{"id":1364,"date":"2023-02-24T04:11:00","date_gmt":"2023-02-24T09:11:00","guid":{"rendered":"https:\/\/www.brunerd.com\/blog\/?p=1364"},"modified":"2023-02-27T22:31:58","modified_gmt":"2023-02-28T03:31:58","slug":"avoid-unicode-mangling-in-jamf-with-shef","status":"publish","type":"post","link":"https:\/\/www.brunerd.com\/blog\/2023\/02\/24\/avoid-unicode-mangling-in-jamf-with-shef\/","title":{"rendered":"Avoid Unicode Mangling in Jamf with shef"},"content":{"rendered":"\n

Computers are “the equivalent of a bicycle for our minds” as Steve Job once said<\/a>. Sure enough, like a bicycle, they also require maintenance! Sometimes that requires soliciting the user to take action. Asking at the right time<\/em> is very important (see my post Don’t Be a Jerk<\/a>) and so is brevity. I’ve found a sprinkling of emoji and other symbols can help your message cut through the noise. A bright red stop sign \ud83d\uded1 can convey its meaning with only a glance, especially for non-native speakers of your language. Your tooling however, may trip you up and mangle your text.<\/p>\n\n\n\n

While AppleScript<\/a>, Swift Dialog<\/a> and JamfHelper<\/a> can all handle Unicode, Jamf databases tables by default do not<\/em> support 4-byte Unicode! I touched on this in my post: jpt 1.0 text encoding fun<\/a>. While the database engine might support it, chances are your tables are Latin1<\/code> which top out at 3-byte UTF-8 encoded characters. 4-byte characters get mangled. Let’s take a look at this in action:<\/p>\n\n\n

\n
\"\"<\/a>
Looks good while editing…<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"<\/a>
Once saved, the mangling is clear<\/figcaption><\/figure>\n<\/div>\n\n\n

Only the gear makes it through because it’s actually two 3-byte characters (U+2699 \u2699 gear plus variation selector U+FE0F) but who has time to check which one is which? How about an encoding tool written in shell, that allows you to escape and format Unicode for shell scripts and\/or Jamf script parameters that can make it through unscathed and un-mangled? Sound good? Great! I present shef<\/a>, the Sh<\/strong>ell E<\/strong>ncoder and F<\/strong>ormatter!<\/p>\n\n\n\n

Give it some text, specify an encoding scheme and\/or quoting style and out comes your string ready for use! You can put the resulting string in your script as a variable or as a script parameter in a Jamf policy depending on quoting options. I’ve done the hard work of finding the various ways to escape special characters for shell and made this handy script for you!<\/p>\n\n\n\n

shef Examples<\/h2>\n\n\n\n

First, let’s make a simple script for Jamf that processes the output of shef<\/a>. I’ve chosen AppleScript so you can play along at home even if you don’t have Jamf installed, this can also be applied to “wrapper scripts” that leverage other tools like Swift Dialog<\/a> or JamfHelper<\/a>. At it’s heart is the simple technique of encoding the input with shef<\/a> then decoding it in the script before presentation. If you use bash<\/code> use echo -e<\/code> if you use sh<\/code> or zsh<\/code> then just echo<\/code> will work, it’s that simple. Our example script simpleAlert-AS.sh<\/a>, keeps things small and as minimal as possible but keep in mind my fuller-featured tool shui<\/a> for more robust scripts where you may need text or password entry, file picking, etc. without external dependencies but don’t<\/em> want to bother learning AppleScript.<\/p>\n\n\n\n

#!\/bin\/bash\n#simpleAlert-AS - Copyright (c) 2023 Joel Bruner (https:\/\/github.com\/brunerd)\n#Licensed under the MIT License\n\n#Simple Applescript alert dialog for Jamf - just a title, a message and an OK button\n#Accepts hex (\\xnn) and octal (\\0nnn) escaped UTF-8 encoded characters (since the default Jamf db character set mangles 4 byte Unicode)\n#Use shef to encode your strings\/files for use in this script: https:\/\/github.com\/brunerd\/shef\n\n#function to interpret the escapes and fixup characters that can screw up Applescript if unescaped \\ and \"\nfunction interpretEscapesFixBackslashesAndQuotes()(echo -e \"${@}\" | sed -e 's\/\\\\\/\\\\\\\\\/g' -e 's\/\"\/\\\\\"\/g')\n\nfunction jamflog(){\n\tlocal logFile=\"\/var\/log\/jamf.log\"\n\t#if we cannot write to the log or it does not exist, unset and tee simply echoes\n\t[ ! -w \"${logFile}\" ] && unset logFile\n\t#this will tee to jamf.log in the jamf log format: <Day> <Month> DD HH:MM:SS <Computer Name> ProcessName[PID]: <Message>\n\techo \"$(date +'%a %b %d %H:%M:%S') ${myComputerName:=\"$(scutil --get ComputerName)\"} ${myName:=\"$(basename \"${0}\" | sed 's\/\\..*$\/\/')\"}[${myPID:=$$}]: ${1}\" | tee -a \"${logFile}\" 2>\/dev\/null\n}\n\n\n#process our input then escape for AppleScript\nmessage=$(interpretEscapesFixBackslashesAndQuotes \"${4}\")\ntitle=$(interpretEscapesFixBackslashesAndQuotes \"${5}\")\n#could be a path or a built-in icon (stop, caution, note)\nicon=\"${6}\"\n#invoke the system open command with this argument (URL, preference pane, etc...)\nopen_item=\"${7}\"\n\n#these are the plain icons (Applescript otherwise badges them with the calling app)\ncase \"${icon}\" in\n\t\"stop\") icon=\"\/System\/Library\/CoreServices\/CoreTypes.bundle\/Contents\/Resources\/AlertStopIcon.icns\";;\n\t\"caution\") icon=\"\/System\/Library\/CoreServices\/CoreTypes.bundle\/Contents\/Resources\/AlertCautionIcon.icns\"\n\t\t#previous icon went away in later macOS RIP\n\t\t[ ! -f \"${icon}\" ] && icon=\"\/System\/Library\/CoreServices\/Problem Reporter.app\/Contents\/Resources\/ProblemReporter.icns\";;\n\t\"note\") icon=\"\/System\/Library\/CoreServices\/CoreTypes.bundle\/Contents\/Resources\/AlertNoteIcon.icns\";;\nesac\n\n#make string only if path is valid (otherwise dialog fails)\nif [ -f \"${icon}\" ]; then\n\twithIcon_AS=\"with icon file (POSIX file \\\"${icon}\\\")\"\nfi\n\njamflog \"Prompting user: $(stat -f %Su \/dev\/console)\"\n\n#prompt the user, giving up and moving on after 1 day (86400 seconds)\n\/usr\/bin\/osascript <<-EOF\nactivate\nwith timeout of 86400 seconds\n\tdisplay dialog \"${message}\" with title \"${title}\" ${withIcon_AS} buttons {\"OK\"} default button \"OK\" giving up after \"86400\"\nend timeout\nEOF\n\nif [ -n \"${open_item}\" ]; then\n\tjamflog \"Opening: ${open_item}\"\n\topen \"${open_item}\"\nfi\n\nexit 0<\/code><\/pre>\n\n\n\n
Upload the above script to your Jamf (if you have one), label parameters 4, 5, 6, and 7 as:  Message<\/code>, Title<\/code>, Icon Path<\/code>, an Open After OK<\/code>,  respectively. If running local keep this in mind and put 1, 2, 3<\/code> as place holder arguments for the first three parameters.<\/p>\n\n\n
\n
\"\"<\/a>
Jamf Script parameters names for <\/figcaption><\/figure>\n<\/div>\n\n\n
Let’s come up with an example message for our users. How about the common refrain coming from MacAdmins around the world:<\/p>\n\n\n\n
\n
\ud83d\uded1 Stop.
\u2699\ufe0f Run your updates.
\ud83d\ude4f Thanks!<\/p>\n<\/blockquote>\n\n\n\n
Now if I tried to pass this text to a script within a Jamf policy, all those emoji would get mangled by Jamf as we saw above. Let’s use shef<\/code> to encode the string for Jamf:<\/p>\n\n\n\n
% shef <<'EOF'\nheredoc> \ud83d\uded1 Stop.\nheredoc> \u2699\ufe0f Run your updates.\nheredoc> \ud83d\ude4f Thanks!\nheredoc> EOF\n\\xF0\\x9F\\x9B\\x91 Stop.\\n\\xE2\\x9A\\x99\\xEF\\xB8\\x8F Run your updates.\\n\\xF0\\x9F\\x99\\x8F Thanks!<\/code><\/pre>\n\n\n\n
For the example above I am using a “here-doc”; in practice you can simply supply shef<\/code> a file path<\/strong>. The output encodes all the newlines in the ANSI-C style of \\n<\/code> and the emoji have all been replaced by their UTF-8 encodings using the hexadecimal escaping of \\x<\/code>. We can take this output and use it in our Jamf policy. The message will get through both textually and symbolically. As an additional feature bonus I added a simple open<\/code> action in the script. Supply the file path to the Software Update panel and after the user clicks OK, it will be opened. Scope a policy like this to anyone with pending updates with Daily frequency:<\/p>\n\n\n
\n
\"\"<\/a>
Escape the mangling in Jamf!<\/figcaption><\/figure>\n<\/div>\n\n\n
If you are just running the script locally and calling from the Terminal you’ll want to specify a quoting option. Exclamations are tricky and shells usually love to interpret trailing exclamations as a history expansion command, shef<\/code> does its best to avoid this:<\/p>\n\n\n\n
bash-3.2$ shef -Qd <<'EOF'\n> \ud83d\uded1 Stop.\n> \u2699\ufe0f Run your updates.\n> \ud83d\ude4f Thanks!\n> EOF\n\"\\xF0\\x9F\\x9B\\x91 Stop.\\n\\xE2\\x9A\\x99\\xEF\\xB8\\x8F Run your updates.\\n\\xF0\\x9F\\x99\\x8F Thanks\"\\!\"\"\n\nbash-3.2$ .\/simpleAlert-AS.sh 1 2 3 \"\\xF0\\x9F\\x9B\\x91 Stop.\\n\\xE2\\x9A\\x99\\xEF\\xB8\\x8F Run your updates.\\n\\xF0\\x9F\\x99\\x8F Thanks\"\\!\"\" \"Software Updates Pending\" stop \"\/System\/Library\/PreferencePanes\/SoftwareUpdate.prefPane\"<\/code><\/pre>\n\n\n
\n
\"\"<\/a>
Our un-mangled output<\/figcaption><\/figure>\n<\/div>\n\n\n
Stop the Mangling Madness!<\/h2>\n\n\n\n
Wrapping things up: use shef<\/a><\/code> to encode strings with Unicode so they survive storage in Jamf’s Latin1 encoded db tables. shui<\/a><\/code> my fuller featured AppleScript dialog tool is ready to accept shef<\/a><\/code><\/code> encoded strings. You can also use shef<\/a><\/code><\/code> minify your text for your shell script. In fact, I used it to encode the help text file down to a single quoted line for use within itself.  That’s 1 happy customer and counting, hope you find it useful too!<\/p>\n","protected":false},"excerpt":{"rendered":"
Computers are “the equivalent of a bicycle for our minds” as Steve Job once said. Sure enough, like a bicycle, they also require maintenance! Sometimes that requires soliciting the user to take action. Asking at the right time is very important (see my post Don’t Be a Jerk) and so is brevity. I’ve found a […]<\/p>\n","protected":false},"author":2,"featured_media":1369,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19,46,12,56,39],"tags":[20,47,22,24,23],"class_list":["post-1364","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bash","category-jamf","category-scripting","category-shef","category-zsh","tag-bash","tag-jamf","tag-scripting","tag-shell","tag-zsh"],"_links":{"self":[{"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/posts\/1364","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/comments?post=1364"}],"version-history":[{"count":17,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/posts\/1364\/revisions"}],"predecessor-version":[{"id":1392,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/posts\/1364\/revisions\/1392"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/media\/1369"}],"wp:attachment":[{"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/media?parent=1364"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/categories?post=1364"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.brunerd.com\/blog\/wp-json\/wp\/v2\/tags?post=1364"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}