Detecting and affecting Lockdown Mode in macOS Ventura

Lockdown mode is new feature for macOS Ventura and for many MacAdmins we’ve been wondering how to detect this state. Why? Lockdown mode affects how macOS and Mac apps behave. This is something a helpdesk might like to know when trying to troubleshoot an issue. Also, due to some ambiguous wording by Apple, they made it seem like MDM Config Profiles could not be installed at all when in Lockdown mode, however this is not always the case. The hunt was on!

Detecting Lockdown Mode

I was looking everywhere last week: ps process lists, nvram, system_profiler, kextstat, launchctl, sysdiagnose, a defaults read dump, etc. I was looking high and low for “lock” “down” and “mode” and I got a hit in the com.apple.Safari domain in the sandboxed ~/Library/Containers/Safari path. While it turns out that Safari will in some cases write the button label LockdownModeToolbarIdentifier to that pref domain, it requires Safari to be launched and for the toolbar to be in non-default layout, otherwise the label name is never written! So that was a dead end.

Then a little birdie on MacAdmins pointed me in the right direction and blogged about it and wrote a Jamf extension attribute! 😅 Turns out I had missed the value sitting at the top of the defaults read dump! (d’oh) It was there the whole time in .GlobalPreferences, I just hadn’t done a diff like I should have! That would have revealed the key uses the LDM acronym/mnemonic: LDMGlobalEnabled Funnily enough, when I searched for this key on Google I got 5 hits and all of them for iOS, like this one at the Apple dev forums. However they were all about Swift and iOS, here’s how to do it in shell for the current user:

defaults read .GlobalPreferences.plist LDMGlobalEnabled 2>/dev/null

It’s a boolean value, that will not exist if Lockdown mode has never been enabled, when enabled it will report 1 from defaults and when disabled the key will remain and report 0. What stands out is that this is a per-user preference. Since it makes you reboot I had supposed it was a system-wide setting but sure enough if you log out and into another user, Lockdown mode is disabled. Perhaps that makes sense but I’m not quite sure about that?

Affecting Lockdown Mode

This totally blew me away: You can enable and disable macOS Lockdown mode by writing to your .GlobalPreferences preference domain!

#turn lockdown mode off
defaults write .GlobalPreferences.plist LDMGlobalEnabled -bool false
#turn lockdown mode on
defaults write .GlobalPreferences.plist LDMGlobalEnabled -bool true

That’s right, it’s not written to a rootless/SIP protected file like TCC.db! Just run the command as the user and it’ll turn toggle the behavior for most things. Here’s some details of my findings:

  • Configuration profiles – a restart of System Settings is not required, it will prohibit the manual installation of a .mobileconfig profile file. When Apple says “Configuration profiles can’t be installed” this is what they mean: User installed “double-click” installations of .mobileconfig files cannot be done. When they say “the device can’t be enrolled in Mobile Device Management or device supervision while in Lockdown Mode”, this only applies to these user-initiated MDM enrollments using a web browser that downloads .mobileconfig files. Lockdown mode does not prohibit enrollment into MDM that’s assigned via Apple Business Manager (ABM/DEP). You can initiate enrollment with the Terminal command: sudo profiles renew -type enrollment A Mac in Lockdown mode will be able to successfully enroll into an MDM assigned by ABM. Once enrolled, new Config Profiles can also be installed via that same MDM, even in Lockdown Mode.
  • Messages – a restart of Messages is not required, all messages will be blocked immediately, attachments or not. I’m not sure if that’s a bug or not since Apple only mentions attachments, not plain messages. It does not matter if the sender is in your Contacts or whether you have initiated contact with them before (like in Facetime). Messages will be delivered to any other devices not in Lockdown mode. If Lockdown mode is turned off, those blocked messages may be delivered if sent recently enough but will appear out of sequence. For example, a device that never had Lockdown Mode turned on would see messages: 1,2,3,4,5 while a device that turns it on and then off would see: 1,2,5,3,4
  • Facetime – restart not required, it will immediately begin blocking calls from anyone you have not called previously from that device. Unlike Messages though, it will show a Notification of the blockage.
  • Safari – app restart required. This differs from everything else, however Safari also gives the best visual indications that Lockdown mode is enabled! On the Start Page you’ll see “Lockdown Ready”, once at at website you’ll likely see “Lockdown Enabled” unless you’ve uncheck Enable Lockdown Mode in the top menubar SafariSettings for <site>… in which case you’ll see “Lockdown Off” in red.
Safari’s Lockdown Mode Toolbar states
  • Safari – Another subtle visual cue of Lockdown mode, that aligns with Apple’s “web fonts might not be displayed” guidance, can be seen on a Jamf user-initiated MDM enrollment screen, instead of a check mark you’ll see a square, take heed and turn back now! Since once you get the .mobileconfig files and fumble your way to System SettingsPrivacy & Security, scroll to the bottom of the list to Profiles (UX gripe: it used to just open the dang panel when you double clicked on them!) you’ll be blocked from installing it as seen above.
  • System Settings – an app restart is required for Privacy & Security to reflect the current state of LDMGlobalEnabled, if it was on and you disable via defaults once you launch System Settings again, it’ll let you turn it back on with a reboot and everything!

Wrapping Up

I didn’t try out the other lockdown mode behaviors for things like new Home management invitations or Shared Albums in Photos. Still it’s quite surprising that despite the System Settings GUI making you reboot to turn it on, Lockdown mode is a per-user setting that can seemingly be enabled and disabled dynamically with a simple defaults command run by the user. With the exception of Safari and System Settings it does not require Messages and Facetime to restart! There might be other caveats, it’s hard to tell. Perhaps this is all in the realm of “works as designed” for Apple but when you, the customer, don’t know what that exact design is, it can be quite a surprise!

One more (unrelated) thing…

Since this post might get a few eyeballs, I’d also like to shine a light on the perplexing fact that Safari is the only browser that still doesn’t support the four year old ES2018 feature of RegExp lookbehind assertions?! I mean, sure it was a Google engineer who kindly filed this heads up to the WebKit team back in July of 2017 when it was a draft and a full year before it was ratified (Bug 174931 – Implement RegExp lookbehind assertions) but even a silly corporate rivalry couldn’t explain the seeming obstinance in letting this feature languish. I don’t get it, it just doesn’t make sense! There’s a nicely visualized page of where things stand and Safari is keeping company with IE 11 on this one.

Make these red islands green, Apple!

So take a look at the comments on the WebKit bug, some are quite funny, others just spot on, and there’s even one from yours truly. Perhaps add your own? Maybe when a bug gets 100 comments something special happens and we all get cake? 🎂

Respecting Focus and Meeting Status in Your Mac scripts (aka Don’t Be a Jerk)

In four words Barbara Krueger distills the Golden Rule into an in-your-face admonishment: Don’t be a jerk. It makes for a great coffee mug but how does this relate to shell scripting for a Mac admin and engineer? Well, sometimes a script can only go so far and you need user consent and cooperation to get the job done. Not being a jerk about it can pay off!

A good way to get cooperation from your users is to build upon a foundation of trust and respect. While IT has a long list of to-dos for users: “Did you reboot? Did you run updates? Did you open a ticket? Did you really reboot?” the users might have one for us: “Respect our screens when we’re in the middle of a client meeting and respect when Focus mode is turned on.” And they are right. That’s also two things. “Give an inch and they take a mile” I tell ya!

So how can we in IT can be considerate of our users? First, don’t do anything Nick Burns from SNL does. Second, use the functions below (and in my GitHub) to check if a Zoom, Teams, Webex, or Goto meeting is happening or if Do Not Disturb/Focus mode is on. When non-user-initiated scripts (i.e. daily pop-ups/nags/alerts) run they can bail or wait if the user is busy. If it were real life (and it is!) we wouldn’t walk into a meeting and bug a user, in front of everyone, to run update. If we did, they’d be more likely to remember how rude we were rather than actually running the updates. So let’s get their attention when they will be most receptive to what we have to say.

Detecting Online Meetings Apps

First up is inMeeting_Zoom which simply checks for the CptHost process and returns success or fail. Notice how this simple behavior can be used with an if/then statement. The return code is evaluated by the if, a zero is success and a non-zero is a failure. && is a logical AND and || is a logical OR

#!/bin/sh
#inMeeting_Zoom (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Zoom {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	pgrep "CptHost" &>/dev/null && return 0 || return 1
}

if inMeeting_Zoom; then
	echo "In Zoom meeting... don't be a jerk"
else
	echo "Not in Zoom meeting"
fi

Next is another process checker for Webex: inMeetng_Webex. What is a bit more unique is the process appears in ps in parentheses as (WebexAppLauncher)however pgrep cannot find this process (because the actual name has been rewritten by the Meeting Center process). We instead use a combination of ps and grep. A neat trick with grep is to use a [] regex character class to surround a single character, this keeps grep from matching itself in the process list. That way you don’t need to have an extra grep -v grep to clean up the output.

#!/bin/sh
#inMeeting_Webex (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Webex {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	ps auxww | grep -q "[(]WebexAppLauncher)" && return 0 || return 1
}

if inMeeting_Webex; then
	echo "In Zoom meeting... don't be a jerk"
else
	echo "Not Webex in meeting"
fi

Goto Meeting is more straightforward, although it should be noted that regardless of quote type, single or double, the parentheses must be escaped with a backslash. Otherwise, it’s the same pattern, look for the process name which only appears during a meeting or during the meeting preview and return 0 or 1 for if to evaluate, find it here: inMeeting_Goto

#!/bin/sh
#inMeeting_Goto (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Goto() {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	pgrep "GoTo Helper \(Plugin\)" &>/dev/null && return 0 || return 1
}

if inMeeting_Goto; then
	echo "In Goto meeting... don't be a jerk"
else
	echo "Not in Goto meeting"
fi

Lastly, Teams is a bit more complex, rather than looking for the presence of a process, we instead look for a JSON file in the user’s /Library/Application Support/Microsoft/Teams folder which has the current call status for both the app and the web plugin (the other methods above are for the app only). We’ll use the ljt to extract the value from the JSON. In fact I wrote ljt after starting to write this blog last week and realizing that jpt (weighing in at 64k) was just overkill. As a bonus to doing that, I just realized that bash functions can contain functions! Long ago I ditched using () in shell function declarations and just used the function keyword. Empty parentheses seemed decorative rather than functional since it’s not like it’s a C function that needs parameter names and types. However the lack of parentheses () apparently, prevents a function from being declared inside a function! Below I just wanted to make sure ljt doesn’t get separated from inMeetings_Teams

#!/bin/bash
#inMeeting_Teams (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Teams ()(

	function ljt () ( #v1.0.3
		[ -n "${-//[^x]/}" ] && set +x; read -r -d '' JSCode <<-'EOT'
		try {var query=decodeURIComponent(escape(arguments[0]));var file=decodeURIComponent(escape(arguments[1]));if (query[0]==='/'){ query = query.split('/').slice(1).map(function (f){return "["+JSON.stringify(f)+"]"}).join('')}if(/[^A-Za-z_$\d\.\[\]'"]/.test(query.split('').reverse().join('').replace(/(["'])(.*?)\1(?!\\)/g, ""))){throw new Error("Invalid path: "+ query)};if(query[0]==="$"){query=query.slice(1,query.length)};var data=JSON.parse(readFile(file));var result=eval("(data)"+query)}catch(e){printErr(e);quit()};if(result !==undefined){result!==null&&result.constructor===String?print(result): print(JSON.stringify(result,null,2))}else{printErr("Node not found.")}
		EOT
		queryArg="${1}"; fileArg="${2}";jsc=$(find "/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/" -name 'jsc');[ -z "${jsc}" ] && jsc=$(which jsc);[ -f "${queryArg}" -a -z "${fileArg}" ] && fileArg="${queryArg}" && unset queryArg;if [ -f "${fileArg:=/dev/stdin}" ]; then { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "${fileArg}"; } 1>&3 ; } 2>&1); } 3>&1;else { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "/dev/stdin" <<< "$(cat)"; } 1>&3 ; } 2>&1); } 3>&1; fi;if [ -n "${errOut}" ]; then /bin/echo "$errOut" >&2; return 1; fi
	)

	consoleUser=$(stat -f %Su /dev/console)
	consoleUserHomeFolder=$(dscl . -read /Users/"${consoleUser}" NFSHomeDirectory | awk -F ': ' '{print $2}')
	storageJSON_path="${consoleUserHomeFolder}/Library/Application Support/Microsoft/Teams/storage.json"
	
	#no file, no meeting
	[ ! -f "${storageJSON_path}" ] && return 1

	#get both states
	appState=$(ljt /appStates/states "${storageJSON_path}" | tr , $'\n' | tail -n 1)
	webappState=$(ljt /webAppStates/states "${storageJSON_path}"| tr , $'\n' | tail -n 1)
	
	#determine app state
	if [ "${appState}" = "InCall" ]	|| [ "${webAppState}" = "InCall" ]; then
		return 0
	else
		return 1
	fi
)


if inMeeting_Teams; then
	echo "In Teams Meeting... don't be a jerk"
else
	echo "Not in Teams Meeting"
fi

Detecting Focus (formerly Do Not Disturb)

Last but not least is determining Focus (formerly Do Not Disturb) with doNotDisturb. As you can see there’s been a few different ways this has been implemented over the years. In macOS 10.13-11 the state was stored inside of a plist. For macOS 12 Monterey they’ve switched from a plist to a JSON file. A simple grep though is all that’s needed to find the key name storeAssertionRecords. If it is off, that string is nowhere to be find, when it’s on it’s there. Simple (as in Keep it Simple Stoopid)

#!/bin/bash
#doNotDisturb (grep) (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

#An example of detecting Do Not Disturb (macOS 10.13-12)

function doNotDisturb()(

	OS_major="$(sw_vers -productVersion | cut -d. -f1)"
	consoleUserID="$(stat -f %u /dev/console)"
	consoleUser="$(stat -f %Su /dev/console)"
	
	#get Do Not Disturb status
	if [ "${OS_major}" = "10" ]; then
		#returns c-cstyle boolean 0 (off) or 1 (on)
		dndStatus="$(launchctl asuser ${consoleUserID} sudo -u ${consoleUser} defaults -currentHost read com.apple.notificationcenterui doNotDisturb 2>/dev/null)"

		#eval c-style boolean and return shell style value
		[ "${dndStatus}" = "1" ] && return 0 || return 1
	#this only works for macOS 11 - macOS12 does not affect any of the settings in com.apple.ncprefs
	elif [ "${OS_major}" = "11" ]; then
		#returns "true" or [blank]
		dndStatus="$(/usr/libexec/PlistBuddy -c "print :userPref:enabled" /dev/stdin 2>/dev/null <<< "$(plutil -extract dnd_prefs xml1 -o - /dev/stdin <<< "$(launchctl asuser ${consoleUserID} sudo -u ${consoleUser} defaults export com.apple.ncprefs.plist -)" | xmllint --xpath "string(//data)" - | base64 --decode | plutil -convert xml1 - -o -)")"

		#if we have ANYTHING it is ON (return 0) otherwise fail (return 1)
		[ -n "${dndStatus}" ] && return 0 || return 1
	elif [ "${OS_major}" -ge "12" ]; then
		consoleUserHomeFolder=$(dscl . -read /Users/"${consoleUser}" NFSHomeDirectory | awk -F ': ' '{print $2}')
		file_assertions="${consoleUserHomeFolder}/Library/DoNotDisturb/DB/Assertions.json"

		#if Assertions.json file does NOT exist, then DnD is OFF
		[ ! -f "${file_assertions}" ] && return 1

		#simply check for storeAssertionRecords existence, usually found at: /data/0/storeAssertionRecords (and only exists when ON)
		! grep -q "storeAssertionRecords" "${file_assertions}" 2>/dev/null && return 1 || return 0
	fi
)
if doNotDisturb; then
	echo "DnD/Focus is ON... don't be a jerk"
else
	echo "DnD/Focus is OFF"
fi

Since Focus can remain on indefinitely an end user may never see your pop-up. If so, build a counter with a local plist to record and increment the number of attempts. After a threshold has been reached you can then break through to the user (I certainly do).

Detecting Apps in Presentation Mode

A newer addition to this page is some code to detect fullscreen presentation apps that I cannot take credit for. Adam Codega, one of the contributors to Installomator hipped me to a cool line of code that was added in PR 268. It leverages pmset to see what assertions have been made to the power management subsystem. It uses awk to look for the IOPMAssertionTypes named NoDisplaySleepAssertion and PreventUserIdleDisplaySleep with some additional logic to throw out false positives from coreaudiod. In testing I’ve found this able to detect the presentation modes of Keynote, Powerpoint and Google Slides in Slideshow mode in Chrome (but not Safari), your mileage may vary for other apps. Another caveat is that when a YouTube video is playing in a visible tab, it will assert a NoDisplaySleepAssertion, however these will be named “Video Wake Lock” whereas a Slideshow presentation mode will have its name assertions named “Blink Wake Lock”. So I am adding an additional check to throw our “Video Wake Locks”. This may be more of a can of worms than you’d like, if so, user education to set Focus mode may be the way to go. A functionalized version can be found here: inPresentationMode

#!/bin/sh
#inPresentationMode (20220319) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#with code from Installomator (PR 268) (https://github.com/Installomator/Installomator) Copyright 2020 Armin Briegel
#Licensed under the MIT License

function inPresentationMode {
	#Apple Dev Docs: https://developer.apple.com/documentation/iokit/iopmlib_h/iopmassertiontypes
	#ignore assertions without the process in parentheses, any coreaudiod procs, and "Video Wake Lock" is just Chrome playing a Youtube vid in the foreground
	assertingApps=$(/usr/bin/pmset -g assertions | /usr/bin/awk '/NoDisplaySleepAssertion | PreventUserIdleDisplaySleep/ && match($0,/\(.+\)/) && ! /coreaudiod/ && ! /Video\ Wake\ Lock/ {gsub(/^.*\(/,"",$0); gsub(/\).*$/,"",$0); print};')
	[ -n "${assertingApps}" ] && return 0 || return 1
}

if inPresentationMode; then
	echo "In presentation mode... don't be a jerk"
else
	echo "Not in presentation mode..."
fi

All together now

Putting it all together here’s how you can test multiple functions in a single if statement, just chain them together with a bunch of || ORs

#!/bin/bash
#Joel Bruner - demo of meeting/focus aware functions for your script

#pretend we've declared all the functions above and copy and pasted them in here
function doNotDisturb()(:)
function inMeeting_Teams()(:)
function inMeeting_Zoom(){:}
function inMeeting_Goto(){:}
function inMeeting_Webex(){:}
function inPresentationMode(){:}

#test each one with || OR conditionals
#the FIRST successful test will "short-circuit" and no more functions will be run
if doNotDisturb || inPresentationMode || inZoomMeeting || inMeeting_Goto || inMeeting_Webex || inMeeting_Teams; then
	echo "In a meeting, presentation, or Focus is On... don't be a jerk"
	#do something else, like wait 
else
	echo "Not in a meeting..."
	#alert the user or do whatever you needed to do that might impact them
fi

Using these functions in your scripts can help respect your users’ online meeting times and Focus states. Also it doesn’t hurt to document it somewhere and toot your own horn in a user facing KB or wiki. If and when a user complains about that pop-up that destroyed their concentration and their world, you can show them the forethought and effort you’ve taken to be as considerate as possible regarding this perceived incursion. This usually has the effect of blowing their mind 🤯 that someone in IT is actually trying to be considerate!

P.S. I’m pretty stoked that Prism.js can really jazz up my normally dreary grey code blocks! 😍🤓 I found a good WordPress tutorial here

jpt 1.0, the JSON power tool in 2022

I’m happy to announce that jpt, the JSON power tool, has been updated to 1.0 and is available on my GitHub Releases page! It’s been over a year since the last release and I’ve been of working on, learning, and pondering what a really useful JSON tool could be and then working like heck to make it a reality.

jpt is a command line utility for formatting, querying, and modifying JSON. It can run on any version of macOS and most any Unix/Linux with jsc (JavaScriptCore) installed, otherwise it’s dependency free! For systems administrators and MacAdmins who need first class JSON tooling in their own shell scripts it can be embedded as a shell function in either bash or zsh. jpt was built to stand the test of time, needing only jsc and a shell, it will work on the newest macOS all the way back to OS X 10.4 Tiger!

There’s a year’s worth of bug fixes, improvements, and new features, let’s take a look at some of them.

Notable Features and Improvement

• jpt can now input and output multi-JSON documents like JSON Text Sequences (RFC 7464) plus other non-standard formats like NDJSON and JSON Lines and concatenated JSON. See: jpt 1.0 can deal with multiple JSON texts

• Truncated JSON can be de detected in concatenated/lines JSON and repaired by closing up open strings, arrays and objects. See: Helping truncated JSON data with jpt 1.0

• jpt can now fully rehabilitate JSON5 back to standard JSON. As I learned last year, there’s more mutant JSON than I realized! See: jpt 1.0 and JSON5 rehab

• jpt can query JSON using both JSON Pointer, IETF RFC 6901 and JSONPath, a nascent draft RFC with powerful recursive search and filtering features.

• jpt can manipulate JSON using JSONPatch, JSON Merge Patch, and some new merge modes not found anywhere else. See: merging JSON objects with jpt 1.0

• shell scripters and programmers may find it useful that jpt has extensive string and number encoding capabilities. From classic encodings like hex, octal, and binary to more recent ones like URL/”percent” encoding and Unicode code points. See: jpt 1.0 text encoding fun

• Need help finding visualizing JSON in new ways? jpt can output the structure of JSON in ways not seen, like JSONPath Object Literals which are simply the JSONPath, an equal sign, and the JSON value (e.g. $.ok="got it"). This simple and powerful declarative syntax can help find the exact path to a particular value and can also be used to quickly prototype a JSON object. See: jpt: see JSON differently

Try it out

There’s a lot to explore in jpt 1.0, download it from my GitHub with a macOS installer package in the Releases page. Check out my other jpt blog entries here at brunerd. As the IETF JSONPath standard takes shape I’ll be updating jpt to accomodate that standard. Until then, I hope this tool helps you in your JSON work, play, or research!


macOS Compatibility Fun!

Compatibility Questions

If you work with Macs and Jamf then you know every year there’s a new per OS Extension Attribute (EA) or Smart Group (SG) recipe to determine if macOS will run on your fleets hardware. However I asked myself: What if a single Extension Attribute script could fill the need, requiring only a periodic updating of Model IDs and the addition of new macOSes?

Then I also asked: Could this same script be re-purposed to output both text and CSV, not just for the script’s running host but for a list of Model IDs? And the answer was a resounding yes on all fronts!

EA Answers

So, my fellow Jamf admin I present to you macOSCompatibility.sh in its simplest form you just run the script and it will provide ultra-sparse EA output like: <result>10.14 10.15 11</result> this could then be used as a Smart Group criteria. Something like “macOS Catalina Compatible” would then match all Macs using LIKE 10.15 or “Big Sur Incompatible” would use NOT LIKE 11, of course care would be taken if you were also testing for 10.11 compatibility, however the versionsToCheck variable in the script can limit the default range to something useful and speeds things up the less version there are. I hope this helps Jamf admins who have vast unwieldy fleets where hardware can vary wildly across regions or departments,

CSV Answers

Now if you provide a couple arguments like so: ./macOSCompatibility.sh -c -v ALL ALL > ~/Desktop/macOSCompatibilityMatrix.csv you will get a pretty spiffy CSV that let’s you visualize which Mac models over the years have enjoyed the most and least macOS compatibility. This is my favorite mode, you can use it to assess the OS coverage of past Macs.

See macOSCompatibilityMatrix.csv for an example of the output. If you bring that CSV into Numbers or Excel you can surely liven it up with some Conditional Formatting! This is the barest of examples:

Can you spot the worst and best values?

Text Answers

If you don’t use the -c flag then it’ll just output in plain or text, like so: ./macOSCompatibility.sh -v ALL ALL

iMacPro1,1: 10.13 10.14 10.15 11
MacBook1,1: 10.4 10.5 10.6
MacBook2,1: 10.4 10.5 10.6 10.7
MacBook3,1: 10.5 10.6 10.7
MacBook4,1: 10.5 10.6 10.7
MacBook5,1: 10.5 10.6 10.7 10.8 10.9 10.10 10.11
MacBook6,1: 10.6 10.7 10.8 10.9 10.10 10.11 10.12 10.13
MacBook7,1: 10.6 10.7 10.8 10.9 10.10 10.11 10.12 10.13
MacBook8,1: 10.10 10.11 10.12 10.13 10.14 10.15 11
MacBook9,1: 10.11 10.12 10.13 10.14 10.15 11
MacBook10,1: 10.12 10.13 10.14 10.15 11
MacBookAir1,1: 10.5 10.6 10.7
MacBookAir2,1: 10.5 10.6 10.7 10.8 10.9 10.10 10.11

Wrapping Up

Now, it’s not totally perfect since some models shared Model IDs (2012 Retina and Non-Retina MacBook Pros for example) but for the most part the Intel Mac Model IDs were sane compared to the PPC hardware Model IDs: abrupt jumps, overlaps, and re-use across model familes. Blech! I’m glad Apple “got religion” for Model IDs (for the most part) when Intel CPUs came along. I did attempt to go back to 10.1-10.3 with PPC hardware but it was such a mess it wasn’t worth it. However testing Intel, Apple Silicon and VMs against macOS 10.4 – 11+ seems to have some real use and perhaps you think so too? Thanks for reading!

jpt is unphased by macOS’s malformed JSON

A funny thing happened on the way to writing a new blog entry. Instead of finding an example JSON file that ships with macOS, I stumbled upon a slow creep of malformed JSON files that started in OS X El Capitan and continues into Big Sur. While I can’t do much about deviant JSON files in macOS, I can control how jpt handles them. So like any good Teachable Moment™ and life lesson, I’ve turned lemons into lemonade and out of challenge comes innovation: jpt v0.9.6 is now nonplussed by these uncouth JSON files, with their comments and trailing commas. Like water off ducks back, jpt keeps on keepin’ on, unphased.

Finding the Misfits

To find these misfits I wrote made a script jsonParsingReportCSV-macOS. This will attempt to parse the JSON it finds using the json_pp binary that ships from Apple, the results are output as CSV and you can then easily sort it to spot troublemakers.

I’ve uploaded a couple annotated version of those searches along with the kind of deviation they exhibit: Malformed-macOS-10.15.csv and Malformed-macOS-11.1.csv

What’s their damage, man?

The two most prevalent kinds of JSON malformation I found in macOS were: trailing commas and comments. Trailing commas meaning the last item in an array or object should not have a comma after it. While Javascript has always allowed trailing commas in array literals and later in object literals (ES5), then in function parameters (ES2017), JSON has never allowed trailing commas nor comments. Neither are in the JSON spec (RFC8259).

The biggest group of offenders were the USD generated JSONs within USDKit (a private framework) and Model I/O. USD is interesting because of it’s lineage and purpose: It’s Pixar’s open source software for combining multiple 3D elements within a common scene. Or as Pixar’s page put it: “Universal Scene Description (USD) is the first publicly available software that addresses the need to robustly and scalably interchange and augment arbitrary 3D scenes that may be composed from many elemental assets.” Keep an eye on this framework 🕶

JSON Lines (Don’t Do It)

An outlier in the mix is defaultConfig.json within CoreAnalytics.framework, it is what some call “JSON Lines” or “concatenated JSON”, where each line is its own a JSON value or object. While doing some research on JSON Lines I came upon an intriguing comment in this post by rurban:

“The delimiters don’t get in the way, they protect you from MITM and other attacks. It’s a security feature, not a bug.”

Reini Urban on JSON vs. “concatenated JSON”

Taking a look at Reini’s GitHub and the level of his work, I’m going to give his statement some credence. To that end, I really didn’t want to add another output mode nor change query behavior to accommodate JSON Lines. My current compromise is to simply to attempt conversion of a JSON Lines file into an array: tack on commas and ensconce in square brackets. It Just Works™ if every line is truly a self contained JSON object. If you want to be made aware of this, use the -c option so jpt it will “complain” on stderr and exit with code 1.

The Rabbit Hole of CoreAnalytics

Within the CoreAnalytics private framework is defaultConfig.json, it is a collection of 4019 objects currently that describe the events and actions Apple is interested in collecting usage information on. They want to know if you’re actually using all those new features in CarPlay, Reminders, Photos, eGPU App usage, etc… If you want to see all the key names try this: jpt $..addTransform.name /System/Library/PrivateFrameworks/CoreAnalytics.framework/Versions/A/Resources/defaultConfig.json 

Do you actually use this stuff? Apple wants to know if it’s worth their while…

A Comment on JSON Comments

Comments in JSON has inspired debate far and wide, this post by getify sums it up very nicely:

A JSON encoder MUST NOT output comments. A JSON decoder MAY accept and ignore comments.

Douglas Crockford in some old Yahoo group that even Wayback can’t find anymore

So that’s just what jpt does now, it will never encode comments but it will strip out any and all comments it recognizes automatically, no switches needed. Which style comments? All of them (or most of them):

Commentkindlineage
# commentlineshell, Perl, PHP, Python, R, Ruby, MySQL
// commentlineC(99), Javascript, Java, Swift
/* comment */blockC, Java, Javascript, Swift
; commentlineassembly language (ASM)
-- commentlineAda, Applescript, Haskell, Lua, SQL
<!-- comment -->blockXML
*** comment ***linemacOS’s ionodecache.json

If you need a test file you can use my worstJSONEver.json. A most horrible JSON file if there ever was one! jpt soldiers on, use -c to see what it does to clean the input (-i0 for no level indents). It shows the initial parse error and each attempt at recovery.

Fixing the Misfits

There is no fixing, there is only acceptance. Since most of these files are in SIP protected areas of macOS so it will take an act of God, a well filed bug report, for Apple to correct these in a future update. But they usually only act if there’s a demonstrable bug, in this case it’s more of an observation about the condition of the JSON files than a clear issue. If someone is really bothered, they could disable SIP and go in and fix them, however future macOS updates will likely revert these changes. Since I haven’t tried this, who knows, macOS may get ornery and trip OS integrity protections, rendering your system un-bootable, YMMV 🤷‍♂️. The real question is if they impact how macOS works and I’m guessing they don’t. The apps and frameworks which use these off-spec JSON file must be doing something to correct the issues before parsing. Rest assured though, if you attempt to parse one of these files with jpt there will be no issues, it Just Works™. However if you would like to know it did something, use -c to “complain” about comments, commas, JSON Lines, and hard wrapping. It will send those notices to stderr and exit with a status of 1.

jpt: malformed JSON? Bring it on.

So after all this I don’t have a good example page for jpt’s multitude of features and usage (yet) but what I do have is a JSON Power Tool with more “dummy-proof” features than it did before. I’ll take that as a win! Check out the latest release of jpt at my Github.

jpt (JSON Power Tool) is a non-compiled tool that can manipulate and query JSON in a variety of ways, it runs on macOS (10.4 – 11.0) PPC/Intel/Apple Silicon; Linux with jsc; Windows with Linux subsystem for Windows and jsc. It can be used standalone or embedded within your shell scripts. I hope you find it useful. Thanks for reading!

Jamf & FileVault 2: Tips & Tricks (and more)

Raiders of the Lost Feature Requests

So there’s this old feature request at Jamf Nation (stop me if you’ve heard this one…) it’s almost 6 years old: Add ability to report on FV2 Recovery Keys (and/or access them via API) In fact, maybe you came here from there, watch out don’t loop! Continue!

The pain point is this: Keys are sent back to Jamf Pro (JSS) but then can only be gotten at manually/interactively through the web interface, not via API nor another method. For cases of mass migration to another JSS it sure would be nice to move those keys over rather than decrypt/re-encrypt. Well, I’ve got a few insights regarding this that I’d like to share that may help. ‘Cuz hey it’s 2020 and we’ve learned that hoarding is just silly.

Firstly, it should be pointed out that neither ye olde “Recovery Key Redirection” payload nor it’s replacement “Recovery Key Escrow” are needed to get keys to the JSS. There is another method and it’s what is used by the built-in “Filevault Encryption” policy payload to get the keys back to your JSS. Jamf references this method in this old script at their GitHub. I revamped the core bits a couple years ago in a (nearly 7 year old) feature request: Manually Edit FileVault 2 Recovery Key

Telling the JSS Your Secrets

The takeaway from that is to realize we have a way to explicitly send keys to the JSS by placing 2 XML files in the /Library/Application Support/JAMF/run folder: file_vault_2_id.xml and file_vault_2_recovery_key.xml. Also note, Jamf has updated the process for the better in the last two years: a jamf recon (or two) is no longer required to send the key and validate it, instead JamfDaemon will send it immediately when both the files are detected. Which is nice, but it’s the subsequent recon validations where we have an opportunity to get grabby.

Cold Lamping, Hard Linking

So here’s the fun part: When recon occurs there’s lots of file traffic in /Library/Application Support/JAMF/tmp all sorts of transient scripts hit this folder. What we can do is make hard links to these files as they come in so when the link is removed in tmp another exists elsewhere and the file remains (just in our new location). EAGrabber.sh does exactly that (and a little bit more)

EAGrabber.sh can be easily modified to narrow it’s focus to the FileVault 2 key only, deleting the rest. What you do with the key is up to you: Send it somewhere else for safe keeping or keep it on device temporarily for a migration to another Jamf console. A script on the new JSS could then put that key on-disk into file_vault_2_recovery_key.xml file which will then import and validate, no decrypt/recrypt necessary. Hope this helps.

Cuidado ¡ Achtung ! Alert

Jamf admins take note: Do you have hard coded passwords in your extension attributes or scripts? If so, then all your scripts are belong to us. Now, go read Obfuscation vs. Encryption from Richard Purves. Read it? OK, now consider what happens if you were to add a routine to capture the output of ps aww along with a hard-linking loop like in EAGraber.sh. If you are passing API credentials from policies via parameter, then ps can capture those parameters and even if you try and obscure them, if we’ve captured the script we can de-obfuscate them. This is a good reason to be really careful with what your API accounts can do. If you have an API account with Computer record Read rights that gets passed into a script via policy and you use LAPS, then captured API credentials could be used to harvest LAPS passwords via API. Keep this in mind and we’ll see if any meaningful changes will occur in recon and/or the script running process in the future (if you open a ticket you can reference PI-006270 regarding API credentials in the process list). In the meantime make API actions as short lived as possible and cross your fingers that only you, good and noble #MacAdmins read this blog. 🤞

New Projects: jpt and shui, Now Available

Between March and October 2020 I had some great ideas for command line Mac utilities the MacAdmin could apprecite and I had the time to devote to their realization. I’m excited to present these two open source projects, available on GitHub: jpt and shui. I hope they can add richness to your shell scripts’ presentation and capabilities without requiring additional external dependancies.

jpt – the “JSON Power Tool” is a Javascript and shell script polyglot that leverages jsc, the JavascriptCore binary that is standard on every Mac since 10.4 and since the jpt is purposefully written in ES5 to maintain maximum compatibility, why yes, this tool does run on both PPC and Intel Macs all the way back to OS X Tiger and then all the way forward to the latest 11.0 macOS Big Sur! Many Linux distros like CentOS and Ubuntu come with jsc pre-installed also, even Windows with the Linux Subsystem installed can run jsc and therefore can run the jpt!

What you can do with the jpt? Query JSON documents using either the simple yet expressive JSONPath syntax or the singular and precise JSON Pointer (RFC6901) syntax. The output mode is JSON but additional creative output modes can render JSONPaths, JSON Pointer paths, or even just the property names with their “constructor” types (try -KC with -J or -R) Textual output can be encoded in a variety of formats (hex/octal/URI encoding, Unicode code points, etc…), data can be modified using both JSON Patch (RFC6902) operations (add, replace, remove, copy, move, test) and also JSON Merge Patch (RFC7386) operations. JSON can be worked with in new ways, try -L for “JSONPath Object Literal” output to see what I mean. Or you simply feed jsc a file to pretty-print (stringify) to /dev/stdout. I’ll be writing more about this one for sure.
Github project page: jpt
Tagged blog posts: scripting/jpt

shui – first-class Applescript dialog boxes in your shell scripts without needing to remember esoteric Applescript phrasings! If you think it’s odd for code to have possessive nouns and are more comfortable in shell, you’re not alone. shui can be embedded in either bash or zsh scripts but it can also output Applescript if you really want to know how the sausage is made or want to embed in your script without shui. Hopefully shui will let you forget those awkward Applescript phrasing and focus on your shell script’s features and functionality. It uses osascript to execute the Applescript and launchctl to invoke osascript in the correct user context so user keyboard layouts are respected (vs. root runs). Check out the project page for demo videos and then give shui a try.
Project page: shui
Tagged blog posts: scripting/shui

Time to Die: When Mac app and package certificates expire

So I had a Draft about this last week but an usure feeling. I thought: “Do I really have anything to contribute? Folks are already aware of this. What can I really add?” So, I let it slide, then Thurday came and there was a hiccup: Apple hadn’t refreshed all their packages! Oh noes. Could this have been something my script could have helped head off? No but it might have made examining the packages Thursday afternoon a bit easier. So with that, I present certChecker.command

I’d advise just copy pasting the raw script into a new BBEdit document and saving it as “certChecker.command“. The extension matters, BBEdit will set the executable bit and you won’t have to mess with cleaning off the com.apple.quaratine flag as you would if you downloaded it.

Not much of a looker at the command line but comes together nicely in Quick Look

The need to find expired packages is critical if you use Jamf Pro to deploy your packages. macOS will throw an error if you try and install a pkg with an expired cert via the installer command line tool. It will suggest you use the -allowUntrusted flag, however this is not an option with Jamf Pro. You either have to get a new resigned package or repack it. Repacking is basically expanding the pacakge then flattening it again. This will strip out all certs. repackPKGs.command is useful for when vendors have forgotten to issue new packages and you need a working package.

Cert validation caveat: some packages can expire yet remain valid! These are signed with “trusted time stamps” and Suspicious Package explains this quite well. Their tool can better help you assess what a package will do when it’s expiration occurs. For certChecker.command I use pkgutil’s assesment for package files but for apps all we have is the the “not after” date.

Speaking of apps: the Install macOS Mojave.app from the App Store was giving me the “damaged” message on Thursday around 1PM CST…

The application certs were all VALID, my script said so, what gives? Doing a bit of sleuthing revealed InstallESD.dmg to be different between the two installers. I used find to run a md5 on all the files in each installer and send the output to two files. Running find from inside the apps keeps the base paths the same and allows for easy comparison with diff or XCode’s FileMerge (a favorite).

#compare two apps in Terminal
cd <ye olde installer app>
find . -type f -exec md5 {} \; | sort > ~/Desktop/YE_OLDE.txt
cd <new installer app>
find . -type f -exec md5 {} \; | sort > ~/Desktop/NEWNESS.txt
cd ~/Desktop
diff YE_OLDE.txt NEWNESS.txt

FileMerge inside XCode.app is a great visual diff’er
Turns out we have some stowaways in the old installer.
All is well, app cert and InstallESD packages are looking good.

By ~3pm CST they’d gotten everything fixed and the packages inside were all good. Not since the great expiration of 2015, have we had to care about expiring Apple packages. This time Apple has only pushed the app certs out 1 1/2 years to April 12th, 2021. That’s not long! The packages inside InstallESD.dmg are good however until April 14, 2029. Which one will win? Will we care (will we be able to roll back?) I’ll leave that as an exercise/rhetorical question for someone else.

In the meantime you might have other packages lurking in your distribution points that need updating or repacking. If so give certChecker.command a whirl to ferret them out. I hope it saves you some time and effort <insert reference to Roy Batty and “tears in the rain here” :>

macOS shell games: long live bash

TL;DR – Bash ain’t goin’ nowhere on Mac, both version-wise and in terms of its presence. Looking at the longevity of other shells on the system, it will likely be around for a good while longer.

There’s been a lot of hand wringing and angst online about bash and zsh becoming the new default shell. Some folks feel Apple is signaling deprecation and removal and have the crushing feeling they must convert all their bash script to zsh. I think that’s a bit unnecessary.

True, the default shell is changing from bash to zsh, as Apple notes here. This is indeed a Good Thing™ as zsh shell has been one of the most frequently updated shells on macOS. Bash, on the other hand, has been stuck at varying versions of 3.2 for 12 years now! On the plus side, sysadmins have “enjoyed” predictable and stable behavior from bash during this time. Sure, you’d love new features but when you are scripting for the enterprise, across multiple OS versions, this is just the sort of thing you want: boringness and dependability.

As far as deprecations go, the only thing Apple has signaled as being deprecated eventually are scripting language runtimes (not shells):

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 from within Terminal. (51097165)

mac OS Catalina 10.15 Release Notes

I’ve done some digging and culled the shell versions from OS X 10.0 to macOS 10.15, along with their respective release dates. I think it shows that shells, no matter how old and crusty, tend to be long lived and not soon removed on macOS.

Here’s a quick way to check your shell versions (except for dash):

macOSzshbash/shcsh/tcshkshdash
10.03.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.08.00 (1998-10-02)
10.13.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.10.00 (2000-11-19)
10.24.0.4 (2001-10-26)2.05b.0 (2002-07-17)6.10.00 (2000-11-19)
10.34.1.1 (2003-06-19)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)
10.44.2.3 (2005-03-00)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)M p (1993-12-28)
10.54.3.4 (2007-04-19)3.2.17 (2007-05-01)6.14.00 (2005-03-23)M s+ (1993-12-28)
10.64.3.4 (2008-11-03)3.2.48 (2008-11-18)6.15.00 (2007-03-03)M s+ (1993-12-28)
10.74.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)M s+ (1993-12-28)
10.84.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)JM 93u (2011-02-08)
10.95.0.2 (2012-12-12)3.2.51 (2010-03-17)6.17.00 (2009-07-10)JM 93u (2011-02-08)
10.105.0.5 (2014-01-06)3.2.57 (2014-11-07)6.17.00 (2009-07-10)AJM 93u+ (2012-08-01)
10.115.0.8 (2015-05-31)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.125.2 (2015-12-02)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.135.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.145.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.155.7.1 (2019-02-03)3.2.57 (2014-11-07)6.21.00 (2019-05-08)AJM 93u+ (2012-08-01)dash-9 (1993)

As you can see, zsh has has been updated with almost every new release of macOS. Bash really hit a wall with 3.2 and as many have noted, it was v4’s change in licensing to GPLv3 that caused this (sh is really bash in sh compatibility mode so the versions are intertwined). csh/tcsh has the same duality thing going on and took a notably giant 7 year leap in 10.15 to a version from 2019. ksh has remained at the same version just as long as bash yet I don’t think anyone is fretting that ksh will be deprecated or removed. Finally, dash is just a weirdo that apparently disdains versioning! I used a combination of what /bin/dash and man dash to get some sort of crude answer.

So there you go: In my opinion, all signs point to bash being yet another shell on macOS for some time. Removing bash from macOS would break a lot of stuff and while that reason alone hasn’t stopped Apple before, I think they will let sleeping dogs lie. Go ahead and learn zsh, master it, customize it, or make it “sexy” but take the rumors of its demise on macOS with a grain of salt and dose of skepticism.

Install macOS High Sierra, HFS Compression, and OS X 10.8

If you have downloaded Install macOS High Sierra.app on a non-macOS 10.8 machine then copy said installer to a 10.8 Mac, the installer will be unusable. Why? HFS Compression. It seems Install macOS High Sierra.app uses HFS compression on numerous files and it seems 10.8 has issues with that and will not “see” those files.

To quickly demonstrate which files are compressed, we will use stat and examine the st_flags attribute, 0=no compression, 32=compression (thanks ghost of Mac OS X Hints!)

IFS=$'\n\t'; for file in $(find /Applications/Install\ macOS\ High\ Sierra.app -type f); do echo "$(stat -f %f "$file"): $file)"; done

You’ll see many files with 32 as the flag, these will not be readable in 10.8

The trick is to copy the file and do away with the HFS compression, we can use ditto for this:

ditto --nopreserveHFSCompression /Applications/Install\ macOS\ High\ Sierra.app [destination folder]

This will produce an installer compatible with 10.8… sure you could have downloaded it via the Mac App Store by now on a 10.8 machine, but now you know.