Track and Tackle com.apple.macl

Starting in macOS Catalina, an extended attribute (XA) named com.apple.macl is being added to the files and folder you work with. What does it do? How does it work? When does it get added? Where is the documentation?

All these are good questions, but there’s no official documentation. A good start is this assemblage of articles, here’s the highlights from that page and a few more…

Apple talked about this vaguely in their WWDC 2019 session 701, “Advances in macOS Security“, the Files and Folders fun starts at 21:21 but there’s nothing really about the implementation

Tom Bridge talked about com.apple.macl to the Penn State Macadmins back in August before Catalina’s release. He’s one of the first to notice this XA (and talk about it). Although, he says that curl will add this XA to downloads, it thankfully does not. Most likely it happened when he dragged it to Terminal or performed some other action on it.

Howard Oakley of the prolific Eclectic Light Company blog kicks the tires a bit more and finds some interesting quirks, as he always does.

Jeff Johnson though, really hits the nail on the head in illustrating what’s going on when you simply drag and drop a file or folder into Terminal and com.apple.macl is appended. He dropped this, December 18th and well… the holidays were around the corner and we all had better stuff to do!

It’s 2020 now and time to come back around to this. A very helpful post in the Apple Dev forums from an Apple employee, Quinn “the Eskimo”, decodes the data structure a bit more as well as what some of the conditions for having this XA appended are:

When a user selects a “protected” file or folder in an NSOpenPanel in a non-sandboxed app on Catalina, consent is inferred and the app can access it.

It seems that 01 00 is a header of some form and … is a UUID associated with my test app.  I dug into how that UUID is set up and, well, it’s complex, and more of an implementation detail than I care to go into here on DevForums.

https://forums.developer.apple.com/thread/124121

Well, I would have loved the complex implementation detail! But I’ll tell you this: The app UUID in com.apple.macl is unique to EVERY computer. Jeff Johnson was right when he said: “The macl is effectively untraceable“.

However you can begin to see the shape of things when you write a tool to output the UUID in CSV like I have. Behold maclTrack.command

Run it in Terminal, give it files or folders as arguments, it’ll report on them, you can even specify a maximum depth for folders (-d) and silence reporting on items lacking com.apple.macl (-s), pipe it into tee to see the output and write it to a file.

This poor file got clobbered by multiple app UUIDs

Something interesting I noticed from using this tool is seeing the “Header” as Quinn called it, differ from 0100 to 0200. I believe 0200 is for drag and drop operations vs. a regular save which results in 0100.

Again, the app UUIDs you see on the files on your Mac will never be seen on another Mac ever – why? Because UUIDs are meant to be truly unique. Have you read the man page for uuidgen? They ain’t eff’in around!

 The uuidgen command generates a Universally Unique IDentifier (UUID), a 128-bit value guaranteed to be unique over both space and time.

Apple man page for uuidgen

So what’s the point of having a long lived XA that you can’t get rid of and that’s only useful on your local Mac? If it’s a permissions granting XA (the opposite of com.apple.quarantine which imposes restrictions) then why not let the user remove it? In some ways it’s a cross computer file tracking mechanism, albeit “anonymized”. Perhaps allowing it’s removal though (aka writing “nothing” to the XA) also allows for something to be written? This is where I could get in trouble speculating, it’s just a guess.

Perhaps you’ll forgive me that, since I have found a way to clear com.apple.macl without disabling SIP! Zip it. No, not the one spawned Finder’s “Compress” menu item, I mean /usr/bin/zip! I noticed command line zip would obliterate XAs during a project many years ago and it still does my friends! So behold maclTackle.command

This script has considerably less engineering to it because it’s a PoC and I’m really don’t want to take responsibility for something going awry should you try to clear com.apple.macl on your entire Desktop folder (no I haven’t tested that). But it will strip the XAs off a file that’s for sure! NOTE: USE A NON-IMPORTANT FILE with this script (make a duplicate). The script will overwrite the original (don’t like the behavior? It’s a script: Comment it out!)

🎶 Gonna wash that macl right outta my file… 🎶

Apple may come along and close this loophole now that it’s been pointed out or perhaps they won’t since they’d have to publish the modfication? Perhaps they’ll continue to purge “undesirable” binaries from macOS? They say they will stop shipping Python, perl and ruby in macOS some day and you know what xattr is written in? That’s right Python! ¯\_(ツ)_/¯

Alright, come and get it while the gettin’s good! Thanks for reading!

Time to Die: When Mac app and package certificates expire

So I had a Draft about this last week but an usure feeling. I thought: “Do I really have anything to contribute? Folks are already aware of this. What can I really add?” So, I let it slide, then Thurday came and there was a hiccup: Apple hadn’t refreshed all their packages! Oh noes. Could this have been something my script could have helped head off? No but it might have made examining the packages Thursday afternoon a bit easier. So with that, I present certChecker.command

I’d advise just copy pasting the raw script into a new BBEdit document and saving it as “certChecker.command“. The extension matters, BBEdit will set the executable bit and you won’t have to mess with cleaning off the com.apple.quaratine flag as you would if you downloaded it.

Not much of a looker at the command line but comes together nicely in Quick Look

The need to find expired packages is critical if you use Jamf Pro to deploy your packages. macOS will throw an error if you try and install a pkg with an expired cert via the installer command line tool. It will suggest you use the -allowUntrusted flag, however this is not an option with Jamf Pro. You either have to get a new resigned package or repack it. Repacking is basically expanding the pacakge then flattening it again. This will strip out all certs. repackPKGs.command is useful for when vendors have forgotten to issue new packages and you need a working package.

Cert validation caveat: some packages can expire yet remain valid! These are signed with “trusted time stamps” and Suspicious Package explains this quite well. Their tool can better help you assess what a package will do when it’s expiration occurs. For certChecker.command I use pkgutil’s assesment for package files but for apps all we have is the the “not after” date.

Speaking of apps: the Install macOS Mojave.app from the App Store was giving me the “damaged” message on Thursday around 1PM CST…

The application certs were all VALID, my script said so, what gives? Doing a bit of sleuthing revealed InstallESD.dmg to be different between the two installers. I used find to run a md5 on all the files in each installer and send the output to two files. Running find from inside the apps keeps the base paths the same and allows for easy comparison with diff or XCode’s FileMerge (a favorite).

#compare two apps in Terminal
cd <ye olde installer app>
find . -type f -exec md5 {} \; | sort > ~/Desktop/YE_OLDE.txt
cd <new installer app>
find . -type f -exec md5 {} \; | sort > ~/Desktop/NEWNESS.txt
cd ~/Desktop
diff YE_OLDE.txt NEWNESS.txt

FileMerge inside XCode.app is a great visual diff’er
Turns out we have some stowaways in the old installer.
All is well, app cert and InstallESD packages are looking good.

By ~3pm CST they’d gotten everything fixed and the packages inside were all good. Not since the great expiration of 2015, have we had to care about expiring Apple packages. This time Apple has only pushed the app certs out 1 1/2 years to April 12th, 2021. That’s not long! The packages inside InstallESD.dmg are good however until April 14, 2029. Which one will win? Will we care (will we be able to roll back?) I’ll leave that as an exercise/rhetorical question for someone else.

In the meantime you might have other packages lurking in your distribution points that need updating or repacking. If so give certChecker.command a whirl to ferret them out. I hope it saves you some time and effort <insert reference to Roy Batty and “tears in the rain here” :>

macOS shell games: long live bash

TL;DR – Bash ain’t goin’ nowhere on Mac, both version-wise and in terms of its presence. Looking at the longevity of other shells on the system, it will likely be around for a good while longer.

There’s been a lot of hand wringing and angst online about bash and zsh becoming the new default shell. Some folks feel Apple is signaling deprecation and removal and have the crushing feeling they must convert all their bash script to zsh. I think that’s a bit unnecessary.

True, the default shell is changing from bash to zsh, as Apple notes here. This is indeed a Good Thing™ as zsh shell has been one of the most frequently updated shells on macOS. Bash, on the other hand, has been stuck at varying versions of 3.2 for 12 years now! On the plus side, sysadmins have “enjoyed” predictable and stable behavior from bash during this time. Sure, you’d love new features but when you are scripting for the enterprise, across multiple OS versions, this is just the sort of thing you want: boringness and dependability.

As far as deprecations go, the only thing Apple has signaled as being deprecated eventually are scripting language runtimes (not shells):

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 from within Terminal. (51097165)

mac OS Catalina 10.15 Release Notes

I’ve done some digging and culled the shell versions from OS X 10.0 to macOS 10.15, along with their respective release dates. I think it shows that shells, no matter how old and crusty, tend to be long lived and not soon removed on macOS.

Here’s a quick way to check your shell versions (except for dash):

macOSzshbash/shcsh/tcshkshdash
10.03.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.08.00 (1998-10-02)
10.13.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.10.00 (2000-11-19)
10.24.0.4 (2001-10-26)2.05b.0 (2002-07-17)6.10.00 (2000-11-19)
10.34.1.1 (2003-06-19)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)
10.44.2.3 (2005-03-00)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)M p (1993-12-28)
10.54.3.4 (2007-04-19)3.2.17 (2007-05-01)6.14.00 (2005-03-23)M s+ (1993-12-28)
10.64.3.4 (2008-11-03)3.2.48 (2008-11-18)6.15.00 (2007-03-03)M s+ (1993-12-28)
10.74.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)M s+ (1993-12-28)
10.84.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)JM 93u (2011-02-08)
10.95.0.2 (2012-12-12)3.2.51 (2010-03-17)6.17.00 (2009-07-10)JM 93u (2011-02-08)
10.105.0.5 (2014-01-06)3.2.57 (2014-11-07)6.17.00 (2009-07-10)AJM 93u+ (2012-08-01)
10.115.0.8 (2015-05-31)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.125.2 (2015-12-02)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.135.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.145.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)
10.155.7.1 (2019-02-03)3.2.57 (2014-11-07)6.21.00 (2019-05-08)AJM 93u+ (2012-08-01)dash-9 (1993)

As you can see, zsh has has been updated with almost every new release of macOS. Bash really hit a wall with 3.2 and as many have noted, it was v4’s change in licensing to GPLv3 that caused this (sh is really bash in sh compatibility mode so the versions are intertwined). csh/tcsh has the same duality thing going on and took a notably giant 7 year leap in 10.15 to a version from 2019. ksh has remained at the same version just as long as bash yet I don’t think anyone is fretting that ksh will be deprecated or removed. Finally, dash is just a weirdo that apparently disdains versioning! I used a combination of what /bin/dash and man dash to get some sort of crude answer.

So there you go: In my opinion, all signs point to bash being yet another shell on macOS for some time. Removing bash from macOS would break a lot of stuff and while that reason alone hasn’t stopped Apple before, I think they will let sleeping dogs lie. Go ahead and learn zsh, master it, customize it, or make it “sexy” but take the rumors of its demise on macOS with a grain of salt and dose of skepticism.

Determining the current console user in macOS

There’s a few methods floating out on the web, but I’d just like to point out a quick and reliable way to get the current console user on macOS using stat. This works great with Fast User Switching and if the mac is at the login screen it will return root as the user

consoleUser=$(stat -f %Su /dev/console)

There you go! Now go forth and script.

Also, if you are using backticks ` to capture output (a.k.a. command substitution) then consider using the more modern and nest-able $(…) method, here’s a good reason why: Why is $(…) preferred over `…` (backticks)?

How to determine if your Mac has a T2 chip

Do you know if your MacBook Pro has a T2 chip in it?
Here’s an easy way to tell:

Goto Identify your MacBook Pro model

Then look for the the murderous autonomous robot. Easy!

You can also run this:

/usr/libexec/PlistBuddy -c "print :0:_items:0:ibridge_model_name" /dev/stdin 2>/dev/null <<< "$(/usr/sbin/system_profiler -xml SPiBridgeDataType -nospawn)"

<<< – is an über cool “here string”

PlistBuddy is great at parsing data defaults might balk at, it can also take /dev/stdin!

Nothing (bash style)

The brunerd blog has been a whole lot of nothing lately, so let’s commemorate and celebrate with an entry featuring the premiere of the bash tag and category… hmm, which one is more useful? We’ll find out. Now as for nothing, there’s a legit command, special built-in to do nothing in bash!

man page for bash command :

: [arguments]

No effect; the command does nothing beyond expanding arguments and performing any specified redirections.  A zero exit code is returned.

 

Isn’t that lovely? When you need to do nothing, you can call :
Perhaps you are writing a script and aren’t sure what you’ll do for that if statement —

if [ "${this}" == "that" ]; then
#actually not sure what to do here, let's do nothing
:
else
echo "This else, I'll do"
fi

I made a gist here and I think I’ll keep making more. I’ll feature them here with the bash tag, if only as an auxillary backup of my brain and to add to the collective patchwork quilt of code we all use to get our ideas off the ground, when our reach is just a hare shy of grasp and we need a little help.

Also in the gist was a snippet of a novel use by kbeck at Extensis in his UTC removal script to use : to redirect a here document into itself and server as a comment block that doesn’t need # on every line to be treated as a comment —

: <<COMMENTBLOCK

[ ] 2016-0324: enclose me; love me. i am a block of comments

COMMENTBLOCK

Spiffy!

RecoveryHD Updater now with PKG building

** Update: Recovery HD updaters packages for 10.11 will only work when run in OS X 10.11, previous packages could run in different OSesnot so for 10.11

Updated April 6, 2015: createRecoveryHDUpdater 15.4.6.command – built PKG now is targetable to volumes other than /

Released Feb 28, 2015: createRecoveryHDUpdater 15.2.28.command

Recovery15.2.28

Building off the previous work the updated script will now create a nicely named dmg

recoverydmg_icon

Containing both an interactive script for updating disks and a new script to create a .pkg!

recoverydmg_2scripts

 

The fruits of RecoveryHD PKG Maker.command:

recoverypkg_iconCustom Bundle ID is interactively provided, the PackageVersion is matched to the OS release. Signing identity is asked for in the script but not tested (my cert might be broken, YMMV). The target of the package is always / – an exercise for the reader to change the postinstall script to use Installer’s variables to affect the target The exercise to make the .pkg targetable has been exercised by me… Enjoy!

 

BASH parameter fun

Some exercises in handling parameters in a script:

Using BASH_ARGV to loop through all arguments

#!/bin/bash
#debugging output
[ -f /tmp/debug ] && set -x

#Loop Though the arguments provided
#BASH places arguments in reverse order in BASH_ARGV array
#Start at end of array (ARGC – 1)
for (( i=$(( $BASH_ARGC – 1 )); i>=0; i– )); do
item=”${BASH_ARGV[$i]}”
echo “\$$(( $BASH_ARGC – $i )): $item”
done

Loop through all arguments with special handling of first and last only

#!/bin/bash
[ -f /tmp/debug ] && set -x
##loop through parameters, start with end of BASH_ARGV
for (( loopNumber=1, i=$(( $BASH_ARGC - 1 )); i>=0; i--, loopNumber++ )); do
#do something special for first and last
if [ $i -eq $(( $BASH_ARGC - 1 )) ]; then
conditionalString="(I am the first one!)"
elif [ $i -eq 0 ]; then
conditionalString="(I am the last one!)"
else
unset conditionalString
fi
echo Arg $loopNumber: ${BASH_ARGV[$i]} $conditionalString
done

Loop through all arguments with special handling of first and everything else

#!/bin/bash
[ -f /tmp/debug ] && set -x

##loop through parameters, start with end of BASH_ARGV
for (( loopNumber=1, i=$(( $BASH_ARGC – 1 )); i>=0; i–, loopNumber++ )); do

#if not the last don’t restart dock
if [ $i -eq $(( $BASH_ARGC – 1 )) ]; then
unset conditionalString
conditionalString=”(I am the first)”
else
conditionalString=”(I am not the first)”
fi

echo Arg $loopNumber: ${BASH_ARGV[$i]} $conditionalString
done

Loop through all arguments with special handling of the last and everything else

#!/bin/bash
[ -f /tmp/debug ] && set -x

##loop through parameters, start with end of BASH_ARGV
for (( loopNumber=1, i=$(( $BASH_ARGC – 1 )); i>=0; i–, loopNumber++ )); do

#if not the last don’t restart dock
if [ ! $i -eq 0 ]; then
conditionalString=”(I am not the last)”
else
unset conditionalString
#conditionalString=”(I am the last)”
fi

echo Arg $loopNumber: ${BASH_ARGV[$i]} $conditionalString
done

(WordPress is bugging the heck out of me with the code tags breaking when their are line breaks… sorry, that’s an exercise for the Googler)

Downloading Safari 6

So when Safari 6 came out, Apple did away with download links from their Support site.
Why? Who knows. But it’s a real pain when you have børked install or install a Developer preview (which sometimes will not register in Software Updates and need to get up to the latest release.)

I contributed this tip to Apple Discussions in the HT5275 Safari 6 download thread and am reposting here as well since I repeatedly keep going back to get the command I authored! (No I don’t run Apple Software Update server, I have my own distribution methods for work that don’t involve merging software update catalogs and other ridiculous things OS X Server requires you to do if you dare run various OS X versions and don’t run the latest server). I parenthetically digress.

Here’s the Terminal command to spit out some Safari6 URLs (OS X 10.6-10.8):

curl -s $(strings /System/Library/PrivateFrameworks/SoftwareUpdate.framework/SoftwareUpdate | grep http | grep sucatalog) | grep Safari6 | grep pkg

It’ll give you something like this:
<string>http://swcdn.apple.com/content/downloads/18/32/041-6651/147ugedbeiqpz43czixlgawayfoa7tjyjv/Safari6.1MountainLion.pkg</string><string>http://swcdn.apple.com/content/downloads/47/40/041-6648/a9y00qwi8esz3gl5v2gfik1p3rgvn7zftz/Safari6.1Lion.pkg</string>

I know, not pretty right? But what you can do then is copy and paste the URL into Safari’s Address/Search bar and hit Option-Enter which will then download it.

Viola. A Safari 6 package.

Update for 10.9:
Now when you grep SoftwareUpdate there are multiple URLs for the seed programs, the easiest way to deal with this and still be generic is to loop through all the URLs grepping for Safari

for URL in $(strings /System/Library/PrivateFrameworks/SoftwareUpdate.framework/SoftwareUpdate | grep http | grep sucatalog); do curl -s "$URL" | grep Safari | grep pkg; done

XProtect Plugin Checker

Are you a system administrator or power user who needs a quick way to check if your Flash or Java plugins have been blocked by Apple? Like so:XProtectPluginChecker Menu 2

Then XProtectPluginChecker is the status menu app for you!

XProtect Plugin Checker iconClick to download.

Written in BASH, sprinkled with PlistBuddy/awk/sed-Fu with a dash of mdfind magic, and wrapped with Platypus, the aim is for XProtect Plugin Checker to be able to dynamically accommodate any additions Apple might make to the Plugin Blacklist, although only Flash and Java seem to be in their crosshairs right now, who knows? Silverlight might be next, given its lack of updates (going on 10 months).

A bonus idea for making this more useful is to make a launchd script that watches the XProtect plist file(s) then fires off the XProtectPluginChecker script when they change, emailing the results to you, because knowing is half the battle!

If you find this useful or if you have some hiccups with it, let me know. UPDATE: Now with comments enabled! :]

Also related, myXProtectStatus is a similar style menu status script that lists the malware threats Apple is blocking.