Open sourcing Airbnb's Puppet module for Munki

It’s probably no secret that we use Puppet to configure our macOS fleet at Airbnb. And whilst we have given several talks about how we use Puppet on macOS, there was still a lot of hand waving about how we had our modules set up.

So, after several months of me saying “I should open source that”, here is our first open source Puppet module: puppet-munki.

What does it do?

As you may have guessed, it is a Puppet module that installs and configures Munki. More specifically:

  • It will install your specified version of Munki, making sure all of its services are loaded (so no need to reboot when upgrading!)
  • It will generate and install a configuration profile that covers all (probably!) of Munki’s preferences
  • Supports local only manifests so managed installs and uninstalls can be specified with the module
  • Is fully configurable with Hiera, so configuration can be specified as generally or as granularly as you need
  • Supports an optional background auto run after Munki has been installed, allowing Munki to get to work installing things whilst your initial Puppet run continues
  • Will repair (re-install Munki and the profile) if Munki hasn’t run successfully after a configurable number of days
  • Includes some Facts about the Munki version installed, the packages installed and a function to determine whether Munki has installed an item (useful if you are installing a package with Munki but need to perform additional configuration with Puppet)

Is something broken? Want to help make this better?

Whilst this module has been in development here for several months, there is always room for improvement. Please file issues and pull requests if you have any suggestions or problems.

This isn’t the only module we have planned for release, so watch this space.

High Sierra and my open source tools

With this week’s release of the first beta of High Sierra, I wanted to quickly update everyone on the current status of the tools I’ve released, and where I see the future going for them.

Sal

Sal appears to work fine on High Sierra. Some of the external scripts may need updating, but right now I’m not seeing anything that’s broken. I don’t see anything changing in this regard.

Crypt

Amazingly, the way we interact with FileVault as admins seems to not have changed at all, so Crypt has been reported to work perfectly (I will confirm when I get back to my testing computers that I don’t mind nuking). Apple has had a habit of wiping out the authorization database for basically every update during Sierra’s life, so this isn’t really a surprise. Whether managing the entries with your configuration management tool, or simply reinstalling the package will get things working again.

Now we know it works, I will be pushing ahead with migrating it to Swift 3 so it can be built on a modern version of Xcode. If anyone wants to help with this effort, please help out on the PR.

Imagr

Imagr currently works, but is only able to restore HFS+ images, due to a limitation in asr. I don’t see this restriction changing any time soon, so I would strongly suggest those who currently image to consider moving to a different workflow. At this time, all other functions of Imagr (running scripts, installing packages) are expected to continue functioning as they presently do.

If you use these tools, I really need help testing them. 10.13 has brought about so many changes that we need to be extra vigilant testing them. Even if you can’t code, bug reports are really valuable.

MacDevOps YVR

For those of you who are at the fantastic MacDevOps:YVR, why on earth are you reading this? Regardless, here are the slides from my talk “Something something commercial, something something open source” (and thank you for coming and not throwing things at me, you’re all lovely people). I will update this post when the video is available.

Using Python in Puppet Facts

There comes a time when writing Facts in Ruby just isn’t going to cut it - when you need to access Objective C frameworks, for example. Whilst Ruby can’t access these, Python is waiting in the wings ready to come to your rescue.

There is the concept of External Facts - Facts that are written in whatever the system can run, and with Puppet 3.4 / Facter 2.0.1, they can even be distributed with pluginsync.

So let’s say you wanted a Fact that reported what a preference is set to (the GlobalProtect VPN client’s portal in this example):

facts.d/global_protect_portal_pref.py
#!/usr/bin/python
import Foundation
import sys
value = Foundation.CFPreferencesCopyAppValue('Portal', 'com.paloaltonetworks.GlobalProtect')
out = value or ''
sys.stdout.write(out)

All done, right? Well, until you try to run this on a box without the Python Objective-C bridge, anyway. Like you Linux machines that also use this Puppet Server.

We’ve hit one of the drawbacks of External Facts vs regular Facts in that you can’t confine your Fact to a particular operating system (you are also unable to access the values from other Facts).

Fortunately, Facter can execute shell commands. And you can feed in strings at the command line for /usr/bin/python to run for you.

lib/facter/global_protect_portal_pref.rb
# global_protect_portal_pref.rb
Facter.add(:global_protect_portal_pref) do
confine kernel: 'Darwin'
setcode do
portal = nil
output = Facter::Util::Resolution.exec("/usr/bin/python -c \"import Foundation; import sys; value = Foundation.CFPreferencesCopyAppValue('Portal', 'com.paloaltonetworks.GlobalProtect'); out = value or ''; sys.stdout.write(out);\"")
if output != ''
portal = output
end
portal
end
end

With this pattern, we are able to use values from other Facts, and we can confine where our Fact will run so we don’t get errors on operating systems that don’t support what we’re doing.

Using Caddy to HTTPS all the things

Caddy is a lightweight web server that amongst it’s features, has integration with LetsEncrypt to automatically request certificates. This means that you now have absolutely no excuse anymore to run your apps over plain old HTTP anymore. Let me be clearer. If you are running web services over HTTP, regardless of whether it touches the internet or not, you are doing it wrong.

Read more →