graham gilbert

Mac administration and assorted nerdity

Personal Automation: Munki (Part 2)

| Comments

The first step to getting any Mac set up is to get some software onto it. I’m not going to cover how to set up Munki or AutoPkg – there are lots of other places for that information.

As a sysadmin, I’m forever testing things. Rather than destroy my own machine, I like to do this in Virtual Machines. My preferred virtualisation solution is VMware Fusion, but unfortunately it’s not very easy to deploy out of the box. You need to do a little bit of work to get it into a package that you can import into Munki, but fortunately the process is well documented on VMware’s site.

The next piece of ‘non standard’ software I need is Homebrew. The installation method listed on their site is to run a terminal command as the current user. The first part of this is obviously fine – Munki has several methods to run scripts (payload free packages, nopkg), but it runs everything as root. Fortunately, as I’m deploying my own machine, I can make some assumptions about where Homebrew will be installed. The first assumption I can make is that there will only be one user on the machine, and the second is that I’m going to be logged in most of the time (as my laptop is encrypted, it’s either off or logged in).

I’m going to utilise a nopkg pkginfo file to perform the installation. The first part of our script to install Homebrew is to make sure that a user (me!) is logged in. Homebrew doesn’t like being owned by root, so first we need to make sure that there is a user logged in.

1
2
3
4
5
6
7
8
#!/bin/bash

CURRENT_USER=`/bin/ls -l /dev/console | /usr/bin/awk '{ print $3 }'`

if [ "$CURRENT_USER" == 'root' ]; then
    # this can't run at the login window, we need the current user
    exit 1
fi

So now we know that there’s a user logged in, and who that user is. Time to install Homebrew as the current user.

1
2
3
4
5
6
7
8
mkdir -p /usr/local
mkdir -p /usr/local/homebrew
mkdir -p /usr/local/bin
chown $CURRENT_USER:_developer /usr/local/homebrew
chown $CURRENT_USER:_developer /usr/local/bin

#download and install homebrew
su $CURRENT_USER -c "/bin/bash -o pipefail -c '/usr/bin/curl -skSfL https://github.com/mxcl/homebrew/tarball/master | (cd /usr/local ; /usr/bin/tar xz -m --strip 1 -C homebrew; ln -s /usr/local/homebrew/bin/brew /usr/local/bin/brew)'"

As we’re using a nopkg with Munki rather than a payload free package, we’ve not left any receipts, so Munki doesn’t know if Homebrew is installed. We’re going to use an installs array to tell Munki what to look for when determining whether Homebrew is installed or not.

1
2
3
4
5
6
7
8
9
<key>installs</key>
  <array>
      <dict>
          <key>path</key>
          <string>/usr/local/bin/brew</string>
          <key>type</key>
          <string>file</string>
      </dict>
  </array>

You might be crying “but Homebrew needs the Xcode Command Line Tools installed!” – and you’d be 100% correct. You have the option of importing the downloaded package into Munki, but I have adapted Tim Sutton’s script into a nopkg. To find out what’s installed, I ran fseventer and chose a random file to act as my installs array. I’ve posted the pkginfos for both the Xcode CLI tools and all of the Homebrew installs on Github.

Personal Automation (Part 1)

| Comments

Earlier this year, I professed my love of Boxen – the personal automation solution based on Puppet released by Github. Indeed, it served me well for quite some time, but I began to find myself spending more time fixing Boxen than actually getting things done. As Boxen was designed for internal use at Github, it set some things up how they liked them – which wasn’t necesarily how I liked them. Sysadmins have similar needs to developers, but not exactly the same.

Then I updated Boxen. All of my modules were out of date, so I spent a good couple of hours updating all of them so they worked again. Ugh.

So I started looking at moving to my own solution. One of my major irritations when using Boxen was that it didn’t really handle updating your apps – you got whatever version the module author decided to install and then you had to hope that there was an updatng mechanism built in. I’ve said before that there is no better method of getting software onto your Mac then Munki, so the first decison was straightforward. The rest took a little thought.

The six P’s

My first requirement was that I shouldn’t need to run anything to get my configuration to apply. Boxen requires that you run the boxen command periodically across each of your Macs to get the configuration applied. This wasn’t always practical. I needed something that would run in the background and keep itself up to date.

As I said before, I really disliked how Boxen installs software. Munki does a much better job, and AutoPkg makes it trivial to make sure you have the latest software version. Being a sysadmin, I need more than simple drag and drop apps and packages though – I make extensive use of Homebrew to install command line tools like Packer, so I needed to come up with a way of installing these with Munki.

However, Munki isn’t the best tool for managing my configuration. I’ve been using Puppet to manage the Macs I look after for nearly three years now, and I wanted to base my system on it as I’ve already done a lot of the work with making OS X specific modules. I also wanted to use the modules made for Boxen as much as possible (some made too many assumptions about where they were running, so couldn’t be re-used.

So to recap:

  • Munki for software deployment.
  • Puppet must run in the background periodically
  • The configuration must update itself – I don’t want to have to sync code across machines.
  • Where possible, re-use existing Puppet modules

Over the next few posts, I’ll go over the different parts of this solution, how I put it together and how you might be able to use this.

Automate Yourself Out of a Job

| Comments

A massive thanks to everyone who came to my talk today, and thanks to everyone who helped out with the Q&A at the end. All of the code I used in the talk is up on GitHub and here are the slides. Hopefully the video will convey how much fun it actually was – it could have been a disaster, so I’m hugely grateful to everyone who contributed to the discussion at the end.

Making Packages With AutoPkg

| Comments

Over the past few weeks, I’ve had the same conversation over and over: people telling me that once they get started using Munki, their next step will be to start using AutoPkg. I gave each person the same response: “you’re doing it wrong”.

AutoPkg a has a reputation of being difficult to use. This is totally unjustfied. You don’t need to be using Munki for it to be useful, you don’t need to set it up to run automatically via Jenkins or a LaunchDaemon. If you need to get software into a package, AutoPkg is the easiest way.

Installing AutoPkg

Head over to the releases page on AutoPkg’s GitHub repository and download the latest version (0.3.0 at the time of writing). It’s an Apple package, so double click it and get it installed. If you have Gate Keeper enabled, you’ll need to right-click on the package and choose to install it from there, as it’s not been signed.

Recipes

AutoPkg is useless without recipes. Fortunately, there are hundreds that have already been made by the community.

We’ll add the set of recipes maintained by AutoPkg’s authors, which contains some of the most common software. Open up a terminal window and enter :

1
$ autopkg repo-add https://github.com/autopkg/recipes

You’ll see AutoPkg downloading and adding the recipes to your Mac.

1
2
3
4
5
6
7
8
Attempting git clone...

Adding /Users/grahamgilbert/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes to RECIPE_SEARCH_DIRS...
Updated search path:
  '.'
  '~/Library/AutoPkg/Recipes'
  '/Library/AutoPkg/Recipes'
  '/Users/vagrant/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes'

Using the thing

Let’s see what recipes we just added. Still in your terminal, enter:

1
$ autopkg list-recipes

You’ll see a whole load of output like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Adium.download
Adium.munki
Adium.pkg
AdobeAIR.pkg
AdobeAcrobatPro9Update.download
AdobeAcrobatPro9Update.munki
AdobeAcrobatProXUpdate.download
AdobeAcrobatProXUpdate.munki
AdobeAir.munki
AdobeFlashPlayer.download
AdobeFlashPlayer.munki
AdobeFlashPlayer.pkg
AdobeFlashPlayerExtractPackage.munki
...

The naming convention in AutoPKG is SoftwareName.output. For for example, to run a recipe that downloads Google Chrome and adds it to Munki, you would use the GoogleChrome.munki recipe, but if you just wanted to download it an make a package, you’d use the GoogleChrome.pkg recipe. It just so happens that making a package of Chrome is exactly what we want to do.

Back into your terminal and enter:

1
$ autopkg run GoogleChrome.pkg

The AutoPkg robot will churn away and you’ll get some output similar to:

1
2
3
4
5
6
7
8
9
Processing GoogleChrome.pkg...

The following new items were downloaded:
    /Users/grahamgilbert/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg

The following packages were built:
    Identifier               Version          Pkg path
    ----------               -------          --------
    com.google.Chrome        35.0.1916.153    /Users/grahamgilbert/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome-35.0.1916.153.pkg

And when it’s all finished, you’ll be left with a nice package that you can use anywhere you’d use finely crafted packages – ARD, AutoDMG or even Casper if you’re that way inclined (although Allister Banks has been working on a way of automating importing packages into the JSS – see his recent talk for more on that subject).

Doing it all again

What happens next time you want to build an updated package?

1
$ autopkg run GoogleChrome.pkg

Right?

Well, kinda.

What happens if Google changes the URL AutoPkg uses to download Chrome? Fortunately we’re using the community provided recipes, and if something’s broken they usually get fixed pretty quickly. We just need to tell AutoPkg to update the installed recipes.

1
$ autopkg update-repo all

And then we’re able to build our package safe in the knowledge that someone else has done all of the hard work for us.