A stylised depiction of Rob

Primary Unit

Scraping Entourage

Early last year myself and my flatmate got into Entourage, which is incredible. We devoured all eight seasons in a few weeks. One thing that I love about it is the soundtrack. It’s a mix of older and newer stuff and, frankly, I have no idea about music so it’s nice to have a pool of well-picked tracks to dip in to.

In December, when my flatmate rewatched the series, I decided to do something with the soundtrack. Getting the music (for 96 episodes) wasn’t as easy as downloading an album — it took some programming, a few headaches, and a bit of tedium. What follows was ultimately successful, but I’d hesitate to call it a success story.

Sourcing the music

HBO’s website does list the tracks used in each episode, but you can’t get at them as it’s all Flash. However, Tunefind has good data, which I imagine some industrious person has transcribed from the official list. In some cases it’s more complete.

But the advantages of having users assemble such data are weighed against the mistakes they may make and inconsistencies that creep in. I’ll address the practical implications later. First we need to extract the track details from the website.

The programs in this post are incredibly rough. I share them in the hope they’ll help others and spark ideas, not as good examples.

All of the code is collected in a multi-file Gist.

python:
 1:  #!/usr/local/bin/python3
 2:  
 3:  import re
 4:  import json
 5:  from bs4 import BeautifulSoup
 6:  from urllib.request import urlopen
 7:  
 8:  base_url = 'http://www.tunefind.com'
 9:  seasons_index = '/show/entourage'
10:  response = urlopen(base_url + seasons_index)
11:  response_text = response.read().decode()
12:  soup = BeautifulSoup(response_text)
13:  
14:  seasons_div = soup.find('div', class_='lefttext sidebarIndent')
15:  seasons_urls = [(base_url + a_tag['href'])
16:                  for a_tag in seasons_div.find_all('a')]
17:  
18:  tracks_list = []
19:  
20:  for s in seasons_urls:
21:    season = BeautifulSoup(urlopen(s).read().decode())
22:    episode_urls = [(base_url + a_tag['href']) for a_tag
23:                    in season.find_all('a',
24:                      {'name': re.compile(r'episode\d+')})]
25:  
26:    for e in episode_urls:
27:      episode = BeautifulSoup(urlopen(e).read().decode())
28:  
29:      for raw in episode.find_all(class_='tf-songevent-text'):
30:        match = re.search(r'(.+)\n\s+by (.+)', raw.text.strip())
31:        if match:
32:          tracks_list.append(match.groups())
33:  
34:  for track in tracks_list:
35:    # filter list for duplicates
36:    if tracks_list.count(track) > 1:
37:      tracks_list.remove(track)
38:  
39:  with open('/Users/robjwells/Desktop/tracks.json', 'w') as tracks_json:
40:    json.dump(tracks_list, tracks_json)

Tunefind’s website has index pages for each series with links to each of their seasons, which link to pages for each episode that contain track details. The scraping code iterates over the seasons (lines 14–24), then the episodes (lines 26–32).

The loop starting on line 20 appends a tuple of the track title and artist name, found in line 30, to tracks_list. This is bluntly weeded for exact duplicates in lines 34–37. (Subtly different entries for the same track aren’t affected, so they have to be picked out when adding the tracks to Spotify, detailed below.)

Lastly the list is written to a file; I chose JSON because I wasn’t sure what I was going to do next. The tuples are converted to lists, which is fine for our purposes.

Doing something with it

We’ve now got a nice file of titles and artists — 789 tracks in total. The next question is how to use it. My goal was to construct a Spotify playlist, as the service has a large catalogue of tracks available for free and it would take no effort to use once set up.

Initially I considered using the Spotify API, but it appeared too daunting and the quality of the data would derail attempts to add tracks programmatically. Instead I settled on scripting the desktop client. That’s easier said than done, as its AppleScript support is poor.

When I publish I always check the post’s links. Thank god I did, because Spotify has recently overhauled its developer site. The metadata API looks far more approachable.

However, discrepancies between the Tunefind data and Spotify’s catalogue would still likely cause headaches. Also, adding tracks to a playlist is still reserved for its C API.

After mulling over how much manual work I was willing to do, I came up with the idea for an AppleScript scaffold that, since I couldn’t script Spotify directly, would script around Spotify: using its URL scheme to search for tracks and a dialog box with controls to move through the list.

Underneath that would be Python that managed the track list, assembled the URL and handled the controls:

python:
 1:  #!/usr/local/bin/python3
 2:  
 3:  import json
 4:  import subprocess
 5:  
 6:  position = open('/Users/robjwells/Desktop/position', 'r+')
 7:  reported = open('/Users/robjwells/Desktop/reported', 'a')
 8:  
 9:  
10:  def asrun(script):
11:    "Run the given AppleScript and return the standard output and error."
12:  
13:    osa = subprocess.Popen(['osascript', '-'],
14:                           stdin=subprocess.PIPE,
15:                           stdout=subprocess.PIPE)
16:    return osa.communicate(script.encode())[0]
17:  
18:  
19:  with open('/Users/robjwells/Desktop/tracks.json') as tracks_json:
20:    tracks_list = json.load(tracks_json)
21:  
22:  ascript = '''
23:  tell application "LaunchBar"
24:     perform action "Open Location" with string "spotify:search:{0}"
25:  end tell
26:  
27:  tell application "Finder"
28:     set dialog_result to display dialog "Ready for next track?" ¬
29:         buttons {{"Report", "Stop", "Next"}} default button "Next"
30:     return button returned of dialog_result
31:  end tell
32:  '''
33:  
34:  
35:  def prep_track(t):
36:    joint = ' '.join(t)
37:    return joint.replace(' ', '+')
38:  
39:  
40:  def prompt(t):
41:    script = ascript.format(prep_track(t))
42:    result = asrun(script).decode().strip()
43:    if result == 'Stop':
44:      return False
45:    elif result == 'Report':
46:      reported.write(json.dumps(t))
47:      reported.write('\n')
48:    position.seek(0)
49:    position.write(str(tracks_list.index(t) + 1))
50:    return True
51:  
52:  
53:  raw_pos = position.read()
54:  if raw_pos:
55:    pos = int(raw_pos)
56:  else:
57:    pos = 0
58:  
59:  for track in tracks_list[pos:]:
60:    if not prompt(track):
61:      break
62:  
63:  position.close()
64:  reported.close()

According to the file metadata, I created this script at 12.50am. Some of it is hilariously bad. I haven’t tidied up these scripts but I had to change a bit where I trampled all over my own global names. It’s in desperate need of re-ordering so I’m going to work through the script in the order parts are used, not in which they’re written.

At its core, it’s a little Python engine that uses Dr Drang’s asrun function to run an AppleScript via osascript.

We start by opening a position file to store our place in the track list and a reported file to store any tracks which aren’t in the Spotify catalogue (lines 6 & 7). Next we open and parse the JSON track list (lines 19 & 20).

Lines 53–57 determine from the position file where to begin in the track list, with pos used to slice it in line 59. The selected part of the list is iterated over and each track tuple fed to the prompt function (lines 40–50).

This formats the track (using prep_track to join the artist and title, and replace spaces with plus signs) and inserts it into the AppleScript (line 41), which is then run.

The AppleScript opens the Spotify search URL and displays a dialog box:

Screenshot of the dialog box on top of the Spotify search results

The chosen button is returned and assigned to result. If the searched-for track couldn’t be found, the “Report” button adds it to the reported file (lines 46 & 47) and proceeds as if “Next” had also been clicked: writing the index of the next track to the position file and returning True to cause the loop in lines 59–61 to continue. “Stop” returns false and ends the loop — and so the script.

Within the AppleScript itself (lines 22–32), LaunchBar is used to open the Spotify. This is holdover from when I was trying to use LaunchBar’s “Browse in Spotify” action (select Spotify and press space) instead of the search URL. It didn’t work. There’s no reason to use LaunchBar anymore, this would work fine:

open location "search:spotify:{0}"

Where {0} is a replacement field.

Once the sliced list has been iterated over, we clean up by closing the position and reported files (lines 63 & 64).

The script does a decent job of working around the inability to directly control Spotify. It leaves me to pick an appropriate track from the search results shown, drag it to a playlist and chose the appropriate next action from the dialog. It turned a mountain of tedious work into a reasonably-sized hill.

Being human, though, I occasionally picked the wrong button in the dialog, meaning some tracks weren’t reported or were reported in error. Ah well. I was still left with a playlist of about 570 tracks.

A musical interlude

Since creating it in mid-December, I’ve been listening to the playlist and marking my favourite tracks so I could buy them on iTunes. There’s too many to do by hand, so a bit of automation is called for.

Dragging a track from the Spotify client gives you an http://open.spotify.com/track/… URL. Opening it in a browser will likely get you the newfangled web player, which isn’t much use, but tools such as curl still return the source to the detail view you used to get. We’ll use this to turn the URLs back into track titles and artist names, and from there into iTunes links.

python:
 1:  #!/usr/local/bin/python3
 2:  
 3:  import os
 4:  import sys
 5:  import requests
 6:  from bs4 import BeautifulSoup as bs
 7:  
 8:  spot_path = os.path.join(os.getcwd(), sys.argv[1])
 9:  itunes_search = 'https://itunes.apple.com/search?term={}&country=gb'
10:  
11:  with open(spot_path) as spot_file:
12:    spot_links = spot_file.read().splitlines()
13:  
14:  
15:  def spotify_to_itunes(link):
16:    spot_soup = bs((requests.get(link)).text)
17:    title = spot_soup.h1.text
18:    artist = spot_soup.h2.a.text
19:    plussed = '+'.join([title, artist]).replace(' ', '+')
20:    itunes_response = requests.get(itunes_search.format(plussed))
21:    if itunes_response.json()['resultCount']:
22:      return itunes_response.json()['results'][0]['trackViewUrl']
23:    else:
24:      return ' '.join([artist, title])
25:  
26:  for line in spot_links:
27:    print(spotify_to_itunes(line))

Dragging several tracks from Spotify gives you URLs separated by newlines, which I saved to a file that is read and split in lines 11 & 12. We iterate over each URL at the end of the script, printing the result of passing the URL to spotify_to_itunes, which does the hard work.

We get the source for the track’s detail view in line 16 and pull out the title and artist, which are joined and the spaces replaced with plus signs in line 19 to make the string URL safe — like in the previous script.

This combined string is used to search the iTunes catalogue (line 20, with the URL in line 9). If there are any results we return the URL for the top hit, otherwise we return the artist name and track title so it can be looked up manually.

Taking the URL for the top iTunes result is potentially hazardous, but here I’m happy enough to put my faith in its search. Also the script takes two requests to process each track (lines 16 & 20) so it is slow, but as it only needs to be run once so I don’t think it really matters.

I decided not to rig up the AppleScript side here, because in my mind it doesn’t require the same kind of industrial processing. At this point it’s a matter of taking time to listen to the track previews and decide which to buy.

Also in contrast to the previous scripts, I print to stdout here instead of writing directly to a file. I don’t have reason for this but I guess partly it’s because there’s no data structure to preserve, just a set of lines to transform.

At this point I’ve run the script and redirected the output to a file. My plan is open it in BBEdit and use ⌘L to select each, send it to LaunchBar (using the Instant Send feature) and open the URLs in Safari and the leftover tracks in iTunes’s search. Thankfully the iTunes links just load a web preview and don’t activate iTunes itself.

I’ve done this already with the tracks in the reported file, run through a modified version of the above script. It works reasonably well.

Wrapping up

While writing (and re-writing) this post I began to doubt its value. These aren’t polished scripts and this shouldn’t be taken as a guide, but at the same time it is how I solved the problem. Examining the choices made — however little thought went into them — has been instructive for me and hopefully will give people facing similar problems something to think about.

And while I’d hesitate to call this a real problem, the cobbled-together solution has at least produced something I’m still listening to after two months.

Die, bookmarks bar, die

I hate the bookmarks bar. This thing:

The bookmarks bar in Safari

It’s all about the mouse here. Keyboard access is through ⌘1 for the first entry and so on, which often leaves me counting along to find which number I need.

What’s the number for Pinboard: Unread? One… two… three… four!

That’s gross and slow. It’s exactly the same reason why I use InDesign’s quick apply panel at work and don’t even try to remember the per-style keyboard shortcuts (which, handily, only work with the number pad).

It also applies to keyboard shortcuts more generally — if you don’t use them often enough you’ll forget what the shortcut is, annoying you later and often taking more time that it would to grab the mouse.

Assigning less-arbitrary shortcuts can be helpful, but again only if you use them fairly regularly.

Back to that horrid bookmark bar, though. How do we construct a Launchbar-style, search-based panel to get at those bookmarks? We’ll use Launchbar itself. (I guess you can do this in Quicksilver or Alfred, but I haven’t tried.)

Safari’s AppleScript support is fairly poor — it doesn’t expose your bookmarks to scripts. But it does have a handy do JavaScript command, which is all that those bookmarklets do: run JavaScript.

Grab the code (stored as the bookmark’s address) and — importantly — decode it to turn entities such as %7B into {. Lop off the leading javascript: and insert it into this tiny AppleScript:

applescript:
tell application "Safari"
  set bookmarklet to "your_bookmarklet_code_here"
  set current_tab to the current tab of the front window
  do JavaScript bookmarklet in current_tab
end tell

Save that script into a new folder — mine’s in ~/Library/Scripts/Applications/Safari/Bookmarklets/

Lastly add that new folder to Launchbar’s index. I have my entry set to access on sub-search only, so to get to the bookmarklet scripts I type bookm → and I end up with this panel:

Launchbar’s list of my Safari bookmarklet scripts

And we’re set: a quick, searchable, keyboard-friendly list. If, like me, you only use the bookmarks bar for JavaScript bookmarklets you can go ahead and hide it as an added bonus.

Date suffixes in Python

A little while ago I wrote about using TextExpander to write dates, the bulk of which was given over to an Applescript that returned a long date complete with date suffixes.

Since I’ve been writing more and more Python, I thought it would be fun to rewrite it to see the difference between the two.

First, here’s my original Applescript:

applescript:
 1:  set theDate to the day of the (current date)
 2:  set theDay to the weekday of the (current date)
 3:  set theMonth to the month of the (current date)
 4:  set theYear to the year of the (current date)
 5:  
 6:  set lastChar to (the last character of (theDate as string)) as number
 7:  
 8:  if lastChar > 3 or lastChar is 0 or (theDate > 10 and theDate < 21) then
 9:    set theDate to (theDate as string) & "th"
10:  else
11:    set theSuffixes to {"st", "nd", "rd"}
12:    set theDate to (theDate as string) & (item lastChar of theSuffixes)
13:  end if
14:  
15:  return (theDay & ", " & theMonth & " " & theDate & ", " & theYear) as string

And now the Python:

python:
 1:  #!/usr/local/bin/python3
 2:  
 3:  from datetime import date
 4:  
 5:  today = date.today()
 6:  date_string = today.strftime('%A, %B #, %Y')
 7:  day = today.day
 8:  
 9:  if (3 < day < 21) or (23 < day < 31):
10:    day = str(day) + 'th'
11:  else:
12:    suffixes = {1: 'st', 2: 'nd', 3: 'rd'}
13:    day = str(day) + suffixes[day % 10]
14:  
15:  print(date_string.replace('#', day), end='')

Originally the Python script used the /usr/bin/env python3 shebang line, as I do in almost all of my scripts. Unfortunately that doesn’t work with TextExpander as its $PATH is set to /usr/bin:/bin:/usr/sbin:/sbin, so you’ll need to hard-code the path to your Python interpreter.

A date object, kind of similar to Applescript’s current date, is constructed on line 5. This is used to build a formatted date on line 6, which is complete bar the day of the month. I swap it out for a # placeholder because both strftime options for the day fall short: %d is padded with a zero, and %e is padded with a space. So instead I pull out the day separately in line 7.

The date-testing logic on line 9 is simpler and more explicit in this version, testing first for any date that takes a -th. Those dates that require a different suffix get dumped to the else clause.

A bit of modulo arithmetic on line 13 reduces the date integer to the units digit, which is used to key into a dictionary holding the suffixes.

The day and its suffix are concatenated, and are inserted into the formatted date on line 15 using the replace string method (god I love Python’s string methods) and printed, with end='' suppressing the standard newline.

Though the line count is the same between the two, the Python version has about half the characters of the Applescript. I think the Applescript’s verbosity — which I don’t help with that conditional — makes it harder to understand at a glance, while the Python is very clear, excepting the strftime format string.

As a side note, I don’t actually use either of these scripts for my long date, having given in to “Weekday month day year” without a suffix or any commas. This can be constructed using TextExpander’s own date macros, which mirror strftime formatting: %A %B %e %Y. Still, a fun exercise.

Restart in Windows: The script strikes back

A quick and hopefully final update on my short AppleScript to restart from Mac OS X into your Bootcamp partition.

The last time we looked at this I had cobbled together some AppleScript, grep and awk for a fairly reliable script, but it required some fiddling if you were still using Leopard (10.5).

I’ve revised the script again, this time relying on the $NF variable in awk to extract the last field. That makes using diskutil list easy, and so I now recommend the following for all users, including those on Leopard or Tiger:

applescript:
set deviceID to (do shell script "diskutil list ¬
  | awk '/YourBootcampPartition/ {print $NF}'")
do shell script "bless -device /dev/" & deviceID & ¬
  " -legacy -setBoot -nextonly" ¬
  with administrator privileges
tell application "Finder" to restart

Here’s the shell part to get the identifier on its own:

bash:
diskutil list | awk '/YourBootcampPartition/ {print $NF}'

To be clear, the name of the partition must go between the two slashes.

Aside from making it a purely shell script, which causes problems with prompting for the user’s credentials, I can’t see how this can get any simpler and so I fully expect this to be the last time I write about it.

I’ve uploaded the script as a Gist, if you’d like to star or fork it.

Setting a date with TextExpander

I suck at dates. Badly.

Thankfully, there are a few ways to get around this in software — not least TextExpander, Smile’s ace text-substitution program. (I don’t use it nearly enough, but I’m looking to fix that.)

TextExpander has a lot of date placeholders, so it’s obvious that I’m not the only one that needs help with this — but I may well be the first to be this bad. Here’s a list of the snippets in my Time & Date folder:

;mdate   ->   March 6 2013
;*ddate  ->   Wednesday March 6 2013
;ds      ->   _0306_0914
;time    ->   09:14
;bd      ->   20130306
;ldate   ->   March 6, 2013
;sdate   ->   06/03/2013
;dt      ->   09:14, March 6, 2013

Those eight snippets include seven different ways to get the date, and three different ways to get the time. The worst part about this is that I can never remember which abbreviation corresponds to which date format.

Ok, let’s fix this. How many formats do I really need? I need a short date, a more human-friendly long date, the time, and the format used at work.

So that’s four — only half as bad as before!

Time

I’m sticking with my old ;time snippet here, a colon-separated 24-hour time. If I need a more human-friendly time, typing 10am or 8pm or whatever is as easy as typing an abbreviation (and TextExpander only does uppercase AM/PM, which I dislike). Here’s the snippet:

%H:%M   ->    10:23

Short date

Because, well, XKCD, I’m going to use the ISO 8601 date format for this. It means I can dump my separate backwards date snippet and avoids any confusion across DD/MM/YY and MM/DD/YY countries.

%Y-%m-%d    ->  2013-03-06

Long date

Since this is implicitly not the format to use when you’re counting characters and one that’s explicit enough to avoid confusion, there’s quite a bit of personal freedom here.

This one’s going to be more complicated than the previous two, as I want to use date suffixes (March 2nd, April 21st, etc). But TextExpander doesn’t include a suffix placeholder, so let’s take advantage of its script support.

AppleScript’s built-in current date command returns an object from which we can extract all the pieces we need to construct a date, like so:

applescript:
set theDate to the day of the (current date)
set theDay to the weekday of the (current date)
set theMonth to the month of the (current date)
set theYear to the year of the (current date)

Now to build the suffix. Most dates have the suffix “th”, so we want to focus on the special cases: 1, 2, 3, 21, 22, and 31. We could check for each of these but there’s a smarter way.

We need to pull the date’s last character, which determines the suffix, so we can work with both single and double-digit dates:

applescript:
set lastChar to (the last character of (theDate as string))¬
 as number

Now we check for the most common situation (“th”) by testing lastChar against a range:

applescript:
if lastChar is 0 or lastChar > 3 or (theDate > 10 and¬
 theDate < 21) then
    set theDate to (theDate as string) & "th"

If we put the other suffixes in order and in a list, we can use the last character of the date itself to extract the correct suffix:

applescript:
set theSuffixes to {"st", "nd", "rd"}
set theDate to (theDate as string) & (item lastChar¬
 of theSuffixes)

Now let’s roll it up and return a constructed date:

applescript:
set theDate to the day of the (current date)
set theDay to the weekday of the (current date)
set theMonth to the month of the (current date)
set theYear to the year of the (current date)

set lastChar to (the last character of (theDate as string))¬
 as number

if lastChar > 3 or lastChar is 0 or (theDate > 10 and¬
 theDate < 21) then
    set theDate to (theDate as string) & "th"
else
    set theSuffixes to {"st", "nd", "rd"}
    set theDate to (theDate as string) & (item lastChar¬
     of theSuffixes)
end if

return (theDay & ", " & theMonth & " " & theDate & ", " &¬
 theYear) as string

End result

Hey! You made it! I was sure I’d lose everyone in the middle of breaking down that AppleScript.

So, here’s what I’m left with:

;iso    ->    2013-03-06
;ldt    ->    Wednesday, March 6th, 2013
;tm     ->    10:21

Mix in one for work:

;*dt    ->    Wednesday March 6 2013

And we’re all set.

Restart in Windows: Revenge of the Script

UPDATE 2013-05-25

I’ve changed the script to fix the fiddliness on Leopard. Please see my new post for details. I’ve left the rest of this post mainly intact, but please bear in mind that the code has changed slightly.

Want to quickly restart your Mac in your Windows Bootcamp partition, without having to hold down option/alt as the computer starts?

Click this link, which will open an AppleScript in your default editor, and replace “YourBootcampPartition” in the first line with your Windows partition’s actual name (making sure that is exactly the same, as it’s case sensitive). Then save it as a .scpt file in your user scripts folder, which you can access through the system-wide script menu (which is turned on in AppleScript Editor’s preferences).

Then, to restart, just pick it from that script menu.

Extra, extra

Using Leopard (10.5) or earlier? You’ll have to change the first line as “diskutil info” doesn’t work with volume names. Try this:

applescript:
set deviceID to (do shell script "diskutil list | ¬
  grep YourBootcampPartition | awk '{print $8}'")

That should return something like “disk0s4”. If it doesn’t change “print $8” to a different number, probably 7. (I recommend you try this in the Terminal until you’ve got it right, then adjust the AppleScript. The number refers to the column of the information.)

The above isn’t true anymore. Please see my latest post about it.

Don’t want to have to type your password in every time? Add “password “YourPassword” ” after “ -nextonly” ” on the second line, then save your script as run-only. I do not recommend this. Your password will not be encrypted. It is only slightly less insecure than keeping your password in a text file. Please, do not do this.

Using it

I’ve got it saved to my ~/Library/Scripts folder, and have Launchbar set to index that folder so I can just type “Restart in Windows” into the prompt, hit enter, type my password and then I’m set.

You could also use FastScripts, a really nice replacement for the system script menu that you can use to give the script a keyboard shortcut.

I don’t recommend that you save it as an application, which would let you add it to your dock. I did that once and got stuck in a restart loop, thanks to Lion’s ability to re-open applications that were running when you restarted. You can disable that feature in Mountain Lion, but I still strongly advise that you keep it as a plain old script file.

The code

applescript:
set deviceID to (do shell script "diskutil info ¬
YourBootcampPartition | grep Identifier | awk '{print $3}'")

do shell script "bless -device /dev/" & deviceID & ¬
" -legacy -setBoot -nextonly" with administrator privileges

tell application "Finder" to restart

The story

I talk about the origins in my first post about the script. Basically, holding down option on my keyboard was unreliable because the computer had booted up too fast for the Bluetooth connection. I’d been using Bootchamp, a menubar utility, for a while but you couldn’t trigger a restart from Launchbar and had to mount the partition first.

The first version relied on the disk identifier never changing. You can’t rely on a partition having the same identifier after a restart, so the script would silently fail.

The next version relied on the disk identifier being in a certain character position, which (aside from being a dirty hack) was also unreliable if you had more than 10 disks connected.

The most recent (and probably final) version uses grep and awk on the command line to fetch the disk identifier after pulling up the information for the partition. (In Leopard it has to get the identifier from the list of every partition, as “diskutil info [PartitionName]” only works in Snow Leopard and later. As such it’s a little less reliable and needs a bit more tweaking by users.)

For such a simple script (three whole lines!) this has taken more work than I originally anticipated. But it’s also taught me a fair bit about the command line and how to retrieve information from it.

So, yeah. I think it’s done now. If you have any questions or comments please message me on Twitter or add a comment to the script’s gist.

What’s in the box!?

Become vengeance, David. Become wrath.

Accessing AppleScript records using a variable

AppleScript gets a lot of stick for being a rubbish language. And it is. But before last night there wasn’t anything that really threw me and made me think “wow, that’s incredibly, impressively, mind-meltingly dumb”. (Admittedly, I’m not a programmer.)

But it happened. Here’s the example code:

applescript:
set myRecords to {Abc:"1", Def:"2", Ghi:"3"}
set myVar to "Abc"
get myVar of myRecords

Try that and you’ll get a big fat error message. Why? Because there’s no property in myRecords that’s literally called myVar.

“But it’s a variable,” I hear you sob quietly into Script Debugger. I know, I know. You were after the value of property “Abc”. Unfortunately AppleScript doesn’t let you access records in this way. If you want to access that property you need to refer to its “real” name directly in the code.

Even to me this is ridiculous. I’ve learnt a little JavaScript with Codecademy recently and you can do this very easily:

javascript:
var theRecords = {
    Abc: 1,
    Def: 2,
    Ghi: 3
};
var theVar = "Abc";
console.log(theRecords[theVar]);

This becomes important when you want to do something else and then refer back to the record.

What I wanted to do is store a simple name for a more complex string, and then later pass the simple name to the record and have it return the longer, hard-to-remember string. But there was no way I could hard-code the property name without defeating the entire purpose of the script I was writing. I just wanted to pass a variable containing the simple name string.

But no. Not in AppleScript.

Well, not quite. The language does not support doing this, true. But the language does let you contain a script within a script — which allows us to get past the “no variables to records” rubbish.

Thanks go to the heroic jobu on the MacScripter forums, who posted this code in 2006. (Despite being so long ago, this technique doesn’t seem to have gotten much attention, as looking at many, many forum posts about this issue has shown.)

Let’s have a look:

applescript:
set myRecords to {Abc:"1", Def:"2", Ghi:"3"}
set myVar to "Abc"

return (searchRecord(myVar, myRecords))

to searchRecord(theKey, theRecord)
    run script "on run{theKey,theRecord}
        return (" & theKey & " of theRecord )
        end" with parameters {theKey, theRecord}
end searchRecord

The important thing to note is that when the variable (in this case, myVar) is passed to the run script code, the variable’s contents are passed. That’s what you’d expect it to do, but what that means is that the variable is essentially hard-coded into this new script — you’re not recreating myVar, you’re just passing along myVar’s contents.

Boom. A regular “get value of property x of the record.” And that data is then returned from the internal script to the parent, containing script. You can then call this as you would a subroutine handler, whenever you need to get a value for an unknown-at-compile-time property.

Essentially, the shopkeeper won’t sell to you in person, but if you step outside and ring him then you’re set.

Yeah. It’s a hack. Thankfully a short, understandable one, but a hack nonetheless. It’s crazy and this should be built into the language. Here’s JavaScript’s way again, for reference:

javascript:
objectName[myVariable];

Why is this missing? God knows.

I’m pretty sympathetic towards AppleScript. It’s the first language I’ve ever done any kind of programming in, and it’s been — and continues to be — useful at home and essential at work. I do like it, and there’s things you can’t do without it — for better and for worse.

I plan to learn Python at some point so I may end up leaning towards “for worse” quite heavily in the future, but at the moment I’m happy using AppleScript to accomplish the (mostly work-related) things I need to do, even with all of its oddities.

(My “favourite” weird things are:

  • Getting the wrong tell target (generally)

    In BBEdit (for example) do I target “text window 1” or “text of text window 1”? It seems to be “text of” but I’m sure there are situations where the other is needed. The reasons why they’re two separate things aren’t clear.

  • Bizarrely-scoped InDesign properties

    Did you know that “page range,” used when you export a .pdf, is an application-wide setting that you have to specify beforehand, not when you actually export the .pdf? Yeah.

  • Setting page-item InDesign properties

    Let’s create a new frame. When we make it we can give it a bunch of properties, such as its position, size, stroke, fill, layer. How about object style? It’s listed in the options! Let’s set it alongside the position and size. Nope! It fails silently and you’re left with an unstyled frame. The way around this is to assign your new frame to a variable, and then apply the object style after you create it.

There’s a bunch more, like slash-separated paths being treated differently from colon-separated paths — often requiring a line of code to convert them, because in many cases you can only get one type. But whatever. I’m a little worried that “proper” programming languages won’t be masochistic enough.)

Jesus Christ. Somebody call somebody. Call somebody.

Everyday automation

Learning AppleScript can really pay off.

It isn’t always the easiest language to work with, as anyone who’s ever used it will tell you. Commands that look like they should work sometimes don’t, you occasionally have to use bizarre methods to refer to that thing right there. It’s not a picnic.

But I’ve been using it for a little while and am pretty comfortable with it. I’ve ended up using it more and more for everyday tasks, or one-off situations that would usually require a lot of donkey work.

Let me share one that’s just happened.

I was browsing the New Yorker’s print shop and — I thought — was opening up the individual pages for covers I liked and wanted to go back to after I’d finished browsing.

It turns out that I was just opening up the preview images in tabs. Whoops. And their URLs don’t contain a unique code that’s shared with their purchase pages. Ah. And there are 10 of them. Erm.

The one similarity they do have is that the filename is kinda sorta repeated on the purchase page, but it’s hyphen-separated so (I thought, wrongly) you can’t just copy and paste it into a search field.

So I cooked up this little script in about a minute:

applescript:
tell application "Safari"
    set theURL to (get the URL of the front document)
end tell

set printsOffset to (get the offset of "/prints/" in theURL)
set jpgOffset to (get the offset of ".jpg" in theURL)

set hyphenString to characters (printsOffset + 8) ¬
through (jpgOffset - 1) of theURL as text
set AppleScript's text item delimiters to "-"
set theWords to (get the text items of hyphenString)
set AppleScript's text item delimiters to " "
set searchString to theWords as string
set AppleScript's text item delimiters to "" -- return to default

tell application "Safari"
    tell the front document
        search the web for searchString
    end tell
end tell

Then I saved it, gave it a shortcut in FastScripts, opened up the first tab in Safari, hit the shortcut, tabbed to select the first URL in the search results, then hit enter to load it. Rinse, repeat.

As I was writing this I tried just pasting the filename into Google and — shocker — that works. Which makes me look a little dumb.

And the script isn’t even that sophisticated; I could have used a repeat command to loop through each tab, for example.

However my wider point about getting your teeth into automation still stands. I’ve written loads of little scripts like this over the past few months and, combined, they’ve saved me bags of time and plenty of frustration.

Hell, the InDesign script I wrote to set a text frame’s name (≥CS5) to its script label (≤CS4) alone saved me half an hour.

The problem of AppleScript being frustrating to learn — and continually frustrating with certain applications’ opaque AppleScript dictionaries and command parsing — does stand in the way of more people using it (which is why Automator exists, as limited as it is).

But if you’re tempted, even a little, give it a go. If you don’t have any programming experience I recommend AppleScript 1-2-3. (Don’t buy the Kindle edition. You want the paper copy.)

Also make sure to check out Script Debugger 5, which at $199 (about £170 including VAT) is a bit pricey if you’re just starting but makes things so much easier it’s a joke. It’s like switching from MSPaint to Photoshop. You’ll never go back to (Apple)Script Editor.

Update 2012-09-12

“Learning AppleScript can really pay off. Once you’ve suffered through its oddities enough to be able to write it without weeping, you’ll be tempted to write scripts that achieve things you can do with copy and paste, like this one.”

Maybe I should have thought harder about that first line.

Update: Restart in Windows

Update 2013-05-25

Please ignore this post and look at my most recent post about the script.

I came a across a more sophisticated method of getting your Bootcamp partition’s device ID for my restart in Windows AppleScript, so you may want to grab the new code (also below).

It no longer uses my hack to pull the device ID from a certain character position in the result of a shell command — which would break if you had more than 10 disks or partitions anyway — and instead just uses grep and awk to strip out the identifier. It should be pretty solid.

applescript:
set deviceID to (do shell script "diskutil info [Your Bootcamp ¬
partition's name] | grep Identifier | awk '{print $3}'")
do shell script "bless -device /dev/" & deviceID & " -legacy -setBoot ¬
-nextonly" password "XXXXXXX" with administrator privileges
tell application "Finder" to restart

Restarting in Bootcamp the easy way

Update 2013-05-25

Please ignore this post and look at my most recent post about the script.

I’ve used a Mac — and Windows on a Mac — since 2007 but I’ve just, only just, figured out a way to quickly restart from OSX into Windows.

And it’s really easy to use it on your computer too, even if the words code or script make you freak out a bit. (If they don’t, skim the post for the meat but make sure to read the last few paragraphs.)

The setup

I use LaunchBar to run a script that mounts my Bootcamp partition, sets it as the startup disk for the next time only and then reboots into it. It means I don’t have hold down Option (⌥) while the computer boots up, which I’ve found very unreliable now that I use a bluetooth keyboard. (It might actually just be the SSD booting quicker than the keyboard can switch on. That’s a nice problem to have.)

As it’s just a bit of AppleScript you can also run it from FastScripts or the system-wide script menu.

The script

In this old post, I suggest you place your password in the script and am naive about the protection offered by saving it as run-only. Please ignore this post and read my new one.

applescript:
set deviceID to (do shell script "diskutil info [Your Bootcamp ¬
partition's name] | grep Identifier | awk '{print $3}'")
do shell script "bless -device /dev/" & deviceID & " -legacy -setBoot ¬
-nextonly" password "XXXXXXX" with administrator privileges
tell application "Finder" to restart

Update 07/09/2012: I’ve changed the script to get the device ID in a more sophisticated way, so it’s far less likely to break even if you have more than 10 disks.

Update 21/07/2012: OSX doesn’t guarantee that your Windows partition will get the same device ID every time the computer boots, which means my previous method (which was dependent on the device ID not changing) can and will break if you have more than one HDD.

I’ve changed the script to use the partition’s name and then pull the correct device ID from the result of diskutil info [Partition Name]. If you have more than 10 disks you may run into problems.


First of all, open up AppleScript Editor (called Script Editor before Lion) from the Utilities subfolder of your Applications folder and copy the code above into a new window.

Then open Disk Utility from the Utilities folder, select your Bootcamp partition, click the Info button and change [Partition Name] in the script to its exact name. It should look something like this:

applescript:
set terminalRAW to do shell script "diskutil info BOOTCAMP"

If the partition’s name is more than a single word, put it in quotes with backslashes in front, like this: \"Windows HDD\".

Now change myPassword to your user account’s password and save the script, making sure to check the box next to Run Only at the bottom of the save sheet.

(Saving a run-only script makes sure no-one will be able to open it and grab your password. If you’d rather not put it in there to begin with just delete password "myPassword" and the system will prompt you for it every time you run the script.)

What should I do with it?

Generally the best place to put your finished script is in ~/Library/Scripts. This means it’ll appear in the system-wide script menu, which you can enable from AppleScript Editor’s preferences, as well as FastScripts. (By default the user library is hidden in Lion so just copy that path, switch to the Finder, click Go in the menu bar and choose Go to Folder and paste it in).

To use it in LaunchBar you’ll want to go into its indexing preferences (⌘⌥I), select Actions, then Options, and check the box next to ~/Library/Scripts. I use “Restart in Windows” to keep things simple.

Why not save it as an application?

I tried this and it didn’t work out well. Even though it should quit immediately after the final command (restart) is passed, Lion’s resume-after-restart feature (where it reopens running applications) can put the computer into a loop where booting back into OSX from Windows causes the computer to boot into Windows.

(If you accidentally get into this position perform a safe boot by holding shift from the Mac chime until you see the Apple logo. Then log in, delete the application and empty the trash.)

Thankfully the only advantage to having it as an application, that I can see, is that you can put it in the dock. (And you don’t want to be one of those people.)