- Facebook and the New Colonialism (the Atlantic). The articles cites a scholar in post-colonialism identifying traits of colonialist behaviours. Prompting the slightly unsettling exercise of trying to apply said list to other things, such as proponents of Gold Open Access:
Take the six signs of post-colonial behaviour in this article and apply to APCs: https://t.co/bh2HlkFVig
— janeschmidt (@janeschmidt) February 12, 2016
- Several intriguing topics in the Planet Money Valentine’s List episode. Yet another word of praise for Marie Kondo, once again carefully avoiding to talk about the root cause of “clutter”, which is that we need to stop buying so much stuff. But I guess you can’t say that if you’re an economist. But you can if you’re a Star columnist (or you can pin this down as yet another Millenial thing if you’re a minimalist blogger). This brilliant line by Matt Levine: the essence of finance is time travel. And also why we all use Q-tips to clean our ears even though we shouldn’t.
- Early 2016 is brutal to artists. But it’s the news of Umberto Eco’s death that unsettled me the most, and Geoff Manaugh of BLDGBLOG echoed that feeling perfectly in his eulogy. Foucault’s Pendulum is one of my favourite novels of all time.
- Last week saw a brief spike in interest from non-librarians towards the “digital divide” in academic publishing (an overly simplistic way to put it would be Open Access vs paywall-model). Besides the fact that none of it is new, this gem of an interaction with SciHub is indicative of how broken the model of the librarian-as-an-information-broker is:
@GaviaLib that is bullshit
— Sci Hub (@Sci_Hub) February 18, 2016
In the midst of all this, this study about how the use of SciHub’s low-tech predecessor #icanhazpdf highlights that there might be a steady demand for open access content from academics, but that they might not be actually aware of the alternate routes to get it. This is also interesting from a purely data-analysis perspective. Archivist Allana Mayer mentioned using TAGS as a way to collect and analyse such tweets, and helpfully shared her search equation.
- Toronto’s Bloor-Danforth subway line (now called line 2) is turning 50 this February. This exhibit looks worthy of a visit.
Edit (January 2019): It took me a long time to fully import my Aperture libraries to Lightroom! I started this process in February 2016 and migrated my last pictures in January 2019. A few things have since then changed, so I’ve updated this post accordingly.
A TL;DR version of this post is available below, summarizing the steps in my migration workflow.
The release of Aperture in 2005 aligned well with my increasing interest in photography at the time. Like many Apple products before, Aperture offered a unique combination of powerful features, ease of use, and good integration with other Apple products and workflows. After years of trying out various tools to (first) organize and (as I started researching photographic techniques) retouch my photos, Aperture was combining both in a unique way. I was convinced and soon bought a copy of the software. And again, like many Apple products before, a competitor soon released a very similar product. Adobe Lightroom came out in early 2007, and quickly become the market leader.
Even as I watched many of my photography friends progressively adopting over to Lightroom, correctly identifying the winning horse early on, I refused to see the writing on the wall and stuck to Aperture. My behaviour in that aspect is pretty acutely described in Alex Allen’s essay “Why we treat tech companies like religions” (mentioned on Spark #295):
And when you’re that tied in to an ecosystem, you have a motive to want to defend and validate the choice you’ve made
But of course all my photography friends were right to choose Lightroom, as demonstrated by Apple finally announcing they were pulling the plug in 2014. We are now in 2016 and I believe my period of
mourning denial has lasted long enough. It is time for me to migrate to Lightroom. I spent some time thinking of the best way to do this and experimenting, and I’ve decided to document the process here in case someone finds it useful.
1. Upgrade to Aperture 3.6
Lightroom has a handy feature for importing Aperture libraries (Adobe being all to happy to welcome the Aperture castaways…). After briefly considering starting from scratch, I decided to use it to at least import all the metadata I had been carefully adding to my pictures (geolocation, tags, etc.). However the Lightroom import wizard warned me that my version of Aperture wasn’t the latest, and that it could cause some errors while importing, without however offering any details on what exactly would be impacted. I wasn’t able to find more information online about that either. In the end, I decided to upgrade to the latest version of Aperture, which wasn’t entirely straightforward with my setup. I have an older Mac Pro which I am loathe to run on the latest OS X version for performance issues. The last version of Aperture (3.6) does not run on anything prior to Yosemite, however. I ended up upgrading my Mac Book Air to El Capitan, install the latest Aperture on it and use it to upgrade my libraries. Convoluted, yes.
Edit (January 2019): I did eventually upgrade my Mac Pro to El Capitan and was able to finish my migration with fewer complications.
To make things even easier, I had bought the software (and any subsequent updates) as a physical DVD (2005, early adopter, remember?). But Apple at some point decided to distribute updates only through their App Store. Updates are only available to users who have previously bought the software on the App Store. Problem for me, since I had purchased a boxed version of the software. After scouring the forums for a solution that did not involve downloading a shady torrent file, I found mention of a user in the same situation who got Apple Support to add Aperture to their App Store purchases. I dutifully called the support line (my favourite activity in the universe). Let’s just say it wasn’t a short call. And it required me refusing to purchase Aperture again at some point (sic) and being a big enough pain that I finally got redirected to someone who understood the situation and provided me with a one-time code to “purchase” Aperture at no cost on the App Store. I was then able to upgrade to the latest version. Finally.
2. Split my library into manageable chunks
Now armed with the latest (and last) version of Aperture, I tried starting the Aperture import wizard in Lightroom, which asked me to locate an Aperture library and then started “verifying” it before offering me any further option. I waited several hours before eventually aborting the operation, since there was no sign how much longer this would take. It was evident my Aperture library was too large to be imported in one go, I would have to split it into smaller chunks. Doing it in batches also made me less nervous about missing a step and then having to start all over again, or messing everything up somehow.
I decided to do this by year. I first created a series of smart collections, filtering by date
Then I exported each as a separate library. I didn’t check the boxes to copy the originals or the previews into the exported library. I wanted to keep the originals in their current location. As for the previews, I knew I was going to have to rebuild them anyways (see next step).
The only drawback from this method is that Aperture doesn’t recreate the full collection structure when creating sub libraries this way:
I only realized this after importing a few year’s worth of images into Lightroom. This is annoying but not really a big problem, as I can always filter images by date or keyword and recreate most of these sub-collections in the future if I ever need to.
3. Preserving adjustments – kind of
Sadly, I knew that all nondestructive adjustments made to my photos in Aperture would be lost when importing into Lightroom. However, the Lightroom importer will try to import “high quality previews” if they exist. So to at least keep a flattened version of the photos I had edited in Aperture, I made sure to generate a fullsize preview of all edited photos. I started by making a smart library containing only images to which I had applied adjustments:
I then set the previews to full-size by selecting the largest possible size and quality setting in Preferences:
Finally I forced Aperture to recreate a full-size preview for all those images by selecting all images in my smart collection, and selecting Photos->Generate Previews while holding down the Option key (this will force generate them even if they are up to date). This increased the size of my Aperture libraries significantly (by a factor of 3), which was expected.
Edit (January 2019): I came back several months (OK years) later to finish the import process for my older images and my library wasn’t increasing in size during that step, which I found odd. I checked the location of the referenced files and sure enough, Aperture had lost track of the location of the originals. I fixed it by using File->Locate Referenced Files before reprocessing the previews.
I repeated this process for each of the yearly libraries I had created in the previous step.
When I first tried to import my new smaller Aperture libraries into Lightroom, I realized the importer was taking forever trying to validate the last opened Aperture library before letting me select which one I wanted to import. To stop wasting time on this step, I created a new empty Aperture library and set it as my current library before switching to Lightroom to start the import process.
4. Import into Lightroom
Edit (January 2019): It turns out that the Aperture importer plugin has a tendency to hang after importing 1000 images or so in recent versions of Lightroom Classic. The workaround is to install version 6.0 of Lightroom Classic (from 2015, then called Photoshop Lightroom CC) and use that version to run the importer. Older versions can be installed from the Adobe Creative Cloud app by clicking the little arrow next to the Install/Update buttons and choose Manage>Other versions:
I decided to import the photos into a blank Lightroom library, to limit interference with existing workflows. After creating a new library, I then ran the Import from Aperture plugin, selecting each of my smaller Aperture libraries in turn to import from, and setting the target path to be consistent with my new Lightroom workflow.
In the options panel, I made sure to check the first checkbox to import the full-size previews for adjusted images I had generated in the previous step. I also opted to leave the referenced files in their current location on my hard drive.
I then took a deep breath and clicked Import. I found the process to be reasonably quick, about 20-30 minutes for each year of photos. After each import, I checked that the images were all properly referenced, checked a few folders and keywords. At first I was alarmed that I was ending up with many more pictures in Lightroom than in the original Aperture libraries, before remembering that the additional images came from the generated previews.
Aside from the fact that the adjustments I had made in Aperture couldn’t be imported and would have to be replicated one by one if I ever needed to reprocess an image, I found the transition to be reasonably smooth. I only realized later in the process that using smart albums to split my Aperture library into smaller libraries lost some granularity in my collections structure, but that’s a relatively minor annoyance. Some metadata cleanup was necessary, notably I noticed that accented characters in keywords appear to have been improperly converted. I decided to fix those over time as needed.
The other cleanup operation that I had to do was that some images (notably scanned images, and some coming from older cameras and smartphones) were not appearing in the right order. It turns out, Lightroom relies on the “Capture date” field to order images in chronological order and that field wasn’t properly filled for those images. This can be easily fixed by filtering by date unknown and then adjusting the capture time to mirror the file creation date for those images.
In the end, I realized that what took me the most time (apart from convincing myself to move to Lighroom already) was figuring out each step of this workflow. Once I had them in place, it was a matter of replicating a series of operations and letting the importer do most of the work. Since it took me a bit of research to put this workflow together, I figured it might interest someone else, thus I documented it here. Hope it helps!
TL;DR – My Aperture to Lightroom migration workflow
- Be sure to run the importer in Lightroom version 6.0 (2015). The importer doesn’t work well in newer versions. You can install older versions of Adobe apps by clicking the little arrow next to the Install button in the Adobe Creative Cloud app.
- Make sure you have a recent backup of your Aperture library, just in case.
- Check if you have images in the Aperture trash. Either empty it before going further, or if you want to keep your trashed images, apply a tag to them, because Lightroom will import them and mix them up with the others. Note that you can’t add a keyword to images when they’re in the trash, so you might need to put them in a temporary collection, add your tag or mark them as “rejected”. Note that if you use smart albums to split your library, the trashed images will not be imported.
- Split your Aperture library into smaller ones (e.g. by date) by creating smart albums with the proper limiters and then exporting each album as an Aperture library.
- For each of the smaller Aperture libraries:
- Make a smart album with the “Adjustments… are applied” rule
- Go to Preferences, under Previews select Photo Preview: Don’t limit
- Select all images in your adjustment smart album, press the Option key and select Photos->Generate previews (this will force generate them even if they are up to date). This will increase the size of your library significantly. If it doesn’t, something is probably not quite right! Start by checking that Aperture can locate all original files (File->Locate Referenced Files) and fix if necessary before trying again.
- BEFORE starting the import process in Lightroom, make a new empty Aperture library and exit Aperture with this library active. This will reduce the time Lightroom will try to “check” Aperture’s library when launching the importer.
- Import each of your Aperture libraries by using Lightroom’s Import from Aperture plugin, making sure to set the target location for referenced files to be consistent with your current Lightroom workflow. Click the Options panel and select all checkboxes to extract the full size previews you created in step 5 and import them as separate files, and to keep the referenced files in their current location instead of duplicating them.
- Once everything has been imported, cleanup your metadata as required, checking for missing capture dates.
- Another item to file under “obsolete or unfashionable technologies I hold on to”: Is the Relational Database Doomed? Here I am teaching the gospel of SQL when all the cool kids swear by NoSQL.
- Speaking of being a grumpy old guy, why is it that when I look at the amazing images taken by Opportunity’s 12 years on Mars, all I see are track marks and litter on a once pristine planet? This reminds me strongly of Mariner 9 by Kelly Richardson, which I’ve seen displayed at the National Gallery last year.
- My friend Xavier sent me the coolest belated Christmas gift ever, a tablecloth printed with the beautiful 1:50,000 swisstopo map of Western Switzerland. He used a service called Spoonflower to custom-print the fabric. Of course I immediately started thinking of all the cool things I could do, notably a wallpaper with contour lines, e.g. of the Niagara escarpement…
- This is our 4th winter in Toronto and so far it’s been much milder than the previous years. It snowed again last night, but nothing like the snows of yesteryear.
- Keeping on the arctic theme, negatives from Shackleton’s second Antarctica expedition have been found and were finally developed after more than a century. Instagram, 1914 style.
- Is the age of the hydroelectric dam over? Fist this fascinating 99% invisible episode about fish cannons is full of offhand comments that dams are being routinely dismantled in the US and how it’s a good thing for salmon and humans. Then I learn in the New Yorker about crumbling African dams that could soon become “the dam industry’s Chernobyl.” I do hope we have good alternatives for replacing all the lost power generation and off-peak storage. The answer to the latter could lay in part a few kilometers from me in Lake Ontario, where energy storage using underwater compressed air is being tested as I write this.
- On this topic, I keep going back to the superb What can a technologist do about climate science piece published by Bret Victor last November. He lists a few alternative solutions for energy storage, including the slightly crazy idea of using surplus energy to drive a train uphill, then letting it roll back downhill to harvest that energy back.
- Keepalive is a piratebox hidden in a boulder in northern Germany. Lighting a fire on its side with generate enough power to bring the server to life and share PDF survival guides over WiFi. I just hope there is a USB port on the boulder to power up the lost wanderers’ devices (and that they remembered how to start a fire without needing a survival guide)…
- Ever wondered why businesses ask you how likely you’d recommend a product of service to a friend? It informs the Net Promoter Score, a metric that is used to measure customer satisfaction. Publishers are using it too!
- This nice visualisation tool for place names motivated me to start playing with geo data again. The Overpass API allows bulk download of OSM nodes, and Overpass Turbo has a nice interface to try out Overpass queries.
At the OLA Super Conference Hackfest last week, I chose to work on trying to enable TLS/SSL encryption on library websites using Let’s Encrypt. Digital privacy has been a hot topic of discussion in libraries lately, most prominently around the efforts of the Library Freedom Project. Enabling TLS/SSL (HTTPS) encryption on library websites and online catalogue is among the first steps recommended by the LITA Patron Privacy Technologies Interest Group to protect the privacy of patrons.
Unfortunately, TLS/SSL encryption is a fairly complex (and costly) process. While large institutions often manage their own web servers and have complete control over them, many small public libraries resort to web hosting services. Some of these services will arrange for HTTPS certificates at a premium, but the process is often not straightforward. Enter Let’s Encrypt, which opened as a public beta service last December. As a “free, automated, and open certificate authority (CA)”, Let’s Encrypt aims to enable anyone who owns a domain name to receive a trusted certificate at no cost, and more easily than by using commercial certificate authorities.
With the helpful support of Dan Scott, we spent the morning trying to install and run Let’s Encrypt, generate certificates and enable them on our own personal websites and, for one brave librarian, on her live public library page.
The first step was to download and run Let’s Encrypt. We all had Mac laptops and so the following may only apply to Mac OS X. The source code for generating certificates is distributed on GitHub and the following steps were required to make it work, following this tutorial.
- Install git. We all had different setups on our laptops. Using Homebrew to download and install git as well as the other required packages worked well. As for me, I found out that recently upgrading the OS to El Capitan broke my Xcode toolchain, including git. This fix found on Stackexchange worked well to restore the Xcode command-line tools without having to go the trouble of downloading the whole Xcode package:
$ xcode-select --install
- Download (git clone) Let’s Encrypt from GitHub:
$ git clone https://github.com/letsencrypt/letsencrypt $ cd letsencrypt
- Installing any remaining dependencies. It turns out that
letsencrypt-autochecks for missing dependencies every time it is run. So running the following will install all that is needed using Homebrew (as well as install Homebrew itself if not already present).
$ ./letsencrypt-auto --help
Since it checks dependencies at every run and will potentially want to install missing ones in
/usr/bin, Let’s Encrypt will request root access every time it is run. Not only is this a bit unexpected and unsettling, because root access should not be required to generate certificates that will not be applied locally, but it will cause other problems when retrieving the generated certificate, as we will see below.
Once Let’s Encrypt was up and running on our laptops, the next step was to generate the certificates. By default,
letsencrypt-auto assumes it is run on the same server that is hosting the website, but in our case we wanted to set up encryption on hosted domains. For this, we need to tell Let’s Encrypt to only generate the certificates, which we would then upload to our hosting providers. This is done using the
The syntax we used was
./letsencrypt-auto certonly -a manual --rsa-key-size 4096 -d domain.com -d www.domain.com
Note that this will generate two distinct certificates, one for domain.com (without www) and one for www.domain.com. Again, this will ask for root password, as discussed above.
The next step is important to ensure that the user requesting a new certificate for a particular domain has legitimate claims to that domain. To this end, Let’s Encrypt will generate a hash string that needs to be copied to a specific file on the server that hosts the domain in question. The hash, the name of the file and the path are provided:
Make sure your web server displays the following content at http://domain.com/.well-known/acme-challenge/weoEFKS-aasksdSKCKEIFIXCNKSKQwa3d35ds30_sDKIS before continuing: weoEFKS-aasksdSKCKEIFIXCNKSKQwa3d35ds30_sDKIS.Rso39djaklj3sdlkjckxmsne3a If you don't have HTTP server configured, you can run the following command on the target server (as root): mkdir -p /tmp/letsencrypt/public_html/.well-known/acme-challenge cd /tmp/letsencrypt/public_html printf "%s" weoEFKS-aasksdSKCKEIFIXCNKSKQwa3d35ds30_sDKIS.Rso39djaklj3sdlkjckxmsne3a > .well-known/acme-challenge/weoEFKS-aasksdSKCKEIFIXCNKSKQwa3d35ds30_sDKIS
This can be copied over to the web server either using FTP, a web based file manager, or by opening a SSH connection to the server. If the SSH route is chosen, then the commands provided on lines 9-11 above can be used to generate the challenge file. Note that the hash and filename provided above are examples. Use the actual text provided by Let’s Encrypt.
Once the file is live on the website, hit enter to finish the process. Let’s Encrypt will visit the newly created page on our website, check that it’s ours and generate the certificates. If more than one domain were specified (e.g. with and without www), the process is repeated for each domain.
The generated key and files are saved in
/etc/letsencrypt/live/domain.com. However since Let’s Encrypt was run as root (see above), these files belong to the root user and trying to display them returns a Permission denied error. To display them, use
$ sudo ls -l /etc/letsencrypt/live/domain.com total 32 lrwxr-xr-x 1 root wheel 33 Feb 5 22:03 cert.pem -> ../../archive/domain.com/cert1.pem lrwxr-xr-x 1 root wheel 34 Feb 5 22:03 chain.pem -> ../../archive/domain.com/chain1.pem lrwxr-xr-x 1 root wheel 38 Feb 5 22:03 fullchain.pem -> ../../archive/domain.com/fullchain1.pem lrwxr-xr-x 1 root wheel 36 Feb 5 22:03 privkey.pem -> ../../archive/domain.com/privkey1.pem
Note that if you specified more than one domain when running
letsencrypt-auto , LetsEncrypt will generate a single certificate covering all specified domains. It will appear on
/etc/letsencrypt/live under the name of the first domain that you specified.
As can be seen, the files stored in
/etc/letsencrypt/live are actually symlinks to files stored in
sudo every time we want to access those files is bothersome, so we can use
chown to change their owner back to us:
$ sudo chown -R username /etc/letsencrypt/
Update: Enabling TLS/SSL on hosted websites
The next and final step is to copy the content of the generated keys in the SSL setup interface on our web host. Unfortunately, it turned out that neither of us had this functionality turned on by our service providers and we ended up having to write them to request enabling SSL.
I have asked my provider to enable SSL on my domain, and after a week or so they wrote back saying that they had not only done the necessary configuration changes to allow me to upload my own certificates, but they had implemented a cpanel extension allowing their customers to generate their own certificates without going through the hassle described above! This is excellent news, and I shall soon try it out for my other domains, but for now I wanted to try loading the certificates I had generated. Here’s how I did it (my hosting admin interface uses cpanel 54.0.15).
The previous version of this post linked to this post (in German, which provides a screenshot of a different configuration interface.
Under Security -> SSL/TLS, I chose the “Install and Manage SSL for your site (HTTPS)” option, which installs certificate and key in one single step. I then selected my main domain name in the drop-down menu, and copied the contents of
cert.pem in the field labeled CRT and
privkey.pem to the KEY field. I left the CABUNDLE field blank and hit “Install Certificate”.
On Mac OS X, piping anything to
pbcopy will place it in the Clipboard, ready to be pasted anywhere. So this is how I copied the contents of the certificate file before pasting it on the cpanel form:
$ cat cert1.pem | pbcopy
And that was it! I got a nice confirmation message that the certificate was installed, detailing all domain names it was covering. It also helpfully listed other domain names I have pointing to my site but not covered by the certificate, warning me that using them will cause browsers to raise a security issue.
I’m using WordPress to manage this blog, and it needed to be reconfigured so as to serve all inserted images as HTTPS, and thus avoiding mixed content issues:
Once this was done, my browser started rewarding me with a nice padlock icon on all my pages, confirming that I had successfully enabled HTTPS on my domain! I also ran it through an SSL checker for good measure.
The availability of Let’s Encrypt as a free and open alternative to commercial certificate authorities is an important step towards a more secure Internet. However, the current beta version of Let’s Encrypt still requires some familiarity with command-line interfaces, web development tools and an understanding of how TLS/SSL works. Better documentation and a more user-friendly interface will certainly go a long way in making the process easier. The necessity to run the client as root is another barrier that will hopefully be lifted as the software evolves. Finally, even though generating certificates is now freely accessible, setting them up on hosted websites still require the service providers to activate this option.
There is an extension for cpanel that claims to allow end-users to easily setup Let’s Encrypt certificates on their website. Maybe as demand grows, hosting providers will begin enabling this extension for their customers and HTTPS will then truly become an easy option for everyone to use. Since my hosting provider recently enabled this option, I plan to try it out soon and will report back if I do.
Thanks a lot to Dan and the OLA Super Conference Hackathon organisers and facilitators, as well as the other attendees with whom I worked on this project! I certainly learned a lot.
- Several takes on the #DeleteAcademiaEdu thing. First off, let’s not forget that academia.edu is a (for profit) social network, not a repository. In the academic rat’s race, every mean to get one’s work out there is justified, so deleting profiles might be a luxury only established folks can afford. Also, not all researchers have access to an institutional repository. Personally, I kept my (few) publications on my former institution’s IR and only linked them from my academia.edu profile. This is the approach I recommend when asked.
- Thanks to this post on the Duke Library blog also advocating a balanced approach to the academia.edu debate, I learned the existence of an open-source alternative network called VIVO, which has been implemented in many schools and allows for integration with IRs.
- The Last Days of Target (Canadian Business). What brought down the giant in its ill-fated Canada venture? Messy data.
- Alors oui, un nouveau clavier, voilà ce qu’il faut pour sauver l’orthographe française. Je me dépêche d’en parler avant qu’il soit nécessaire d’obtenir l’autorisation des ayants-droit pour insérer un lien hypertexte. Slow clap.