Skip to content

timtom.ch

Thomas' home on the web

Looking to hire me?

Check out my library technology and open data consulting practice.

RSS My photography blog

  • Bibliothèque de Drummondville
    This award-winning library combines contemporary design with a colourful reference to local industrial heritage.
  • Bibliothèque du Centre Canadien d’Architecture, Montréal
    Combining modern elements with a respect for classical proportions, the design of the Canadian Centre for Architecture reflects its mission as an universal archive of the art.
  • Finsbury Library, London
    The reorganization of the London boroughs robbed this library of its ambition as a cultural centre but gifted inhabitants of the nearby housing estates with a spacious library.
  • London Metropolitan Archives
    A former print shop in Clerkenwell is now used as the archives of the Corporation of London.
  • Bibliothèque Luc-Lacourcière, Beaumont
    The 18th century presbytery of Beaumont was once a grander building serving as a schoolhouse before being reconstructed as it first stood.
  • Bibliothèque historique du Séminaire de Québec
    Spanning four centuries of learning inside a former 19th century chapel, the library of the Québec Seminary is the keeper of a rich encyclopedic collection as old as the city itself.
  • Baubibliothek ETH Zürich
    The ETH Zürich's architecture library on the Hönggerberg is a mecca for architecture lovers in Switzerland and abroad.
  • Pestalozzi-Bibliothek Oerlikon
    At the heart of a bustling transit hub, the early modern Oerlikon post office now serves as a location for the Zürich public library.
  • Dawson College Library, Montréal
    Among the first examples of Beaux-Arts architecture in Canada, the former Notre-Dame Congregation chapel in Montréal is now the Dawson college library.
  • Nova Scotia Legislative Library, Halifax
    Inside the oldest seat of government in Canada, the Nova Scotia Legislative Library is a gem of Victorian craftsmanship.

Non-photo posts

  • California Suitcase
  • Impressions from the Artist Project 2017
  • Automating and sending speedtest.net data to web services
  • Link dump 2016/8: Open science, books on a ship, waves in the Alps, maker projects
  • Link dump 2016/7: Maps

Elsewhere

  • LinkedIn
  • Twitter
  • Instagram
  • Flickr
  • GitHub
  • Facebook

Support

Archives

  • November 2017
  • February 2017
  • March 2016
  • February 2016
  • January 2016
  • November 2015

Categories

  • Code
  • DIY
  • Life
  • Link dumps
  • Photography
  • Uncategorized
  • Work

Category: Work

Guessing the language of a book based on its title

In the midst of endless report-writing, I was faced with an interesting challenge at work this week. We are trying to aggregate e-book usage data for the members of our consortium, and we were interested in figuring out how well the French language content is faring compared to the English titles that make the bulk of the collection.

Unfortunately, one of our vendors do not include language data in either their title lists or the usage reports. Before trying to recoup the usage reports with the full e-book metadata I could get from the MARC records, I tried to run the title list through the guess_language library by way of a simple Python script:

from guess_language import guess_language
import csv
with open('2015-01_ProQuest_titles.csv', 'rb') as csvfile:
    PQreader = csv.DictReader(csvfile)
    for row in PQreader:
        title = row['Title']
        language = guess_language(title.decode('utf-8'))
        print language, title

The results were a disaster:

pt How to Dotcom : A Step by Step Guide to E-Commerce
en My Numbers, My Friends : Popular Lectures on Number Theory
en Foundations of Differential Calculus
en Language and the Internet
en Hollywood & Anti-Semitism : A Cultural History, 1880-1941
de Agape, Eros, Gender : Towards a Pauline Sexual Ethic
la International Law in Antiquity
fr Delinquent-Prone Communities
en Modernist Writing & Reactionary Politics

guess_language works by identifying trigrams, combinations of three characters that are more prevalent in one language than another. While it works reasonably well on whole sentences and short text snippets, the particular construction of a book title seems to throw the method entirely off-kilter.

As I was pondering the next steps, I came to realize that I could also filter titles based on language directly in the vendor database and then export to a CSV file… which solved my issue in seconds but wasn’t half as fun as playing around with computational linguistics. Back to writing reports, I guess.

Share this:

  • Twitter
  • Facebook
  • Email
Posted on January 20, 2016January 20, 2016Categories Code, WorkTags ebooks, language, linguistics, metadata, pythonLeave a comment on Guessing the language of a book based on its title
Proudly powered by WordPress