Skip to content

timtom.ch

Thomas' home on the web

Looking for pictures?

Check out my portfolio

RSS My photography blog

  • Universitätsbibliothek Basel
    Completed in 1968, the Basel University Library was a controversial replacement of an earlier neo-Baroque building that has now become an architectural icon.
  • Bibliothèque Publique et Universitaire de Neuchâtel
    The town of Neuchâtel in western Switzerland owes much to its most generous denizen, David de Pury, who might have gained his fortune through the exploitation of slave labour. The ambitious Neuchâtel native left his landlocked home on foot in […]
  • Berkeley Law Library
    From wine cellar to midcentury modern glory, bespoke furniture and a problematic name: the UC Berkeley Law library's legacy.
  • Uni-/PH-Gebäude Luzern
    Inside the University of Luzern's main building and library in a converted mail sorting facility.
  • C.V. Starr East Asian Library, UC Berkeley
    A tour through the C.V. Starr East Asian Library on the University of California Berkeley campus.
  • Fachbibliothek Germanistik, Basel
    A visit to the University of Basel's German language department library inside the "Engelhof", a 14th century house in the old town.
  • Vallila Library, Helsinki
    Touring the Vallila branch of the Helsinki Public Library network by Finnish architect Juha Leiviskä.

Non-photo posts

  • California Suitcase
  • Impressions from the Artist Project 2017
  • Automating and sending speedtest.net data to web services
  • Link dump 2016/8: Open science, books on a ship, waves in the Alps, maker projects
  • Link dump 2016/7: Maps

Elsewhere

  • LinkedIn
  • Twitter
  • Instagram
  • Flickr
  • GitHub
  • Facebook

Support

Archives

  • November 2017
  • February 2017
  • March 2016
  • February 2016
  • January 2016
  • November 2015

Categories

  • Code
  • DIY
  • Life
  • Link dumps
  • Photography
  • Uncategorized
  • Work

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

Tag: linguistics

Guessing the language of a book based on its title

In the midst of endless report-writing, I was faced with an interesting challenge at work this week. We are trying to aggregate e-book usage data for the members of our consortium, and we were interested in figuring out how well the French language content is faring compared to the English titles that make the bulk of the collection.

Unfortunately, one of our vendors do not include language data in either their title lists or the usage reports. Before trying to recoup the usage reports with the full e-book metadata I could get from the MARC records, I tried to run the title list through the guess_language library by way of a simple Python script:

from guess_language import guess_language
import csv
with open('2015-01_ProQuest_titles.csv', 'rb') as csvfile:
    PQreader = csv.DictReader(csvfile)
    for row in PQreader:
        title = row['Title']
        language = guess_language(title.decode('utf-8'))
        print language, title

The results were a disaster:

pt How to Dotcom : A Step by Step Guide to E-Commerce
en My Numbers, My Friends : Popular Lectures on Number Theory
en Foundations of Differential Calculus
en Language and the Internet
en Hollywood & Anti-Semitism : A Cultural History, 1880-1941
de Agape, Eros, Gender : Towards a Pauline Sexual Ethic
la International Law in Antiquity
fr Delinquent-Prone Communities
en Modernist Writing & Reactionary Politics

guess_language works by identifying trigrams, combinations of three characters that are more prevalent in one language than another. While it works reasonably well on whole sentences and short text snippets, the particular construction of a book title seems to throw the method entirely off-kilter.

As I was pondering the next steps, I came to realize that I could also filter titles based on language directly in the vendor database and then export to a CSV file… which solved my issue in seconds but wasn’t half as fun as playing around with computational linguistics. Back to writing reports, I guess.

Share this:

  • Twitter
  • Facebook
  • Email
Posted on January 20, 2016January 20, 2016Categories Code, WorkTags ebooks, language, linguistics, metadata, pythonLeave a comment on Guessing the language of a book based on its title
Proudly powered by WordPress
loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.