Crowdsourcing – Metadata Games & The Smithsonian Transcription Center

“Crowdsourcing” is a bit of a buzzword, and projects that incorporate it often fail in the same ways that many projects with buzz fail. Institutions have a tendency to undertake flashy new initiatives that betray a lack of nuanced understanding of what makes crowdsourcing work, and once the excitement wears off they fail to maintain them in the long term, or they quietly tuck them away. Today I am going to look at one project that suffers from some of these symptoms, and one that rises above them.

Metadata Games

Crowdsourcing pairs well with gamification. If your crowdsourcing platform is intrinsically motivating, then you’re more likely to attract contributors, and those contributors will likely put in better work. Gamification, if done well, can accomplish this, drawing in contributors with an enjoyable experience. The Metadata Games project takes advantage of this synergy, soliciting metadata tagging through web browser based flash games.

Metadata Games was created by Dartmouth University’s games lab, Tiltfactor. It’s an open source platform that currently interfaces with 45 collections at 11 institutions, including The Boston Public Library and the Digital Public Library of America. The project comprises seven browser games, including three built specifically for the British Library, and two mobile apps developed for both Android and iOS.

I looked specifically at their collaboration with the British Library. The three games, “Ships Tag,” “Book Tag,” and “Portrait Tag,” seem to each focus on a different collection at the British Library, but the Metadata Games website is rather vague about this. Each of the three games is identical, save for the logo, color scheme, and the type of item the player works with.

Pictured above is a screenshot of the game Book Tag. The player is presented with a randomly selected image from the British Library’s collection. The player must simply type as many tags related to the image as they can think of in the box below, each separated by a comma. Once the player is satisfied, they can submit the tags or skip to another image. Clicking the image brings up a more detailed view of it, but it is a simple static image. The player cannot zoom in on it or pan around it. There is a Wikipedia tab on the right that opens up a search box, allowing the player to make a search on Wikipedia that opens in a separate tab. Once the player has tagged four different images, the player is shown a grid containing thumbnails of all four images they’d tagged, and told their score. The score is extremely simple; each tag the player submitted is worth one point. That’s it. The player is told which of their tags “matched tags,” but it’s not clear what that means. It might mean that those tags matched other players’ tags, or matched librarians’ tags. Further investigation on the Metadata Games website does not turn up this information.

Each game includes a link to the British Library’s home page, but there is no way to reach the specific item you’re looking at, or the collection it’s from. I wanted to learn more about some of the images I saw, or see whether the tags I’d submitted were active in the library’s catalog. Unfortunately, the game provided absolutely no means of reaching an item’s catalog entry, or even any information whatsoever about the item beyond the simple image they’d shown me.

In game design terms, these things are pretty dismal. There is absolutely the bare minimum of gamification required to even call them games. The only thing differentiating them from a simple tagging application is the score, which is no different from a simple tally of the number of tags you’d entered. These could hardly even be considered games. They’re not particularly fun to play with, and I can’t imagine them drawing in many players. While they do fit the British Library’s very succinct mission of “Advancing the world’s knowledge,” they don’t do a great job of it.

I would love to know more about the ways that the British Library incorporates the user-created metadata from these games, but there is no way of finding out. The games are hosted on Metadata Games’ website, and I couldn’t find even a single mention of them on the British Library’s website. There is no way to access the games from the British Library’s website, and no indication that the metadata from the games is actually being used. I wish there were more transparency and documentation here. It doesn’t inspire a lot of confidence in potential players that their contributions are doing anything.

The Smithsonian Transcription Center

Luckily, there are great examples of crowdsourcing done right that we can point to. The Smithsonian Institution runs a website called TheSmithsonian Transcription Center where they run various crowdsourcing projects.

The Smithsonian Institute has vast numbers of handwritten and typed documents that have never been transcribed into a machine-readable form. The Transcription Center is crowdsourcing the transcription of these documents. As their “About” page states:

The Smithsonian Transcription Center seeks to engage the public in making our collections more accessible. We're working hand-in-hand with digital volunteers to transcribe historic documents and collection records to facilitate research and excite the learning in everyone.

Since its inception in 2013, the Transcription Center has hosted over 1,000 transcription projects, with contributions from over 6,392 volunteers. They maintain a robust web interface for undertaking transcriptions, with each project’s interface specifically tailored to its purpose. They have a thorough social media presence that actively supports volunteers in their work. Their website is organized in a very approachable way, and is designed to help you quickly jump into a project that you can be useful to. Each project has a personalized tutorial to help new users learn how they can best contribute, and contact information for a Smithsonian employee assigned to the project.

I decided to help out by transcribing a page from the Museum of Natural History’s Department of Botany’s project “OTHER FASCINATING EUPHORB GENERA SET 6”. The project consisted of 500 photographs of plant life from the genera Chamaesyce. Each image included not just the plant specimen, but also handwritten and typed notes about it. My job was to transcribe those notes. The interface guided me through the process, with very specific, labeled fields for me to input data into, arranged in an intuitive order. Hovering my mouse over each field provided further explanations and clarifications to help me make sure I was entering the information in correctly. I was able to submit my transcription without needing to create an account or log in. At the bottom of the page there was a Disqus comment/share widget, so I could easily share an interesting entry I’d worked on through social media.

Once I finished the transcription, I submitted it. This moved that page’s status from “Needs Transcription” to “Needs Review”. At this point, any user who’d created a free account could review my transcription for accuracy. Once approved, my transcription would move on to a Smithsonian employee, who would give it a final review before either approving it, or sending it back for a new transcription. I was glad to see that the Smithsonian had a solid and transparent plan for ensuring that their transcriptions were accurate.

Overall, the transcription process was very satisfying and motivating. Though they did not try to gamify it, I still had fun with it. I could see myself going back to it just for personal enjoyment.

NOTE: All sources have been hyperlinked within the text.