2014-02-21

Co.Labs

An Open Source Library For Adding Sound Bytes To Print Journalism

A journalism student in Chicago has built a free tool for enriching text with sound—one of a growing number of new tools trying to improve the reader experience without adding tons of code.



When Texas Senator Wendy Davis gave her 13-hour filibuster against an anti-abortion law last summer, journalists filled the web with a heap of multimedia: videos, tweets, GIFs, and Internet memes that often outshined their written articles. But the Washington Post blog "The Fix" didn’t make its readers choose between listening and reading its filibuster coverage. By using SoundCite, a web tool that streams an audio clip behind the text, the Post’s account of Davis’s speech uniquely captured the event’s emotions in digestible snippets without awkwardly breaking up its written text.

"When the Wendy Davis filibuster went down—which was inherently such an audio-rich event—we realized we could do something compelling with it," Caitlin Dewey, the blog post’s author, wrote in an email.

The tool’s original developer is Medill School of Journalism senior Tyler Fisher, who originally conceived SoundCite to improve the way online music articles weaved sound clips and their text together. An arrow appears next to the words that are animated by the audio, and a moving progress bar slides across the text, so you stay focused on reading. At just over a year old, SoundCite has not only made its appearance in the music-sphere, like at WBEZ Chicago Public Radio, but has also entered other types of newsrooms, like the Washington Post, JSOnline, and Education Week.

Startups aimed at content publishers are growing in numbers, with tools like Gui.de and SpokenLayer and Readrboard offering to give publishers easy access to the kind of functionality that users have come to expect from other parts of the social web.

Unlike these companies, however, Fisher doesn’t aspire to make money off of SoundCite. In fact, he sees the project as core to a new type of storytelling. "A lot of people don’t think about the usability of [multimedia], and I don’t think they think about how the multimedia is contributing to the story," he says.

An App For Music Critics

SoundCite got its start when Fisher took an advanced design class at Medill in the spring of 2012. The class required that he develop a web tool to solve a journalistic problem.

"In a past life, Tyler aspired to be a rock critic," says Jeremy Gilbert, Tyler’s professor for the class. "He spent equal time writing what the music sounded like and writing what the music meant. With the clips, he wanted to concentrate on what the music meant and let the music play for you," Gilbert says.

"A lot of sites have good players, but you usually have to play the whole song. That break in the narrative isn’t good for the experience," Gilbert adds.

Fisher’s earliest version of SoundCite consisted of JavaScript, HTML, and CSS that relied on the user to create MP3s loaded directly in the HTML page. "It wasn’t powered by anything," Gilbert says. "Users couldn’t easily pick out the start and end points they wanted."

"I was convinced that it could be used for much more than just music," Gilbert says. Gilbert decided to work independently with Fisher after the class ended. Eventually, they were able to connect with SoundCloud’s API, making it easier for non-coders to create and choose time points within the sound file. What’s more is users no longer had to wrangle MP3s onto their site; they could use SoundCloud’s uploading feature and let them deal with hosting the clip.

Collaborating With Expert Coders

At that point, SoundCite still had JavaScript code that could interfere with other pages on a given website. Depending on where the tool was used, SoundCite’s version of JQuery was likely to interfere with pages that didn’t use that exact JQuery version.

Knowing that they needed help from the experts, Fisher and Gilbert applied to have Northwestern's Knight Lab host and support continuing development of SoundCite. "We needed to harden the code," says Gilbert. With no break in momentum, they transferred SoundCite from Gilbert’s personal server to the Knight Lab that summer.

The Knight Lab, which is jointly funded by the Knight and Robert R. McCormick foundations, selectively accepts journalism-driven tech projects, whether they are from a Medill class or elsewhere in the professional world, and helps push them to usability. It employs two full-time software developers.

At the same time the Knight developers got up to speed on SoundCite, Fisher was accepted as a Knight Fellow for the 2012-2013 academic year. He essentially got paid to keep improving SoundCite while working in the lab.

Solving the JQuery issue meant isolating the version of JQuery that SoundCite used. "We had to make sure it kept itself tidy and didn’t step on other things," says Joe Germuska, the Knight Lab’s technology director, stressing that SoundCite’s loose JavaScript could have interfered with news sites’ advertising elements.

Germuska looked to the Internet for help and found a solution on Stack Overflow. "As is often the case, someone has already solved the problem. You just have to find it," says Germuska.

Another issue was rewriting the code so that non-coders wouldn’t be scared off by it when it came time to paste it into their pages. "We really had to work on usability and create clean code," says Gilbert. "The Knight Lab really helped us with that."

Working On Mobile

"The main problem with SoundCite [now] is that it doesn’t work with mobile," Fisher says. Mobile browsers don’t support Flash, and the SoundCloud API uses Flash to pass its clips to third-party applications. The solution would be to create in-house code so that SoundCite wouldn’t have to rely on Flash. This would mean hosting users’ sound clips on internal servers, and this isn’t an attractive option to Fisher.

"We definitely don’t want to get in the business of hosting audio because of copyrights and all those legality issues," says Fisher. "The problem with audio codecs is that the MP3 is owned by someone," he says, "and some web browsers like Mozilla Firefox don’t want to pay for that, so they don’t support the MP3. Some browsers do play them. Some web browsers don’t support the WAV file, so nobody agrees on what types of audio files should be supported." So it’s good that SoundCloud uses Flash because Flash is universal, he says.

Besides, Fisher says optimizing SoundCite for mobile is outside of his scope. He is willing to give users documentation on how to host audio on their sites and has placed SoundCite’s code on GitHub. "Everything the Knight Lab does is open source. So anyone can modify the code if they want to," he says. And someone has. Today, there is one WordPress plug-in for SoundCite on GitHub.

"The way people look at it is that the audio isn’t integral to the piece, so it doesn’t ruin the piece for mobile [when the clip isn’t there]. But it does add something," Fisher says, bringing up the Post article.

With the code available to everyone, users can tweak SoundCite to work for whatever needs they have, whether it’s mobile or fitting in with their web page’s scheme. "Education Week was able to customize the look of SoundCite because its HTML was publicly available," says Fisher.

Keeping It Open

The Knight Lab plans to continue encouraging journalists to use SoundCite and promote collaboration, as they have done in the past. "We approached [WBEZ] directly and guided them through an early implementation," says Ryan Graff, communications director at the Knight Lab. "After that, the other projects were word of mouth." In the case of the Post article, a Post intern knew Fisher and mentioned SoundCite to her colleagues.

At the moment, Fisher is spending a chunk of the school year interning at National Public Radio and will return to Northwestern this spring. After graduating this June, he would ideally "be working with technology and news, somewhere in that intersection." And Gilbert has since moved on from Northwestern and is now deputy director of digital for National Geographic.

"I would hope that anyone who uses SoundCite would think first about what that is adding to their story and if that is necessary," says Fisher. For web journalists who want their well-picked words read, adding multimedia might keep readers on their page for a few seconds, but it does not guarantee that their prose will command their attention. But SoundCite does writers justice by illuminating the text, keeping the reader’s focus on the writing. And that’s what matters.

[Image: Flickr user Paul Lowry]






Add New Comment

3 Comments

  • This is a nice story, except that the headline is completely incorrect. No part of SoundCite creates a podcast as the headline would have you believe.

  • Thanks -- We've corrected the headline.

    Indeed, SoundCite lets a writer stream any audio clip, whether it's music, birds chirping, a speech, or even her own voice, in line with the text. It's a neat way to meaningfully add media to an online article.

    Case-in-point: In her WaPo post, Dewey wrote about the "ensuing celebrations," while you could literally hear the room cheering without taking your eyes off of the text to search for the clip.