Features / Tool
Christopher Groskopf, master of CSVs, breaks down the magical powers of csvkit.
CSV Fingerprints creator Victor Powell talks about the tool’s inception, inner workings, and potential to help data-slingers in newsrooms finally ditch Excel.
Last week, NPR’s Visuals team released their dailygraphics rig, which offers workflow for small-scale visualizations, interactives, and graphics, along with “automated machinery for creating, deploying and embedding these mini-projects.” Their introductory blog post breaks down how to set up and use the rig, and the code is open source and ready to use. Alyson Hurt joined last week’s OpenNews community call to talk a little about the project, and we chatted with her and Christopher Groskopf afterward about how the rig came to be, what kind of skills are required to use it, and their aim to improve code quality and culture through process-improving tools.
If your websites have SSL enabled (when users log in, for example), or if you use VPN software to secure your network, or if you run your own mail servers, your newsroom might be affected by Heartbleed. Here’s what to do next.
I was asked to join BBC News Labs a couple a weeks ago to work on a project that, when it was first briefly explained to me by email, left me clueless about what it was about. (Imagine the discomfort before my job interview with Matt Shearer, Innovation Manager at the Lab.)
The project is called #newsVane—and yes, we refer to it with the hash sign every time, don’t ask me why.
We see a moment coming when the collection of endless streams of data is commonplace. As this transition accelerates it is becoming increasingly apparent that our existing toolset for dealing with streams of data is lacking. Over the last 20 years we have invested heavily in tools that deal with tabulated data, from Excel, MySQL, and MATLAB to Hadoop, R, and Python+Numpy. These tools, when faced with a stream of never-ending data, fall short and diminish our creative potential.
In response to this shortfall we have created streamtools—a new, open source project by the New York Times R&D Lab which provides a general purpose, graphical tool for dealing with streams of data. It offers a vocabulary of operations that can be connected together to create live data processing systems without the need for programming or complicated infrastructure. These systems are assembled using a visual interface that affords both immediate understanding and live manipulation of the system.
Today, we’re launching Source Jobs, a new place to list jobs for the newsroom designers and developers already populating our Community section—and for the curious developers and designers who don’t yet realize that their future lies in journalism. As the global journalism-code community continues to grow, our goal is to offer a simple, scalable listings service that newsrooms can edit on their own.
The code and thinking behind NPR’s implementation of the JPEG “filmstrip” technique in “Planet Money Makes A T-Shirt.”
Freedom of the Press Foundation executive director Trevor Timm discusses SecureDrop’s evolution and future prospects.
2013 OpenNews fellow Brian Abelson has been conducting research on pageviews as a metric, and on the relationship between pageviews and promotion at the New York Times during his fellowship there. This article is cross-posted from his blog.
Al Jazeera America’s Michael Keller introduces the three new open source libraries behind AJA’s displaced Syrians interactive app.
Introducing the double-whammy of Simple Map D3 and Tulip, a new mapping app from MinnPost.
Introducing csvdedupe, an open source command line tool for de-duplication and entity resolution.
Last November, the BBC News team created a front-end regression tool that collects and diffs screenshots to automatically highlight discrepancies produced (intentionally or otherwise) by CSS changes. Last week, the team open-sourced Wraith. We spoke with David Blooman, who developed the tool last fall and worked with Simon Thulbourn to prepare it for public release.
Ultralight CMSes are, in many ways, the product of hacking or infecting the CMS. Here’s a breakdown of a few popular ones, complete with setup instructions, pro/cons, and newsroom case studies.
Yesterday, ProPublica released Transcribable, a new open source tool that makes orderly crowdsourced transcription available to any organization that uses Ruby on Rails. ProPublica’s Al Shaw introduced the project to the public in a post on ProPublica’s Nerd Blog yesterday and here answers all our questions about the project.
We asked for your thoughts on Strongbox, the New Yorker’s new implementation of DeadDrop. Our first wave of responses includes the New York Times’ Jacob Harris, the Overview Project’s Jonathan Stray, and Mike Tigas, OpenNews Fellow at ProPublica.
Yesterday morning, the ProPublica apps team released a series of documents outlining their coding philosophy, app design and development practices, data validation techniques, and more. We spoke with Scott Klein about how his team’s processes evolved and how they made the time to document it all.