Tyler Dukes on combining the power of data-sorting tools with old-fashioned digging.
As part of the orientation week for the 2014 class of Knight-Mozilla OpenNews Fellows, fellow nerd-cuber Mike Tigas and I led a hackathon at Mozilla’s headquarters in San Francisco…
At the Chicago Tribune, we had a simple goal: to automatically tweet contributions to Illinois politicians of $1,000 or more, which campaigns are required to report within five business days. To see, in something approximating real time, which campaigns are bringing in the big bucks and who those big-buck-bearers are. The Illinois State Board of Elections (ISBE) has helpfully published exactly this data for years online, in a format that appears to have changed very little since at least the mid-2000s. There’s no API for this data, but the stability of the format is encouraging. A scraper is hardly an ideal tool for anything intended to last for a while and produce public-facing data, but if we can count on the format of the page not to change much over at least the next several months, it’s probably worth it.
The U.S. Treasury’s Daily Treasury Statement lists actual cash spending down to the million on everything the government spent money on each day, as well as how it funded the spending. But, the Treasury only releases these files in PDF or fixed-width text files like this one, making any analysis very difficult.
To liberate the data and make it easy to analyze federal money flows across time, we created Treasury.IO. The system we built downloads and parses the fixed-width files into a standard schema, creating a SQLite database that can be directly queried via a URL endpoint.
Jacob Harris on the hows and whys of designing interactives to survive the future.
Alan Palazzolo on how the MinnPost team rocks it without a big budget
Joe Germuska on the iterative, human-centered process that’s made the new Census Reporter project especially awesome.
The Center for Investigative Reporting continues their work visualizing Department of Veterans Affairs’ data. Here, they discuss their development process.
Chase Davis lays down some data science upon us to change how you think about the questions you’re asking of your data
As the government shutdown grinds into its third day, many news developers, civic data hackers, and open gov activists are starting to feel the hurt due to the suspension of most government data feeds, APIs, and websites. How they’re adapting and collaborating to fill the gaps of the shutdown.
Jake Harris opens a serious barrel of monkeys about when and how to issue corrections for data journalism.
Matt Waite on what to do when things don’t work out like you planned.
The Wall Street Journal’s Jeremy Singer-Vine recently released Reporter, an open source tool that makes it easy to hide and reveal the code behind common forms of data visualization presented on the web. We spoke with him about the tool’s makeup, design goals, and future development plan.
John Keefe on tracking the cicada pestilence with open source sensor journalism and crowdsourced data collection
Matt Waite says just because you can make it doesn’t mean you should
Third in a three-part series by Adrian Holovaty about hairy data-parsing problems from a journalist’s perspective
Jake Harris on how dead accounts and spambots can mess with your Twitter data mojo
Read Dave Stanton‘s essential primer on basic statistical principles and you won’t get caught with your data pants down.
In this OpenNews Learning special, Jonathan Stray presents an equation-free statistics talk on data and the shape of randomness.