Learning How to Make a News App in Two Days
- By Al Shaw
- Ben Chartoff, Harlo Holmes, Brian Jacobs, Aurelia Moser, Gabriela Rodriguez, Al Shaw, Mike Tigas, Marcos Vanetta
- OpenNews, ProPublica
As part of the orientation week for the 2014 class of Knight-Mozilla OpenNews Fellows, fellow nerd-cuber Mike Tigas and I led a hackathon at Mozilla’s headquarters in San Francisco…
Project To Scrape, Perchance to Tweet
At the Chicago Tribune, we had a simple goal: to automatically tweet contributions to Illinois politicians of $1,000 or more, which campaigns are required to report within five business days. To see, in something approximating real time, which campaigns are bringing in the big bucks and who those big-buck-bearers are. The Illinois State Board of Elections (ISBE) has helpfully published exactly this data for years online, in a format that appears to have changed very little since at least the mid-2000s. There’s no API for this data, but the stability of the format is encouraging. A scraper is hardly an ideal tool for anything intended to last for a while and produce public-facing data, but if we can count on the format of the page not to change much over at least the next several months, it’s probably worth it.
Project Introducing Treasury.IO
- By Michael Keller, Cezary Podkul
- Brian Abelson, Jake Bialer, Burton DeWilde, Michael Keller, Thomas Levine, Ashley Williams
The U.S. Treasury’s Daily Treasury Statement lists actual cash spending down to the million on everything the government spent money on each day, as well as how it funded the spending. But, the Treasury only releases these files in PDF or fixed-width text files like this one, making any analysis very difficult.
To liberate the data and make it easy to analyze federal money flows across time, we created Treasury.IO. The system we built downloads and parses the fixed-width files into a standard schema, creating a SQLite database that can be directly queried via a URL endpoint.
Learning And Remember, this Is for Posterity
Jacob Harris on the hows and whys of designing interactives to survive the future.
Learning Building News Apps on a Shoestring
Alan Palazzolo on how the MinnPost team rocks it without a big budget
Learning Well Hello, Census
Joe Germuska on the iterative, human-centered process that’s made the new Census Reporter project especially awesome.
Project Let research apps, MVC JavaScript, and APIs work for you
- By Agustin Armendariz, Michael Corey, Aaron Williams
- Agustin Armendariz, Michael Corey, Aaron Williams
- The Center for Investigative Reporting
The Center for Investigative Reporting continues their work visualizing Department of Veterans Affairs’ data. Here, they discuss their development process.
Learning Using Big Data to Ask Big Questions
Chase Davis lays down some data science upon us to change how you think about the questions you’re asking of your data
Roundtable US Shutdown Scuttles Data
- By Dan Sinker
- Waldo Jaquith, Chris Keller, Mike Migurski, Ryan Pitts, Matt Stiles, Ben Welsh
- Census Reporter, Code for America, KPCC, NPR, The Los Angeles Times
As the government shutdown grinds into its third day, many news developers, civic data hackers, and open gov activists are starting to feel the hurt due to the suspension of most government data feeds, APIs, and websites. How they’re adapting and collaborating to fill the gaps of the shutdown.
Learning The Times Regrets the Programmer Error
Jake Harris opens a serious barrel of monkeys about when and how to issue corrections for data journalism.
Learning Kill All Your Darlings
Matt Waite on what to do when things don’t work out like you planned.
Project All About Reporter
The Wall Street Journal’s Jeremy Singer-Vine recently released Reporter, an open source tool that makes it easy to hide and reveal the code behind common forms of data visualization presented on the web. We spoke with him about the tool’s makeup, design goals, and future development plan.
Learning People Power Prevails!
John Keefe on tracking the cicada pestilence with open source sensor journalism and crowdsourced data collection
Learning Telling Your Boss “No”
Matt Waite says just because you can make it doesn’t mean you should
Learning Sane Data Updates Are Harder than You Think: Part 3
Third in a three-part series by Adrian Holovaty about hairy data-parsing problems from a journalist’s perspective
Learning They Are Tweet Zombies!! They Are Followers!!
Jake Harris on how dead accounts and spambots can mess with your Twitter data mojo
Learning Know Your Stats
Read Dave Stanton‘s essential primer on basic statistical principles and you won’t get caught with your data pants down.
Learning Drawing Conclusions from Data
In this OpenNews Learning special, Jonathan Stray presents an equation-free statistics talk on data and the shape of randomness.
Learning The Perils of Polling Twitter
Jake Harris on just a few of the myriad reasons why using tweets as data is less than ideal.
