Learning data Consider the Boolean

Consider the Boolean

The challenge of using binary data structures in a complicated world.

Project data scraping elections Scraping Nevada

Scraping Nevada

Derek Willis breaks down the three stages of scraping (denial, annoyance, and acceptance) while confronting the election-results form from hell.

Learning data Marriage Data: It’s Complicated

Marriage Data: It's Complicated

D’Vera Cohn on Everything You Ever Wanted to Know About Marriage Data But Were Afraid to Ask

Learning data Everything You Ever Wanted to Know About Elections Scraping

Everything You Ever Wanted to Know About Elections Scraping

Jeremy Merrill and Ken Schwencke explore the fine art of anticipating and catching errors while wrangling the eccentricities of US elections data.

Learning data The Census of Governments Has Your Number

The Census of Governments Has Your Number

Michael Maciag‘s walk-through of this under-utilized goldmine.

Learning data Finding Stories in Census Data

Finding Stories in Census Data

Emily Alpert Reyes on how to find promising needles in Census haystacks.

Learning data Gender, Twitter, and the Value of Taking Things Apart

Gender, Twitter, and the Value of Taking Things Apart

Jake Harris reverse-engineers Twee-Q to evaluate its use of data (and see if his ratio is as disappointing as Twee-Q says it is)

Project JavaScript data From the BBC News Labs: Datastringer

From the BBC News Labs: Datastringer

Basile Simon walks through the process of building a new tool that aims to help reporters cover beats, and that was prompted by work by Knight-Mozilla Fellows and a presentation at Hacks/Hackers London.

Learning data When and How to Use Census Microdata

When and How to Use Census Microdata

Robert Gebeloff‘s primer on working microdata magic

Project D3 data Comparing the Net Cost of College

Comparing the Net Cost of College

The Chronicle of Higher Education set out to compare net cost of colleges and found an unexpected discrepancy. The team describes the piece they created to help explain the difficulty in comparing net costs.

Project linked data Covering the European Elections with Linked Data

Covering the European Elections with Linked Data

The BBC News Labs team explores ways of exposing linked data in public-facing election coverage, and encounters some interesting challenges.

Learning data Pushing Hot Buttons with Census.gov

Pushing Hot Buttons with Census.gov

Ronald Campbell on using census data to find facts in a world of speculation

Learning data Distrust Your Data

Distrust Your Data

Jacob Harris on six ways to make mistakes with data—and how to avoid them.

Learning data How to Use the Census Bureau’s American Community Survey like a Pro

How to Use the Census Bureau's American Community Survey like a Pro

Paul Overberg explains base tables and how to get the best data from them (hint: ask good questions!).

Learning data Newsroom Analytics: A Primer

Newsroom Analytics: A Primer

Jacqui Maher says it’s not just the numbers, it’s what they mean about the audience.

Project data How We Made the SOTU Twitter Visualization

How We Made the SOTU Twitter Visualization

People tweet what they think, when they think it—and, crucially, we wanted to provide a visualization for the State of the Union speech which reflected that. This wouldn’t be a (shudder) word cloud based on frequencies but a way to track the conversation on Twitter as it was directly influenced by the President’s speech.

Learning data You Got the Documents. Now What?

You Got the Documents. Now What?

Jonathan Stray’s guide to turning documents into data you can run with.

Learning data Human-Assisted Reporting Gets the Story

Human-Assisted Reporting Gets the Story

Tyler Dukes on combining the power of data-sorting tools with old-fashioned digging.

Learning data How to Make a News App in Two Days

How to Make a News App in Two Days

As part of the orientation week for the 2014 class of Knight-Mozilla OpenNews Fellows, fellow nerd-cuber Mike Tigas and I led a hackathon at Mozilla’s headquarters in San Francisco…

Project Python scraping Twitter data To Scrape, Perchance to Tweet

To Scrape, Perchance to Tweet

At the Chicago Tribune, we had a simple goal: to automatically tweet contributions to Illinois politicians of $1,000 or more, which campaigns are required to report within five business days. To see, in something approximating real time, which campaigns are bringing in the big bucks and who those big-buck-bearers are. The Illinois State Board of Elections (ISBE) has helpfully published exactly this data for years online, in a format that appears to have changed very little since at least the mid-2000s. There’s no API for this data, but the stability of the format is encouraging. A scraper is hardly an ideal tool for anything intended to last for a while and produce public-facing data, but if we can count on the format of the page not to change much over at least the next several months, it’s probably worth it.