Introducing FourScore

Project OpenNews code convening sentiment grid Introducing FourScore

Speedy sentiment-grid making from WNYC + Al Jazeera America

FourScore is a collaboration between the WNYC Data News team and Al Jazeera America. This project was documented and released as part of the first OpenNews Code Convening.

In 2011, the New York Times collected reader responses to the death of Osama Bin Laden on a two-dimensional grid. Since then, WNYC has used a similar format with great results, like this Valentine’s Day chart of relationship status and happiness.

from the WNYC Valentine’s Day chart

At the 2014 OpenNews code convening, we took on the task of turning these examples into something reusable that could allow other organizations to produce something similar with a bare minimum of technical know-how. The result was FourScore, a library that allows you to set a few configuration options to produce your very own interactive sentiment grid. It even works in IE8, and maybe doesn’t totally not work in IE7.

How It Works

Cloning the repo gets you an almost-working sentiment grid. The only two things you have to define are a data source that provides the X/Y coordinates of existing submissions, and a data destination to which a new submission can be sent. Both the source and destination can be a Google spreadsheet (more on that in a minute). Beyond that, you can configure to your heart’s content by changing the default options for things like grid size (do you want 5x5? 10x10?):

"gridSize": 5

or color scheme (maybe you want red or yellow instead of blue or a Hot Dog on a Stick combination of all three):

"colors": ["yellow","red","blue"]

Or what other form elements are displayed:

"fields": [
  {
    "name": "Favorite sandwich",
    "type": "text"
    ...
  }
]

What We Did

We had three main goals going into the project:

  1. Make the form setup and configuration as simple as possible.
  2. Maximize browser compatibility and styleability.
  3. Find a UI that better balances the competing goals of participation and presentation.

Open source” vs. “available source”

In some sense, the WNYC sentiment grid was already “open source.” Virtually all of the code is client-side JavaScript; you could view-source in your browser, figure out how it works and then reuse or mimic it as needed. But this presumes that you have both the expertise and patience needed to deconstruct someone else’s spaghetti code, and you probably don’t. Ultimately you’d be better off starting from scratch than investing the time to get into the author’s brain and jerry-rigging the code for a different situation.

There’s a big gap between source code that is available but esoteric and undocumented and open source code that is designed with reuse in mind. That’s the gap we wanted to bridge in Portland, turning a one-off into a many-off, which means worrying about things like:

  • Dependencies and browser compability. We can’t assume that a web producer in a tiny newsroom has the savvy to manage dependency issues, or that every audience has the same browser/device profile as ours. Maximizing compatibility is important.
  • Breaking things up into logical pieces. One of the keys to useful open source is separating out customization from the guts of an app, so that users can change the things they need to change without touching the rest. If you have to modify any actual JavaScript code to do basic customization, we’ve failed. We tried to centralize typical options in a config file and made a point of putting all the styling into stylesheets where it belongs. Boo inline styles set in JavaScript, yay stylesheets! We further subdivided into a “base” and “theme” stylesheet, the former of which contains core styles that you probably don’t want to edit, and the latter of which controls the look-and-feel stuff like colors, spacing, and breakpoints that you probably do.
  • Documentation. If a codebase is put on GitHub and its behavior isn’t documented, does it make a sound? Documentation is the unsung workhorse of code reuse. It’s not fun to write, but it tends to make the difference between a project that gets used and a project that gathers dust.
  • Workflow. Code doesn’t exist in a vacuum; it’s important to understand the workflow around it. This app requires data coming in and going out, but a random web producer may not have the ability to set up a database or a Rails app to handle that data. One of our priorities was figuring out a workflow that would allow you to handle data without writing any code, which is easier said than done. We came up with a way to wire things to a Google Form and spreadsheet, but even that isn’t quite enough, since you need to be comfortable in the web inspector to figure out the form element IDs Google uses and add them into your app. Enter this bookmarklet, which, when used while viewing a Google form, spits out a sample config file for FourScore that includes the details of the form you created. (Using Google spreadsheets as a backend has plenty of problems, which we cover in the docs).

The twist ending of these tasks is that as much as they are a public service for other people who want to use your code, they are also good for you. Scrutinizing dependencies in your code makes it better. Documenting your code makes it better. Breaking your messy omnibus code into smaller pieces makes it better. Writing useful open source code makes you a better coder.

Adventures in IE8

The original New York Times project was done in Flash (remember Flash?) and the existing WNYC projects were produced with some Highcharts trickery that starts to get weird on old browsers. We decided early on that we needed to make browser compatibility a top priority because IE8 is still a thing, and small newsrooms can’t justify using this sort of tool if it flat-out breaks for a big chunk of their users. We also wanted to avoid a long list of dependencies, so something like an SVG->VML conversion or a big list of shims for modern array methods was out. We decided to see how far we could get with no SVG, no canvas, and no fancy CSS properties.

This posed a lot of weird challenges. Something as simple as drawing axis lines without messing up the grid proportions becomes quite an adventure. We ended up with a lot of commit messages like:

Author: Noah
Date:   Tue Apr 15 19:55:25 2014 -0400

    dumb hack for IE8

And was it worth it, or should we have just drawn a line in the sand and taken the modern browser way out? I’m honestly not sure. It was an awful lot of gymnastics for a minority of users, but at the same time, access matters, and underserved audience segments are among the most likely to be using an old browser or an old device. We don’t want want to marginalize them if we can help it.

No More Disc-hover-y

There’s a big flaw in the presentation used in both the New York Times and WNYC examples: the grid is the interface for participation (you click to add yourself) and the interface for reading content (you hover around, hunting for interesting comments to appear). This kind of sucks. There’s not much point in even collecting comments or demographic info from respondents if it’s going to be so hard to surface, and as a user there’s not much reason to stick around once you’ve marked a spot. Relying on hover for discovery presumes that your users have tons of patience (they don’t), aren’t using touchscreens (they are), and don’t mind being turned into prospectors (they do). Repeat after me:

In order to better separate participating and the payoff of seeing what other people have said, we broke out the existing submissions into a separate box. We also added micromaps, so you could see at a glance where any given comment sits on the grid, and some basic filters so you could see submissions by quadrant.

We’d imagine extending this so that you see things like a bar chart breaking down the demographics of each quadrant. Because the layout is controlled entirely with CSS, it’s easy to make it responsive, flip the order, or prioritize one more strongly over the other.

Where We Go from Here

Not bad for two days and a plane ride, but there’s still plenty of work to do smoothing out rough edges, improving the docs, and otherwise making FourScore something more people would actually use. Got a question, bug report, or pull request? Let us know.

About Michael Keller

Michael Keller is a reporter and developer on the Al Jazeera America Interactive Multimedia Team where he alternates between the phone and Sublime Text 2. He is also the co-founder of csv soundsystem, a New York City-based hacker collective and datathon dreamteam.

About Noah Veltman

comments powered by Disqus