Investigative journalists are just “scraping” by …

New tools for shining light on complex public data

'Refined' tools for shining light on complex public data

Dan Nguyen at ProPublica, discussing the topic of public records gathering as a programming challenge, has an excellent summary of how to use advanced tools for what used to be called computer-assisted reporting.  For example, check out Google Refine, which is downloadable software, can quickly sort and reconcile the imperfections in real-world data.

ProPublica used these tools in their  investigation of the financial ties between drug companies and doctors, Dollars for Docs.  Dan observes that several drug companies had been ordered to disclose who they paid to speak and consult on their behalf, “but they made the records hard to analyze, seemingly making the data “impossible to download.” In the end, ProPublica is proud to report that ”There is no data on the Internet that is actually impossible to download.”

This entry was posted in Best_Practices, Tools. Bookmark the permalink.

One Response to Investigative journalists are just “scraping” by …

  1. Pingback: Alexander7

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>