Computerworld - Reporters wrangle all sorts of data, from analyzing property tax valuations to mapping fatal accidents -- and, here at Computerworld, for stories about IT salaries and H-1B visas. In fact, tools used by data-crunching journalists are generally useful for a wide range of other, non-journalistic tasks -- and that includes software that's been specifically designed for newsroom use. And, given the generally thrifty culture of your average newsroom, these tools often have the added appeal of little or no cost.
I came back from last year's National Institute for Computer-Assisted Reporting (NICAR) conference with 22 free tools for data visualization and analysis -- most of which are still popular and worth a look. At this year's conference, I learned about other free (or at least inexpensive) tools for data analysis and presentation.
Like that previous group of 22 tools, these range from easy enough for a beginner (i.e., anyone who can do rudimentary spreadsheet data entry) to expert (requiring hands-on coding). Here are eight of the best:
What it does: This utility suite available from Christopher Groskopf's GitHub account has a host of Unix-like command-line tools for importing, analyzing and reformatting comma-separated data files.
What's cool: Sure, you could pull your file into Excel to examine it, but CSVKit makes it quick and easy to preview, slice and summarize.
For example, you can see all your column headers in a list -- which is handy for super-wide, many-column files -- and then just pull data from a few of those columns. In addition to inputting CSV files, it can import several fixed-width file formats -- for example, there are libraries available for the specific fixed-width formats used by the Census Bureau and Federal Elections Commission.
Two simple commands will generate a data structure that can, in turn, be used by several SQL database formats (Mr. Data Converter handles only MySQL). The SQL code will create a table, inferring the proper data type for each field as well as the insert commands for adding data to the table.
The Unix-like interface will be familiar to anyone who has worked on a *nix system, and makes it easy to save multiple frequently used commands in a batch file.
Drawbacks: Working on a command line means learning new text commands (not to mention the likely risk of typing errors), which might not be worthwhile unless you work with CSV files fairly often. Also, be advised that this tool suite is written in Python, so Windows users will need that installed on their system as well.
Skill level: Expert
Runs on: Any Windows, Mac or Linux system with Python installed.
Learn more: The documentation includes an easy-to-follow tutorial. There's also a brief introductory slide presentation that was given at the NICAR conference last month.
Related tools: Google Refine is a desktop application that can do some rudimentary file analysis as well as its core task of data cleaning; and The R Project for Statistical Computing can do more powerful statistical analysis on CSV and other files.
What it does: This popular jQuery plug-in (which was designed and created by Allan Jardine) creates sortable, searchable HTML tables from a variety of data sources -- say, an existing, static HTML table, a JavaScript array, JSON or server-side SQL.
| Quarter ending | Unit sales (millions) | Device |
|---|---|---|
| 2010-06 | 3.3 | iPad |
| 2010-09 | 4.2 | iPad |
| 2010-12 | 7.3 | iPad |
| 2010-12 | 16.2 | iPhone |
| 2010-12 | 4.1 | Mac |
| 2011-03 | 4.7 | iPad |
| 2011-03 | 18.6 | iPhone |
| 2011-03 | 3.8 | Mac |
| 2011-06 | 9.3 | iPad |
| 2011-06 | 20.3 | iPhone |
| 2011-06 | 4.0 | Mac |
| 2011-09 | 11.1 | iPad |
| 2011-09 | 17.0 | iPhone |
| 2011-09 | 4.9 | Mac |
| 2011-12 | 15.4 | iPad |
| 2011-12 | 37.0 | iPhone |
| 2011-12 | 5.2 | Mac |
What's cool: In addition to sortable tables, results can be searched in real time (results are narrowed further with each search-entry keystroke).
Drawbacks: Search capability is fairly basic and cannot be narrowed by column or by using wildcard or Boolean searches.
Skill level: Expert
Runs on: JavaScript-enabled Web browsers
Learn more: Numerous examples on the DataTables site show many ways to use this plug-in.
Free Insider GuideCopyright © 1994 - 2013 Computerworld Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of Computerworld Inc. is prohibited. Computerworld and Computerworld.com and the respective logos are trademarks of International Data Group Inc.
