Skip to content
November 23, 2012 / Damien Irving

RunMyCode.org – the answer to open computing?

In theory, a journal paper should include enough detail for the reader to be able to reproduce the paper’s central finding. However, for the weather and climate sciences, which are essentially a branch of computational science these days, reproducibility is severely hampered if the reader doesn’t have access to the author’s data and source code. This dilemma is very nicely (and somewhat bluntly) summarised by Donoho et al (2009):

“In stark contrast to the sciences relying on deduction or empiricism, computational science is far less visibly concerned with the ubiquity of error. At conferences and in publications, it’s now completely acceptable for a researcher to simply say, ‘here is what I did, and here are my results.’ Presenters devote almost no time to explaining why the audience should believe that they found and corrected errors in their computations. The presentation’s core isn’t about the struggle to root out error — as it would be in mature fields — but is instead a sales pitch: an enthusiastic presentation of ideas and a breezy demo of an implementation. Computational science has nothing like the elaborate mechanisms of formal proof in mathematics or meta-analysis in empirical science. Many users of scientific computing aren’t even trying to follow a systematic, rigorous discipline that would in principle allow others to verify the claims they make. How dare we imagine that computational science, as routinely practiced, is reliable!”

Most scientific journals state that authors should make their data available on request, however policies on software availability are far less consistent. A number of progressive journals are beginning to demand that authors submit source code along with their manuscript (e.g. see the policy for Science here), but these journals are in the minority.

Given the highly politicised nature of weather and climate science, one gets the feeling that our profession and it’s associated journals will be at the forefront of the push towards open computing. As some commentators suggest (e.g. Ince et al, 2012), the full release of source programs should (and may soon be) universally mandatory for results that depend on computation. For individual researchers, this begs an obvious question: is there an easy and convenient way for me to make my data and source code available upon publication? Enter RunMyCode.org.

Earlier this year, RunMyCode.org was launched as a website where authors can create a companion page for their journal publications (see Stodden et al, 2012 for details). On a RunMyCode.org companion page, readers can not only access the author’s code and data, but can also execute that code directly on the cloud using either the author’s data or their own. Most of the initial companion pages posted on the website were created by people working in finance and economics, however I contacted RunMyCode.org this week and they very promptly installed my requested software packages onto their cloud system (Python, in this case).

So rather than get caught out when the open computing era fully arrives, I’ve decided that I’m going to jump in right now. For my next publication, I’ll endeavour to publish a RunMyCode.org companion page. I’ve even located a useful wiki on best practices for researchers publishing computational results, while the Reproducible Research and RRPlanet websites also have some pretty useful information. I’m sure the companion page process will make for interesting blog content, so I’ll be sure to let you know how I go!

Advertisements

One Comment

Leave a Comment
  1. drclimate / Nov 30 2012 12:09

    Figshare (http://figshare.com/) also looks like a good option for sharing supplementary material to go with your journal papers (particularly figures), however it doesn’t have the cloud computing capabilities of RunMyCode

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: