Skip to content
May 1, 2014 / Damien Irving

What’s in your bag?

This post has been superseded by the toolbox page.

Warning: For those who aren’t into sports, try and push through the first paragraph – there’s a useful analogy I’m working towards, I promise!

If you’re a golfer, or if you know someone who is, you’ll be aware that there is a lot of equipment involved. The 14 clubs in a typical golf bag each have a very specific function. Some are for hitting the ball low and far, others high and short, out of sand, off a tee, on the putting green, etc. A common preoccupation among golf players involves checking out what clubs other people have in their bag. Which brand do they use? What grip thickness, shaft flexibility and club head size have they gone for? Do they think they impart more backspin on the ball with their new sand wedge, or their old one?

This phenomenon certainly isn’t restricted to golf players. Cyclists pore over each other’s bikes, while hikers can talk about their camping gear for hours. Likewise, when you attend a workshop or even a social event in the weather and climate sciences, conversation invariably turns to your research setup. What programming language, operating system and reference management software do you use? Are you a Microsoft Word or LaTeX person? What program do you use to edit your graphics? While this might seem like idle chit-chat, I’ve been alerted to some very useful software packages through conversations exactly like this. As such, I wanted to share my research setup:

Operating system: Ubuntu (desktop), OS X (laptop)

Data analysis and visualisation:

  • Simple manipulation of netCDF files (e.g. changing metadata/attributes, renaming variables): NCO
  • Simple data analysis tasks on netCDF files (e.g. temporal and spatial averaging): CDO
  • Quick viewing of netCDF files: UV-CDAT
  • Data analysis: xray, which is built on top of Pandas and NumPy
  • Plotting: Iris / Cartopy (geographic maps) and Seaborn (regular histograms, line graphs, etc), which are built on top of matplotlib and basemap
  • Workflow automation: Unix Shell, Make

Code development: IPython notebook

Source code editing: NEdit

Version control: Git (version control software), GitHub (hosting service)

Simple data processing: OpenOffice Calc

Word processing: OpenOffice Writer for short documents and LaTeX for papers and theses. I used to use the Texmaker editor for writing LaTeX documents, but increasingly I’m using an online editor called Authorea instead because it’s great for collaborative LaTeX editing (see my post about it here).

Reference management: Mendeley

Presentations: Microsoft PowerPoint, which I then make available via Speaker Deck

Graphics editing (including conference posters): Inkscape

Cloud storage/backup: SpiderOak

 

Golf players also tend to have a favourite and least favourite club in their bag, and always have their eye on the latest clubs for sale in the local pro shop. Similarly, with respect to my research setup:

  • I would love to find an alternative to using PowerPoint for presentations. The rest of my setup consists of free and open source software, but I haven’t been able to kick the PowerPoint habit. There are a number of LaTeX packages for creating presentations, as well as OpenOffice Impress and Prezi, so I really have no excuse.
  • I would like to make the move to vi for my source code editing. It allows you to edit code at the command line (which means it doesn’t require a pop-up window) and is available with all Linux distributions, so you can never get caught out when using a new/foreign computer.
  • Like many scientific Python programmers, I’ve also got my eye on Julia. High-level programming languages like Python and MATLAB are great because they’re easy for humans to read (which speeds up the code development process), but the trade-off is that they run much slower than low-level compiled languages like C or Fortran. Julia is a new language that’s the best of both worlds: a high-level (i.e. pretty), compiled (i.e. fast) language.

 

What’s in your bag at the moment, and what have you got your eye on in the pro shop?

Advertisements

12 Comments

Leave a Comment
  1. Nick / Nov 29 2012 17:12

    I recently used Scribus – http://www.scribus.net/ – for a poster and a presentation. It took about half a day to get used to but once I got past that I really enjoyed it. Nice interface, heaps of control, you can use eps files for everything.

    • drclimate / Nov 30 2012 10:01

      Thanks Nick. Now that you mention it, the one thing missing from my ‘bag’ was an open source alternative to Adobe InDesign, for doing conference posters (Inkscape is more of an Adobe Illustrator alternative). At my last job we had a graphics department to do posters for us, so I never needed to, but now that I’m back out in the real world I think Scribus is exactly what I need. I’m going to go and download it now!

  2. xiaoxinyo / May 8 2014 16:37

    Operating system: Windows 8

    Programming: Python / UV-CDAT / Matplotlib / Basemap

    Code development: Originally python but started to use ipython yesterday and the real time debugging is just amazing.

    Source code editing: Emacs

    Version control: I don’t do any version control but I probably should. (like really, should)

    Word processing: Microsoft Word(I bought it through the home use program using my college email account for $10. I don’t think openoffice is bad but, hey, ms office is an eye candy!)

    Cloud storage/backup: skydrive

    SSH client: PuTTY plus Xming for X-windows support

  3. Damien Irving / Jul 22 2014 17:01

    I’m seriously considering moving to xray (xray.readthedocs.org/en/stable/) as my main data type, as opposed to numpy arrays or cmds2 transient variables in Python.

    • Jason / Oct 12 2015 08:29

      Could you share your reasons to migrate from cdms2 to xray? I’m more familiar with cdms2 (and cdat), but am thinking about moving to alternatives too. In particular from the vcs in cdat to iris or basemap, because I can’t get vcs to work properly after I got a new laptop (and honestly I feel the newer UVCDAT is even slower than the old cdat). So could you give me some guidance on how to create the geo-plots using iris after all the data analysis performed in transient variables, so that I can reduce the code modification to minimal? Thanks.

      • Damien Irving / Oct 12 2015 11:17

        Hi Jason. xray is hands down the best data processing library out there for weather/climate/ocean scientists. It is built on Pandas (which is a fantastic new data processing library) and has an active community working on it to answer all your questions, whereas cdms2/cdat is only built on numpy (which doesn’t have all the cool new features of Pandas) and hasn’t been updated or worked on in about a decade. I got to the point where I was having issues with cdms2 that couldn’t be fixed because nobody is working on the code-base anymore, so that was the final straw and I moved across to xray.

        Similarly, iris/cartopy is built on matplotlib and basemap and is hands down the best Python library out there for plotting geospatial data. At the moment I usually do my data processing with xray, write the results to a netCDF file, and then read that netCDF file with a separate iris script for plotting. I’ve suggested to the xray developers (see https://github.com/xray/xray/issues/621) that they could include a function for converting an xray.DataArray to an iris.cube, but until that happens you could either write your own converter or use the intermediary netCDF file approach.

        Hope that helps!

      • Jason / Oct 13 2015 03:16

        Hi Damien,

        Thanks for your response. Seems that I can’t reply to a reply so I just post it here.

        I didn’t know that cdms2 hasn’t been maintained for that long, the UVCDAT is releasing updates regularly though. I just managed to get xray, basemap and cdat-lite work in conda on my machine, the vcs is still much a bomb and creates a segfault every time it gets involved, headache…

        xray sounds like a promising choice and I will definitely look into it, and the move to an alternative plotting module seems necessary enough. It’ll be some pain re-building the tools one has been familiar with, I think that illustrates the point of choosing carefully a long-term investment. Thanks for the suggestions.

        Besides, you mentioned Julia in one the posts, how is that going on?

  4. timcdlucas / Oct 14 2014 22:32

    Fun.

    Operating systems: Ubuntu. Mint (as of last week).

    Programming: R and Python.

    Code development: RGedit, IPython qtconsole

    Source code editing: GEdit

    Version control: Git, Github (hosting service), GHI (terminal access to github issues.)

    Simple data processing: R, LibreOffice Calc

    Word processing: LaTeX unless I HAVE to use Word/LibreOffice

    Reference management: Bibtex

    Presentations: LaTeX Beamer

    Posters: LaTeX

    Graphics editing: Inkscape (vector), GIMP (bitmap).

    Cloud storage/backup: Dropbox

    Music: Clementine/grooveshark <- most important.

Trackbacks

  1. Mozilla Science Lab Week in Review, Oct. 13 – 19 | Mozilla Science Lab
  2. Software installation explained | Dr Climate
  3. Workflow automation | Dr Climate
  4. My Software Setup – matlipson

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: