Last week I spent a day going over the basics of the Unix shell, version control using git and some Python with engineering students. Here are some of the things I said, links to interesting tools and my insights from crowd feedback.
Motivation
Every year, new project students arrive on campus and are asked to do larger projects than they are used to. The methods they have used for data processing up to now are usually not up to the task. The ubiquity of the spreadsheet means that most of them have opened their files in a spreadsheet and then manually applied whatever operations they needed. In my fourth year projects and also at postgraduate level, the problems of scale mean that this method probably won’t work.
I’ve been inspired by the people from Software Carpentry to set up a longish session where I help students to install the software they need and start to understand the power of the tools. The goal is to get them over the initial stage where you don’t know anything to a place where they can search for the answer themselves. I don’t have permission from the Software Carpentry guys to call this one of their events, and I only loosely based my session on theirs. They appear to have more of a life-sciences focus, whereas I tried to use engineering-type data in my session.
I also chose to do this in Windows, but to use a Unix shell in order to ease the transition for those who would end up using Linux for their simulation. It doesn’t make sense to throw someone into the deep end here. It’s better to feel like you can do something at the end of a session than being made to feel powerless.
Installers
I chose the following installers for the session:
The Windows version of git. This is a good start as it includes git-bash, which also has a great subset of the unix tools.
The Python(x,y) distribution of Python. There are newer distributions around, but this one is still up to date and very user-friendly. If you are going to be doing science in Python, you need a distribution like this, as the batteries-included philosophy of Python doesn’t extend to science (yet).
Unix shell
I start with a review of where Unix comes from. As an aside, I highly recommend In the beginning… was the commandline for a pep-talk about how awesome the command line is as well as the most complimentary description of Emacs I have ever read. It helps to keep a picture of a teletype in mind when working on the terminal as it informs many of the design decisions.
Now that everyone is in line-by-line mode, we can start the installer. The first two screens provide talking points about the differences between the terminal and the shell, as well as a profitable digression on the problem of different end-of-line conventions.
After running the git installer, you have access to git bash, which you can start from the start menu.
Of course, we start with ls, and explain the idea of flags as well as the concept of a working directory (pwd and cd are introduced here, as well as . and ..). Then I move on to other commands, including echo, which allows me to explain how blobbing is handled by the shell rather than by each command we will be using. Most of this part is just me showing how useful the shell can be, with examples motivating cat, cut, grep, head, wc and some others. As the examples spin out, we start to use redirection and piping. I think it is more important to leave with a sense of awe at the possibilities, as well as some basic understanding of what to search for than any real expertise in the command line. It may also be useful to go through the Software Carpentry shell lessons.
Version control
Now that everyone is roughly comfortable with the command line, it’s time to start using git bash for what it’s named for: git.
I’m not going to recapitulate the whole lesson here, especially because it’s better written up at Software Carpentry but the core is instilling a good set of nomenclature built around pictures which represent the structures git uses. First, here is how the repository commands work:
Figure showing the parts of a git repository, with commands to move between them
So this takes us up to basic commits. Now, we need to think about how commits relate to one another and how branching works. I use a visualisation similar to what GitHub uses, with dots for commits and arrow-boxes for branches.
If you’re reading this to learn git, I recommend that you do the git code school for the basics and then work through the challenges on Learn Git Branching, which will give you a way to think about how branching and commit chains work:
This interactive branching game is a great way to learn about git branching.
Python for Octave/Matlab users
All of the students in the session had learned programming in their undergraduate course, but most of them had learned Octave. This means that basic ideas from programming are already understood, but that the differences between Octave and Python are more important, because they may be working with a different mental model.
I started with the way names work in Python, explaining the idea that Python variables shouldn’t be thought of as containers, as they are in Octave, but rather like labels. I did some of the stuff interactively, but if you’re reading this now, you only really need this article, which explains it with some great visuals. I can also highly recommend the Online Python Tutor site which allows you to generate these kinds of diagrams for running code and see each step. Here’s what that looks like for one of the examples in the article
I also covered some tips on looping, mentioning the excellent Loop like a Native talk.
Once these gotchas were covered, I also switched from the IPython terminal to the IPython Notebook. I gave a brief overview of how everything works and also mentioned that they need to study up on all the interesting magics.
Then it was time to pack up!
I can see why the SC guys do this as a two-day workshop, and I think if I do this again I will encourage students to bring files from their particular projects so that they can do some actual work rather than just examples. It’s always more meaningful to learn on a problem you actually want to solve than on some contrived example.
Leave a Reply