Negative Feedback

Year: 2009

  • You can’t get there from here

    I am reading “The design of everyday things” by Donald A. Norman. While I was writing the previous post about Eclipse I realised that a part of my problem may be my lack of a consistent mental model for the Project and Workspace that Eclipse forces you to use.

    I have seen the modern computer user interact with computers by observing my students and colleagues at university. One of the things that has become very fuzzy to most users is the concept of what a file “is”. When I get questions about LaTeX from newbies, they often get confused about the editor they are using to edit their files, the files themselves, and the compiler that produces the final output. I lived through a low-level time and still use the terminal a lot, so the distinction between files and the programs that act on them is very clear in my mind, partly because I often operate on the same files with many different programs. The insight I had while working with Eclipse is that the project gives substance in the IDE to the grouping of files that I just assume are grouped quite well by directories.
    When I think of my projects, I think of a directory with a memorable name that groups files, which can either be source files, scripts, library/module files or data. I sometimes use a Makefile to automate generation of outputs that need to be done in a certain sequence or when things change. I understand this process.
    IDEs seem to operate on a different mental model, which I have not exactly figured out. You create a “Project”. I am not exactly sure where this project “is”. I assume when you add a file to the project, an entry is created in some kind of project file that is kept somewhere. In the case of Eclipse, these project files are in the workspace directory, in the case of Netbeans, they are in the directory you created to hold your project. So, fundamentally, you are not supposed to edit these files directly, you are supposed to interact with your project via the IDE. And perhaps this is the biggest problem for me. I don’t see the gears turning in the IDEs, so it’s hard for me to form a model of what will happen when I do things. And most importantly, I can’t use my default method of trying out a few things in the terminal before automating them because there is aren’t really separate programs working together.
    This forms part of a larger problem for me. I have spent a lot of time (about 8 years) becoming proficient at the Unix commandline. I have learned a new language containing new words. In my mind “grep” is a verb meaning to look for something. I use it without thinking. Other things I use in the commandline without thinking: cut, paste, join, comm, diff, tr, sed (with its language), awk (with its language) and many of the features of bash that make it an efficient environment for entering commands. The keystrokes of readline (the library used by bash for input) are similar to the keystrokes in emacs (and as it happens, many of them are supported by the widgets in Mac OS X). I have also learned several computer languages (I will make that the topic of another post), and because I hate repetitive work so much, I find these tools help me to automate them away.
    Now, I’m not saying all these things were easy to learn, but I have learned them and I am here now with all this knowledge in my head and an open terminal on every computer I use. I actually thought I would try to avoid the terminal today as an experiment, but it turns out I had to ssh into my computer at varsity to copy a file. I suppose I could do it with a vastly more involved setup that allowed me to browse that computer like one on my computer, but I am also sure that it would take me a lot longer than just going to a terminal and entering the commands.
    So a lot of the cultural cues that are in graphical environments are not meaningful to me. In fact, many of them are downright confusing. Here I must say that most of the Mac’s software is graphical and well enough designed that it doesn’t bother me at all to use, with the exception of Xcode, which I still haven’t quite figured out. Some of the stuff is pretty opaque. I have also realised that I have stopped trying to right click on stuff — something Windows users do all the time, because the Mac interface guidelines require actions to be available without right clicking as well, so it’s almost never required.
    So, a long rant about where I am with computer use. What’s the punchline? I find myself in a place where I am pretty efficent and can do most of the things I want to do without much trouble using the commandline or the tools I know. I am however aware that there are very good reasons to prefer environments that are more supportive and require less memory (from the user’s perspective). I just don’t know if I can there from where I am now without feeling stupid and helpless along the way. The irony is that the environments that make me feel stupid and helpless are the ones that are supposed to save people from feeling stupid and helpless in mine (the commandline).

  • Choosing an IDE – part 2, Eclipse

    I have been using Eclipse for some project editing and a couple of things have improved since the last time I looked at this. I have gotten a faster laptop, Eclipse Ganymede has come out and Pydev has improved hugely due to the commercial arm merging with the open source one.

    The good bits

    Editing python files is easy and efficient. The discovery of interpreters and installed modules is good and the type inference (which is the hardest part of getting Python editors to work well) is quite good — better than Netbeans.

    The strange bits

    (things I think aren’t broken as much as I just don’t understand why it’s done that way):
    Why is it that, when I edit a file in another editor, then open Eclipse to that project, it does all the checking that allows it to say ‘this resource is out of sync’, but that I have to follow the instructions “Go to File|Refresh”. Surely it should just refresh itself, or at least make the action clickable in the message. Why doesn’t it just load the file as it is when it opens?
    What is up with workspaces? I understand (kindof) what they are, but what are the advantages of having workspaces rather than just storing the project information in a file in a directory that I can sync and use on another machine, or zip up and send someone without using the IDE?

    How does my workflow fit into the “Project” mold? Let’s say I have my PhD, which is a real-world project. Normally I would have a PhD directory and bang stuff related to my PhD in there. Now, there’s the thesis document itself which is written in LaTeX, which I can edit using TeXlipse. There’s also loads of programs written in many different languages, many with some levels of support in Eclipse. What do I do? What seems to make a bit of sense is to create a PhD workspace and then have projects for each of the programs so that I can edit them in the right mode and have all the right compile options and whatnot. If I just have one big PhD project, how do I tell Eclipse to change behaviour based on the file I’m editing like Emacs does? I mean, if I have some Fortran sources in front of me, I probably want to compile them when I click on the play button. If I have a Python script open I probably want to run it. I guess this is from the use of Eclipse to edit “Applications” — single, well-defined programs that consist of many files that all end up going into the application and where there is a clear build target to build and run. Perhaps with some tests. My projects just aren’t like that.

    The bad bits

    Refactoring (one of the big reasons I want to use an IDE) is broken for me. Many simple things broke in strange ways. “Extract variable” would place the variable definition inside a function defined earlier in the file, which would break everything if I did it. It’s still better than Netbeans, because the Python plugin for Netbeans has renaming as the only refactoring option. On the other hand, the rope support in Emacs means that refactoring just works.
    Syntax highlighting is also strangely limited. I have grown very used to editing stuff with syntax highlighting and support for code formatting. Python is ok in Eclipse, but Makefile syntax is not supported (edit: Makefiles are supported by installing the C/C++ dev tools), neither are bash or awk, which I use quite a lot.
    I am also very annoyed that stuff isn’t indented as I type, but rather that I have to do periodic “clean my code” up formatting (even if it’s a single key). I have become spoiled by Emacs, which can usually do the right thing when you press tab (who inserts a real tab in code ever anyway?). So your code always looks pristine. I don’t know if it really boosts my productivity, but it’s nice to have.
    The last thing has to do with how I manage my files on my laptop and my desktop. I sync all my files to my destop when I get to work and sync back to my laptop when I leave. This is important functionality that has to work without me holding its hand. I use unison, which never breaks — I have been using it daily for almost 10 years and it has not failed me in that whole time. The information that Eclipse stores in the workspaces just doesn’t translate all that well between multiple machines. I am guessing this is because I have different versions of Eclipse running on my Mac and on my Linux box (because I haven’t upgraded my Linux box). It’s not really Eclipses fault, but it’s annoying none the less.

    The verdict

    I get the persistent idea that I’m doing it wrong — that if I just knew a couple of tricks in Eclipse it would all start to come together, and I must say I enjoy the passive information stuff a lot (the documentation pops up when you start typing, there’s autocompletion of methods, there’s a lot of highlighting that lets you know there are problems in the code. That’s all great, and I would even consider doing my next project in pure python just for those benefits, which is what I suppose drives the one language for everything mentality I see so much these days, but for my existing code base, I’m not going to get the benefits without a lot of swearing using Eclipse.
    If anyone reading this has used Eclipse (or another IDE I have not tried yet) and is thinking “Geez, he should just do this or read that”, pleas post a comment. I read manuals. I figure stuff out. I can really see how Eclipse can help if you plan to write a Python program that’s all in python all the time, but if there are other languages in there, it becomes less useful.

  • Choosing an IDE – part 1, The Problem

    As I have mentioned before, I am a cli guy. I honestly cannot imagine getting through a day without using the terminal. Drag and drop is all well and fine, but wildcard support and iteration in the shell is hard to do in finder (or explorer or nautilus or konqueror etc). That being said, I can see the benefit of many of the things that a modern IDE supplies. I am a big fan of Emacs, but the bells and whistles are very alluring.

    I have some specific and interesting needs due to the variety of tools I have used to cobble together my solutions to problems that I have had in the past. Domain specific languages have filled many needs in these problems, so I need a lot of different syntaxes highlighted.
    As an example of a more or less typical project I will describe my project assignment tools. The problem is assigning projects to students for our project subject. The idea is that each lecturer comes up with a couple of project proposals, the students choose their top 10 (in order) and a program then matches projects to students by minimising the “unhappiness” while obeying constraints on how many students each lecturer should get etc.
    The process uses the following components, held together by a Makefile:
    • To allow the students to specify their projects, we use an Excel spreadsheet that verifies that their choices are valid etc. They fill in the project selection spreadsheet and e-mail it to us.
    • A Python script reads and compiles these spreadsheets into a table
    • A second Python script converts this table into a MathProg data file
    • GLPK is used to solve the problem (specified using MathProg)
    • Unix tools (grep, sed, awk, join, tr, xargs) are used to massage the run results
    • The massaged results are processed by python to generate an HTML file
    • The HTML file is coloured using CSS
    The parameters of the optimisation will be tuned and the optimisation re-run many times. I will also need to answer many questions about specific students based on names or student numbers, which is easily obtained by grepping the intermediate results generated by the above process. These are the reasons why a single Python program doing the above would be a lot larger than the current files are: I would need to implement persistence and memoisation to avoid rerunning lots of code (which is currently handled by make), in addition to coding querying interfaces to allow queries of the results.
    But back to IDEs. My ideal IDE does not force me to use one language for everything. It supports my use of domain specific languages by at least colouring the code, but ideally providing intelligent indentation, bracket matching and language help. It allows complex code refactoring and is aware of scope and different senses of variables. A debugger is a plus, but because I mostly use dynamic languages, I find I use the debugger infrequently. A last hurdle is that I either need the same program on both of the platforms I use (mac and linux) or use a scheme for projects that doesn’t break when the project is edited by another program (plays well with others).
    Much of what I’ve described is delivered by Emacs. It has code highlighting, tabbing and bracket matching for almost every language ever developed. It also has debugging and refactoring for python via interaction with pdb and rope, although I will admit it’s not as slick as some modern IDEs.
    After hearing a lot of people rave about the advantages of IDEs I have decided to give them another shot. So far I have installed Eclipse and Netbeans, as they both run under Mac and Linux. I will be posting some of my experiences here. Please leave a comment if you have some suggestions I should try out. I am really keen to boost my productivity, so I won’t shoot down any suggestions out of and (except that I have tried vi and didn’t like it that much).

  • Understanding and intent

    When our ancestors were attacked by animals, they needed a way to predict their behaviour to come up with a counter strategy. An effective way of doing this is to think about the animal’s intent. “It’s trying to catch me, so if I run here, it will run there” etc. Of course, humans who were good at avoiding being eaten were selected and here we are with our brains wired for modelling intent. Unfortunately, many things to not have intent. Rocks, the weather, computers are all examples of things that are not “trying” to do anything. They exist and behave according to deterministic rules. I say unfortunately, because our over-active intent modelling comes up with intent anyay. Children easily think that simple inanimate things are out to get them (or ascribe some kind of autonomy to them), but grown-ups are often caught by the same trap when they try to understand the motivations of the computers or other complex devices that they use.

    I am in two minds about this phenomenon. On the one hand I think that it leads to sloppy thinking, where we are ascribing thought to something that has none, or leveraging a broken model to extrapolate far outside its realm of usefulness. On the other hand, it does make things easier to understand. I think my refusal to use intent when I was teaching programming is one of the reasons my students didn’t understand what I was saying. How many times have you heard things like “So, the rock wants to fall, because it’s being pulled by gravity, but there’s this thing in the way, so it has to go around it” (perhaps only if you listen to the Naked Scientist or are an engineering educator). This is clearly wrong in the particulars: The rock has no intent, it doesn’t “know” anything about what’s in the way — it is experiencing an acceleration proportional to the resultant force acting on it, that’s about as far as it goes — but it is right in allowing someone to understand what will happen.
    So the riddle is this: how do I leverage this mode of thinking in myself and others to increase understanding or make explanations easier but avoid coming up with a new religion to explain the intent of rocks?

  • Front end loafing

    I am a big fan of planning. I spend a lot of time on increasing my future capabilities. I learn new programming languages, explore new environments and try to stay up to date on research directions. All of this means that I don’t do as much right now as I probably could. Hopefully, however, it does mean that I do more now than I could have done if I hadn’t followed this strategy in the past. My industry has a name for getting more stuff done up front: Front End Loading, hence the title of this post.

    There is a fine balance between getting more done now and being more productive later. I tend to be a ‘sharpen the axe‘ kind of guy — I believe that you work more efficiently with sharp tools, that sharp tools require some time to learn and maintain and that that time tends to be less than what you save. Unfortunately, like any habit, this needs to be kept in check as well. When times are tough, my first instinct is to take a step back and find a better tool. Sometimes that is just procrastination. Sometimes it is time to put your head down and keep chopping.
    PS: This whole post was really just a test of twitterfeed.