Negative Feedback

Tag: productivity

  • The impact of environment: how much do IDEs contribute?

    Those who know me will know this is a much-beloved topic of mine. I am setting this stuff down in an attempt to purge some of these thoughts from my brain.

    During a conversation with a colleague yesterday, the idea of language productivity came up. It’s an old argument: sure, you can write less lines of code in say Python than in Java (see my post on this topic), but with a modern IDE, you’ll end up ahead as the IDE will be doing a lot of that typing for you. This forms part of a bigger conversation about the relative power of different languages, the relative merits of various IDEs and the way the brain works, before and after exposure to various models of computation. So, I’ve been maintaining a variety of links to research on these topics, and I’ll try to state my current understanding succinctly in the next couple of paragraphs. I would really value any feedback that you could give me (for the handful of people who actually read this). Although I enjoy reading anecdotes of personal experiences, I would appreciate most if you could point to proper scientific studies on the topic.


    So, this study, from 2000, compared C, C++, Java, Perl, Python, Rexx, and Tcl for a particular programming task: in this case a program to find which words could be spelled with particular phone numbers using the normal keypad letter mappings. They found that the “scripting language” programs were significantly shorter and took less time to write, but did not perform significantly poorer than the compiled languages. An interesting feature is that the compiled languages (C, C++ and Java) required considerably longer programs and took longer to write. The lines/hour for all languages were roughly comparable. They don’t mention which IDEs the programmers were using, but one assumes that at least some of them were using “modern” IDEs. A 2003 interview with Guido van Rossum, highlights this idea of Python requiring less “finger typing”. He’s obviously biased as the creator of Python, but considering that he also wrote the Python interpreter (in C) and has a large realm of experience in various languages, one would think he has had some time to experience the differences.
    Another study, from 2009 may explain a part of this. They found that reading code was largely a function of the number of tokens. This is important, because it implies that less tokens can aid in understanding already written code and have an effect on writing it. Importantly, they also find that simple domain mappings are easier to comprehend than complicated ones.
    I have not been able to find much quantitative research on the benefits of IDEs. Although I acknowledge that environment makes a big difference, I tend to think about code in my head without an IDE, which makes the actual language quite important for me. To some extent one must acknowledge that there is a personal aspect as well – some people prefer a particular kind of approach, as shown in this article about language mavens vs tool mavens, which I’ve linked to before. I’ve slowly been learning Eclipse, and it is quite amazing how much of the mundane Java stuff it can automate. I’m also a great believer in refactoring tools (I use rope, with bindings for Emacs). At the same time it is clear to me that much of the stuff you get in Eclipse could be addressed by streamlining the language a bit.
    Finally, the larger picture about fitting into an environment. This is where the established languages have the upper hand. Popular languages are more widely known, have more books and environments available and have the corporate OK. Also, if the libraries of the operating system you are using have been crafted for a particular language, there is a clear advantage to aligning yourself with that. As a researcher, though, it is often a bit of a conundrum: the real publishable results are more important than the interface, so you find the majority of computational research code has no GUI, and is only lightly coupled to the host OS. This is especially true of parallel code which inevitably has to run on headless cluster boxes.
    I don’t want to make this another post about choosing a programming language, but I will restate my final strategy: use the most expressive language available for prototyping – this will inevitably be a dynamically typed language with a REPL. For me, this is Python. This hooks into the constant lines per hour result of before. When the program produces the correct output and the algorithm is set, re-implement the slow bits in a lower level language. For me this is Fortran, but it could be C++ or whatever has good support for your problem domain. This strategy has placed me a in a poor position to benefit from IDE support. For that, you typically want to do everything in a combination of IDE and well-supported language (this is the classic Java/C++/C# for everything approach).
    Anyone with links to proper quantifiable research on programmer productivity (especially the effect of IDEs)?

  • Productivity: perception vs reality

    I am haunted by the thought that somewhere someone is doing what I am doing more efficiently. I pride myself on knowing efficient ways to do things. Note that this is somewhat different from “being efficient”, which in many cases involves not learning new ways of doing things you will probably just do once. I have slowly learned that I am fooling myself when I figure out ways in which to do things better next time, as there probably won’t be a next time and my time is largely wasted.

    That said, the feelings aren’t going away, and in fact I have learned things that make it even more difficult to figure out what’s going on. Witness this discussion – the meat of which is that keyboard shortcuts feel faster but mousing is actually faster as measured by a stopwatch. Oh no! Now I need to time myself doing common tasks instead of relying how productive I feel? This brings up an even more interesting question: What is more important to me day to day, being productive or feeling productive?
    The sad truth is that I have come to the conclusion that much of my investment into learning lots of different ways of doing things and even the environments that I enjoy the most have been focussed on making me feel productive rather than improving my actual productivity. Unfortunately I am now experiencing the sunk cost dilemma. I have spent almost 9 years of my life learning (and I stress that I have aimed to become proficient rather than just getting by) Linux, grep, sed, awk and all the unix tools and gnu plotutils so that I can do tons of stuff from the terminal. I have learned Emacs, memorising its arcane and non-transferrable shortcut keys and how to interface it with the shell commands. In Emacs I have learned AUCTeX for latex editing, where I often write code for pictures. I also use org-mode a lot for task management. I suppose my investment into learning Python was the least contentious of these decisions. On the way I have also shed Matlab and ignored powerful but expensive tools like Mathematica in favour of SciPy and Maxima, and lately Sage. I have replaced Simulink with Modelica.
    I am now beset by doubts and I wonder if I shouldn’t have taken the blue pill after all and when I started getting into Open Source when I graduated, just continued with my Windows box, installed Visual Studio and rekindled my original love for C++, used MS Word for my Masters and Matlab or even Excel for all my data processing, or perhaps laid my hands on Mathematica or MathCad. Would I not be more capable now and would I have not gotten more things done in the end by not going against the flow?
    Unfortunately there is no easy answer. I am typing this on a MacBook Pro, which I love because Mac OS X has a nice unix layer beneath it but also because there are less hardware issues than I have endured using Linux. I have not yet been successful in using XCode to do anything meaningful because I don’t want to write an app that I can’t use on my Linux box at work. I have found myself developing code that I am forced to maintain and run simply because I am the only guy on my floor that has Linux on his machine and I have not wrapped the stuff in a GUI so that our secretary can run it. So I experience the need post facto of being able to create GUI apps and have other people run them, and of having a common environment that people can help me with.
    However, every time I look into the “more productive” way I feel like gouging my eyes out. I hate word processors of pretty much every description, as the whole philosophy of “when it looks right it is right” galls me, and figures don’t float – what’s up with that? When I try to use Eclipse for development, I am up against the problems of using such environments for dynamic languages, in addition to the fact that all my current projects’ directory structures don’t lend themselves to Pydev’s idea of what a Python project should look like. In fact, when I try to use statically typed languages, I feel totally put out by having to think of the type information in the first place. I cannot find an easy way to get from where I am now into the mainstream, and it annoys me.
    I have vowed to stop looking around for now, until I have finished my PhD, but once that is done, should I become a good little MS drone? Have I wasted my time learning these skills or can I justify my rationalisation that I am more productive now? Does that even matter when I feel more productive (and happier) when I am using my current environment?