Negative Feedback

Tag: rant

  • The science of the general

    This is a draft of an upcoming comment hopefully to be published in Chemical Technology. Feel free to shoot with some comments, even if you’re not a chemical engineer.

    Chemical Engineering as a discipline by that name has been taught at universities since 1888. At this time it was seen as a small side-branch of Mechanical Engineering incorporating applied industrial chemistry, mainly serving the emerging chemical industries of the time. The curricula of these early courses would become more recognisable to modern Chemical Engineers when AD Little formalised the concept of unit operations in 1915, while the concept of material and energy balances had gained traction by 1935. Thermodynamics and process control had appeared by 1945, followed by chemical kinetics and process design by 1955 (see Origins of Chemical Engineering for a complete timeline). Henri Poincaré said “There is no science but the science of the general”, and one sees this instinct for generalisation at work as fundamental knowledge and general analysis supplanted a curriculum filled with specific examples from industry.

    During roughly the same time, computing devices were progressing from simple adding machines to fully programmable computers. Although computers had been used for process control on chemical plants since the late 1950s, the first personal computer was only released in 1974 and Chemical Engineering simulation software for the PC was released in 1981. In a recent Nature interview with Neil Turok from the African Institute for Mathematical Sciences, I was struck by his description of their curriculum design process. They had initially developed a “conventional syllabus involving math and physics”, but realised there wasn’t enough time to teach it. They then made a key observation: “… if you took a photograph of a scientist at work today, everyone looks the same … sitting at a computer and trying to make sense of what’s on the screen.” I think one could confidently replace scientist with engineer in that observation.

    The funny thing is that computers are often seen as a means to an end, something that can remain opaque to our understanding as long as the programs we use work. Although the same approach could arguably be applied to much of the equipment on our plants, we still teach Thermodynamics, Calculus and other subjects that are frequently described as not being used in the workplace. We do this because these subjects lead to a more fundamental, more general understanding of our plants. There is far greater likelihood of a particular Chemical Engineer using a computer in his day than any particular unit operation, but it is not common for us to teach the equivalent fundamentals behind computing as applied to the problems of Chemical Engineering. Instead, we focus on manual solution of sets of equations using methods that have very little in common with the computer techniques used by process simulators. A friend of mine summed it up well: “We’re teaching as though it were still the 70s, as though there is a reasonable chance of having to solve a problem on site without the benefit of a computer”.

    Perhaps a part of the problem lies in the demarcation of domains. The fact that computer science is a discipline in itself, or that there may be an electronic or computer engineering department in the same school as our Chemical Engineering departments may lead us to focus on the “real Chemical Engineering”, but the existence of an industrial chemistry course or a mechanical engineering course did not stop our predecessors from carving out the niche that would become Chemical Engineering. We would do well to remember that most of the simulators we use today were developed by Chemical Engineers rather than generic software houses.

    Another argument is that the principles don’t change much from manual calculation to computer implementation. However, the methods of solution and the fundamentals of computing have made such great strides in the last 100 years that understanding how to solve a set of mass balance equations by hand does not really enable one to reason about the algorithm being employed by a simulator. Not only that, but the algorithms and software that are available today enable the solution of whole different class of problems.

    We need to move beyond the simple treatment of computers as tools to solve our problems and appreciate that there is a general science underpinning their operation. As much as understanding Reaction Kinetics or Thermodynamics enables us to understand the design and
    failure modes of reactors, understanding the computational structure of our simulators, optimisation solvers and spreadsheets can help us to understand how to employ these new information unit operations more effectively. It may also enable us to develop the
    next set of tools for ourselves.

  • Lecturer rant: there should be a system for marks

    Even after six years at the university I am often amazed at how stupid we are.

    I use “we” there because I am part of faculty. I am amazed because of these stupid things (and these are just the ones I know of):

    Yearbooks

    The university yearbooks are written and maintained by hand. Yep, if you clicked on some of those links and saw lists of repetitive seemingly machine-generated content, you may infer that this is all on a database somewhere and simply sucked in to this document by some reporting system. You would be wrong. There are people in each department tasked with maintaining the system. Changes to the document are posed in the form of a manually generated diff and applied by humans. This means there is no automated consistency checking and no way of generating the dependency tree of a particular subject to answer a question like “I have failed GFU 320 – does this cost me a year?” because the dependencies are simply listed in the document. I have written scripts that scrape the PDFs for useful information like the number of periods allocated etc to check that they are consistent with the timetable (see the next point) and generate graphs like the one below that show the dependencies of our course. Why is there not an automatically generated graph like this with clickable links into the subject descriptions on the UP website? The graph is generated using Graphviz

    Timetables

    There is no single, up-to-date timetable for the university as a whole. Once a year the timetable book gets generated from a program called Syllabus Plus. After this point, all the advanced features of Syllabus, including distributed access by multiple people, custom web-based timetable views etc are ignored and the Word file containing the manually reformatted timetable information sorted by subject code is sent to each department. Each member of faculty then has to traverse this document searching for their subject and draw up a timetable in the traditional two-dimensional grid form themselves. This is a huge waste of effort! I have written scripts that read a dump prepared from Syllabus by our helpful IT staff in an Excel file that changes subtly each year even though I have given a detailed spec of the format I require and prepares timetables for the school of engineering.

    Unfortunately, this is not the end, because the venue bookings are handled on a totally different system which does not write back to Syllabus. This means that at any given moment it is literally impossible to reconstruct the exact timetable as it stands without talking to every lecturer involved, as the ad-hoc changes are only relayed to them and not recorded centrally beyond the subject name (not the group or other information you may need to generate the timetable). The whole situation is silly and costs everyone involved hours of their time as they sift through the book trying to verify that the information is correct. What makes it worse is that several other universities are allowing staff access to the syllabus plus system while we are using what boils down to one step above manual scheduling.

    Student marks

    Student marks are calculated by each lecturer using home-rolled spreadsheets that are almost guaranteed to have an error in somewhere. The rules to determine for instance if a student qualifies for a supp are subtle and often incorrectly implimented. To give you an idea, here is the logic for if a student qualifies for a supp in the school of engineering from the 2010 yearbook:

    In the School of Engineering a supplementary examination is only granted in instances where:

    1. A final mark of between 45% and 49% was achieved;
    2. A final mark of between 40% and 44% was achieved and where the candidate also achieved either a semester mark or an examination mark of 50% or higher;
    3. A pass mark has been obtained, but the required subminimum in the examination section of the module or divisions thereof has not been obtained.
    4. A final mark of between 40% and 49% has been obtained in first-year modules on 100 level.
    I have made a graph showing the regions for a 50% semester mark exam split:

    Is it not glaringly obvious that this should at least be coded up in a reusable spreadsheet function that can be distributed to everyone? Of course, that’s just a sticking-plaster, because the system is broken in more subtle ways.
    At the beginning of each course, an automatically-generated class list is sent to you by admin. It used to be a CSV file, now it is an Excel file (already my scripting has taken a knock right there). You then calculate marks in any way you want and send the marks back. Unfortunately they now send you a revised class list which does not have the same students in the same order as the previous ones. So what could have been a simple copy-and-paste now either becomes tedious lookup calculations or even more tedious manual lining up of the data. I think it is entirely obvious that we need things like the following ideas:
    • Accelerated marking system you scan in the memo and the papers, define active regions on the memo and mark directly into the system. The papers have a barcode sticker from a pile you issue the students at the beginning of each year, so your (lecturer) workflow involves going into the system and clicking through the memo for each paper. Totally transparent to the student and externals, a lot faster than carting around tons of papers and shuffling through them.
    • Grade calculation based on a tree structure with calculation nodes most all grade calculation can be seen as a tree-like aggregation of primary grades. Final mark is weighted average of semester and exam, each paper is sum of questions, which is sum of subquestions etc. You need a builder and navigator, with colour highlighting and live stats to answer questions like ‘why is this guy’s semester mark so low’ or ‘what will my class average look like if I adjust my semester mark’.
    • Audit trails all changes should be historied (BIG fail with spreadsheet-based systems). If an adjustment is made, this should be noted for this student so that one can track the total ‘free marks’ a particular student has received. This can also be used in
    • Automatic edge case detection: Students that are ‘close’ to particular boundaries like distinctions or failing or supps should be identified by the system and automatic adjustments should be suggested. This would of course be historied so that you can review how much action has been taken. With a proper system, this would include all the student’s subjects, so that the system would warn you if a student (for instance) only needs your subject to pass his whole course and has not received a lot of free mark. I have even thought a good system could be that you start your course with 0.5 percent per test (so 0.5 for semester mark, 0.5 for exam and 0.5 for final mark), so 1.5 per course. This is the adjustment we allow today anyway, as we round each of those marks. Lecturers can use some of these marks for adjustments. The system could even find optimal distribution of the free marks to maximize the chance that the student succeeds in the course. I mean, we’re only probably repeatable within about 2% to 5% on marks anyway.
    • Aggregated statistics lecturers should be able to see the distributions of the other courses this group has, and be warned about strange anomalies like an otherwise good student scoring really low or an average student doing particularly well.
    The thing that amazes me most is that we have all this student manpower doing projects to design websites and database systems and intelligent algorithms for fault-finding and inference, but we’re not using any of it on our own systems! I am sure that the development of a system with some of the features above would not be beyond our student body. A part of the problem is that no-one sees development of such systems as part of their job. If one person were to try to develop such a system they would end up wasting a lot of their own time, as the total loss of time is distributed among so many people. This means there should be a higher-up decision about introducing more efficiency in the system rather than a lone developer working on it. Unfortunately, the small losses aren’t accounted for in an actionable way and things keep going like they’re going. Perhaps I will should send a link to this blog post to my head of department…

  • Turing completeness is a trap

    Or why you can do everything in Excel but it may not be a good idea.

    My wife’s struggle with a spreadsheet from work yesterday got me thinking about this topic again. It has perplexed me for a long time that people use Excel for so much that other tools are clearly more suited for. There are many rants on the Internet about using Excel for database-like activities, and I will probably write a bit more on that later. However, I think the key phrase in the overuse of any powerful tool is “but I can do that in my tool, too”.

    What I mean is that one could approach someone who is an expert Excel user and say “I don’t like Excel for engineering calculations because it doesn’t allow me to track the units of the numbers like for instance Frink or Mathcad does”. This guy would say “but you can do that in Excel – just do X or Y”. Somewhere they are thinking “I will use another tool when I find something that Excel can’t do”. However, they are stuck in a Turing Tarpit. The problem is that all Turing complete languages are equivalent in power in this strange and abstract sense that it is possible to do the same calculation in both. This does not say anything about how easy it will be to do that calculation, or how maintainable the code will be – these are requirements that have very little to do with the computational power.

    So, you if you are taking the view that you will use Excel as long as it is possible to do the thing you want to do, you literally don’t need any other tool. Unfortunately the same can be said about any esoteric programming language like INTERCAL or x86 assembly. It is perfectly valid to say “but I can do that calculation in INTERCAL – I don’t need any other languages”. In a strict sense this is true, but this really just points out how little Turing completeness actually buys you in terms of useful programming structures.

    At this point it seems like a good idea to mention the other thing that keeps people in Excel. I think the marketers of spreadsheets have been using the “no programming required” line for so long that everyone thinks that spreadsheets are different from programming. In fact, they can be understood as functional programming languages with a two-dimensional (or three-dimensional if you take sheets into account) structure. Of course, there’s also VBA if you aren’t sufficiently resourceful to figure out how to do everything using only the built-in functions and cells, but the spreadsheet itself is also a computational device. So a lot of people think that “using Excel” is different from “programming”, when it differs mainly in the environment.

    Bottom line: if you are a single-language kind of person (who only uses Excel or only uses C) you need to understand that it is true that you will never find a program that your favourite tool won’t be able to solve in a strict sense, but that it is also true that other tools may make it a lot easier.

  • Why is no-one using electronic signatures?

    At the end of every semester we go through a little dance at the university called external examination. We send the exams to an external examiner and they have to indicate the paper is well set and fair, then we send them some of the marked papers and they indicate that they have been fairly marked and that the marks are reasonable. This process is monitored and our accreditation body (ECSA) requires that we keep the paper trail on file to check that everything was done right. They require a signature on a hard copy of the documentation at every stage of the process.

    At the moment I am also going about the business of registering as a professional engineer, which also requires much signing of documents. After the signing by the people involved, it has to be signed by a commissioner of oaths and sent on to ECSA as a hard copy.
    These situations have me wondering yet again why people don’t use electronic means for these processes. Perhaps I should append “in South Africa” or “in the Engineering community in South Africa”. An argument I hear quite often is that you need a handwritten signature for legal purposes. This is balony, as one can read in Act No. 25 of 2002: Electronic Communications and Transactions Act. A good summary of what the act means for digital signatures in SA can be found here, but I will give an even more summarised version: digital communication has exactly the same legal standing as hard-copy communication in South Africa. So there is no legal reason for people not to just e-mail one another and write their names at the bottom of the mail for non-contract communication just like any other time. When a legally binding contract is entered into, we need an “advanced digital signature”, which boils down to signatures using public key systems like S/MIME or PGP.
    So, why is no-one using these systems? I think the first is that most people still print out everything they want to keep, and digital signatures tend to be invisible. Printing out the document invalidates the security of a cryptographic signature and there is no formal way of ascertaining whether the document printed out was signed or not. Big problem for people who don’t use computerised filing systems. Unfortunately, this means they do something very bad: they trust faxed or printed out signatures. Many people are ok with a document as long as it looks as if it has been signed. This is ludicrously simple to cheat, as all the things that make a physical signature difficult to fake are not present on the fax. I am absolutely certain that I could trace a signature and have it look legit after faxing. I am even more certain (because I have done it) that a couple of minutes with a photo manipulation software gives me a signature I can put on any document I wish.
    Which brings me to the second reason no-one uses digital signatures: people think of computers primarily as an extension of the “real world”. The mental model encouraged by Office software vendors is that the electronic document is just like the real thing, only on the computer. This is great when the metaphor is solid, but many things that are used in computers just don’t have any physical equivalent, and these things become difficult to understand. This doesn’t mean they can’t be used (what’s the real-world equivalent of Facebook?), it just means you need to package it right. I reckon Adobe’s idea with a little ribbon that appears on a PDF when it’s been validated is almost there, but it’s not ubiquitous enough for people to recognise what it means, and I don’t think there is a good way of validating the document after it has been printed.
    I think there is a real gap in the market for a cryptographic signature that would work with mobile phones — a datamatrix in the corner that would match an image hash of the whole document. Lots of interesting implementation details there, but a good way to make hard copies trustable.
    So, what am I going to do to stop the madness of printing out documents, signing them and scanning them or faxing them? Pretty much what I’m doing already – affix a picture of my signature to the PDF and get on with my life. Legally, I could just as well have written my name, but it keeps people happy. I will also refer them to this blog entry and the law when they claim a legal necessity for hard-copy signatures.
    What do you think – when will the hard copies go away?