{"id":62,"date":"2011-12-29T14:14:00","date_gmt":"2011-12-29T14:14:00","guid":{"rendered":"https:\/\/sandrock.co.za\/carl\/2011\/12\/29\/the-fallacy-of-the-general-purpose-tool\/"},"modified":"2011-12-29T14:14:00","modified_gmt":"2011-12-29T14:14:00","slug":"the-fallacy-of-the-general-purpose-tool","status":"publish","type":"post","link":"https:\/\/sandrock.co.za\/carl\/2011\/12\/the-fallacy-of-the-general-purpose-tool\/","title":{"rendered":"The fallacy of the general purpose tool"},"content":{"rendered":"

I carry a Leatherman on my belt every day wherever I go. It has become a bit of an extension of my body, which I only really notice when I have to fly somewhere and I’m forced to take it off for the flight. Then I get to my destination and I can’t open the cable ties on my luggage. At the same time, I have a whole pinboard of tools up in my garage, which overlap somewhat with my Leatherman: a dedicated set of pliers, some dedicated screwdrivers and scissors and so on. This is because the is not as good as any of these dedicated tools.<\/p>\n

<\/div>\n
I think the same goes for programming languages, and therefore I have learned quite a few, just like I have stocked my pinboard with tools that do one thing very well. Many people find learning new languages onerous and will try to find the one language that does everything they want to do well. Their arguments against learning a new language often involves something like “but I can do that just fine in my language, and then I don’t have to learn a new language”. Here’s an example from real life that illustrates the advantages of special purpose languages.<\/div>\n
<\/div>\n
My father approaches me with this problem. He uses an accounting package which can import OFX files<\/a> (they’re an SGML\/XML format for financial records), but there is a slight problem. The ZAR needs to be changed to USD and the dates (which are tags of the form 20111201 – yes, no closing tag as it’s SGML) have to be changed from YYYYMMDD to YYYYDDMM. Now, turns out that Python has a nice OFX module. So does Java<\/a>. But the fastest way to make the changes to his files is probably sed:<\/div>\n
\n
sed -r -i -e 's\/ZAR\/USD\/' -e 's\/<DT(.*)>
([0-9]{4})([0-9]{2})([0-9]{2})\/<DT1>243\/' filename.ocx<\/dt><\/dt><\/pre>\n
This does the replacement in-place, is really fast and is quick enough to throw together.  This is the kind of thing that sed shines at.  In fact, it is exactly what it was designed to do, so it is unsurprising that it does it so well.<\/p><\/div>\n
<\/div>\n
To do the same thing in Python (even without using the OFX module) requires a bit more effort:<\/div>\n
<\/div>\n
<\/p>\n
import<\/span> os
import<\/span> re
from<\/span> tempfile import<\/span> mkstemp

filename =<\/span> \"test.txt\"<\/span>
patternStrings =<\/span> [<\/span>\"ZAR\"<\/span>,<\/span> r\"<TD([^>]+)>([0-9]{4})([0-9]{2})([0-9]{2})\"<\/span>]<\/span>
replacements =<\/span> [<\/span>\"USD\"<\/span>,<\/span> r\"<TD1>243\"<\/span>]<\/span>

# Compile patterns<\/span>
patterns =<\/span> [<\/span>re.<\/span>compile<\/span>(<\/span>p)<\/span> for<\/span> p in<\/span> patternStrings]<\/span>

# Create temporary file to hold outputs<\/span>
_,<\/span> tempfile =<\/span> mkstemp(<\/span>)<\/span>

# Process file<\/span>
outfile =<\/span> open<\/span>(<\/span>tempfile,<\/span> 'w'<\/span>)<\/span>
for<\/span> line in<\/span> open<\/span>(<\/span>filename)<\/span>:<\/span>
    for<\/span> pattern,<\/span> replacement in<\/span> zip(<\/span>patterns,<\/span> replacements)<\/span>:<\/span>
        line =<\/span> pattern.<\/span>sub(<\/span>replacement,<\/span> line)<\/span>
    outfile.<\/span>write(<\/span>line)<\/span>

outfile.<\/span>close(<\/span>)<\/span>

os.<\/span>rename(<\/span>tempfile,<\/span> filename)<\/span>
<\/pre>\n
<\/p>\n
Needless to say, the difference is even more pronounced in Java due to the large amount of boilerplate needed.<\/div>\n
\n
import<\/span> java<\/span>.<\/span>io<\/span>.<\/span>File<\/span>;<\/span>
import<\/span> java<\/span>.<\/span>io<\/span>.<\/span>FileReader<\/span>;<\/span>
import<\/span> java<\/span>.<\/span>io<\/span>.<\/span>FileWriter<\/span>;<\/span>
import<\/span> java<\/span>.<\/span>io<\/span>.<\/span>IOException<\/span>;<\/span>
import<\/span> java<\/span>.<\/span>util<\/span>.<\/span>Scanner<\/span>;<\/span>
import<\/span> java<\/span>.<\/span>util<\/span>.<\/span>regex<\/span>.<\/span>Pattern<\/span>;<\/span>

public<\/span> class<\/span> Fixer {<\/span>
    static<\/span> File<\/span> infile =<\/span> new<\/span> File<\/span>(<\/span>\"test.txt\"<\/span>)<\/span>;<\/span>
    static<\/span> String<\/span>[<\/span>]<\/span> patternStrings =<\/span> {<\/span>\"ZAR\"<\/span>,<\/span> \"<TD([^>]*)>([0-9]{4})([0-9]{2})([0-9]{2})\"<\/span>}<\/span>;<\/span>
    static<\/span> String<\/span>[<\/span>]<\/span> replacementStrings =<\/span> {<\/span>\"USD\"<\/span>,<\/span> \"<TD$1>$2$4$3\"<\/span>}<\/span>;<\/span>

    public<\/span> static<\/span> void<\/span> main(<\/span>String<\/span>[<\/span>]<\/span> args)<\/span> throws<\/span> IOException<\/span> {<\/span>
        Pattern[<\/span>]<\/span> patterns =<\/span> new<\/span> Pattern[<\/span>patternStrings.<\/span>length]<\/span>;<\/span>
        Scanner in =<\/span> new<\/span> Scanner(<\/span>new<\/span> FileReader<\/span>(<\/span>infile)<\/span>)<\/span>;<\/span>
        
        \/\/ Compile patterns<\/span>
        for<\/span> (<\/span>int<\/span> i=<\/span>0<\/span>;<\/span> i<<\/span>patternStrings.<\/span>length;<\/span> i+<\/span>+<\/span>)<\/span> {<\/span>
            patterns[<\/span>i]<\/span> =<\/span> Pattern.<\/span>compile(<\/span>patternStrings[<\/span>i]<\/span>)<\/span>;<\/span>
        }<\/span>
        
        \/\/ Create temporary file to hold outputs<\/span>
        File<\/span> tempfile =<\/span> File<\/span>.<\/span>createTempFile(<\/span>\"tmp\"<\/span>,<\/span> \"tmp\"<\/span>)<\/span>;<\/span>

        \/\/ Process file<\/span>
        FileWriter<\/span> outfile =<\/span> new<\/span> FileWriter<\/span>(<\/span>tempfile)<\/span>;<\/span>
        while<\/span> (<\/span>in.<\/span>hasNextLine(<\/span>)<\/span>)<\/span> {<\/span>
            String<\/span> line =<\/span> in.<\/span>nextLine(<\/span>)<\/span>;<\/span>
            for<\/span> (<\/span>int<\/span> i=<\/span>0<\/span>;<\/span> i <<\/span> patterns.<\/span>length;<\/span> i+<\/span>+<\/span>)<\/span>
                line =<\/span> patterns[<\/span>i]<\/span>.<\/span>matcher(<\/span>line)<\/span>.<\/span>replaceAll(<\/span>replacementStrings[<\/span>i]<\/span>)<\/span>;<\/span>
            
            outfile.<\/span>write(<\/span>line +<\/span> System<\/span>.<\/span>getProperty(<\/span>\"line.separator\"<\/span>)<\/span>)<\/span>;<\/span>
        }<\/span>        
        outfile.<\/span>close(<\/span>)<\/span>;<\/span>
        
        \/\/ move temp file back to filename<\/span>
        tempfile.<\/span>renameTo(<\/span>infile)<\/span>;<\/span>
    }<\/span>
}<\/span>
<\/pre>\n<\/div>\n
Now, at this point someone is bound to say “but wait, people don’t write desktop apps in sed!”.  But that’s quite the point – Java was designed with programming in the large in mind, but it is really tedious for short programs.  In many cases, learning a small domain specific language and using it to solve your small problem is faster than learning how to do the same thing in your “one language”.<\/div>\n
<\/div>\n
I wish I had more time to put examples together, but this situation just presented itself and got me thinking.<\/div>\n","protected":false},"excerpt":{"rendered":"
I carry a Leatherman on my belt every day wherever I go. It has become a bit of an extension of my body, which I only really notice when I have to fly somewhere and I’m forced to take it off for the flight. Then I get to my destination and I can’t open the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[3],"tags":[187],"class_list":["post-62","post","type-post","status-publish","format-standard","hentry","category-programming","tag-ramble"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/posts\/62","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/comments?post=62"}],"version-history":[{"count":0,"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/posts\/62\/revisions"}],"wp:attachment":[{"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/media?parent=62"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/categories?post=62"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sandrock.co.za\/carl\/wp-json\/wp\/v2\/tags?post=62"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}