Category: Uncategorized

Where “added value” comes from

One of the things that have often bothered me is the idea of added value. I’ll admit to not having read enough market theory, but it’s been pretty easy to understand how refining an ore leads to a product with higher value due to its increased utility. For bulk chemicals and products, the market is very efficient and bulk product prices tend to be marginally above production cost. Where things become weird is with services and branded products. Many of us have thought about the irrationality of paying twice as much for the same shirt simply because it bears a particular logo.

Then you start thinking about counterfeiting, especially about the concept of originality. Let’s say you’re willing to pay R100000 for an original Pierneef. If a person approaches you with a perfect copy of this painting, why would you not pay the same amount for the copy? The answer is that you are attaching some value to originality. I suppose it should be said that market theorists would probably explain this by saying perfect copies deminish the value of the painting by increasing the supply.

While driving to fetch some pizza from the best pizza place in Pretoria, it occurred to me that cognitive biases could help to explain much of the strategies associated with good business beyond the bulk manufacturing sense engineers tend to think of. I’m still working on the details, but briefly, people do not generally have access to perfect rationality. The biases listed in that wikipedia page all show how our perceptions and estimation of the odds are actually a bit off. When you can craft a strategy that bends as many of these biases in your favour as possible, you will end up with a product that will appear to be worth more than its utility to most people, and you will be able to get a lot more money for it. I’ll be thinking about this more and posting, but leave a comment if you think I’m on to something (or even better, if you can point me to alternative explanations).

2010-01-12

PySUNDIALS problems

I’m still installing JModelica. I am now in the annoying “nothing broke during install, but somethings still fail when I use them” phase. I am using the instructions here, and using the example to test if things are working because JModelica doesn’t seem to have a test suite. This is what is happening now:


In [1]: import jmodelica.examples.cstr as cstr
/Library/Python/2.6/site-packages/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated
import sets
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)

/Users/alchemyst/ in ()

/usr/local/Python/jmodelica/examples/cstr.py in ()
  9 from jmodelica.initialization.ipopt import NLPInitialization
 10 from jmodelica.initialization.ipopt import InitializationOptimizer
---> 11 from jmodelica.simulation.sundials import TrajectoryLinearInterpolation
 12 from jmodelica.simulation.sundials import SundialsDAESimulator
 13 from jmodelica.optimization import ipopt

/usr/local/Python/jmodelica/simulation/sundials.py in ()
 14
 15 try:
---> 16     from pysundials import cvodes
 17     from pysundials import ida
 18     from pysundials import nvecserial

/Library/Python/2.6/site-packages/pysundials/cvodes.py in ()
 39 import ctypes
 40 import sundials_core
---> 41 import nvecserial
 42
 43 realtype = nvecserial.realtype

/Library/Python/2.6/site-packages/pysundials/nvecserial.py in ()
 56 PVector = ctypes.POINTER(_NVector)
 57
---> 58 nvecserial.N_VNew_Serial.restype = PVector
 59
 60 nvecserial.N_VLinearSum_Serial.argtypes = [realtype, PVector, realtype, PVector, PVector]

/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.pyc in __getattr__(self, name)
356         if name.startswith('__') and name.endswith('__'):
357             raise AttributeError(name)
--> 358         func = self.__getitem__(name)
359         setattr(self, name, func)
360         return func

/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.pyc in __getitem__(self, name_or_ordinal)
361
362     def __getitem__(self, name_or_ordinal):
--> 363         func = self._FuncPtr((name_or_ordinal, self))
364         if not isinstance(name_or_ordinal, (int, long)):
365             func.__name__ = name_or_ordinal

AttributeError: dlsym(0x102e957d0, N_VNew_Serial): symbol not found

It’s clear something is wrong with pysundials. On closer inspection, it seems that the dynamic library isn’t loading correctly. A part of the problem is explained by this:


$ nm /usr/local/lib/libsundials*dylib

/usr/local/lib/libsundials_cvode.1.0.0.dylib:
0000000000000000 s __mh_dylib_header
              U dyld_stub_binder

And many more like it. It seems like the .dylib built by the sundials build process doesn’t include the correct names. After much browsing and learning about how .dylibs work, I realise that this can be solved with a simple change to the configure script call (typical – it was actually documented: use –enable-shared in the build!):


FFLAGS="-arch x86_64" ./configure --prefix=/usr/local --enable-examples --enable-shared
make
sudo make install

Now nm shows the items in the dylib.

I know sundials is working right because the examples compile and run fine. Now for pysundials:


In [2]: from pysundials import cvode
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)

/Users/alchemyst/tmp/pysundials-2.3.0-rc2/examples/cvode/serial/ in ()

/Library/Python/2.6/site-packages/pysundials/cvode.py in ()
   350         if ret  352 cvode.CVodeSetFdata.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
   353 cvode.CVodeSetFdata.restype = ctypes.c_int
   354

/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.pyc in __getattr__(self, name)
   356         if name.startswith('__') and name.endswith('__'):
   357             raise AttributeError(name)
--> 358         func = self.__getitem__(name)
   359         setattr(self, name, func)
   360         return func

/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.pyc in __getitem__(self, name_or_ordinal)
   361
   362     def __getitem__(self, name_or_ordinal):
--> 363         func = self._FuncPtr((name_or_ordinal, self))
   364         if not isinstance(name_or_ordinal, (int, long)):
   365             func.__name__ = name_or_ordinal

AttributeError: dlsym(0x1014d04f0, CVodeSetFdata): symbol not found

And the original thing I was trying to do:


In [1]: import jmodelica.examples.cstr as cstr
/Library/Python/2.6/site-packages/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated
 import sets
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)

/Users/alchemyst/tmp/pysundials-2.3.0-rc2/examples/cvode/serial/ in ()

/usr/local/Python/jmodelica/examples/cstr.py in ()
     9 from jmodelica.initialization.ipopt import NLPInitialization
    10 from jmodelica.initialization.ipopt import InitializationOptimizer
---> 11 from jmodelica.simulation.sundials import TrajectoryLinearInterpolation
    12 from jmodelica.simulation.sundials import SundialsDAESimulator
    13 from jmodelica.optimization import ipopt

/usr/local/Python/jmodelica/simulation/sundials.py in ()
    14
    15 try:
---> 16     from pysundials import cvodes
    17     from pysundials import ida
    18     from pysundials import nvecserial

/Library/Python/2.6/site-packages/pysundials/cvodes.py in ()
    44 NVector = nvecserial.NVector
    45
---> 46 cvodes = sundials_core.loadlib("cvodes")
    47
    48 #Linear Multistep method constants


/Library/Python/2.6/site-packages/pysundials/sundials_core.pyc in loadlib(libname)
    62                 lib = ctypes.CDLL(libpaths[libname])
    63         except OSError, e:
---> 64                 raise OSError("%snCannot load shared library %s. Please check you config file and ensure the paths to the shared libraries are correct."%(e, libpaths[libname]))
    65         return lib
    66

OSError: dlopen(/usr/local/libsundials_cvodes.dylib, 6): image not found
Cannot load shared library /usr/local/libsundials_cvodes.dylib. Please check you config file and ensure the paths to the shared libraries are correct.

So the error has changed to it not finding the library. Something must be screwy with paths. Frustrating afternoon.

2009-12-21

Building JModelica on Mac OS X Snow Leopard

I am again working on my PhD and I need to get my development environment sorted out. I have almost had it with OpenModelica, because it is so hard to get going on my Mac. I have tried to contribute code to the build process, but the configure script still bombs out based on environment variables that have to be set by hand and so forth. I have therefore decided to try JModelica instead. I duly followed the start of the install instructions, including grabbing a version of the repository using svn (I was not warned that the repo is about 130 MiB, though!).

Installing Ipopt

Ipopt is an interior point solver, part of the COIN-OR project. JModelica includes some nifty optimisation functionality via Optimica so it is quite good to have Ipopt (even though it is not required). So I download the latest Ipopt tarball and now I’m faced with more dependancies: BLAS and LAPACK. After some toying around with these packages, I learned the following:

The Mac dev tools inclode cBLAS and cLAPACK as part of the vecLib framework (something to remember). However, these don’t play well with gfortran on 10.6
You can download and build them yourself with little effort, but most build tools assume you are going to use -lblas to access it, which is a bit harder to override.
Ipopt actually has scripts for downloading these for compiling Ipopt, as documented here (although, crucially, not in the install documentation I was following in the distribution)
The Ipopt instructions advise using the HSL libraries, but the process you have to go through to get them takes time, I’ve done it, but I still don’t have access, so I’m just going for Mumps instead.

So, after wasting much time, all I really needed to do was


cd ThirdParty
for lib in Blas Lapack ASL Metis Mumps; do
cd $lib;
./get.$lib
cd ..
done
cd ..
./configure --prefix=/usr/local # to get stuff where I want it
make

Well, after that the configure script worked, but there was a build error (or a couple of them, all with the same form):


lpBlas.cpp:211: error: ‘dtrsm’ was not declared in this scope
IpBlas.cpp:211: error: ‘DTRSM’ was not declared in this scope
IpBlas.cpp:211: error: ‘F77_FUNC’ cannot be used as a function

Looking at the source, it was clear that F77_FUNC was supposed to be a macro that helps with fortran execution or something. Further investigation (grep -R “#define F77_FUNC”) shows that it’s supposed to be a mangling thing. Even further investigation shows that one of the configure messages that wizzed by is “WARNING: unknown Fortran name-mangling scheme”). The configure script then just silently doesn’t define F77_FUNC, leading to these errors. After scrutinising

Ipopt/config.log

I noticed that ld was complaining about the archetecture. I modify my call to configure thus:

FFLAGS="-arch=x86_64"

Now make runs! Make test succeeds and I can do

sudo make install

and get to the main game.

Building JModelica

So, back to the original build instructions. I modify the configure again to point to the right places.


cd JModelica
mkdir build
cd build
../configure --with-ipopt=/usr/local/ --prefix=/usr/local
make

Three minutes later, success! Now do build docs (which requires doxygen, obtained via fink and graphviz, using the Mac install package), which takes ages.

Python support

Last step is getting Python support, one of the other cool things about JModelica. The docs list the following packages:

Package	URL	Steps I took to install
JPype	http://jpype.sourceforge.net/	Download, run sudo python setup.py install
lxml	http://codespeak.net/lxml/	sudo easy_install lxml
NumPy 1.2.0	http://numpy.scipy.org/	Already installed
Scipy 0.7	http://www.scipy.org/	Already installed
PySundials 2.3	http://sourceforge.net/projects/pysundials/	Install SUNDIALS, change reference in src/sundials_core_aux.c from sundials/sundials_smalldense.h to sundials/sundials_dense.h. sudo python setup.py install
pyreadline =1.5	http://ipython.scipy.org/dist/	sudo easy_install pyreadline
Matplotlib	http://matplotlib.sourceforge.net/	already installed
Nose	http://code.google.com/p/python-nose/	sudo easy_install nose

And now it still doesn’t work, because Matplotlib is causing segfaults. I guess it’s time to call it a day — more in getting Matplotlib to go tomorrow.

2009-12-16

How can GUI programming be so hard (for me)?
I do a lot of casual programming every day, to solve problems that I have at that moment. This mostly involves a quick one-liner in bash. Examples of this: finding out which of my students are repeating the subject by comparing this year’s class list to last year’s, figuring out which episodes of a series I still need to download, and so on. When I find myself doing the same thing more than once, that one-liner gets pushed into a script file that I can call at a later time. The shell allows me to pipe the output from one of these scripts into another or prepare input in many different ways, filtering by multiple criteria and allowing me to select files easily.
From time to time, I get the urge to convert one or more of these things into a nice GUI. To paraphrase a quote, now I have two or more problems (Apologies to Jamie Zawinski). First I need to choose a GUI toolkit that supports the platforms I use every day: Mac OS on the laptop and Linux on my desktop at work. All my scripting stuff ‘just works’. I use a blend of (in order of size of job) grep, sed, awk, bash and python, all of which are transparently available on both platforms and have never given me a day of trouble (except perhaps python, but that’s a different story). Enter the GUI and things get seriously complicated. Python ships with Tkinter, which is universally acknowledged to be pretty bad. It also has bindings for GTK, Qt and wxWidgets. I have not succeeded in getting any of these to work on both my computers at the same time.

A prime motivation for GUIfying these utility scripts is so that other people can use them more easily. Unfortunately that also means that this all has to work on Windows as well. This consideration, along with the fact that getting a fully functional version of Python running on a windows box takes time if not that much effort, has led me to consider Java for my GUI stuff.
So, now the problem — I remember doing a lot of GUI programming with Delphi back in the day (when I was still at school). I wrote several programs that did useful things and didn’t think it was all that hard. But now that I have mastered the command line, everything I need to do seems so hard in the GUI environment. Java has no standard GUI editor, so I have kind of drifted toward Netbeans for the nice GUI layout tool that it has. No matter how hard I try, though, I can’t find the motivation to stick with figuring out GUI programming for the hours that it takes me to get a useful result; where useful is defined as something that actually does something instead of just popping up a window with buttons and all. I get that all of that functionality (having a clickable widget appear on screen, being able to close the window etc) comes at a cost, but figuring out the details takes be ages, and seems entirely orthogonal to the problem I am actually trying to solve.

For instance, here’s the python code for a program that gives me a list of students in common between a couple of files:
```
import sys, csv
classlists = dict((f, [r for i, r in enumerate(csv.reader(file(f))) if i > 1]) for f in sys.argv[1:])
numbers = [set(student[0] for student in classlist) for classlist in classlists.itervalues()]
uniqnumbers = reduce(set.intersection, numbers)
for n in sorted(uniqnumbers): 
     print n
```
How long would that have taken to GUIfy? Frustratingly long.
2009-11-02