Managing scientific data and interchange with industry

I spoke at a recent symposium on the difficulties in managing data interchange with industry.

Just for some context, “industry” here stands for personnel involved in process control and modelling activities. My direct experience is in the petrochemical, mining and paper and pulp industries. I also show an example of data from a piece of analytical equipment.

I’ve uploaded the source on GitHub here, and you can see the notebook without downloading it by using NBViewer.

I’ve spoken about the pattern of intermediates on this blog before (in a slightly different context).

Notably absent from this discussion are any kind of database, since my experience is that if you can’t e-mail it, it might as well not exist when talking to the kinds of people I work with. I’ve used sqlite quite extensively myself, but I have found most people want something they can double-click and just have it work.

If you have different experiences, I’m always eager to learn. I am specifically interested in how people who use client-server databases for their data handle the problem of interchange.