ImperialViolet

People who read this via RSS probably don't even notice my del.icio.us feed at the top. (The RSS for that is here).

Recently, I have mosting been reading about concurrency. This isn't new - I've been thinking about it for ages (in fact, some people wish I would shutup about it sometimes). But previously my thinking has been about how to manage complex, stateful servers without lock-hell or inversion-of-control. I've now written a Python module (twistless) which does this very nicely. (Source availible upon prodding, but it needs polishing before a public release)

But that (and my other code, like Neuroses) is based around syncthreading which manages to avoid locks by only really have a single thread. The complexity is vanquished, but it doesn't actually use an SMP machine.

This was in the hope that `it will always be fast enough tomorrow'. The thinking was that doubling the speed of your library wasn't worth hand-crafted inverted control code because the CPUs would catch up soon enough anyway. But anyone who has been watching cpu speeds will have noticed that this is no longer true - we've hit the ceiling and CPUs are now growing outwards (multiple cores) rather than upwards (ever faster clock speeds).

Herb Sutter has a good text in DDJ about this.

Doing `true' concurrency without lock-hell is really tough. I don't know how to do it. (Maybe you're smart enough to write complex, multi-threaded code with fine-grained, deadlock and priority-inversion free locking - but I'm not, and nor are a lot of people.) Erlang does quite well in this area with very real results (working ATM switches etc), but it's based around the idea of message-passing threads, which isn't my cup of tea.

The most exciting thing I've read about this is Composable memory transactions which uses a begin-commit style for shared memory (with retry and orElse - see the paper). Unfortunately, it's built on Haskell. (A language which calls IO, exceptions and external calls the "awkward squad" was built on the wrong foundations I'm afraid. Having said that - here's a great introduction on how to do them in Haskell).

Unfortunately, I can't see that the ideas in that paper fit neatly into any current language. (The authors had to hack GHC quite a lot - and even then it only runs syncthreads in a single kernel-thread!) so maybe a new language is in order to play about with them.