Socket Efficiency

From KallestadWiki

I've been looking into ways to implement various KDF features, and the bottom line is this - IPC is absolutely necessary. From various searches, it doesn't seem like there has been a push forward for an easy to use highly efficient IPc mechanism. The problem is that there are a lot of ways to slice and dice things, and socket programming is something that most people don't want to delve into if they don't have to.

Threads, Processes, and Microthreads

I'm a huge fan of Perl, but one aspect of Perl is lacking and has been for quite some time - threading. Perl does have thread support, but you have to enable threads at compile time and a lot of useful CPAN modules are simply not thread safe. Despite that, I looked into a lot of perl mechanisms from threading to POE.

I'm no fan of Java, so I started looking into other languages with easy thread support, and I found Python. I grabbed a book and taught myself the basics of Python a week or two ago. From there, I found Stackless Python, and distracted myself with other languages like Haskell and Erlang. I keep coming back to Python/Stackless as the best solution for this problem because of Microthreads.

Threads are basically sub-processes for a process. They are similar to child processes, but generally have much less overhead (although in Perl, threads have much more overhead than you would think). Microthreads are similar to threads, but rather than using an OS level threading library to separate tasks, an internal library of the parent process manages the task queue. It's the most efficient "threading mechanism" - you can spawn several thousand microthreads with minimal overhead, but trying to do that with threads or child processes is a bit of a bear.

IO Bound Mechanisms

Microthreads aren't the ideal solution for everything, but