TEAM.Commons: Threading - part 1

This is the 4th of several posts about TEAM.Common, a set of functionality I use in every project and that I'd like to share. The index is here: http://rodolfograve.blogspot.com/2010/09/teamcommons-introduction.html

(Multi)Threading is a complex, error prone task. You must take a lot of details into account everytime you're going to do it. On the other side, it's a very powerful tool and refusing to use it is not a valid solution. At least not if you want to do some large processings or interactive UIs.

AJAX (web), BackgroundWorker and Dispatcher (desktop), have solved the interactive UIs part (at least I feel comfortable enough with those these days), but I didn't feel the same about large processings that could be parallelized.

At some point I got tired of avoiding multithreading until there was no other option and got my self to implement some abstractions that would allow me and my team to implement parallel processes really, really quickly, with industry level robustness.

The result is TEAM.Commons.Threading. Next thing is an example so that you can decide if you like it and want to go on reading this post, or stop right here.

// Create the list of consumers... we are creating 5 consumers for this example
var consumers = Enumerable.Range(1, 5).Select(x =>
        new DelegatedConsumerWorker<string>(
            "Consumer" + x.ToString(),
            TimeSpan.FromSeconds(1),
            2,
            item => Console.WriteLine("Processed: " + item)
        )).ToArray();

// Create a producer based on an IEnumerable of strings from "Item-01" to "Item-20".
var producer = new EnumeratorProducer<string>(Enumerable.Range(1, 20).Select(x => "Item-" + x.ToString("00")));

// Create the MainProcess... all the internal logic for parallelization is here.
ProducerConsumersProcess<string> p = 
    new ProducerConsumersProcess<string>(
        producer,
        TimeSpan.FromMilliseconds(800),
        consumers,
        3);

// Start the process. This call blocks until the parallel process fails or ends.
p.Start();

Console.WriteLine("Process finished!");

Output is:

Processed: Item-01
Processed: Item-02
Processed: Item-03
Processed: Item-04
Processed: Item-05
Processed: Item-06
Processed: Item-08
Processed: Item-09
Processed: Item-07
Processed: Item-11
Processed: Item-12
Processed: Item-10
Processed: Item-14
Processed: Item-16
Processed: Item-17
Processed: Item-13
Processed: Item-15
Processed: Item-18
Processed: Item-19
Processed: Item-20
Process finished!

If you run the code (TEAM.Commons.Threading.SampleConsole) you'll see how the execution pauses at some points. That's because the buffer gets emptied and the consumers are waiting for the producer to produce more items. In the real world you should adjust the parameters to avoid these pauses: buffer size, producer pause time when buffer is full, consumers pause time when buffer is empty.

Next post will be about the details of TEAM.Commons.Threading.

Remember, you can get all this code for free at bitbucket: http://bitbucket.org/rodolfograve/team.commons/overview

Check it out to get ideas, or simply use it as it is. It's working out for me and my team.