I am building a node-based drag-and-drop editor, where each node represents one action (for example, read this file, or sort this data, etc.) Outputs and inputs of nodes can be connected.
One of the features I’d like to implement is automatic parallelization, so that if a path branches off I can automatically begin a thread to handle each branch. I’m concerned about a few issues, however:
- If a path branches off, but then later joins back together, I will need to synchronize them somehow
- If there are multiple start-nodes (where execution begins), their paths will have to be managed separately and then possibly dynamically joined/merged
- I want to limit how many threads are created so that I don’t suddenly have 20 threads deadlocked
Essentially, I’d like to know if any strategies for doing something like this exist (not looking for code necessarily; just theory). Could scheduling algorithms help?
Thanks for your advice! I look forward to hearing your suggestions.
Note: I’m using C# 3.5, so none of the fun parallel-tasking abilities are available to me. If necessary, I will make the switch to C# 4.0, but I’d like to avoid this.
The Task Parallel Library might be exactly what you’re looking for.
I imagine your node-based drag-and-drop editor to look like this:
Every node is essentially a Task. A Task can be anything — read a file from disk, download some data from the web, or compute anything.
When a Task has finished, it can ContinueWith one or more other Tasks, passing the result of the old Task to the new Tasks.
A Task can also consist of waiting for multiple Tasks to finish. WhenAll these Tasks have finished, this Task can continue with another Task, passing the result of all Tasks to the new task.
The TPL will schedule all these Tasks on a Thread Pool, so Threads can be reused and each Task doesn’t need to have its own Thread. The TPL will find the optimal number of Threads for the system it is running on.
The Visual Studio Async CTP adds native language support for asynchronous operations to C#, which makes working with Tasks really easy and fun.
With the TPL it is just a matter of creating Tasks and composing them according to the node layout.
Complete program code for the above example: