architecture - Multithreading: Each system on a different thread or a thread pool?

I'm building a fairly involved game using OpenGL and C++. I've been thinking about how to implement multithreading, and the two options are:

1) Each system (Graphics, Audio, Physics, et cetera) gets their own dedicated thread that they run off of.

2) Implement a thread pool, that picks up a jobs from a queue (priority queue), executes it and gets re-inserted into the pool to wait for new available job.

I was wondering which one is generally considered better: in terms of performance, maintenance, scalability, code smell, et cetera.

Answer

One important concern when deciding units of parallelization is usually to avoid sharing data between threads. Multiple threads operating on the same data is always hairy, because:

• if those accesses are not properly synchronized, you can encounter bugs triggered by race conditions which appear seemingly randomly and are extremely difficult to reproduce and analyze.


• if they are properly synchronized, those synchronization techniques can often lead to performance problems and even deadlocks (two threads are locking different data structures and waiting for the other thread to release theirs)

So you generally want threads to communicate with each other as little as possible.

Unfortunately, systems often operate on data delivered by other systems. A good example is the graphics system which renders the game state which gets constantly changed by the physics system. That means that access to the game state must be synchronized. If you render the game state while the physics system changes it, you might occasionally encounter really weird artifacts on the screen. For example, let's say the rendering system wants to render a character who is swinging a sword. The character and their sword are implemented as separate entities by the physics system. At the moment where the rendering system renders both entities, the physics system might have already updated the position of the character but not yet the position of the sword. So you occasionally see the sword getting rendered outside of the fist of the character.

There are two solutions to this synchronization problem, and neither is good. The first is to have one system wait until the other is finished. But if you run your systems in sequence anyway, then you gain nothing by multithreading and can just as well run them on the same thread. The second is to have two copies of the game state. While the rendering engine renders the old state, the physics engine calculates the next state. Then you have a synchronization point, switch out the two states, and proceed with the next frame. But this doubles the amount of RAM you need for your game state.

Another problem with using one thread per system is that the resource consumption between systems is often very unequal. There is little benefit in having 4 threads when 3 of them stall most of the time while only one of them actually maxes out its CPU core. You ideally want to distribute your load on all CPU cores equally.

For these reasons, you might rather look for units of parallelization within your systems.

If you need to run an update on 2000 objects, and those updates do not depend on the states of other objects, then you can use 4 threads and have each one process a set of 500 objects. In the idealized case, this cuts the processing time of that system down to one quarter.

Keep in mind that creating threads is expensive, so you should always use a thread-pool which maintains a number of long-living threads and passes units of work to these threads. I have also seen thread-pool implementations which allow you to pass a job together with a list of other jobs it depends on. The thread-pool will then wait with enqueueing that job until all the dependencies have finished. This makes it a lot safer to implement multithreading across system boundaries if those systems have dependencies on each other.