In computer science, in the operating systems, respectively their design and their internal, there are:
Producer-consumer problem
Readers-Writers
Limited Buffer
The software architecture in turn teaches the concept of pipes and filters.
Indication:
You have a class of God with over 100 functions. Each is called with an argument entry (message) and returns an output (transformed message).
Process:
You divide it into 100 classes. Each is a producer or an independent consumer, or both, a producer-consumer at the same time. Communication between them is via a queue (publication subscribe).
How do you solve the remaining challenge, the challenge of load balancing? The producer generates 100 msg / s and the consumer accepts 50 msg / s (suppose the calculation is linked, not linked to the I / O. More threads calculate more).
A slow consumer is detected by his waiting queue, which means that the messages accumulate. You have a multicore processor and, to begin with, the consumer is a single thread.
Comment or answer how you can automatically scale a slow consumer from one thread (computation-related) to multiple threads (or reduce the number of threads of the same consumer from one thread to another). a single thread when the queue is not full enough to use all the threads).