Parallel processing¶
ParallelPipeline
distributes the work between
multiple processes.
This can speed up processing massively in CPU-bound applications.
-
class
morphocut.parallel.
ParallelPipeline
(num_workers=None, queue_size=2, multiprocessing_context=None, parent=None)[source]¶ Parallel processing of the stream in multiple processes.
- Parameters
num_workers (int, optional) – Number of worker processes. Default: Number of CPUs in the system.
queue_size (int, optional) – Upperbound limit on the number of items that can be placed in the input queue of each worker. If queue_size is less than or equal to zero, the queue size is infinite.
multiprocessing_context (optional) – Result of
multiprocessing.get_context()
.parent (
Pipeline
) – The parent pipeline.
Note
ParallelPipeline
creates distinct copies of its nodes in each worker thread that are not accessible from the main thread.Example
with Pipeline() as pipeline: # Regular sequential processing ... with ParallelPipeline(): # Parallelized processing in this block, # work is distributed between all cores. ... pipeline.run()