Billiard

So, how does the whole thing work?

Every Python program is executed in a Process, which is a new instance of the Python interpreter. The prefork pool, ultimately, is an implementation of the billiard pool. Billiard is a fork of Python 2.7 multiprocessing pool and was originally part of the Celery code base. [verify!]

Starting with Celery version xxx, billiard was carved out of Celery into a project of its own. Billiard is used in and is a dependency for Celery and is maintained by the Celery team.

Celery uses billiard instead of Python’s multiprocessing pool because there are features missing in Python’s multiprocessing pool to provide long running services.

How does the worker parent process communicate with the forks? This is done via multiprocessing pipes. What is a Pipe? In multiprocessing, a pipe is a connection between two processes in Python. It is used to send data from one process which is received by another process.