Multiprocessing is a powerful tool for parallel computing in Python. It allows you to run multiple processes concurrently, which can significantly speed up your programs, especially when dealing with CPU-bound tasks.

Getting Started

To use multiprocessing in Python, you'll need to import the multiprocessing module. Here's a simple example of how to use it:

from multiprocessing import Process

def worker():
    print('Worker process started')

if __name__ == '__main__':
    p = Process(target=worker)
    p.start()
    p.join()

In this example, we define a worker function that will be executed in a separate process. We then create a Process object with the target argument set to our worker function. We start the process with p.start() and wait for it to finish with p.join().

Process Pool

The ProcessPoolExecutor is a high-level interface for asynchronously executing callables.

from multiprocessing import Pool

def square(x):
    return x * x

if __name__ == '__main__':
    with Pool(4) as p:
        result = p.map(square, range(10))
        print(result)

In this example, we use a ProcessPoolExecutor to create a pool of 4 worker processes. We then use the map method to apply the square function to each element in the range of 10.

Sharing Data

When using multiprocessing, it's important to be aware of how data is shared between processes. The multiprocessing module provides several mechanisms for sharing data, such as Queue, Pipe, and Value.

Here's an example of using a Queue to share data between processes:

from multiprocessing import Process, Queue

def worker(q):
    q.put([42, None, 'hello'])

if __name__ == '__main__':
    q = Queue()
    p = Process(target=worker, args=(q,))
    p.start()
    print(q.get())  # Output: [42, None, 'hello']
    p.join()

In this example, we use a Queue to share data between the main process and the worker process.

Conclusion

Multiprocessing can be a powerful tool for speeding up your Python programs. By understanding how to use the multiprocessing module, you can take advantage of multiple CPU cores and improve the performance of your applications.

For more information, check out our advanced multiprocessing tutorial.

Multiprocessing Example