Python Multiprocessing Tutorial 🚀

Welcome to our comprehensive guide on Python Multiprocessing! In this tutorial, we'll dive deep into the world of concurrent programming and learn how to leverage multiple CPU cores to execute tasks simultaneously. Let's get started!

🎯 Understanding Multiprocessing

In simple terms, Multiprocessing is a Python library that allows us to create multiple processes within a single Python script. Each process runs parallelly, independently, and can execute CPU-bound tasks concurrently.

📝 Note:

Processes are independent, meaning they have separate memory spaces.
They communicate using synchronization primitives like Queue, Pipe, or Semaphore.

💡 Pro Tip:

Multiprocessing can significantly improve the performance of CPU-bound tasks by making efficient use of multiple CPU cores.

✅ Creating a Process

To create a new process, we use the multiprocessing.Process class. Here's a simple example:

python

from multiprocessing import Process

def print_numbers(prefix):
    for i in range(5):
        print(f'{prefix}: {i}')

if __name__ == '__main__':
    proc1 = Process(target=print_numbers, args=('Task1',))
    proc2 = Process(target=print_numbers, args=('Task2',))

    proc1.start()
    proc2.start()

    proc1.join()
    proc2.join()

In this example, we create two processes that print numbers from 0 to 4 with a prefix "Task1" and "Task2" respectively. The join() method ensures that the parent process waits for child processes to complete before exiting.

💡 Pro Tip:

Always use if __name__ == '__main__': to ensure that the code runs only when the script is executed directly and not when it's imported as a module.

🎯 Passing Arguments

We can pass arguments to processes using the args parameter. The arguments are passed as a tuple:

python

def print_numbers(prefix, limit):
    for i in range(limit):
        print(f'{prefix}: {i}')

if __name__ == '__main__':
    proc1 = Process(target=print_numbers, args=('Task1', 5))
    proc2 = Process(target=print_numbers, args=('Task2', 10))

    proc1.start()
    proc2.start()

    proc1.join()
    proc2.join()

In this example, we pass two arguments to each process: the prefix and the limit.

🎯 Sharing Data Between Processes

Since processes have separate memory spaces, they can't directly access each other's variables. To share data between processes, we use multiprocessing.Manager. Here's an example:

python

from multiprocessing import Manager, Process

def update_counter(counter, limit):
    for _ in range(limit):
        counter.value += 1

def print_counter(counter):
    print(f'Counter: {counter.value}')

if __name__ == '__main__':
    manager = Manager()
    counter = manager.Value('i', 0)
    proc1 = Process(target=update_counter, args=(counter, 5000))
    proc2 = Process(target=print_counter, args=(counter,))

    proc1.start()
    proc2.start()

    proc1.join()
    proc2.join()

In this example, we use a Value object from multiprocessing.Manager to share a counter between two processes.

🎯 Quiz

That's it for this tutorial on Python Multiprocessing! We hope you found it helpful and informative. Stay tuned for more tutorials on Python, and keep coding! 💻🎉