Parallel processing

Example of parallel runs

parallel calculating the squares of numbers 1, 2, 3, 4

using 4 processors

#!/usr/bin/env python

import multiprocessing

def parallel_runs():

pool = multiprocessing.Pool(processes=4)

input_list = [1, 2, 3, 4]

result_list = pool.map(parfunc, input_list) # 'map' waits until the result is ready

print(result_list)

# square-function to run in parallel

def parfunc(x):

return x**2

if __name__ == '__main__':

parallel_runs()

pool.map get's as input a function and only one iterable argument; output is a list of the corresponding results. To run in parallel function with multiple arguments, partial can be used to reduce the number of arguments to the one that is replaced during parallel processing.

https://docs.python.org/3.4/library/multiprocessing.html

get number of CPU's

try:

numProcessors = multiprocessing.cpu_count()

except NotImplementedError: # win32 environment variable NUMBER_OF_PROCESSORS not defined

print('Cannot detect number of CPUs')

numProcessors = 1