Example of parallel runs (pool.map)
parallel calculating the squares of numbers 1, 2, 3, 4
using 4 processors
#!/usr/bin/env python
import multiprocessing
def parallel_runs():
pool = multiprocessing.Pool(processes=4)
input_list = [1, 2, 3, 4]
result_list = pool.map(parfunc, input_list) # 'map' waits until all results are ready
print(result_list)
# square-function to run in parallel
def parfunc(x):
return x**2
if __name__ == '__main__':
parallel_runs()
pool.map get's as input a function and only one iterable argument; output is a list of the corresponding results. To run in parallel function with multiple arguments, partial can be used to reduce the number of arguments to the one that is replaced during parallel processing.
https://docs.python.org/3.4/library/multiprocessing.html
get number of CPU's
try:
numProcessors = multiprocessing.cpu_count()
except NotImplementedError: # win32 environment variable NUMBER_OF_PROCESSORS not defined
print('Cannot detect number of CPUs')
numProcessors = 1
Read more:
→ pool.map - using multiple arguments