Ran into an interesting issue with the Python multiprocessing library under Windows, where the code that I had written would hang. It was the equivalent of this:
from multiprocessing import Pool
...
with Pool() as pool:
results = pool.starmap(foo, arglist)
This was in part because of how Windows was forking / pulling back the results.
The work-around I used was the following:
if os.name == 'nt':
from multiprocessing.pool import ThreadPool as Pool
else:
from multiprocessing import Pool
...
with Pool() as pool:
results = pool.starmap(foo, arglist)
Which did the trick. This is because ThreadPool uses Python's native threading (which is Global Interpreter Locked, yes) to handle multi-processing instead of the OS-level forking / spawning.
Posting this here in case I run into this again.