(continued from previous page) (continues on next page) 17 Sometimes you want to let a log file grow to a certain size, then open a new file and log to that. You may want to keep a certain number of these files, and when that many files have been created, rotate the files so that the number of files and the size of the files both remain bounded. For this usage pattern, the logging package provides a $_{RotatingFileHandler}$: to this (remembering to first import$_{concurrent.futures}$ ): This variant shows how you can e.g. apply configuration for particular loggers - e.g. the foo logger has a special handler which stores all events in the foo subsystem in a file mplog-foo.log . This will be used by the logging machinery in the main process (even though the logging events are generated in the worker processes) to direct the messages to the appropriate destinations. If you want to use$_{concurrent.futures.ProcessPoolExecutor}$ to start your worker processes, you need to create the queue slightly differently. Instead of wp.join() # And now tell the logging thread to finish up, too q.put( None ) lp.join() workers =[] for i in range(10 ): worker =multiprocessing.Process(target =worker_process, args=(queue, worker_configurer)) workers.append(worker) worker.start() for w in workers: w.join() with concurrent.futures.ProcessPoolExecutor(max_workers =10 ) as executor: for i in range(10 ): executor.submit(worker_process, queue, worker_configurer) import glob import logging import logging.handlers LOG_FILENAME ='logging_rotatingfile_example.out ' # Set up a specific logger with our desired output level my_logger=logging.getLogger('MyLogger ') my_logger .setLevel(logging .DEBUG) # Add the log message handler to the logger 10 Using file rotation queue =multiprocessing.Queue(-1) you should use queue =multiprocessing.Manager() .Queue(-1) # also works with the examples above and you can then replace the worker creation from this: 9.1 Using concurrent.futures.ProcessPoolExecutor