Batch evaluators#
A collection of batch evaluators for process based parallelism.
All batch evaluators have the same interface and any function with the same interface can be used used as batch evaluator in estimagic.
- estimagic.batch_evaluators.pathos_mp_batch_evaluator(func, arguments, *, n_cores=1, error_handling='continue', unpack_symbol=None)[source]#
Batch evaluator based on pathos.multiprocess.ProcessPool.
This uses a patched but older version of python multiprocessing that replaces pickling with dill and can thus handle decorated functions.
- Parameters
func (Callable) – The function that is evaluated.
arguments (Iterable) – Arguments for the functions. Their interperation depends on the unpack argument.
n_cores (int) – Number of cores used to evaluate the function in parallel. Value below one are interpreted as one. If only one core is used, the batch evaluator disables everything that could cause problems, i.e. in that case func and arguments are never pickled and func is executed in the main process.
error_handling (str) – Can take the values “raise” (raise the error and stop all tasks as soon as one task fails) and “continue” (catch exceptions and set the traceback of the raised exception. KeyboardInterrupt and SystemExit are always raised.
unpack_symbol (str or None) – one argument. If “*”, the elements of arguments are positional arguments for func. If “**”, the elements of arguments are keyword arguments for func.
- Returns
The function evaluations.
- Return type
- estimagic.batch_evaluators.joblib_batch_evaluator(func, arguments, *, n_cores=1, error_handling='continue', unpack_symbol=None)[source]#
Batch evaluator based on joblib’s Parallel.
- Parameters
func (Callable) – The function that is evaluated.
arguments (Iterable) – Arguments for the functions. Their interperation depends on the unpack argument.
n_cores (int) – Number of cores used to evaluate the function in parallel. Value below one are interpreted as one. If only one core is used, the batch evaluator disables everything that could cause problems, i.e. in that case func and arguments are never pickled and func is executed in the main process.
error_handling (str) – Can take the values “raise” (raise the error and stop all tasks as soon as one task fails) and “continue” (catch exceptions and set the output of failed tasks to the traceback of the raised exception. KeyboardInterrupt and SystemExit are always raised.
unpack_symbol (str or None) – one argument. If “*”, the elements of arguments are positional arguments for func. If “**”, the elements of arguments are keyword arguments for func.
- Returns
The function evaluations.
- Return type