law.patches#
Collection of minor patches for luigi. These patches are only intended to support extra features for law, rather than changing default luigi behavior.
- before_run(func, force=False)[source]#
Adds a function func to the list of callbacks that are invoked right before luigi starts running scheduled tasks. Unless force is True, a function that is already registered is not added again and False is returned. Otherwise, True is returned.
- patch_all()[source]#
Runs all patches. This function ensures that a second invocation has no effect.
- before_run(func, force=False)[source]#
Adds a function func to the list of callbacks that are invoked right before luigi starts running scheduled tasks. Unless force is True, a function that is already registered is not added again and False is returned. Otherwise, True is returned.
- patch_all()[source]#
Runs all patches. This function ensures that a second invocation has no effect.
- patch_default_retcodes()[source]#
Sets the default luigi return codes in
luigi.retcodes.retcode
to:already_running: 10
missing_data: 20
not_run: 30
task_failed: 40
scheduling_error: 50
unhandled_exception: 60
- patch_worker_add_task()[source]#
Patches the
luigi.worker.Worker._add_task
method to skip dependencies of the triggered task when running in a sandbox, as dependencies are already controlled from outside the sandbox. Also, the severity of luigi’s interface logging is increased when running in a sandbox.
- patch_worker_add()[source]#
Patches the
luigi.worker.Worker._add
method to make sure that no dependencies are yielded when the triggered task is added to the worker when running in a sandbox and that the task is added to the scheduler with the id of the outer task.
- patch_worker_run_task()[source]#
Patches the
luigi.worker.Worker._run_task
method to store the worker id and the id of its first task in the task. This information is required by the sandboxing mechanism.
- patch_worker_get_work()[source]#
Patches the
luigi.worker.Worker._get_work
method to only return information of the sandboxed task when running in a sandbox. This way, actual (outer) task and the sandboxed (inner) task appear to a central as the same task and communication for exchanging (e.g.) messages becomes transparent.
- patch_worker_factory()[source]#
Patches the
luigi.interface._WorkerSchedulerFactory
to include sandboxing information when creating a worker instance.
- patch_keepalive_run()[source]#
Patches the
luigi.worker.KeepAliveThread.run
to immediately stop the keep-alive thread when running within a sandbox.
- patch_cmdline_parser()[source]#
Patches the
luigi.cmdline_parser.CmdlineParser
to store the original command line arguments for later processing in thelaw.config.Config
, and to update the way that parameter objects are called to parse empty strings.
- patch_schedule_and_run()[source]#
Patches
luigi.interface._schedule_and_run
to invoke all callbacks registered viabefore_run()
right before luigi starts running scheduled tasks. This is achieved by patchingluigi.worker.Worker.run
within the scope ofluigi.interface._schedule_and_run
.
- patch_interface_logging()[source]#
Patches
luigi.setup_logging.InterfaceLogging._default
to avoid adding multiple tty stream handlers to the logger named “luigi-interface” and to preserve any previously set log level. Also, the formatters of its stream handlers are amended in order to colorize parts of luigi log messages.