law.patches

Collection of minor patches for luigi. These patches are only intended to support extra features for law, rather than changing default luigi behavior.

before_run(func: Callable, force: bool = False) → bool

Adds a function func to the list of callbacks that are invoked right before luigi starts running scheduled tasks. Unless force is True, a function that is already registered is not added again and False is returned. Otherwise, True is returned.

patch_all() → None

Runs all patches. This function ensures that a second invocation has no effect.

before_run(func: Callable, force: bool = False) → bool

Adds a function func to the list of callbacks that are invoked right before luigi starts running scheduled tasks. Unless force is True, a function that is already registered is not added again and False is returned. Otherwise, True is returned.

patch_all() → None

Runs all patches. This function ensures that a second invocation has no effect.

patch_default_retcodes() → None

Sets the default luigi return codes in luigi.retcodes.retcode to:

• already_running: 10

• missing_data: 20

• not_run: 30

• task_failed: 40

• scheduling_error: 50

• unhandled_exception: 60

patch_worker_add_task() → None

Patches the luigi.worker.Worker._add_task method to skip dependencies of the triggered task when running in a sandbox, as dependencies are already controlled from outside the sandbox. To reduce redundant logs, the severity of luigi’s interface logging is increased when running in a sandbox. In addition, info logs about repeatedly added tasks are suppressed.

patch_worker_add() → None

Patches the luigi.worker.Worker._add method to make sure that no dependencies are yielded when the triggered task is added to the worker when running in a sandbox and that the task is added to the scheduler with the id of the outer task.

patch_worker_run_task() → None

Patches the luigi.worker.Worker._run_task method to store the worker id and the id of its first task in the task. This information is required by the sandboxing mechanism.

patch_worker_get_work() → None

Patches the luigi.worker.Worker._get_work method to only return information of the sandboxed task when running in a sandbox. This way, actual (outer) task and the sandboxed (inner) task appear to a central as the same task and communication for exchanging (e.g.) messages becomes transparent.

patch_worker_factory() → None

Patches the luigi.interface._WorkerSchedulerFactory to include sandboxing information when creating a worker instance.

patch_keepalive_run() → None

Patches the luigi.worker.KeepAliveThread.run to immediately stop the keep-alive thread when running within a sandbox.

patch_cmdline_parser() → None

Patches the luigi.cmdline_parser.CmdlineParser to store the original command line arguments for later processing in the law.config.Config, and to update the way that parameter objects are called to parse empty strings.

patch_schedule_and_run() → None

Patches luigi.interface._schedule_and_run to invoke all callbacks registered via before_run() right before luigi starts running scheduled tasks. This is achieved by patching luigi.worker.Worker.run within the scope of luigi.interface._schedule_and_run.

patch_interface_logging() → None

Patches luigi.setup_logging.InterfaceLogging._default to avoid adding multiple tty stream handlers to the logger named “luigi-interface” and to preserve any previously set log level. Also, the formatters of its stream handlers are amended in order to colorize parts of luigi log messages.