law.decorator

Helpful decorators to use with tasks.

Example usage:

class MyTask(law.Task):

    @log
    @safe_output(skip=KeyboardInterrupt)
    def run(self):
        ...

The usage of a decorator without invocation (e.g. @log) is equivalent to the one with invocation (@log()), for law to distuinguish between the two cases always use keyword arguments when configuring decorators. Default arguments are applied in either case.

Functions:

factory(**default_opts)	Factory function to create decorators for tasks' run methods.
log()	Wraps a bound method of a task and redirects output of both stdout and stderr to the file defined by the tasks's log_file parameter or default_log_file attribute.
safe_output([skip])	Wraps a bound method of a task and guards its execution.
delay([t, stddev, pdf])	Wraps a bound method of a task and delays its execution by t seconds.
notify([on_success, on_failure])	Wraps a bound method of a task and guards its execution.
timeit()	Wraps a bound method of a task and logs its execution time in a human readable format using the task's logger instance in info mode.
localize([input, output, input_kwargs, ...])	Wraps a bound method of a task and temporarily changes the input and output methods to return localized targets.
require_sandbox([sandbox])	Wraps a bound method of a sandbox task and throws an exception when the method is called while the task is not sandboxed yet.

factory(**default_opts)

Factory function to create decorators for tasks’ run methods. Default options for the decorator function can be given in default_opts. The returned decorator can be used with or without function invocation. Example:

@factory(digits=2)
def runtime(fn, opts, task, *args, **kwargs):
    t0 = time.time()
    try:
        return fn(task, *args, **kwargs)
    finally:
        t1 = time.time()
        diff = round(t1 - t0, opts["digits"])
        print("runtime: {}".format(diff))

...

class MyTask(law.Task):

    @runtime
    def run(self):
        ...

    # or
    @runtime(digits=3):
    def run(self):
        ...

In most cases, the created decorators are used to decorate run methods. As intended by luigi, run methods can become generators by yielding tasks to declare dynamic dependencies. As luigi will resume the run method from scratch everytime a new, incomplete dependency is yielded, decorators are required to be idempotent. Therefore, a plain definition as shown in the example above is not sufficient. A decorator that accepts generator functions should look like the following:

@factory(digits=2, accept_generator=True)
def runtime(fn, opts, task, *args, **kwargs):
    def before_call():
        t0 = time.time()
        return t0

    def call(t0):
        return fn(task, *args, **kwargs)

    def after_call(t0):
        t1 = time.time()
        diff = round(t1 - t0, opts["digits"])
        print("runtime: {}".format(diff))

    # optional:
    def on_error(e, t0):
        # called when an exception occured in call,
        # return True to prevent the error from being raised
        ...

    return before_call, call, after_call[, on_error]

before_call() is invoked only once. It can be used to setup objects, etc, and its return value is passed as a single argument to both call() and after_call(), even when None. The former function should (at least) call the actual wrapped function and return its result while the latter is intended to execute custom logic afterwards. The use of a 4th function for handling exceptions is optional. It is called when an exception is raised inside call() with the exception instance and the return value of before_call as arguments. When its return value is True, the error is not raised and the return value of the wrapped function becomes None.

A decorator that accepts generator functions can also be used to decorate plain, non-generator functions, but not vice-versa. Decorated functions can be called with a keyword argument skip_decorators set to True to directly call the originally wrapped function without the stack of decorators.

log()

Wraps a bound method of a task and redirects output of both stdout and stderr to the file defined by the tasks’s log_file parameter or default_log_file attribute. If its value is "-" or None, the output is not redirected. Does not accept generator functions.

safe_output(skip=None)

Wraps a bound method of a task and guards its execution. If an exception occurs, and it is not an instance of skip, the task’s output is removed prior to the actual raising. Accepts generator functions.

delay(t=5.0, stddev=0.0, pdf='gauss')

Wraps a bound method of a task and delays its execution by t seconds. Accepts generator functions.

notify(on_success=True, on_failure=True)

Wraps a bound method of a task and guards its execution. Information about the execution (task name, duration, etc) is collected and dispatched to all notification transports registered on wrapped task via adding law.NotifyParameter parameters. Example:

class MyTask(law.Task):

    notify_mail = law.NotifyMailParameter()

    @notify
    # or
    @notify(sender="foo@bar.com", recipient="user@host.tld")
    def run(self):
        ...

When the notify_mail parameter is True, a notification is sent to the configured email address. Also see config-notifications. Accepts generator functions.

timeit()

Wraps a bound method of a task and logs its execution time in a human readable format using the task’s logger instance in info mode. Accepts generator functions.

localize(input=True, output=True, input_kwargs=None, output_kwargs=None)

Wraps a bound method of a task and temporarily changes the input and output methods to return localized targets. When input (output) is True, Task.input() (Task.output()) is adjusted. input_kwargs and output_kwargs can be dictionaries that are passed as keyword arguments to the respective localization method. Does not accept generator functions.

require_sandbox(sandbox=None)

Wraps a bound method of a sandbox task and throws an exception when the method is called while the task is not sandboxed yet. This is intended to prevent undesired results or non-verbose error messages when the method is invoked outside the requested sandbox. When sandbox is set, it can be a (list of) pattern(s) to compare against the task’s effective sandbox and in error is raised if they don’t match. Accepts generator functions.