Ctrl+K

Configuration

Configuration#

Please note that configuration options of law contrib packages might also appear in sections of the general law configuration.

Features of the law configuration parser#

The law home directory#

The law home directory is the default location of a small number of files and directories that are created when working with law. Please refer to the particular configuration options below for more information. The law home directory defaults to “$LAW_HOME” or “$HOME/.law”.

Configuration file resolution order#

Law configuration files are looked up in a certain order: “$LAW_CONFIG_FILE”, “./law.cfg”, “config” in the law home directory, or “etc/law/config”. The config parser only reads the first existing file. Please also note the optional synchronization with environment variables described below.

Inheritance and extensions#

The law configuration can inherit from or can be extended by additional configuration files. See the options “inherit” and “extends” in the “[core]” section of the law configuration below. The only difference between the two is that options defined in “inherit” do not overwrite those in the actual config, whereas options of “extend” do (options of extended config files are added _after_ those in the actual config).

Option referencing#

To avoid that some configurations are copied to other options (which is required in some cases), the law configuration parser supports references. The syntax for references to options in the same section is “&::other_option”, and “&::other_section::other_option” for options in other sections. None is returned in case a reference cannot be resolved. The same applies to circular references.

Environment variable expansion#

Configuration values that contain environment variables are automatically expanded by most getters of the law configuration parser, such as “get_expanded”, “get_expanded_int”, etc. If desired, you can avoid this expanion by backslash-escaping characters such as “$” or “~” to “$” or “~”.

The “None” value#

Getters of the law configuration parser such as “get_default”, get_expanded”, “get_expanded_int”, etc., are configured to return a default value (which itself defaults to “None” (the NoneType)) in case an option is not existing. Sometimes it can be useful to actively declare an option as missing. This can be achieved by setting an option to “None” (string).

Synchronization with environment variables#

In some scenarios it might be useful to control the value of some options via environment variables, for instance, when testing features or prototyping outside of an existing configuration file. For these cases, a synchronization mechanism is implemented that reads variables named “LAW__the_section__the_option” and overwrites the respective option with its value. This is enabled by default, but can be configured by the “sync_env” option in the “[core]” section.

Synchronization with luigi configuration#

law is built on top of luigi and does not replace it. Therefore, another luigi-specfic configuration file is actually required. However, law provides a mechanism to define the luigi configuration right within the law configuration file. All sections starting with “luigi” plus underscore, e.g. “[luigi_worker]” are forwarded to luigi. Environment variable expansion is performed as well. This behavior is configurable by the “sync_luigi_config” option in the “[core]” section.

law configuration#

[core]#

law_home

  • Description: The law home directory is the default location of a small number of files and directories that are created when working with law.

  • Type: string

  • Default: “$LAW_HOME”, “$HOME/.law”

index_file

  • Description: The location of the file that contains a list of all tasks, their module, and their parameters for the purpose of fast autocompletion on the command line.

  • Type: string

  • Default: “$LAW_INDEX_FILE”, “index” in the law home directory

software_dir

  • Description: The default location where softwar is copied to by the “law software” command. This can be especially useful for (e.g.) forwarding law and its dependencies into a container. Type “law software –help” for more info.

  • Type: string

  • Default: “$LAW_SOFTWARE_DIR”, “software” in the law home directory

inherit

  • Description: A comma-separated list of files that this config is inheriting, i.e., their options do _not_ overwrite the ones defined in _this_ file. Relative paths are resolved with respect to _this_ file. Brace expansion, such as “myfile{1,2}.cfg” to “myfile1.cfg, myfile2.cfg” is applied.

  • Type: string, list of strings

  • Default: None

extend

  • Description: A comma-separated list of files that this config is extended by, i.e., their options _do_ overwrite the ones defined in _this_ file. Relative paths are resolved with respect to _this_ file. Brace expansion, such as “myfile{1,2}.cfg” to “myfile1.cfg, myfile2.cfg” is applied.

  • Type: string, list of strings

  • Default: None

sync_env

  • Description: A boolean flag that decides whether the config should be synchronized with environment variables that have the format “LAW__the_section__the_option”. When “True”, the synchronization takes place after all defaults and config files were read.

  • Type: boolean

  • Default: True

sync_luigi_config

  • Description: A boolean flag that decides whether sections starting with “luigi” plus underscore should be synchronized with the luigi configuration.

  • Type: boolean

  • Default: True

[logging]#

law

  • Description: The log level of the law root logger. Accepted values are the default python log levels, i.e., NOTSET, DEBUG, INFO, WARNING, ERROR, or CRITICAL.

  • Type: string

  • Default: “$LAW_LOG_LEVEL”, “WARNING”

gfal2

  • Description: Operations on remote targets are performed using the gfal2 package. This value defines the level of the “gfal2” logger.

  • Type: string

  • Default: “WARNING”

Other loggers can be configured conveniently by adding “my_logger: my_level” to this section. For instance, “law.workflow.remote: DEBUG” will set the log level of the logger defined in “law/workflow/remote.py” to “DEBUG”. Loggers that are not part of the “law” namespace are supported as well.

[modules]#

This section only accepts options without values. They are interpreted as a list of modules that are loaded and checked for law tasks classes when “law index” is executed. This command creates the index file that contains a list of task names and their parameters for fast autocompletion on the command line.

[task]#

colored_repr

  • Description: A boolean flag that decides whether the output of “task.repr()” and “repr(task)” is colorized by default.

  • Type: boolean

  • Default: False

colored_str

  • Description: A boolean flag that decides whether the output of “str(task)” is colorized by default.

  • Type: boolean

  • Default: True

interactive_format

  • Description: The style of interactive task outputs, e.g. when adding “–print-status N” to a command. Possible values: plain, compact, fancy, fancy_compact.

  • Type: string

  • Default: “fancy”

interactive_line_breaks

  • Description: A boolean flag that decides whether smart line breaks should be used when printing interactive task outputs to terminal. Automatically disabled when the terminal width cannot be determined.

  • Type: boolean

  • Default: True

interactive_line_width

  • Description: The width of interactive task output when “interactive_line_breaks” is enabled. When zero or negative, the terminal size is obtained dynamically if possible.

  • Type: integer

  • Default: 0

interactive_status_skip_seen

  • Description: A boolean flag that decides whether during interactive task status traversal, e.g. when adding “–print-status N” to a command, tasks whose status has already been checked are not checked again in case they appear more than once in the dependency tree.

  • Type: boolean

  • Default: False

[target]#

colored_repr

  • Description: A boolean flag that decides whether the output of “target.repr()” and “repr(target)” is colorized by default.

  • Type: boolean

  • Default: False

colored_str

  • Description: A boolean flag that decides whether the output of “str(target)” is colorized by default.

  • Type: boolean

  • Default: True

expand_path_repr

  • Description: A boolean flag that decides whether environment variables in paths of file targets are expanded in “repr(target)” and “str(target)”.

  • Type: boolean

  • Default: False

filesize_repr

  • Description: A boolean flag that decides whether the file size is part of the representation for targets inheriting from law.FileSystemTarget.

  • Type: boolean

  • Default: False

default_local_fs

tmp_dir Description:: The directory in which temporary local targets are stored. Type: string Default: “$LAW_TARGET_TMP_DIR”, “tempfile.gettempdir()”

tmp_dir_perm

  • Description: When the tmp_dir does not exist, it is created with this permission. When “None” or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer, None

  • Default: 0o0770

Options of contrib packages#

default_wlcg_fs

  • Description: The name of the section to lookup for getting the configuration of a law.wlcg.WLCGFileSystem.

  • Type: string

  • Default: “wlcg_fs”

default_dropbox_fs

  • Description: The name of the section to lookup for getting the configuration of a law.dropbox.DropboxFileSystem.

  • Type: string

  • Default: “dropbox_fs”

[local_fs]#

base

  • Description: The base path for accessing files on this file system. Its default refers to the root of the file system which should serve most uses cases where local file targets assigned to it define absolute paths on their own. However, setting a different value will prepend it to the paths used in all its operations.

  • Type: str

  • Default: “/”

Options defined by law.target.file.FileSystem#

has_permissions

  • Description: A boolean flag that signifies whether the file system should handle permissions, i.e., whether “chmod” operations should be evaluated. When “False”, the current umask is used.

  • Type: boolean

  • Default: True

default_file_perm

  • Description: The default permission of newly created files. When “None”, or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer, None

  • Default: None

default_dir_perm

  • Description: The default permission of newly created directories. When “None” or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer, None

  • Default: None

create_file_dir

  • Description: A boolean flag that signifies whether directories should be created when a file is created via open, copy, move or dump operations.

  • Type: boolean

  • Default: True

[wlcg_fs]#

Note: Remote file operations are handled by GFAL2. For more info, see cern-fts/gfal2.

Options defined by law.target.file.FileSystem#

has_permissions

  • Description: A boolean flag that signifies whether the file system should handle permissions, i.e., whether “chmod” operations should be evaluated. When “False”, the current umask is used.

  • Type: boolean

  • Default: True

default_file_perm

  • Description: The default permission of newly created files. When “None” or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer, None

  • Default: None

default_dir_perm

  • Description: The default permission of newly created directories. When “None” or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer, None

  • Default: None

create_file_dir

  • Description: A boolean flag that signifies whether directories should be created when a file is created via open, copy, move or dump operations. When “False”, the “gfal_transfer_create_parent” options should be set to “True” to ensure that intermediate directories are created.

  • Type: boolean

  • Default: False

Options defined by law.target.remote.RemoteFileInterface#

base

  • Description: The base path for accessing files on a WLCG storage element. It should with a protocol, such as “root://” or “gsiftp://”, that can be handled by gfal. This value is required for the law.wlcg.WLCGFileSystem to work. Brace expansion into a list of paths is applied.

  • Type: string, list of strings

  • Default: None

base_stat, base_exists, base_chmod, base_unlink, base_rmdir, base_mkdir, base_listdir, base_filecopy

  • Description: Different base paths for certain file or directory operations. When empty, the value of “base” is used for that operation. As an example, if you use a base path starting with “srm://…”, you might want to choose a different protocol for listdir operations as SRM is too inefficient for directory listings (runtime linear to number of items). Brace expansion into a list of paths is applied.

  • Type: string, list of strings

  • Default: None

retries

  • Description: The default number of retries to perform when a file operation fails. The value can also be set per operation.

  • Type: integer

  • Default: 1

retry_delay

  • Description: The default amount of time to wait before retrying a failed file operation. The default unit is seconds. The value can also be set per operation.

  • Type: integer, string

  • Default: “5s”

random_base Desciption: A boolean flag that, if a base path above is given as a list of paths, decides whether, by default, a random path should be chosen per file operation. When “True” and a file operation fails, the next attempt is made with a different yet still random base path. The exact behavior can also be set per operation. Type: boolean Default: True

Options defined by law.target.remote.RemoteFileSystem#

validate_copy

  • Description: A boolean flag that decides whether, by default, a validation should be performed after each file copy (or move) operation. The exact behavior can also be set per operation.

  • Type: boolean

  • Default: False

use_cache

  • Description: A boolean flag that decides whether, by default, certain operations (copy, move, open, etc.) should use the local cache (when existing). The exact behavior can also be set per operation.

  • Type: boolean

  • Default: False

Options defined by law.target.remote.RemoteCache#

cache_root

  • Description: The directory in which cached files from remote file operation should be stored. When “None” or empty, file caching is disabled. When “__TMP__”, a temporary directory is created.

  • Type: string

  • Default: None

cache_cleanup

  • Description: A boolean flag that decides whether the cache should be removed when the program terminates.

  • Type: boolean

  • Default: False

cache_max_size

  • Description: The maximum size of the cache directory. When a file should be cached, but the maximum size is reached, files are deleted from the cache in order of their smallest modification time until enough disk space is freed. The default unit is MB. When zero or negative, the maximum cache size is equal to 90% of the available disk space.

  • Type: integer, string

  • Default: “0MB”

cache_mtime_patience

  • Description: The maximum amount of seconds that the modification time of remote of cached files can differ before considering a file invalid. A negative value means causes all files to be considered valid, independent of modification times. This feature might only be useful for file systems that are accessible via multiple protocols that might respond to stat requests with different timestamps for files that are known to be identical (e.g. large, identical files that are provided simultaneously by multiple central storage sites).

  • Type: float

  • Default: 1.0

cache_file_perm

  • Description: The permission of files in the cache. When “None” or empty, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: 0o0660

cache_dir_perm

  • Description: The permission of the cache root directory in case it is not existing yet and has to be created. When “None” or empty, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: 0o0770

cache_wait_delay

  • Description: The amount of time to wait for a file in the cache to be unlocked. The default unit is seconds.

  • Type: integer, string

  • Default: “5s”

cache_max_waits

  • Description: The maximum number of times to wait for a duration of “cache_wait_delay” for a file in the cache to be unlocked before an error is thrown that the file is unavailable.

  • Type: integer

  • Default: 120

cache_global_lock

  • Description: A boolean flag that decides whether each cache operation is locked by a global locking mechanism that prevents simultaneous cache access.

  • Type: boolean

  • Default: False

Options defined by law.gfal.GFAlFileInterface#

gfal_atomic_contexts

  • Description: A boolean flag that decides whether each operation of the gfal2 interface should be placed in its own gfal2 context. For the law.wlcg.WLCGFileSystem it is highly recommended to set this option to “True” to enfore that FTP connections are closed as soon as possible and remote resources are freed properly.

  • Type: boolean

  • Default: True

gfal_transfer_timeout

  • Description: The number of seconds after which file operations should be considered timed out.

  • Type: integer

  • Default: 3600

gfal_transfer_checksum_check

  • Description: A boolean flag that decides whether checksums should be checked after operations that create new files.

  • Type: boolean

  • Default: False

gfal_transfer_nbstreams

  • Description: The number of parallel streams for copying files.

  • Type: integer

  • Default: 1

gfal_transfer_overwrite

  • Description: A boolean flag that decides whether existing files should by overwritten by operations that would recreate them.

  • Type: boolean

  • Default: True

gfal_transfer_create_parent

  • Description: A boolean flag that decides whether intermediate, missing directories should be created automatically in certan file operations.

  • Type: boolean

  • Default: True

gfal_transfer_strict_copy

  • Description: A boolean flag that decides whether file copying should be done without source and destination checks.

  • Type: boolean

  • Default: False

[dropbox_fs]#

Note: Remote file operations are handled by GFAL2. For more info, see cern-fts/gfal2. To access files on your Dropbox, you to have to register an application to obtain credentials. Visit https://www.dropbox.com/developers for more information.

Options defined by law.dropbox.DropboxFileSystem#

app_key
app_secret
access_token

Options defined by law.target.file.FileSystem#

has_permissions

  • Description: A boolean flag that signifies whether the file system should handle permissions, i.e., whether “chmod” operations should be evaluated. When “False”, the current umask is used.

  • Type: boolean

  • Default: True

default_file_perm

  • Description: The default permission of newly created files. When “None” or empty, or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: None

default_dir_perm

  • Description: The default permission of newly created directories. When “None” or empty, or when “has_permissions” is “False”, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: None

create_file_dir

  • Description: A boolean flag that signifies whether directories should be created when a file is created via open, copy, move or dump operations. When “False”, the “gfal_transfer_create_parent” options should be set to “True” to ensure that intermediate directories are created.

  • Type: boolean

  • Default: False

Options defined by law.target.remote.RemoteFileInterface#

base

  • Description: The base path for accessing files on your Dropbox. Normally, it should with “dropbox://dropbox.com/…”. This value is required for the law.dropbox.DropboxFileSystem to work.

  • Type: string, list of strings

  • Default: None

base_stat, base_exists, base_chmod, base_unlink, base_rmdir, base_mkdir, base_mkdir_rec, base_listdir, base_filecopy

  • Description: Different base paths for certain file or directory operations. When empty, the value of “base” is used for that operation. For the law.dropbox.DropboxFileSystem, setting different base paths is usually not required as Dropbox does not provide different entry points per operation type.

  • Type: string, list of strings

  • Default: None

retries

  • Description: The default number of retries to perform when a file operation fails. The value can also be set per operation.

  • Type: integer

  • Default: 1

retry_delay

  • Description: The default amount of time to wait before retrying a failed file operation. The default unit is seconds. The value can also be set per operation.

  • Type: integer, string

  • Default: “5s”

random_base Desciption: A boolean flag that, if a base path above is given as a list of paths, decides whether, by default, a random path should be chosen per file operation. When “True” and a file operation fails, the next attempt is made with a different yet still random base path. The exact behavior can also be set per operation. Type: boolean Default: True

Options defined by law.target.remote.RemoteFileSystem#

validate_copy

  • Description: A boolean flag that decides whether, by default, a validation should be performed after each file copy (or move) operation. The exact behavior can also be set per operation.

  • Type: boolean

  • Default: False

use_cache

  • Description: A boolean flag that decides whether, by default, certain operations (copy, move, open, etc.) should use the local cache (when existing). The exact behavior can also be set per operation.

  • Type: boolean

  • Default: False

Options defined by law.target.remote.RemoteCache#

cache_root

  • Description: The directory in which cached files from remote file operation should be stored. When “None” or empty, file caching is disabled. When “__TMP__”, a temporary directory is created.

  • Type: string

  • Default: None

cache_cleanup

  • Description: A boolean flag that decides whether the cache should be removed when the program terminates.

  • Type: boolean

  • Default: False

cache_max_size

  • Description: The maximum size of the cache directory. When a file should be cached, but the maximum size is reached, files are deleted from the cache in order of their smallest modification time until enough disk space is freed. The default unit is MB. When zero or negative, the maximum cache size is equal to 90% of the available disk space.

  • Type: integer, string

  • Default: “0MB”

cache_file_perm

  • Description: The permission of files in the cache. When “None” or empty, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: 0o0660

cache_dir_perm

  • Description: The permission of the cache root directory in case it is not existing yet and has to be created. When “None” or empty, the current umask is used. The value can be expressed by an octal number.

  • Type: integer

  • Default: 0o0770

cache_wait_delay

  • Description: The amount of time to wait for a file in the cache to be unlocked. The default unit is seconds.

  • Type: integer, string

  • Default: “5s”

cache_max_waits

  • Description: The maximum number of times to wait for a duration of “cache_wait_delay” for a file in the cache to be unlocked before an error is thrown that the file is unavailable.

  • Type: integer

  • Default: 120

cache_global_lock

  • Description: A boolean flag that decides whether each cache operation is locked by a global locking mechanism that prevents simultaneous cache access.

  • Type: boolean

  • Default: False

Options defined by law.gfal.GFAlFileInterface#

gfal_atomic_contexts

  • Description: A boolean flag that decides whether each operation of the gfal2 interface should be placed in its own gfal2 context.

  • Type: boolean

  • Default: True

gfal_transfer_timeout

  • Description: The number of seconds after which file operations should be considered timed out.

  • Type: integer

  • Default: 3600

gfal_transfer_checksum_check

  • Description: A boolean flag that decides whether checksums should be checked after operations that create new files.

  • Type: boolean

  • Default: False

gfal_transfer_nbstreams

  • Description: The number of parallel streams for copying files.

  • Type: integer

  • Default: 1

gfal_transfer_overwrite

  • Description: A boolean flag that decides whether existing files should by overwritten by operations that would recreate them.

  • Type: boolean

  • Default: overwrite

gfal_transfer_create_parent

  • Description: A boolean flag that decides whether intermediate, missing directories should be created automatically in certan file operations.

  • Type: boolean

  • Default: True

gfal_transfer_strict_copy

  • Description: A boolean flag that decides whether file copying should be done without source and destination checks.

  • Type: boolean

  • Default: False

[job]#

job_file_dir

  • Description: The location of the directory in which a job file factory saves temporary files that are used for job submission.

  • Type: string

  • Default: “$LAW_JOB_FILE_DIR”, “tempfile.gettempdir()”

job_file_dir_mkdtemp

  • Description: A boolean flag that decides if a temporary directory inside job_file_dir should be created in which the actual job files are stored. It is recommended to set this option to “True” in order to have a clear separation between files created by different job file factories.

  • Type: boolean

  • Default: True

job_file_dir_cleanup

  • Description: A boolean flag that decides whether a job file factory should remove its file diretory once it is destructed. Set this option to “False” when the job submission system requires all jobs files to be present as long as jobs are running, or in case you want to debug the files later on.

  • Type: boolean

  • Default: False

Options of contrib packages#

arc_job_file_dir, arc_job_file_dir_mkdtemp, arc_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “arc” prefix, but only apply to the law.arc.ARCJobFileFactory. When “None” or not existing, the values above are used.

arc_chunk_size_submit

  • Description: Number of jobs that can be submitted in parallel inside a single call to law.arc.ARCJobManager.submit, i.e., in a single “arcsub” command.

  • Type: integer

  • Default: 25

arc_chunk_size_cancel

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.arc.ARCJobManager.cancel, i.e., in a single “arckill” command.

  • Type: integer

  • Default: 25

arc_chunk_size_cleanup

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.arc.ARCJobManager.cleanup, i.e., in a single “arcclean” command.

  • Type: integer

  • Default: 25

arc_chunk_size_query

  • Description: Number of jobs whose status can be queried in parallel inside a single call to law.arc.ARCJobManager.query, i.e., in a single “arcstat” command.

  • Type: integer

  • Default: 25

glite_job_file_dir, glite_job_file_dir_mkdtemp, glite_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “glite” prefix, but only apply to the law.glite.GLiteJobFileFactory. When “None” or not existing, the values above are used.

glite_chunk_size_cancel

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.glite.GLiteJobManager.cancel, i.e., in a single “glite-ce-job-cancel” command.

  • Type: integer

  • Default: 25

glite_chunk_size_cleanup

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.glite.GLiteJobManager.cleanup, i.e., in a single “glite-ce-job-purge” command.

  • Type: integer

  • Default: 25

glite_chunk_size_query

  • Description: Number of jobs whose status can be queried in parallel inside a single call to law.glite.GLiteJobManager.query, i.e., in a single “glite-ce-job-status” command.

  • Type: integer

  • Default: 25

htcondor_job_file_dir, htcondor_job_file_dir_mkdtemp, htcondor_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “htcondor” prefix, but only apply to the law.htcondor.HTCondorJobFileFactory. When “None” or not existing, the values above are used. The only exception is “htcondor_job_file_dir_cleanup” whose default value is False.

htcondor_chunk_size_submit

  • Description: Number of jobs that can be submitted in parallel inside a single call to law.htcondor.HTCondorJobManager.submit, i.e., in a single “condor_submit” command.

  • Type: integer

  • Default: 25

htcondor_chunk_size_cancel

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.htcondor.HTCondorJobManager.cancel, i.e., in a single “condor_rm” command.

  • Type: integer

  • Default: 25

htcondor_chunk_size_query

  • Description: Number of jobs whose status can be queried in parallel inside a single call to law.htcondor.HTCondorJobManager.query, i.e., in a single “condor_q” or “condor_history” command.

  • Type: integer

  • Default: 25

htcondor_merge_job_files

  • Description: A boolean flag that decides whether multiple job description files should be merged into a single file before submission. When “False”, the “htcondor_chunk_size_submit” option is not considered either.

  • Type: boolean

  • Default: True

lsf_job_file_dir, lsf_job_file_dir_mkdtemp, lsf_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “lsf” prefix, but only apply to the law.lsf.LSFJobFileFactory. When “None” or not existing, the values above are used. The only exception is “lsf_job_file_dir_cleanup” whose default value is False.

lsf_chunk_size_cancel

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.lsf.LSFJobManager.cancel, i.e., in a single “bkill” command.

  • Type: integer

  • Default: 25

lsf_chunk_size_query

  • Description: Number of jobs whose status can be queried in parallel inside a single call to law.lsf.LSFJobManager.query, i.e., in a single “bjobs” command.

  • Type: integer

  • Default: 25

slurm_job_file_dir, slurm_job_file_dir_mkdtemp, slurm_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “slurm” prefix, but only apply to the law.slurm.SlurmJobFileFactory. When “None” or not existing, the values above are used. The only exception is “slurm_job_file_dir_cleanup” whose default value is False.

slurm_chunk_size_cancel

  • Description: Number of jobs that can be cancelled in parallel inside a single call to law.slurm.SlurmJobManager.cancel, i.e., in a single “scancel” command.

  • Type: integer

  • Default: 25

slurm_chunk_size_query

  • Description: Number of jobs whose status can be queried in parallel inside a single call to law.slurm.SlurmJobManager.query, i.e., in a single “squeue” or “sacct” command.

  • Type: integer

  • Default: 25

crab_job_file_dir, crab_job_file_dir_mkdtemp, crab_job_file_dir_cleanup

  • Description: These three options are identical to the ones above without the “crab” prefix, but only apply to the law.cms.CrabJobFileFactory. When “None” or not existing, the values above are used.

crab_work_area

  • Description: The directory in which the law.cms.CrabJobManager will create crab project directories upon submission. Defaults to “crab_job_file_dir”.

  • Type: string

  • Default: job.crab_job_file_dir

crab_sandbox_name

  • Description: The name of the “cmssw” sandbox to use which provides the crab executable. In its simplest form, this is just the version string of the CMSSW release to use. However, additional settings can be appended with the “::” delimiter in the format “cmssw_version::setting=value::setting=value” for more configurablity. Supported settings are “setup”, an additional setup script that is executed inside the src directory during installation, “dir”, a custom install directory, “arch”, a custom scram architecture, and “cores”, the number of CPU cores to use for installation.

  • Type: string

  • Default: CMSSW_10_6_30

crab_password_file

  • Description: A file containing the X509 certificate password for automatic proxy delegations by the law.crab.CrabJobManager.

  • Type: string

  • Default: None

[notifications]#

mail_recipient
mail_sender
mail_smtp_host
mail_smtp_port

Options of contrib packages#

slack_token
slack_channel
slack_mention_user

  • Description: When set, notifications sent by law.slack.notify_slack mention this user explicitly.

  • Type: string

  • Default: None

telegram_token
telegram_chat
telegram_mention_user

[bash_sandbox]#

Note: This section defines the default options for all bash sandboxes. To configure options per bash initialization file, create a section “bash_sandbox_<init_file>” with the desired options.

stagein_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and to which input files are staged-in outside of the sandbox and provided to the sandboxed task. When “None”, no stage-in is performed and the task is assumed to be able to access inputs directly from within the sandbox.

  • Type: string

  • Default: “stagein”

stageout_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and provided to sandboxed tasks to store outputs and from which those files are staged-out outside the sandbox. When “None”, no stage-out is performed and the task is assumed to be able to store outputs directly from within the sandbox.

  • Type: string

  • Default: “stageout”

law_executable

  • Description: The law executable to use within sandboxes, e.g. “law” or “python -m law”.

  • Type: string

  • Default: “law”

login

  • Description: A boolean flag that decides whether the bash sandbox should be invoked as a login shell.

  • Type: boolean

  • Default: False

[bash_sandbox_env]#

Note: Here you can define environment variables via key-value pairs that are accessible in a bash sandbox. When an option has no value, i.e., when only a key is given, the value of the variable of the host environment is used. Also note that, unless escaped with a blackslach, “$” and “~” are expanded with variables of the outer task environment. The environment variables defined in this section are applied to all bash sandboxes. To configure variables per bash initialization file, create a section “bash_sandbox_env_<init_file>” with the desired values.

[venv_sandbox]#

Note: This section defines the default options for all venv sandboxes. To configure options per bash initialization file, create a section “venv_sandbox_<venv_dir>” with the desired options.

stagein_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and to which input files are staged-in outside of the sandbox and provided to the sandboxed task. When “None”, no stage-in is performed and the task is assumed to be able to access inputs directly from within the sandbox.

  • Type: string

  • Default: “stagein”

stageout_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and provided to sandboxed tasks to store outputs and from which those files are staged-out outside the sandbox. When “None”, no stage-out is performed and the task is assumed to be able to store outputs directly from within the sandbox.

  • Type: string

  • Default: “stageout”

law_executable

  • Description: The law executable to use within sandboxes, e.g. “law” or “python -m law”.

  • Type: string

  • Default: “law”

[venv_sandbox_env]#

Note: Here you can define environment variables via key-value pairs that are accessible in a venv sandbox. When an option has no value, i.e., when only a key is given, the value of the variable of the host environment is used. Also note that, unless escaped with a blackslach, “$” and “~” are expanded with variables of the outer task environment. The environment variables defined in this section are applied to all venv sandboxes. To configure variables per venv directory, create a section “venv_sandbox_env_<venv_dir>” with the desired values.

[docker_sandbox]#

Note: This section defines the default options for all docker sandboxes. To configure options per image, create a section “docker_sandbox_<image_name>” with the desired options.

stagein_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and to which input files are staged-in outside of the sandbox and provided to the sandboxed task. When “None”, no stage-in is performed and the task is assumed to be able to access inputs directly from within the sandbox.

  • Type: string

  • Default: “stagein”

stageout_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and provided to sandboxed tasks to store outputs and from which those files are staged-out outside the sandbox. When “None”, no stage-out is performed and the task is assumed to be able to store outputs directly from within the sandbox.

  • Type: string

  • Default: “stageout”

law_executable

  • Description: The law executable to use within sandboxes, e.g. “law” or “python -m law”.

  • Type: string

  • Default: “law”

uid

  • Description: The user id of the account inside the docker container. When “None”, the container is started with its default value.

  • Type: integer

  • Default: None

gid

  • Description: The grouo id of the account inside the docker container. When “None”, the container is started with its default value.

  • Type: integer

  • Default: None

forward_dir

  • Description: The container directory in which forwarded files and directories are located.

  • Type: string

  • Default: “/law_forward”

python_dir

  • Description: The container directory in which forwarded python modules are located, relative to “forward_directory”.

  • Type: string

  • Default: “py”

bin_dir

  • Description: The container directory in which forwarded executables are located, relative to “forward_directory”.

  • Type: string

  • Default: “bin”

[docker_sandbox_env]#

Note: Here you can define environment variables via key-value pairs that are accessible in a docker sandbox. When an option has no value, i.e., when only a key is given, the value of the variable of the host environment is used. Also note that, unless escaped with a blackslach, “$” and “~” are expanded with variables of the outer task environment. The environment variables defined in this section are applied to all docker sandboxes. To configure variables per image, create a section “docker_sandbox_env_<image_name>” with the desired values.

[docker_sandbox_volumes]#

Note: Here you can define volumes that are mounted in docker containers via key-value pairs that represent host and container directory, respectively. The volumes defined in this section are applied to all docker sandboxes. To configure volumes per image, create a section “docker_sandbox_volumes_<image_name>” with the desired values.

[singularity_sandbox]#

Note: This section defines the default options for all singularity sandboxes. To configure options per image, create a section “singularity_sandbox_<image_name>” with the desired options.

stagein_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and to which input files are staged-in outside of the sandbox and provided to the sandboxed task. When “None”, no stage-in is performed and the task is assumed to be able to access inputs directly from within the sandbox.

  • Type: string

  • Default: “stagein”

stageout_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and provided to sandboxed tasks to store outputs and from which those files are staged-out outside the sandbox. When “None”, no stage-out is performed and the task is assumed to be able to store outputs directly from within the sandbox.

  • Type: string

  • Default: “stageout”

law_executable

  • Description: The law executable to use within sandboxes, e.g. “law” or “python -m law”.

  • Type: string

  • Default: “law”

uid

  • Description: The user id of the account inside the docker container. When “None”, the container is started with its default value.

  • Type: integer

  • Default: None

gid

  • Description: The grouo id of the account inside the docker container. When “None”, the container is started with its default value.

  • Type: integer

  • Default: None

forward_dir

  • Description: The container directory in which forwarded files and directories are located.

  • Type: string

  • Default: “/law_forward”

python_dir

  • Description: The container directory in which forwarded python modules are located, relative to “forward_directory”.

  • Type: string

  • Default: “py”

bin_dir

  • Description: The container directory in which forwarded executables are located, relative to “forward_directory”.

  • Type: string

  • Default: “bin”

allow_binds

  • Description: A boolean flag that decides whether singularity binds are allowed.

  • Type: boolean

  • Default: True

forward_law

  • Description: A boolean flag that decides whether the law python package and executable should be forwarded into the conatiner. When “False”, the container is assumed to have law accessible.

  • Type: boolean

  • Default: True

[singularity_sandbox_env]#

Note: Here you can define environment variables via key-value pairs that are accessible in a singularity sandbox. When an option has no value, i.e., when only a key is given, the value of the variable of the host environment is used. Also note that, unless escaped with a blackslach, “$” and “~” are expanded with variables of the outer task environment. The environment variables defined in this section are applied to all singularity sandboxes. To configure variables per image, create a section “singularity_sandbox_env_<image_name>” with the desired values.

[singularity_sandbox_volumes]#

Note: Here you can define volumes that are bound to singularity containers via key-value pairs that represent host and container directory, respectively. The volumes defined in this section are applied to all singularity sandboxes. To configure volumes per image, create a section “singularity_sandbox_volumes_<image_name>” with the desired values.

[cmssw_sandbox]#

Note: This section defines the default options for all cmssw sandboxes. To configure options per cmssw version, create a section “cmssw_sandbox_<cmssw_version>” with the desired options.

stagein_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and to which input files are staged-in outside of the sandbox and provided to the sandboxed task. When “None”, no stage-in is performed and the task is assumed to be able to access inputs directly from within the sandbox.

  • Type: string

  • Default: “stagein”

stageout_dir_name

  • Description: Name of a directory which is placed automatically inside a temporary directory and provided to sandboxed tasks to store outputs and from which those files are staged-out outside the sandbox. When “None”, no stage-out is performed and the task is assumed to be able to store outputs directly from within the sandbox.

  • Type: string

  • Default: “stageout”

law_executable

  • Description: The law executable to use within sandboxes, e.g. “law” or “python -m law”.

  • Type: string

  • Default: “law”

login

  • Description: A boolean flag that decides whether the bash beneath the sandbox should be invoked as a login shell.

  • Type: boolean

  • Default: False

[cmssw_sandbox_env]#

Note: Here you can define environment variables via key-value pairs that are accessible in a cmssw sandbox. When an option has no value, i.e., when only a key is given, the value of the variable of the host environment is used. Also note that, unless escaped with a blackslach, “$” and “~” are expanded with variables of the outer task environment. The environment variables defined in this section are applied to all cmssw sandboxes. To configure variables per cmssw version, create a section “cmssw_sandbox_env_<cmssw_version>” with the desired values.

Contents