arc

arc#

ARC contrib functionality.

Class ARCWorkflow#

class ARCWorkflow(*args, **kwargs)#

Bases: BaseRemoteWorkflow

workflow_proxy_cls#

alias of ARCWorkflowProxy

arc_workflow_run_context()#

Hook to provide a context manager in which the workflow run implementation is placed. This can be helpful in situations where resurces should be acquired before and released after running a workflow.

arc_job_resources(job_num, branches)#

Hook to define resources for a specific job with number job_num, processing branches. This method should return a dictionary.

arc_dump_intermediate_job_data()#

Whether to dump intermediate job data to the job submission file while jobs are being submitted.

arc_post_submit_delay()#

Configurable delay in seconds to wait after submitting jobs and before starting the status polling.

arc_poll_callback(poll_data)#

Configurable callback that is called after each job status query and before potential resubmission. It receives the variable polling attributes poll_data (PollData) that can be changed within this method.

If False is returned, the polling loop is gracefully terminated. Returning any other value does not have any effect.

arc_post_poll_callback(success, duration)#

Configurable callback that is called after the polling loop has ended. It receives a boolean success that indicates whether the job polling was successful, and the duration of the job polling in seconds.

Class ARCJobManager#

class ARCJobManager(job_list=None, ce=None, threads=1)#

Bases: BaseJobManager

submit(job_file, job_list=None, ce=None, retries=0, retry_delay=3, silent=False, _processes=None)#

Abstract atomic or group job submission. Can throw exceptions. Should return a list of job ids.

cancel(job_id, job_list=None, silent=False, _processes=None)#

Abstract atomic or group job cancellation. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

cleanup(job_id, job_list=None, silent=False, _processes=None)#

Abstract atomic or group job cleanup. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

query(job_id, job_list=None, silent=False, _processes=None)#

Abstract atomic or group job status query. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

Class ARCJobFileFactory#

class ARCJobFileFactory(file_name='arc_job.xrsl', command=None, executable=None, arguments=None, input_files=None, output_files=None, postfix_output_files=True, output_uri=None, overwrite_output_files=True, job_name=None, log='log.txt', stdout='stdout.txt', stderr='stderr.txt', custom_content=None, absolute_paths=True, **kwargs)#

Bases: BaseJobFileFactory

create(postfix=None, **kwargs)#

Abstract job file creation method that must be implemented by inheriting classes.