Ctrl+K

htcondor

htcondor#

HTCondor contrib functionality.

Class HTCondorWorkflow#

class HTCondorWorkflow(*args, **kwargs)#

Bases: BaseRemoteWorkflow

workflow_proxy_cls#

alias of HTCondorWorkflowProxy

Class HTCondorJobManager#

class HTCondorJobManager(pool=None, scheduler=None, user=None, threads=1)#

Bases: BaseJobManager

cleanup(*args, **kwargs)#

Abstract atomic or group job cleanup. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

cleanup_batch(*args, **kwargs)#

Cleans up a batch of jobs given by job_ids via a thread pool of size threads which defaults to its instance attribute. When chunk_size, which defaults to chunk_size_cleanup, is not negative, job_ids are split into chunks of that size which are passed to cleanup().

When callback is set, it is invoked after each successful job (or job chunk) cleaning with the index of the corresponding job id (starting at 0) and either None or an exception if any occurred. All other kwargs are passed to cleanup().

Exceptions that occured during job cleaning are stored in a list and returned. An empty list means that no exceptions occured.

submit(job_file, pool=None, scheduler=None, retries=0, retry_delay=3, silent=False)#

Abstract atomic or group job submission. Can throw exceptions. Should return a list of job ids.

cancel(job_id, pool=None, scheduler=None, silent=False)#

Abstract atomic or group job cancellation. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

query(job_id, pool=None, scheduler=None, user=None, silent=False)#

Abstract atomic or group job status query. Can throw exceptions. Should return a dictionary mapping job ids to per-job return values.

Class HTCondorJobFileFactory#

class HTCondorJobFileFactory(file_name='htcondor_job.jdl', command=None, executable=None, arguments=None, input_files=None, output_files=None, log='log.txt', stdout='stdout.txt', stderr='stderr.txt', postfix_output_files=True, universe='vanilla', notification='Never', custom_content=None, absolute_paths=False, **kwargs)#

Bases: BaseJobFileFactory

create(postfix=None, **kwargs)#

Abstract job file creation method that must be implemented by inheriting classes.

Contents