law.target.file

law.target.file#

Custom luigi file system and target objects.

Class FileSystem#

class FileSystem(name: str | None = None, *, has_permissions: bool = True, default_file_perm: int | None = None, default_dir_perm: int | None = None, create_file_dir: bool = True, **kwargs)[source]#

Bases: FileSystem

abstractmethod exists(path: str | Path, *, stat: bool = False, **kwargs) → bool | stat_result | None[source]#

Return True if file or directory at path exist, False otherwise

Parameters:

path (str) – a path within the FileSystem to check for existence.

abstractmethod isdir(path: str | Path, **kwargs) → bool[source]#

Return True if the location at path is a directory. If not, return False.

Parameters:

path (str) – a path within the FileSystem to check as a directory.

Note: This method is optional, not all FileSystem subclasses implements it.

abstractmethod remove(path: str | Path, *, recursive: bool = True, silent: bool = True, **kwargs) → bool[source]#

Remove file or directory at location path

Parameters:

  • path (str) – a path within the FileSystem to remove.

  • recursive (bool) – if the path is a directory, recursively remove the directory and all of its descendants. Defaults to True.

abstractmethod mkdir(path: str | Path, *, perm: int | None = None, recursive: bool = True, silent: bool = True, **kwargs) → bool[source]#

Create directory at location path

Creates the directory at path and implicitly create parent directories if they do not already exist.

Parameters:

  • path (str) – a path within the FileSystem to create as a directory.

  • parents (bool) – Create parent directories when necessary. When parents=False and the parent directory doesn’t exist, raise luigi.target.MissingParentDirectory

  • raise_if_exists (bool) – raise luigi.target.FileAlreadyExists if the folder already exists.

abstractmethod listdir(path: str | Path, *, pattern: str | None = None, type: Literal['f', 'd'] | None = None, **kwargs) → list[str][source]#

Return a list of files rooted in path.

This returns an iterable of the files rooted at path. This is intended to be a recursive listing.

Parameters:

path (str) – a path within the FileSystem to list.

Note: This method is optional, not all FileSystem subclasses implements it.

abstractmethod copy(src: str | Path, dst: str | Path, *, perm: int | None = None, dir_perm: int | None = None, **kwargs) → str[source]#

Copy a file or a directory with contents. Currently, LocalFileSystem and MockFileSystem support only single file copying but S3Client copies either a file or a directory as required.

abstractmethod move(src: str | Path, dst: str | Path, *, perm: int | None = None, dir_perm: int | None = None, **kwargs) → str[source]#

Move a file, as one would expect.

Class FileSystemTarget#

class FileSystemTarget(path: str | Path, fs: FileSystem | None = None, **kwargs)[source]#

Bases: Target, FileSystemTarget

open: Callable | None = None#
exists(**kwargs) → bool | stat_result | None[source]#

Returns True if the path for this FileSystemTarget exists; False otherwise.

This method is implemented by using fs.

remove(*, silent: bool = True, **kwargs) → bool[source]#

Remove the resource at the path specified by this FileSystemTarget.

This method is implemented by using fs.

abstract property fs: FileSystem#

The FileSystem associated with this FileSystemTarget.

Class FileSystemFileTarget#

class FileSystemFileTarget(path: str | Path, fs: FileSystem | None = None, **kwargs)[source]#

Bases: FileSystemTarget

open(mode: str, **kwargs) → AbstractContextManager[IO][source]#

Open the FileSystem target.

This method returns a file-like object which can either be read from or written to depending on the specified mode.

Parameters:

mode (str) – the mode r opens the FileSystemTarget in read-only mode, whereas w will open the FileSystemTarget in write mode. Subclasses can implement additional options. Using b is not supported; initialize with format=Nop instead.

Class FileSystemDirectoryTarget#

class FileSystemDirectoryTarget(path: str | Path, fs: FileSystem | None = None, **kwargs)[source]#

Bases: FileSystemTarget

open: Callable | None = None#

Functions#

get_path(target: T) → str[source]#
get_scheme(uri: str | Path) → str | None[source]#
has_scheme(uri: str | Path) → bool[source]#
add_scheme(path: str | Path, scheme: str) → str[source]#
remove_scheme(uri: str | Path) → str[source]#
localize_file_targets(struct, *args, **kwargs) → Generator[Any, None, None][source]#

Takes an arbitrary struct of targets, opens the contexts returned by their FileSystemFileTarget.localize() implementations and yields their localized representations in the same structure as passed in struct. When the context is closed, the contexts of all localized targets are closed.

Contents