pype.process

class pype.process.DockerConfig(exec_path, runtime, cache_dir=None)

Bases: object

Configuration for Docker and container runtime environments.

This class manages container runtime configuration and provides properties to identify the runtime type (Docker, Singularity, uDocker).

Attributes:

exec_path: Path to the container runtime executable runtime: Name of the container runtime (‘docker’, ‘singularity’, ‘udocker’) cache_dir: Optional cache directory, required for Singularity containers

Parameters:

• exec_path (Path)

• runtime (str)

• cache_dir (Path | None)

exec_path: Path

runtime: str

cache_dir: Path | None = None

property is_singularity: bool

Check if runtime is Singularity.

property is_udocker: bool

Check if runtime is uDocker.

__init__(exec_path, runtime, cache_dir=None)

Parameters:

• exec_path (Path)

• runtime (str)

• cache_dir (Path | None)

Return type:

None

pype.process.get_module_cmd()

Get the environment modules command interface.

pype.process.module_avail(module=None, modulepath=None)

Parameters:

• module (str | None)

• modulepath (str | None)

pype.process.program_string(program_dict)

Format program string from dictionary.

Parameters:

program_dict (Dict[str, str])

Return type:

str

class pype.process.Volume(path, output=False, bind_prefix='/var/lib/pype')

Bases: object

Volume binding configuration for containerized environments.

Handles the mounting of files and directories between host and container environments, managing read-only and read-write access modes.

Attributes:

path: Path to the file or directory on the host system output: If True, mount as read-write; if False, mount as read-only bind_prefix: Base path in the container where volumes will be mounted

Parameters:

• path (str)

• output (bool)

• bind_prefix (str)

path: str

output: bool = False

bind_prefix: str = '/var/lib/pype'

remove_mode()

Remove mode specifier for udocker compatibility.

Return type:

None

singularity_volume()

Format for Singularity binding syntax.

Return type:

None

to_str()

Get formatted volume string.

Return type:

str

__init__(path, output=False, bind_prefix='/var/lib/pype')

Parameters:

• path (str)

• output (bool)

• bind_prefix (str)

Return type:

None

class pype.process.Command(cmd, log, profile, name='')

Bases: object

High-level interface for executing commands with container support.

This class provides a unified interface for running commands locally or within containers (Docker, Singularity, uDocker). It handles: - Command execution and piping - Volume mounting and file access - Container runtime configuration - Environment modules integration - Input/output file tracking

Attributes:

cmd: Command to execute as list of arguments name: Identifier for the command (used in logs) profile: Configuration profile for the environment log: Logger instance for command execution tracking docker: Container runtime configuration inputs: List of input files required by the command outputs: List of output files produced by the command volumes: List of volumes to mount in container environments

Parameters:

• cmd (str)

• log (Any)

• profile (Any)

• name (str)

__init__(cmd, log, profile, name='')

Initialize command execution environment.

Args:

cmd: Command string to execute log: Logger instance for command output profile: Environment profile configuration name: Optional identifier for the command

Parameters:

• cmd (str)

• log (Any)

• profile (Any)

• name (str)

add_output(out_file)

Add output file to track.

Parameters:

out_file (str)

Return type:

None

add_input(in_file, match='exact')

Add input file(s) to track.

Parameters:

• in_file (str)

• match (str)

Return type:

None

add_volume(path, output=False)

Add volume to track.

Parameters:

• path (str)

• output (bool)

Return type:

None

normalize_volumes()

Normalize volumes to avoid duplicates.

Return type:

None

add_namespace(namespace)

Add namespace to command.

Parameters:

namespace (Dict[str, Any])

Return type:

None

run(local_script=False)

Execute the command.

Parameters:

local_script (bool)

Return type:

None

docker_run(local_script)

Execute command within a container environment.

Handles the complexities of running commands in containers including: - Volume mounting and path translation - User permissions - Working directory setup - Container-specific command modifications

Args:

local_script: If True, indicates the command is a local script that

needs to be mounted in the container

Parameters:

local_script (bool)

Return type:

None

local_run()

Execute command locally.

Return type:

None

module_run()

Execute command with environment modules.

Return type:

None

pipe_in(command, local_script=False)

Pipe input from another command.

Parameters:

• command (Command)

• local_script (bool)

Return type:

None

child_close()

Close child process.

Return type:

None

close()

Close command and return result.

Return type:

Any

replace_values_in_code(code_file)

Replace values in code file with bind volumes.

Parameters:

code_file (str)

Return type:

None

pype.process.is_in_directory(x, y)

Check if x is in directory y.

Parameters:

• x (str)

• y (str)

Return type:

bool

pype.process.has_same_basedir(x, y)

Check if x and y have the same base directory.

Parameters:

• x (str)

• y (str)

Return type:

bool

class pype.process.Namespace(program_dict, log, profile)

Bases: object

Environment and dependency management system.

Manages different execution environments (path-based, environment modules, containers) and their dependencies. Supports: - Path-based program execution - Environment modules loading - Container image specification - Dependency resolution

The namespace format is ‘type@item’ where: - type: One of ‘path’, ‘env_module’, or ‘docker’ - item: The specific program/container/module to use

For example: - ‘docker@ubuntu:latest’ - ‘env_module@gcc/9.3.0’ - ‘path@/usr/local/bin/python’

Parameters:

• program_dict (Dict[str, Any])

• log (Any)

• profile (Any)

__init__(program_dict, log, profile)

Initialize namespace configuration.

Args:

program_dict: Dictionary containing program configuration:

• namespace: String in format ‘type@item’

• version: Program version

• dependencies: Optional list of dependent programs

log: Logger instance for namespace operations profile: Environment profile containing program configurations

Raises:

CommandNamespaceError: For invalid namespace format or unsupported types

Parameters:

• program_dict (Dict[str, Any])

• log (Any)

• profile (Any)

first()

Return type:

str