mutcleaner.utils#

Utility functions for mutation and sequence processing.

mutcleaner.utils.download(url, local_path, overwrite=False, chunk_size=8192, timeout=30, max_retries=3, retry_delay=1.0, headers=None, verify_ssl=True, expected_checksum=None, checksum_algorithm='md5', show_progress=True, create_dirs=True)[source]#

Download data from a URL and save to local path with progress tracking.

Parameters:

  • url (str) – URL to download data from

  • local_path (Union[str, Path]) – Local path where the downloaded file will be saved

  • overwrite (bool) – Whether to overwrite existing files

  • chunk_size (int) – Size of chunks to download at a time (in bytes)

  • timeout (int) – Request timeout in seconds

  • max_retries (int) – Maximum number of retry attempts if download fails

  • retry_delay (float) – Delay between retry attempts in seconds

  • headers (Optional[Dict[str, str]]) – Additional HTTP headers to send with the request

  • verify_ssl (bool) – Whether to verify SSL certificates

  • expected_checksum (Optional[str]) – Expected checksum of the downloaded file for verification

  • checksum_algorithm (str) – Algorithm to use for checksum verification (“md5”, “sha1”, “sha256”)

  • show_progress (bool) – Whether to show download progress bar

  • create_dirs (bool) – Whether to create parent directories if they don’t exist

Return type:

Path

Returns:

Path object pointing to the downloaded file

Raises:

  • ValueError – If URL is invalid or checksum verification fails

  • FileExistsError – If file exists and overwrite=False

  • requests.RequestException – If download fails after all retries

Examples

Basic usage:

>>> file_path = download_origin_data(
...     "https://example.com/data.csv",
...     "data/raw_data.csv"
... )
>>> print(f"Downloaded to: {file_path}")
Downloaded to: data/raw_data.csv

With checksum verification:

>>> file_path = download_origin_data(
...     "https://example.com/important_data.xlsx",
...     "data/important_data.xlsx",
...     expected_checksum="5d41402abc4b2a76b9719d911017c592",
...     checksum_algorithm="md5"
... )

With custom headers and retry settings:

>>> headers = {"User-Agent": "MyApp/1.0"}
>>> file_path = download_origin_data(
...     "https://api.example.com/dataset.json",
...     "data/dataset.json",
...     headers=headers,
...     max_retries=5,
...     retry_delay=2.0
... )

Download without progress bar:

>>> file_path = download_origin_data(
...     "https://example.com/data.tsv",
...     "data/data.tsv",
...     show_progress=False,
...     overwrite=True
... )
mutcleaner.utils.download_antitoxin_pard3_source_file(dir, *, overwrite=False)[source]#

Download the source file for Antitoxin_ParD3 dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to Antitoxin ParD3 Epistasis Dataset source file

mutcleaner.utils.download_archstabms1e10_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source files for the ArchStabMS1E10 dataset.

Parameters:

  • dir (str) – The target directory where the files will be saved.

  • overwrite (bool) – Whether to overwrite existing files.

  • sub_dataset (Optional[str]) – Name of the sub-dataset to download. If None, all registered ArchStabMS1E10 source files are downloaded.

Return type:

Dict[str, str]

Returns:

Mapping from file names to downloaded local file paths.

mutcleaner.utils.download_chitosanase_dtm_source_file(dir, *, overwrite=False)[source]#

Download the source file for Chitosanase dTm Dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to Chitosanase dTm Dataset source file

mutcleaner.utils.download_ctxm_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file for CTX-M dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to CTX-M dataset source file

mutcleaner.utils.download_ddg_dtm_source_file(dir, *, dataset_type, overwrite=False, sub_dataset=None)[source]#

Download the source file for ddG or dTm datasets from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved.

  • dataset_type (Literal['ddg', 'dtm']) – Dataset type to download. - “ddg”: download ΔΔG datasets - “dtm”: download ΔTm datasets

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

  • sub_dataset (Optional[str]) – Sub-dataset to download. If None, download all datasets of the selected type.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to the ddG or dTm dataset source file

mutcleaner.utils.download_human_domainome_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download source files for the Human Domainome Dataset.

Parameters:

  • dir (str) – The target directory where the files will be saved.

  • overwrite (bool) – Whether to overwrite existing files.

  • sub_dataset (Optional[Literal['Human Domainome Sup2 Dataset', 'Human Domainome Sup4 Dataset']]) – Literal[ “Human Domainome Sup2 Dataset”, “Human Domainome Sup4 Dataset”, ]

  • default=None (],) – Sub-dataset to download. If None, download all Human Domainome source files.

Return type:

Dict[str, str]

Returns:

Mapping from file names to downloaded local file paths.

mutcleaner.utils.download_human_myoglobin_source_file(dir, *, overwrite=False)[source]#

Download the source file for HumanMyoglobin dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to HumanMyoglobin dataset source file

mutcleaner.utils.download_mgnify_ddg_source_file(dir, *, overwrite=False)[source]#

Download the source file for MGnify ddG Dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to MGnify ddG Dataset source file

mutcleaner.utils.download_protein_cdna_proteolysis_source_file(dir, *, overwrite=False)[source]#

Download the source file for Protein cDNA Proteolysis Dataset from the original source.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to Protein cDNA Proteolysis Dataset source file

mutcleaner.utils.download_proteingym_source_file(dir, *, overwrite=False)[source]#

Download the source file for ProteinGym DMS Substitutions Dataset from the original source.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to the ProteinGym DMS Substitutions Dataset source file

mutcleaner.utils.download_rbd_ace2_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file(s) for the RBD_ACE2 dataset from Hugging Face.

Parameters:

  • dir (str) – The target directory where the file(s) will be saved.

  • overwrite (bool) – Whether to overwrite existing files.

  • sub_dataset (Optional[Literal['Omicron_EG5_FLip_BA286', 'Omicron_XBB_BQ', 'Omicron', 'DMS_variants', 'Delta']]) – If provided, download only the specified RBD ACE2 sub-dataset.

Return type:

Dict[str, str]

Returns:

key: file name, value: local file path

mutcleaner.utils.download_rbd_antibody_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file(s) for the curated RBD_Antibody dataset bundle.

Parameters:

  • dir (str) – The target directory where the file(s) will be saved.

  • overwrite (bool) – Whether to overwrite existing files.

  • sub_dataset (Optional[Literal['Moderna', 'Rockefeller', 'Vir_mAbs']]) – If provided, download only the specified RBD antibody sub-dataset.

Return type:

Dict[str, str]

Returns:

key: file name, value: local file path

mutcleaner.utils.download_trpb_source_file(dir, *, overwrite=False)[source]#

Download the source file for TrpB dataset from the original source.

Parameters:

  • dir (str) – The target directory where the file will be saved

  • overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to TrpB dataset source file

mutcleaner.utils.list_datasets_with_built_in_cleaners()[source]#

List built-in datasets with predefined processing pipelines.

These are public datasets for which this package includes pre-defined data cleaning pipelines. The datasets themselves are not distributed with the package and must be downloaded manually.

You can also define custom cleaner functions for your own datasets using the same @pipeline_step framework.

Predefined datasets:

  • cDNA Proteolysis Dataset

  • ProteinGym DMS Substitutions Dataset

  • Human Domainome Dataset

  • ΔΔG Dataset

  • ΔTm Dataset

  • Antitoxin ParD3 Epistasis Dataset

  • TrpB Epistasis Dataset

  • Human Myoglobin Epistasis Dataset

  • CTXM Epistasis Dataset

  • RBD Antibody Dataset

  • RBD ACE2 Dataset

  • Chitosanase dTm Dataset

  • MGnify ddG Dataset

Return type:

None

mutcleaner.utils.show_download_instructions(dataset_key)[source]#

Show download instructions for a specific dataset.

Return type:

None

Modules

cleaner_workers

data_source

dataset_builders

label_resolvers

Label resolvers for aggregating per-group target columns.

mutation_converter

raw_data_downloader

sequence_io

Utilities for reading and writing sequence files without BioPython dependency.

type_converter

Type conversion utilities for pandas DataFrames.