mutcleaner.utils.raw_data_downloader#

Functions

`download`(url, local_path[, overwrite, ...])	Download data from a URL and save to local path with progress tracking.
`download_antitoxin_pard3_source_file`(dir, *)	Download the source file for Antitoxin_ParD3 dataset from the original source.
`download_archstabms1e10_source_file`(dir, *)	Download the source files for the ArchStabMS1E10 dataset.
`download_chitosanase_dtm_source_file`(dir, *)	Download the source file for Chitosanase dTm Dataset from the original source.
`download_ctxm_source_file`(dir, *[, ...])	Download the source file for CTX-M dataset from the original source.
`download_ddg_dtm_source_file`(dir, *, ...[, ...])	Download the source file for ddG or dTm datasets from the original source.
`download_human_domainome_source_file`(dir, *)	Download source files for the Human Domainome Dataset.
`download_human_myoglobin_source_file`(dir, *)	Download the source file for HumanMyoglobin dataset from the original source.
`download_mgnify_ddg_source_file`(dir, *[, ...])	Download the source file for MGnify ddG Dataset from the original source.
`download_protein_cdna_proteolysis_source_file`(dir, *)	Download the source file for Protein cDNA Proteolysis Dataset from the original source.
`download_proteingym_source_file`(dir, *[, ...])	Download the source file for ProteinGym DMS Substitutions Dataset from the original source.
`download_rbd_ace2_source_file`(dir, *[, ...])	Download the source file(s) for the RBD_ACE2 dataset from Hugging Face.
`download_rbd_antibody_source_file`(dir, *[, ...])	Download the source file(s) for the curated RBD_Antibody dataset bundle.
`download_trpb_source_file`(dir, *[, overwrite])	Download the source file for TrpB dataset from the original source.

mutcleaner.utils.raw_data_downloader.download(url, local_path, overwrite=False, chunk_size=8192, timeout=30, max_retries=3, retry_delay=1.0, headers=None, verify_ssl=True, expected_checksum=None, checksum_algorithm='md5', show_progress=True, create_dirs=True)[source]#

Download data from a URL and save to local path with progress tracking.

Parameters:

url (str) – URL to download data from
local_path (Union[str, Path]) – Local path where the downloaded file will be saved
overwrite (bool) – Whether to overwrite existing files
chunk_size (int) – Size of chunks to download at a time (in bytes)
timeout (int) – Request timeout in seconds
max_retries (int) – Maximum number of retry attempts if download fails
retry_delay (float) – Delay between retry attempts in seconds
headers (Optional[Dict[str, str]]) – Additional HTTP headers to send with the request
verify_ssl (bool) – Whether to verify SSL certificates
expected_checksum (Optional[str]) – Expected checksum of the downloaded file for verification
checksum_algorithm (str) – Algorithm to use for checksum verification (“md5”, “sha1”, “sha256”)
show_progress (bool) – Whether to show download progress bar
create_dirs (bool) – Whether to create parent directories if they don’t exist

Return type:

Path

Returns:

Path object pointing to the downloaded file

Raises:

ValueError – If URL is invalid or checksum verification fails
FileExistsError – If file exists and overwrite=False
requests.RequestException – If download fails after all retries

Examples

Basic usage:

>>> file_path = download_origin_data(
...     "https://example.com/data.csv",
...     "data/raw_data.csv"
... )
>>> print(f"Downloaded to: {file_path}")
Downloaded to: data/raw_data.csv

With checksum verification:

>>> file_path = download_origin_data(
...     "https://example.com/important_data.xlsx",
...     "data/important_data.xlsx",
...     expected_checksum="5d41402abc4b2a76b9719d911017c592",
...     checksum_algorithm="md5"
... )

With custom headers and retry settings:

>>> headers = {"User-Agent": "MyApp/1.0"}
>>> file_path = download_origin_data(
...     "https://api.example.com/dataset.json",
...     "data/dataset.json",
...     headers=headers,
...     max_retries=5,
...     retry_delay=2.0
... )

Download without progress bar:

>>> file_path = download_origin_data(
...     "https://example.com/data.tsv",
...     "data/data.tsv",
...     show_progress=False,
...     overwrite=True
... )

mutcleaner.utils.raw_data_downloader.download_antitoxin_pard3_source_file(dir, *, overwrite=False)[source]#

Download the source file for Antitoxin_ParD3 dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to Antitoxin ParD3 Epistasis Dataset source file

mutcleaner.utils.raw_data_downloader.download_archstabms1e10_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source files for the ArchStabMS1E10 dataset.

Parameters:

dir (str) – The target directory where the files will be saved.
overwrite (bool) – Whether to overwrite existing files.
sub_dataset (Optional[str]) – Name of the sub-dataset to download. If None, all registered ArchStabMS1E10 source files are downloaded.

Return type:

Dict[str, str]

Returns:

Mapping from file names to downloaded local file paths.

mutcleaner.utils.raw_data_downloader.download_chitosanase_dtm_source_file(dir, *, overwrite=False)[source]#

Download the source file for Chitosanase dTm Dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to Chitosanase dTm Dataset source file

mutcleaner.utils.raw_data_downloader.download_ctxm_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file for CTX-M dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to CTX-M dataset source file

mutcleaner.utils.raw_data_downloader.download_ddg_dtm_source_file(dir, *, dataset_type, overwrite=False, sub_dataset=None)[source]#

Download the source file for ddG or dTm datasets from the original source.

Parameters:

dir (str) – The target directory where the file will be saved.
dataset_type (Literal['ddg', 'dtm']) – Dataset type to download. - “ddg”: download ΔΔG datasets - “dtm”: download ΔTm datasets
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.
sub_dataset (Optional[str]) – Sub-dataset to download. If None, download all datasets of the selected type.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to the ddG or dTm dataset source file

mutcleaner.utils.raw_data_downloader.download_human_domainome_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download source files for the Human Domainome Dataset.

Parameters:

dir (str) – The target directory where the files will be saved.
overwrite (bool) – Whether to overwrite existing files.
sub_dataset (Optional[Literal['Human Domainome Sup2 Dataset', 'Human Domainome Sup4 Dataset']]) – Literal[ “Human Domainome Sup2 Dataset”, “Human Domainome Sup4 Dataset”, ]
default=None (],) – Sub-dataset to download. If None, download all Human Domainome source files.

Return type:

Dict[str, str]

Returns:

Mapping from file names to downloaded local file paths.

mutcleaner.utils.raw_data_downloader.download_human_myoglobin_source_file(dir, *, overwrite=False)[source]#

Download the source file for HumanMyoglobin dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to HumanMyoglobin dataset source file

mutcleaner.utils.raw_data_downloader.download_mgnify_ddg_source_file(dir, *, overwrite=False)[source]#

Download the source file for MGnify ddG Dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to MGnify ddG Dataset source file

mutcleaner.utils.raw_data_downloader.download_protein_cdna_proteolysis_source_file(dir, *, overwrite=False)[source]#

Download the source file for Protein cDNA Proteolysis Dataset from the original source.

Return type:: Dict[str, str]
Returns:: key: file name, value: file path pointing to Protein cDNA Proteolysis Dataset source file

mutcleaner.utils.raw_data_downloader.download_proteingym_source_file(dir, *, overwrite=False)[source]#

Download the source file for ProteinGym DMS Substitutions Dataset from the original source.

Return type:: Dict[str, str]
Returns:: key: file name, value: file path pointing to the ProteinGym DMS Substitutions Dataset source file

mutcleaner.utils.raw_data_downloader.download_rbd_ace2_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file(s) for the RBD_ACE2 dataset from Hugging Face.

Parameters:

dir (str) – The target directory where the file(s) will be saved.
overwrite (bool) – Whether to overwrite existing files.
sub_dataset (Optional[Literal['Omicron_EG5_FLip_BA286', 'Omicron_XBB_BQ', 'Omicron', 'DMS_variants', 'Delta']]) – If provided, download only the specified RBD ACE2 sub-dataset.

Return type:

Dict[str, str]

Returns:

key: file name, value: local file path

mutcleaner.utils.raw_data_downloader.download_rbd_antibody_source_file(dir, *, overwrite=False, sub_dataset=None)[source]#

Download the source file(s) for the curated RBD_Antibody dataset bundle.

Parameters:

dir (str) – The target directory where the file(s) will be saved.
overwrite (bool) – Whether to overwrite existing files.
sub_dataset (Optional[Literal['Moderna', 'Rockefeller', 'Vir_mAbs']]) – If provided, download only the specified RBD antibody sub-dataset.

Return type:

Dict[str, str]

Returns:

key: file name, value: local file path

mutcleaner.utils.raw_data_downloader.download_trpb_source_file(dir, *, overwrite=False)[source]#

Download the source file for TrpB dataset from the original source.

Parameters:

dir (str) – The target directory where the file will be saved
overwrite (bool) – Whether to overwrite the file if it already exists. Default is False.

Return type:

Dict[str, str]

Returns:

key: file name, value: file path pointing to TrpB dataset source file