Cthulhu.downloader

Functions

download_ExoMol_file(url, f, l_folder)

Download a file from ExoMol and decompress it if needed.

download_HITRAN_line_list(mol_ID, iso_ID, folder[, ...])

Download line list using the fetch() function already in HITRAN.

HITEMP_table()

Recreate the table found on the HITEMP main page in order to simplify later processes

download_HITEMP_line_list(mol_ID, iso_ID, out_folder)

Download a line list from the HITEMP database

check_HITEMP_file_exists(folder, file)

Check to see if a given HITEMP file already exists. This is to ensure that HITEMP downloads that stall midway

process_hdf_chunk(chunk, upper_ds, lower_ds, logA_ds, ...)

Process chunk of data from convert_to_hdf

convert_to_hdf([file, mol_ID, iso_ID, alkali, ...])

Convert a given file to HDF5 format.

create_directories([molecule, isotopologue, ...])

Create new folders on local machine to store the relevant data

calc_num_ExoMol_trans(html_tags)

Calculate the number of .trans files in the line list

create_ExoMol_tag_array(url, broad_URL)

Create a list of html tags that contain the URLs from which to later download files

iterate_ExoMol_tags(tags, l_folder, line_list)

Iterate through every html tag and download the file contained by the URL in the href.

find_input_dir(input_dir, database, molecule, isotope, ...)

Find the directory on a user's machine that contains the data needed to create a cross-section

parse_directory(directory, database)

Determine which linelist and isotopologue this directory contains data for (assumes data was downloaded using our script)

Module Contents

Cthulhu.downloader.download_ExoMol_file(url, f, l_folder)

Download a file from ExoMol and decompress it if needed.

Parameters:

  • url (String) – The URL of a given ExoMol file.

  • f (String) – The filename of the resulting downloaded file.

  • l_folder (String) – Local directory where the file will be stored.

Return type:

None.

Cthulhu.downloader.download_HITRAN_line_list(mol_ID, iso_ID, folder, nu_min=1, nu_max=100000)

Download line list using the fetch() function already in HITRAN.

Parameters:

  • mol_ID (int) – HITRAN molecular ID.

  • iso_ID (int) – HITRAN isotopologue ID.

  • folder (String) – Local directory where the line list is to be stored.

  • nu_min (int, optional) – Minimum wavenumber for which the line list is downloaded. The default is 1.

  • nu_max (int, optional) – Maximum wavenumber for which the line list is downloaded. The default is 100,000.

Return type:

None.

Cthulhu.downloader.HITEMP_table()

Recreate the table found on the HITEMP main page in order to simplify later processes

Returns:

hitemp – Recreated version of the table seen on the HITEMP main page (i.e., the table here: https://hitran.org/hitemp/)

Return type:

DataFrame

Cthulhu.downloader.download_HITEMP_line_list(mol_ID, iso_ID, out_folder)

Download a line list from the HITEMP database

Parameters:

  • mol_ID (int) – HITEMP molecule ID.

  • iso_ID (int) – HITEMP isotopologue ID.

  • out_folder (String) – Local directory where the line list is to be stored.

Return type:

None.

Cthulhu.downloader.check_HITEMP_file_exists(folder, file)

Check to see if a given HITEMP file already exists. This is to ensure that HITEMP downloads that stall midway through, for whatever reason, don’t have to restart from the first line list.

Parameters:

  • folder (String) – Local directory where file is to be stored.

  • file (String) – File name.

Returns:

Return a String which is either ‘par’, ‘hdf’, or ‘neither’ depending on which file type of this exists, if at all.

Return type:

String

Cthulhu.downloader.process_hdf_chunk(chunk, upper_ds, lower_ds, logA_ds, total_written)

Process chunk of data from convert_to_hdf

Parameters:

  • chunk (_type_) – _description_

  • upper_ds (_type_) – _description_

  • lower_ds (_type_) – _description_

  • logA_ds (_type_) – _description_

  • total_written (_type_) – _description_

Returns:

_description_

Return type:

_type_

Cthulhu.downloader.convert_to_hdf(file='', mol_ID=0, iso_ID=0, alkali=False, database='', chunk_size=5000000, compression_type='lzf')

Convert a given file to HDF5 format.

Parameters:

  • file (String, optional) – File name. The default is ‘’.

  • mol_ID (int, optional) – HITRAN/HITEMP molecule ID. The default is 0.

  • iso_ID (int, optional) – HITRAN/HITEMP isotopologue ID. The default is 0.

  • alkali (bool, optional) – Whether or not the species is an alkali metal. The default is False.

  • database (String, optional) – Database that the line list came from. The default is ‘’.

  • chunk_size (int, optional) – The size of the chunks to be read and written into the hdf5 file. The default is 5 million.

  • compression_type (String, optional) – The compression algorithm to be used during the writing to hdf5. The default is ‘lzf’, due to its speed.

Return type:

None.

Cthulhu.downloader.create_directories(molecule='', isotopologue='', line_list='', database='', mol_ID=0, iso_ID=0, ionization_state=1, VALD_data_dir='')

Create new folders on local machine to store the relevant data

Parameters:

  • molecule (String, optional) – Molecule name. The default is ‘’.

  • isotopologue (String, optional) – Isotopologue name. The default is ‘’.

  • line_list (String, optional) – Species line list. For HITRAN, HITEMP, and VALD, the line list is the same as the database. The default is ‘’.

  • database (String, optional) – Database the line list was derived from. The default is ‘’.

  • mol_ID (int, optional) – Molecular ID number as specified on HITRAN / HITEMP. The default is 0.

  • iso_ID (int, optional) – Isotopologue ID number as specified on HITRAN / HITEMP. The default is 0.

  • ionization_state (int, optional) – Ionization state of atomic species. The default is 1.

  • VALD_data_dir (String, optional) – Local directory VALD line list is stored in. The default is ‘’.

Returns:

line_list_folder – Local directory containing the line list.

Return type:

String

Cthulhu.downloader.calc_num_ExoMol_trans(html_tags)

Calculate the number of .trans files in the line list

Parameters:

html_tags (list) – HTML tag list containing download links (i.e. with the .trans, .broad, .pf, and .states files).

Returns:

counter – Number of .trans files in the line list.

Return type:

int

Cthulhu.downloader.create_ExoMol_tag_array(url, broad_URL)

Create a list of html tags that contain the URLs from which to later download files

Parameters:

  • url (String) – The ExoMol URL for the webpage that contains download links for the line list files.

  • broad_URL (String) – ExoMol URL which contains download links for the broadening files.

Returns:

combined_tags – HTML tags that contain URLs to download .broad, .pf, .states, and .trans files.

Return type:

list

Cthulhu.downloader.iterate_ExoMol_tags(tags, l_folder, line_list)

Iterate through every html tag and download the file contained by the URL in the href.

Parameters:

  • tags (list) – List containing all the html tags.

  • l_folder (String) – Local directory where the line list is to be stored.

  • line_list (String) – Name of line list to download.

Return type:

None.

Cthulhu.downloader.find_input_dir(input_dir, database, molecule, isotope, ionization_state, linelist)

Find the directory on a user’s machine that contains the data needed to create a cross-section

Parameters:

  • input_dir (String) – ‘Prefix’ of the directory containing the line list files. If the files were downloaded using our defaults scripts, input_dir will be ‘./input’ relative to the Cthulhu main folder.

  • database (String) – Database the line list was downloaded from.

  • molecule (String) – Molecule for which the cross-section is to be created.

  • isotope (String) – Isotopologue of the molecule for which the cross-section is to be created.

  • ionization_state (int) – Ionization state of the atom (neutral atom is 1).

  • linelist (String) – Line list that is being used. HITRAN/HITEMP/VALD used as the line list name for these databases respectively. ExoMol has its own named line lists.

Returns:

input_directory – Local directory containing line list and other important data (e.g. broadening files) for computing cross section.

Return type:

String

Cthulhu.downloader.parse_directory(directory, database)

Determine which linelist and isotopologue this directory contains data for (assumes data was downloaded using our script)

Parameters:

  • directory (String) – Local directory containing the line list file[s], broadening data, and partition function.

  • database (String) – Database line list is derived from.

Returns:

  • linelist (String) – Line list for which the cross-section is to be calculated.

  • isotopologue (String) – Molecular isotopologue for which the cross-section is to be calculated.