Cthulhu.downloader ================== .. py:module:: Cthulhu.downloader Functions --------- .. autoapisummary:: Cthulhu.downloader.download_ExoMol_file Cthulhu.downloader.download_HITRAN_line_list Cthulhu.downloader.HITEMP_table Cthulhu.downloader.download_HITEMP_line_list Cthulhu.downloader.check_HITEMP_file_exists Cthulhu.downloader.process_hdf_chunk Cthulhu.downloader.convert_to_hdf Cthulhu.downloader.create_directories Cthulhu.downloader.calc_num_ExoMol_trans Cthulhu.downloader.create_ExoMol_tag_array Cthulhu.downloader.iterate_ExoMol_tags Cthulhu.downloader.find_input_dir Cthulhu.downloader.parse_directory Module Contents --------------- .. py:function:: download_ExoMol_file(url, f, l_folder) Download a file from ExoMol and decompress it if needed. :param url: The URL of a given ExoMol file. :type url: String :param f: The filename of the resulting downloaded file. :type f: String :param l_folder: Local directory where the file will be stored. :type l_folder: String :rtype: None. .. py:function:: download_HITRAN_line_list(mol_ID, iso_ID, folder, nu_min=1, nu_max=100000) Download line list using the fetch() function already in HITRAN. :param mol_ID: HITRAN molecular ID. :type mol_ID: int :param iso_ID: HITRAN isotopologue ID. :type iso_ID: int :param folder: Local directory where the line list is to be stored. :type folder: String :param nu_min: Minimum wavenumber for which the line list is downloaded. The default is 1. :type nu_min: int, optional :param nu_max: Maximum wavenumber for which the line list is downloaded. The default is 100,000. :type nu_max: int, optional :rtype: None. .. py:function:: HITEMP_table() Recreate the table found on the HITEMP main page in order to simplify later processes :returns: **hitemp** -- Recreated version of the table seen on the HITEMP main page (i.e., the table here: https://hitran.org/hitemp/) :rtype: DataFrame .. py:function:: download_HITEMP_line_list(mol_ID, iso_ID, out_folder) Download a line list from the HITEMP database :param mol_ID: HITEMP molecule ID. :type mol_ID: int :param iso_ID: HITEMP isotopologue ID. :type iso_ID: int :param out_folder: Local directory where the line list is to be stored. :type out_folder: String :rtype: None. .. py:function:: check_HITEMP_file_exists(folder, file) Check to see if a given HITEMP file already exists. This is to ensure that HITEMP downloads that stall midway through, for whatever reason, don't have to restart from the first line list. :param folder: Local directory where file is to be stored. :type folder: String :param file: File name. :type file: String :returns: Return a String which is either 'par', 'hdf', or 'neither' depending on which file type of this exists, if at all. :rtype: String .. py:function:: process_hdf_chunk(chunk, upper_ds, lower_ds, logA_ds, total_written) Process chunk of data from convert_to_hdf :param chunk: _description_ :type chunk: _type_ :param upper_ds: _description_ :type upper_ds: _type_ :param lower_ds: _description_ :type lower_ds: _type_ :param logA_ds: _description_ :type logA_ds: _type_ :param total_written: _description_ :type total_written: _type_ :returns: _description_ :rtype: _type_ .. py:function:: convert_to_hdf(file='', mol_ID=0, iso_ID=0, alkali=False, database='', chunk_size=5000000, compression_type='lzf') Convert a given file to HDF5 format. :param file: File name. The default is ''. :type file: String, optional :param mol_ID: HITRAN/HITEMP molecule ID. The default is 0. :type mol_ID: int, optional :param iso_ID: HITRAN/HITEMP isotopologue ID. The default is 0. :type iso_ID: int, optional :param alkali: Whether or not the species is an alkali metal. The default is False. :type alkali: bool, optional :param database: Database that the line list came from. The default is ''. :type database: String, optional :param chunk_size: The size of the chunks to be read and written into the hdf5 file. The default is 5 million. :type chunk_size: int, optional :param compression_type: The compression algorithm to be used during the writing to hdf5. The default is 'lzf', due to its speed. :type compression_type: String, optional :rtype: None. .. py:function:: create_directories(molecule='', isotopologue='', line_list='', database='', mol_ID=0, iso_ID=0, ionization_state=1, VALD_data_dir='') Create new folders on local machine to store the relevant data :param molecule: Molecule name. The default is ''. :type molecule: String, optional :param isotopologue: Isotopologue name. The default is ''. :type isotopologue: String, optional :param line_list: Species line list. For HITRAN, HITEMP, and VALD, the line list is the same as the database. The default is ''. :type line_list: String, optional :param database: Database the line list was derived from. The default is ''. :type database: String, optional :param mol_ID: Molecular ID number as specified on HITRAN / HITEMP. The default is 0. :type mol_ID: int, optional :param iso_ID: Isotopologue ID number as specified on HITRAN / HITEMP. The default is 0. :type iso_ID: int, optional :param ionization_state: Ionization state of atomic species. The default is 1. :type ionization_state: int, optional :param VALD_data_dir: Local directory VALD line list is stored in. The default is ''. :type VALD_data_dir: String, optional :returns: **line_list_folder** -- Local directory containing the line list. :rtype: String .. py:function:: calc_num_ExoMol_trans(html_tags) Calculate the number of .trans files in the line list :param html_tags: HTML tag list containing download links (i.e. with the .trans, .broad, .pf, and .states files). :type html_tags: list :returns: **counter** -- Number of .trans files in the line list. :rtype: int .. py:function:: create_ExoMol_tag_array(url, broad_URL) Create a list of html tags that contain the URLs from which to later download files :param url: The ExoMol URL for the webpage that contains download links for the line list files. :type url: String :param broad_URL: ExoMol URL which contains download links for the broadening files. :type broad_URL: String :returns: **combined_tags** -- HTML tags that contain URLs to download .broad, .pf, .states, and .trans files. :rtype: list .. py:function:: iterate_ExoMol_tags(tags, l_folder, line_list) Iterate through every html tag and download the file contained by the URL in the href. :param tags: List containing all the html tags. :type tags: list :param l_folder: Local directory where the line list is to be stored. :type l_folder: String :param line_list: Name of line list to download. :type line_list: String :rtype: None. .. py:function:: find_input_dir(input_dir, database, molecule, isotope, ionization_state, linelist) Find the directory on a user's machine that contains the data needed to create a cross-section :param input_dir: 'Prefix' of the directory containing the line list files. If the files were downloaded using our defaults scripts, input_dir will be './input' relative to the Cthulhu main folder. :type input_dir: String :param database: Database the line list was downloaded from. :type database: String :param molecule: Molecule for which the cross-section is to be created. :type molecule: String :param isotope: Isotopologue of the molecule for which the cross-section is to be created. :type isotope: String :param ionization_state: Ionization state of the atom (neutral atom is 1). :type ionization_state: int :param linelist: Line list that is being used. HITRAN/HITEMP/VALD used as the line list name for these databases respectively. ExoMol has its own named line lists. :type linelist: String :returns: **input_directory** -- Local directory containing line list and other important data (e.g. broadening files) for computing cross section. :rtype: String .. py:function:: parse_directory(directory, database) Determine which linelist and isotopologue this directory contains data for (assumes data was downloaded using our script) :param directory: Local directory containing the line list file[s], broadening data, and partition function. :type directory: String :param database: Database line list is derived from. :type database: String :returns: * **linelist** (*String*) -- Line list for which the cross-section is to be calculated. * **isotopologue** (*String*) -- Molecular isotopologue for which the cross-section is to be calculated.