hats.io.paths ============= .. py:module:: hats.io.paths .. autoapi-nested-parse:: Methods for creating partitioned data paths .. !! processed by numpydoc !! Attributes ---------- .. autoapisummary:: hats.io.paths.PARTITION_ORDER hats.io.paths.PARTITION_DIR hats.io.paths.PARTITION_PIXEL hats.io.paths.MARGIN_ORDER hats.io.paths.MARGIN_DIR hats.io.paths.MARGIN_PIXEL hats.io.paths.HIVE_COLUMNS hats.io.paths.DATASET_DIR hats.io.paths.PARTITION_INFO_FILENAME hats.io.paths.PARQUET_METADATA_FILENAME hats.io.paths.PARQUET_COMMON_METADATA_FILENAME hats.io.paths.DATA_THUMBNAIL_FILENAME hats.io.paths.POINT_MAP_FILENAME hats.io.paths.SKYMAP_FILENAME Functions --------- .. autoapisummary:: hats.io.paths.dataset_directory hats.io.paths.pixel_directory hats.io.paths.get_healpix_from_path hats.io.paths.dict_to_query_urlparams hats.io.paths.pixel_catalog_file hats.io.paths.new_pixel_catalog_file hats.io.paths.get_partition_info_pointer hats.io.paths.get_common_metadata_pointer hats.io.paths.get_parquet_metadata_pointer hats.io.paths.get_data_thumbnail_pointer hats.io.paths.get_point_map_file_pointer hats.io.paths.get_skymap_file_pointer Module Contents --------------- .. py:data:: PARTITION_ORDER :value: 'Norder' .. py:data:: PARTITION_DIR :value: 'Dir' .. py:data:: PARTITION_PIXEL :value: 'Npix' .. py:data:: MARGIN_ORDER :value: 'margin_Norder' .. py:data:: MARGIN_DIR :value: 'margin_Dir' .. py:data:: MARGIN_PIXEL :value: 'margin_Npix' .. py:data:: HIVE_COLUMNS .. py:data:: DATASET_DIR :value: 'dataset' .. py:data:: PARTITION_INFO_FILENAME :value: 'partition_info.csv' .. py:data:: PARQUET_METADATA_FILENAME :value: '_metadata' .. py:data:: PARQUET_COMMON_METADATA_FILENAME :value: '_common_metadata' .. py:data:: DATA_THUMBNAIL_FILENAME :value: 'data_thumbnail.parquet' .. py:data:: POINT_MAP_FILENAME :value: 'point_map.fits' .. py:data:: SKYMAP_FILENAME :value: 'skymap.fits' .. py:function:: dataset_directory(catalog_base_dir: str | pathlib.Path | upath.UPath | None) -> upath.UPath Create path pointer for the dataset directory. This will not create the directory. The directory name will take the HiPS standard form of:: /dataset/ :Parameters: **catalog_base_dir** : str | Path | UPath | None base directory of the catalog (includes catalog name) :Returns: UPath dataset directory name .. !! processed by numpydoc !! .. py:function:: pixel_directory(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel_order: int, pixel_number: int | None = None, directory_number: int | None = None) -> upath.UPath Create path pointer for a pixel directory. This will not create the directory. One of pixel_number or directory_number is required. The directory name will take the HiPS standard form of:: /dataset/Norder=/Dir= Where the directory number is calculated using integer division as:: (pixel_number/10000)*10000 :Parameters: **catalog_base_dir** : str | Path | UPath | None base directory of the catalog (includes catalog name) **pixel_order** : int the healpix order of the pixel **pixel_number** : int | None the number of the healpix pixel at ``pixel_order`` **directory_number** : int | None directory number (or inferred from pixel number) :Returns: UPath directory name .. !! processed by numpydoc !! .. py:function:: get_healpix_from_path(path: str) -> hats.pixel_math.healpix_pixel.HealpixPixel Find the `pixel_order` and `pixel_number` from a string like the following:: Norder=/Dir=/Npix=.parquet NB: This expects the format generated by the `pixel_catalog_file` method :Parameters: **path** : str path to parse :Returns: HealpixPixel Constructed HealpixPixel object representing the pixel in the path. ``INVALID_PIXEL`` if the path doesn't match the expected pattern for any reason. .. !! processed by numpydoc !! .. py:function:: dict_to_query_urlparams(query_params: dict | None = None) -> str Converts a dictionary to a url query parameter string :Parameters: **query_params: dict | None** (Default value = None) set of URL query parameters. :Returns: str query parameter string to append to a url .. !! processed by numpydoc !! .. py:function:: pixel_catalog_file(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel: hats.pixel_math.healpix_pixel.HealpixPixel, query_params: dict | None = None, npix_suffix: str = '.parquet') -> upath.UPath Create path *pointer* for a pixel catalog file. This will not create the directory or file. The catalog file name will take the HiPS standard form of:: /Norder=/Dir=/Npix=.parquet Where the directory number is calculated using integer division as:: (pixel_number/10000)*10000 :Parameters: **catalog_base_dir** : str | Path | UPath | None base directory of the catalog (includes catalog name) **pixel** : HealpixPixel the healpix pixel to create path to **query_params: dict | None** (Default value = None) Params to append to URL. Ex:: {'cols': ['ra', 'dec'], 'fltrs': ['r>=10', 'g<18']} **npix_suffix: str** (Default value = ".parquet") extension for the parquet file (or `/` if a directory) :Returns: UPath catalog file name .. !! processed by numpydoc !! .. py:function:: new_pixel_catalog_file(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel: hats.pixel_math.healpix_pixel.HealpixPixel, create_dirs: bool = True, npix_suffix: str = '.parquet', npix_parquet_name: str | None = None) -> upath.UPath Create path for a single new pixel catalog file. In the event that you are writing to a per-pixel directory, you will also need a name for the new parquet file. The catalog file name will take the HiPS standard form of:: /Norder=/Dir=/Npix=.parquet Where the directory number is calculated using integer division as:: (pixel_number/10000)*10000 :Parameters: **catalog_base_dir** : str | Path | UPath | None base directory of the catalog (includes catalog name) **pixel** : HealpixPixel the healpix pixel to create path to **create_dirs: bool** if True, create all parent directories, including the per-pixel directory when ``npix_suffix == "/"``. Default is True. **npix_suffix: str** suffix appended to the ``Npix=`` component. For regular files, this is typically a file extension such as ``".parquet"``. If set to ``"/"``, ``Npix=`` is treated as a directory and the returned path points to a file within that directory. **npix_parquet_name: (str)** name of the parquet file inside the per-pixel directory when ``npix_suffix == "/"``. Defaults to ``"Npix=.parquet"``. Ignored unless ``npix_suffix == "/"``. :Returns: UPath catalog file name .. !! processed by numpydoc !! .. py:function:: get_partition_info_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) -> upath.UPath Get file pointer to ``partition_info.csv`` metadata file :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) :Returns: UPath File Pointer to the catalog's ``partition_info.csv`` file .. !! processed by numpydoc !! .. py:function:: get_common_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) -> upath.UPath Get file pointer to `_common_metadata` parquet metadata file :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) :Returns: UPath File Pointer to the catalog's `_common_metadata` file .. !! processed by numpydoc !! .. py:function:: get_parquet_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) -> upath.UPath Get file pointer to `_metadata` parquet metadata file :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) :Returns: UPath File Pointer to the catalog's `_metadata` file .. !! processed by numpydoc !! .. py:function:: get_data_thumbnail_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) -> upath.UPath Get file pointer to `data_thumbnail` parquet file :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) :Returns: UPath File Pointer to the catalog's `data_thumbnail` file .. !! processed by numpydoc !! .. py:function:: get_point_map_file_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) -> upath.UPath Get file pointer to `point_map.fits` FITS image file. :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) :Returns: UPath File Pointer to the catalog's `point_map.fits` FITS image file. .. !! processed by numpydoc !! .. py:function:: get_skymap_file_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath, order: int | None = None) -> upath.UPath Get file pointer to `skymap.fits` or `skymap.K.fits` FITS image file. :Parameters: **catalog_base_dir: str | Path | UPath** base directory of the catalog (includes catalog name) **order: int | None** (Default value = None) desired order for the map, if looking for a down-sampled map. :Returns: UPath File Pointer to the FITS image file. .. !! processed by numpydoc !!