hipscat.pixel_math.partition_stats#
Utilities for generating and manipulating object count histograms
Module Contents#
Functions#
|
Use numpy to create an histogram array with the right shape, filled with zeros. |
|
Generate a histogram of counts for objects found in data |
|
Generate alignment from high order pixels to those of equal or lower order |
|
Generate mapping from destination pixel to all the constituent pixels. |
|
Generate alignment from high order pixels to those of equal or lower order |
|
Special case of creating a destination pixel map where you want to |
- empty_histogram(highest_order)[source]#
Use numpy to create an histogram array with the right shape, filled with zeros.
- Parameters:
highest_order (int) – the highest healpix order (e.g. 0-10)
- Returns:
one-dimensional numpy array of long integers, where the length is equal to the number of pixels in a healpix map of target order, and all values are set to 0.
- generate_histogram(data: pandas.DataFrame, highest_order, ra_column='ra', dec_column='dec')[source]#
Generate a histogram of counts for objects found in data
- Parameters:
data (
pd.DataFrame) – tabular object datahighest_order (int) – the highest healpix order (e.g. 0-10)
ra_column (str) – where in the input to find the celestial coordinate, right ascension
dec_column (str) – where in the input to find the celestial coordinate, declination
- Returns:
one-dimensional numpy array of long integers where the value at each index corresponds to the number of objects found at the healpix pixel.
- Raises:
ValueError – if the ra_column or dec_column cannot be found in the input file.
- generate_alignment(histogram, highest_order=10, lowest_order=0, threshold=1000000)[source]#
Generate alignment from high order pixels to those of equal or lower order
We may initially find healpix pixels at order 10, but after aggregating up to the pixel threshold, some final pixels are order 4 or 7. This method provides a map from pixels at order 10 to their destination pixel. This may be used as an input into later partitioning map reduce steps.
- Parameters:
histogram (
np.array) – one-dimensional numpy array of long integers where the value at each index corresponds to the number of objects found at the healpix pixel.highest_order (int) – the highest healpix order (e.g. 5-10)
lowest_order (int) – the lowest healpix order (e.g. 1-5). specifying a lowest order constrains the partitioning to prevent spatially large pixels.
threshold (int) – the maximum number of objects allowed in a single pixel
- Returns:
one-dimensional numpy array of integer 3-tuples, where the value at each index corresponds to the destination pixel at order less than or equal to the highest_order.
The tuple contains three integers:
order of the destination pixel
pixel number at the above order
the number of objects in the pixel
- Raises:
ValueError – if the histogram is the wrong size, or some initial histogram bins exceed threshold.
- generate_destination_pixel_map(histogram, pixel_map)[source]#
Generate mapping from destination pixel to all the constituent pixels.
- Parameters:
histogram (
np.array) – one-dimensional numpy array of long integers where the value at each index corresponds to the number of objects found at the healpix pixel.pixel_map (
np.array) – one-dimensional numpy array of integer 3-tuples. Seegenerate_alignment()for more details on this format.
- Returns:
dictionary that maps the integer 3-tuple of a pixel at destination order to the set of indexes in histogram for the pixels at the original healpix order
- compute_pixel_map(histogram, highest_order=10, lowest_order=0, threshold=1000000)[source]#
Generate alignment from high order pixels to those of equal or lower order
We may initially find healpix pixels at order 10, but after aggregating up to the pixel threshold, some final pixels are order 4 or 7. This method provides a map from destination healpix pixels at a lower order to the origin pixels at the highest order. This may be used as an input into later partitioning map reduce steps.
- Parameters:
histogram (
np.array) – one-dimensional numpy array of long integers where the value at each index corresponds to the number of objects found at the healpix pixel.highest_order (int) – the highest healpix order (e.g. 0-10)
lowest_order (int) – the lowest healpix order (e.g. 1-5). specifying a lowest order constrains the partitioning to prevent spatially large pixels.
threshold (int) – the maximum number of objects allowed in a single pixel
- Returns:
dictionary that maps the HealpixPixel (a pixel at destination order) to a tuple of origin pixel information.
The tuple contains the following:
0 - the total number of rows found in this destination pixel
1 - the set of indexes in histogram for the pixels at the original healpix order.
- Raises:
ValueError – if the histogram is the wrong size, or some initial histogram bins exceed threshold.
- generate_constant_pixel_map(histogram, constant_healpix_order)[source]#
Special case of creating a destination pixel map where you want to preserve some constant healpix order across the sky.
- NB:
this method filters out pixels with no counts
no upper thresholds are applied, and single pixel may contain many rows
- Parameters:
histogram (
np.array) – one-dimensional numpy array of long integers where the value at each index corresponds to the number of objects found at the healpix pixel.constant_healpix_order (int) – the desired healpix order (e.g. 0-10)
- Returns:
dictionary that maps non-empty bins as HealpixPixel to a tuple of origin pixel information.
The tuple contains the following:
0 - the total number of rows found in this destination pixel, same as the origin bin
1 - the set of indexes in histogram for the pixels at the original healpix order, which will be a list containing only the origin pixel.
- Raises:
ValueError – if the histogram is the wrong size