The random points are drawn with a density of 100 points per sq. arcmin, for each patch and each filter that have been processed by the pipeline. We draw the points and probe the mask information at each random position using a python module running within the HSC pipeline environment. The source code can be accessed on gitHub here: https://github.com/jcoupon/maskUtils.
The random points can be used for several purposes; (1) a random sample for clustering analysis, (2) to identify problematic areas, (3) to compute the survey area and the fraction of masked areas, etc. Available information carried by the random points, per filter and per patch, is:
- patch and tract number
- patch and tract geometry (non-overlapping regions: patch_inner and tract_inner)
- pixel mask bits such as flags_pixel_interpolated_any
- local PSF size: recorded as the second moment matrix (see FAQ: how to measure the size of an object, to convert into FWHM)
sky_mean: mean of sky value in 2″ diameter apertures (merge.footprint.sky == Trueobjects). Constant within patch.sky_std: standard deviation of sky value in 2″ diameter apertures (merge.footprint.sky == Trueobjects). Constant within patch. To compute the limiting magnitude at 5 sigma, do:lim_mag = -2.5 log10(5*sky_std) + 27.0adjust_densityparameter: a random number between 0 and 1 to adjust the sky density. A cut likeadjust_density < 0.5will decrease the sky density by a factor of two- number of visits contributed to coadds (countinputs)
pix_variance: pixel variance at random point position (note that the variance given for the object is the sum of the pixel variance over the source footprint).- Milky Way extinction value (Schlegel map).
Known issues
- centroid_sdss_flags and flags_badcentroid flags must be ignored because they are meaningless for randoms. If set to False, 85% of randoms are lost.
- The bright-star masks are too large for a few very bright stars and include about 10% of bright galaxies. See also the Known Problems page.
- Some functions of the object database are not available in the randoms database.
Examples
Query the database
-- This is a simple sql query to get all columns from the random database
-- in one single patch
SELECT *
FROM pdr1_wide.random AS ran
WHERE tract=8523
AND patch_s='3,5'
AND iflags_pixel_bright_object_center is not True
AND iflags_pixel_edge is not True
Do a joint object/random query
-- The object version.
SELECT object_id, ra, dec, imag_kron, imag_kron_err
FROM pdr1_wide.forced
WHERE boxSearch(coord, 34.0, 36.0, -5.0, -4.5)
AND NOT iflags_pixel_bright_object_center
AND NOT iflags_pixel_edge
AND icountinputs > 4
-- The random version.
-- Note that the boxSearch function is not available in
-- the random database.
-- For the randoms, the "NOT iflags_pixel_edge" is mandatory
-- near the border of the survey.
SELECT object_id, ra, dec
FROM pdr1_wide.random
WHERE ra BETWEEN 34.0 AND 36.0
AND dec BETWEEN -5.0 AND -4.5
AND NOT iflags_pixel_bright_object_center
AND NOT iflags_pixel_edge
AND icountinputs > 4
Add a cut on PSF size
Note that in the random database, the moments are in pixel, whereas in the object data, it's in arcsec (hence the 0.17 factor for the random). The factor 2sqrt(2) is to convert from sigma to FWHM, assuming the PSF is gaussianrandom: shape_detradius(array[ishape_sdss_psf_11, ishape_sdss_psf_22, ishape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 < 1.0
object: shape_detradius(array[ishape_sdss_psf_11, ishape_sdss_psf_22, ishape_sdss_psf_12])*2.0*sqrt(2.0) < 1.0
Compute the survey area
The random point density is 100 per arcmin2. Thus the area in deg2 is simply the number of random points given a selection:SELECT COUNT(*)/100.0/3600.0
FROM pdr1_udeep.random
WHERE icountinputs > 10;
Adjust the random local density
The "adjust_density" is a random number in the interval [0:1]. For example, to divide the random point density by a factor of two (to get 50 /arcmin2), set:iflag_adjust_density < 0.5
Compute the patch mean sky and sky variance
The mean and standard deviation of the sky object in 2" diameter aperture per patch is stored as sky_mean and sky_std. To get these quantities per patch and compute the limiting magnitude, e.g. at 5 sigma, one may do this directly using the database-- This query gets the patch mean sky and variance
SELECT ran.isky_mean, ran.isky_std, -2.5*LOG(5.0*ran.isky_std)+27.0 as imag_limit
FROM pdr1_wide.random AS ran
WHERE ran.tract=8523
AND ran.patch_s='3,5'
LIMIT
1
Compute the masked fraction
-- This query returns the masked fraction in bright object masks for tract 8523.
SELECT 1.0 - CAST(sum(
CASE WHEN iflags_pixel_edge is not True AND iflags_pixel_bright_object_center is not True
THEN 1
ELSE 0
END ) AS DECIMAL )
/ CAST(sum(
CASE WHEN iflags_pixel_edge is not True
THEN 1
ELSE 0 END ) AS DECIMAL)
as masked_fraction
FROM pdr1_wide.random AS ran
WHERE tract=8523
Identify areas with large seeing
-- This query gets 1% of random objects in the entire Wide and record the position as well as the local seeing FWHM.
SELECT ran.ra, ran.dec,
shape_detradius(array[ran.gshape_sdss_psf_11, ran.gshape_sdss_psf_22, ran.gshape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 as gPSF,
shape_detradius(array[ran.rshape_sdss_psf_11, ran.rshape_sdss_psf_22, ran.rshape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 as rPSF,
shape_detradius(array[ran.ishape_sdss_psf_11, ran.ishape_sdss_psf_22, ran.ishape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 as iPSF,
shape_detradius(array[ran.zshape_sdss_psf_11, ran.zshape_sdss_psf_22, ran.zshape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 as zPSF,
shape_detradius(array[ran.yshape_sdss_psf_11, ran.yshape_sdss_psf_22, ran.yshape_sdss_psf_12])*2.0*sqrt(2.0)*0.17 as yPSF
from pdr1_wide.random AS ran
where iadjust_density < 0.01