==================== IVOA Obscore Extension for Radio data ==================== :Status: ObsCoreExtensionForRadioData 1.0 PEN 2025-09-15 .. role:: raw-latex(raw) :format: latex .. .. _`ObsCoreExtensionForRadioData:acknowledgments`: Acknowledgments =============== The authors would like to thank all the participants in DM-WG and Radioastronomy-IG discussions for their ideas, critical reviews, and contributions to this document. We acknowledge also the support of ESCAPE (European Science Cluster of Astronomy and Particle Physics ESFRI Research Infrastructures) funded by the EU Horizon 2020 research and innovation program (Grant Agreement no 824064). .. _`ObsCoreExtensionForRadioData:introduction`: 1 Introduction ============== :doc:`ObsCore <../ObsCore/ObsCore>` specification :doc:`ObsCore <../ObsCore/ObsCore>` defines both a minimal datamodel to describe datasets and a table consistent with the model which can be served by :doc:`TAP <../TAP/TAP>` services. It has been successful to define a lot of data discovery services in astronomy. The emergence of the Radioastronomy Interest Group in the IVOA in April 2020 confirmed the strong interest of the radio astronomy community to distribute their data in the VO. Many teams now distribute their data using VO standards [1]_. While reduced radio data products, such as images or spectral cubes, are mostly covered by the :doc:`ObsCore <../ObsCore/ObsCore>` model, the lower level observational data (interferometric visibilities, single dish data in SDFITS, filterbank or whatever other specific formats) require additional description parameters. For interferometry, this need was already exposed in 2010 by Anita Richards in a IVOA Note "Radio interferometry data in the VO" [2]_ which captures precisely the requirements in the radio community. Various discovery use cases have been collected in the radio community and gathered in the :ref:`ObsCoreExtensionForRadioData:ADQLusecases` appendix. The current specification suggests addition of new features in the :doc:`ObsCore <../ObsCore/ObsCore>` metadata profile to fill the gap. With the expansion of large radio astronomy projects such as LOFAR, NenuFAR, the future SKA, ngVLA, etc... and the emergence of interesting research topics matching data in all electromagnetic regimes, the Virtual Observatory framework can facilitate a wider access to radio data for experts and non-specialists radio astronomers in order to support collaborations in multi-wavelength, multi-messenger astronomy. .. container:: float :name: ObsCoreExtensionForRadioData: `image `__ :raw-latex:`\label{fig:architecture}` This Radio :doc:`ObsCore <../ObsCore/ObsCore>` extension in the IVOA, relies on :doc:`ObsCore <../ObsCore/ObsCore>` itself and expands it to allow data providers to describe their radio data further the ObsTAP framework. Its goal is to clarify how :doc:`ObsCore <../ObsCore/ObsCore>` metadata can be used in the radio context and to add new specific features to the existing :doc:`ObsCore <../ObsCore/ObsCore>` metadata. .. _`ObsCoreExtensionForRadioData:sec:specificities`: 2 Radio data specifities from the Data Discovery point of view ============================================================== On the lower end of the radio spectrum, radio astronomers generally make use of frequencies for designating the spectral ranges of their observation. The standard :doc:`ObsCore <../ObsCore/ObsCore>` attributes *em_min, em_max* are expressed in wavelength and are not really convenient. That’s why we should also provide a mechanism for translation into frequencies .But this should not be done by duplicating the same information in two different attributes. Receivers with a (ultra)wide bandwidth, up to tens of GHz, are nowadays commonly used for both interferometric and Single Dish (herefater SD) radio observations. Given that the spatial field of view and resolution linearly depend on wavelength, these quantities may significantly vary across the observed bandwidth in a radio observation. Generally only a representative value (for instance the receiver nominal frequency) for these two parameters can be given. It is noticeable that this is the case for any measuring system allowing a large interval of :math:`\lambda/D` (where :math:`\lambda` and :math:`D` are the wavelength and the measuring system aperture scale). Similarly, the resolution power quantity, commonly provided to describe optical spectroscopic data, is generally not used in the radio domain. Instead one could introduce a new :doc:`ObsCore <../ObsCore/ObsCore>` element for the absolute spectral resolution, in frequency unit, for which a representative value for each observation can be given. Modern radio instrumentation offers the possibility of several spectral windows within the same observation with significant separation or different resolutions. Such observations may be represented at the highest granularity as a set of combined data sets represented by several entries in an :doc:`ObsCore <../ObsCore/ObsCore>` table. However it’s up to data provider to decide which level of granularity is best adapted in order to optimize data discoverability and ease data access, depending on the scientific content of the observation. .. _`ObsCoreExtensionForRadioData:subsec:sd`: 2.1 Single dish data -------------------- Single Dish (SD) observations can be done with different types of receiving systems. Typical frontends are mono-feed, multi-feed and phased array feed (PAF), the last two suitable to efficiently span wider parts of the sky. Data can be acquired by various backend systems providing either the total intensity (integrated over the whole available bandwidth) or the spectroscopic/spectropolarimetric intensity (acquired in each spectral channel/sample). Data are saved as raw counts resulting from the digitization of the voltage signal measured by the receiving system. Commonly-used SD data formats are registered FITS standard conventions (FITS, SDFITS and MBFITS) or unregistered conventions like the standard FITS-based format delivered by the INAF radio telescopes. The combination of telescope, frontend and backend permits the realization of various observing strategies characterized by specific spatial and/or spectral patterns. Typical SD observing strategies are: ``on-source``, ``frequency switching``, ``on-off`` observations, ``raster`` or ``on-the-fly`` (OTF) mapping, ``on-the-fly-cross-scan``, ``skydip`` calibrations (see Fig :ref:`ObsCoreExtensionForRadioData:fig:SD`). For each spatial position in the observation, SD data gather emission for any of the spectral samples in the given frequency band and polarization. If multi-feed/PAFs are used, a set of spatial positions are simultaneously measured. Scan modes should be described in :doc:`ObsCore <../ObsCore/ObsCore>` in order to allow astronomers to better understand the structure of the data which will be retrieved. Spatial resolution in the SD case is intended as the beam size. This holds true for any type of receivers, since also for multi-feed/PAF ones the spatial resolution is ruled by the size of the individual beam. Contrary to what usually happens for interferometric observations, for some radio telescopes a SD observation (scan) contains only one scientific target (for example INAF ones). In any case, each target in an observation is listed as a separate entry in an :doc:`ObsCore <../ObsCore/ObsCore>` table sharing the same *obs_id*. Complex frequency setups are possible in the same observation, as already mentioned in Sect. :ref:`ObsCoreExtensionForRadioData:sec:specificities`. The :doc:`ObsCore <../ObsCore/ObsCore>` parameter *t_resolution*, defined as the minimal interpretable interval between two points along the time axis (being it an average or representative value), has a limited application for SD data except for ``on-source`` tracking observations like those for pulsar/FRB studies. Typically, time is not an independent variable in SD measurements and it can be saved together with spatial/spectral/intensity information as a timestamp associated to each data sample. A more comprehensive discussion on :doc:`ObsCore <../ObsCore/ObsCore>` parameters for time-domain data is given in the Pulsar and FRB Radio Data Discovery and Access IVOA Note [3]_. .. container:: float :name: ObsCoreExtensionForRadioData:fig:SD .. raw:: latex \centering |image1| .. _`ObsCoreExtensionForRadioData:sec:visibility`: 2.2 Visibility data -------------------- Visibility data are sets of complex numbers corresponding to the amplitude and phase of correlation coefficients measured between pair of antennas (i.e., a baseline), at a given time, a given wavelength or polarisation. The visibility data are a sparse representation of the observed sky. The visibility data sets can be processed to obtain interferometric images, through inverse Fourier algorithms. Each visibility measurement corresponds to an interferometric fringe system on the sky. The imaging algorithms include a calibration step allowing to set the center of the reconstructed image, setting this direction as a phase reference. The visibilities are then usually represented in a spatial frequency plane, called the *uv* plane, whose orientation is perpendicular to phase reference direction. The instantaneous PSF (Point Spread Function) of an interferometer is the Fourier transform of all baselines sampled in the *uv* plane. Hence, the quality of the reconstructed images are directly related to the set of baselines used for the measurements. Visibility data are usually organized as sets of matrices for various phase references (i.e., pointing, or fields) and configuration of the baselines, such as their distances and orientations. Such matrices may or may not be regularly sampled in time, wavelength and polarisation. As for any other observation product described with :doc:`ObsCore <../ObsCore/ObsCore>`, the description may be split into several records in the :doc:`ObsCore <../ObsCore/ObsCore>` table, when :doc:`ObsCore <../ObsCore/ObsCore>` parameters cannot represent the variety of the observation results coverage (e.g., if there are several observed “fields”, requiring different *s_ra* and *s_dec* value, or various groups of spectral bands, etc.) We consider that consistent :doc:`ObsCore <../ObsCore/ObsCore>` records as described above define datasets with a dataproduct type set to “visibility”. Contrary to what occurs with direct imaging observations, the PSF of the interferometer is filtering spatial scales (large scales, when the small baselines are insufficiently sampled; and vice versa for small scales with long baselines). For large spectral ranges, the variations of the field of view and the spatial resolution along the axis may become so large that the typical value cannot be sufficient to characterize the dataset. Ranges of values for such parameters are required to accurately describe such datasets. The quality of the data strongly depends from the distribution of the visibility measurements in the *uv* plane : the more complete the *uv* sampling plane, the better the reconstructed image. The *uv* plane distribution can be characterized by several numbers. The minimal and maximum distance between measurements in the *uv* plane provide assessments for spatial resolution and largest angular scale. Beside this a *uv* plane filling factor of the distribution will allow to predict the quality of reconstruction of the image in the distance plane (sky). Eventually, the ellipticity of the distribution is a measure of the distortions that can affect the reconstruction. Radio astronomers also check the quality of the visibility data by looking at some maps of the data structure. The *uv* coverage map can show how complete and regular is the sampling in the *uv* plane and give an hint of resolution and maximum angular scale. The visualisation of the dirty beam, which is the Fourier transform of the *uv* sampling function gives an hint of the intrinsic quality of possible reconstruction. As maps they are not queriable. So links to these kind of maps will not be exposed in the extension table but only via a :doc:`DataLink <../DataLink/DataLink>` service. If none of these *uv* characterization features are available to be exposed in the service we can still predict ranges of some of those by using parameters of the instrumental configuration. Important features are the antenna diameter (or maximum antenna diameter), the number of antennas and the minimum and maximum distance between antennas of the array. In addition to these specifities most of the scan modes shown on figure :ref:`ObsCoreExtensionForRadioData:fig:SD` also apply to some interferometry observations and should be described. .. _`ObsCoreExtensionForRadioData:sec:ObsCoreRadDef`: 3 ObsCore attributes definition valid for radio data ==================================================== For radio data some of the definitions on :doc:`ObsCore <../ObsCore/ObsCore>` data model attributes need to be adjusted to fit the peculiarity of metadata for datasets partition, *uv* space, etc. Here is a list of common :doc:`ObsCore <../ObsCore/ObsCore>` parameters already available for the radio data discovery. .. _`ObsCoreExtensionForRadioData:obs_id`: 3.1 obs_id ---------- Astronomers usually know what they identify as a single observation: a complex set of measurements made in a given sequence of time. *obs_id* should define unambiguously each observation. It is provided by the observation pipeline to identify what is collected for one observation operation. .. _`ObsCoreExtensionForRadioData:obs_publisher_did`: 3.2 obs_publisher_did --------------------- Radio data observations can be split in several subparts with homogeneous spatial, time, spectral coverage intervals, spectral resolution, etc. Each part can be described by a single dataset and has its own *obs_publisher_did*. It has to be unique in the Virtual Observatory domain. It identifies a dataset with homogeneous properties in terms of coverage on all physical axes : temporal, spectral, spatial. .. _`ObsCoreExtensionForRadioData:sec:fov`: 3.3 s_fov --------- This attribute measures the size of the field covered. It usually depends on the spectral interval and of the telescope diameter. A typical value for the size of the field of view is to be computed on the observation by taking into account the sky scan geometry and the receiver type in use. *s_fov* coincides with the instantaneous field of view :math:`\lambda / D` only for pointed observations (for instance, an ON in the SD case) obtained with a mono-feed receiver. In this case, :math:`\lambda` is the receiver nominal wavelength and D coincides with the telescope diameter (SD case) or the largest diameter of the array antennae or telescopes (interferometric case). In interferometry, the correlator can also restrict the fov depending on the trade-off set to build the signal. Nominal wavelength SHOULD be taken as the mid value of the spectral range except if data providers have good reasons to propose another value which should be documented in the FIELD DESCRIPTION tag in that case. .. _`ObsCoreExtensionForRadioData:sec:res`: 3.4 s_resolution ---------------- In the case of SD using mono- or multi-feed/PAF receivers this is the beam size inferred from the wavelength and telescope diameter. In the case of interferometry, a typical value for the spatial resolution will be given by :math:`\lambda / L` where :math:`\lambda` is the receiver nominal wavelength and L is the longest distance in the *uv* plane. As above nominal wavelength SHOULD be taken as the mid value of the spectral range except if data providers want for specific reasons. For beamforming applied to SD *s_resolution* is set by the size of one individual electronically-formed beam. .. _`ObsCoreExtensionForRadioData:s_region`: 3.5 s_region ------------ The shape of the covered region. For single dish data it will strongly depend on the scanning mode and type of receiver in use. This shape will be the typical contour of the detectable beam for interferometry. Of course it cannot be accurate. .. _`ObsCoreExtensionForRadioData:o_ucd`: 3.6 o_ucd --------- This is UCD string to qualify what is the observable quantity varying along the axes. In the current UCD vocabulary (UCD1+ controlled vocabulary - Updated List of Terms Version 1.5) there appear to be no primary words suitable to describe raw SD data. *o_ucd=phot.flux.density* does not seem appropriate, since the single dish measured quantity is expressed in raw counts coming from the digitization of a voltage signal generated in the receiver chain by the incoming electromagnetic field. *o_ucd=phys.voltage* is validated for addition into the next version for UCD1+ controlled vocabulary - Updated List of Terms Version 1.6. In the case of visibility data the "observable" is a complex number representing Fourier coefficients of the image Fourier transform. Its UCD string is *stat.fourier*. .. _`ObsCoreExtensionForRadioData:t_exptime`: 3.7 t_exptime ------------- Total duration of the observation for the given dataset or :doc:`ObsCore <../ObsCore/ObsCore>` entry. For instance in case of multiple targets, *t_exptime* will be computed for each source and written in the appropriate :doc:`ObsCore <../ObsCore/ObsCore>` Table entry. The specific case of time series is addressed in another specification [4]_. .. _`ObsCoreExtensionForRadioData:t_resolution`: 3.8 t_resolution ---------------- The :doc:`ObsCore <../ObsCore/ObsCore>` parameter *t_resolution* (see Sect. :ref:`ObsCoreExtensionForRadioData:subsec:sd`) has a limited application for SD data except for ``on-source`` tracking observations like those for pulsar/FRB studies and could be set to the exposure time or could be NULL. For time-domain data, *t_resolution* can be set according to the Pulsar and FRB Radio Data Discovery and Access IVOA Note [5]_. For interferometric observations it is the integration time set at the correlation level. .. _`ObsCoreExtensionForRadioData:dataproduct_type-and-dataproduct_subtype`: 3.9 dataproduct_type and dataproduct_subtype -------------------------------------------- Radio astronomy data cover a wide variety of data product types from visibility for raw interferometry data to cubes, images, spectra, time series or even measurements (in the case of single dish on scan mode). Single dish observations in some modes show specifities which are not covered by the current :doc:`ObsCore <../ObsCore/ObsCore>` *dataproduct_type* vocabulary. This is the case of spatial profiles obtained with on the fly cross scan or of the tables of flux measurements obtained on a regular spatial grid but with specific time stamp for each spot as in the raster map mode. A new external standard IVOA vocabulary is currently defined for data product types [6]_ and tackles some of these specificities. However some of them SHOULD be covered in the *dataproduct_subtype* attribute if no new term is introduced in the standard vocabulary. .. _`ObsCoreExtensionForRadioData:obscore-extension-specific-for-radio-data`: 4 ObsCore extension specific for radio data =========================================== Tables :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt`, :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt_interferometry` and :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt_instrumental` show the querying parameters we propose to include into the :doc:`ObsCore <../ObsCore/ObsCore>` radio extension table in order to better describe radio single dish and visibility data. .. _`ObsCoreExtensionForRadioData:spatial-parameters`: 4.1 Spatial parameters ---------------------- | For extended spectral range datasets *s_fov_min, s_fov_max* are estimated like in the typical value case (see subsection :ref:`ObsCoreExtensionForRadioData:sec:fov`). In the case of SD pointed observations with mono-feed receivers and the majority of interferometric observations the minimum and maximum :math:`\lambda` values in the spectral range(s) will be used in the formula :math:`\lambda / D` to estimate respectively *s_fov_min* and *s_fov_max*. | In the case of mapping scans or multi-feed/PAF receivers *s_fov_min* and *s_fov_max* are derived as the minimum and maximum sizes of the circular region encompassing the covered area. *s_resolution_min, s_resolution_max* are estimated like the typical value by the formula :math:`\lambda / L` (see subsection :ref:`ObsCoreExtensionForRadioData:sec:res`) where :math:`\lambda` is replaced respectively by the minimum and maximum wavelength of the spectral range(s). The size L is the telescope diameter for SD observations and the largest distance in the *uv* plane for interferometry. Beam forming may represent an exception to this rule, see :ref:`ObsCoreExtensionForRadioData:sec:res`. In the case of interferometry, we introduce the new *s_largest_angular_scale* which is estimated as :math:`\lambda/l` where :math:`\lambda` is the typical wavelength (and again typical value SHOULD be estimated as the mid value of the spectral range apart from documented exceptions) and l is the typical smallest distance in the *uv* plane. This parameter is not relevant for other observation modes. The largest angular scale is also variable along the spectral range. That’s why we bound it with *s_largest_angular_scale_min* and *s_largest_angular_scale_max* estimated as respectively :math:`\lambda\_min/l` and :math:`\lambda\_max/l` .. _`ObsCoreExtensionForRadioData:frequency-characterization`: 4.2 Frequency characterization ------------------------------ As was stated above (:ref:`ObsCoreExtensionForRadioData:sec:specificities`) radio astronomers use frequency quantities to characterize their datasets. Therefore we introduce one additional parameters in the extension : *f_resolution* for absolute spectral resolution, which is a more stable parameter than the resolution power in the radio domain. For users willing to access spectral ranges in frequencies we can imagine several scenarii: - compute two free parameters *f_min* and *f_max* this way *f_min* = c / *em_max* and *f_max* = c / *em_min* with c = 299 792 458 m/s - express queries and results in terms of frequencies by using the same formulae in the :doc:`ADQL <../ADQL/ADQL>` queries (see :ref:`ObsCoreExtensionForRadioData:sec:FreqRanges`) - let the interface do these conversions Using the :doc:`ADQL <../ADQL/ADQL>` User Defined Functions :doc:`udf-catalogue <../udf-catalogue/udf-catalogue>` in queries for unit conversion as well as *ivo_interval_overlaps, ivo_specconv* would simplify the interface for the user and avoid columns duplication for the spectral coverage . .. _`ObsCoreExtensionForRadioData:spatial-frequency-coverage-for-visibilities`: 4.3 Spatial frequency coverage for visibilities ------------------------------------------------ These parameters are valid for interferometry only. The absolute *uv*\ \_distance_min and *uv*\ \_distance_max in the *uv* plane give some outlier minimum and maximum scale in some direction. To compute the ellipse’s eccentricity of the *uv* distribution a principal component analysis (PCA) with 2 components is performed over the data points sampling the *uv* plane to select the main axis of data scattering. The first component is used to rotate the distribution of uv in a way that the major variation of the distribution is leaning towards the :math:`x` axis of a bi dimensional :math:`xy` Cartesian plane. The major axis length and the minor axis length of the ellipse are therefore defined as the semi distance between the most positive point along the :math:`x`/:math:`y` axis and the most negative point among the :math:`y` axis. For instance, if the range of the rotated UV will cover on the :math:`x \in [-10, 10]` the major axis distance would be 10, a similar procedure is done on the y axis. This procedure allows the definition of the *uv* distribution eccentricity, *uv_distribution_ecc* computed as follows: .. math:: \begin{equation} uv\_distribution\_ecc = \sqrt{1-\frac{b^2}{a^2}} \end{equation} where a is the major axis length and b is the minor axis length. The filling factor of the *uv* plane (hereafter *uv_distribution_fill*) is computed as the average number of samples found in a :math:`N^{uv}_{samples}`\ x\ :math:`N^{uv}_{samples}` equi-spaced grid enclosing the rotated ellipse. In formulas, the boundaries of a cell (i,j) are defined by the boundaries .. math:: \begin{equation} u \in [u_{min} + \frac{u_{max} - u_{min}}{N^{uv}_{samples}} \cdot i , u_{min} + \frac{u_{max} - u_{min}}{N^{uv}_{samples}} \cdot (i + 1)] \end{equation} and .. math:: \begin{equation} v \in [v_{min} + \frac{v_{max} - v_{min}}{N^{uv}_{samples}} \cdot j , v_{min} + \frac{v_{max} - v_{min}}{N^{uv}_{samples}} \cdot (j + 1)] \end{equation} where :math:`u_{max}`/:math:`v_{max}` are the respective maximum u/v of the *uv* sample and :math:`u_{min}`/:math:`v_{min}` is the minimum u/v of the *uv* sample. Given the above boundaries the number of samples within a cell (i,j) will be :math:`n^{uv}_{i,j}` and uv_distribution_fill will be then computed as .. math:: \begin{equation} uv\_distribution\_fill = \frac{\sum^{N^{uv}_{samples}}_{i=1} \sum^{N^{uv}_{samples}}_{j=1} n^{uv}_{i,j} }{(N^{uv}_{samples}) ^ 2}, \end{equation} in the preliminary analysis :math:`N^{uv}_{samples} = 1000`. .. _`ObsCoreExtensionForRadioData:observational-configuration-and-instrumental-parameters`: 4.4 Observational configuration and instrumental parameters ----------------------------------------------------------- These parameters are intended to describe the main telescope(s) characteristics for both SD antennas and interferometers. Such instrumental characteristics give an approximate idea on the spanned angular scales, field of view, product types, etc. The more global parameter to define is the instrument type allowing to discriminate single dish observations from interferometry or beam forming ones. Parameters *instr_tel_number, instr_tel_min_dist* and *instr_tel_max_dist* are related to interferometers only while *instr_tel_diameter, instr_feed* are valid also for SD. We note that *instr_feed* could also account for the number of beams in the case of a beam forming/PAF receiver. The scanning strategy adopted in an observation is described by the parameter *scan_mode*. This parameter covers both spatial and frequency scanning modes (see Sect. :ref:`ObsCoreExtensionForRadioData:subsec:sd` for details and table :ref:`ObsCoreExtensionForRadioData:tab:scanmode` for possible values). It is applicable to SD as well as to interferometry cases. .. raw:: latex \begin{longtable}{ | p{4.5cm}| p{7cm}| } \hline \textbf{value}&\textbf{definition}\cr \hline \hline \texttt{on-source}&pointed measurement\cr \hline \texttt{on-off}&switched measurements between two positions (source and background)\cr \hline \texttt{raster-map}&successive measurement spots on a predefined mesh (generally regular and rectangular)\cr \hline \texttt{on-the-fly-cross-scan}&successive continuous measurements along a predefined spatial pattern\cr \hline \texttt{on-the-fly-map}&successive continuous measurements along parallel directions\cr \hline \texttt{skydip}&successive continuous measurements along a large span in elevation at a given azimuth (acquired for calibration purposes)\cr \hline \texttt{frequency-switching}&subsequent measurements of the same target at two different central frequencies acquired with the same receiver \cr % \texttt{target}&\texttt{extrasolar target follow up}\cr \hline \caption{Values and definitions of the \emph{scan\_mode} attribute.} \label{tab:scanmode} \end{longtable} Pointing mode distinguishes observations pointed on a fixed target from observations fixed on a given elevation and azimuth. The :doc:`ObsLocTAP <../ObsLocTAP/ObsLocTAP>` specification :doc:`ObsLocTAP <../ObsLocTAP/ObsLocTAP>` defines the term *tracking_type* for describing this as well as a vocabulary for these modes. We include the same terms here in the present extension. The possible values for radio astronomy data are the following: - ``sidereal`` : observations pointed on a fixed target, as defined in :doc:`ObsLocTAP <../ObsLocTAP/ObsLocTAP>` - ``fixed-az-el-transit`` : observations fixed on a given elevation and azimuth, as in :doc:`ObsLocTAP <../ObsLocTAP/ObsLocTAP>` - Solar-system-object-tracking : moving target, like moon or solar bodies .. _`ObsCoreExtensionForRadioData:auxiliary-datasets-for-data-quality-estimation`: 4.5 Auxiliary datasets for data quality estimation -------------------------------------------------- Auxiliary datasets such as *uv* distribution map, dirty beam maps, frequency/amplitude plots, phase/amplitude plots are useful for astronomers to check data quality. In that case, :doc:`DataLink <../DataLink/DataLink>` :doc:`DataLink <../DataLink/DataLink>` may provide a solution to attach these auxiliary data files to :doc:`ObsCore <../ObsCore/ObsCore>` records. The link between a data set and its associated data files, is described by a set of parameters in the specification as shown in the datalink table example below. .. raw:: latex \begin{flushleft} \begin{tabular}{| l l l l l | |} \hline {\bf parameter} & {\bf DL record } & {\bf DL record} \\ \hline ID & ivo://aaa/bb & ivo://aaa/bb \\ (of ObsCore record ) & & \\ \hline access\_url &http://atca/aaa & http://atca/bbb\\ \hline description & \emph{uv} coverage map & dirty beam image \\ \hline semantics & auxiliary& calibration\\ \hline content\_type & image/png& image/png \\ \hline content\_length & 25000 & 1000 000\\ \hline content\_qualifier & \footnotesize{\url{www.ivoa.net/rdf/product-type#image}} & \footnotesize{\url{www.ivoa.net/rdf/product-type#image}}\\ \hline local\_semantics & uv\_distribution\_map& dirty\_beam \\ \hline \end{tabular} \end{flushleft} .. _`ObsCoreExtensionForRadioData:sec:implementation`: 5 The ivoa.obscore_radio table ============================== The :doc:`ObsCore <../ObsCore/ObsCore>` Extension for Radio is accessed as a table within a :doc:`TAP <../TAP/TAP>` :doc:`TAP <../TAP/TAP>` service. At this point, the name of this table is fixed to ``ivoa.obscore_radio``. Within the IVOA, it is forbidden to use this name for anything else than a table compliant with this specification. A :doc:`TAP <../TAP/TAP>` service that has ``ivoa.obscore_radio`` must also have a table compliant to any version 1 of :doc:`ObsCore <../ObsCore/ObsCore>`, i.e., a table ``ivoa.obscore`` containing only the basic :doc:`ObsCore <../ObsCore/ObsCore>` attributes. The two tables must share exactly the *obs_publisher_did* column such that a ``NATURAL JOIN`` will yield per-dataset rows of obscore and radio extension metadata. To ensure that all compliant services can execute the same queries, all columns described in following tables  :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt` ,  :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt_interferometry` and :ref:`ObsCoreExtensionForRadioData:tab:ExtensionAtt_instrumental` must be gathered in the ``ivoa.obscore_radio`` table, although some of them may be NULL if no value apply . At least a foreign key into ``ivoa.obscore`` will typically make the extension table user-visible. The intention is that clients will write queries like :: SELECT [any obscore and obscore_radio columns or expressions] FROM ivoa.obscore NATURAL JOIN ivoa.obscore_radio WHERE [constraints] .. raw:: latex \begin{landscape} \begin{longtable}{ l p{4cm}| p{4cm}| l l |} \sptablerule \textbf{column name}&\textbf{definition}&\textbf{utype}&\textbf{ucd}&\textbf{unit}\cr \sptablerule \texttt{ s\_resolution\_min}&\texttt{ Angular resolution, longest baseline and max frequency dependent}&{ Char.SpatialAxis.\newline Resolution.Bounds.\newline Limits.LoLim}&{pos.angResolution;stat.min}&{arcsec}\cr \sptablerule \texttt{s\_resolution\_max}&\texttt{Angular resolution, longest baseline and min frequency dependent}&\texttt{Char.SpatialAxis.\newline Resolution.Bounds.\newline Limits.HiLim}&{pos.angResolution;stat.max}&arcsec\cr \sptablerule \texttt{s\_fov\_min}&\texttt{field of view diameter, min value, max frequency dependent}&\texttt{Char.SpatialAxis.\newline Coverage.Bounds.\newline Extent.LowLim}&{phys.angSize;instr.fov;\newline stat.min}°\cr \sptablerule \texttt{s\_fov\_max}&\texttt{field of view diameter, max value, min frequency dependent}&\texttt{Char.SpatialAxis.\newline Coverage.Bounds.\newline Extent.HiLim}&{phys.angSize;instr.fov;\newline stat.max}°\cr \sptablerule \texttt{f\_resolution}&\texttt{absolute spectral resolution in frequency}&\texttt{Char.SpectralAxis.\newline Coverage.Bounds\newline Limits.HiLim}&{em.freq;stat.max}&kHz\cr \sptablerule \caption{ObsCore extension proposal for radio data in general.} \label{tab:ExtensionAtt} \end{longtable} \end{landscape} .. raw:: latex \begin{landscape} \begin{longtable}{l p{4.5cm}| p{4cm}| l l |} \hline \ \textbf{column name}&\textbf{definition}&\textbf{utype}&\textbf{ucd}&\textbf{unit}\\ \hline \texttt{s\_largest\_angular\_scale}&\texttt{maximum scale in dataset, shortest baseline and for typical frequency}&\texttt{Char.SpatialAxis.\newline Resolution.Scale.\newline Limits.HiLim}&{phys.angSize;stat.max}&arcsec \\ \hline \texttt{s\_largest\_angular\_scale\_min}&\texttt{smallest maximum scale in dataset, shortest baseline and for highest frequency}&\texttt{Char.SpatialAxis.\newline Resolution.Scale.\newline Limits.HiLim.Low}&{phys.angSize;stat.max}&arcsec \\ \hline \texttt{s\_largest\_angular\_scale\_max}&\texttt{largest maximum scale in dataset, shortest baseline and for lowest frequency}&\texttt{Char.SpatialAxis.\newline Resolution.Scale.\newline Limits.HiLim.Hi}&{phys.angSize;stat.max}&arcsec \\ \hline %\texttt{f\_min}&\texttt{spectral coverage min in frequency}&\texttt{Char.SpectralAxis.\newline Coverage.Bounds\newline Limits.LoLim}&{em.freq;stat.min}&Mhz&radio\cr %\hline %\texttt{f\_max}&\texttt{spectral coverage max in frequency}&\texttt{Char.SpectralAxis.\newline Coverage.Bounds\newline Limits.HiLim}&{em.freq;stat.max}&Mhz&radio\cr %\hline %\texttt{t\_exp\_min}&\texttt{minimum integration time per sample}&\texttt{Char.TimeAxis.\newline Sampling.Extent\newline LoLim}&{time.duration;obs.exposure;\newline stat.min}&s&radio\cr %\hline %\texttt{t\_exp\_max}&\texttt{maximum integration time per sample}&\texttt{Char.TimeAxis.\newline Sampling.Extent\newline HiLim}&{time.duration;obs.exposure;\newline stat.max}&s&radio\cr %\hline %\texttt{t\_exp\_mean}&\texttt{average integration time per sample}&\texttt{Char.TimeAxis.\newline Sampling.Extent\newline HiLim}&{time.duration;obs.exposure\newline stat.mean}&s&radio\cr %\hline \texttt{uv\_distance\_min}&\texttt{minimal distance in uv plane}&\texttt{Char.UVAxis.Coverage\newline .Bounds.Limits.LoLim}&stat.fourier;pos;stat.min&m \\ \hline \texttt{uv\_distance\_max}&\texttt{maximal distance in uv plane}&\texttt{Char.UVAxis.Coverage\newline .Bounds.Limits.LoLim}&stat.fourier;pos;stat.max&m \\ \hline \texttt{uv\_distribution\_ecc}&\texttt{eccentricity of uv distribution}&\texttt{Char.UVAxis.Coverage\newline .Bounds.Eccentricity}&stat.fourier;pos& null \\ \hline \texttt{uv\_distribution\_fill}&\texttt{filling factor of uv distribution}&\texttt{Char.UVAxis.Coverage\newline .Bounds.FillingFactor}&stat.fourier;pos;\newline arith.ratio& null \\ \hline %\texttt{s\_beam\_dirty}&\texttt{dirty beam}&\texttt{- (via DataLink}&{via DataLink}&&interferometry\cr %\hline %\texttt{phase\_amplitude}&\texttt{phase amplitude plot}&\texttt{via DataLink}&{via DataLink}&&interferometry\cr %\hline %\texttt{frequency\_amplitude}&\texttt{dirty beam}&\texttt{via DataLink}&{via DataLink}&&interferometry\cr %\hline \caption{ObsCore extension columns proposal for radio interferometry.} \label{tab:ExtensionAtt_interferometry} \end{longtable} \end{landscape} .. raw:: latex \begin{landscape} \begin{longtable}{l p{4.2cm} | p{4cm} | l l l } \hline \textbf{column name}&\textbf{definition}&\textbf{utype}&\textbf{ucd}&\textbf{unit}\\ \hline \texttt{instr\_tel\_number}&\texttt{number of antennas in array}&\texttt{Provenance.ObsConfig.\newline Instrument.Array.\newline AntNumber}&meta.number;instr.param& \\ \hline % same for all antenae features \texttt{instr\_tel\_min\_dist}&\texttt{minimum distance between antennas in array}&\texttt{Provenance.ObsConfig.\newline Instrument.Array.\newline MinDist}&instr.baseline;stat.min&m \\ \hline \texttt{instr\_tel\_max\_dist}&\texttt{maximum distance between antennas in array}&\texttt{Provenance.ObsConfig.\newline Instrument.Array.\newline MaxDist}&instr.baseline;stat.max&m \\ \hline \texttt{instr\_tel\_diameter}&\texttt{diameter of telecope or antennas in array}&\texttt{Provenance.ObsConfig.\newline Instrument.Array.\newline Diameter}&instr.param&m \\ \hline \texttt{instr\_feed}&\texttt{number of feeds}&\texttt{Provenance.ObsConfig.\newline Instrument.Feed}&instr.param& \\ \hline \texttt{scan\_mode}&\texttt{sky and spectral axis scan mode }&\texttt{Provenance.\newline Observation.\newline sky\_scan\_mode}&instr.param& \\ \hline \texttt{tracking\_type}&\texttt{target tracking modes}&\texttt{Provenance.\newline Observation.\newline tracking\_mode}&instr.param& \\ \hline \caption{ObsCore radio extension columns for instrumental parameters.} \label{tab:ExtensionAtt_instrumental} \end{longtable} \end{landscape} .. _`ObsCoreExtensionForRadioData:sec:registry`: 6 Registry Aspects ================== Services compliant with this specification are registered using :doc:`VODataService <../VODataService/VODataService>` :doc:`VODataService <../VODataService/VODataService>` tablesets. Compliant tables use the utype .. math:: \hbox{\nolinkurl{ivo://ivoa.net/std/ObsCore#radioExt-1.0}.} While it is admitted that the table only sits in the tableset of the embedding :doc:`TAP <../TAP/TAP>` service, implementors are urged to use a separate registry record with the main :doc:`TAP <../TAP/TAP>` service as an auxiliary capability :cite:p:`2019ivoa.spec.0520D`. In this way, meaningful information on coverage in space, time, and spectral axes as per :doc:`VODataService <../VODataService/VODataService>` 1.2 can be communicated to the Registry, which data providers are urged to do. However, discovering the ``obscore_radio`` table alone would be irrelevant. The same service delivering the ``obscore_radio`` table MUST also contain the :doc:`ObsCore <../ObsCore/ObsCore>` table. Being sure our service contains these both tables, the user is able to build any natural JOIN query between these two tables. To obtain access URLs of all :doc:`TAP <../TAP/TAP>` services that have compliant tables together with their table names (which in this major version are fixed to ``ivoa.obscore_radio``), use a :doc:`RegTAP <../RegTAP/RegTAP>` :cite:p:`2019ivoa.spec.1011D` query like: :: SELECT DISTINCT(access_url), table_name FROM rr.res_table NATURAL JOIN rr.capability NATURAL JOIN rr.interface WHERE standard_id LIKE 'ivo://ivoa.net/std/tap%' AND intf_role='std' AND table_utype LIKE 'ivo://ivoa.net/std/ObsCore#radioExt-1.%' AND EXISTS (select 1 from rr.res_table where table_name LIKE '%obscore') In the current status of the :doc:`ObsCore <../ObsCore/ObsCore>` specification the last statement in the WHERE clause is the simplest one to ensure the service also delivers the main obscore table. In the future this statement could be replaced by :: AND EXISTS (select 1 from rr.res_table where table_utype LIKE 'ivo://ivoa.net/std/obscore#core-1.%') When we will have other extensions (for example for time series or high energy data) we may want to discover services supporting several extensions in addition to the :doc:`ObsCore <../ObsCore/ObsCore>` main table. Searching ObsTAP services with multiple extensions could be done by a query to the relational registry such as: :: SELECT DISTINCT(access_url), table_name FROM rr.res_table NATURAL JOIN rr.capability NATURAL JOIN rr.interface WHERE standard_id LIKE 'ivo://ivoa.net/std/tap%' AND intf_role='std' AND table_utype LIKE 'ivo://ivoa.net/std/ObsCore#radioExt-1.%' AND EXISTS (select 1 from rr.res_table where table_utype LIKE 'ivo://ivoa.net/std/ObsCore#timeExt-1.0' AND EXISTS (select 1 from rr.res_table where table_name LIKE '%obscore') assuming that the standardID for the time extension currently in progress will be .. math:: \hbox{\nolinkurl{ivo://ivoa.net/std/ObsCore#timeExt-1.0}} In addition the tableset schema containing the :doc:`ObsCore <../ObsCore/ObsCore>` main table and potentially some of the extensions SHOULD use the root :doc:`ObsCore <../ObsCore/ObsCore>` standardID utype : .. math:: \hbox{\nolinkurl{ivo://ivoa.net/std/ObsCore}} Then the following query would allow to discover all services exposing :doc:`ObsCore <../ObsCore/ObsCore>` metadata as well as the extension tables they deliver. :: SELECT DISTINCT(access_url), table_name, schema_name FROM rr.res_table NATURAL JOIN rr.capability NATURAL JOIN rr.interface NATURAL JOIN rr.res_schema WHERE standard_id LIKE 'ivo://ivoa.net/std/tap%' AND intf_role='std' AND schema_utype LIKE 'ivo://ivoa.net/std/ObsCore' .. raw:: latex \newpage Appendices ========== A Appendices ============ .. [1] https://ivoa.net/documents/Notes/RadioVOImp/index.html .. [2] https://wiki.ivoa.net/internal/IVOA/SiaInterface/Anita-InterferometryVO.pdf .. [3] https://wiki.ivoa.net/internal/IVOA/RadioastronomyInterestGroupFifthVirtualMeeting/PulsarRadioDataAccess.pdf .. [4] https://www.ivoa.net/documents/ObsCoreTimeExtension/20240717/index.html, IVOA working draft .. [5] https://wiki.ivoa.net/internal/IVOA/RadioastronomyInterestGroupFifthVirtualMeeting/PulsarRadioDataAccess.pdf .. [6] https://www.ivoa.net/rdf/product-type/2023-06-26/product-type.html .. |image1| image:: SingleDish.png :width: 90.0%