==================== TAPRegExt: A :doc:`VOResource <../VOResource/VOResource>` Schema Extension for Describing :doc:`TAP <../TAP/TAP>` Services ==================== Official bibliographic entry for published version :cite:p:`2012ivoa.spec.0827D`. :Status: TAPRegExt 1.1 WD 2015-11-17 .. role:: raw-latex(raw) :format: latex .. .. _`TAPRegExt:introduction`: 1 Introduction ============== The Table Access Protocol :doc:`TAP <../TAP/TAP>` :doc:`TAP <../TAP/TAP>` allows VO clients to send queries to remote database servers and receive the results in standard formats. In addition, it defines means to discover database schemata on the remote side, to upload data from the local disk or third-party hosts, and more. :doc:`TAP <../TAP/TAP>` builds upon a variety of other standards, premier among which is the Universal Worker Service :cite:p:`2016ivoa.spec.1024H`, which describes how client and server can negotiate the execution of a query and the retrieval of results without having to maintain a continuous connection. To accommodate a wide variety of requirements, the :doc:`TAP <../TAP/TAP>` specification offers implementors many choices on optional features, resource limits, or locally defined functionality. One purpose of TAPRegExt is to allow the service to communicate such choices to remote clients using the mechanisms laid down in the VO Service Interfaces standard :doc:`VOSI <../VOSI/VOSI>`. Clients also need to discover :doc:`TAP <../TAP/TAP>` services offering certain kinds of data. Central to this is the concept of a registry in which resources can be described and consequently discovered by users and applications in the VO. Registries receive resource descriptions as defined in the IVOA standard :cite:p:`2018ivoa.spec.0625P`. In this schema, support for a standard service protocol is described as a service’s capability; the associated metadata is contained within the service resource description’s ```` element. TAPRegExt defines this capability element for :doc:`TAP <../TAP/TAP>` services. In the context of registering :doc:`TAP <../TAP/TAP>` services, an important role filled by TAPRegExt is the communication of supported data models to the Registry. The specification comes with a non-normative example document [1]_ showing this specification’s major features. It also declares a separate capability for the :doc:`VOSI <../VOSI/VOSI>` capabilities endpoint. The example is written as a response from a :doc:`TAP <../TAP/TAP>` service’s capabilities endpoint. When embedded in a :doc:`VOResource <../VOResource/VOResource>` record, the capability elements would be direct children of the :raw-latex:`\xmlel{vr:Resource}` element. .. _`TAPRegExt:architecture`: 1.1 TAPRegExt within the VO Architecture ---------------------------------------- .. container:: float :name: TAPRegExt:fig:arch .. container:: center `role_diagram.pdf `__ This specification directly relates to other IVOA standards in the following ways: :doc:`VOResource <../VOResource/VOResource>`, v1.1 :cite:p:`2018ivoa.spec.0625P` Descriptions of services that support :doc:`TAP <../TAP/TAP>` are encoded using the :doc:`VOResource <../VOResource/VOResource>` XML schema. TAPRegExt is an extension of the :doc:`VOResource <../VOResource/VOResource>` core schema. :doc:`TAP <../TAP/TAP>`, v1.1 :doc:`TAP <../TAP/TAP>` The :doc:`TAP <../TAP/TAP>` standard defines some of the concepts that TAPRegExt deals with. The :doc:`TAP <../TAP/TAP>` standard document indirectly refers to this document in the specification of its capabilities endpoint. UWS, v1.1 :cite:p:`2016ivoa.spec.1024H` The UWS standard describes additional parameters the choices of which are communicated using TAPRegExt. :doc:`StandardsRegExt <../StandardsRegExt/StandardsRegExt>` :doc:`StandardsRegExt <../StandardsRegExt/StandardsRegExt>` TAPRegExt uses the StandardKeyEnumeration mechanism introduced in :doc:`StandardsRegExt <../StandardsRegExt/StandardsRegExt>` to define controlled vocabularies. This standard also relates to other IVOA standards: IVOA Support Interfaces, v1.1 :doc:`VOSI <../VOSI/VOSI>` :raw-latex:`\hfil`:raw-latex:`\break `:doc:`VOSI <../VOSI/VOSI>` describes the standard interfaces to discover metadata about services; this document defines the response :doc:`TAP <../TAP/TAP>` services should provide on the ``capabilities`` endpoint described by :doc:`VOSI <../VOSI/VOSI>`. IVOA defined data models Data models specified by the IVOA can define the structure of database tables holding instances of those data models. The first examples of such definitions are :doc:`ObsCore <../ObsCore/ObsCore>` :doc:`ObsCore <../ObsCore/ObsCore>` and :doc:`RegTAP <../RegTAP/RegTAP>` :cite:p:`2019ivoa.spec.1011D`. Services providing access to such tables declare that fact within TAPRegExt instance documents. .. _`TAPRegExt:taextension`: 2 The Extension =============== .. _`TAPRegExt:nsloc`: 2.1 The Schema Namespace and Location ------------------------------------- The namespace associated with the TAPRegExt :doc:`VOResource <../VOResource/VOResource>` extension is .. math:: \mbox{\texttt{http://www.ivoa.net/xml/TAPRegExt/v1.0}.} The namespace is unchanged from version 1.0 of this standard as no changes that could break clients are introduced. Just like the namespace URI for the :doc:`VOResource <../VOResource/VOResource>` schema, the TAPRegExt namespace URI can be interpreted as a URL. Resolving it returns the current XML schema document that defines the TAPRegExt schema (cf. Appendix :ref:`TAPRegExt:app:fullschema`). Authors of :doc:`VOResource <../VOResource/VOResource>` instance documents may choose to provide a location for the :doc:`VOResource <../VOResource/VOResource>` XML schema document and its extensions using the :raw-latex:`\xmlel{xsi:schemaLocation}` attribute. While generators are free to provide any schema location (e.g., a local mirror), this specification recommends using the TAPRegExt namespace URI as its location URL, as in, :: xsi:schemaLocation="http://www.ivoa.net/xml/TAPRegExt/v1.0 http://www.ivoa.net/xml/TAPRegExt/v1.0" Note that you must give the ``xsi:schemaLocation`` of the TAPRegExt schema when the capability defined here is part of a published registry resource record as per the IVOA Registry Interface standard :cite:p:`2018ivoa.spec.0723D`. This does not apply to the use in a :doc:`TAP <../TAP/TAP>` server’s capabilities endpoint. .. _`TAPRegExt:dms`: 2.2 Declaring Instantiated Data Models -------------------------------------- The IVOA defines certain data models that can be instantiated in database tables exposed by a :doc:`TAP <../TAP/TAP>` service. This allows a query built exclusively on a data model or a set of data models to work on all :doc:`TAP <../TAP/TAP>` services exposing tables instantiating the data model(s). In TAPRegExt, a data model is identified by its IVOA identifier :cite:p:`2016ivoa.spec.0523D`. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:DataModelType} Type Schema Documentation} \noindent{\small{}An IVOA defined data model, identified by an IVORN intended for machine consumption and a short label intended for human comsumption.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:DataModelType} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:DataModelType} Attributes} \begingroup\small\begin{bigdescription} \item[ivo-id] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The IVOID of the data model. \item[Occurrence] required \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:langs`: 2.3 Languages Supported ----------------------- :doc:`TAP <../TAP/TAP>` services may offer a variety of query languages. In TAPRegExt, the :raw-latex:`\xmlel{language}` element allows the communication of what languages are available on a service. :doc:`TAP <../TAP/TAP>` defines values of the ``LANG`` parameter to have either the form ``-`` or the form ````, where the latter form leaves the choice of the version to the server. Therefore, a language is defined using a name and one or more versions. The recommended way to associate larger amounts of documentation with a language entry in a capability element is via registration of the language using the mechanisms defined in StdRegExt :doc:`StandardsRegExt <../StandardsRegExt/StandardsRegExt>` and associating the registry record with the language element through the latter’s :raw-latex:`\xmlel{ivo-id}` attribute. The IVOID for the only language mandatory for :doc:`TAP <../TAP/TAP>` services, :doc:`ADQL <../ADQL/ADQL>` 2.0, is ``ivo://ivoa.net/std/ADQL#v2.0``. . The type of the :raw-latex:`\xmlel{ivo-id}` attribute on version is :raw-latex:`\xmlel{xs:anyURI}` as opposed to :raw-latex:`\xmlel{vr:IdentifierURI}` since the latter does not allow fragment identifiers in :doc:`VOResource <../VOResource/VOResource>` 1.0. The description constrains the value to be an IVOID (i.e., a URI with a schema of ivo:), though. The same reasoning applies to the :raw-latex:`\xmlel{ivo-id}` attributes of :raw-latex:`\xmlel{outputFormat}` and :raw-latex:`\xmlel{uploadMethod}`. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:Language} Type Schema Documentation} \noindent{\small{}A query language supported by the service.\par} \noindent{\small{}Each language element can describe one or more versions of a language. Either name alone or name-version can be used as values for the server's LANG parameter.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:Language} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:Language} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{name}] \begin{description} \item[Type] a prefixless XML name \item[Meaning] The name of the language without a version suffix. \item[Occurrence] required \end{description} \item[Element \xmlel{version}] \begin{description} \item[Type] a string with optional attributes \item[Meaning] A version of the language supported by the server. \item[Occurrence] required; multiple occurrences allowed. \end{description} \item[Element \xmlel{description}] \begin{description} \item[Type] string: \xmlel{xs:token} \item[Meaning] A short, human-readable description of the query language. \item[Occurrence] optional \end{description} \item[Element \xmlel{languageFeatures}] \begin{description} \item[Type] composite: \xmlel{tr:LanguageFeatureList} \item[Meaning] Optional features of the query language, grouped by feature type. \item[Occurrence] optional; multiple occurrences allowed. \item[Comment] This includes listing user defined functions, geometry support, or similar concepts. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:Version} Type Schema Documentation} \noindent{\small{}One version of the language supported by the service.\par} \noindent{\small{}If the service supports more than one version of the language, include multiple version elements. It is recommended that you use a version numbering scheme like MAJOR.MINOR in such a way that sorting by ascending character codes will leave the most recent version at the bottom of the list.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:Version} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:Version} Attributes} \begingroup\small\begin{bigdescription} \item[ivo-id] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] An optional IVOID of the language. \item[Occurrence] optional \item[Comment] To more formally define a language supported by a service, a resource record for the language can be created, either centrally on the Registry of Registries or by other registry operators. When such a record exists, the ivo-id attribute of language should point to it. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} Query languages may support optional features. For :doc:`ADQL <../ADQL/ADQL>`, the most prominent of those are user-defined functions, i.e., functions not defined in the language standard but added by the operators of the service, and geometry functions. Such optional features may be communicated to the service client in :raw-latex:`\xmlel{tr:languageFeatures}` elements. Each such list is labelled with a :raw-latex:`\xmlel{type}` attribute indicating the type of language option being described. This string should be an IVOID whose semantics in this context, along with the semantics of the content of its descendant :raw-latex:`\xmlel{feature/form}` elements, can be documented in association with the language in question. TAPRegExt itself defines the following feature types: .. raw:: latex \begin{bigdescription} \item[\nolinkurl{ivo://ivoa.net/std/TAPRegExt\#features-udf}] Each feature declares a user-defined ADQL (or similar) function supported. The content of the \xmlel{form} element must be the signature of the function, written to match the \texttt{signature} nonterminal in the following grammar: \begin{verbatim} signature ::= "->" funcname ::= arglist ::= "(" { "," } ")" arg ::= \end{verbatim} The \texttt{type\_name} nonterminal is not defined by the ADQL grammar in version 2.0. For the purposes of TAPRegExt, it is sufficient to assume it expands to ``some sort of SQL type specifier'' (which may include spaces and parentheses). For an enumeration of common types in ADQL, refer to the last column of the table in section 2.5 of the TAP standard \citep{2019ivoa.spec.0927D}. Example: \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
match(pattern TEXT, string TEXT) -> INTEGER
match returns 1 if the POSIX regular expression pattern matches anything in string, 0 otherwise.
\end{lstlisting} \item[\nolinkurl{ivo://ivoa.net/std/TAPRegExt\#features-adqlgeo}] Each feature declares support for one of the geometry functions defined by ADQL (support for these functions is in general optional for ADQL implementations, though TAP imposes some constraints on what combinations of support are permitted). The signature of these functions, where supported, is fixed by ADQL; the content of the \xmlel{form} element is just the name of the function. Example: \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
CONTAINS
\end{lstlisting} \end{bigdescription} .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:LanguageFeatureList} Type Schema Documentation} \noindent{\small{}An enumeration of non-standard or non-mandatory features of a specific type implemented by the language.\par} \noindent{\small{}A feature type is a language-dependent concept like {"}user defined function{"}, {"}geometry support{"}, or possibly {"}units supported{"}. A featureList gives all features of a given type applicable for the service. Multiple featureLists are possible. All feature in a given list are of the same type. This type is declared using the mandatory type attribute, the value of which will typically be an IVOID. To see values defined in TAPRegExt, retrieve the ivo://ivoa.net/std/TAPRegExt resource record and look for keys starting with {"}features-{"}.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:LanguageFeatureList} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:LanguageFeatureList} Attributes} \begingroup\small\begin{bigdescription} \item[type] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The type of the features given here. \item[Occurrence] required \item[Comment] This is in general an IVOID. TAPRegExt itself gives IVOIDs for defining user defined functions and geometry support. \end{description} \end{bigdescription}\endgroup \vspace{0.5ex}\noindent\textbf{\xmlel{tr:LanguageFeatureList} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{feature}] \begin{description} \item[Type] composite: \xmlel{tr:LanguageFeature} \item[Meaning] A language feature of the type given by the type attribute. \item[Occurrence] optional; multiple occurrences allowed. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:LanguageFeature} Type Schema Documentation} \noindent{\small{}A non-standard or non-mandatory feature implemented by the language..\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:LanguageFeature} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:LanguageFeature} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{form}] \begin{description} \item[Type] string: \xmlel{xs:token} \item[Meaning] Formal notation for the language feature. \item[Occurrence] required \item[Comment] The syntax for the content of this element is defined by the type attribute of its parent language list. \end{description} \item[Element \xmlel{description}] \begin{description} \item[Type] string: \xmlel{xs:string} \item[Meaning] Human-readable freeform documentation for the language feature. \item[Occurrence] optional \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:outforms`: 2.4 Output Formats ------------------ A :doc:`TAP <../TAP/TAP>` service may offer a variety of output formats. What output formats are available is defined using :raw-latex:`\xmlel{outputFormat}` elements. They declare an RFC 2046 media type :cite:p:`std:MIME` as well as aliases (the shorthand forms the server also accepts in the FORMAT parameter). If desired, the format can be further described with an IVOID in the ivo-id attribute; TAPRegExt provides keys for some variants of VOTables which are not interoperably distinguishable by their MIME types so far: :raw-latex:`\normalfont`\ ``output-votable-td`` A :doc:`VOTable <../VOTable/VOTable>` in which all DATA elements contain a TABLEDATA element :raw-latex:`\normalfont`\ ``output-votable-binary`` A :doc:`VOTable <../VOTable/VOTable>` in which all DATA elements contain a STREAM element with a BINARY child :raw-latex:`\normalfont`\ ``output-votable-binary2`` A :doc:`VOTable <../VOTable/VOTable>` in which all DATA elements contain a STREAM element with a BINARY2 child .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:OutputFormat} Type Schema Documentation} \noindent{\small{}An output format supported by the service.\par} \noindent{\small{}All TAP services must support VOTable output, with media types as requested by the FORMAT parameter if applicable (cf.~section 2.7.1 of the TAP standard). The primary identifier for an output format is the RFC 2046 media type. If you want to register an output format, you must use a media type (or make one up using the x- syntax), although the concrete media syntax is not enforced by the schema. For more detailed specification, an IVOID may be used.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:OutputFormat} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:OutputFormat} Attributes} \begingroup\small\begin{bigdescription} \item[ivo-id] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] An optional IVOID of the output format. \item[Occurrence] optional \item[Comment] When the media type does not uniquely define the format (or a generic media type like application/octet-stream or text/plain is given), the IVOID can point to a key or StandardsRegExt document defining the format more precisely. To see values defined in TAPRegExt, retrieve the ivo://ivoa.net/std/TAPRegExt resource record and look for keys starting with {"}output-{"}. \end{description} \end{bigdescription}\endgroup \vspace{0.5ex}\noindent\textbf{\xmlel{tr:OutputFormat} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{mime}] \begin{description} \item[Type] string: \xmlel{xs:token} \item[Meaning] The media type of this format. \item[Occurrence] required \item[Comment] The format of this string is specified by RFC 2046. The service has to accept this string as a value of the FORMAT parameter. \end{description} \item[Element \xmlel{alias}] \begin{description} \item[Type] string: \xmlel{xs:token} \item[Meaning] Other values of FORMAT ({"}shorthands{"}) that make the service return documents with the media type. \item[Occurrence] optional; multiple occurrences allowed. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:uploadmethods`: 2.5 Upload Methods ------------------ :doc:`TAP <../TAP/TAP>` services should allow the upload of VOTables. They can support various methods to do this: HTTP upload, retrieval by URL, but also :doc:`VOSpace <../VOSpace/VOSpace>` or possibly retrieval using Grid protocols. Since an actual specification of the details of such protocols is far beyond the scope of a registry document and probably would not benefit clients anyway, the upload methods are given as IVOIDs. IVOIDs for the standard upload methods are provided within the resource record ``ivo://ivoa.net/std/TAPRegExt``. The IVOIDs are built by using the keys as fragments after the TAPRegExt IVOID. It is permitted to register upload methods under authorities other than ivoa.net. The registry records can then provide more in-depth information. For the upload methods defined in the :doc:`TAP <../TAP/TAP>` specification, however, the IVOIDs of the keys in the TAPRegExt resource record must be used to enable clients to identify supported methods using string comparisons. This document defines the following protocol identifiers: - ``upload-inline`` – HTTP upload as per section 2.5.2 of the :doc:`TAP <../TAP/TAP>` standard :doc:`TAP <../TAP/TAP>`. - ``upload-http`` – retrieval from an http URL. - ``upload-https`` – retrieval from an https URL. - ``upload-ftp`` – retrieval from an ftp URL. Thus, a service offering upload by retrieving from ftp and http URLs would say: .. code:: xml .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:UploadMethod} Type Schema Documentation} \noindent{\small{}An upload method as defined by IVOA.\par} \noindent{\small{}Upload methods are always identified by an IVOID. Descriptions can be obtained by dereferencing this IVOID. To see values defined in TAPRegExt, retrieve the ivo://ivoa.net/std/TAPRegExt resource record and look for keys starting with {"}upload-{"}. You can register custom upload methods, but you must use the standard IVOIDs for the upload methods defined in the TAP specification.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:UploadMethod} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:UploadMethod} Attributes} \begingroup\small\begin{bigdescription} \item[ivo-id] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The IVOID of the upload method. \item[Occurrence] optional \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:reslimits`: 2.6 Resource Limits ------------------- :doc:`TAP <../TAP/TAP>` services usually impose certain limits on resource usage by clients, e.g., a maximum run time per query, or a maximum number of rows in the result set. Services assign such limits to newly created jobs and may allow raising the limits by means of queries or query parameters (e.g., the size of the result set is limited by the ``MAXREC`` parameter, whereas the date of job destruction may be changed by posting to the ``destruction`` parameter). Services may put some limit to how far the resource limitations may be raised. TAPRegExt’s :raw-latex:`\xmlel{capabilities}` element allows the declaration of such limits. These declarations are primarily intended for human consumption and should give conservative guidelines. Thus, the operators of a service implementing a complex, possibly dynamic limits policy should give lower estimates here. If a service supports authentication and has different limits depending on what user is authenticated, it should make an effort to guess the limits applying to a given client (e.g., when authentication tokens are present in the request). Limits reported to the Registry should reflect limits for unauthenticated use unless the service does not admit unauthenticated requests. The resource limits applying to newly created jobs are given in :raw-latex:`\xmlel{default}` elements, the limits beyond which users cannot raise the limits are given in :raw-latex:`\xmlel{hard}` elements. Note that the absence of a specification of limits does not imply that no limits are enforced. .. _`TAPRegExt:limits-on-time`: 2.6.1 Limits on Time ~~~~~~~~~~~~~~~~~~~~ This document defines two time-like resource limits: - :raw-latex:`\xmlel{retentionPeriod}` – the time from job creation until ``destruction``. - :raw-latex:`\xmlel{executionDuration}` – the maximal run time given to a query. All values in time-like limits are given in seconds. Both :raw-latex:`\xmlel{retentionPeriod}` and :raw-latex:`\xmlel{executionDuration}` are of type :raw-latex:`\xmlel{tr:TimeLimits}`. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:TimeLimits} Type Schema Documentation} \noindent{\small{}Time-valued limits, all values given in seconds.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:TimeLimits} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:TimeLimits} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{default}] \begin{description} \item[Type] integer \item[Meaning] The value of this limit for newly-created jobs, given in seconds. \item[Occurrence] optional \end{description} \item[Element \xmlel{hard}] \begin{description} \item[Type] integer \item[Meaning] The value this limit cannot be raised above, given in seconds. \item[Occurrence] optional \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:limits-on-data`: 2.6.2 Limits on Data ~~~~~~~~~~~~~~~~~~~~ Limits on data are expressed much like time limits in that they give :raw-latex:`\xmlel{default}` and a :raw-latex:`\xmlel{hard}` value as well. Both those values have a unit attribute that can either be ``byte`` or ``row`` for data limits. This document defines two resource limits on data: - :raw-latex:`\xmlel{outputLimit}` – if :raw-latex:`\xmlel{unit}` is ``row`` here, the :raw-latex:`\xmlel{default}` gives the value of TAP’s ``MAXREC`` parameter the service will use when none is specified. - :raw-latex:`\xmlel{uploadLimit}` – the maximum size of uploads. This is not a :doc:`TAP <../TAP/TAP>` adjustable parameter. The :raw-latex:`\xmlel{default}` value advises clients about the server’s wishes as to a limit above which some sort of acknowledgement should be requested from the user. The :raw-latex:`\xmlel{hard}` limit gives the maximum size of an upload to the server. Data limits are defined using the :raw-latex:`\xmlel{tr:DataLimits}` and :raw-latex:`\xmlel{tr:DataLimit}` types: .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:DataLimits} Type Schema Documentation} \noindent{\small{}Limits on data sizes, given in rows or bytes.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:DataLimits} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:DataLimits} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{default}] \begin{description} \item[Type] an integer with optional attributes \item[Meaning] The value of this limit for newly-created jobs. \item[Occurrence] optional \end{description} \item[Element \xmlel{hard}] \begin{description} \item[Type] an integer with optional attributes \item[Meaning] The value this limit cannot be raised above. \item[Occurrence] optional \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:DataLimit} Type Schema Documentation} \noindent{\small{}A limit on some data size, either in rows or in bytes.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:DataLimit} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:DataLimit} Attributes} \begingroup\small\begin{bigdescription} \item[unit] \begin{description} \item[Type] string with controlled vocabulary \item[Meaning] The unit of the limit specified. \item[Occurrence] required \item[Allowed Values]\hfil \begin{longtermsdescription}\item[byte] \item[row] \end{longtermsdescription} \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} .. _`TAPRegExt:intfdecl`: 2.7 Interface Declaration ------------------------- Version 1.0 of this specification used a :doc:`VODataService <../VODataService/VODataService>` :raw-latex:`\xmlel{vs:ParamHTTP}` interface to declare the :doc:`TAP <../TAP/TAP>` service’s base URL. This was mainly done to maintain common service discovery patterns – other DAL protocols use :raw-latex:`\xmlel{vs:ParamHTTP}`-typed interfaces as well –, but it ignored the semantics of the type, which in particular is that the access URL of such an interface is what clients use in queries. Against that, the :doc:`TAP <../TAP/TAP>` specification actually leaves open what directly dereferencing the base URL should result in. Instead, all functionality is offered through children of the base URL. The abuse of :raw-latex:`\xmlel{vs:ParamHTTP}` later had a severe fallout as discussed in :cite:t:`note:caproles`. However, existing :doc:`TAP <../TAP/TAP>` clients rely on the interface declaration for discovery, and hence all capability elements with a TAPRegExt version 1 :raw-latex:`\xmlel{standardID}` must include a :raw-latex:`\xmlel{vs:ParamHTTP}`-typed interface with - a child element :raw-latex:`\xmlel{accessURL}` with its content the base URL of the :doc:`TAP <../TAP/TAP>` service (the parent of sync, async, etc.) and its :raw-latex:`\xmlel{use}` attribute set to ``base``. - its :raw-latex:`\xmlel{role}` attribute set to ``std`` - its :raw-latex:`\xmlel{version}` attribute set to the version of the :doc:`TAP <../TAP/TAP>` specification implemented. In order to open the path to advanced use cases (e.g., involving authenticated access), services implementing TAPRegExt 1.1 must include another interface element of type :raw-latex:`\xmlel{tr:DALIInterface}`. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:DALIInterface} Type Schema Documentation} \noindent{\small{}An interface for a complex, multi-endpoint interfaces as regulated by DALI.\par} \noindent{\small{}In addition to the standard vr:Interface elements, DALIInterfaces have endpoints, listed by name; that name doubles as a path segment to append to the interface's access URL, yielding the URI at which the endpoint is operated.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:DALIInterface} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:DALIInterface} Extension Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{endpoint}] \begin{description} \item[Type] composite: \xmlel{tr:Endpoint} \item[Meaning] An endpoint accessible through this interface. \item[Occurrence] optional; multiple occurrences allowed. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} The :raw-latex:`\xmlel{tr:DALIInterface}`-typed interface must satisfy the same requirements as the :raw-latex:`\xmlel{vs:ParamHTTP}`-typed one as regards its :raw-latex:`\xmlel{accessURL}` child and :raw-latex:`\xmlel{role}` and :raw-latex:`\xmlel{version}` attributes. Furthermore, this interface now enumerates the various endpoints exposed below the base URI in :raw-latex:`\xmlel{endpoint}` elements:raw-latex:`\todo{We may want to support vocab on endpoint; or do we want an implicit prefix as in datalink?}`. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:Endpoint} Type Schema Documentation} \noindent{\small{}An endpoint of a complex interface.\par} \noindent{\small{}An endpoint is characterised and addressed by its name; they can further be defined through RDF triples. This is a generic extension mechanism for endpoint-specific metadata, primarily intended for custom, vendor-specific extensions.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:Endpoint} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:Endpoint} Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{name}] \begin{description} \item[Type] string: \xmlel{xs:token} \item[Meaning] The endpoint name, which is also the last component of the path of its URI. \item[Occurrence] required \item[Comment] Names without dashes are reserved for IVOA use and are expected to work the same way on all services. Well-known examples for such endpoint names include sync, async, and tables. \end{description} \item[Element \xmlel{meta}] \begin{description} \item[Type] a string with optional attributes \item[Meaning] Auxiliary information on this endpoint. \item[Occurrence] optional; multiple occurrences allowed. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} For :doc:`TAP <../TAP/TAP>`, at least the ``sync``, ``async``, and ``tables`` endpoints must be declared, as well as ``examples`` if present. The :doc:`VOSI <../VOSI/VOSI>` capabilities URL, while in :doc:`TAP <../TAP/TAP>` also a child of the base URL, continues to be declared in separate capability in this version rather than a :doc:`DALI <../DALI/DALI>` interface endpoint. TAPRegExt specifically discourages returning different capability contents from different interfaces (e.g., with and without authentication) of a service. While in general, not overly much configurability should be enabled on the level of endpoints, operators can attach extra per-endpoint metadata using their :raw-latex:`\xmlel{meta}` children. These support a subset of the attributes defined by RDFa lite :cite:p:`std:RDFaLite11`, enough to make statements consisting of a subject (:raw-latex:`\xmlel{about}`), a predicate (:raw-latex:`\xmlel{property}`), and an object (content or :raw-latex:`\xmlel{resource}`): .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:MetaTriple} Type Schema Documentation} \noindent{\small{}A container for an RDFa triple giving information related to an endpoint.\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:MetaTriple} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:MetaTriple} Attributes} \begingroup\small\begin{bigdescription} \item[about] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The subject of the statement. \item[Occurrence] optional \item[Comment] If missing, the endpoint itself is assumed as the subject. \end{description} \item[property] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The property of the statement. \item[Occurrence] required \item[Comment] This is a reference to an RDF resource. IVOA standards may define semantics for scheme-less URI; non-IVOA properties must use full URIs with at least scheme and authority; in this version, no vocab attributes are supported. \end{description} \item[resource] \begin{description} \item[Type] a URI: \xmlel{xs:anyURI} \item[Meaning] The object of the statement. \item[Occurrence] optional \item[Comment] If missing, the text content of the element is used as the object. \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} This is intended to give operators the possibility to communicate extra information to custom clients primarily by defining properties, which are simply URIs controlled by the operator and should dereference to documents explaining the semantics of the properties, ideally authored in HTML. The object of the statement is typically given by the element contents, but where the object cannot be inlined, the :raw-latex:`\xmlel{meta}`’s :raw-latex:`\xmlel{resource}` attribute can be used instead. For instance, to convey a custom method to upload tables on an (authenticated) tables endpoint, an operator could say: .. code:: xml tables table-upload To prevent a fragmentation of the :doc:`TAP <../TAP/TAP>` standard, such extensions should be restricted to single service providers and custom clients and be considered for addition to the :doc:`TAP <../TAP/TAP>` standard as other parties take them up. Because it models the most essential aspects of RDF, the :raw-latex:`\xmlel{meta}` element, in principle, allows operators to build almost arbitrarily complex metadata structures when other :raw-latex:`\xmlel{meta}` elements are used as subjects. Continuing the example above, the operator could declare media types and labels for upload formats like this: .. code:: xml tables table-upload FITS Binary application/fits VOTable application/x-votable+xml Again, this facility is intended to support prototyping features as well as facilitate provider-specific extensions aiding custom tooling on top of standard interfaces. Interoperable :doc:`TAP <../TAP/TAP>` functionality should probably not need to rely on per-endpoint metadata of this complexity. .. _`TAPRegExt:caprec`: 2.8 The Capability Record ------------------------- Using the types defined above, the :raw-latex:`\xmlel{tr:TableAccess}` type can be defined. Note that it is a type, not a (global) element. In instance documents, you will typically place it in a capability element with an explicit type specification, like this: .. code:: xml ... :raw-latex:`\xmlel{tr:TableAccess}`-typed capabilites can be used with arbitrary standardIDs; this is different from previous TAPRegExt versions, which fixed it to ``ivo://ivoa.net/std/TAP``. This URI should still be given for :doc:`TAP <../TAP/TAP>` 1.0 services. Services having other capabilities (e.g., :doc:`TAP <../TAP/TAP>` 1.1 or even UWS) will use other standardIDs. .. raw:: latex \begin{generated} \begingroup \renewcommand*\descriptionlabel[1]{% \hbox to 5.5em{\emph{#1}\hfil}}\vspace{2ex}\noindent\textbf{\xmlel{tr:TableAccess} Type Schema Documentation} \noindent{\small{}The capabilities of a TAP server.\par} \noindent{\small{}The capabilities attempt to define most issues that the TAP standard leaves to the implementors ({"}may{"}, {"}should{"}).\par} \vspace{1ex}\noindent\textbf{\xmlel{tr:TableAccess} Type Schema Definition} \begin{lstlisting}[language=XML,basicstyle=\footnotesize] \end{lstlisting} \vspace{0.5ex}\noindent\textbf{\xmlel{tr:TableAccess} Extension Metadata Elements} \begingroup\small\begin{bigdescription}\item[Element \xmlel{dataModel}] \begin{description} \item[Type] a string with optional attributes \item[Meaning] Identifier of IVOA-approved data model supported by the service. \item[Occurrence] optional; multiple occurrences allowed. \end{description} \item[Element \xmlel{language}] \begin{description} \item[Type] composite: \xmlel{tr:Language} \item[Meaning] Language supported by the service. \item[Occurrence] required; multiple occurrences allowed. \end{description} \item[Element \xmlel{outputFormat}] \begin{description} \item[Type] composite: \xmlel{tr:OutputFormat} \item[Meaning] Output format supported by the service. \item[Occurrence] required; multiple occurrences allowed. \end{description} \item[Element \xmlel{uploadMethod}] \begin{description} \item[Type] composite: \xmlel{tr:UploadMethod} \item[Meaning] Upload method supported by the service. \item[Occurrence] optional; multiple occurrences allowed. \item[Comment] The absence of upload methods indicates that the service does not support uploads at all. \end{description} \item[Element \xmlel{retentionPeriod}] \begin{description} \item[Type] composite: \xmlel{tr:TimeLimits} \item[Meaning] Limits on the time between job creation and destruction time. \item[Occurrence] optional \end{description} \item[Element \xmlel{executionDuration}] \begin{description} \item[Type] composite: \xmlel{tr:TimeLimits} \item[Meaning] Limits on executionDuration. \item[Occurrence] optional \end{description} \item[Element \xmlel{outputLimit}] \begin{description} \item[Type] composite: \xmlel{tr:DataLimits} \item[Meaning] Limits on the size of data returned. \item[Occurrence] optional \end{description} \item[Element \xmlel{uploadLimit}] \begin{description} \item[Type] composite: \xmlel{tr:DataLimits} \item[Meaning] Limits on the size of uploaded data. \item[Occurrence] optional \end{description} \end{bigdescription}\endgroup \endgroup \end{generated} Appendices ========== .. _`TAPRegExt:app:fullschema`: A Obtaining the Schema ====================== The recommended way to obtain the TAPRegExt schema is to retrieve the namespace URI, http://www.ivoa.net/xml/TAPRegExt/v1.0, which will usually involve one or two redirects. Despite to suggestive name, this will always return the latest version of the TAPRegExt schema in major version 1; see :cite:t:`2018ivoa.spec.0529H` for the background of this somewhat unfortunate circumstance. The schema accompanying this specific version is published alongside this document [2]_. The document at this URI is only intended for use in review or debugging. .. _`TAPRegExt:changes`: B Changes from Previous Versions ================================ .. _`TAPRegExt:changes-from-rec-1.0`: B.1 Changes from REC-1.0 ------------------------ - Adding tr:DALIInterface to the schema. - Dropping the appendix with an example document. Using a standalone, validated file and auxiliaryurl instead. .. _`TAPRegExt:changes-20110127`: B.2 Changes from WD-20110127 ---------------------------- - userDefinedFunction was generalized to feature within languageFeatures. - The uploadmethods StandardKeyEnumeration was replaced by a resource record for TAPRegExt as a whole. This now includes keys of output formats and features as well; therefore, upload method names in their new IVOIDs are prefixed with upload- - Schema version was bumped to 1.0 (yes, we indulge in unversioned schema changes before this becomes REC). - uploadLimit interpretation was changed: The default limit is now “advisory” and to be interpreted as such by clients, the hard limit is what is actually required by the server. - There’s now an optional ivo-id attribute on the version element within language. - There’s now an optional ivo-id attribute on output formats. .. _`TAPRegExt:changes-20110727`: B.3 Changes from WD-20110727 ---------------------------- - The namespace in the schema is now ``http://www.ivoa.net/xml/TAPRegExt/v1.0`` consistent with what has already been stated in the text. - The IVOID for :doc:`ADQL <../ADQL/ADQL>` is now ``ivo://ivoa.net/std/ADQL#v2.0``; it is defined here to be in ADQL’s record since we do not want to wait for the :doc:`ADQL <../ADQL/ADQL>` standard to be fixed, but :doc:`ADQL <../ADQL/ADQL>` versioning should really not be done here, so a TAPRegExt IVOID is out of the question. - The IVOID of the TAPRegExt standard is now ``ivo://ivoa.net/std/TAPRegExt`` to conform with other standard IVOIDs. Unfortunately, this changes all other IVOIDs dependent on this. - We now allow AnyURI on the ivo-id of language to allow fragment identifiers as, e.g., in :doc:`ADQL <../ADQL/ADQL>`. .. _`TAPRegExt:changes-20120208`: B.4 Changes from PR-20120812 ---------------------------- - Fixed units in limits to ``"row"`` and ``"byte"``. .. _`TAPRegExt:changes-rec-1.0`: B.5 Changes from REC-1.0 ------------------------ - No longer restricting TableAccess’ standardID to the (old) :doc:`TAP <../TAP/TAP>` standardID. TAPRegExt capabilities are now allowed with arbitrary standardIDs - Changed former use of the term “IVORN” to “IVOID” (or Registry part, as appropriate) to comply with a proposed clarification of terminology in Identifiers 2.0 - The example now shows an entire capabilities response - Repaired obscore data model URI in the example - dataModel/@ivo-id is now typed xs:anyURI to allow fragment identifiers on it - Migrated to ivoatex .. [1] :raw-latex:`\auxiliaryurl{sample.xml}` .. [2] :raw-latex:`\auxiliaryurl{TAPRegExt-v1.0.xsd}`