Internet-Draft api-catalog well-known URI November 2024
Smith Expires 29 May 2025 [Page]
Workgroup:
Network Working Group
Internet-Draft:
draft-ietf-httpapi-api-catalog-06
Published:
Intended Status:
Standards Track
Expires:
Author:
K. Smith
Vodafone

api-catalog: a well-known URI and link relation to help discovery of APIs

Abstract

This document defines the "api-catalog" well-known URI and link relation. It is intended to facilitate automated discovery and usage of the APIs published by a given organisation or individual. A request to the api-catalog resource will return a document providing information about, and links to, the publisher's APIs.

About This Document

This note is to be removed before publishing as an RFC.

The latest revision of this draft can be found at https://ietf-wg-httpapi.github.io/api-catalog/draft-ietf-httpapi-api-catalog.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-ietf-httpapi-api-catalog/.

Discussion of this document takes place on the Building Blocks for HTTP APIs Working Group mailing list (mailto:httpapi@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/httpapi/. Subscribe at https://www.ietf.org/mailman/listinfo/httpapi/.

Source for this draft and an issue tracker can be found at https://github.com/ietf-wg-httpapi/api-catalog.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 29 May 2025.

▲

Table of Contents

1. Introduction

An organisation or individual may publish Application Programming Interfaces (APIs) to encourage requests for interaction from external parties. Such APIs must be discovered before they may be used - i.e., the external party needs to know what APIs a given publisher exposes, their purpose, any policies for usage, and the endpoint to interact with each API. To facilitate automated discovery of this information, and automated usage of the APIs, this document proposes:

1.1. Goals and non-goals

The primary goal is to facilitate the automated discovery of a Publisher's public API endpoints, along with metadata that describes the purpose and usage of each API, by specifying a well-known URI that returns an API catalog document. The API catalog document is primarily machine-readable to enable automated discovery and usage of APIs, and it may also include links to human-readable documentation.

Non-goals: this document does not mandate paths for API endpoints. i.e., it does not mandate that my_example_api's endpoint should be https://www.example.com/.well-known/api-catalog/my_example_api , nor even to be hosted at www.example.com (although it is not forbidden to do so).

1.2. Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. These words may also appear in this document in lower case as plain English words, absent their normative meanings.

The term "content negotiation" and "status code" are from [HTTP]. The term "well-known URI" is from [WELL-KNOWN]. The term "link relation" is from [WEB-LINKING].

The term "Publisher" refers to an organisation, company or individual that publishes one or more APIs for usage by external third parties. A fictional Publisher named "example" is used throughout this document. The examples use the FQDNs "www.example.com", "developer.example.com", "apis.example.com", "apis.example.net", "gaming.example.com", "iot.example.net",where the use of the .com and .net TLDs and various subdomains are simply to illustrate that the "example" Publisher may have their API portfolio distributed across various domains for which they are the authority. For scenarios where the Publisher "example" is not the authority for a given .example. domain then that is made explicit in the text.

In this document, "API" means the specification resources required for an external party (or in the case of 'private' APIs, an internal party) to implement software which uses the Publisher's Application Programming Interface.

The specification recommends the use of TLS, hence "HTTPS" and "https://" are used throughout.

2. Using the 'api-catalog' well-known URI

The api-catalog well-known URI is intended for HTTPS servers that publish APIs.

A Publisher supporting this URI:

4. The API Catalog document

The API Catalog is a document listing hyperlinks to a Publisher's APIs. The Publisher may host this API Catalog document at any URI(s) they choose. As illustration, the API Catalog document URI of https://www.example.com/my_api_catalog.json can be requested directly, or via a request to https://www.example.com/.well-known/api-catalog, which the Publisher will resolve to https://www.example.com/my_api_catalog.

The Publisher MUST publish the API Catalog document in the Linkset format application/linkset+json (section 4.2 of [RFC9264]). In addition, the Publisher MAY make additional formats available via content negotiation (section 5.3 of [HTTP]) to their /.well-known/api-catalog location. A non-exhaustive list of such formats that support the automated discovery, and machine (and human) usage of a Publisher's APIs, is listed below.

The API Catalog document MUST include hyperlinks to API endpoints, and is RECOMMENDED to include useful metadata, such as usage policies, API version information, links to the OpenAPI Specification [OAS] definitions for each API, etc.. If the Publisher does not include these metadata directly in the API Catalog document, they SHOULD make that metadata available at the API endpoint URIs they have listed (see Appendix A.2 for an example).

Some suitable API Catalog document formats include:

If a Publisher already lists their APIs in a format other than linkset but wish to utilise the /.well-known/api-catalog URI, then:

An API Catalog may itself contain links to other API Catalogs, by using the 'api-catalog' relation type for each link. An example of this is given in Appendix A.3.

5. Operational considerations

5.1. Accounting for APIs distributed across multiple domains

A Publisher ("example") may have their APIs hosted across multiple domains that they manage: e.g., at www.example.com, developer.example.com, apis.example.com, apis.example.net etc. They may also use a third-party API hosting provider which hosts APIs on a distinct domain.

To account for this scenario, it is RECOMMENDED that:

  • The Publisher also publish the api-catalog well-known URI at each of their API domains e.g. https://apis.example.com/.well-known/api-catalog, https://developer.example.net/.well-known/api-catalog etc.

  • An HTTPS GET request to any of these URIs returns the same result, namely, the API Catalog document.

  • Since the physical location of the API Catalog document is decided by the Publisher, and may change, the Publisher choose one of their instances of /.well-known/api-catalog as a canonical reference to the location of the latest API Catalog. The Publisher's other instances of ./well-known/api-catalog SHOULD redirect to this canonical instance of /.well-known/api-catalog to ensure the latest API Catalog is returned.

For example, if the Publisher's primary API portal is https://apis.example.com, then https://apis.example.com/.well-known/api-catalog SHOULD resolve to the location of the Publisher's latest API Catalog document. If the Publisher is also the domain authority for www.example.net, which also hosts a selection of their APIs, then a request to https://www.example.net/.well-known/api-catalog SHOULD redirect to https://apis.example.com/.well-known/api-catalog .

If the Publisher is not the domain authority for www.example.net - or any third-party domain that hosts any of the Publisher's APIs - then the Publisher MAY include a link in its own API Catalog to that third-party domain's API Catalog. For example, the API Catalog available at https://apis.example.com/.well-known/api-catalog) may list APIs hosted at apis.example.com and also link to the API Catalog hosted at https://www.example.net/.well-known/api-catalog using the "api-catalog" link relation:

{
  "linkset": [
    {
      "anchor": "https://www.example.com/.well-known/api-catalog",
      "item": [
        {
          "href": "https://developer.example.com/apis/foo_api"
        },
        {
          "href": "https://developer.example.com/apis/bar_api"
        },
        {
          "href": "https://developer.example.com/apis/cantona_api"
        }
      ],
      "api-catalog": "https://www.example.net/./well-known/api-catalog"
    }
  ]
}

5.2. Internal use of api-catalog for private APIs

A Publisher may wish to use the api-catalog well-known URI on their internal network, to signpost authorised users (e.g. company employees) towards internal/private APIs not intended for third-party use. This scenario may incur additional security considerations, as noted in Section 8.

5.3. Scalability guidelines

In cases where a Publisher has a large number of APIs, potentially deployed across multiple domains, then two challenges may arise:

  • Maintaining the catalog entries to ensure they are up to date and any errors corrected.

  • Restricting the catalog size to help reduce network and client-processing overheads.

In both cases a Publisher may benefit from grouping their APIs, providing an API Catalog document for each group - and use the main API Catalog hosted at /.well-known/api-catalog to provide links to these. For example a Publisher may decide to group their APIs according to a business category (e.g. 'gaming APIs', 'anti-fraud APIs' etc.) or a technology category (e.g. ''IOT', 'networks', 'AI' etc.), or any other criterion. This grouping may already be implicit where the Publisher has already published their APIs across multiple domains, e.g. at gaming.example.com, iot.example.net, etc.

Section 4.1 below shows how the API Catalog at /.well-known/api-catalog can use the api-catalog link relation to point to other API Catalogs.

The Publisher SHOULD consider caching and compression techniques to reduce the network overhead of large API Catalogs.

5.4. Monitoring and maintenance

Publishers are RECOMMENDED to follow operational best practice when hosting API Catalog(s), including but not limited to:

  • Health. The Publisher SHOULD monitor availability of the API Catalog, and consider alternate means to resolve requests to /.well-known/api-catalog during planned downtime of hosts.

  • Performance. Although the performance of APIs listed in an API Catalog can demand high transactions per second and low-latency response, the retrieval of the API Catalog itself to discover those APIs is less likely to incur strict performance demands. That said, the Publisher SHOULD monitor the response time to fulfil a request for the API Catalog, and determine any necessary improvements (as with any other Web resource the Publisher serves). For large API Catalogs, the Publisher SHOULD consider the techniques described in Section 5.3.

  • Usage. Since the goal of the api-catalog well-known URI is to facilitate discovery of APIs, the Publisher may wish to correlate requests to the /.well-known/api-catalog URI with subsequent requests to the API URIs listed in the catalog.

  • Current data. The Publisher SHOULD include the removal of stale API entries from the API Catalog as part of their API release lifecycle. The Publisher MAY decide to include metadata regarding legacy API versions or deprecated APIs to help users of those APIs discover up-to-date alternatives.

  • Correct metadata. The Publisher SHOULD include human and/or automated checks for syntax errors in the API Catalog. Automated checks include format validation (e.g. to ensure valid JSON syntax) and linting to enforce business rules - such as removing duplicate entries and ensuring descriptions are correctly named with valid values. A proofread of the API Catalog as part of the API release lifecycle is RECOMMENDED to detect any errors in business grammar (for example, an API entry that is described with valid syntax, but has been allocated an incorrect or outdated description.)

  • Security best practice, as set out in Section 8

5.5. Integration with existing API management frameworks

A Publisher may already utilise an API management framework to produce their API portfolio. These frameworks typically include the publication of API endpoint URIs, deprecation and redirection of legacy API versions, API usage policies and documentation, etc. The api-catalog well-known URI and API Catalog document are intended to complement API management frameworks by facilitating the discovery of the framework's outputs - API endpoints, usage policies and documentation - and are not intended to replace any existing API discovery mechanisms the framework has implemented.

Providers of such frameworks may include the production of an API Catalog and the publication of the /.well-known/api-catalog URI as a final pre-release (or post-release) step in the release management workflow. The following steps are recommended:

If the ./well-known/api-catalog URI has not been published previously, the framework provider should:

  • Collate and check the metadata for each API that will be included in the API Catalog. This metadata is likely to already exist in the framework.

  • Determine which metadata to include in the API Catalog, following the requirements set out in Section 4 and the considerations set out in Section 5.

  • Map the chosen metadata to the format(s) described in Section 4. Where only the hyperlinks to APIs are to be included in the API Catalog, then the structure suggested in Appendix A.2 may be followed. Where possible the API Catalog SHOULD include further metadata per the guidance in Section 4, in which case the structure suggested in Appendix A can be utilised and adapted (ensuring compliance to [RFC9264]) to reflect the nature of the chosen metadata.

  • Publish the /.well-known/api-catalog URI following the guidance set out in Section 2.

If the ./well-known/api-catalog URI has previously been published, the framework provider should:

  • Include a step in the release management lifecycle to refresh the API Catalog following any changes in API hyperlinks or published metadata. This could include placing triggers on certain metadata fields, so that as they are updated in pre-production on the API framework, the updates are pushed to a pre-production copy of the API Catalog to be pushed live when the release is published by the framework.

6. Conformance to RFC8615

The requirements in section 3 of [WELL-KNOWN] for defining Well-Known Uniform Resource Identifiers are met as described in the following sub-sections.

6.1. Path suffix

The api-catalog URI SHALL be appended to the /.well-known/ path-prefix for "well-known locations".

6.2. Formats and associated media types

A /.well-known/api-catalog location MUST support the Linkset [RFC9264] format of application/linkset+json, and MAY also support the other formats via content negotiation.

7. IANA Considerations

7.1. The api-catalog well-known URI

This specification registers the "api-catalog" well-known URI in the Well-Known URI Registry as defined by [WELL-KNOWN].

  • URI suffix: api-catalog

  • Change Controller: IETF

  • Specification document(s): THIS-RFC

  • Status: permanent

7.3. The api-catalog Profile URI

This specification registers "THIS-RFC-URL" in the "Profile URIs" registry according to [RFC7284].

  • Profile URI: THIS-RFC-URL

  • Common Name: API Catalog

  • Description: A profile URI to request or signal a linkset representing an API Catalog.

  • Reference: THIS-RFC

RFC Editor's Note: IANA is kindly requested to replace all instances of THIS-RFC and THIS-RFC-URL with the actual RFC number/URL once assigned.

8. Security Considerations

For all scenarios:

For the public-facing APIs scenario: security teams SHOULD additionally audit the API Catalog to ensure no APIs intended solely for internal use have been mistakenly included. For example, a catalog hosted on https://developer.example.com should not expose unnecessary metadata about any internal domains (e.g. https://internal.example.com).

For the internal/private APIs scenario: the Publisher SHOULD take steps to ensure that appropriate controls - such as CORS policies and access control lists - are in place to ensure only authorised roles and systems may access an internal api-catalog well-known URI.

9. References

9.1. Normative References

[HTTP]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, , <https://www.rfc-editor.org/rfc/rfc9110>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC6573]
Amundsen, M., "The Item and Collection Link Relations", RFC 6573, DOI 10.17487/RFC6573, , <https://www.rfc-editor.org/rfc/rfc6573>.
[RFC7284]
Lanthaler, M., "The Profile URI Registry", RFC 7284, DOI 10.17487/RFC7284, , <https://www.rfc-editor.org/rfc/rfc7284>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC9264]
Wilde, E. and H. Van de Sompel, "Linkset: Media Types and a Link Relation Type for Link Sets", RFC 9264, DOI 10.17487/RFC9264, , <https://www.rfc-editor.org/rfc/rfc9264>.
[WEB-LINKING]
Nottingham, M., "Web Linking", RFC 8288, DOI 10.17487/RFC8288, , <https://www.rfc-editor.org/rfc/rfc8288>.
[WELL-KNOWN]
Nottingham, M., "Well-Known Uniform Resource Identifiers (URIs)", RFC 8615, DOI 10.17487/RFC8615, , <https://www.rfc-editor.org/rfc/rfc8615>.

9.2. Informative References

[APIsjson]
Kin Lane and Steve Willmott, "APIs.json", , <http://apisjson.org/format/apisjson_0.16.txt>.
[HAL]
Mike Kelly, "JSON Hypertext Application Language", , <https://datatracker.ietf.org/doc/html/draft-kelly-json-hal-11>.
[OAS]
Darrel Miller, Jeremy Whitlock, Marsh Gardiner, Mike Ralphson, Ron Ratovsky, and Uri Sarid, "OpenAPI Specification 3.1.0", , <https://spec.openapis.org/oas/latest>.
[RESTdesc]
Ruben Verborgh, Erik Mannens, Rick Van de Walle, and Thomas Steiner, "RESTdesc", , <http://apisjson.org/format/apisjson_0.16.txt>.
[RFC8631]
Wilde, E., "Link Relation Types for Web Services", RFC 8631, DOI 10.17487/RFC8631, , <https://www.rfc-editor.org/rfc/rfc8631>.
[WebAPIext]
Mike Ralphson and Nick Evans, "WebAPI type extension", , <https://webapi-discovery.github.io/rfcs/rfc0001.html>.

Appendix A. Example API Catalog documents

This section is informative and provides and example of an API Catalog document using the RECOMMENDED linkset format.

A.1. Using Linkset with RFC8615 relations

This example uses the linkset format [RFC9264], and the following link relations defined in [RFC8631]:

  • "service-desc", used to link to a description of the API that is primarily intended for machine consumption.

  • "service-doc", used to link to API documentation that is primarily intended for human consumption.

  • "service-meta", used to link to additional metadata about the API, and is primarily intended for machine consumption.

  • "status", used to link to the API status (e.g. API "health" indication etc.) for machine and/or human consumption.

Client request:

GET .well-known/api-catalog HTTP/1.1
Host: example.com
Accept: application/linkset+json

Server response:

HTTP/1.1 200 OK
Date: Mon, 01 Jun 2023 00:00:01 GMT
Server: Apache-Coyote/1.1
Content-Type: application/linkset+json;
    profile="THIS-RFC-URL"

{
  "linkset": [
  {
    "anchor": "https://developer.example.com/apis/foo_api",
    "service-desc": [
      {
        "href": "https://developer.example.com/apis/foo_api/spec",
        "type": "application/yaml"
      }
    ],
    "status": [
      {
        "href": "https://developer.example.com/apis/foo_api/status",
        "type": "application/json"
      }
    ],
    "service-doc": [
      {
        "href": "https://developer.example.com/apis/foo_api/doc",
        "type": "text/html"
      }
    ],
    "service-meta": [
      {
        "href": "https://developer.example.com/apis/foo_api/policies",
        "type": "text/xml"
      }
    ]
  },
  {
    "anchor": "https://developer.example.com/apis/bar_api",
    "service-desc": [
      {
        "href": "https://developer.example.com/apis/bar_api/spec",
        "type": "application/yaml"
      }
    ],
    "status": [
      {
        "href": "https://developer.example.com/apis/bar_api/status",
       "type": "application/json"
      }
    ],
    "service-doc": [
      {
        "href": "https://developer.example.com/apis/bar_api/doc",
        "type": "text/plain"
      }
    ]
  },
  {
    "anchor": "https://apis.example.net/apis/cantona_api",
    "service-desc": [
      {
        "href": "https://apis.example.net/apis/cantona_api/spec",
        "type": "text/n3"
      }
    ],
    "service-doc": [
      {
        "href": "https://apis.example.net/apis/cantona_api/doc",
        "type": "text/html"
      }
    ]
  }
  ]
}

A.2. Using Linkset with bookmarks

This example also uses the linkset format [RFC9264], listing the API endpoints in an array of bookmarks. Each link shares the same context anchor (the well-known URI of the API Catalog) and "item" [RFC9264] link relation (to indicate they are an item in the catalog). The intent is that by following a bookmark link, a machine-client can discover the purpose and usage policy for each API, hence the document targeted by the bookmark link should support this.

Client request:

GET .well-known/api-catalog HTTP/1.1
Host: example.com
Accept: application/linkset+json

Server response:

HTTP/1.1 200 OK
Date: Mon, 01 Jun 2023 00:00:01 GMT
Server: Apache-Coyote/1.1
Content-Type: application/linkset+json;
    profile="THIS-RFC-URL"

{ "linkset":
 [
   { "anchor": "https://www.example.com/.well-known/api-catalog",
     "item": [
       {"href": "https://developer.example.com/apis/foo_api"},
       {"href": "https://developer.example.com/apis/bar_api"},
       {"href": "https://developer.example.com/apis/cantona_api"}
     ]
   }
 ]
}

In this example, a request to the /.well-known/api-catalog URI returns an array of links of relation type 'api-catalog'. This can be useful to Publishers with a large number of APIs, who wish to group them in smaller catalogs (as described in Section 5.3).

Client request:

GET .well-known/api-catalog HTTP/1.1
Host: example.com
Accept: application/linkset+json

Server response:

HTTP/1.1 200 OK
Date: Mon, 01 Jun 2023 00:00:01 GMT
Server: Apache-Coyote/1.1
Content-Type: application/linkset+json;
    profile="THIS-RFC-URL"

{
  "linkset": [
    {
      "anchor": "https://www.example.com/.well-known/api-catalog",
      "api-catalog": [
        {
          "href": "https://apis.example.com/iot/api-catalog"
        },
        {
          "href": "https://ecommerce.example.com/api-catalog"
        },
        {
          "href": "https://developer.example.com/gaming/api-catalog"
        }
      ]
    }
  ]
}

Appendix B. Acknowledgements

Thanks to Jan Algermissen, Phil Archer, Tim Bray, Ben Bucksch, Sanjay Dalal, David Dong, Mallory Knodel, Max Maton, Darrel Miller, Mark Nottingham, Roberto Polli, Joey Salazar, Rich Salz, Herbert Van De Sompel, Tina Tsou and Erik Wilde for their reviews, suggestions and support.

Author's Address

Kevin Smith
Vodafone
URI: https://www.vodafone.com