Google Cloud Pub/Sub Protocol Binding for CloudEvents - Version 1.0

Abstract

The Google Cloud Pub/Sub Protocol Binding for CloudEvents defines how events are mapped to Pub/Sub 1.1 request and response messages.

Status of this document

This document is a working draft.

1. Introduction

CloudEvents is a standardized and protocol-agnostic definition of the structure and metadata description of events. This specification defines how the elements defined in the CloudEvents specification are to be used in Pub/Sub requests and response messages.

1.1. Conformance

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119.

1.2. Relation to Pub/Sub

Events can be produced using a Topic or consumed using Push or Pull based Subscriptions.

1.3. Content Modes

The CloudEvents specification defines three content modes for transferring events: structured, binary and batch. The Pub/Sub binding does not currently support batch mode. Every compliant implementation SHOULD support both structured and binary modes.

In the structured content mode, event metadata attributes and event data are placed into the Pub/Sub message body section using an event format.

In the binary content mode, the value of the event data attribute is placed into the Pub/Sub payload data as-is, with the datacontenttype attribute value declaring its media type; all other event attributes are mapped to Pub/Sub attributes.

1.4. Event Formats

Event formats, used within the structured content mode, define how an event is expressed in a particular data format. All implementations of this specification MUST support the JSON event format, but MAY support any additional, including proprietary, formats.

1.5. Security

This specification does not introduce any new security features for Pub/Sub, or mandate specific existing features to be used.

2. Use of CloudEvents Attributes

This specification does not further define any of the CloudEvents event attributes.

This mapping is intentionally robust against changes, including the addition and removal of event attributes, and also accommodates vendor extensions to the event metadata.

2.1. datacontenttype Attribute

The datacontenttype attribute SHOULD contain a RFC2046 compliant media-type expression.

2.2. Data

The event data (also known as the event payload) SHOULD contain opaque application data that is encoded as declared by the datacontenttype attribute.

An application MAY hold the information in any in-memory representation of its choosing, but as the value is transposed into a Pub/Sub message as defined in this specification, the assumption is that the event data is represented as a sequence of bytes.

For instance, if the declared datacontenttype is application/json;charset=utf-8, the expectation is that the event data is represented as UTF-8 encoded JSON text Pub/Sub message data.

3. Pub/Sub Message Mapping

With Pub/Sub, the content mode is chosen by the sender of the event. Protocol usage patterns that might allow solicitation of events using a particular content mode might be defined by an application, but are not defined here.

The receiver of the event can distinguish between the two content modes by inspecting the Pub/Sub Attribute content-type of the Pub/Sub message. If the attribute is present and its value is prefixed with the CloudEvents media type application/cloudevents, indicating the use of a known event format, the receiver SHOULD use structured mode, otherwise it MUST default to binary mode.

If a receiver finds a CloudEvents media type as per the above rule, but with an event format that it cannot handle, for instance application/cloudevents+avro, it MAY still treat the event as binary and forward it to another party as-is.

3.1. Binary Content Mode

The binary content mode accommodates any shape of event data, and allows for efficient transfer and without transcoding effort.

The binary content mode can only be used if the metadata attributes fit within the limits of Pub/Sub messages.

3.1.1. content-type

For the binary mode, the Pub/Sub Attribute content-type, if present, MUST be used as the CloudEvents datacontenttype attribute.

3.1.2. Event Data Encoding

The event data byte-sequence MUST be used as the value of the Pub/Sub message data payload.

3.1.3. Metadata Attributes

All CloudEvents attributes MUST be individually mapped to and from distinct Pub/Sub message attributes.

3.1.3.1 Pub/Sub Attribute Names

The naming convention for the Pub/Sub attribute mapping of the CloudEvents core context attributes is that each attribute name MUST be prefixed with "ce-".

Examples:

* `time` maps to `ce-time`
* `id` maps to `ce-id`
* `specversion` maps to `ce-specversion`

Attribute names are lower-case.

3.1.3.2 Pub/Sub Attribute Values

The value for each Pub/Sub attribute MUST be constructed from the respective CloudEvents attribute type's canonical string representation.

3.1.4 Examples

This example shows the binary mode mapping of an event to a Pub/Sub Message:

--- Attributes ---
{
  "ce-specversion": "1.0",
  "ce-type": "com.example.someevent",
  "ce-time": "2020-03-10T03:56:24Z",
  "ce-id": "1234-1234-1234",
  "ce-source": "/mycontext/subcontext",
    .... further attributes ...
  "ce-datacontenttype": "application/json; charset=utf-8",
}
--- Data ---
{
    ... application data ...
}

This example show a publish request on the REST API of the above Pub/Sub Message:

POST https://pubsub.googleapis.com/v1/{topic}:publish HTTP/1.1
Content-Type: application/json; charset=utf-8
Authorization: Bearer ... token ...

{
  "messages": [
    {
      "attributes": {
        "ce-specversion": "1.0",
        "ce-type": "com.example.someevent",
        "ce-time": "2020-03-10T03:56:24Z",
        "ce-id": "1234-1234-1234",
        "ce-source": "/mycontext/subcontext",
          .... further attributes ...
        "ce-datacontenttype": "application/json; charset=utf-8",
      },
      "data": "... base64 encoded Data JSON (see above) ..."
    }
  ]
}

3.2. Structured Content Mode

The structured content mode keeps event metadata and data together in the payload, allowing simple forwarding of the same event across multiple routing hops, and across multiple transports.

3.2.1. content-type

The Pub/Sub Attribute content-type MUST be set to the media type of an event format.

Example for the JSON format:

"content-type": "application/cloudevents+json; charset=UTF-8"

3.2.2. Event Data Encoding

The chosen event format defines how all attributes and event data are represented.

The event metadata and data are then rendered in accordance with the event format specification and the resulting data becomes the Pub/Sub message data.

3.2.3. Metadata Attributes

Implementations MAY include the same Pub/Sub attributes as defined for the binary mode.

All CloudEvents metadata attributes MUST be mapped into the payload, even if they are also mapped into Pub/Sub attributes.

3.2.4 Examples

This example shows the structured mode mapping of an event to a Pub/Sub Message:

--- Attributes ---
{
  "content-type": "application/cloudevents+json; charset=utf-8"
}
--- Data ---
{
  "specversion": "1.0",
  "type": "com.example.someevent",
  "time": "2018-04-05T03:56:24Z",
  "id": "1234-1234-1234",
  "source": "/mycontext/subcontext",
  "datacontenttype": "application/json; charset=utf-8",
    .... further attributes ...
  "data": {
    ... application data ...
  }
}

This example show a publish request on the REST API of the above Pub/Sub Message:

POST https://pubsub.googleapis.com/v1/{topic}:publish HTTP/1.1
Content-Type: application/json; charset=utf-8
Authorization: Bearer ... token ...

{
  "messages": [
    {
      "attributes": {
        "content-type": "application/cloudevents+json; charset=utf-8"
      },
      "data": "... base64 encoded Data JSON (see above) ..."
    }
  ]
}

4. References

PUBSUB Google Cloud Pub/Sub
PUBSUB-MESSAGE The Google Cloud Pub/Sub message
PUBSUB-MESSAGE-QUOTAS The Google Cloud Pub/Sub message quotas
RFC2046 Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types
RFC2119 Key words for use in RFCs to Indicate Requirement Levels
RFC3629 UTF-8, a transformation format of ISO 10646
RFC3986 Uniform Resource Identifier (URI): Generic Syntax

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!