Skip to content

Infrastructural Components⚓︎

The previous page showed how to define the application components that implement the services exposed by a data product through its ports. This page explains how to define the infrastructural components required to execute these applications.

Infrastructural Component Object⚓︎

Infrastructural components are internal components of a data product. This means that, while the interface components are publicly available on demand to all consumers, access to infrastructural components can be restricted.

All internal components of a data product are contained in the Internal Components Object, accessible through the internalComponents field in the root Data Product Descriptor Entity. Within this object, the field infrastructuralComponents contains the list of all Infrastructural Component of the data product.

open-data-mesh descriptor components

Fields⚓︎

An Infrastructural Component has the following two mandatory fields:

  • fullyQualifiedName (string:fqn): This is the unique universal identifier of the component. It MUST be a URN of the form urn:dpds:{mesh-namespace}:dataproducts:{product-name}:{product-major-version}:infrastructure:{infra-name}.
  • version (string:version): This is the semantic version number of the data product's infrastructural component. The version of infrastructural components has no direct impact on the version of the data product they belong to. A change in the major version of an infrastructural component can anyway impact the version of one interface component and so indirectly the version of the data product.

Other non-mandatory descriptive fields like platform and infrastructureType are also available. Moreover, the Infrastructural Component](../resources/specifications/1.0.0-DRAFT.md#infrastructure-component) can be extended with other fields with "x-" prefix as needed.

Example⚓︎

The following example shows two infrastructural components that respectively describe

  1. The topology of Kafka topics required to store
    • technical events generated by the cdcIngestion application while offloading data from the TMS source.
    • domain events generated by eventProcessor application as a composition of technical events that belong to the same transaction intercepted by cdcIngestion on the TMS source.
  2. The schema and tables required by dbSinkConnector application to materialize from streamed domain events the updated state of each Trip.
JSON
{
    "infrastructuralComponents": [
        {
            "fullyQualifiedName": "urn:dpds:com.company-xyz:dataproducts:tripExecution:1:infrastructure:eventStore",
            "version": "1.0.1",
            "description": "The kafka topics topology required to store technical events offloaded from TMS by the CDC and the domain events generetaed by eventProcessor application",
            "platform": "aws:eu-south-1:confluent",
            "infrastructureType":"storage-resource"
        }, {
            "fullyQualifiedName": "urn:dpds:com.company-xyz:dataproducts:tripExecution:1:infrastructure:stateStore",
            "version": "1.0.0",
            "description": "The database used by dbSinkConnector application to store the Trip updated state",
            "platform": "aws:eu-south-1:postgress",
            "infrastructureType":"storage-resource"
        }
    ]
}

The information provided so far are useful to track the infrastructural assets related to a specific data product. There is not anyway sufficient information to allow an external agent to provision the infrastructure autonomously. To this end in the Infrastructural Component there is the provisioningService filed that is described in much detail in the next sections.

Trip Execution Data Product Descriptor⚓︎

The Trip Execution Data Product is composed of two infrastructural components.

The following files contain the complete definition of these infrastructural components:

You can read it on GitHub or cut and paste its content to JSON Crack for more interactive visualization.

Summary⚓︎

This page has shown that:

  • a data product is composed not only of its public interface but also of internal components
  • infrastructural components required by application components that implement services exposed by a data product through its ports are internal components described through an Infrastructural Component.
  • Infrastructural Component contains general information related to the specific infrastructure component but also all the required information to provision the infrastructure in an automatic and reproducible way.
  • the DPDS uses infrastructure as code (IaC) to define how to provision infrastructure but does not enforce the use of any particular IaC tool, preserving in this way its technology independence.

The next page describes how to reuse portions of a data product descriptor document to make it more readable and less redundant, reducing file size and maintenance cost.