Skip to content

Application Components⚓︎

The previous page showed how to define the interface of services exposed by a data product. This page explains how to define the application components that implement these services.

Application Component⚓︎

Application components are internal components of a data product. This means that, while the interface components are publicly available on demand to all consumers, access to application components can be restricted.

All internal components of a data product are contained in the Internal Components Object, accessible through the internalComponents field in the root Data Product Descriptor Entity. Within this object, the field applicationComponents contains the list of all Application Components of the data product.

open-data-mesh descriptor components

Fields⚓︎

An Application Component has the following two mandatory fields:

  • fullyQualifiedName (string:fqn): This is the unique universal identifier of the component. It MUST be a URN of the form urn:dpds:{mesh-namespace}:dataproducts:{product-name}:{product-major-version}:applications:{app-name}
  • version (string:version): This is the semantic version number of the data product's application component. The version of application components has no direct impact on the version of the data product they belong to. A change in the major version of an application component can anyway impact the version of one interface component and so indirectly the version of the data product.

Other non-mandatory descriptive fields like platform and applicationType are also available. Moreover, the Application Component can be extended with other fields with "x-" prefix as needed.

Example⚓︎

The following example shows three application components that respectively describe 1. an application that loads data from a downstream legacy system (i.e. TMS) in real-time to CDC technology (i.e. Debezium). 1. a streaming application that transforms the technical events loaded in the streaming platform (i.e. Confluent platform) by the previous application in domain events. a streaming application that uses the domain events generated by the previous application to update the trip status in a downstream table stored in a Postgres DB.

All three applications are deployed in eu-south-1 region of AWS Cloud.

JSON
{
    "applicationComponents": [
        {
            "fullyQualifiedName": "urn:dpds:com.company-xyz:dataproducts:tripExecution:1:applications:cdcIngestion",
            "version": "1.0.1",
            "description": "The app that offload the Trip aggregate from  TMS using CDC",
            "platform": "aws:eu-south-1:debezium",
            "applicationType":"stream-sourcing"
        }, {
            "fullyQualifiedName": "urn:dpds:com.company-xyz:dataproducts:tripExecution:1:applications:eventProcessor",
            "version": "1.1.0",
            "description": "The app that transform tecnical events in domain events",
            "platform": "aws:eu-south-1:confluent",
            "applicationType":"stream-transformation"
        }, {
            "fullyQualifiedName": "urn:dpds:com.company-xyz:dataproducts:tripExecution:1:applications:dbSinkConnector",
            "version": "1.1.3",
            "description": "The app that store the updated state of a Trip in the target postgres db",
            "platform": "aws:eu-south-1:confluent",
            "applicationType":"stream-transformation"
        }
    ]
}

The information provided so far are useful to track the application assets related to a specific data product. There is not anyway sufficient information to allow an external agent to build and deploy the application autonomously. To this end in the Application Component, there are the buildService and deploymentService fields that are described in much detail in the next sections.

Build Service Object⚓︎

The Build Info Object contains the information required to build an application component. DPDS uses integration pipelines as code to guarantee the possibility to automatize the build process in a reproducible way. Anyway, to maintain technological independence the DPDS does not enforce the usage of any specific CICD tool.

Fileds⚓︎

A Build Info Object has the following three mandatory fields:

  • service ( string:name or External Resource Object): This is the logical name or the actual endpoint of the service to call to build the application component. If the logical name (ex. jenkins, aws-pipelines, azure-devops, etc...) is used as field value the actual endpoint value resolution is demanded by the data product experience plane* of the underlying platform.
  • template (object or External Resource Object): This is the definition of the pipeline to execute to build the application. It is passed as is to the build service implementation specified in the service field.
  • configurations (object or External Resource Object): This is the map of all configuration properties that must be used by the build service at build time. It is passed as is to the build service implementation specified in the service field.

Example⚓︎

The following example shows an application that could be built using a Jenkins-backed build service. The jenkinsfile that contains the description of the pipeline is available on an external bitbucket repository. The only configuration parameter passed into the build service* at runtime is the target stage.

JSON
{
    "service": "jenkins",
    "template": {
        "mediaType": "text/jenkinsfile",
        "$href": "https://bitbucket.org/company-xyz/trip-execution-app-sourcing/src/master/jenkinsfile"
    },
    "configurations": {
        "TARGET_STAGE": "BUILD",
    }
}

Deployment Service Object⚓︎

The Deployment Info Object contains the information required to deploy an application component. DPDS uses integration pipelines as code to guarantee the possibility to automatize the deployment process in a reproducible way. Anyway, to maintain technological independence the DPDS does not enforce the usage of any specific CICD tool.

Fileds⚓︎

A Deployment Info Object has the following three mandatory fields:

  • service ( string:name or External Resource Object): This is the logical name or the actual endpoint of the service to call to deploy the application component. If the logical name (ex. jenkins, aws-pipelines, azure-devops, etc...) is used as field value the actual endpoint value resolution is demanded by the data product experience plane* of the underlying platform.
  • template (object or External Resource Object): This is the definition of the pipeline to execute to deploy the application. It is passed as is to the deployment service implementation specified in the service field.
  • configurations (object or External Resource Object): This is the map of all configuration properties that must be used by the build service at deployment time. It is passed as is to the deployment service implementation specified in the service field.

Example⚓︎

The following example shows an application that could be deployed using a Jenkins-backed deployment service. The jenkinsfile that contains the description of the pipeline is available on an external bitbucket repository. The only configuration parameters passed into the deployment service* at runtime are the initial stage and the end state that identify the portion of the pipeline that must be executed.

JSON
{
    "service": {
        "mediaType": "application",
        "$href": "https://jenkins.company-xyz.com/api/v1/planes/utility/integration-services/builds"
    },
    "template": {
        "mediaType": "text/jenkinsfile",
        "$href": "https://bitbucket.org/company-xyz/trip-execution-app-sourcing/src/master/jenkinsfile"
    },
    "configurations": {
        "START_STAGE": "BUILD",
        "TARGET_STAGE": "DEPLOY",
    }
}

Trip Execution Data Product Descriptor⚓︎

The Trip Execution Data Product is composed of three application components.

The following files contain the complete definition of these application components:

You can read it on GitHub or cut and paste its content to JSON Crack for more interactive visualization.

Summary⚓︎

This page has shown that:

  • a data product is composed not only of its public interface but also of internal components
  • applications that implement the exposed interfaces are internal components described through an Application Component.
  • Application Component contains general information related to the specific application but also all the required information to build and deploy the application in an automatic and reproducible way.
  • the DPDS uses pipeline as code to define how to build and deploy an application but does not enforce the use of any particular CICD tool, preserving in this way its technology independence.

The next page describes the contents of the infrastructuralComponents field used to provide all the required information to provision the infrastructure required by application components that implement the services exposed by the product through its ports.

Last update: November 26, 2022
Created: November 7, 2022