Application Pipeline¶

Combining multiple feature of the Industrial Edge Control (iectl) allows the automatic build and deployment of applications, no matter if through a CI/CD pipeline or a manually executed script.

The steps to achieve this are:

Set up static configuration and secrets required for the application and its deployment
Determine the app id and next version from already deployed versions of the app, if any
Build the docker images of the application and save them as tar files
Alternatively, pull external images from a registry
Create the application file including the reverse proxy and app configuration
Finally, deploy the application to the Industrial Edge Management and Device

Static Configuration and Secrets¶

To create an Industrial Edge Application it is necessary to define some constants like the application and repository name. Additionally the application could define its application id at this stage as well. Combined with a externally provided version number this would allow to skip the next step of determining the app id and next version to deploy.

Local EnvironmentGitLab CIGitHub Actions

export APP_NAME="My Example Application"
export REPO_NAME="my-application"
export APP_DESC="This is an example application."
# Optional
APP_ID="PlKqV9txs9eJeiCK7UmEJfA8zj6fl4Vf"
NEXT_VERSION="1.0.0"
# As well as Deployment related variables
export DEVICE_NAME="ievd-erl-001"

variables:
  APP_NAME: "My Example Application"
  REPO_NAME: "my-application"
  APP_DESC: "This is an example application."
  # Optional
  APP_ID: "PlKqV9txs9eJeiCK7UmEJfA8zj6fl4Vf"
  NEXT_VERSION: "1.0.0"
  # As well as Deployment related variables
  DEVICE_NAME: "ievd-erl-001"

env:
  APP_NAME: "My Example Application"
  REPO_NAME: "my-application"
  APP_DESC: "This is an example application."
  # Optional
  APP_ID: "PlKqV9txs9eJeiCK7UmEJfA8zj6fl4Vf"
  NEXT_VERSION: "1.0.0"
  # As well as Deployment related variables
  DEVICE_NAME: "ievd-erl-001"

Secrets like the credentials for the Industrial Edge Management need to be provided as well. Make sure the IEM url starts with https:// and the credentials have sufficient permissions to deploy applications.

Local EnvironmentGitLab CIGitHub Actions

IEM_URL="https://iem.example.com"
IEM_USER="apiuser"
IEM_PASSWORD="********"

set -a
source .env
set +a

It is also possible to simulate some environment variables provided by the CI/CD system locally by exporting them:

export CI_COMMIT_MESSAGE="$(git log -1 --pretty=%B)"
export CI_COMMIT_TAG="$(git describe --tags --exact-match 2>/dev/null || true)"

Consult the official documentation for GitLab CI/CD variables to set up the required secrets:

GitLab CI/CD variables

Consult the official documentation for GitHub Actions secrets:

Using secrets in GitHub Actions

Hint

If you want to deploy to an IEM with self-signed certificates, don't forget to set the IEM_TLS_SKIP variable to true to skip the certificate validation in the iectl commands.

Determine the app id and next version¶

For the Industrial Edge Platform to understand that two applications are the same the application ID and repository name must match. Using the iectl tool it is possible to query the application ID and the highest version currently in the IEM. With the semver tool the next version can be calculated.

Local EnvironmentGitLab CIGitHub Actions

iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PW}" --name "iemconfig"

if APP_DETAILS=$(iectl iem device-apps app-details --app-name "${APP_NAME}" 2>/dev/null); then
  APP_ID=$(printf '%s' "$APP_DETAILS" | jq -r '.applicationId')
  HIGHEST_VERSION=$(printf '%s' "$APP_DETAILS" | jq -r '.versions[].version' | sort -V | tail -1)

  # Bump to next release candidate version
  if [ -n "$HIGHEST_VERSION" ] && [ "$HIGHEST_VERSION" != "null" ]; then
    NEXT_VERSION=$(semver bump prerel "rc." "${HIGHEST_VERSION}")
  else
    NEXT_VERSION="1.0.0-rc1"
  fi
else
  # App doesn't exist - generate new ID and start with first version
  APP_ID=$(openssl rand -hex 16)
  NEXT_VERSION="1.0.0-rc1"
  echo "App does not exist yet. Generated new APP_ID: ${APP_ID}"
fi

determine-version:
  stage: prepare
  script:
    - iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PASSWORD}" --name "iemconfig"
    - |
      if APP_DETAILS=$(iectl iem device-apps app-details --app-name "${APP_NAME}" 2>/dev/null); then
        APP_ID=$(printf '%s' "$APP_DETAILS" | jq -r '.applicationId')
        HIGHEST_VERSION=$(printf '%s' "$APP_DETAILS" | jq -r '.versions[].version' | sort -V | tail -1)

        # Bump to next release candidate version
        if [ -n "$HIGHEST_VERSION" ] && [ "$HIGHEST_VERSION" != "null" ]; then
          NEXT_VERSION=$(semver bump prerel "rc." "${HIGHEST_VERSION}")
        else
          NEXT_VERSION="1.0.0-rc1"
        fi
      else
        # App doesn't exist - generate new ID and start with first version
        APP_ID=$(openssl rand -hex 16)
        NEXT_VERSION="1.0.0-rc1"
        echo "App does not exist yet. Generated new APP_ID: ${APP_ID}"
      fi
    - echo "APP_ID=${APP_ID}" >> build.env
    - echo "NEXT_VERSION=${NEXT_VERSION}" >> build.env
  artifacts:
    reports:
      dotenv: build.env

jobs:
  determine-version:
    runs-on: ubuntu-latest
    outputs:
      app_id: ${{ steps.version.outputs.app_id }}
      next_version: ${{ steps.version.outputs.next_version }}
    steps:
      - name: Determine app version
        id: version
        run: |
          iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PASSWORD}" --name "iemconfig"

          if APP_DETAILS=$(iectl iem device-apps app-details --app-name "${APP_NAME}" 2>/dev/null); then
            APP_ID=$(printf '%s' "$APP_DETAILS" | jq -r '.applicationId')
            HIGHEST_VERSION=$(printf '%s' "$APP_DETAILS" | jq -r '.versions[].version' | sort -V | tail -1)

            # Bump to next release candidate version
            if [ -n "$HIGHEST_VERSION" ] && [ "$HIGHEST_VERSION" != "null" ]; then
              NEXT_VERSION=$(semver bump prerel "rc." "${HIGHEST_VERSION}")
            else
              NEXT_VERSION="1.0.0-rc1"
            fi
          else
            # App doesn't exist - generate new ID and start with first version
            APP_ID=$(openssl rand -hex 16)
            NEXT_VERSION="1.0.0-rc1"
            echo "App does not exist yet. Generated new APP_ID: ${APP_ID}"
          fi

          echo "app_id=${APP_ID}" >> $GITHUB_OUTPUT
          echo "next_version=${NEXT_VERSION}" >> $GITHUB_OUTPUT

Build the docker images¶

To create the application file in the later stage the docker images of the application need to be available as tar files.

Local EnvironmentGitLab CIGitHub Actions

docker build -t ${REPO_NAME}/frontend:${NEXT_VERSION} ${REPO_NAME}/frontend/
mkdir -p tars
docker save ${REPO_NAME}/frontend:${NEXT_VERSION} -o tars/frontend.tar

build:
  image: docker:29
  services:
    - docker:29-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  script:
    - docker build -t ${REPO_NAME}/frontend:${NEXT_VERSION} ${REPO_NAME}/frontend/
    - mkdir -p tars
    - docker save ${REPO_NAME}/frontend:${NEXT_VERSION} -o tars/frontend.tar
  artifacts:
    paths:
      - tars/

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Build and export
        uses: docker/build-push-action@v6
        with:
          context: ${{ env.REPO_NAME }}/frontend/
          tags: ${{ env.REPO_NAME }}/frontend:${{ needs.determine-version.outputs.next_version }}
          outputs: type=docker,dest=tars/frontend.tar
      - name: Upload image artifacts
        uses: actions/upload-artifact@v4
        with:
          name: docker-images
          path: tars/

Pull external images¶

If your application relies on external images, e.g. from Docker Hub or a private registry, you can pull them and save them as tar files as well to make them available for the application file creation.

Local EnvironmentGitLab CIGitHub Actions

docker pull postgres:latest
docker save postgres:latest -o tars/database.tar

pull:
  image: gcr.io/go-containerregistry/crane:debug
  script:
  - crane pull postgres:latest tars/database.tar
  artifacts:
    paths:
      - tars/

jobs:
  pull:
    runs-on: ubuntu-latest
    steps:
      - name: Set up crane
        uses: imjasonh/setup-crane@v0.1
      - name: Pull and save external images
        run: |
          mkdir -p tars
          crane pull postgres:latest tars/database.tar
      - name: Upload image artifacts
        uses: actions/upload-artifact@v4
        with:
          name: external-images
          path: tars/

Create the application file¶

With the application information and the docker images the application binary can now be generated. After initializing the workspace the app can be created using the application id from the first step. When creating the application version, the nginx configuration as well as the paths to the image tar files are provided as well. Optionally app configurations can be added as well. Finally the app gets exported as a .app file.

Local EnvironmentGitLab CIGitHub Actions

iectl config add publisher --name "publisherconfig" --workspace ".ws"
iectl publisher ws init

iectl publisher sa create --appname "${APP_NAME}" --reponame "${REPO_NAME}" --appdescription "${APP_DESC}" --iconpath "${REPO_NAME}/icon.png" $([ -n "${APP_ID}" ] && echo "--appId ${APP_ID}")

iectl publisher sa version create --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" \
    --yamlpath ${REPO_NAME}/docker-compose.yml \
    --imagetarjson "$(jq -n --arg frontend "tars/frontend.tar" --arg backend "tars/backend.tar" --arg database "tars/database.tar" '{frontend: $frontend, backend: $backend, database: $database}')" \
    --nginxjson "$(jq -c 'map_values(map(.headers |= tojson))' ${REPO_NAME}/nginx.json)" \
    --changelogs "${CI_COMMIT_MESSAGE}" --defaultmountpath \
    --redirecttype "FromBoxReverseProxy" --redirectsection "frontend" --redirecturl "${REPO_NAME}/"

iectl publisher sa app-config add --appname "${APP_NAME}" --configname "AppConfig" --configdescription "Configure the Application Title" --hostpath "./cfg-data/" --templatename "ConfigTemplate" --templatedescription "Configuration Template Schema v1" --jsonschema --filepath "${REPO_NAME}/config/config.json"

mkdir -p export
iectl publisher sa version export --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" --exportpath "export"

create-app:
  stage: package
  needs:
    - job: determine-version
      artifacts: true
    - job: build
      artifacts: true
    - job: pull
      artifacts: true
  script:
    - iectl config add publisher --name "publisherconfig" --workspace ".ws"
    - iectl publisher ws init
    - >
      iectl publisher sa create --appname "${APP_NAME}" --reponame "${REPO_NAME}" --appdescription "${APP_DESC}" --iconpath "${REPO_NAME}/icon.png"
      $([ -n "${APP_ID}" ] && echo "--appId ${APP_ID}")
    - |
      iectl publisher sa version create --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" \
          --yamlpath ${REPO_NAME}/docker-compose.yml \
          --imagetarjson "$(jq -n --arg frontend "tars/frontend.tar" --arg backend "tars/backend.tar" --arg database "tars/database.tar" '{frontend: $frontend, backend: $backend, database: $database}')" \
          --nginxjson "$(jq -c 'map_values(map(.headers |= tojson))' ${REPO_NAME}/nginx.json)" \
          --changelogs "${CI_COMMIT_MESSAGE}" --defaultmountpath \
          --redirecttype "FromBoxReverseProxy" --redirectsection "frontend" --redirecturl "${REPO_NAME}/"
    - iectl publisher sa app-config add --appname "${APP_NAME}" --configname "AppConfig" --configdescription "Configure the Application Title" --hostpath "./cfg-data/" --templatename "ConfigTemplate" --templatedescription "Configuration Template Schema v1" --jsonschema --filepath "${REPO_NAME}/config/config.json"
    - mkdir -p export
    - iectl publisher sa version export --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" --exportpath "export"
  artifacts:
    paths:
      - export/

jobs:
  create-app:
    runs-on: ubuntu-latest
    needs: [determine-version, build, pull]
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Download docker image artifacts
        uses: actions/download-artifact@v4
        with:
          name: docker-images
          path: tars/
      - name: Download external image artifacts
        uses: actions/download-artifact@v4
        with:
          name: external-images
          path: tars/
      - name: Create application file
        env:
          APP_ID: ${{ needs.determine-version.outputs.app_id }}
          NEXT_VERSION: ${{ needs.determine-version.outputs.next_version }}
        run: |
          iectl config add publisher --name "publisherconfig" --workspace ".ws"
          iectl publisher ws init

          iectl publisher sa create --appname "${APP_NAME}" --reponame "${REPO_NAME}" --appdescription "${APP_DESC}" --iconpath "${REPO_NAME}/icon.png" $([ -n "${APP_ID}" ] && echo "--appId ${APP_ID}")

          iectl publisher sa version create --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" \
              --yamlpath ${REPO_NAME}/docker-compose.yml \
              --imagetarjson "$(jq -n --arg frontend "tars/frontend.tar" --arg backend "tars/backend.tar" --arg database "tars/database.tar" '{frontend: $frontend, backend: $backend, database: $database}')" \
              --nginxjson "$(jq -c 'map_values(map(.headers |= tojson))' ${REPO_NAME}/nginx.json)" \
              --changelogs "${CI_COMMIT_MESSAGE}" --defaultmountpath \
              --redirecttype "FromBoxReverseProxy" --redirectsection "frontend" --redirecturl "${REPO_NAME}/"

          iectl publisher sa app-config add --appname "${APP_NAME}" --configname "AppConfig" --configdescription "Configure the Application Title" --hostpath "./cfg-data/" --templatename "ConfigTemplate" --templatedescription "Configuration Template Schema v1" --jsonschema --filepath "${REPO_NAME}/config/config.json"

          mkdir -p export
          iectl publisher sa version export --appname "${APP_NAME}" --versionnumber "${NEXT_VERSION}" --exportpath "export"
      - name: Upload application file
        uses: actions/upload-artifact@v4
        with:
          name: app-file
          path: export/

Deploy the application¶

With the application binary in hand it can now be uploaded to the IEM and from there installed to an Industrial Edge Device for testing.

Local EnvironmentGitLab CIGitHub Actions

iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PW}" --name "iemconfig"

iectl iem device-apps upload --app-file-path "export/${APP_ID}_${NEXT_VERSION}.app" --follow

DEVICE_ID=$(iectl iem device list | jq -r --arg name "${DEVICE_NAME}" '.data[] | select(.deviceName == $name) | .deviceId')
VERSION_ID=$(iectl iem device-apps app-details --app-name "${APP_NAME}" | jq -r --arg ver "${NEXT_VERSION}" '.versions[] | select(.version == $ver) | .versionId')

BATCH_ID=$(iectl iem job batch-create --appid "${APP_ID}" \
    --versionId "${VERSION_ID}" \
    --operation "installApplication" \
    --infoMap "$(jq -n --arg device "${DEVICE_ID}" '{devices: [$device]}')" | jq -r '.data')

# Wait for batch to be processed
for i in {1..30}; do STATUS=$(iectl iem job batch-status --batchId "${BATCH_ID}" | jq -r '.data'); echo "$STATUS"; [ "$STATUS" = "PROCESSED" ] && break || sleep 1; done

JOB_ID=$(iectl iem job list --id "${BATCH_ID}" | jq -r '.data[0].installedJobId')

iectl iem job device-job-wait --id "${JOB_ID}" --timeout 300

deploy:
  stage: deploy
  needs:
    - job: determine-version
      artifacts: true
    - job: create-app
      artifacts: true
  script:
    - iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PASSWORD}" --name "iemconfig"
    - iectl iem device-apps upload --app-file-path "export/${APP_ID}_${NEXT_VERSION}.app" --follow
    - DEVICE_ID=$(iectl iem device list | jq -r --arg name "${DEVICE_NAME}" '.data[] | select(.deviceName == $name) | .deviceId')
    - VERSION_ID=$(iectl iem device-apps app-details --app-name "${APP_NAME}" | jq -r --arg ver "${NEXT_VERSION}" '.versions[] | select(.version == $ver) | .versionId')
    - |
      BATCH_ID=$(iectl iem job batch-create --appid "${APP_ID}" \
          --versionId "${VERSION_ID}" \
          --operation "installApplication" \
          --infoMap "$(jq -n --arg device "${DEVICE_ID}" '{devices: [$device]}')" | jq -r '.data')
    - |
      # Wait for batch to be processed
      for i in {1..30}; do STATUS=$(iectl iem job batch-status --batchId "${BATCH_ID}" | jq -r '.data'); echo "$STATUS"; [ "$STATUS" = "PROCESSED" ] && break || sleep 1; done
    - JOB_ID=$(iectl iem job list --id "${BATCH_ID}" | jq -r '.data[0].installedJobId')
    - iectl iem job device-job-wait --id "${JOB_ID}" --timeout 300

jobs:
  deploy:
    runs-on: ubuntu-latest
    needs: [determine-version, create-app]
    steps:
      - name: Download application file
        uses: actions/download-artifact@v4
        with:
          name: app-file
          path: export/
      - name: Deploy application
        env:
          APP_ID: ${{ needs.determine-version.outputs.app_id }}
          NEXT_VERSION: ${{ needs.determine-version.outputs.next_version }}
        run: |
          iectl config add iem --url "${IEM_URL}" --user "${IEM_USER}" --password "${IEM_PASSWORD}" --name "iemconfig"

          iectl iem device-apps upload --app-file-path "export/${APP_ID}_${NEXT_VERSION}.app" --follow

          DEVICE_ID=$(iectl iem device list | jq -r --arg name "${DEVICE_NAME}" '.data[] | select(.deviceName == $name) | .deviceId')
          VERSION_ID=$(iectl iem device-apps app-details --app-name "${APP_NAME}" | jq -r --arg ver "${NEXT_VERSION}" '.versions[] | select(.version == $ver) | .versionId')

          BATCH_ID=$(iectl iem job batch-create --appid "${APP_ID}" \
              --versionId "${VERSION_ID}" \
              --operation "installApplication" \
              --infoMap "$(jq -n --arg device "${DEVICE_ID}" '{devices: [$device]}')" | jq -r '.data')

          # Wait for batch to be processed
          for i in {1..30}; do STATUS=$(iectl iem job batch-status --batchId "${BATCH_ID}" | jq -r '.data'); echo "$STATUS"; [ "$STATUS" = "PROCESSED" ] && break || sleep 1; done

          JOB_ID=$(iectl iem job list --id "${BATCH_ID}" | jq -r '.data[0].installedJobId')

          iectl iem job device-job-wait --id "${JOB_ID}" --timeout 300

NOTICE

In the same manner the application can also be published to the IE Hub to be distributed from there to multiple IEM instances in the tenant.