Performance & Smoke Testing for Nexeed Gateway (OpConXml)

This document describes how the automated smoke & performance tests for the Nexeed Gateway (OpConXml protocol) are structured, executed, configured, evaluated, and how to extend or run them locally.

The tests validate both functional connectivity (round‑trip correctness incl. return codes) and latency (round‑trip time thresholds) between the Gateway and an OpConXml endpoint under near‑production conditions.

Overview

The performance test job in Azure DevOps runs as part of the Release Pipeline task: OpConXml Smoke and Performance Testing si0vmc4560.

It spins up multiple Docker containers:

SmokeTestHelperService (project: Bosch.Nexeed.Gateway.SmokeTestService) – hosts the OpConXmlClientController used to drive client traffic and measure timings.
TestRunner – executes a Postman/Newman collection (NX_IAS_Gateway_OpConXml_SmokeTest.postman_collection.json) that configures the Gateway (DDL + general config) and triggers test requests.

High-Level Architecture

Flow summary:

The TestRunner posts configuration to the Gateway under test (DDL + generalGatewayConfig) via Postman requests.
The SmokeTestHelperService (OpConXmlClientController) acts as an OpConXml client, sending XML requests to the Gateway and measuring round‑trip time.
The Gateway forwards the operation to either:
- A mock HTTP server (default on test host, e.g. D:\Temp\HttpServerSmokeTest).
- The real backend system (when environment is configured with real endpoints) for true performance characterization.
The controller aggregates latency metrics and validates response return codes

Controller Logic (Latency Measurement)

Source: OpConXmlClientController (excerpt from SmokeTestServiceController context).

Key behaviors: * Calculates per‑message round‑trip delay: received.Timestamp - sent.Timestamp (ms, rounded to 2 decimals). * Sorts delays descending; computes average. * Threshold enforcement: - Individual round‑trip must be ≤ 60 ms. - Average round‑trip must be ≤ 60 ms. * Error conditions returned as HTTP 500 with diagnostic payload: - Connection setup failure. - Timeout waiting for responses. - Mismatch send/received counts. - Response return code not equal to 0 (<result returnCode="0"> expected). - Performance threshold breach (individual or average).

Postman / Newman TestRunner

Collection: NX_IAS_Gateway_OpConXml_SmokeTest.postman_collection.json.

Roles:

Prepares Gateway configuration (DDL + generalGatewayConfig).
Triggers performance measurement endpoints exposed by SmokeTestHelperService.

Azure DevOps Pipelines

Build pipeline: Nexeed Gateway Deploy (Version Information) – produces artifacts.

Release pipeline stages relevant for performance:

Build & Push E2E Docker image.
OpConXml Smoke and Performance Testing si0vmc4560 – runs containers & executes Newman collection.

Response Examples

Successful performance summary:

{
	"averageRoundtripDuration": 42.18,
	"individualRoundtripDurations": [55.12, 44.90, 40.11, 39.88, 33.89]
}

Extending Tests

To add scenarios:

Extend the Postman collection with new request sequences (e.g. provisioning different DDL sets).
Provide variant XmlMessage payloads capturing new business operations.
If parallel measurements are required, enhance controller logic to support correlation/ordering (currently disabled for ParallelMessages > 1 && MessageCount > 1).

Limitations

only run one instance of the Gateway. The configuration is with a POST request to one Pod. If multiple pods are scaled, only one will receive the configuration
Sometimes the first request of the first time deployment is slow. This should be investigated. Probably some things have to be initialized or loaded on the first try.
NGC configuration not used in the test. We are POSTING configurations to the configuration endpoint

Further reads:

Please be careful. Some of the docupedia articles might be outdated.