Performance testing

The performance test system consists of these components

OPC UA Server (OpCon.HAL.Devices.OPCUA.Tests.OPCUAServer)
- OPC UA Server simulating the southside of gateway
- Variables are read/written there and events for the tests are triggered from the OPC UA server
PerformanceTestExecutor (Bosch.Nexeed.Gateway.TestsExecutor)
- Webservice that can be controlled by web requests (POST, GET, …)
- Manages the test execution
- Starts the tests with a webrequest from the Performance-Unit-Test
- Synchronizes with the gateway instance with a webrequest from the Gateway
Gateway (Bosch.Nexeed.Gateway)
- Software under test
- Must be run with special configuration so that Mocks are used and synchronization with PerformanceTestExecutor is done
Performance-Unit Test (Bosch.Nexeed.Gateway.PerformanceTests)
- Triggers the test with request to PerformanceTestExecutor
- Aggregates results and Asserts in the end
- Detailed output written to: .\test\_PerformanceTests\Bosch.Nexeed.Gateway.PerformanceTests\TestResults

Configuration of the components

OPC UA Server:
- Should be enough to run the .exe
- Config is taken from Resources
  - NodeSet.xml
  - UaSdk.xml
PerformanceTestExecutor:
- In appsettings the path to the Gateway-Configuration must be specified
- It is required to read/write the same OPC UA variables as the gateway
Gateway:
- In appsettings.json, set these values:
  "Ddl": { "UseMock": true, "MockOutputPath": "C:\\OpCon\\Logs\\Gateway\\Resources\\DdlMockOutput", "MockResponsePath": "test\\_IntegrationTests\\Nexeed.Gateway.IntegrationTests\\Resources\\DdlResponse\\Active" }, "PerformanceTesting": { "UseSystem": true }
- If the gateway is started with the above configuration, performance logs are taken while certain dependencies are replaced by performance testing once (e.g., ITcpConnectionService)
- Gateway-Configuration same as PerformanceExecutor

How to run the tests locally

To start via pipeline script

Save yourself the trouble and restart everything for each testcase/time you are debugging. The test environment is very complex and could be "dirty" e.g., OPC UA variable values and internal structures of PerformanceAnalyzer etc.

Start OPCUA-Server exe
Bosch.Nexeed.Gateway.TestsExecutor from one Visual Studio
Bosch.Nexeed.Gateway from second Visual Studio
Start the Unit Test from third Visual Studio e.g.: "OneExecution_TwentyMachines_OneMessageSentToEachStation_DurationSmallerThan50Ms"

Performance Tests in Pipeline/Test26

The pipeline for the performance tests will execute the tests on server fe0bci-test26.de.bosch.com

There is a folder containing the configuration. Sometimes the appsettings in this folder have to be updated, especially when there were changes to the settings used in the gateway.

Overall-Folder: C:\performance-testing
Config-Folder: C:\performance-testing\configs
Detailed logs can be found in this path: C:\performance-testing\logs
Detailed-Results-Folder: C:\performance-testing\Gateway\test\_PerformanceTests\Bosch.Nexeed.Gateway.PerformanceTests\TestResults

The output of the pipeline is reported to the Teams channel: Performance Tests in CodeTransformers Group

The result of the unit tests indicate if the performance is sufficient (less than 50ms for a message to be processed by the gateway). If the performance is not sufficient, the test will fail and the pipeline will be marked as failed.