SOT
    • Introduction
    • User manual
      • General Descriptions
        • Fine-grained access control
        • Ticket Management Groups
      • Ticket Management
        • Filter options
        • Ticket details
        • Ticket archiving
        • Ticket anonymization
        • Notifications
        • Ticket escalation mapping
      • Maintenance status widget
    • Operations manual
      • Overview
      • System Architecture and Interfaces
      • System Requirements
        • tm/ticket-service
      • Migration from Previous Versions
        • Migration to 7.3.0+
        • Migration to 7.4.0+
        • Migration to 7.4.1+
        • Migration to 7.8.0+
        • Migration to 7.9.0+
        • Migration to 7.10.0+
        • Migration to 7.11.0+
        • Migration to 7.12.0+
        • Migration to 7.13.0+
        • Migration to 7.13.1+
        • Migration to 7.14.0+
        • Migration to 7.16.0+
        • Migration to 7.17.0+
        • Migration to 7.20.0+
      • Setup and Configuration
        • Setup
          • Installation guide
        • Recommendations
        • Helm Configuration
        • Ticket Management global variables
        • tm/ticket-service
        • Ticket Management IAS dependencies
      • Start and Shutdown
      • Regular Operations
      • Failure Handling
      • Backup and Restore
      • Logging and Monitoring
        • Module Health Verification Endpoints and K8S Probes
      • Known Limitations
        • Maximum 2100 parameter support of MSSQL can lead to query failure
        • Users from different tenants can have issues when calling MACMA/MDM concurrently
        • Lost messages of Notification service if maximum retry is exceeded
    • Developer documentation
      • Getting Started
      • How-Tos
        • Domain Context Registration
        • Integrating Ticket Management Views
      • Failure Handling
        • Embedded View Blocked by Content Security Policy (CSP)
    • API documentation
      • HTTP API
    • Glossary
Ticket Management
  • Smart Operations Toolkit
    • Deviation Processor
    • Multitenant Access Control
    • Notification Service
    • Ticket Management
    • Web Portal
  • Shopfloor Management
    • Andon Live
    • KPI Reporting
    • Operational Routines
    • Shift Book
    • Shopfloor Management Administration
  • Product & Quality
    • Process Quality
    • AI Services
  • Machine & Equipment
    • Condition Monitoring
    • Device Portal
  • Enterprise & Shopfloor Integration
    • Information Router
    • Master Data Management

SOT Learning Portal
  • Ticket Management
  • Operations manual
  • Logging and Monitoring
  • Module Health Verification Endpoints and K8S Probes
7.22.2 7.22.0

Module Health Verification Endpoints and K8S Probes

  • Health endpoint: Will be used by monitoring to determine the health state of a microservice. The health endpoint will also include the states of the liveness and readiness states. It is not intended to be used by the container management system, but for privileged (authorization required) users like operators or tools of an operator. For that reason the endpoints are exposed and will provide additional information than a simple ok, nok status.

  • Readiness endpoint: Will be polled by Kubernetes to check, if a Microservice (pod) can accept traffic.

  • Liveness endpoint: Will be polled by Kubernetes to check, if a Microservice internal state is valid. Kubernetes will restart the Container if the liveness endpoint fails for the configured number of times.

Ticket Service

  • Health Endpoint

    • /BCI_TS/v1/actuator/health

    • The endpoint deviates from the BCI requirements of the health endpoint by not authorizing on resource with id health and type urn:com:bosch:bci:operation with privilege EXECUTE; any authenticated user or client can request this endpoint

    • unauthorized: 401

      • No authorization (token) was provided with the request

    • down: 503

      • Status 503 is returned when the Health State is down. Reason for being down and details can be found in the response body.

      • We think that the microservice is not healthy in case of lost connection to Database, RabbitMQ, MACMA, MDM, Portal, Deviation Processor and NOTIFICATION.

    • up: 200

      • details are provided for any authenticated user

  • Liveness Endpoint

    • /BCI_TS/v1/actuator/health/liveness

    • down: 503

      • only during startup and shutdown it will be down

    • up: 200

  • Readiness Endpoint

    • /BCI_TS/v1/actuator/health/readiness

    • down: 503

      • If the Database or MACMA is down we will not accept requests to the microservice

    • up: 200

  • Dependencies

    • RabbitMQ

      • Use cases:
        Lost connection to RabbitMQ.
        Reasons:

        • RabbitMQ instance crashed, restarted, etc.

        • Network issue between microservice and RabbitMQ

      • Impact

        • The lost connection to RabbitMQ will be logged as warning.

        • The microservice is trying to reconnect infinitely.

        • We keep the Service alive to keep the UI running.

        • Durable RabbitMQ objects (e.g. queues and bindings) are not impacted.

        • Non-Durables queues are receiving request again when RabbitMQ is up.

        • When accessing /health endpoint, we receive 504 in case of RabbitMQ is down, because we are trying to reconnect infinitely. And after timeout is exceeded, we got 504 back.

        • After restarting RabbitMQ, the health endpoint responds with an UP status for RabbitMQ.

    • Database

      • Use cases:
        Lost connection to Database.
        Reasons:

        • Database instance crashed, restarted, etc.

        • Network issue between microservice and Database

      • Impact

        • The lost connection to Database will be logged as warning.

        • The microservice is trying to reconnect infinitely.

        • The service is kept alive but it won’t be ready to accept any requests.

        • When accessing /health endpoint, we receive 504 in case of DB is down, because we are trying to reconnect infinitely. And after timeout is exceeded, we got 504 back.

        • After restarting the db, our service connects automatically and successfully to DB.

    • MACMA

      • Use cases:
        Lost connection to MACMA.
        Reasons:

        • MACMA instance crashed, restarted, etc.

        • Network issue between microservice and MACMA

      • Impact

        • When accessing /health endpoint, we receive 503 in case of the whole MACMA is down.

        • The lost connection to MACMA will be logged as error

        • The microservice is trying to reconnect 50 times by default (configurable via MACMA_RETRYER_MAXATTEMTPS)

        • The service is kept alive but it can only accept those request whose token is still valid and where MACMA is not involved. The other request (e.g. new token is required or token is expired) cannot be accepted.

        • The application’s UI cannot be accessed.

        • After restarting MACMA, everything works fine again (get token, send requests) without restarting ticket service

    • Portal

      • Use cases:
        Lost connection to Portal.
        Reasons:

        • Portal instance crashed, restarted, etc.

        • Network issue between microservice and Portal

      • Impact

        • The lost connection/failed requests to Portal will be logged as error

        • The microservice is trying to reconnect 50 times by default (configurable via PORTAL_RETRYER_MAXATTEMTPS)

        • The application’s UI cannot be accessed

        • API calls via Postman are still working

        • After restarting portal, everything works fine again without restarting ticket service

    • MDM

      • Use cases:
        Lost connection to MDM.
        Reasons:

        • MDM instance crashed, restarted, etc.

        • Network issue between microservice and MDM

      • Impact

        • The microservice is trying to reconnect 50 times by default (configurable via MMPD_RETRYER_MAXATTEMTPS)

        • The lost connection/failed requests to MDM will be logged as error

        • The API is still functional with the existing Master Data in TS Database.

        • Masterdata changes (Integration Events) after MDM stopped is not reflected to TS, MDM needs to retry publishing (Stop TS. Create new device in MDM, link it to a facility, Stop MMPD, Start TS). After triggering masterdata reload via API, masterdata appears.

        • After restarting MDM, everything works fine again without restarting ticket service

    • Deviation Processor

      • Use cases:
        Lost connection to Deviation Processor.
        Reasons:

        • Deviation Processor instance crashed, restarted, etc.

        • Network issue between microservice and Deviation Processor

      • Impact

        • The lost connection/failed requests to Deviation Processor will be logged as error

        • The microservice is trying to reconnect 10 times by default (configurable via SMDP_RETRYER_MAXATTEMTPS)

        • If the retry attempt is exceeded, the registration as reactor module at Deviation Processor won’t be performed anymore, until the ticket-service is restarted

        • After restarting Deviation Processor, everything works fine again without restarting ticket service

    • NOTIFICATION

      • Use cases:
        Lost connection to NOTIFICATION.
        Reasons:

        • NOTIFICATION instance crashed, restarted, etc.

        • Network issue between microservice and NOTIFICATION

      • Impact

        • The lost connection/failed requests to NOTIFICATION will be logged as error

        • The microservice is trying to reconnect 50 times by default (configurable via NOTIFICATION_RETRYER_MAXATTEMTPS)

        • If the retry is exceeded, this ticket status change information won’t be published to Notification Service and get lost for Notification Service

        • After restarting NOTIFICATION, everything works fine again without restarting ticket service

Contents

© Robert Bosch Manufacturing Solutions GmbH 2023-2026, all rights reserved

Changelog Corporate information Legal notice Data protection notice Third party licenses