Modules

SDP software modules.

The SDP is built from software modules which produce a number of different types of artefacts. The components of the system are built as Open Container Initiative (OCI) images which are deployed on a Kubernetes cluster using a Helm chart. The OCI images depend on libraries containing common code. The diagram shows the SDP modules and the dependencies beween them.

The source code is hosted in the SKA Science Data Processor group in GitLab, in the following repositories:

• Integration

  Integration of components into the SDP system. Contains the Helm chart to deploy the SDP, the system integration tests, and this documentation.

• Configuration Library

  Library providing the interface to the configuration database.

• Processing Scripting Library

  Library providing the high-level interface for writing processing scripts.

• Local Monitoring and Control

  Tango devices providing the local monitoring and control (LMC) of the SDP.

• Processing Controller

  Processing controller.

• Helm Deployer

  Helm deployer.

• Console

  Command-line interface to control and monitor the SDP.

• Processing Scripts

  Processing scripts to be executed by the SDP in response to requests.

• Helm Deployer Charts

  Charts used by the Helm deployer to deploy processing scripts and execution engines.

• SDP Notebooks

  Contains Jupyter Notebooks that describe how one can control and monitor SDP via the console or using pytango. Also contains example notebooks of SDP features. This module does not appear in the diagram, since it is a separate entity, not part of the SDP design. It uses BinderHub to connect to a remote cluster and to interact with SDP.

• QA Metrics

  Contains a dashboard as well as an API to get stats based on the currently running receive data. More information can be found at the QA Metrics Documentation Pages. The base setup also configures ingresses for the QA system. They can be found at /<namespace>/qa/api/ping and /<namespace>/qa/display.