Modules
The SDP is built from software modules which produce a number of different types of artefacts. The components of the system are built as Open Container Initiative (OCI) images which are deployed on a Kubernetes cluster using a Helm chart. The OCI images depend on libraries containing common code. The diagram shows the SDP modules and the dependencies between them.
The source code is hosted in the SKA Science Data Processor group in GitLab, in the following repositories:
-
Integration of components into the SDP system. Contains the Helm chart to deploy the SDP, the system integration tests, and this documentation.
-
Library providing the interface to the configuration database.
-
Etcd is the highly available key-value database underpinning the configuration library.
-
Tango devices providing the local monitoring and control (LMC) of the SDP.
-
The Queue Connector Tango devices provide the interface for exchanging real-time data with other telescope subsystems.
-
Processing controller.
-
Helm deployer.
-
Command-line interface to control and monitor the SDP.
-
Library providing the high-level interface for writing processing scripts.
-
Processing scripts to be executed by the SDP in response to requests.
-
Charts used by the Helm deployer to deploy processing scripts and processing deployments.
-
Contains Jupyter notebooks that describe how one can control and monitor SDP via the console or using PyTango. Also contains notebooks to demonstrate SDP features. This module does not appear in the diagram, since it is a separate entity, not part of the SDP design. It uses BinderHub to connect to and interact with SDP deployed on a remote cluster.