Running the SDP using make targets
The ska-sdp-integration repository contains a Makefile which provides targets to simplify deploying and testing the SDP. The following commands should be run from the root directory of the git repository.
Preparing Minikube to deploy the SDP
If you are deploying the SDP in a Minikube cluster, there are two steps needed to prepare the cluster:
- Enable the ingress add-on by running
minikube addons enable ingress
- Create the
nfss1
storage class. This is needed to for running the visibility receive processing script
make create-storageclass
These steps only need to be done once for each cluster.
Environment variables
The environment variables to control the deployment and testing of the SDP are defined at the top of the Makefile. They are described in the table below.
Environment variable | Default value | Description |
---|---|---|
KUBE_NAMESPACE |
test |
Kubernetes namespace for deploying the control system |
KUBE_NAMESPACE_SDP |
test-sdp |
Kubernetes namespace for deploying the processing scripts |
HELM_RELEASE |
test |
Name of the Helm chart release |
HELM_REPO |
charts |
Location of the Helm chart repository, may be a directory or name of a remote repository |
HELM_VALUES |
resources/values/test.yaml |
Values file(s) to use to install/upgrade the SDP deployment, may be a list of files separated by spaces |
TEST_INGRESS |
http://$(shell minikube ip) |
Ingress to use to execute tests. The default value assumes a Minikube cluster is being used |
TEST_TANGO_CLIENT |
gql |
Client to use to connect to Tango devices in the tests |
TEST_MARKER |
"not alternating_scans" |
Pytest markers to be used in the tests |
PYTEST_FLAGS |
"" |
Additional flags passed to pytest |
HELM_UNINSTALL_HAS_WAIT |
"1" |
Whether helm uninstall supports the --wait flag |
If you are using different values from the default ones, you will have to export the environment variables, or use them together with the commands. Alternatively, you can temporarily update the Makefile, but make sure you are working from a branch so that you don’t accidentally commit your changes.
For example:
$ export KUBE_NAMESPACE=default
$ export KUBE_NAMESPACE_SDP=sdp
$ make install-sdp
The above will install SDP into the default
namespace and use the sdp
namespace
for processing script deployments.
Creating namespaces
Create the namespaces specified in the environment variables with:
$ make create-namespaces
Updating chart dependencies
If you’re using the local copy of the chart to install SDP, you can make sure you have the latest chart dependencies available with:
$ make update-chart-dependencies
Deploying the SDP
$ make install-sdp
By default, the SDP will be installed using the local copy of the Helm chart in
the charts
directory.
If instead you want to use the published version of the SDP chart from the SKA
Central Artefact Repository, then you can do that using the HELM_REPO
variable. First, add the Helm chart repository:
$ helm repo add ska https://artefact.skao.int/repository/helm-internal
Then install SDP using this repository:
$ HELM_REPO=ska make install-sdp
The install-sdp
command upgrades the deployment if it exists,
or installs it if it doesn’t. It has a --wait
switch added to it,
which means the command will only return once the deployment has
finished installing/upgrading.
Running the integration tests
The Integration GitLab repository contains several tests, which make sure the SDP system behaves as expected. These can be found in the tests directory, together with BDD scenarios and resources to run the tests.
The tests are marked with pytest markers. The TEST_MARKER
environment variable
specifies which tests will run when using the make targets. For example, if you
only want to run the visibility receive test, you would need to use
TEST_MARKER="visibility_receive"
.
The tests interact with the Tango devices via Tango GraphQL. You need to install this chart first:
$ make install-test-tangogql
Note
If you are using Minikube with the docker driver, then you must enable tunnelling for the tests to work, which is done by running this command in a separate terminal:
$ minikube tunnel
It may ask you for an administrator password to open privileged ports. The command must remain running for the tunnel to be active. You must also set the ingress URL to point to localhost:
$ export TEST_INGRESS=http://127.0.0.1
Once the Tango GraphQL pod is started, you can run the tests with
$ make test
Note: by default all tests are run, including the visibility receive one.
The visibility receive test requires a couple of helper pods, which connect to persistent volumes. These contain MeasurementSet data (stored using Git LFS in the repository, see Troubleshooting), which are used for sending and validating the received data. These pods are automatically created and removed by the test, however, for manual testing purposes, you may want to create them yourself:
$ make create-test-volumes
Finally, run the visibility receive test:
$ TEST_MARKER="visibility_receive" make test
Cleaning up and removing SDP
Once you finished work with SDP, you can use the following commands to clean up your deployment.
Remove resources used for tests:
$ make delete-test-volumes
$ make uninstall-test-tangogql
Uninstall SDP:
$ make uninstall-sdp
Delete namespaces (optional)
$ make delete-namespaces