Find and download the output data

This documentation outlines how to retrieve the output data from a successful scan.

Note

All data products are identified by a unique execution block ID.

How you retrieve the data product depends on where and how you installed your instance of the SDP. You can use the Data Product Dashboard if you deployed on an environment such the ITF. If you deployed locally or in an environment where the Data Product Dashboard is not available, you should have created a local PVC in which the data is stored.

Data Product Dashboard

Note: The Data Product Dashboard is only available if you installed the SDP in a SKA enviroment which has been connected to the shared dashboard.

1. You can access the data product dashboard at Data Product Dashboard. This is on the STFC SDHP cluster.

2. How to use Data Product Dashboard

   Once connected to the dashboard, a list of data products will appear. Click on the relevant data product and a side bar should appear with information about the data product. There is a download button if you wish to download the data product. For more information click here.

Kubernetes Commands (if deployed Locally or no Data Product Dashboard available)

1. Connect to Kubernetes container

   In your second namespace where the script pods are run, you need to list all the containers in the namespace. If you are using k9s, exec into the pod with a name similar to xxxxx-vis-receive-xxxx. If you want to use Kubernetes, use the get pods command to get a list of containers.

   kubectl get pods -n <namespace-proc> -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{"\t"}{.metadata.namespace}{"\t"}{range .spec.containers[*]}{.name}{"=>"}{.image}{","}{end}{end}'|sort|column -t
   

   You need to look for a container called plasma-store with a pod name similar to xxxxx-vis-receive-xxxx.

   Now you need to exec into this container. If you are in k9s, select the container and press the s key.:

   kubectl exec -i -t <pod_name> --container <container_name> -- /bin/bash
   

2. Commands needed to download the data product

   Once in the container you need to go to the following folder and list the data products in the folder:

   cd /mnt/data/
   ls
   

   You are looking for a file with the same execution block id that you used to define the AssignResources config.

   Now you can exit the container, and run the following command to copy the file to your local

   kubectl cp <namespace-proc>/<pod_name>:/mnt/data/<filename> <local_destination>