Skip to content

Backup and Restore

The procedure described below explains how to do a full backup of your ESS Pro installation, and how to restore it.

It is also possible to do incremental backups for the s3 storage and the PostgreSQL databases. Doing so would avoid the need to stop the services.

Full backups

You need to backup a couple of things to be able to restore your deployment:

  1. Follow the documentation to stop ESS Pro services
  2. The database. You need to backup your database and restore it on a new deployment.
  3. If you are using the provided Postgres database, build a dump using the command kubectl exec --namespace ess -it sts/ess-postgres -- pg_dumpall -U postgres > dump.sql. Adjust to your own Kubernetes namespace and release name if required.
  4. If you are using your own Postgres database, please build your backup according to your database documentation.
  5. Your values files used to deploy the chart
  6. The chart will generate some credentials in a Secret if you do not provide them. To copy them to a local file, you can run the following command: kubectl get secrets -l "app.kubernetes.io/managed-by=matrix-tools-init-secrets" -n ess -o yaml > secrets.yaml. Adjust to your own Kubernetes namespace if required.
  7. The chart will generate some flags/markers in a ConfigMap to ensure that helm upgrade with different values doesn't put the installation in an invalid state. To copy them to a local file, you can run the following command: kubectl get configmap -l "app.kubernetes.io/managed-by=matrix-tools-deployment-markers" -n ess -o yaml > configmaps.yaml. Adjust to your own Kubernetes namespace if required.
  8. The media files: if you are using S3 storage, follow your S3 provider backup recommendations. If you are storing media in persistent volume, Synapse stores media that should be backed up. On a default K3s setup, you can find where synapse media is stored on your node using the command kubectl get pv -n ess -o yaml | grep synapse-media.
  9. Run the helm upgrade --install.... command again to restore your workload's pods.

Restore procedure

  1. Recreate the namespace and the backed-up secret in step 3:
kubectl create ns ess
kubectl -n ess apply -f secrets.yaml
kubectl -n ess apply -f configmaps.yaml
  1. Redeploy the chart using the values backed-up in step 2.
  2. Stop Synapse and Matrix Authentication Service workloads:
kubectl scale sts -l "app.kubernetes.io/component=matrix-server" -n ess --replicas=0
kubectl scale deploy -l "app.kubernetes.io/component=matrix-authentication" -n ess --replicas=0
  1. Restore the PostgreSQL dump. If you are using the provided PostgreSQL database, this can be achieved using the following commands:
# Drop newly created databases and roles
kubectl exec -n ess sts/ess-postgres -- psql -U postgres -c 'DROP DATABASE matrixauthenticationservice'
kubectl exec -n ess sts/ess-postgres -- psql -U postgres -c 'DROP DATABASE synapse'
kubectl exec -n ess sts/ess-postgres -- psql -U postgres -c 'DROP ROLE synapse_user'
kubectl exec -n ess sts/ess-postgres -- psql -U postgres -c 'DROP ROLE matrixauthenticationservice_user'
kubectl cp dump.sql ess-postgres-0:/tmp -n ess
kubectl exec -n ess sts/ess-postgres -- bash -c "psql -U postgres -d postgres < /tmp/dump.sql"

Adjust to your own Kubernetes namespace and release name if required.

  1. If you are not using S3 for Synapse media, restore the synapse media files using kubectl cp to copy them in Synapse pod. If you are using K3s, you can find where the new persistent volume has been mounted with kubectl get pv -n ess -o yaml | grep synapse-media and copy your files in the destination path.
  2. Follow the documentation to start ESS Pro services