Skip to content

Migrate existing instance to external database

By default, Grafana instances use an embedded SQLite database. While it should be sufficient for most instances, very active instances may consider an external database for better scalability.

If an existing instance becomes limited by the embedded SQLite database, the following steps can be used to migrate to an external database:

  1. Request a mysql instance from the Database on Demand service
  2. Provision a second grafana instance in the same project on https://app-catalogue.cern.ch, following the instance creation instructions a second time with:
  3. a different name, e.g. my-grafana-new
  4. a temporary hostname, e.g. my-grafana-new.web.cern.ch
  5. the external database connection details under Config (see Grafana database configuration reference)
  6. Use the following instructions to copy files from existing instance to the new instance. These steps assume a session on lxplus8.cern.ch and use Grafana's database-migrator scripts.
# specify mysql database connection details
MYSQL_DB=grafana
MYSQL_HOST=dbod-XXXXX.cern.ch
MYSQL_PORT=5500
MYSQL_USER=grafana
read -s -p 'mysql password: ' MYSQL_PWD

# authenticate with the app-catalogue Openshift cluster using CLI
oc sso-login app-catalogue
oc project my-grafana-project

# we should have 2 instances, identify source and destination pod
oc get pod
sourcepod=<old-intance-name>-grafana-app-12345678-12345
destpod=<new-instance-name>-grafana-app-1234abcd-123ab

# copy files over
# WARNING: any change done in the source instance after this point will not be replicated into the new instance
tmpcopydir=$(mktemp -d)
oc rsync ${sourcepod}:/var/lib/grafana/ ${tmpcopydir}/

# convert and import SQL as per https://github.com/grafana/database-migrator
tmpscriptdir=$(mktemp -d)
git clone https://github.com/grafana/database-migrator.git ${tmpscriptdir}
# sqlite3 not installed on lxplus8, use a container via podman
(cd ${tmpscriptdir}; alias sqlite3="podman run --rm -v ${tmpcopydir}:${tmpcopydir} -w /tmp -u 0 -i docker.io/keinos/sqlite3 sqlite3"; source ./sqlitedump.sh ${tmpcopydir}/grafana.db > ${tmpscriptdir}/grafana.sql)

# now we're ready to overwrite the destination instance database and files
# replace files
oc rsync ${tmpcopydir}/ ${destpod}:/var/lib/grafana/
# replace destination mysql database contents (overwrites any existing content)
mysql -u "${MYSQL_USER}" -p"${MYSQL_PWD}" -h "${MYSQL_HOST}" -P "${MYSQL_PORT}" -D "${MYSQL_DB}" < ${tmpscriptdir}/grafana.sql
# restart destination grafana instance
oc delete pod ${destpod}
  1. Verify that the new instance (my-grafana-new.web.cern.ch) is working properly, then we can delete the old instance and replace it with the new instance:
  2. Edit the old instance and delete it (Actions -> Delete Grafana). Alternatively, change its Hostname to a temporary name (e.g. my-grafana-old.web.cern.ch) and delete it after a few days.
  3. Edit the new instance and change its Hostname to the original instance's Hostname (e.g. my-grafana.web.cern.ch)