In some situations, you might not be able to upgrade NVIDIA CUDA drivers on a regular basis. To work around this issue, NVIDIA provides compatibility drivers that allow users to use newer features without requiring a full upgrade. For information about compatibility drivers, see https://docs.nvidia.com/deploy/cuda-compatibility/index.html.

Installing the Drivers

Use the following commands to install the CUDA 11 compatibility drivers on Ubuntu:

wget 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin

mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 

apt-key adv --fetch-keys 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub

add-apt-repository "deb 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" 

apt update 

nvidia-smi 

apt install cuda-compat-11-0 

nvidia-smi 

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.0/compat/ 

nvidia-smi

After the last nvidia-smi, ensure that CUDA shows the correct version.

Updating systemd Files

After installing the drivers, update the systemd files in /lib/systemd/system/heavydb.service.

In the service section, add or update the environment property

Environment=LD_LIBRARY_PATH=/usr/local/cuda-11.0/compat:$LD_LIBRARY_PATHbash

The file should look like that

[Unit] 
Description=HEAVY.AI database server 
After=network.target remote-fs.target

[Service] 
Environment=LD_LIBRARY_PATH=/usr/local/cuda-11.0/compat:$LD_LIBRARY_PATH
User=heavyai 
Group=heavyai 
WorkingDirectory=/opt/heavyai
ExecStart=/opt/heavyai/bin/heavydb --config /var/lib/heavyai/heavy.conf 
KillMode=control-group 
SuccessExitStatus=143 
LimitNOFILE=65536 
Restart=always

[Install] 
WantedBy=multi-user.target

Then force the reload of the systemd configuration

// Some code

In this section, you will find recipes to upgrade from the OmniSci to the HEAVY.AI platform and upgrade between versions of the HEAVY.AI platform.

Upgrade Paths supported

It's not always possible to upgrade from your actual product version to the last one, but one or more intermediate upgrade steps are needed.

As an example, if you are using an OmniSci version older than 5.5, you must upgrade to 5.5, then to 6.0 and after that to 7.0, while if you are using 6.0-6.4.4 you can upgrade to 7.0.0 in a single step.

This section is giving a recipe to upgrade from Omnisci platform 5.5+ to Heavi.ai 6.0.

Considerations when Upgrading from Omnisci to HEAVY.AI Platform

If you are upgrading from Omnisci to HEAVY.AI, there are a lot of additional steps compared to a simple sub-version upgrade.

Before Upgrading to Release 6.0

The omnisci the database will not be automatically renamed to the new default name heavyai.This will be done manually and it's documented in the upgrade steps.

Essential Changes for release 6.0 of HEAVY.AI compared to Omnisci

The following table describes the changes to environment variables, storage locations, and filenames in Release 6.0 compared to Release 5.x. Except where noted, revised storage subfolders, symlinks for old folder names, and filenames are created automatically on server start.

Change descriptions in bold require user intervention.

Upgrade Instructions

Assumptions

This upgrade procedure is assuming that you are using the default storage location for both Omnisci and HEAVY.AI.

Upgrading Using Docker

Stop all containers running Omnisci services.

In a terminal window, get the Docker container IDs:

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep omnisci\/

You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

Stop the HEAVY.AI Docker container. For example:

sudo docker container stop 9e01e520c3

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zxvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme.

sudo mv /var/lib/omnisci /var/lib/heavyai
sudo mv /var/lib/heavyai/data /var/lib/heavyai/storage

Create a new configuration file for heavydb changing the data parameter to point to the renamed data directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" 
>/var/lib/heavyai/heavy.conf

Rename the Omnisci license file (EE and FREE only).

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Download and run the 6.0 version of the HEAVY.AI Docker image.

Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:v6.0.0

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:v6.0.0

Check that Docker is up and running using a docker ps command:

sudo docker container ps --format "{{.Id}} {{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see output similar to the following:

9e01e520c30c heavyai/heavyai-ee-cuda Up 48 seconds ago 

Using the new container ID rename the default omnisci database to heavyai:

sudo docker exec -it 9e01e520c30c \
echo "alter database omnisci rename to heavyai;" \
| bin/heavysql omnisci 

Check that everything is running as expected.

Upgrading to HEAVY.AI Using Package Managers or Tarball

To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data.

Back up the Omnisci Database

Stop the Omnisci services.

sudo systemctl stop omnisci_web_server omnisci_server

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zcvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Create a user named heavyai who will be the owner of the HEAVY.AI software and data on the filesystem.

sudo useradd --shell /bin/bash --user-group --create-home --group wheel heavyai

sudo useradd --shell /bin/bash --user-group --create-home --group sudo heavyai

Set a password for the user. It'll need when sudo-ing.

sudo passwd heavyai

Login with the newly created user

sudo su - heavyai

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme and change the ownership to heavyai user.

sudo chown -R heavyai:heavyai /var/lib/omnisci

sudo mv /var/lib/omnisci /var/lib/heavyai

mv /var/lib/heavyai/data /var/lib/heavyai/storage
mkdir /var/lib/heavyai/storage/catalogs

Check that everything is in order and the semaphore directory is created

ls -la /var/lib/heavyai/storage/

All the directories must belong to heavyai user and the directory catalogs would be present

total 32
drwxr-xr-x  8 heavyai heavyai 4096 lug 15 16:03 .
drwxr-xr-x  4 heavyai heavyai 4096 lug 15 16:02 ..
drwxrwxr-x  2 heavyai heavyai 4096 lug 15 16:03 catalogs
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_catalogs
drwxr-xr-x 52 heavyai heavyai 4096 lug 15 15:54 mapd_data
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_export
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_log
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 omnisci_disk_cache
-rw-r--r--  1 heavyai heavyai 1229 lug 15 16:07 omnisci-licence

Rename the license file. (EE and FREE only)

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Install the HEAVY.AI Software

Install the HEAVY.AI software following all the instructions for your Operative System. CentOS/RHEL and Ubuntu.

Update the configuration file and rename the default database

Log in with the heavyai user and be sure that the heavyai services are stopped.

sudo systemctl stop heavy_web_server heavydb

Create a new configuration file for heavydb changing the data parameter to point to the /var/lib/heavyai/storage directory and the frontend to the new install directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" \
>/var/lib/heavyai/heavy.conf

All the settings of the upgraded database will be moved to the new configuration file.

Now we have to complete the database migration.

Remove the semaphore directory.

rmdir /var/lib/heavyai/storage/catalogs

To complete the upgrade, start the HEAVY.AI servers.

sudo systemctl start heavydb heavy_web_server

Check if the database migrated, running this command and checking for the Rebrand migration complete message.

sudo systemctl status heavydb

Rename the default omnisci database to heavyai. Run the command using an administrative user (typically admin) with his password (default HyperInteractive)

echo "alter database omnisci rename to heavyai;" \
| /opt/heavyai/bin/heavysql -p HyperInteractive -u admin omnisci 

Restart the database service and check that everything is running as expected.

Remove Omnisci Software from the System

After all the checks confirmed that the upgraded system is stable, clean up the system to remove the Omnisci install and relative system configuration. Remove permanently the configuration of the services.

sudo rm /lib/systemd/omnisci_server*.service
sudo rm /lib/systemd/omnisci_web_server*.service
sudo systemctl daemon-reload
sudo systemctl reset-failed

Remove the installed software.

sudo rm -Rf /opt/omnisci

Delete the YUM or APT repositories.

sudo rm /etc/yum.repos.d/omnisci.repo

sudo rm /etc/apt/sources.list.d/omnisci.list

This section is giving a recipe to upgrade between fully compatible products version.

Upgrading from Omnisci

If you need to upgrade from Omnisci to HEAVY.AI 6.0 or later, please refer to the specific recipe.

Upgrading Using Docker

To upgrade HEAVY.AI in place in Docker

In a terminal window, get the Docker container ID.

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep omnisci\/

You should see output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

Stop the HEAVY.AI Docker container. For example:

docker container stop 9e01e520c30c

Optionally, remove the HEAVY.AI Docker container. This removes unused Docker containers on your system and saves disk space.

docker container rm 9e01e520c30c

Backup the Omnisci data directory (typically /var/lib/omnisci)

tar zcvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Download the latest version of the HEAVY.AI Docker image according to the Edition and device you are actually coming from Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.

sudo docker run -d --gpus=all \
  -v /var/lib/heavyai:/var/lib/heavyai \
  -p 6273-6278:6273-6278 \
  heavyai/heavyai-ee-cuda:latest

sudo docker run -d -v \
/var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:latest

sudo docker run -d --gpu=all \
  -v /var/lib/heavyai:/var/lib/heavyai \
  -p 6273-6278:6273-6278 \
  heavyai/core-os-cuda:latest

sudo docker run -d -v \
/var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:latest

Check that the docker is up and running a docker ps commnd:

sudo docker container ps --format "{{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see an output similar to the following.

heavyai/heavyai-ee-cuda Up 48 seconds ago 

This runs both the HEAVY.AI database and Immerse in the same container.

See also the note regarding the CUDA JIT Cache in Optimizing Performance.

Upgrading HEAVY.AI Using Package Managers and Tarball

To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data

Stop the HEAVY.AI services.

sudo systemctl stop heavydb heavy_web_server

Back up your $HEAVYAI_STORAGE directory (the default location is /var/lib/heavyai).

Run the appropriate set of commands depending on the method used to install the previous version of the software.

sudo yum update heavyai.x86_64

sudo apt update
sudo apt upgrade heavyai

sudo mv /opt/heavyai /opt/heavyai_backup

When the upgrade is complete, start the HEAVY.AI services.

sudo systemctl start heavydb heavy_web_server