Upgrading from Omnisci to HEAVY.AI 6.0

This section is giving a recipe to upgrade from Omnisci platform 5.5+ to HEAVY.AI 6.0.

If the version of Omnisci is older than 5.5 an intermediate upgrade step to the 5.5 version is needed. Check the docs on how to do the upgrade.

Considerations when Upgrading from Omnisci to HEAVY.AI Platform

If you are upgrading from Omnisci to HEAVY.AI, there are a lot of additional steps compared to a simple sub-version upgrade.

Before Upgrading to Release 6.0

IMPORTANT - Before you begin, stop all the running services / docker images of your Omnisci installation and create a backup $OMNISCI_STORAGE folder (typically /var/lib/omnisci). A backup is essential for recoverability; do not proceed with the upgrade without confirming that a full and consistent backup is available and ready to be restored.

The omnisci the database will not be automatically renamed to the new default name heavyai.This will be done manually and it's documented in the upgrade steps.

All the dumps created with the dump command on Omnisci cannot be restored after the database is upgraded to this version.

Essential Changes for release 6.0 of HEAVY.AI compared to Omnisci

The following table describes the changes to environment variables, storage locations, and filenames in Release 6.0 compared to Release 5.x. Except where noted, revised storage subfolders, symlinks for old folder names, and filenames are created automatically on server start.

Change descriptions in bold require user intervention.

Description	Omnisci 5.x	HEAVY.AI 6.0
Environmental variable for storage location	$OMNISCI_STORAGE	$HEAVYAI_BASE
Default location for $HEAVYAI_BASE / $OMNISCI_STORAGE	/var/lib/omnisci	/var/lib/heavyai
Fixed location for Docker $HEAVYAI_BASE / $OMNISCI_STORAGE	/omnisci-storage	/var/lib/heavyai
The folder containing catalogs for $HEAVYAI_BASE / $OMNISCI_STORAGE	data/	storage/
Storage subfolder - data	data/mapd_data	storage/data
Storage subfolder - catalog	data/mapd_catalogs	storage/catalogs
Storage subfolder - import	data/mapd_import	storage/import
Storage subfolder - export	data/mapd_export	storage/export
Storage subfolder - logs	data/mapd_log	storage/log
Server INFO logs	omnisci_server.INFO	heavydb.INFO
Server ERROR logs	omnisci_server.ERROR	heavydb.ERROR
Server WARNING logs	omnisci_server.WARNING	heavydb.WARNING
Web Server ACCESS logs	omnisci_web_server.ACCESS	heavy_web_server.ACCESS
Web Server ALL logs	omnisci_web_server.ALL	heavy_web_server.ALL
Install directory	/omnisci (Docker) /opt/omnisci (bare metal)	/opt/heavyai/ (Docker and bare metal)
Binary file - core server (located in install directory)	bin/omnsici_server	bin/heavydb
Binary file - web server (located in install directory)	bin/omnisci_web_server	bin/heavy_web_server
Binary file - command- line SQL utility	bin/omnisql	bin/heavysql
Binary file - JDBC jar	bin/omnisci-jdbc-5.10.2-SNAPSHOT.jar	bin/heavydb-jdbc-6.0.0-SNAPSHOT.jar
Binary file - Utilities (SqlImporter) jar	bin/omnisci-utility-5.10.2-SNAPSHOT.jar	bin/heavydb-utility-6.0.0-SNAPSHOT.jar
HEAVY.AI Server service (for bare metal install)	omnisci_server	heavydb
HEAVY.AI Web Server service (for bare metal install)	omnisci_web_server	heavy_web_server
Default configuration file	omnisci.conf	heavy.conf

Description

Omnisci 5.x

HEAVY.AI 6.0

Environmental variable for storage location

$OMNISCI_STORAGE

$HEAVYAI_BASE

Default location for $HEAVYAI_BASE / $OMNISCI_STORAGE

/var/lib/omnisci

/var/lib/heavyai

Fixed location for Docker $HEAVYAI_BASE / $OMNISCI_STORAGE

/omnisci-storage

/var/lib/heavyai

The folder containing catalogs for $HEAVYAI_BASE / $OMNISCI_STORAGE

data/

storage/

Storage subfolder - data

data/mapd_data

storage/data

Storage subfolder - catalog

data/mapd_catalogs

storage/catalogs

Storage subfolder - import

data/mapd_import

storage/import

Storage subfolder - export

data/mapd_export

storage/export

Storage subfolder - logs

data/mapd_log

storage/log

Server INFO logs

omnisci_server.INFO

heavydb.INFO

Server ERROR logs

omnisci_server.ERROR

heavydb.ERROR

Server WARNING logs

omnisci_server.WARNING

heavydb.WARNING

Web Server ACCESS logs

omnisci_web_server.ACCESS

heavy_web_server.ACCESS

Web Server ALL logs

omnisci_web_server.ALL

heavy_web_server.ALL

Install directory

/omnisci (Docker) /opt/omnisci (bare metal)

/opt/heavyai/ (Docker and bare metal)

Binary file - core server (located in install directory)

bin/omnsici_server

bin/heavydb

Binary file - web server (located in install directory)

bin/omnisci_web_server

bin/heavy_web_server

Binary file - command- line SQL utility

bin/omnisql

bin/heavysql

Binary file - JDBC jar

bin/omnisci-jdbc-5.10.2-SNAPSHOT.jar

bin/heavydb-jdbc-6.0.0-SNAPSHOT.jar

Binary file - Utilities (SqlImporter) jar

bin/omnisci-utility-5.10.2-SNAPSHOT.jar

bin/heavydb-utility-6.0.0-SNAPSHOT.jar

HEAVY.AI Server service (for bare metal install)

omnisci_server

heavydb

HEAVY.AI Web Server service (for bare metal install)

omnisci_web_server

heavy_web_server

Default configuration file

omnisci.conf

heavy.conf

Upgrade Instructions

The order of these instructions is significant. To avoid problems, follow the order of the instruction provided and don't skip any step.

Assumptions

This upgrade procedure is assuming that you are using the default storage location for both Omnisci and HEAVY.AI.

$OMNISCI_STORAGE

$HEAVYAI_BASE

/var/lib/omnisci

/var/lib/heavyai

Upgrading Using Docker

Stop all containers running Omnisci services.

In a terminal window, get the Docker container IDs:

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep omnisci\/

You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

Stop the HEAVY.AI Docker container. For example:

sudo docker container stop 9e01e520c3

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zxvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme.

sudo mv /var/lib/omnisci /var/lib/heavyai
sudo mv /var/lib/heavyai/data /var/lib/heavyai/storage

Create a new configuration file for heavydb changing the data parameter to point to the renamed data directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" 
>/var/lib/heavyai/heavy.conf

Rename the Omnisci license file (EE and FREE only).

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Download and run the 6.0 version of the HEAVY.AI Docker image.

Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:v6.0.0

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:v6.0.0

Check that Docker is up and running using a docker ps command:

sudo docker container ps --format "{{.Id}} {{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see output similar to the following:

9e01e520c30c heavyai/heavyai-ee-cuda Up 48 seconds ago

Using the new container ID rename the default omnisci database to heavyai:

sudo docker exec -it 9e01e520c30c \
echo "alter database omnisci rename to heavyai;" \
| bin/heavysql omnisci

Check that everything is running as expected.

Upgrading to HEAVY.AI Using Package Managers or Tarball

To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data.

Back up the Omnisci Database

Stop the Omnisci services.

sudo systemctl stop omnisci_web_server omnisci_server

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zcvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Create a user named heavyai who will be the owner of the HEAVY.AI software and data on the filesystem.

sudo useradd --shell /bin/bash --user-group --create-home --group wheel heavyai

sudo useradd --shell /bin/bash --user-group --create-home --group sudo heavyai

Set a password for the user. It'll need when sudo-ing.

sudo passwd heavyai

sudo su - heavyai

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme and change the ownership to heavyai user.

sudo chown -R heavyai:heavyai /var/lib/omnisci

sudo mv /var/lib/omnisci /var/lib/heavyai

mv /var/lib/heavyai/data /var/lib/heavyai/storage

Create the "semaphore" catalog directory; we'll have to remove it later "

mkdir /var/lib/heavyai/storage/catalogs

Check that everything is in order and that the "semaphore" directory is created,

ls -la /var/lib/heavyai/storage/

All the directories must belong to the heavyai user, and the directory catalogs would be present

total 32
drwxr-xr-x  8 heavyai heavyai 4096 lug 15 16:03 .
drwxr-xr-x  4 heavyai heavyai 4096 lug 15 16:02 ..
drwxrwxr-x  2 heavyai heavyai 4096 lug 15 16:03 catalogs
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_catalogs
drwxr-xr-x 52 heavyai heavyai 4096 lug 15 15:54 mapd_data
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_export
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_log
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 omnisci_disk_cache
-rw-r--r--  1 heavyai heavyai 1229 lug 15 16:07 omnisci-licence

Rename the license file. (EE and FREE only)

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Install the HEAVY.AI Software

Install the HEAVY.AI software following all the instructions for your Operative System. CentOS/RHEL and Ubuntu.

Please follow all the installation and configuration steps until the Initialization step.

Update the configuration file and rename the default database

sudo systemctl stop heavy_web_server heavydb

Create a new configuration file for heavydb, changing the data parameter to point to the /var/lib/heavyai/storage directory and the frontend to the new install directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" \
>/var/lib/heavyai/heavy.conf

All the settings of the upgraded database will be moved to the new configuration file.

Now we have to complete the database migration.

Remove the "semaphore" directory we previously created. (this is a fundamental step needed for the omnsci to heavydb upgrade)

rmdir /var/lib/heavyai/storage/catalogs

To complete the upgrade, start the HEAVY.AI servers.

sudo systemctl start heavydb heavy_web_server

Check if the database migrated, running this command and checking for the Rebrand migration complete message.

sudo systemctl status heavydb

Rename the default omnisci database to heavyai.Run the command using an administrative user (typically admin) with his password (default HyperInteractive)

echo "alter database omnisci rename to heavyai;" \
| /opt/heavyai/bin/heavysql -p HyperInteractive -u admin omnisci

Restart the database service and check that everything is running as expected.

Remove Omnisci Software from the System

After all the checks confirmed that the upgraded system is stable, clean up the system to remove the Omnisci install and relative system configuration. Remove permanently the configuration of the services.

sudo rm /lib/systemd/omnisci_server*.service
sudo rm /lib/systemd/omnisci_web_server*.service
sudo systemctl daemon-reload
sudo systemctl reset-failed

Remove the installed software.

sudo rm -Rf /opt/omnisci

Delete the YUM or APT repositories.

sudo rm /etc/yum.repos.d/omnisci.repo

sudo rm /etc/apt/sources.list.d/omnisci.list

PreviousUpgrading HEAVY.AI NextCUDA Compatibility Drivers