1 of 20

Installation

You can download the latest version of HEAVY.AI for your preferred platform by reaching our support site or referring to the links provided in the instructions.

The CPU (no GPUs) install does not support backend rendering. For example, Pointmap and Scatterplot charts are not available. The GPU install supports all chart types.

The Open Source options do not require a license, and omit HeavyImmerse.

Docker
Ubuntu
Rocky Linux / RHEL
AWS
GCP
Azure
Jupyter
Upgrading
Uninstalling

Free Version

HEAVY.AI Free is a full-featured version of the HEAVY.AI platform available at no cost for non-hosted commercial use.

HEAVY.AI Free includes access to the following:

Up to 32GB of RAM
Supports 1 GPU
Up to 3 active users
Advanced Analytics
Rendering Engine
Immerse Dashboards
HeavyDB
Sharing
Support access through the HEAVY.AI Community

To get started with HEAVY.AI Free:

Go to Get Started with HEAVY.AI, and in the HEAVY.AI Free section, click Free License.
On the Get HEAVY.AI Free page, enter the requested information, agree to the HEAVY.AI EULA and the HeavyIQ EULA Addendum, then click I Agree.
Open the Your HEAVY.AI Free License Key email to view and download the free edition license key. You will need this license key to run HEAVY.AI after you install it.
In the Download HEAVY.AI section of the Free License Key email, click See Install Options to select the best version of HEAVY.AI for your hardware and software configuration. Follow the instructions for the download or cloud version you choose.
Under the Install / Configure section of the Free License Key email, click the Installation Guide link and follow our documentation to install HEAVY.AI and get your environment up and running.

Add Users

You can create additional HEAVY.AI users to collaborate with.

Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Open the SQL Editor.
Use the CREATE USER command to create a new user. For information on syntax and options, see CREATE USER.

Installing on Docker

Installing HEAVY.AI on Docker

In this section you will find the recipes to install HEAVY.AI platform using Docker. We provide instructions for installing Docker and HEAVY.AI using an Ubuntu host machine. However, advanced users may install the CPU version of HEAVY.AI (EE/Free or Open Source) on any host system running Docker, or install the GPU (EE/Free or Open Source) version of HEAVY.AI on any Linux-based system with supported NVIDIA drivers and nvidia-docker packages (See Software Requirements).

HEAVY.AI Installation using Docker on Ubuntu

Follow these steps to install HEAVY.AI as a Docker container on a machine running with on CPU or with supported NVIDIA GPU cards using Ubuntu as the host OS.

Preparation

Prepare your host by installing Docker and if needed for your configuration NVIDIA drivers and NVIDIA runtime.

Install Docker

Remove any existing Docker Installs and if on GPU the legacy NVIDIA docker runtime.

sudo apt-get purge nvidia-docker
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

Add Docker's GPG key using curl and ca-certificates

sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

Add Docker to your Apt repository.

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update your repository.

sudo apt update

Install Docker, the command line interface, and the container runtime.

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Run the following usermod command so that Docker command execution does not require sudo privilege. Log out and log back in for the changes to take effect. (recommended)

sudo usermod  --append --groups docker $USER

Verify your Docker installation.

sudo docker run hello-world

Install NVIDIA Drivers and NVIDIA Container ᴳᴾᵁ ᴼᴾᵀᴵᴼᴺ

Install NVIDIA Drivers

Install NVIDIA Docker Runtime

Use curl to add Nvidia's Gpg key:

curl --silent --location https://nvidia.github.io/nvidia-container-runtime/gpgkey | \
sudo apt-key add -

Update your sources list:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl --silent --location https://nvidia.github.io/nvidia-container-runtime/$distribution/nvidia-container-runtime.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list

Update apt-get and install nvidia-container-runtime:

sudo apt-get update
sudo apt-get install -y nvidia-container-runtime

Edit /etc/docker/daemon.json to add the following, and save the changes:

{
  "default-runtime": "nvidia",
  "runtimes": {
     "nvidia": {
         "path": "/usr/bin/nvidia-container-runtime",
         "runtimeArgs": []
     }
 }
}

Restart the Docker daemon:

sudo pkill -SIGHUP dockerd

Check NVIDIA Drivers

Verify that docker and NVIDIA runtime work together.

sudo docker run --gpus=all \
--rm nvidia/cuda:12.4.1-runtime-ubuntu22.04 nvidia-smi

If everything is working you should get the output of nvidia-smi command showing the installed GPUs in the system.

HEAVY.AI Installation

Create a directory to store data and configuration files

sudo mkdir -p /var/lib/heavyai && sudo chown $USER /var/lib/heavyai

Then a minimal configuration file for the docker installation

echo "port = 6274
http-port = 6278
calcite-port = 6279
data = \"/var/lib/heavyai\"
null-div-by-zero = true

[web]
port = 6273
frontend = \"/opt/heavyai/frontend\"" \
>/var/lib/heavyai/heavy.conf

Ensure that you have sufficient storage on the drive you choose for your storage dir running this command

if test -d /var/lib/heavyai; then echo "There is $(df -kh /var/lib/heavyai --output="avail" | sed 1d) avaibale space in you storage dir"; else echo "There was a problem with the creation of storage dir";  fi;

Optional: Download HEAVY.AI from Release Website

The subsequent section will download and install an image using DockerHub. However, if you wish to avoid pulling from DockerHub and instead download and prepare a specific image, follow the instructions in this section. To Download a specific version, visit one of the following websites, choose the version that you wish to install, right click and select "COPY URL".

Use files ending in -render-docker.tar.gz to install GPU edition and -cpu-docker.tar.gz to install CPU editions.

Then, on the server where you wish to install HEAVY.AI, run the following command (Replacing $DOWNLOAD_URL with the URL from your clipboard).

wget $DOWNLOAD_URL

Await successful download and run ls | grep heavy to see the filename of the package you just downloaded. Copy the filename to your clipboard, and then run the next command replacing $DOWNLOADED_FILENAME with the contents of your clipboard.

docker load < $DOWNLOADED_FILENAME

The command will return a Docker image name. Replace heavyai/heavyai-(...):latest with the image you just loaded.

Download HEAVY.AI from DockerHub and Start HEAVY.AI in Docker.

Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are going to use.

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cuda:latest

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:latest

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cuda:latest

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:latest

Replace ":latest" with ":vX.Y.Z" to pull a specific docker version. (Eg: heavyai-ee-cuda:v8.0.1)

Check that the docker is up and running a docker ps commnd:

sudo docker container ps --format "{{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see an output similar to the following.

heavyai/heavyai-ee-cuda Up 48 seconds ago

Configure Firewall ᴼᴾᵀᴵᴼᴺᴬᴸ

If a firewall is not already installed and you want to harden your system, install theufw.

sudo apt install ufw
sudo ufw allow ssh

To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.

sudo ufw disable
sudo ufw allow 6273:6278/tcp
sudo ufw enable

Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.

Licensing HEAVY.AI ᵉᵉ⁻ᶠʳᵉᵉ ᵒⁿˡʸ

Connect to Heavy Immerse using a web browser to your host on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.

Command-Line Access

You can access the command line in the Docker image to perform configuration and run HEAVY.AI utilities.

You need to know the container-id to access the command line. Use the command below to list the running containers.

sudo docker container ps

You see output similar to the following.

CONTAINER ID        IMAGE                     COMMAND                     CREATED             STATUS              PORTS                                            NAMES
9e01e520c30c        heavyai/heavyai-ee-gpu    "/bin/sh -c '/heavyai..."   50 seconds ago      Up 48 seconds ago   0.0.0.0:6273-6280->6273-6280/tcp                 confident_neumann

Once you have your container ID, in the example 9e01e520c30c, you can access the command line using the Docker exec command. For example, here is the command to start a Bash session in the Docker instance listed above. The -it switch makes the session interactive.

sudo docker exec -it 9e01e520c30c bash

You can end the Bash session with the exit command.

Final Checks

Load Sample Data and Run a Simple Query

HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.

sudo docker exec -it <container-id> \
./insert_sample_data --data /var/lib/heavyai/storage

Where <container-id> is the container in which HEAVY.AI is running.

When prompted, choose whether to insert dataset 1 (7,000,000 rows), dataset 2 (10,000 rows), or dataset 3 (683,000 rows). The examples below use dataset 2.

Enter dataset number to download, or 'q' to quit:
#     Dataset                   Rows    Table Name             File Name
1)    Flights (2008)            7M      flights_2008_7M        flights_2008_7M.tar.gz
2)    Flights (2008)            10k     flights_2008_10k       flights_2008_10k.tar.gz
3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz

Connect to HeavyDB by entering the following command (a password willò be asked; the default password is HyperInteractive):

sudo docker exec -it <container-id> bin/heavysql

Enter a SQL query such as the following:

SELECT origin_city AS "Origin", 
dest_city AS "Destination", 
ROUND(AVG(airtime),1) AS "Average Airtime" 
FROM flights_2008_10k 
WHERE distance < 175 GROUP BY origin_city,
dest_city;

The results should be similar to the results below.

Origin|Destination|Average Airtime
West Palm Beach|Tampa|33.8
Norfolk|Baltimore|36.1
Ft. Myers|Orlando|28.7
Indianapolis|Chicago|39.5
Tampa|West Palm Beach|33.3
Orlando|Ft. Myers|32.6
Austin|Houston|33.1
Chicago|Indianapolis|32.7
Baltimore|Norfolk|31.7
Houston|Austin|29.6

Installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.

Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.

Create a new dashboard and a Scatter Plot to verify that backend rendering is working.

Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.

The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.\

Create a new dashboard and a Table chart to verify that Heavy Immerse is working.

Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data sour
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.

The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.\

¹ In the OS Edition, Heavy Immerse Service is unavailable.

² The OS Edition does not require a license key.

Installing on Ubuntu

In this section, you will find recipes to install HEAVY.AI platform and NVIDIA drivers using package manager like apt or tarball.

HEAVY.AI Installation on Ubuntu

This is an end-to-end recipe for installing HEAVY.AI on a Ubuntu 22.04 machine using CPU and GPU devices.

The order of these instructions is significant. To avoid problems, install each component in the order presented.

Assumptions

These instructions assume the following:

You are installing on a “clean” Ubuntu 22.04 host machine with only the operating system installed.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.

Preparation

Prepare your Ubuntu machine by updating your system, creating the HEAVY.AI user (named heavyai), installing kernel headers, installing CUDA drivers, and optionally enabling the firewall.

Update and Reboot

Update the entire system:

sudo apt update
sudo apt upgrade

2. Install the utilities needed to create Heavy.ai repositories and download archives:

sudo apt install curl
sudo apt install libncurses5

3. Install the headless JDK and the utility apt-transport-https:

sudo apt install default-jre-headless apt-transport-https

4. Reboot to activate the latest kernel:

sudo reboot

Create the HEAVY.AI User

Create a group called heavyai and a user named heavyai, who will be the owner of the HEAVY.AI software and data on the filesystem.

Create the group, user, and home directory using the useradd command with the --user-group and --create-home switches.

sudo useradd --user-group --create-home --group sudo heavyai

2. Set a password for the user:

sudo passwd heavyai

3. Log in with the newly created user:

sudo su - heavyai

Installation

Install the HEAVY.AI using APT and a tarball.

The installation using the APT package manager is recommended to those who want a more automated install and upgrade procedure.

Install NVIDIA Drivers ᴳᴾᵁ ᴼᴾᵀᴵᴼᴺ

If your system uses NVIDIA GPUs, but the drivers not installed, install them now. See Install NVIDIA Drivers and Vulkan on Ubuntu for details.

Installing with APT

Download and add a GPG key to APT.

curl https://releases.heavy.ai/GPG-KEY-heavyai | sudo apt-key add -

Add a source apt depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you are going to use.

echo "deb https://releases.heavy.ai/ee/apt/ stable cuda" \
| sudo tee /etc/apt/sources.list.d/heavyai.list

echo "deb https://releases.heavy.ai/ee/apt/ stable cpu" \
| sudo tee /etc/apt/sources.list.d/heavyai.list

echo "deb https://releases.heavy.ai/os/apt/ stable cuda" \
| sudo tee /etc/apt/sources.list.d/heavyai.list

echo "deb https://releases.heavy.ai/os/apt/ stable cpu" \
| sudo tee /etc/apt/sources.list.d/heavyai.list

Use apt to install the latest version of HEAVY.AI.

sudo apt update
sudo apt install heavyai

If you need to install a specific version of HEAVY.AI, because you are upgrading from Omnisci or for different reasons, you must run the following command:

hai_version="6.0.0"
sudo apt install heavyai=$(apt-cache madison heavyai | grep $hai_version | cut -f 2 -d '|' | xargs)

Installing with a Tarball

First create the installation directory.

sudo mkdir /opt/heavyai && sudo chown $USER /opt/heavyai

Download the archive and install the software. A different archive is downloaded depending on the Edition (Enterprise, Free, or Open Source) and the device used for runtime (GPU or CPU).

curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-render.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

Configuration

Follow these steps to prepare your HEAVY.AI environment.

Set Environment Variables

For convenience, you can update .bashrc with these environment variables

echo "# HEAVY.AI variable and paths
export HEAVYAI_PATH=/opt/heavyai
export HEAVYAI_BASE=/var/lib/heavyai
export HEAVYAI_LOG=$HEAVYAI_BASE/storage/log
export PATH=$HEAVYAI_PATH/bin:$PATH" \
>> ~/.bashrc
source ~/.bashrc

Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain respectively the paths where configuration, license, and data files are stored and where the software is installed. Setting them is strongly recommended.

Initialization

Run the systemd installer to create heavyai services, a minimal config file, and initialize the data storage.

cd $HEAVYAI_PATH/systemd
./install_heavy_systemd.sh

Accept the default values provided or make changes as needed.

The script creates a data directory in $HEAVYAI_BASE/storage (default /var/lib/heavyai/storage) with the directories catalogs, data, export and log.The import directory is created when you insert data the first time. If you are HEAVY.AI administrator, the log directory is of particular interest.

Activation

Start and use HeavyDB and Heavy Immerse. ¹

Heavy Immerse is not available in the OSS Edition, so if running the OSS Edition the systemctl command using the heavy_web_server has no effect.

Enable the automatic startup of the service at reboot and start the HEAVY.AI services.

sudo systemctl enable heavydb --now
sudo systemctl enable heavy_web_server --now

sudo systemctl enable heavydb --now

Configure Firewall ᴼᴾᵀᴵᴼᴺᴬᴸ

If a firewall is not already installed and you want to harden your system, install theufw.

sudo apt install ufw
sudo ufw allow ssh

To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.

sudo ufw disable
sudo ufw allow 6273:6278/tcp
sudo ufw enable

Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.

For more information, see https://help.ubuntu.com/lts/serverguide/firewall.html.

Licensing HEAVY.AI ᵉᵉ⁻ᶠʳᵉᵉ ᵒⁿˡʸ

If you are using Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key.

Skip this section if you are on Open Source Edition ²

Copy your license key of Enterprise or Free Edition from the registration email message. If you do not have a license and you want to evaluate HEAVI.AI in an unlimited
enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition here. If you need a Free License you can get one here.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
.

Final Checks

To verify that everything is working, load some sample data, perform a heavysql query, and generate a Pointmap using Heavy Immerse ¹

Load Sample Data and Run a Simple Query

HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.

cd $HEAVYAI_PATH
sudo ./insert_sample_data --data /var/lib/heavyai/storage

#     Enter dataset number to download, or 'q' to quit:
Dataset           Rows    Table Name          File Name
1)    Flights (2008)    7M      flights_2008_7M     flights_2008_7M.tar.gz
2)    Flights (2008)    10k     flights_2008_10k    flights_2008_10k.tar.gz
3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz

Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):

$HEAVYAI_PATH/bin/heavysql
password: ••••••••••••••••

Enter a SQL query such as the following

SELECT origin_city AS "Origin", 
dest_city AS "Destination", 
AVG(airtime) AS "Average Airtime" 
FROM flights_2008_10k WHERE distance < 175 
GROUP BY origin_city, dest_city;

The results should be similar to the results below.

Origin|Destination|Average Airtime
Austin|Houston|33.055556
Norfolk|Baltimore|36.071429
Ft. Myers|Orlando|28.666667
Orlando|Ft. Myers|32.583333
Houston|Austin|29.611111
Baltimore|Norfolk|31.714286

Create a Dashboard Using Heavy Immerse ᵉᵉ⁻ᶠʳᵉᵉ ᵒⁿˡʸ ¹

After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.

Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.

Create a new dashboard and a Scatter Plot to verify that backend rendering is working.

Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.

The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.\

Create a new dashboard and a Table chart to verify that Heavy Immerse is working.

Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data source.
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.

The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.\

¹ In the OS Edition, Heavy Immerse is unavailable.

² The OS Edition does not require a license key.

Install NVIDIA Drivers and Vulkan on Ubuntu

Installation Prerequisites

Hardware Check

Check that your system has a compatible NVIDIA GPU. If you don't know the exact GPU model in your system run this command:

lspci -v | egrep "NVIDIA"

This command should output one row per installed NVIDIA GPU. Check that your hardware is compatible on our page. Note that you can still install the CPU Edition of HEAVY.AI on machines that do not have an NVIDIA GPU.

Pre-Installation Updates

Upgrade the system and the kernel, then the machine if needed.

sudo apt update

sudo apt upgrade -y
sudo reboot

Install Kernel Headers

Install kernel headers and development packages.

sudo apt install linux-headers-$(uname -r)

Install the extra packages.

sudo apt install pciutils

Installing Vulkan Library

The rendering engine of HEAVY.AI (present in Enterprise Editions) requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself may not be able to start.

Install the Vulkan library and its dependencies using apt.

sudo apt install libvulkan1

Installing NVIDIA Drivers

Installing NVIDIA drivers with support for the CUDA platform is required to run GPU-enabled versions of HEAVY.AI.

You can install NVIDIA drivers in multiple ways, we've outlined three available options below - we recommend Option 1.

It is advisable to keep a record of the installation method used, as upgrading NVIDIA drivers will require the utilization of the same method for successful results.

What is CUDA? What is the CUDA toolkit?

The CUDA Toolkit from NVIDIA provides everything you need to develop GPU-accelerated applications. The CUDA Toolkit includes GPU-accelerated libraries, a compiler, development tools and the CUDA runtime. The CUDA Toolkit is not required to run HEAVY.AI, but you must install the CUDA toolkit if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.

Option 1: Install NVIDIA Drivers with CUDA Toolkit from NVIDIA Website

The minimum CUDA version supported by HEAVY.AI is 11.4. We recommend using a release that has been available for at least two months.

In the "Target Platform" section, follow these steps:

For "Operating System" select Linux
For Architecture" select x86_64
For "Distribution" select Ubuntu
For "Version" select the version of your operating system (20.04)
For "Installer Type" choose deb (network) **
One by one, run the presented commands in the Installer Instructions section on your server.

** You may optionally use any of the "Installer Type" options available.

If you choose to use the .run file option, prior to running the installer you will need to manually install build-essentials using apt and change permissions of the downloaded .run file to allow execution.

Option 2: Install NVIDIA Drivers via .run file using the NVIDIA Website

If you don't know the exact GPU model in your system run this command

lspci -v | egrep "3D|VGA*.NVIDIA" | awk -F '\[|\]' ' { print $2 } '

You'll get an output in the format Product Type, Series and Model

Tesla T4

In this example, the Product type is Tesla the Series is T (as Turing), and the model is T4.

Select the Product Type as the one you got with the command.
Select the correct Product Series and Product Type for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download
On the subsequent page, if you agree to the terms, right click on "Agree and Download" and select "Copy Link Address". You may also manually download and transfer to your server, skipping the next step.
On your server, type wget and paste the URL you copied in the previous step. Press enter to download.

Install the tools needed for installation.

sudo apt install build-essential

Change the permissions of the downloaded .run file to allow execution, and run the installation.

chmod +x NVIDIA-Linux-x86_64-*.run
sudo ./NVIDIA-Linux--x86_64-*.run

Option 3: Install NVIDIA drivers using APT

Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the apt package manager.

Run the command to get a list of the available driver's version

apt list nvidia-driver-*
Listing... Done

nvidia-driver-450/bionic-updates,bionic-security 460.91.03-0ubuntu0.18.04.1 amd64
nvidia-driver-450-server/bionic-updates,bionic-security 450.172.01-0ubuntu0.18.04.1 amd64
nvidia-driver-455/bionic-updates,bionic-security 460.91.03-0ubuntu0.18.04.1 amd64
nvidia-driver-460/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-465/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-470/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-470-server/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-495/bionic-updates,bionic-security 510.60.02-0ubuntu0.18.04.1 amd64
nvidia-driver-510/bionic-updates,bionic-security 510.60.02-0ubuntu0.18.04.1 amd64
nvidia-driver-510-server/bionic-updates,bionic-security 510.47.03-0ubuntu0.18.04.1 amd64

Install the driver version needed with apt

sudo apt install nvidia-driver-<version>

NVIDIA Driver Post-Installation steps

Reboot your system to ensure the new version of the driver is loaded

sudo reboot

Verify Successful NVIDIA driver installation

Run nvidia-smi to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see something like this to confirm that your NVIDIA GPUs and drivers are present.

If you see an error like the following, the NVIDIA drivers are probably installed incorrectly:

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. 
Make sure that the latest NVIDIA driver is installed and running.

Review the installation instructions, specifically checking for completion of install prerequisites, and correct any errors.

Install Vulkan library

The rendering engine of HEAVY.AI requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself can't even start without disabling the back-end renderer.

Install the Vulkan library and its dependencies using apt.

sudo apt install libvulkan1

Advanced Installation

You must install the CUDA toolkit and Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.

Install CUDA Toolkit ᴼᴾᵀᴵᴼᴺᴬᴸ

Install the NVIDIA public repository GPG key.

distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
sudo apt-key adv --fetch-keys \
https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/3bf863cc.pub

Add the repository.

echo "deb http://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64 /" \
| sudo tee /etc/apt/sources.list.d/cuda.list
apt update

List the available Cuda toolkit versions.

apt list cuda-toolkit-* | grep -v config

Listing...
cuda-toolkit-10-0/unknown 10.0.130-1 amd64
cuda-toolkit-10-1/unknown 10.1.243-1 amd64
cuda-toolkit-10-2/unknown 10.2.89-1 amd64
cuda-toolkit-11-0/unknown 11.0.3-1 amd64
cuda-toolkit-11-1/unknown 11.1.1-1 amd64
cuda-toolkit-11-2/unknown 11.2.2-1 amd64
cuda-toolkit-11-3/unknown 11.3.1-1 amd64
cuda-toolkit-11-4/unknown 11.4.4-1 amd64
cuda-toolkit-11-5/unknown 11.5.2-1 amd64
cuda-toolkit-11-6/unknown 11.6.2-1 amd64
cuda-toolkit-11-7/unknown 11.7.0-1 amd64

Install the CUDA toolkit using apt.

sudo apt install cuda-toolkit-<version>

Verification

Check that everything is working and the toolkit has been installed.

/usr/local/cuda/bin/nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

Install Clang ᴼᴾᵀᴵᴼᴺᴬᴸ

You must install Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities. Install Clang and LLVM dependencies using apt.

sudo apt install clang

Verification

Check that the software is installed and in the execution path.

clang --version
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Installing on Rocky Linux / RHEL

In this section you will find a recipe to install the HEAVY.AI platform on Red Hat and derivates like Rocky Linux.

HEAVY.AI Installation on RHEL

This is an end-to-end recipe for installing HEAVY.AI on a Red Hat Enterprise 8.x machine using CPU and GPU devices.

The order of these instructions is significant. To avoid problems, install each component in the order presented.

The same instructions can be used to install on RL / RHEL 9, which some minor modifications.

Assumptions

These instructions assume the following:

You are installing a "clean" Rocky Linux / RHEL 8 host machine with only the operating system.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.

Preparation

Prepare your machine by updating your system and optionally enabling or configuring a firewall.

Update and Reboot

Update the entire system and reboot the system if needed.

sudo dnf -y update
sudo reboot

Install the utilities needed to create HEAVY.AI repositories and download installation binaries.

sudo dnf -y install dnf-utils curl libldap2-dev

JDK

Open a terminal on the host machine.
Install the headless JDK using the following command:

sudo dnf -y install java-1.8.0-openjdk-headless

Create the HEAVY.AI User

Create a group called heavyai and a user named heavyai, who will own HEAVY.AI software and data on the file system.

You can create the group, user, and home directory using the useradd command with the --user-group and --create-home switches:

sudo useradd --user-group --create-home --groups wheel heavyai

Set a password for the user using the passwd command.

sudo passwd heavyai

sudo su - heavyai

Installation

There are two ways to install the heavy.ai software

Using the DNF package manager for installation is highly recommended due to its ability to handle dependencies and streamline the installation process, making it a preferred choice for many users.

Install NVIDIA Drivers ᴳᴾᵁ ᴼᴾᵀᴵᴼᴺ

If your system includes NVIDIA GPUs but the drivers are not installed, it is advisable to install them before proceeding with the suite installation.

Installing with DNF

Create a DNF repository depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you will use.

sudo  dnf config-manager --add-repo \
https://releases.heavy.ai/ee/yum/stable/cuda

sudo yum-config-manager --add-repo \
https://releases.heavy.ai/ee/yum/stable/cpu

sudo yum-config-manager --add-repo \
https://releases.heavy.ai/os/yum/stable/cuda

sudo yum-config-manager --add-repo \
https://releases.heavy.ai/os/yum/stable/cpu

Add the GPG-key to the newly added repository.

sudo dnf config-manager --save \
--setopt="releases.heavy*.gpgkey=https://releases.heavy.ai/GPG-KEY-heavyai"

Use DNF to install the latest version of HEAVY.AI.

sudo dnf -y install heavyai.x86_64

You can use the DNF package manager to list the available packages when installing a specific version of HEAVY.AI, such as when a multistep upgrade is necessary, or a specific version is needed for any other reason. sudodnf --showduplicateslistheavyai Select the version needed from the list (e.g. 7.0.0) and install using the command.

sudodnfinstallheavyai-7.0.0_20230501_be4f51b048-1.x86_64

Installing with a Tarball

Let's begin by creating the installation directory.

sudo mkdir /opt/heavyai && sudo chown $USER /opt/heavyai

Download the archive and install the latest version of the software. The appropriate archive is downloaded based on the edition (Enterprise, Free, or Open Source) and the device used for runtime.

curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-render.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai

Configuration

Follow these steps to configure your HEAVY.AI environment.

Set Environment Variables

For your convenience, you can update .bashrc with these environment variables

echo "# HEAVY.AI variable and paths
export HEAVYAI_PATH=/opt/heavyai
export HEAVYAI_BASE=/var/lib/heavyai
export HEAVYAI_LOG=\$HEAVYAI_BASE/storage/log
export PATH=\$HEAVYAI_PATH/bin:$PATH" \
>> ~/.bashrc
source ~/.bashrc

Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain the paths where configuration, license, and data files are stored and the location of the software installation. It is strongly recommended that you set them up.

Initialization

Run the script that will initialize the HEAVY.AI services and database storage located in the systemd folder.

cd $HEAVYAI_PATH/systemd
./install_heavy_systemd.sh

Accept the default values provided or make changes as needed.

This step will take a few minutes if you are installing a CUDA-enabled version of the software because the shaders must be compiled.

The script creates a data directory in $HEAVYAI_BASE/storage (typically /var/lib/heavyai) with the directories catalogs, dataand log, which will contain the metadata, the data of the database tables, and the log files from Immerse's web server and the database. The log folder is particularly important for database administrators. It contains data about the system's health, performance, and user activities.

Activation

Heavy Immerse is not available in the OS Edition.

Start the services and enable the automatic startup of the service at reboot and start the HEAVY.AI services.

sudo systemctl enable heavydb --now
sudo systemctl enable heavy_web_server --now

sudo systemctl enable heavydb --now

Configure the Firewall ᴼᴾᵀᴵᴼᴺᴬᴸ

If a firewall is not already installed and you want to harden your system, install and start firewalld.

sudo dnf -y install firewalld
sudo systemctl start firewalld
sudo systemctl enable firewalld
sudo systemctl status firewalld

To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access:

sudo firewall-cmd --zone=public --add-port=6273-6274/tcp --add-port=6278/tcp --permanent
sudo firewall-cmd --reload

Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.

Licensing HEAVY.AI ᵉᵉ⁻ᶠʳᵉᵉ ᵒⁿˡʸ

Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.

The $HEAVYAI_BASE directory must be dedicated to HEAVYAI; do not set it to a directory shared by other packages.

Final Checks

Load Sample Data and Run a Simple Query

HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.

cd $HEAVYAI_PATH
sudo ./insert_sample_data --data /var/lib/heavyai/storage

#     Enter dataset number to download, or 'q' to quit:
Dataset           Rows    Table Name          File Name
1)    Flights (2008)    7M      flights_2008_7M     flights_2008_7M.tar.gz
2)    Flights (2008)    10k     flights_2008_10k    flights_2008_10k.tar.gz
3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz

Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):

$HEAVYAI_PATH/bin/heavysql -p HyperInteractive

anEnter a SQL query such as the following:

SELECT origin_city AS "Origin", 
dest_city AS "Destination", 
AVG(airtime) AS "Average Airtime" 
FROM flights_2008_10k WHERE distance < 175 
GROUP BY origin_city, dest_city;

The results should be similar to the results below.

Origin|Destination|Average Airtime
Austin|Houston|33.055556
Norfolk|Baltimore|36.071429
Ft. Myers|Orlando|28.666667
Orlando|Ft. Myers|32.583333
Houston|Austin|29.611111
Baltimore|Norfolk|31.714286

After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.

Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.

Create a new dashboard and a Scatter Plot to verify that backend rendering is working.

Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.

The resulting chart clearly demonstrates that there is a direct correlation between departure delay and arrival delay. This insight can help in identifying areas for improvement and implementing strategies to minimize delays and enhance overall efficiency.\

Create a new dashboard and a Table chart to verify that Heavy Immerse is working.

Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data source
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.

The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.\

¹ In the OS Edition, Heavy Immerse is unavailable.

² The OS Edition does not require a license key.

Install NVIDIA Drivers and Vulkan on Rocky Linux and RHEL

Install Prerequisites

Install the Extra Packages for Enterprise Linux (EPEL) repository and other packages before installing NVIDIA drivers.

sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

RHEL-based distributions require Dynamic Kernel Module Support (DKMS) to build the GPU driver kernel modules. For more information, see https://fedoraproject.org/wiki/EPEL. Upgrade the kernel and restart the machine.

sudo dnf -y upgrade kernel
sudo reboot now

Install Kernel Headers

Install kernel headers and development packages:

sudo dnf -y install kernel-devel-$(uname -r) kernel-headers-$(uname -r)

If installing kernel headers does not work correctly, follow these steps instead:

Identify the Linux kernel you are using by issuing the uname -r command.
Use the name of the kernel (4.18.0-553.el8_10.x86_64 in the following code example) to install kernel headers and development packages:

sudo dnf -y install \
kernel-devel-4.18.0-553.el8_10.x86_64 \
kernel-headers-4.18.0-553.el8_10.x86_64

Install the dependencies and extra packages:

Sudo dnf install -y kernel-devel kernel-headers pciutils dkms

Install NVIDIA Drivers and Vulkan

CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see https://developer.nvidia.com/cuda-zone. You can install drivers in multiple ways. This section provides installation information using the NVIDIA website or using dnf.

Although using the NVIDIA website is more time-consuming and less automated, you are assured that the driver is certified for your GPU. Use this method if you are not sure which driver to install. If you prefer a more automated method and are confident that the driver is certified, you can use the DNF package manager method.

Please check that the driver's version you download meets the HEAVI.AI minimum requirements.

Install NVIDIA Drivers Using the NVIDIA Website

Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website (https://developer.nvidia.com/cuda-downloads).

If you do not know the GPU model installed on your system, run this command:

lspci -v | egrep "3D|VGA*.NVIDIA" | awk -F '\[|\]' ' { print $2 } '

The output shows the product type, series, and model. In this example, the product type is Tesla, the series is T (as Turing), and the model is T4.

Tesla T4

Select the product type shown after running the command above.
Select the correct product series and model for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download.

Please check that the driver's version you download meets the HEAVI.AI minimum requirements.

Move the downloaded file to the server, change the permissions, and run the installation.

chmod +x NVIDIA-Linux-x86_64-*.run
sudo ./NVIDIA-Linux--x86_64-*.runYou might get the following error during installation:

You might receive the following error during installation:

ERROR: The Nouveau kernel driver is currently in use by your system. This driver is incompatible with the NVIDIA driver, and must be disabled before proceeding. Please consult the NVIDIA driver README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.

If you receive this error, blacklist the Nouveau driver by editing the /etc/modprobe.d/blacklist-nouveau.conffile, adding the following lines at the end:

blacklist nouveau blacklist lbm-nouveau options nouveau modeset=0 alias nouveau off alias lbm-nouveau off

Install NVIDIA Drivers Using DNF

Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the DNF package manager.

When installing the driver, ensure your GPU model is supported and meets the HEAVI.AI minimum requirements.

Add the NVIDIA network repository to your system.

sudo dnf config-manager --add-repo \
http://developer.download.nvidia.com/compute/cuda/repos/rhel8/$(uname -i)/cuda-rhel8.repo

Install the driver version needed with dnf. For 8.0, the minimum version is 535.

sudo dnf -y module install nvidia-driver:535-dkms

To load the installed driver, you can run sudo modprobe nvidia or nvidia-smi commands, or , in case of driver upgrade, you can reboot your system to ensure that the new version of the driver is loaded using the command sudo reboot

Check NVIDIA Driver Installation

Run the specified command to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see output confirming the presence of your NVIDIA GPUs and drivers. This verification step ensures that your system can identify and utilize the GPUs as intended.

If you encounter an error similar to the following, the NVIDIA drivers are likely installed incorrectly: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Please ensure that the latest NVIDIA driver is installed and running.

Please review the Install NVIDIA Drivers section and correct any errors.

Install Vulkan

The back-end renderer requires a Vulkan-enabled driver and the Vulkan library to work correctly. Without these components, the database cannot start without disabling the back-end renderer.

To ensure the Vulkan library and its dependencies are installed, use the DNF.

sudo dnf -y install vulkan

For more information about troubleshooting Vulkan, see the Vulkan Renderer section.

Install CUDA Toolkit ᴼᴾᵀᴵᴼᴺᴬᴸ

You must install the CUDA Toolkit if you use advanced features like C++ User-Defined Functions or User-Defined Table Functions to extend the database capabilities.

Add the NVIDIA network repository to your system:

sudo dnf config-manager --add-repo \
http://developer.download.nvidia.com/compute/cuda/repos/rhel8/$(uname -i)/cuda-rhel8.repo

2. List the available CUDA Toolkit versions using the DNF list command

dnf list cuda-toolkit-* | egrep -v config

Available Packages
cuda-toolkit-10-1.x86_64                     10.1.243-1        cuda-rhel8-x86_64
cuda-toolkit-10-2.x86_64                     10.2.89-1         cuda-rhel8-x86_64
cuda-toolkit-11-0.x86_64                     11.0.3-1          cuda-rhel8-x86_64
cuda-toolkit-11-1.x86_64                     11.1.1-1          cuda-rhel8-x86_64
cuda-toolkit-11-2.x86_64                     11.2.2-1          cuda-rhel8-x86_64
cuda-toolkit-11-3.x86_64                     11.3.1-1          cuda-rhel8-x86_64
cuda-toolkit-11-4.x86_64                     11.4.4-1          cuda-rhel8-x86_64
cuda-toolkit-11-5.x86_64                     11.5.2-1          cuda-rhel8-x86_64
cuda-toolkit-11-6.x86_64                     11.6.2-1          cuda-rhel8-x86_64
cuda-toolkit-11-7.x86_64                     11.7.1-1          cuda-rhel8-x86_64
cuda-toolkit-11-8.x86_64                     11.8.0-1          cuda-rhel8-x86_64
cuda-toolkit-12.x86_64                       12.5.0-1          cuda-rhel8-x86_64
cuda-toolkit-12-0.x86_64                     12.0.1-1          cuda-rhel8-x86_64
cuda-toolkit-12-1.x86_64                     12.1.1-1          cuda-rhel8-x86_64
cuda-toolkit-12-2.x86_64                     12.2.2-1          cuda-rhel8-x86_64
cuda-toolkit-12-3.x86_64                     12.3.2-1          cuda-rhel8-x86_64
cuda-toolkit-12-4.x86_64                     12.4.1-1          cuda-rhel8-x86_64
cuda-toolkit-12-5.x86_64                     12.5.0-1          cuda-rhel8-x86_64

3. Install the CUDA Toolkit version using DNF.

sudo dnf -y install cuda-toolkit-<version>.x86_64

4. Check that everything is working correctly:

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

Getting Started on AWS

Getting Started with AWS AMI

You can use the HEAVY.AI AWS AMI (Amazon Web Services Amazon Machine Image) to try HeavyDB and Heavy Immerse in the cloud. Perform visual analytics with the included New York Taxi database, or import and explore your own data.

Many options are available when deploying an AWS AMI. These instructions skip to the specific tasks you must perform to deploy a sample environment.

Prerequisite

You need a security key pair when you launch your HEAVY.AI instance. If you do not have one, create one before you continue.

Go to the EC2 Dashboard.
Select Key Pairs under Network & Security.
Click Create Key Pair.
Enter a name for your key pair. For example, MyKey.
Click Create. The key pair PEM file downloads to your local machine. For example, you would find MyKey.pem in your Downloads directory.

Launching Your Instance

Go to the and select the version you want to use. You can get overview information about the product, see pricing, and get usage and support information.
Click Continue to Subscribe to subscribe.
Read the Terms and Conditions, and then click Continue to Configuration.
Select the Fulfillment Option, Software Version, and Region.
Click Continue to Launch.
On the Launch this software page, select Launch through EC2, and then click Launch.
From the Choose and Instance Type page, select an available EC2 instance type, and click Review and Launch.
Review the instance launch details, and click Launch.
Select a key pair, or click Create a key pair to create a new key pair and download it, and then click Launch Instances.
On the Launch Status page, click the instance name to see it on your EC2 Dashboard Instances page.

Using HEAVY.AI Immerse on Your AWS Instance

To connect to Heavy Immerse, you need your Public IP address and Instance ID for the instance you created. You can find these values on the Description tab for your instance.

To connect to Heavy Immerse:

Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
1. Enter the USERNAME (admin), PASSWORD ( {Instance ID} ), and DATABASE (heavyai). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
On the Dashboards page, click NYC Taxi Rides. Explore and filter the chart information on the NYC Taxis Dashboard.

Importing Your Own Data

Working with your own familiar dataset makes it easier to see the advantages of HEAVY.AI processing speed and data visualization.

To import your own data to Heavy Immerse:

Export your data from your current datastore as a comma-separated value (CSV) or tab-separated value (TSV) file. HEAVY.AI supports Latin-1 ASCII format and UTF-8. If you want to load data with another encoding (for example, UTF-16), convert the data to UTF-8 before loading it to HEAVY.AI.
Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
Enter the USERNAME (admin) and PASSWORD ( {instance ID} ). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
Click Data Manager, and then click Import Data.
Drag your data file onto the table importer page, or use the directory selector.
Click Import Files.
Verify the column names and datatypes. Edit them if needed.
Enter a Name for your table.
Click Save Table.
Click Connect to Table.
On the New Dashboard page, click Add Chart.
Choose a chart type.
Add dimensions and measures as required.
Click Apply.
Enter a Name for your dashboard.
Click Save.

Accessing Your HEAVY.AI Instance Using SSH

Open a terminal window.
Locate your private key file (for example, MyKey.pem). The wizard automatically detects the key you used to launch the instance.
Your key must not be publicly viewable for SSH to work. Use this command to change permissions, if needed:
Connect to your instance using its Public DNS. The default user name is centos or ubuntu, depending on the version you are using. For example:
```
ssh -i MyKey.pem centos@ec2-12-345-678-901.us-west-2.compute.amazonaws.com
```
Use the following command to run the heavysql SQL command-line utility on HeavyDB. The default user is admin and the default password is { Instance ID }:
```
$HEAVYAI_PATH/bin/heavysql
```

Getting Started on GCP

Getting Started with HEAVY.AI on Google Cloud Platform

Follow these instructions to get started with HEAVY.AI on Google Cloud Platform (GCP).

Prerequisites

You must have a Google Cloud Platform account. If you do not have an account, follow to sign up for one.

To launch HEAVY.AI on Google Cloud Platform, you select and configure an instance.

Launching Your HEAVY.AI Instance

On the solution Launcher Page, click Launch on Compute Engine to begin configuring your deployment.

Before deploying a solution with a GPU machine type, avoid potential deployment failure by to make sure that you have not exceeded your limit.

To launch HEAVY.AI on Google Cloud Platform, you select and configure a GPU-enabled instance.

Search for HEAVY.AI on the , and select a solution. HEAVY.AI has four instance types:
- .
- .
- .
- .
On the solution Launcher Page, click Launch to begin configuring your deployment.
On the new deployment page, configure the following:
- Deployment name
- Zone
- Machine type - Click Customize and configure Cores and Memory, and select Extend memory if necessary.
- GPU type. (Not applicable for CPU configurations.)
- Number of GPUs - (Not applicable for CPU configurations.) Select the number of GPUs; subject to quota and GPU type by region. For more information about GPU-equipped instances and associated resources, see .
- Boot disk type
- Boot disk size in GB
- Networking - Set the Network, Subnetwork, and External IP.
- Firewall - Select the required ports to allow TCP-based connectivity to HEAVY.AI. Click More to set IP ranges for port traffic and IP forwarding.
Accept the GCP Marketplace Terms of Service and click Deploy.
In the Deployment Manager, click the instance that you deployed.
Launch the Heavy Immerse client:
- Record the Admin password (Temporary).
- Click the Site address link to go to the Heavy Immerse login page. Enter the password you recorded, and click Connect.
- Copy your license key from the registration email message. If you have not received your license key, contact your Sales Representative or register for your 30-day trial .
- Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
- When prompted, paste your license key in the text box and click Apply.
- Click Connect to start using HEAVY.AI.
On successful login, you see a list of sample dashboards loaded into your instance.

Getting Started on Azure

Getting Started with HEAVY.AI on Microsoft Azure

Follow these instructions to get started with HEAVY.AI on Microsoft Azure.

Prerequisites

You must have a Microsoft Azure account. If you do not have an account, go to the Micrsoft Azure home page to sign up for one.

Configure Your HEAVY.AI Instance

To launch HEAVY.AI on Microsoft Azure, you configure a GPU-enabled instance.

1) Log in to you Microsoft Azure portal.

2) On the left side menu, create a Resource group, or use one that your organization has created.

3) On the left side menu, click Virtual machines, and then click Add.

4) Create your virtual machine:

On the Basics tab:
- In Project Details, specify the Resource group.
- Specify the Instance Details:
  - Virtual machine name
  - Region
  - Image (Ubuntu 16.04 or higher, or CentOS/RHEL 7.0 or higher)
  - Size. Click Change size and use the Family filter to filter on GPU, based on your use case and requirements. Not all GPU VM variants are available in all regions.
- For Username, add any user name other than admin.
- In Inbound Port Rules, click Allow selected ports and select one or more of the following:
  - HTTP (80)
  - HTTPS (443)
  - SSH (22)
On the Disks tab, select Premium or Standard SSD, depending on your needs.
For the rest of the tabs and sections, use the default values.

5) Click Review + create. Azure reviews your entries, creates the required services, deploys them, and starts the VM.

6) Once the VM is running, select the VM you just created and click the Networking tab.

7) Click the Add inbound button and configure security rules to allow any source, any destination, and destination port 6273 so you can access Heavy Immerse from a browser on that port. Consider renaming the rule to 6273-Immerse or something similar so that the default name makes sense.

8) Click Add and verify that your new rule appears.

Azure-specific configuration is complete. Now, follow standard HEAVY.AI installation instructions for your Linux distribution and installation method.

Getting Started on Kubernetes (BETA)

Using HEAVY.AI's Helm Chart on Kubernetes

This documentation outlines how to use HEAVY.AI’s Helm Chart within a Kubernetes environment. It assumes the user is a network administrator within your organization and is an experienced Kubernetes administrator. This is not a beginner guide and does not instruct on Kubernetes installation or administration. It is quite possible you will require additional manifest files for your environment.

Overview

The HEAVY.AI Helm Chart is a template of how to configure deployment of the HEAVY.AI platform. The following files need to be updated/created to reflect the customer's deployment environment.

values.yml
<customer_created>-pv.yml
<customer_created>-pvc.yml

Once the files are updated/created, follow the installation instructions below to install the Helm Chart into your Kubernetes environment.

Where to get the Helm Chart?

The Helm Chart is located in the HEAVY.AI github repository. It can be found here: https://releases.heavy.ai/ee/helm/heavyai-1.0.0.tgz

What’s included?

     Helm-workspace
          ↳heavyai
               ↳Chart.yml
               ↳values.yml
	       ↳templates
	            ↳README.pdf
                    ↳deployment.yml
          ↳misc
               ↳example-heavyai-pv.yml
               ↳example-heavyai-pvc.yml

File Name

Description

Chart.yml

HEAVY.AI Helm Chart. Contains version and contact information.

values.yml

Copy this file and edit values specific to your HEAVY.AI deployment. This is where to note the PVC name. This file is annotated to identify typical customizations and is pre-populated with default values.

README.pdf

These instructions.

deployment.yml

HEAVY.AI platform deployment template. DO NOT EDIT

example-heavyai-pv.yml

Example PV file.

example-heavyai-pv.yml

Example PVC file.

How to install?

Before installing, create a PV/PVC that the deployment will use. Save these files in the regular PVC/PV location used in the customer’s environment. Reference the README.pdf file found in the Helm Chart under templates and the example PV/PVC manifests in the misc folder in the helm chart. The PVC name is then provided to the helm install command.
In your current directory; copy the values.yml file from the HEAVY.AI Helm Chart and customize for your needs.
Run the helm install command with the desired deployment name and Helm Chart.
1. When using a values.yml file:
  $ helm install heavyai --values values.yml heavyaihelmchart-1.0.0.tgz
2. When not using a values.yml file:
  If you only need to change a value or two from the default values.yml file you can use --set instead of a custom values.yml file.
  For example:
  $ helm install heavyai --set pvcName=MyPVCName heavyaihelmchart-1.0.0.tgz

How to uninstall?

To uninstall the helm installed HEAVY.AI instance:

$ helm uninstall heavyai

The PVC and PV space defined for the HEAVY.AI instance is not removed. The retained space must be manually deleted.

Example: values.yml

# Default values for heavyai.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
#
# Version of heavyai to install in the format 'v7.0.0' or 'latest' for the latest version released.
version: v7.0.0
# Persistant volume claim name to use with heavyai.
pvcName: heavyai-pvc
# Namespace to install heavyai in.
nameSpace: heavyai
# Number or GPU's to assign to heavyai or 0 to run the CPU version of heavyai.
gpuNumber: 1
# NodeName to install heavyai on, if you wish to let Kubernetes schedule a host, leave it blank.
nodeName: heavyai-node
# Immerse port redirect of 6273.
hostPortImmerse: 9273
# TCP port redirect of 6274.
hostPortTCP: 9274
# HTTP port redirect of 6278.
hostPortHTTP: 9278

Example: example-heavyai-pvc.yml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: heavyai-pvc
 namespace: heavyai
spec:
 volumeMode: Filesystem
 accessModes:
   - ReadWriteOnce
 resources:
   requests:
     storage: 100Gi
 storageClassName: heavyai

Example: example-heavyai-pv.yml

apiVersion: v1
kind: PersistentVolume
metadata:
 name: heavyai-pv
spec:
 capacity:
   storage: 100Gi
 volumeMode: Filesystem
 accessModes:
   - ReadWriteOnce
 persistentVolumeReclaimPolicy: Retain
 storageClassName: heavyai
 mountOptions:
   - hard
   - nfsvers=4.1
 nfs:
   path: {your nfs path goes here }
   server: { your nfs server name goes here }

Upgrading

In this section, you will find recipes to upgrade from the OmniSci to the HEAVY.AI platform and upgrade between versions of the HEAVY.AI platform.

Supported Upgrade Path

The following table shows the steps needed to move from one version to a later one.

Initial Version

Final Version

Upgrade Steps

OmniSci less than 5.5

HEAVY.AI 7.0

Upgrade to 5.5 --> --> 7.0

OmniSci 5.5 - 5.10

HEAVY.AI 7.0

Upgrade to --> 7.0

HEAVY.AI 6.0

HEAVY.AI 7.0

Upgrade to 7.0

Versions 5.x and 6.0.0 are no longer supported; use these only as needed to facilitate an upgrade to a supported version.

Example: if you are running an OmniSci version older than 5.5, you must first upgrade to 5.5, then upgrade to 6.0 and after that upgrade to 7.0. If you are running 6.0 - 6.4, you can upgrade directly to 7.0 in a single step.

Upgrading HEAVY.AI

This section is giving a recipe to upgrade between fully compatible products version.

As with any software upgrade, it is important that you back up your data before upgrading. Each release introduces efficiencies that are not necessarily compatible with earlier releases of the platform. HeavyAI is never expected to be backward compatible.

Back up the contents of your $HEAVYAI_STORAGE directory.

Upgrading from Omnisci

If you need to upgrade from Omnisci to HEAVY.AI 6.0 or later, please refer to the specific recipe.

Direct upgrades from Omnisci to HEAVY.AI version later than 6.0 aren't allowed nor supported.

Upgrading Using Docker

To upgrade HEAVY.AI in place in Docker

In a terminal window, get the Docker container ID.

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep omnisci\/

You should see output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

Stop the HEAVY.AI Docker container. For example:

docker container stop 9e01e520c30c

Optionally, remove the HEAVY.AI Docker container. This removes unused Docker containers on your system and saves disk space.

docker container rm 9e01e520c30c

Backup the Omnisci data directory (typically /var/lib/omnisci)

tar zcvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Download the latest version of the HEAVY.AI Docker image according to the Edition and device you are actually coming from Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.

sudo docker run -d --gpus=all \
  -v /var/lib/heavyai:/var/lib/heavyai \
  -p 6273-6278:6273-6278 \
  heavyai/heavyai-ee-cuda:latest

sudo docker run -d -v \
/var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:latest

sudo docker run -d --gpu=all \
  -v /var/lib/heavyai:/var/lib/heavyai \
  -p 6273-6278:6273-6278 \
  heavyai/core-os-cuda:latest

sudo docker run -d -v \
/var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:latest

If you don't want to upgrade to the latest version but want to upgrade to a specific version, change thelatesttag with the version needed.

If the version needed is the 6.0 use v6.0.0 as the version tag in the image name

heavyai/heavyai-ee-cuda:v6.0.0

Check that the docker is up and running a docker ps commnd:

sudo docker container ps --format "{{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see an output similar to the following.

heavyai/heavyai-ee-cuda Up 48 seconds ago

This runs both the HEAVY.AI database and Immerse in the same container.

You can optionally add --rm to the Docker run command so that the container is removed when it is stopped.

See also the note regarding the CUDA JIT Cache in Optimizing Performance.

Upgrading HEAVY.AI Using Package Managers and Tarball

To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data

Stop the HEAVY.AI services.

sudo systemctl stop heavydb heavy_web_server

Back up your $HEAVYAI_STORAGE directory (the default location is /var/lib/heavyai).

Run the appropriate set of commands depending on the method used to install the previous version of the software.

sudo yum update heavyai.x86_64

sudo apt update
sudo apt upgrade heavyai

Make a backup of your actual installation

sudo mv /opt/heavyai /opt/heavyai_backup

Download and Install the latest version following the install documentation for your Operative System CentOS/RHEL and Ubuntu

When the upgrade is complete, start the HEAVY.AI services.

sudo systemctl start heavydb heavy_web_server

Upgrading from Omnisci to HEAVY.AI 6.0

This section is giving a recipe to upgrade from Omnisci platform 5.5+ to HEAVY.AI 6.0.

If the version of Omnisci is older than 5.5 an intermediate upgrade step to the 5.5 version is needed. Check the docs on how to do the upgrade.

Considerations when Upgrading from Omnisci to HEAVY.AI Platform

If you are upgrading from Omnisci to HEAVY.AI, there are a lot of additional steps compared to a simple sub-version upgrade.

Before Upgrading to Release 6.0

IMPORTANT - Before you begin, stop all the running services / docker images of your Omnisci installation and create a backup $OMNISCI_STORAGE folder (typically /var/lib/omnisci). A backup is essential for recoverability; do not proceed with the upgrade without confirming that a full and consistent backup is available and ready to be restored.

The omnisci the database will not be automatically renamed to the new default name heavyai.This will be done manually and it's documented in the upgrade steps.

All the dumps created with the dump command on Omnisci cannot be restored after the database is upgraded to this version.

Essential Changes for release 6.0 of HEAVY.AI compared to Omnisci

The following table describes the changes to environment variables, storage locations, and filenames in Release 6.0 compared to Release 5.x. Except where noted, revised storage subfolders, symlinks for old folder names, and filenames are created automatically on server start.

Change descriptions in bold require user intervention.

Description

Omnisci 5.x

HEAVY.AI 6.0

Environmental variable for storage location

$OMNISCI_STORAGE

$HEAVYAI_BASE

Default location for $HEAVYAI_BASE / $OMNISCI_STORAGE

/var/lib/omnisci

/var/lib/heavyai

Fixed location for Docker $HEAVYAI_BASE / $OMNISCI_STORAGE

/omnisci-storage

/var/lib/heavyai

The folder containing catalogs for $HEAVYAI_BASE / $OMNISCI_STORAGE

data/

storage/

Storage subfolder - data

data/mapd_data

storage/data

Storage subfolder - catalog

data/mapd_catalogs

storage/catalogs

Storage subfolder - import

data/mapd_import

storage/import

Storage subfolder - export

data/mapd_export

storage/export

Storage subfolder - logs

data/mapd_log

storage/log

Server INFO logs

omnisci_server.INFO

heavydb.INFO

Server ERROR logs

omnisci_server.ERROR

heavydb.ERROR

Server WARNING logs

omnisci_server.WARNING

heavydb.WARNING

Web Server ACCESS logs

omnisci_web_server.ACCESS

heavy_web_server.ACCESS

Web Server ALL logs

omnisci_web_server.ALL

heavy_web_server.ALL

Install directory

/omnisci (Docker) /opt/omnisci (bare metal)

/opt/heavyai/ (Docker and bare metal)

Binary file - core server (located in install directory)

bin/omnsici_server

bin/heavydb

Binary file - web server (located in install directory)

bin/omnisci_web_server

bin/heavy_web_server

Binary file - command- line SQL utility

bin/omnisql

bin/heavysql

Binary file - JDBC jar

bin/omnisci-jdbc-5.10.2-SNAPSHOT.jar

bin/heavydb-jdbc-6.0.0-SNAPSHOT.jar

Binary file - Utilities (SqlImporter) jar

bin/omnisci-utility-5.10.2-SNAPSHOT.jar

bin/heavydb-utility-6.0.0-SNAPSHOT.jar

HEAVY.AI Server service (for bare metal install)

omnisci_server

heavydb

HEAVY.AI Web Server service (for bare metal install)

omnisci_web_server

heavy_web_server

Default configuration file

omnisci.conf

heavy.conf

Upgrade Instructions

The order of these instructions is significant. To avoid problems, follow the order of the instruction provided and don't skip any step.

Assumptions

This upgrade procedure is assuming that you are using the default storage location for both Omnisci and HEAVY.AI.

$OMNISCI_STORAGE

$HEAVYAI_BASE

/var/lib/omnisci

/var/lib/heavyai

Upgrading Using Docker

Stop all containers running Omnisci services.

In a terminal window, get the Docker container IDs:

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep omnisci\/

You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

Stop the HEAVY.AI Docker container. For example:

sudo docker container stop 9e01e520c3

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zxvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme.

sudo mv /var/lib/omnisci /var/lib/heavyai
sudo mv /var/lib/heavyai/data /var/lib/heavyai/storage

Create a new configuration file for heavydb changing the data parameter to point to the renamed data directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" 
>/var/lib/heavyai/heavy.conf

Rename the Omnisci license file (EE and FREE only).

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Download and run the 6.0 version of the HEAVY.AI Docker image.

Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/heavyai-ee-cpu:v6.0.0

sudo docker run -d --gpus=all \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cuda:v6.0.0

sudo docker run -d \
-v /var/lib/heavyai:/var/lib/heavyai \
-p 6273-6278:6273-6278 \
heavyai/core-os-cpu:v6.0.0

Check that Docker is up and running using a docker ps command:

sudo docker container ps --format "{{.Id}} {{.Image}} {{.Status}}" \
-f status=running | grep heavyai\/

You should see output similar to the following:

9e01e520c30c heavyai/heavyai-ee-cuda Up 48 seconds ago

Using the new container ID rename the default omnisci database to heavyai:

sudo docker exec -it 9e01e520c30c \
echo "alter database omnisci rename to heavyai;" \
| bin/heavysql omnisci

Check that everything is running as expected.

Upgrading to HEAVY.AI Using Package Managers or Tarball

To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data.

Back up the Omnisci Database

Stop the Omnisci services.

sudo systemctl stop omnisci_web_server omnisci_server

Backup the Omnisci data directory (typically /var/lib/omnisci).

tar zcvf /backup_dir/omnisci_storage_backup.tar.gz /var/lib/omnisci

Create a user named heavyai who will be the owner of the HEAVY.AI software and data on the filesystem.

sudo useradd --shell /bin/bash --user-group --create-home --group wheel heavyai

sudo useradd --shell /bin/bash --user-group --create-home --group sudo heavyai

Set a password for the user. It'll need when sudo-ing.

sudo passwd heavyai

sudo su - heavyai

Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme and change the ownership to heavyai user.

sudo chown -R heavyai:heavyai /var/lib/omnisci

sudo mv /var/lib/omnisci /var/lib/heavyai

mv /var/lib/heavyai/data /var/lib/heavyai/storage

Create the "semaphore" catalog directory; we'll have to remove it later "

mkdir /var/lib/heavyai/storage/catalogs

Check that everything is in order and that the "semaphore" directory is created,

ls -la /var/lib/heavyai/storage/

All the directories must belong to the heavyai user, and the directory catalogs would be present

total 32
drwxr-xr-x  8 heavyai heavyai 4096 lug 15 16:03 .
drwxr-xr-x  4 heavyai heavyai 4096 lug 15 16:02 ..
drwxrwxr-x  2 heavyai heavyai 4096 lug 15 16:03 catalogs
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_catalogs
drwxr-xr-x 52 heavyai heavyai 4096 lug 15 15:54 mapd_data
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_export
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 mapd_log
drwxr-xr-x  2 heavyai heavyai 4096 lug 15 15:54 omnisci_disk_cache
-rw-r--r--  1 heavyai heavyai 1229 lug 15 16:07 omnisci-licence

Rename the license file. (EE and FREE only)

mv /var/lib/heavyai/storage/omnisci.license \
/var/lib/heavyai/storage/heavyai.license

Install the HEAVY.AI Software

Install the HEAVY.AI software following all the instructions for your Operative System. CentOS/RHEL and Ubuntu.

Please follow all the installation and configuration steps until the Initialization step.

Update the configuration file and rename the default database

sudo systemctl stop heavy_web_server heavydb

Create a new configuration file for heavydb, changing the data parameter to point to the /var/lib/heavyai/storage directory and the frontend to the new install directory.

cat /var/lib/heavyai/omnisci.conf | \
sed "s/^\(data.*=.*\)/#\1\\ndata = \"\/var\/lib\/heavyai\/storage\"/" | \
sed "s/^\(frontend.*=.*\)/#\1\\nfrontend = \"\/opt\/heavyai\/frontend\"/" \
>/var/lib/heavyai/heavy.conf

All the settings of the upgraded database will be moved to the new configuration file.

Now we have to complete the database migration.

Remove the "semaphore" directory we previously created. (this is a fundamental step needed for the omnsci to heavydb upgrade)

rmdir /var/lib/heavyai/storage/catalogs

To complete the upgrade, start the HEAVY.AI servers.

sudo systemctl start heavydb heavy_web_server

Check if the database migrated, running this command and checking for the Rebrand migration complete message.

sudo systemctl status heavydb

Rename the default omnisci database to heavyai. Run the command using an administrative user (typically admin) with his password (default HyperInteractive)

echo "alter database omnisci rename to heavyai;" \
| /opt/heavyai/bin/heavysql -p HyperInteractive -u admin omnisci

Restart the database service and check that everything is running as expected.

Remove Omnisci Software from the System

After all the checks confirmed that the upgraded system is stable, clean up the system to remove the Omnisci install and relative system configuration. Remove permanently the configuration of the services.

sudo rm /lib/systemd/omnisci_server*.service
sudo rm /lib/systemd/omnisci_web_server*.service
sudo systemctl daemon-reload
sudo systemctl reset-failed

Remove the installed software.

sudo rm -Rf /opt/omnisci

Delete the YUM or APT repositories.

sudo rm /etc/yum.repos.d/omnisci.repo

sudo rm /etc/apt/sources.list.d/omnisci.list

CUDA Compatibility Drivers

This procedure is considered experimental.

In some situations, you might not be able to upgrade NVIDIA CUDA drivers on a regular basis. To work around this issue, NVIDIA provides compatibility drivers that allow users to use newer features without requiring a full upgrade. For information about compatibility drivers, see https://docs.nvidia.com/deploy/cuda-compatibility/index.html.

Installing the Drivers

Use the following commands to install the CUDA 11 compatibility drivers on Ubuntu:

wget 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin

mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 

apt-key adv --fetch-keys 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub

add-apt-repository "deb 
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" 

apt update 

nvidia-smi 

apt install cuda-compat-11-0 

nvidia-smi 

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.0/compat/ 

nvidia-smi

After the last nvidia-smi, ensure that CUDA shows the correct version.

The driver version will still show as the old version.

Updating systemd Files

After installing the drivers, update the systemd files in /lib/systemd/system/heavydb.service.

In the service section, add or update the environment property

Environment=LD_LIBRARY_PATH=/usr/local/cuda-11.0/compat:$LD_LIBRARY_PATHbash

The file should look like that

[Unit] 
Description=HEAVY.AI database server 
After=network.target remote-fs.target

[Service] 
Environment=LD_LIBRARY_PATH=/usr/local/cuda-11.0/compat:$LD_LIBRARY_PATH
User=heavyai 
Group=heavyai 
WorkingDirectory=/opt/heavyai
ExecStart=/opt/heavyai/bin/heavydb --config /var/lib/heavyai/heavy.conf 
KillMode=control-group 
SuccessExitStatus=143 
LimitNOFILE=65536 
Restart=always

[Install] 
WantedBy=multi-user.target

Then force the reload of the systemd configuration

// Some code

Uninstalling

This is a recipe to permanently remove HEAVY.AI Software, services, and data from your system.

Uninstalling HEAVY.AI from Docker

To uninstall HEAVY.AI in Docker, stop and delete the current Docker container.

In a terminal window, get the Docker container ID:

sudo docker container ps --format "{{.Id}} {{.Image}}" \
-f status=running | grep heavyai\/

You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:

9e01e520c30c omnisci/omnisci-ee-gpu

To see all containers, both running and stopped, use the following command:

sudo docker container ps -a

Stop the HEAVY.AI Docker container. For example:

sudo docker container stop 9e01e520c30c

Remove the HEAVY.AI Docker container to save disk space. For example:

sudo docker container rm 9e01e520c30c

Uninstalling HEAVY.AI on Redhat and Ubuntu

To uninstall an existing system installed with Yum, Apt, or Tarball connect using the user that runs the platform, typically heavyai.

Disable and stop all HEAVY.AI services.

sudo systemctl disable heavy_web_server --now
sudo systemctl disable heavydb --now

Remove the HEAVY.AI Installation files. (the $HEAVYAI_PATH defaults to /opt/heavyai)

sudo yum remove heavyai.x86_64

sudo apt remove heavyai

sudo rm -r $(readlink $HEAVYAI_PATH) $HEAVYAI_PATH

Delete the configuration files and the storage removing the $HEAVYAI_BASE directory. (defaults to /var/lib/heavyai)

sudo rm  -r $HEAVYAI_BASE

Remove permanently the configuration of the services.

sudo rm /lib/systemd/heavydb*.service
sudo rm /lib/systemd/heavy_web_server*.service
sudo systemctl daemon-reload
sudo systemctl reset-failed

Ports

HEAVY.AI uses the following ports.

Port

Service

Use