In this section, you will find recipes to install HEAVY.AI platform and NVIDIA drivers using package manager like apt or tarball.
This is an end-to-end recipe for installing HEAVY.AI on a Ubuntu 22.04 machine using CPU and GPU devices.
The order of these instructions is significant. To avoid problems, install each component in the order presented.
These instructions assume the following:
You are installing on a “clean” Ubuntu 22.04 host machine with only the operating system installed.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.
Prepare your Ubuntu machine by updating your system, creating the HEAVY.AI user (named heavyai), installing kernel headers, installing CUDA drivers, and optionally enabling the firewall.
Update the entire system:
sudo apt update
sudo apt upgrade
2. Install the utilities needed to create Heavy.ai repositories and download archives:
sudo apt install curl
sudo apt install libncurses5
3. Install the headless JDK and the utility apt-transport-https
:
sudo apt install default-jre-headless apt-transport-https
4. Reboot to activate the latest kernel:
sudo reboot
Create a group called heavyai
and a user named heavyai
, who will be the owner of the HEAVY.AI software and data on the filesystem.
Create the group, user, and home directory using the useradd
command with the --user-group
and --create-home
switches.
sudo useradd --user-group --create-home --group sudo heavyai
2. Set a password for the user:
sudo passwd heavyai
3. Log in with the newly created user:
sudo su - heavyai
Install the HEAVY.AI using APT and a tarball.
If your system uses NVIDIA GPUs, but the drivers not installed, install them now. See Install NVIDIA Drivers and Vulkan on Ubuntu for details.
Download and add a GPG key to APT.
curl https://releases.heavy.ai/GPG-KEY-heavyai | sudo apt-key add -
Add a source apt depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you are going to use.
echo "deb https://releases.heavy.ai/ee/apt/ stable cuda" \
| sudo tee /etc/apt/sources.list.d/heavyai.list
echo "deb https://releases.heavy.ai/ee/apt/ stable cpu" \
| sudo tee /etc/apt/sources.list.d/heavyai.list
echo "deb https://releases.heavy.ai/os/apt/ stable cuda" \
| sudo tee /etc/apt/sources.list.d/heavyai.list
echo "deb https://releases.heavy.ai/os/apt/ stable cpu" \
| sudo tee /etc/apt/sources.list.d/heavyai.list
Use apt
to install the latest version of HEAVY.AI.
sudo apt update
sudo apt install heavyai
First create the installation directory.
sudo mkdir /opt/heavyai && sudo chown $USER /opt/heavyai
Download the archive and install the software. A different archive is downloaded depending on the Edition (Enterprise, Free, or Open Source) and the device used for runtime (GPU or CPU).
curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-render.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai
curl \
https://releases.heavy.ai/ee/tar/heavyai-ee-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai
curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai
curl \
https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64-cpu.tar.gz \
| sudo tar zxf - --strip-components=1 -C /opt/heavyai
Follow these steps to prepare your HEAVY.AI environment.
For convenience, you can update .bashrc with these environment variables
echo "# HEAVY.AI variable and paths
export HEAVYAI_PATH=/opt/heavyai
export HEAVYAI_BASE=/var/lib/heavyai
export HEAVYAI_LOG=$HEAVYAI_BASE/storage/log
export PATH=$HEAVYAI_PATH/bin:$PATH" \
>> ~/.bashrc
source ~/.bashrc
Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain respectively the paths where configuration, license, and data files are stored and where the software is installed. Setting them is strongly recommended.
Run the systemd
installer to create heavyai services, a minimal config file, and initialize the data storage.
cd $HEAVYAI_PATH/systemd
./install_heavy_systemd.sh
Accept the default values provided or make changes as needed.
The script creates a data directory in $HEAVYAI_BASE/storage
(default /var/lib/heavyai/storage
) with the directories catalogs
, data
, export
and log
.The import
directory is created when you insert data the first time. If you are HEAVY.AI administrator, the log
directory is of particular interest.
Start and use HeavyDB and Heavy Immerse. ¹
Heavy Immerse is not available in the OSS Edition, so if running the OSS Edition the systemctl
command using the heavy_web_server
has no effect.
Enable the automatic startup of the service at reboot and start the HEAVY.AI services.
sudo systemctl enable heavydb --now
sudo systemctl enable heavy_web_server --now
sudo systemctl enable heavydb --now
If a firewall is not already installed and you want to harden your system, install theufw
.
sudo apt install ufw
sudo ufw allow ssh
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.
sudo ufw disable
sudo ufw allow 6273:6278/tcp
sudo ufw enable
For more information, see https://help.ubuntu.com/lts/serverguide/firewall.html.
If you are using Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key.
Skip this section if you are on Open Source Edition ²
Copy your license key of Enterprise or Free Edition from the registration email message. If you do not have a license and you want to evaluate HEAVI.AI in an unlimited
enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition here. If you need a Free License you can get one here.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273
.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin
) and password (HyperInteractive
), and then click Connect.
.
To verify that everything is working, load some sample data, perform a heavysql
query, and generate a Pointmap using Heavy Immerse ¹
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
cd $HEAVYAI_PATH
sudo ./insert_sample_data --data /var/lib/heavyai/storage
# Enter dataset number to download, or 'q' to quit:
Dataset Rows Table Name File Name
1) Flights (2008) 7M flights_2008_7M flights_2008_7M.tar.gz
2) Flights (2008) 10k flights_2008_10k flights_2008_10k.tar.gz
3) NYC Tree Census (2015) 683k nyc_trees_2015_683k nyc_trees_2015_683k.tar.gz
Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):
$HEAVYAI_PATH/bin/heavysql
password: ••••••••••••••••
Enter a SQL query such as the following
SELECT origin_city AS "Origin",
dest_city AS "Destination",
AVG(airtime) AS "Average Airtime"
FROM flights_2008_10k WHERE distance < 175
GROUP BY origin_city, dest_city;
The results should be similar to the results below.
Origin|Destination|Average Airtime
Austin|Houston|33.055556
Norfolk|Baltimore|36.071429
Ft. Myers|Orlando|28.666667
Orlando|Ft. Myers|32.583333
Houston|Austin|29.611111
Baltimore|Norfolk|31.714286
After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273
.
Log into Heavy Immerse by entering the default username (admin
) and password (HyperInteractive
), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.\
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data source.
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.\
Hardware Check
Check that your system has a compatible NVIDIA GPU. If you don't know the exact GPU model in your system run this command:
lspci -v | egrep "NVIDIA"
This command should output one row per installed NVIDIA GPU. Check that your hardware is compatible on our Hardware Requirements page. Note that you can still install the CPU Edition of HEAVY.AI on machines that do not have an NVIDIA GPU.
Pre-Installation Updates
Upgrade the system and the kernel, then the machine if needed.
sudo apt update
sudo apt upgrade -y
sudo reboot
Install kernel headers and development packages.
sudo apt install linux-headers-$(uname -r)
Install the extra packages.
sudo apt install pciutils
The rendering engine of HEAVY.AI (present in Enterprise Editions) requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself may not be able to start.
Install the Vulkan library and its dependencies using apt
.
sudo apt install libvulkan1
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
Installing NVIDIA drivers with support for the CUDA platform is required to run GPU-enabled versions of HEAVY.AI.
Each version of HEAVY.AI has a minimum required driver version, which is documented in the Software Requirements page. You can generally install NVIDIA drivers newer than the minimum required version, but the version listed in our Software Requirements page reflects the NVIDIA driver used for software testing.
You can install NVIDIA drivers in multiple ways, we've outlined three available options below - we recommend Option 1.
Option 1: Install NVIDIA drivers with CUDA toolkit from NVIDIA Website
Option 2: Install NVIDIA drivers via .run file using the NVIDIA Website
Option 3: Install NVIDIA drivers using APT package manager
CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see https://developer.nvidia.com/cuda-zone.
The CUDA Toolkit from NVIDIA provides everything you need to develop GPU-accelerated applications. The CUDA Toolkit includes GPU-accelerated libraries, a compiler, development tools and the CUDA runtime. The CUDA Toolkit is not required to run HEAVY.AI, but you must install the CUDA toolkit if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
Open https://developer.nvidia.com/cuda-toolkit-archive and select the desired CUDA Toolkit version to install.
In the "Target Platform" section, follow these steps:
For "Operating System" select Linux
For Architecture" select x86_64
For "Distribution" select Ubuntu
For "Version" select the version of your operating system (20.04)
For "Installer Type" choose deb (network) **
One by one, run the presented commands in the Installer Instructions section on your server.
Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website (https://www.nvidia.com/download/index.aspx).
If you don't know the exact GPU model in your system run this command
lspci -v | egrep "3D|VGA*.NVIDIA" | awk -F '\[|\]' ' { print $2 } '
You'll get an output in the format Product Type, Series and Model
Tesla T4
In this example, the Product type is Tesla the Series is T (as Turing), and the model is T4.
Select the Product Type as the one you got with the command.
Select the correct Product Series and Product Type for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download
On the subsequent page, if you agree to the terms, right click on "Agree and Download" and select "Copy Link Address". You may also manually download and transfer to your server, skipping the next step.
On your server, type wget
and paste the URL you copied in the previous step. Press enter to download.
Install the tools needed for installation.
sudo apt install build-essential
Change the permissions of the downloaded .run file to allow execution, and run the installation.
chmod +x NVIDIA-Linux-x86_64-*.run
sudo ./NVIDIA-Linux--x86_64-*.run
Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the apt
package manager.
Run the command to get a list of the available driver's version
apt list nvidia-driver-*
Listing... Done
nvidia-driver-450/bionic-updates,bionic-security 460.91.03-0ubuntu0.18.04.1 amd64
nvidia-driver-450-server/bionic-updates,bionic-security 450.172.01-0ubuntu0.18.04.1 amd64
nvidia-driver-455/bionic-updates,bionic-security 460.91.03-0ubuntu0.18.04.1 amd64
nvidia-driver-460/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-465/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-470/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-470-server/bionic-updates,bionic-security 470.103.01-0ubuntu0.18.04.1 amd64
nvidia-driver-495/bionic-updates,bionic-security 510.60.02-0ubuntu0.18.04.1 amd64
nvidia-driver-510/bionic-updates,bionic-security 510.60.02-0ubuntu0.18.04.1 amd64
nvidia-driver-510-server/bionic-updates,bionic-security 510.47.03-0ubuntu0.18.04.1 amd64
Install the driver version needed with apt
sudo apt install nvidia-driver-<version>
Reboot your system to ensure the new version of the driver is loaded
sudo reboot
Run nvidia-smi
to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see something like this to confirm that your NVIDIA GPUs and drivers are present.
The rendering engine of HEAVY.AI requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself can't even start without disabling the back-end renderer.
Install the Vulkan library and its dependencies using apt
.
sudo apt install libvulkan1
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
You must install the CUDA toolkit and Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
If you installed NVIDIA drivers using Option 1 above, the CUDA toolkit is already installed; you may proceed to the verification step below.
Install the NVIDIA public repository GPG key.
distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
sudo apt-key adv --fetch-keys \
https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/3bf863cc.pub
Add the repository.
echo "deb http://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64 /" \
| sudo tee /etc/apt/sources.list.d/cuda.list
apt update
List the available Cuda toolkit versions.
apt list cuda-toolkit-* | grep -v config
Listing...
cuda-toolkit-10-0/unknown 10.0.130-1 amd64
cuda-toolkit-10-1/unknown 10.1.243-1 amd64
cuda-toolkit-10-2/unknown 10.2.89-1 amd64
cuda-toolkit-11-0/unknown 11.0.3-1 amd64
cuda-toolkit-11-1/unknown 11.1.1-1 amd64
cuda-toolkit-11-2/unknown 11.2.2-1 amd64
cuda-toolkit-11-3/unknown 11.3.1-1 amd64
cuda-toolkit-11-4/unknown 11.4.4-1 amd64
cuda-toolkit-11-5/unknown 11.5.2-1 amd64
cuda-toolkit-11-6/unknown 11.6.2-1 amd64
cuda-toolkit-11-7/unknown 11.7.0-1 amd64
Install the CUDA toolkit using apt
.
sudo apt install cuda-toolkit-<version>
Check that everything is working and the toolkit has been installed.
/usr/local/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0
You must install Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities. Install Clang and LLVM dependencies using apt
.
sudo apt install clang
Check that the software is installed and in the execution path.
clang --version
clang version 10.0.0-4ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
For more information, see C++ User-Defined Functions.