Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
In this section, you will find recipes to install HEAVY.AI platform and NVIDIA drivers using package manager like yum or tarball.
HEAVY.AI Free is a full-featured version of the HEAVY.AI platform available at no cost for non-hosted commercial use.
To get started with HEAVY.AI Free:
Go to the Get Started with HEAVY.AI, and in the HEAVY.AI Free section, click Get Free License.
On the Get HEAVY.AI Free page, enter your email address and click I Agree.
Open the HEAVY.AI Free Edition Activation Link email that you receive from HEAVY.AI, and click Click Here to view and download the free edition license. You will need this license to run HEAVY.AI after you install it. A copy of the license is also sent to your email.
In the What's Next section, click See Install Options to select the best version of HEAVY.AI for your hardware and software configuration. Follow the instructions for the download or cloud version you choose.
Install HEAVY.AI, using the instructions for your platform.
Verify that OmniSci is working correctly by following the instructions in the Checkpoint section at the end of the installation instructions. For example, the Checkpoint instructions for the CentOS CPU with Tarball installation is here.
You can create additional HEAVY.AI users to collaborate with.
Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Open the SQL Editor.
Use the CREATE USER command to create a new user. For information on syntax and options, see CREATE USER.
This is an end-to-end recipe for installing HEAVY.AI on a CentOS/RHEL 7 machine using CPU and GPU devices.
The order of these instructions is significant. To avoid problems, install each component in the order presented.
These instructions assume the following:
You are installing on a “clean” CentOS/RHEL 7 host machine with only the operating system installed.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.
Prepare your Centos/RHEL machine by updating your system and optionally enabling or configuring a firewall.
Update the entire system and reboot the system if needed.
Install the utilities needed to create HEAVY.AI repositories and download archives
Open a terminal on the host machine.
Install the headless JDK using the following command:
Create a group called heavyai and a user named heavyai, who will own HEAVY.AI software and data on the file system.
You can create the group, user, and home directory using the useradd command with the --user-group and --create-home switches:
Set a password for the user:
Log in with the newly created user:
Install HEAVY.AI using yum or a tarball.
The installation using the yum package manager is recommended to those who want a more automated install and upgrade procedure.
If your system includes NVIDIA GPUs, but the drivers are not installed, install them now.
Create a yum repository depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you are going to use.
Add the GPG-key to the newly added repository.
Use yum to install the latest version of HEAVY.AI.
If you need to install a specific version of HEAVY.AI because you are upgrading from Omnisci, or for different reasons, you must run the following command:
First create the installation directory.
Download the archive and install the latest version of the software. A different archive is downloaded depending on the edition (Enterprise, Free, or Open Source) and the device used for runtime.
Follow these steps to prepare your HEAVY.AI environment.
For your convenience, you can update .bashrc with these environment variables
Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain respectively the paths where configuration, license, and data files are stored and where the software is installed. Installing them is strongly recommended.
Run the systemd installer to initialize the HEAVY.AI services and the database storage.
Accept the default values provided or make changes as needed.
The script creates a data directory in $HEAVYAI_BASE/storage (typically /var/lib/heavyai) with the directories catalogs, data, export and log. The directory import is created when you insert data the first time. If you are a HeavyDB administrator, the log directory is of particular interest.
Note that Heavy Immerse is not available in the OS SEdition, so if running the OSS Edition the systemctl command using the heavy_web_server has no effect.
Enable the automatic startup of the service at reboot and start the HEAVY.AI services.
If a firewall is not already installed and you want to harden your system, install and start firewalld.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access:
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
Open a terminal window.
Enter cd ~/ to go to your home directory.
Open .bashrc in a text editor. For example, vi .bashrc.
Edit the .bashrc file. Add the following export commands under “User specific aliases and functions.”
Save the .bashrc file. For example, in vi enter[esc]:x!
Open a new terminal window to use your changes.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
The $HEAVYAI_BASE directory must be dedicated to HEAVYAI; do not set it to a directory shared by other packages.
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):
Enter a SQL query such as the following:
The results should be similar to the results below.
After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data sour
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
Follow these instructions to install a headless JDK and configure an environment variable with a path to the library. The “headless” Java Development Kit does not provide support for keyboard, mouse, or display systems. It has fewer dependencies and is best suited for a server host. For more information, see .
See for details.
Start and use HeavyDB and Heavy Immerse.
For more information, see .
If you are on Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key. You can skip this section if you are using Open Source Edition.
Copy your license key from the registration email message. If you have not received your license key, contact your Sales Representative or register for your 30-day trial .
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Pointmap using Heavy Immerse.
In this section, you will find recipes to install HEAVY.AI platform and NVIDIA drivers using package manager like apt or tarball.
Installing OmniSci on Docker
In this section you will find the recipes to install HEAVY.AI platform using Dcoker.
You can download HEAVY.AI for your preferred platform from https://www.heavy.ai/platform/downloads/.
The CPU (no GPUs) install does not support backend rendering. For example, Pointmap and Scatterplot charts are not available. The GPU install supports all chart types.
The Open Source options do not require a license, and do not include Heavy Immerse.
Getting Started with AWS AMI
You can use the HEAVY.AI AWS AMI (Amazon Web Services Amazon Machine Image) to try HeavyDB and Heavy Immerse in the cloud. Perform visual analytics with the included New York Taxi database, or import and explore your own data.
Many options are available when deploying an AWS AMI. These instructions skip to the specific tasks you must perform to deploy a sample environment.
You need a security key pair when you launch your HEAVY.AI instance. If you do not have one, create one before you continue.
Go to the EC2 Dashboard.
Select Key Pairs under Network & Security.
Click Create Key Pair.
Enter a name for your key pair. For example, MyKey.
Click Create. The key pair PEM file downloads to your local machine. For example, you would find MyKey.pem in your Downloads directory.
Go to the AWS Marketplace page for HEAVY.AI and select the version you want to use. You can get overview information about the product, see pricing, and get usage and support information.
Click Continue to Subscribe to subscribe.
Read the Terms and Conditions, and then click Continue to Configuration.
Select the Fulfillment Option, Software Version, and Region.
Click Continue to Launch.
On the Launch this software page, select Launch through EC2, and then click Launch.
From the Choose and Instance Type page, select an available EC2 instance type, and click Review and Launch.
Review the instance launch details, and click Launch.
Select a key pair, or click Create a key pair to create a new key pair and download it, and then click Launch Instances.
On the Launch Status page, click the instance name to see it on your EC2 Dashboard Instances page.
To connect to Heavy Immerse, you need your Public IP address and Instance ID for the instance you created. You can find these values on the Description tab for your instance.
To connect to Heavy Immerse:
Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
If you receive an error message stating that the connection is not private, follow the prompts onscreen to click through to the unsecured website. To secure your site, see Tips for Securing Your EC2 Instance.
Enter the USERNAME (admin), PASSWORD ( {Instance ID} ), and DATABASE (heavyai). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
On the Dashboards page, click NYC Taxi Rides. Explore and filter the chart information on the NYC Taxis Dashboard.
For more information on Heavy Immerse features, see Introduction to Heavy Immerse.
Working with your own familiar dataset makes it easier to see the advantages of HEAVY.AI processing speed and data visualization.
To import your own data to Heavy Immerse:
Export your data from your current datastore as a comma-separated value (CSV) or tab-separated value (TSV) file. HEAVY.AI supports Latin-1 ASCII format and UTF-8. If you want to load data with another encoding (for example, UTF-16), convert the data to UTF-8 before loading it to HEAVY.AI.
Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
Enter the USERNAME (admin) and PASSWORD ( {instance ID} ). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
Click Data Manager, and then click Import Data.
Drag your data file onto the table importer page, or use the directory selector.
Click Import Files.
Verify the column names and datatypes. Edit them if needed.
Enter a Name for your table.
Click Save Table.
Click Connect to Table.
On the New Dashboard page, click Add Chart.
Choose a chart type.
Add dimensions and measures as required.
Click Apply.
Enter a Name for your dashboard.
Click Save.
For more information, see Loading Data.
Follow these instructions to connect to your instance using SSH from MacOS or Linux. For information on connecting from Windows, see Connecting to Your Linux Instance from Windows Using PuTTY.
Open a terminal window.
Locate your private key file (for example, MyKey.pem). The wizard automatically detects the key you used to launch the instance.
Your key must not be publicly viewable for SSH to work. Use this command to change permissions, if needed:
Connect to your instance using its Public DNS. The default user name is centos or ubuntu, depending on the version you are using. For example:
Use the following command to run the heavysql SQL command-line utility on HeavyDB. The default user is admin and the default password is { Instance ID }:
For more information, see heavysql.
This is an end-to-end recipe for installing HEAVY.AI on a Ubuntu 20.04 machine using CPU and GPU devices.
The order of these instructions is significant. To avoid problems, install each component in the order presented.
These instructions assume the following:
You are installing on a “clean” Ubuntu 20.04 host machine with only the operating system installed.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.
Prepare your Ubuntu machine by updating your system, creating the HEAVY.AI user (named heavyai), installing kernel headers, installing CUDA drivers, and optionally enabling the firewall.
Update the entire system:
2. Install the utilities needed to create Heavy.ai repositories and download archives:
3. Install the headless JDK and the utility apt-transport-https:
4. Reboot to activate the latest kernel:
Create a group called heavyai and a user named heavyai, who will be the owner of the HEAVY.AI software and data on the filesystem.
Create the group, user, and home directory using the useradd command with the --user-group and --create-home switches.
2. Set a password for the user:
3. Log in with the newly created user:
Install the HEAVY.AI using APT and a tarball.
The installation using the APT package manager is recommended to those who want a more automated install and upgrade procedure.
If your system uses NVIDIA GPUs, but the drivers not installed, install them now. See Install NVIDIA Drivers and Vulkan on Ubuntu for details.
Download and add a GPG key to APT.
Add a source apt depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you are going to use.
Use apt to install the latest version of HEAVY.AI.
If you need to install a specific version of HEAVY.AI, because you are upgrading from Omnisci or for different reasons, you must run the following command:
First create the installation directory.
Download the archive and install the software. A different archive is downloaded depending on the Edition (Enterprise, Free, or Open Source) and the device used for runtime (GPU or CPU).
Follow these steps to prepare your HEAVY.AI environment.
For convenience, you can update .bashrc with these environment variables
Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain respectively the paths where configuration, license, and data files are stored and where the software is installed. Setting them is strongly recommended.
Run the systemd installer to create heavyai services, a minimal config file, and initialize the data storage.
Accept the default values provided or make changes as needed.
The script creates a data directory in $HEAVYAI_BASE/storage (default /var/lib/heavyai/storage) with the directories catalogs, data, export and log.The import directory is created when you insert data the first time. If you are HEAVY.AI administrator, the log directory is of particular interest.
Start and use HeavyDB and Heavy Immerse. ¹
Heavy Immerse is not available in the OSS Edition, so if running the OSS Edition the systemctl command using the heavy_web_server has no effect.
Enable the automatic startup of the service at reboot and start the HEAVY.AI services.
If a firewall is not already installed and you want to harden your system, install theufw.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
For more information, see https://help.ubuntu.com/lts/serverguide/firewall.html.
If you are using Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key.
Skip this section if you are on Open Source Edition ²
Copy your license key of Enterprise or Free Edition from the registration email message. If you do not have a license and you want to evaluate HEAVI.AI in an unlimited
enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition here. If you need a Free License you can get one here.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
.
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Pointmap using Heavy Immerse ¹
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):
Enter a SQL query such as the following
The results should be similar to the results below.
After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data source.
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
Upgrade the system and the kernel, then the machine if needed.
Install kernel headers and development packages.
Install the extra packages.
The rendering engine of HEAVY.AI (present in Enterprise Editions) requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself may not be able to start.
Install the Vulkan library and its dependencies using apt.
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
Installing NVIDIA drivers with support for the CUDA platform is required to run GPU-enabled versions of HEAVY.AI.
You can install NVIDIA drivers in multiple ways, we've outlined three available options below. If you would prefer not to decide, we recommend Option 1.
Option 1: Install NVIDIA drivers with CUDA toolkit from NVIDIA Website
Option 2: Install NVIDIA drivers via .run file using the NVIDIA Website
Option 3: Install NVIDIA drivers using APT package manager
It is advisable to keep a record of the installation method used, as upgrading NVIDIA drivers will require the utilization of the same method for successful results.
CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see https://developer.nvidia.com/cuda-zone.
The CUDA Toolkit from NVIDIA provides everything you need to develop GPU-accelerated applications. The CUDA Toolkit includes GPU-accelerated libraries, a compiler, development tools and the CUDA runtime. The CUDA Toolkit is not required to run HEAVY.AI, but you must install the CUDA toolkit if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
Open https://developer.nvidia.com/cuda-toolkit-archive and select the desired CUDA Toolkit version to install.
The minimum CUDA version supported by HEAVY.AI is 11.4. We recommend using a release that has been available for at least two months.
In the "Target Platform" section, follow these steps:
For "Operating System" select Linux
For Architecture" select x86_64
For "Distribution" select Ubuntu
For "Version" select the version of your operating system (20.04)
For "Installer Type" choose deb (network) **
One by one, run the presented commands in the Installer Instructions section on your server.
** You may optionally use any of the "Installer Type" options available.
If you choose to use the .run file option, prior to running the installer you will need to manually install build-essentials using apt and change permissions of the downloaded .run file to allow execution.
Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website (https://www.nvidia.com/download/index.aspx).
If you don't know the exact GPU model in your system run this command
You'll get an output in the format Product Type, Series and Model
In this example, the Product type is Tesla the Series is T (as Turing), and the model is T4.
Select the Product Type as the one you got with the command.
Select the correct Product Series and Product Type for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download
On the subsequent page, if you agree to the terms, right click on "Agree and Download" and select "Copy Link Address". You may also manually download and transfer to your server, skipping the next step.
On your server, type wget and paste the URL you copied in the previous step. Press enter to download.
Please check that the driver's version you are downloading meets the HEAVI.AI minimum requirements
Install the tools needed for installation.
Change the permissions of the downloaded .run file to allow execution, and run the installation.
Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the apt package manager.
Be careful when choosing the driver version to install. Ensure that your GPU's model is supported and that meets the HEAVI.AI minimum requirements
Run the command to get a list of the available driver's version
Install the driver version needed with apt
Reboot your system to ensure the new version of the driver is loaded
Run nvidia-smi to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see something like this to confirm that your NVIDIA GPUs and drivers are present.
If you see an error like the following, the NVIDIA drivers are probably installed incorrectly:
Review the installation instructions, specifically checking for completion of install prerequisites, and correct any errors.
The rendering engine of HEAVY.AI requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself can't even start without disabling the back-end renderer.
Install the Vulkan library and its dependencies using apt.
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
You must install the CUDA toolkit and Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
If you installed NVIDIA drivers using Option 1 above, the CUDA toolkit is already installed; you may proceed to the verification step below.
Install the NVIDIA public repository GPG key.
Add the repository.
List the available Cuda toolkit versions.
Install the CUDA toolkit using apt.
Check that everything is working and the toolkit has been installed.
You must install Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities. Install Clang and LLVM dependencies using apt.
Check that the software is installed and in the execution path.
For more information, see C++ User-Defined Functions.
Install the Extra Packages for Enterprise Linux (EPEL) repository and other packages before installing NVIDIA drivers.
For CentOS, use yum to install the epel-release package.
Use the following install command for RHEL.
RHEL-based distributions require Dynamic Kernel Module Support (DKMS) to build the GPU driver kernel modules. For more information, see https://fedoraproject.org/wiki/EPEL. Upgrade the kernel and restart the machine.
Install kernel headers and development packages:
If installing kernel headers does not work correctly, follow these steps instead:
Identify the Linux kernel you are using by issuing the uname -r command.
Use the name of the kernel (3.10.0-862.11.6.el7.x86_64 in the following code example) to install kernel headers and development packages:
Install the dependencies and extra packages:
CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see https://developer.nvidia.com/cuda-zone. You can install drivers in multiple ways. This section provides installation information using the NVIDIA website or using yum.
Although using the NVIDIA website is more time consuming and less automated, you are assured that the driver is certified for your GPU. Use this method if you are not sure which driver to install. If you prefer a more automated method and are confident that the driver is certified, you can use the package-manager method.
Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website (https://developer.nvidia.com/cuda-downloads).
If you do not know the GPU model installed on your system, run this command:
The output shows the product type, series, and model. In this example, the product type is Tesla, the series is T (as Turing), and the model is T4.
Select the product type shown after running the command above.
Select the correct product series and model for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download.
Please check that the driver's version you are downloading meets the HEAVI.AI minimum requirements.
Move the downloaded file to the server, change the permissions, and run the installation.
You might receive the following error during installation:
ERROR: The Nouveau kernel driver is currently in use by your system. This driver is incompatible with the NVIDIA driver, and must be disabled before proceeding. Please consult the NVIDIA driver README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.
If you receive this error, blacklist the Nouveau driver by editing the /etc/modprobe.d/blacklist-nouveau.conffile, adding the following lines at the end:
blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off
Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the yum package manager.
When installing the driver, ensure that your GPU model is supported and meets the HEAVI.AI minimum requirements.
Add the NVIDIA network repository to your system.
Run the available drivers for the download.
Install the driver version needed with yum.
Reboot your system to ensure that the new version of the driver is loaded.
Run nvidia-smi to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see something like this to confirm that your NVIDIA GPUs and drivers are present:
If you see an error like the following, the NVIDIA drivers are probably installed incorrectly:
Review the Install NVIDIA Drivers section and correct any errors.
To work correctly, the back-end renderer requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database cannot start without disabling the back-end renderer.
Install the Vulkan library and its dependencies using yum both CentOS and RHEL.
If installing on RHEL, you must obtain and manually install the vulkan-filesystem package manually. Perform these additional steps:
Download the rpm file
Install the rpm file
You might see a warning similar to the following:
Ignore it now; you can verify NVIDIA driver installation here.
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
You must install the CUDA Toolkit if you use advanced features like C++ User-Defined Functions or User-Defined Table Functions to extend the database capabilities.
Add the NVIDIA network repository to your system:
2. List the available CUDA Toolkit versions:
3. Install the CUDA Toolkit using yum:
4. Check that everything is working correctly:
Follow these steps to install HEAVY.AI as a Docker container on a machine running with on CPU or with supported NVIDIA GPU cards using Ubuntu as the host OS.
Prepare your host by installing Docker and if needed for your configuration NVIDIA drivers and NVIDIA runtime.
Remove any existing Docker Installs and if on GPU the legacy NVIDIA docker runtime.
Use curl to add the docker's GPG key.
Add Docker to your Apt repository.
Update your repository.
Install Docker, the command line interface, and the container runtime.
Run the following usermod command so that docker command execution does not require sudo privilege. Log out and log back in for the changes to take effect. (reccomended)
Verify your Docker installation.
For more information on Docker installation, see the Docker Installation Guide.
Install NVIDIA driver and Cuda Toolkit using Install NVIDIA Drivers and Vulkan on Ubuntu
Use curl to add Nvidia's Gpg key:
Update your sources list:
Update apt-get and install nvidia-container-runtime:
Edit /etc/docker/daemon.json to add the following, and save the changes:
Restart the Docker daemon:
Verify that docker and NVIDIA runtime work together.
If everything is working you should get the output of nvidia-smi command showing the installed GPUs in the system.
Create a directory to store data and configuration files
Then a minimal configuration file for the docker installation
Ensure that you have sufficient storage on the drive you choose for your storage dir running this command
Download HEAVY.AI from DockerHub and Start HEAVY.AI in Docker. Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are going to use.
Check that the docker is up and running a docker ps commnd:
You should see an output similar to the following.
See also the note regarding the CUDA JIT Cache in Optimizing Performance.
If a firewall is not already installed and you want to harden your system, install theufw.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
For more information, see https://help.ubuntu.com/lts/serverguide/firewall.html.
If you are on Enterprise or Free Edition, you need to validate your HEAVY.AI instance using your license key. You must skip this section if you are on Open Source Edition ²
Copy your license key of Enterprise or Free Edition from the registration email message. If you don't have a license and you want to evaluate HEAVY.AI in an enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition here. If you need a Free License you can get one here.
Connect to Heavy Immerse using a web browser to your host on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
You can access the command line in the Docker image to perform configuration and run HEAVY.AI utilities.
You need to know the container-id to access the command line. Use the command below to list the running containers.
You see output similar to the following.
Once you have your container ID, in the example 9e01e520c30c, you can access the command line using the Docker exec command. For example, here is the command to start a Bash session in the Docker instance listed above. The -it switch makes the session interactive.
You can end the Bash session with the exit command.
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Scatter Plot or a Bubble Chart using Heavy Immerse ¹
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Where <container-id> is the container in which HEAVY.AI is running.
When prompted, choose whether to insert dataset 1 (7,000,000 rows), dataset 2 (10,000 rows), or dataset 3 (683,000 rows). The examples below use dataset 2.
Connect to HeavyDB by entering the following command (a password willò be asked; the default password is HyperInteractive):
Enter a SQL query such as the following:
The results should be similar to the results below.
Installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data sour
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
Using HEAVY.AI's Helm Chart on Kubernetes
This documentation outlines how to use HEAVY.AI’s Helm Chart within a Kubernetes environment. It assumes the user is a network administrator within your organization and is an experienced Kubernetes administrator. This is not a beginner guide and does not instruct on Kubernetes installation or administration. It is quite possible you will require additional manifest files for your environment.
The HEAVY.AI Helm Chart is a template of how to configure deployment of the HEAVY.AI platform. The following files need to be updated/created to reflect the customer's deployment environment.
values.yml
<customer_created>-pv.yml
<customer_created>-pvc.yml
Once the files are updated/created, follow the installation instructions below to install the Helm Chart into your Kubernetes environment.
The Helm Chart is located in the HEAVY.AI github repository. It can be found here: https://releases.heavy.ai/ee/helm/heavyai-1.0.0.tgz
Before installing, create a PV/PVC that the deployment will use. Save these files in the regular PVC/PV location used in the customer’s environment. Reference the README.pdf file found in the Helm Chart under templates and the example PV/PVC manifests in the misc folder in the helm chart. The PVC name is then provided to the helm install command.
In your current directory; copy the values.yml file from the HEAVY.AI Helm Chart and customize for your needs.
Run the helm install command with the desired deployment name and Helm Chart.
When using a values.yml file:
$ helm install heavyai --values values.yml heavyaihelmchart-1.0.0.tgz
When not using a values.yml file:
If you only need to change a value or two from the default values.yml file you can use --set instead of a custom values.yml file.
For example:
$ helm install heavyai --set pvcName=MyPVCName heavyaihelmchart-1.0.0.tgz
To uninstall the helm installed HEAVY.AI instance:
$ helm uninstall heavyai
The PVC and PV space defined for the HEAVY.AI instance is not removed. The retained space must be manually deleted.
Getting Started with HEAVY.AI on Microsoft Azure
Follow these instructions to get started with HEAVY.AI on Microsoft Azure.
You must have a Microsoft Azure account. If you do not have an account, go to the Micrsoft Azure home page to sign up for one.
To launch HEAVY.AI on Microsoft Azure, you configure a GPU-enabled instance.
1) Log in to you Microsoft Azure portal.
2) On the left side menu, create a Resource group, or use one that your organization has created.
3) On the left side menu, click Virtual machines, and then click Add.
4) Create your virtual machine:
On the Basics tab:
In Project Details, specify the Resource group.
Specify the Instance Details:
Virtual machine name
Region
Image (Ubuntu 16.04 or higher, or CentOS/RHEL 7.0 or higher)
Size. Click Change size and use the Family filter to filter on GPU, based on your use case and requirements. Not all GPU VM variants are available in all regions.
For Username, add any user name other than admin.
In Inbound Port Rules, click Allow selected ports and select one or more of the following:
HTTP (80)
HTTPS (443)
SSH (22)
On the Disks tab, select Premium or Standard SSD, depending on your needs.
For the rest of the tabs and sections, use the default values.
5) Click Review + create. Azure reviews your entries, creates the required services, deploys them, and starts the VM.
6) Once the VM is running, select the VM you just created and click the Networking tab.
7) Click the Add inbound button and configure security rules to allow any source, any destination, and destination port 6273 so you can access Heavy Immerse from a browser on that port. Consider renaming the rule to 6273-Immerse or something similar so that the default name makes sense.
8) Click Add and verify that your new rule appears.
Azure-specific configuration is complete. Now, follow standard HEAVY.AI installation instructions for your Linux distribution and installation method.
This section is giving a recipe to upgrade between fully compatible products version.
This section is giving a recipe to upgrade between fully compatible products version.
As with any software upgrade, it is important that you back up your data before upgrading. Each release introduces efficiencies that are not necessarily compatible with earlier releases of the platform. HeavyAI is never expected to be backward compatible.
Back up the contents of your $HEAVYAI_STORAGE directory.
If you need to upgrade from Omnisci to HEAVY.AI 6.0 or later, please refer to the specific recipe.
Direct upgrades from Omnisci to HEAVY.AI version later than 6.0 aren't allowed nor supported.
To upgrade HEAVY.AI in place in Docker
In a terminal window, get the Docker container ID.
You should see output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
Stop the HEAVY.AI Docker container. For example:
Optionally, remove the HEAVY.AI Docker container. This removes unused Docker containers on your system and saves disk space.
Backup the Omnisci data directory (typically /var/lib/omnisci)
Download the latest version of the HEAVY.AI Docker image according to the Edition and device you are actually coming from Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.
If you don't want to upgrade to the latest version but want to upgrade to a specific version, change thelatesttag with the version needed.
If the version needed is the 6.0 use v6.0.0 as the version tag in the image name
heavyai/heavyai-ee-cuda:v6.0.0
Check that the docker is up and running a docker ps commnd:
You should see an output similar to the following.
This runs both the HEAVY.AI database and Immerse in the same container.
You can optionally add --rm to the Docker run command so that the container is removed when it is stopped.
See also the note regarding the CUDA JIT Cache in Optimizing Performance.
To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data
Stop the HEAVY.AI services.
Back up your $HEAVYAI_STORAGE directory (the default location is /var/lib/heavyai).
Run the appropriate set of commands depending on the method used to install the previous version of the software.
Make a backup of your actual installation
Download and Install the latest version following the install documentation for your Operative System CentOS/RHEL and Ubuntu
When the upgrade is complete, start the HEAVY.AI services.
In this section, you will find recipes to upgrade from the OmniSci to the HEAVY.AI platform and upgrade between versions of the HEAVY.AI platform.
The following table shows the steps needed to move from one version to a later one.
| Initial Version | Final Version | Upgrade Steps |
|---|---|---|
Versions 5.x and 6.0.0 are no longer supported; use these only as needed to facilitate an upgrade to a supported version.
Example: if you are running an OmniSci version older than 5.5, you must first upgrade to 5.5, then upgrade to 6.0 and after that upgrade to 7.0. If you are running 6.0 - 6.4, you can upgrade directly to 7.0 in a single step.
Getting Started with HEAVY.AI on Google Cloud Platform
Follow these instructions to get started with HEAVY.AI on Google Cloud Platform (GCP).
You must have a Google Cloud Platform account. If you do not have an account, follow these instructions to sign up for one.
To launch HEAVY.AI on Google Cloud Platform, you select and configure an instance.
On the solution Launcher Page, click Launch on Compute Engine to begin configuring your deployment.
Before deploying a solution with a GPU machine type, avoid potential deployment failure by checking your available quota for a project to make sure that you have not exceeded your limit.
To launch HEAVY.AI on Google Cloud Platform, you select and configure a GPU-enabled instance.
Search for HEAVY.AI on the heavyai-launcher-public project on Google Cloud Platform, and select a solution. HEAVY.AI has four instance types:
On the solution Launcher Page, click Launch to begin configuring your deployment.
On the new deployment page, configure the following:
Deployment name
Zone
Machine type - Click Customize and configure Cores and Memory, and select Extend memory if necessary.
GPU type. (Not applicable for CPU configurations.)
Number of GPUs - (Not applicable for CPU configurations.) Select the number of GPUs; subject to quota and GPU type by region. For more information about GPU-equipped instances and associated resources, see GPU Models for Compute Engine.
Boot disk type
Boot disk size in GB
Networking - Set the Network, Subnetwork, and External IP.
Firewall - Select the required ports to allow TCP-based connectivity to HEAVY.AI. Click More to set IP ranges for port traffic and IP forwarding.
Accept the GCP Marketplace Terms of Service and click Deploy.
In the Deployment Manager, click the instance that you deployed.
Launch the Heavy Immerse client:
Record the Admin password (Temporary).
Click the Site address link to go to the Heavy Immerse login page. Enter the password you recorded, and click Connect.
Copy your license key from the registration email message. If you have not received your license key, contact your Sales Representative or register for your 30-day trial here.
Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Click Connect to start using HEAVY.AI.
On successful login, you see a list of sample dashboards loaded into your instance.
This section is giving a recipe to upgrade from Omnisci platform 5.5+ to HEAVY.AI 6.0.
If the version of Omnisci is older than 5.5 an intermediate upgrade step to the 5.5 version is needed. Check the docs on how to do the .
If you are upgrading from Omnisci to HEAVY.AI, there are a lot of additional steps compared to a simple sub-version upgrade.
IMPORTANT - Before you begin, stop all the running services / docker images of your Omnisci installation and create a backup $OMNISCI_STORAGE folder (typically /var/lib/omnisci). A backup is essential for recoverability; do not proceed with the upgrade without confirming that a full and consistent backup is available and ready to be restored.
The omnisci the database will not be automatically renamed to the new default name heavyai.This will be done manually and it's documented in the upgrade steps.
All the dumps created with the dump command on Omnisci cannot be restored after the database is upgraded to this version.
The following table describes the changes to environment variables, storage locations, and filenames in Release 6.0 compared to Release 5.x. Except where noted, revised storage subfolders, symlinks for old folder names, and filenames are created automatically on server start.
Change descriptions in bold require user intervention.
| Description | Omnisci 5.x | HEAVY.AI 6.0 |
|---|
The order of these instructions is significant. To avoid problems, follow the order of the instruction provided and don't skip any step.
This upgrade procedure is assuming that you are using the default storage location for both Omnisci and HEAVY.AI.
Stop all containers running Omnisci services.
In a terminal window, get the Docker container IDs:
You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
Stop the HEAVY.AI Docker container. For example:
Backup the Omnisci data directory (typically /var/lib/omnisci).
Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme.
Create a new configuration file for heavydb changing the data parameter to point to the renamed data directory.
Rename the Omnisci license file (EE and FREE only).
Download and run the 6.0 version of the HEAVY.AI Docker image.
Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.
Check that Docker is up and running using a docker ps command:
You should see output similar to the following:
Using the new container ID rename the default omnisci database to heavyai:
Check that everything is running as expected.
To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data.
Stop the Omnisci services.
Backup the Omnisci data directory (typically /var/lib/omnisci).
Create a user named heavyai who will be the owner of the HEAVY.AI software and data on the filesystem.
Set a password for the user. It'll need when sudo-ing.
Login with the newly created user
Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme and change the ownership to heavyai user.
Create the "semaphore" catalog directory; we'll have to remove it later "
Check that everything is in order and that the "semaphore" directory is created,
All the directories must belong to the heavyai user, and the directory catalogs would be present
Rename the license file. (EE and FREE only)
Please follow all the installation and configuration steps until the Initialization step.
Log in with the heavyai user and ensure the heavyai services are stopped.
Create a new configuration file for heavydb, changing the data parameter to point to the /var/lib/heavyai/storage directory and the frontend to the new install directory.
All the settings of the upgraded database will be moved to the new configuration file.
Now we have to complete the database migration.
Remove the "semaphore" directory we previously created. (this is a fundamental step needed for the omnsci to heavydb upgrade)
To complete the upgrade, start the HEAVY.AI servers.
Check if the database migrated, running this command and checking for the Rebrand migration complete message.
Rename the default omnisci database to heavyai.
Run the command using an administrative user (typically admin) with his password (default HyperInteractive)
Restart the database service and check that everything is running as expected.
After all the checks confirmed that the upgraded system is stable, clean up the system to remove the Omnisci install and relative system configuration. Remove permanently the configuration of the services.
Remove the installed software.
Delete the YUM or APT repositories.
This procedure is considered experimental.
In some situations, you might not be able to upgrade NVIDIA CUDA drivers on a regular basis. To work around this issue, NVIDIA provides compatibility drivers that allow users to use newer features without requiring a full upgrade. For information about compatibility drivers, see .
Use the following commands to install the CUDA 11 compatibility drivers on Ubuntu:
After the last nvidia-smi, ensure that CUDA shows the correct version.
The driver version will still show as the old version.
After installing the drivers, update the systemd files in /lib/systemd/system/heavydb.service.
In the service section, add or update the environment property
The file should look like that
Then force the reload of the systemd configuration
HEAVY.AI uses the following ports.
This is a recipe to permanently remove HEAVY.AI Software, services, and data from your system.
To uninstall HEAVY.AI in Docker, stop and delete the current Docker container.
In a terminal window, get the Docker container ID:
You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
To see all containers, both running and stopped, use the following command:
Stop the HEAVY.AI Docker container. For example:
Remove the HEAVY.AI Docker container to save disk space. For example:
To uninstall an existing system installed with Yum, Apt, or Tarball connect using the user that runs the platform, typically heavyai.
Disable and stop all HEAVY.AI services.
Remove the HEAVY.AI Installation files. (the $HEAVYAI_PATH defaults to /opt/heavyai)
Delete the configuration files and the storage removing the $HEAVYAI_BASE directory. (defaults to /var/lib/heavyai)
Remove permanently the configuration of the services.
| File Name | Description |
|---|---|
Install the HEAVY.AI software following all the instructions for your Operative System. and .
Chart.yml
HEAVY.AI Helm Chart. Contains version and contact information.
values.yml
Copy this file and edit values specific to your HEAVY.AI deployment. This is where to note the PVC name. This file is annotated to identify typical customizations and is pre-populated with default values.
README.pdf
These instructions.
deployment.yml
HEAVY.AI platform deployment template. DO NOT EDIT
example-heavyai-pv.yml
Example PV file.
example-heavyai-pv.yml
Example PVC file.
OmniSci less than 5.5
HEAVY.AI 7.0
Upgrade to 5.5 --> 6.0 --> 7.0
OmniSci 5.5 - 5.10
HEAVY.AI 7.0
Upgrade to 6.0 --> 7.0
HEAVY.AI 6.0
HEAVY.AI 7.0
Upgrade to 7.0
$OMNISCI_STORAGE | $HEAVYAI_BASE |
/var/lib/omnisci | /var/lib/heavyai |
Port | Service | Use |
6273 | heavy_web_server | Used to access Heavy Immerse. |
6274 | heavydb tcp | Used by connectors (heavyai, omnisql, odbc, and jdbc) to access the more efficient Thrift API. |
6276 | heavy_web_server | Used to access the HTTP/JSON thrift API. |
6278 | heavydb http | Used to directly access the HTTP/binary thrift API, without having to proxy through heavy_web_server. Recommended for debugging use only. |
Environmental variable for storage location | $OMNISCI_STORAGE | $HEAVYAI_BASE |
Default location for $HEAVYAI_BASE / $OMNISCI_STORAGE | /var/lib/omnisci | /var/lib/heavyai |
Fixed location for Docker $HEAVYAI_BASE / $OMNISCI_STORAGE | /omnisci-storage | /var/lib/heavyai |
The folder containing catalogs for $HEAVYAI_BASE / $OMNISCI_STORAGE | data/ | storage/ |
Storage subfolder - data | data/mapd_data | storage/data |
Storage subfolder - catalog | data/mapd_catalogs | storage/catalogs |
Storage subfolder - import | data/mapd_import | storage/import |
Storage subfolder - export | data/mapd_export | storage/export |
Storage subfolder - logs | data/mapd_log | storage/log |
Server INFO logs | omnisci_server.INFO | heavydb.INFO |
Server ERROR logs | omnisci_server.ERROR | heavydb.ERROR |
Server WARNING logs | omnisci_server.WARNING | heavydb.WARNING |
Web Server ACCESS logs | omnisci_web_server.ACCESS | heavy_web_server.ACCESS |
Web Server ALL logs | omnisci_web_server.ALL | heavy_web_server.ALL |
Install directory | /omnisci (Docker) /opt/omnisci (bare metal) | /opt/heavyai/ (Docker and bare metal) |
Binary file - core server (located in install directory) | bin/omnsici_server | bin/heavydb |
Binary file - web server (located in install directory) | bin/omnisci_web_server | bin/heavy_web_server |
Binary file - command- line SQL utility | bin/omnisql | bin/heavysql |
Binary file - JDBC jar | bin/omnisci-jdbc-5.10.2-SNAPSHOT.jar | bin/heavydb-jdbc-6.0.0-SNAPSHOT.jar |
Binary file - Utilities (SqlImporter) jar | bin/omnisci-utility-5.10.2-SNAPSHOT.jar | bin/heavydb-utility-6.0.0-SNAPSHOT.jar |
HEAVY.AI Server service (for bare metal install) | omnisci_server | heavydb |
HEAVY.AI Web Server service (for bare metal install) | omnisci_web_server | heavy_web_server |
Default configuration file | omnisci.conf | heavy.conf |
