Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
You can download HEAVY.AI for your preferred platform from https://www.omnisci.com/platform/downloads/.
The CPU (no GPUs) install does not support backend rendering. For example, Pointmap and Scatterplot charts are not available. The GPU install supports all chart types.
The Open Source options do not require a license, and do not include Heavy Immerse.
In this section, you will find recipes to install HEAVY.AI platform and NVIDIA drivers using package manager like apt or tarball.
HEAVY.AI Free is a full-featured version of the HEAVY.AI platform available at no cost for non-hosted commercial use.
To get started with HEAVY.AI Free:
Go to the Get Started with HEAVY.AI, and in the HEAVY.AI Free section, click Get Free License.
On the Get HEAVY.AI Free page, enter your email address and click I Agree.
Open the HEAVY.AI Free Edition Activation Link email that you receive from HEAVY.AI, and click Click Here to view and download the free edition license. You will need this license to run HEAVY.AI after you install it. A copy of the license is also sent to your email.
In the What's Next section, click See Install Options to select the best version of HEAVY.AI for your hardware and software configuration. Follow the instructions for the download or cloud version you choose.
Install HEAVY.AI, using the instructions for your platform.
Verify that OmniSci is working correctly by following the instructions in the Checkpoint section at the end of the installation instructions. For example, the Checkpoint instructions for the CentOS CPU with Tarball installation is here.
You can create additional HEAVY.AI users to collaborate with.
Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Open the SQL Editor.
Use the CREATE USER command to create a new user. For information on syntax and options, see CREATE USER.
In this section you will find a recipe to install the heavy.ai platofrom on Red Hat and derivates like Rocky Linux
In this section you will find a recipe to install the heavy.ai platofrom on Red Hat and derivates like Rocky Linux
Installing OmniSci on Docker
In this section you will find the recipes to install HEAVY.AI platform using Dcoker.
Install the Extra Packages for Enterprise Linux (EPEL) repository and other packages before installing NVIDIA drivers.
RHEL-based distributions require Dynamic Kernel Module Support (DKMS) to build the GPU driver kernel modules. For more information, see https://fedoraproject.org/wiki/EPEL. Upgrade the kernel and restart the machine.
Install kernel headers and development packages:
If installing kernel headers does not work correctly, follow these steps instead:
Identify the Linux kernel you are using by issuing the uname -r command.
Use the name of the kernel (4.18.0-553.el8_10.x86_64 in the following code example) to install kernel headers and development packages:
Install the dependencies and extra packages:
CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see https://developer.nvidia.com/cuda-zone. You can install drivers in multiple ways. This section provides installation information using the NVIDIA website or using dnf.
Although using the NVIDIA website is more time-consuming and less automated, you are assured that the driver is certified for your GPU. Use this method if you are not sure which driver to install. If you prefer a more automated method and are confident that the driver is certified, you can use the DNF package manager method.
Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website (https://developer.nvidia.com/cuda-downloads).
If you do not know the GPU model installed on your system, run this command:
The output shows the product type, series, and model. In this example, the product type is Tesla, the series is T (as Turing), and the model is T4.
Select the product type shown after running the command above.
Select the correct product series and model for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download.
Please check that the driver's version you download meets the HEAVI.AI minimum requirements.
Move the downloaded file to the server, change the permissions, and run the installation.
You might receive the following error during installation:
ERROR: The Nouveau kernel driver is currently in use by your system. This driver is incompatible with the NVIDIA driver, and must be disabled before proceeding. Please consult the NVIDIA driver README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.
If you receive this error, blacklist the Nouveau driver by editing the /etc/modprobe.d/blacklist-nouveau.conffile, adding the following lines at the end:
blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off
Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the DNF package manager.
When installing the driver, ensure your GPU model is supported and meets the HEAVI.AI minimum requirements.
Add the NVIDIA network repository to your system.
Install the driver version needed with dnf. For 8.0, the minimum version is 535.
To load the installed driver, you can run sudo modprobe nvidia or nvidia-smi commands, or , in case of driver upgrade, you can reboot your system to ensure that the new version of the driver is loaded using the command sudo reboot
Run the specified command to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see output confirming the presence of your NVIDIA GPUs and drivers. This verification step ensures that your system can identify and utilize the GPUs as intended.
If you encounter an error similar to the following, the NVIDIA drivers are likely installed incorrectly:
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Please ensure that the latest NVIDIA driver is installed and running.
Please review the Install NVIDIA Drivers section and correct any errors.
The back-end renderer requires a Vulkan-enabled driver and the Vulkan library to work correctly. Without these components, the database cannot start without disabling the back-end renderer.
To ensure the Vulkan library and its dependencies are installed, use the DNF.
For more information about troubleshooting Vulkan, see the Vulkan Renderer section.
You must install the CUDA Toolkit if you use advanced features like C++ User-Defined Functions or User-Defined Table Functions to extend the database capabilities.
Add the NVIDIA network repository to your system:
2. List the available CUDA Toolkit versions using the DNF list command
3. Install the CUDA Toolkit version using DNF.
4. Check that everything is working correctly:
This is an end-to-end recipe for installing HEAVY.AI on a Ubuntu 18.04/20.04 machine using CPU and GPU devices.
The order of these instructions is significant. To avoid problems, install each component in the order presented.
These instructions assume the following:
You are installing on a “clean” Ubuntu 18.04/20.04 host machine with only the operating system installed.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.
Prepare your Ubuntu machine by updating your system, creating the HEAVY.AI user (named heavyai), installing kernel headers, installing CUDA drivers, and optionally enabling the firewall.
Update the entire system:
2. Install the utilities needed to create Heavy.ai repositories and download archives:
3. Install the headless JDK and the utility apt-transport-https:
4. Reboot to activate the latest kernel:
Create a group called heavyai and a user named heavyai, who will be the owner of the HEAVY.AI software and data on the filesystem.
Create the group, user, and home directory using the useradd command with the --user-group and --create-home switches.
2. Set a password for the user:
3. Log in with the newly created user:
Install the HEAVY.AI using APT and a tarball.
The installation using the APT package manager is recommended to those who want a more automated install and upgrade procedure.
If your system uses NVIDIA GPUs, but the drivers not installed, install them now. See Install NVIDIA Drivers and Vulkan on Ubuntu for details.
Download and add a GPG key to APT.
Add a source apt depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you are going to use.
Use apt to install the latest version of HEAVY.AI.
If you need to install a specific version of HEAVY.AI, because you are upgrading from Omnisci or for different reasons, you must run the following command:
First create the installation directory.
Download the archive and install the software. A different archive is downloaded depending on the Edition (Enterprise, Free, or Open Source) and the device used for runtime (GPU or CPU).
Follow these steps to prepare your HEAVY.AI environment.
For convenience, you can update .bashrc with these environment variables
Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain respectively the paths where configuration, license, and data files are stored and where the software is installed. Setting them is strongly recommended.
Run the systemd installer to create heavyai services, a minimal config file, and initialize the data storage.
Accept the default values provided or make changes as needed.
The script creates a data directory in $HEAVYAI_BASE/storage (default /var/lib/heavyai/storage) with the directories catalogs, data, export and log.The import directory is created when you insert data the first time. If you are HEAVY.AI administrator, the log directory is of particular interest.
Start and use HeavyDB and Heavy Immerse. ¹
Heavy Immerse is available in OS Edition, so the systemctl command using the heavy_web_server has no effect.
Enable the automatic startup of the service at reboot and start the HEAVY.AI services.
If a firewall is not already installed and you want to harden your system, install theufw.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
For more information, see https://help.ubuntu.com/lts/serverguide/firewall.html.
If you are using Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key.
Skip this section if you are on Open Source Edition ²
Copy your license key of Enterprise or Free Edition from the registration email message. If you do not have a license and you want to evaluate HEAVI.AI in an unlimited
enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition here. If you need a Free License you can get one here.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
.
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Pointmap using Heavy Immerse ¹
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):
Enter a SQL query such as the following
The results should be similar to the results below.
After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data source.
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
Getting Started with HEAVY.AI on Microsoft Azure
Follow these instructions to get started with HEAVY.AI on Microsoft Azure.
You must have a Microsoft Azure account. If you do not have an account, go to the Micrsoft Azure home page to sign up for one.
To launch HEAVY.AI on Microsoft Azure, you configure a GPU-enabled instance.
1) Log in to you Microsoft Azure portal.
2) On the left side menu, create a Resource group, or use one that your organization has created.
3) On the left side menu, click Virtual machines, and then click Add.
4) Create your virtual machine:
On the Basics tab:
In Project Details, specify the Resource group.
Specify the Instance Details:
Virtual machine name
Region
Image (Ubuntu 16.04 or higher, or CentOS/RHEL 7.0 or higher)
Size. Click Change size and use the Family filter to filter on GPU, based on your use case and requirements. Not all GPU VM variants are available in all regions.
For Username, add any user name other than admin.
In Inbound Port Rules, click Allow selected ports and select one or more of the following:
HTTP (80)
HTTPS (443)
SSH (22)
On the Disks tab, select Premium or Standard SSD, depending on your needs.
For the rest of the tabs and sections, use the default values.
5) Click Review + create. Azure reviews your entries, creates the required services, deploys them, and starts the VM.
6) Once the VM is running, select the VM you just created and click the Networking tab.
7) Click the Add inbound button and configure security rules to allow any source, any destination, and destination port 6273 so you can access Heavy Immerse from a browser on that port. Consider renaming the rule to 6273-Immerse or something similar so that the default name makes sense.
8) Click Add and verify that your new rule appears.
Azure-specific configuration is complete. Now, follow standard HEAVY.AI installation instructions for your Linux distribution and installation method.
In this section, you will find recipes to upgrade from the OmniSci to the HEAVY.AI platform and upgrade between versions of the HEAVY.AI platform.
It's not always possible to upgrade from your actual product version to the last one, but one or more intermediate upgrade steps are needed.
The following table shows the steps needed to move from one software version to another.
| Initial version | Final version | Upgrade Steps |
|---|
Versions 5.x and 6.0.0 are not currently supported; use these only as needed to facilitate an upgrade to a supported version.
As an example, if you are using an OmniSci version older than 5.5, you must upgrade to 5.5, then to 6.0 and after that to 7.0, while if you are using 6.0-6.4.4 you can upgrade to 7.0.0 in a single step.
Upgrade the system and the kernel, then the machine if needed.
Install kernel headers and development packages.
Install the extra packages.
The rendering engine of HEAVY.AI (present in Enterprise Editions) requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself may not be able to start.
Install the Vulkan library and its dependencies using apt.
For more information about troubleshooting Vulkan, see the section.
Installing NVIDIA drivers with support for the CUDA platform is required to run GPU-enabled versions of HEAVY.AI.
You can install NVIDIA drivers in multiple ways, we've outlined three available options below. If you would prefer not to decide, we recommend Option 1.
It is advisable to keep a record of the installation method used, as upgrading NVIDIA drivers will require the utilization of the same method for successful results.
The CUDA Toolkit from NVIDIA provides everything you need to develop GPU-accelerated applications. The CUDA Toolkit includes GPU-accelerated libraries, a compiler, development tools and the CUDA runtime. The CUDA Toolkit is not required to run HEAVY.AI, but you must install the CUDA toolkit if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
The minimum CUDA version supported by HEAVY.AI is 11.4. We recommend using a release that has been available for at least two months.
In the "Target Platform" section, follow these steps:
For "Operating System" select Linux
For Architecture" select x86_64
For "Distribution" select Ubuntu
For "Version" select the version of your operating system (18.04 or 20.04)
For "Installer Type" choose deb (network) **
One by one, run the presented commands in the Installer Instructions section on your server.
** You may optionally use any of the "Installer Type" options available.
If you choose to use the .run file option, prior to running the installer you will need to manually install build-essentials using apt and change permissions of the downloaded .run file to allow execution.
If you don't know the exact GPU model in your system run this command
You'll get an output in the format Product Type, Series and Model
In this example, the Product type is Tesla the Series is T (as Turing), and the model is T4.
Select the Product Type as the one you got with the command.
Select the correct Product Series and Product Type for your installation.
In the Operating System dropdown list, select Linux 64-bit.
In the CUDA Toolkit dropdown list, click a supported version (11.4 or higher).
Click Search.
On the resulting page, verify the download information and click Download
On the subsequent page, if you agree to the terms, right click on "Agree and Download" and select "Copy Link Address". You may also manually download and transfer to your server, skipping the next step.
On your server, type wget and paste the URL you copied in the previous step. Press enter to download.
Install the tools needed for installation.
Change the permissions of the downloaded .run file to allow execution, and run the installation.
Install a specific version of the driver for your GPU by installing the NVIDIA repository and using the apt package manager.
Run the command to get a list of the available driver's version
Install the driver version needed with apt
Reboot your system to ensure the new version of the driver is loaded
Run nvidia-smi to verify that your drivers are installed correctly and recognize the GPUs in your environment. Depending on your environment, you should see something like this to confirm that your NVIDIA GPUs and drivers are present.
If you see an error like the following, the NVIDIA drivers are probably installed incorrectly:
Review the installation instructions, specifically checking for completion of install prerequisites, and correct any errors.
The rendering engine of HEAVY.AI requires a Vulkan-enabled driver and the Vulkan library. Without these components, the database itself can't even start without disabling the back-end renderer.
Install the Vulkan library and its dependencies using apt.
You must install the CUDA toolkit and Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities.
Install the NVIDIA public repository GPG key.
Add the repository.
List the available Cuda toolkit versions.
Install the CUDA toolkit using apt.
Check that everything is working and the toolkit has been installed.
You must install Clang if you use advanced features like C++ User-Defined Functions and or User-Defined Table Functions to extend the database capabilities. Install Clang and LLVM dependencies using apt.
Check that the software is installed and in the execution path.
This is a recipe to permanently remove HEAVY.AI Software, services, and data from your system.
To uninstall HEAVY.AI in Docker, stop and delete the current Docker container.
In a terminal window, get the Docker container ID:
You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
To see all containers, both running and stopped, use the following command:
Stop the HEAVY.AI Docker container. For example:
Remove the HEAVY.AI Docker container to save disk space. For example:
To uninstall an existing system installed with Yum, Apt, or Tarball connect using the user that runs the platform, typically heavyai.
Disable and stop all HEAVY.AI services.
Remove the HEAVY.AI Installation files. (the $HEAVYAI_PATH defaults to /opt/heavyai)
Delete the configuration files and the storage removing the $HEAVYAI_BASE directory. (defaults to /var/lib/heavyai)
Remove permanently the configuration of the services.
This is an end-to-end recipe for installing HEAVY.AI on a Red Hat Enterprise 8.x machine using CPU and GPU devices.
The order of these instructions is significant. To avoid problems, install each component in the order presented.
The same instructions can be used to install on RL / RHEL 9, which some minor modifications.
These instructions assume the following:
You are installing a "clean" Rocky Linux / RHEL 8 host machine with only the operating system.
Your HEAVY.AI host only runs the daemons and services required to support HEAVY.AI.
Your HEAVY.AI host is connected to the Internet.
Prepare your machine by updating your system and optionally enabling or configuring a firewall.
Update the entire system and reboot the system if needed.
Install the utilities needed to create HEAVY.AI repositories and download installation binaries.
Open a terminal on the host machine.
Install the headless JDK using the following command:
Create a group called heavyai and a user named heavyai, who will own HEAVY.AI software and data on the file system.
You can create the group, user, and home directory using the useradd command with the --user-group and --create-home switches:
Set a password for the user using the passwd command.
Log in with the newly created user.
There are two ways to install the heavy.ai software
Using the DNF package manager for installation is highly recommended due to its ability to handle dependencies and streamline the installation process, making it a preferred choice for many users.
If your system includes NVIDIA GPUs but the drivers are not installed, it is advisable to install them before proceeding with the suite installation.
Create a DNF repository depending on the edition (Enterprise, Free, or Open Source) and execution device (GPU or CPU) you will use.
Add the GPG-key to the newly added repository.
Use DNF to install the latest version of HEAVY.AI.
You can use the DNF package manager to list the available packages when installing a specific version of HEAVY.AI, such as when a multistep upgrade is necessary, or a specific version is needed for any other reason.
sudodnf --showduplicateslistheavyai
Select the version needed from the list (e.g. 7.0.0) and install using the command.
sudodnfinstallheavyai-7.0.0_20230501_be4f51b048-1.x86_64
Let's begin by creating the installation directory.
Download the archive and install the latest version of the software. The appropriate archive is downloaded based on the edition (Enterprise, Free, or Open Source) and the device used for runtime.
Follow these steps to configure your HEAVY.AI environment.
For your convenience, you can update .bashrc with these environment variables
Although this step is optional, you will find references to the HEAVYAI_BASE and HEAVYAI_PATH variables. These variables contain the paths where configuration, license, and data files are stored and the location of the software installation. It is strongly recommended that you set them up.
Run the script that will initialize the HEAVY.AI services and database storage located in the systemd folder.
Accept the default values provided or make changes as needed.
This step will take a few minutes if you are installing a CUDA-enabled version of the software because the shaders must be compiled.
The script creates a data directory in $HEAVYAI_BASE/storage (typically /var/lib/heavyai) with the directories catalogs, dataand log, which will contain the metadata, the data of the database tables, and the log files from Immerse's web server and the database.
The log folder is particularly important for database administrators. It contains data about the system's health, performance, and user activities.
Heavy Immerse is not available in the OS Edition.
Start the services and enable the automatic startup of the service at reboot and start the HEAVY.AI services.
If a firewall is not already installed and you want to harden your system, install and start firewalld.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access:
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
The $HEAVYAI_BASE directory must be dedicated to HEAVYAI; do not set it to a directory shared by other packages.
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Connect to HeavyDB by entering the following command in a terminal on the host machine (default password is HyperInteractive):
anEnter a SQL query such as the following:
The results should be similar to the results below.
After installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart clearly demonstrates that there is a direct correlation between departure delay and arrival delay. This insight can help in identifying areas for improvement and implementing strategies to minimize delays and enhance overall efficiency.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data sour
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
HEAVY.AI uses the following ports.
Follow these steps to install HEAVY.AI as a Docker container on a machine running with on CPU or with supported NVIDIA GPU cards using Ubuntu as the host OS.
Prepare your host by installing Docker and if needed for your configuration NVIDIA drivers and NVIDIA runtime.
Remove any existing Docker Installs and if on GPU the legacy NVIDIA docker runtime.
Use curl to add the docker's GPG key.
Add Docker to your Apt repository.
Update your repository.
Install Docker, the command line interface, and the container runtime.
Run the following usermod command so that docker command execution does not require sudo privilege. Log out and log back in for the changes to take effect. (reccomended)
Verify your Docker installation.
Use curl to add Nvidia's Gpg key:
Update your sources list:
Update apt-get and install nvidia-container-runtime:
Edit /etc/docker/daemon.json to add the following, and save the changes:
Restart the Docker daemon:
Verify that docker and NVIDIA runtime work together.
If everything is working you should get the output of nvidia-smi command showing the installed GPUs in the system.
Create a directory to store data and configuration files
Then a minimal configuration file for the docker installation
Ensure that you have sufficient storage on the drive you choose for your storage dir running this command
Download HEAVY.AI from DockerHub and Start HEAVY.AI in Docker. Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are going to use.
Check that the docker is up and running a docker ps commnd:
You should see an output similar to the following.
If a firewall is not already installed and you want to harden your system, install theufw.
To use Heavy Immerse or other third-party tools, you must prepare your host machine to accept incoming HTTP(S) connections. Configure your firewall for external access.
Most cloud providers use a different mechanism for firewall configuration. The commands above might not run in cloud deployments.
Connect to Heavy Immerse using a web browser to your host on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
You can access the command line in the Docker image to perform configuration and run HEAVY.AI utilities.
You need to know the container-id to access the command line. Use the command below to list the running containers.
You see output similar to the following.
Once you have your container ID, in the example 9e01e520c30c, you can access the command line using the Docker exec command. For example, here is the command to start a Bash session in the Docker instance listed above. The -it switch makes the session interactive.
You can end the Bash session with the exit command.
HEAVY.AI ships with two sample datasets of airline flight information collected in 2008, and a census of New York City trees. To install sample data, run the following command.
Where <container-id> is the container in which HEAVY.AI is running.
When prompted, choose whether to insert dataset 1 (7,000,000 rows), dataset 2 (10,000 rows), or dataset 3 (683,000 rows). The examples below use dataset 2.
Connect to HeavyDB by entering the following command (a password willò be asked; the default password is HyperInteractive):
Enter a SQL query such as the following:
The results should be similar to the results below.
Installing Enterprise or Free Edition, check if Heavy Immerse is running as intended.
Connect to Heavy Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
Log into Heavy Immerse by entering the default username (admin) and password (HyperInteractive), and then click Connect.
Create a new dashboard and a Scatter Plot to verify that backend rendering is working.
Click New Dashboard.
Click Add Chart.
Click SCATTER.
Click Add Data Source.
Choose the flights_2008_10k table as the data source.
Click X Axis +Add Measure.
Choose depdelay.
Click Y Axis +Add Measure.
Choose arrdelay.
Click Size +Add Measure.
Choose airtime.
Click Color +Add Measure.
Choose dest_state.
The resulting chart shows, unsurprisingly, that there is a correlation between departure delay and arrival delay.
Create a new dashboard and a Table chart to verify that Heavy Immerse is working.
Click New Dashboard.
Click Add Chart.
Click Bubble.
Click Select Data Source.
Choose the flights_2008_10k table as the data sour
Click Add Dimension.
Choose carrier_name.
Click Add Measure.
Choose depdelay.
Click Add Measure.
Choose arrdelay.
Click Add Measure.
Choose #Records.
The resulting chart shows, unsurprisingly, that also the average departure delay is correlated to the average of arrival delay, while there is quite a difference between Carriers.
Getting Started with AWS AMI
You can use the HEAVY.AI AWS AMI (Amazon Web Services Amazon Machine Image) to try HeavyDB and Heavy Immerse in the cloud. Perform visual analytics with the included New York Taxi database, or import and explore your own data.
Many options are available when deploying an AWS AMI. These instructions skip to the specific tasks you must perform to deploy a sample environment.
You need a security key pair when you launch your HEAVY.AI instance. If you do not have one, create one before you continue.
Go to the EC2 Dashboard.
Select Key Pairs under Network & Security.
Click Create Key Pair.
Enter a name for your key pair. For example, MyKey.
Click Create. The key pair PEM file downloads to your local machine. For example, you would find MyKey.pem in your Downloads directory.
Go to the and select the version you want to use. You can get overview information about the product, see pricing, and get usage and support information.
Click Continue to Subscribe to subscribe.
Read the Terms and Conditions, and then click Continue to Configuration.
Select the Fulfillment Option, Software Version, and Region.
Click Continue to Launch.
On the Launch this software page, select Launch through EC2, and then click Launch.
From the Choose and Instance Type page, select an available EC2 instance type, and click Review and Launch.
Review the instance launch details, and click Launch.
Select a key pair, or click Create a key pair to create a new key pair and download it, and then click Launch Instances.
On the Launch Status page, click the instance name to see it on your EC2 Dashboard Instances page.
To connect to Heavy Immerse, you need your Public IP address and Instance ID for the instance you created. You can find these values on the Description tab for your instance.
To connect to Heavy Immerse:
Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
Enter the USERNAME (admin), PASSWORD ( {Instance ID} ), and DATABASE (heavyai). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
On the Dashboards page, click NYC Taxi Rides. Explore and filter the chart information on the NYC Taxis Dashboard.
Working with your own familiar dataset makes it easier to see the advantages of HEAVY.AI processing speed and data visualization.
To import your own data to Heavy Immerse:
Export your data from your current datastore as a comma-separated value (CSV) or tab-separated value (TSV) file. HEAVY.AI supports Latin-1 ASCII format and UTF-8. If you want to load data with another encoding (for example, UTF-16), convert the data to UTF-8 before loading it to HEAVY.AI.
Point your Internet browser to the public IP address for your instance, on port 6273. For example, for public IP 54.83.211.182, you would use the URL https://54.83.211.182:6273.
Enter the USERNAME (admin) and PASSWORD ( {instance ID} ). If you are using the BYOL version, enter you license key in the key field and click Apply.
Click Connect.
Click Data Manager, and then click Import Data.
Drag your data file onto the table importer page, or use the directory selector.
Click Import Files.
Verify the column names and datatypes. Edit them if needed.
Enter a Name for your table.
Click Save Table.
Click Connect to Table.
On the New Dashboard page, click Add Chart.
Choose a chart type.
Add dimensions and measures as required.
Click Apply.
Enter a Name for your dashboard.
Click Save.
Open a terminal window.
Locate your private key file (for example, MyKey.pem). The wizard automatically detects the key you used to launch the instance.
Your key must not be publicly viewable for SSH to work. Use this command to change permissions, if needed:
Connect to your instance using its Public DNS. The default user name is centos or ubuntu, depending on the version you are using. For example:
Use the following command to run the heavysql SQL command-line utility on HeavyDB. The default user is admin and the default password is { Instance ID }:
This procedure is considered experimental.
In some situations, you might not be able to upgrade NVIDIA CUDA drivers on a regular basis. To work around this issue, NVIDIA provides compatibility drivers that allow users to use newer features without requiring a full upgrade. For information about compatibility drivers, see .
Use the following commands to install the CUDA 11 compatibility drivers on Ubuntu:
After the last nvidia-smi, ensure that CUDA shows the correct version.
The driver version will still show as the old version.
After installing the drivers, update the systemd files in /lib/systemd/system/heavydb.service.
In the service section, add or update the environment property
The file should look like that
Then force the reload of the systemd configuration
This section is giving a recipe to upgrade between fully compatible products version.
This section is giving a recipe to upgrade between fully compatible products version.
As with any software upgrade, it is important that you back up your data before upgrading. Each release introduces efficiencies that are not necessarily compatible with earlier releases of the platform. HeavyAI is never expected to be backward compatible.
Back up the contents of your $HEAVYAI_STORAGE directory.
If you need to upgrade from Omnisci to HEAVY.AI 6.0 or later, please refer to the specific recipe.
Direct upgrades from Omnisci to HEAVY.AI version later than 6.0 aren't allowed nor supported.
To upgrade HEAVY.AI in place in Docker
In a terminal window, get the Docker container ID.
You should see output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
Stop the HEAVY.AI Docker container. For example:
Optionally, remove the HEAVY.AI Docker container. This removes unused Docker containers on your system and saves disk space.
Backup the Omnisci data directory (typically /var/lib/omnisci)
Download the latest version of the HEAVY.AI Docker image according to the Edition and device you are actually coming from Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.
If you don't want to upgrade to the latest version but want to upgrade to a specific version, change thelatesttag with the version needed.
If the version needed is the 6.0 use v6.0.0 as the version tag in the image name
heavyai/heavyai-ee-cuda:v6.0.0
Check that the docker is up and running a docker ps commnd:
You should see an output similar to the following.
This runs both the HEAVY.AI database and Immerse in the same container.
You can optionally add --rm to the Docker run command so that the container is removed when it is stopped.
To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data
Stop the HEAVY.AI services.
Back up your $HEAVYAI_STORAGE directory (the default location is /var/lib/heavyai).
Run the appropriate set of commands depending on the method used to install the previous version of the software.
Make a backup of your actual installation
When the upgrade is complete, start the HEAVY.AI services.
Getting Started with HEAVY.AI on Google Cloud Platform
Follow these instructions to get started with HEAVY.AI on Google Cloud Platform (GCP).
You must have a Google Cloud Platform account. If you do not have an account, follow to sign up for one.
To launch HEAVY.AI on Google Cloud Platform, you select and configure an instance.
On the solution Launcher Page, click Launch on Compute Engine to begin configuring your deployment.
Before deploying a solution with a GPU machine type, avoid potential deployment failure by to make sure that you have not exceeded your limit.
To launch HEAVY.AI on Google Cloud Platform, you select and configure a GPU-enabled instance.
Search for HEAVY.AI on the , and select a solution. OmniSci has four instance types:
.
.
.
.
On the solution Launcher Page, click Launch to begin configuring your deployment.
On the new deployment page, configure the following:
Deployment name
Zone
Machine type - Click Customize and configure Cores and Memory, and select Extend memory if necessary.
GPU type. (Not applicable for CPU configurations.)
Number of GPUs - (Not applicable for CPU configurations.) Select the number of GPUs; subject to quota and GPU type by region. For more information about GPU-equipped instances and associated resources, see .
Boot disk type
Boot disk size in GB
Networking - Set the Network, Subnetwork, and External IP.
Firewall - Select the required ports to allow TCP-based connectivity to HEAVY.AI. Click More to set IP ranges for port traffic and IP forwarding.
Accept the GCP Marketplace Terms of Service and click Deploy.
In the Deployment Manager, click the instance that you deployed.
Launch the Heavy Immerse client:
Record the Admin password (Temporary).
Click the Site address link to go to the Heavy Immerse login page. Enter the password you recorded, and click Connect.
Copy your license key from the registration email message. If you have not received your license key, contact your Sales Representative or register for your 30-day trial .
Connect to Immerse using a web browser connected to your host machine on port 6273. For example, http://heavyai.mycompany.com:6273.
When prompted, paste your license key in the text box and click Apply.
Click Connect to start using HEAVY.AI.
On successful login, you see a list of sample dashboards loaded into your instance.
This section is giving a recipe to upgrade from Omnisci platform 5.5+ to Heavi.ai 6.0.
If the version of Omnisci is older than 5.5 an intermediate upgrade step to the 5.5 version is needed. Check the docs on how to do the .
If you are upgrading from Omnisci to HEAVY.AI, there are a lot of additional steps compared to a simple sub-version upgrade.
IMPORTANT - Before you begin, stop all the running services / docker images of your Omnisci installation and create a backup $OMNISCI_STORAGE folder (typically /var/lib/omnisci). A backup is essential for recoverability; do not proceed with the upgrade without confirming that a full and consistent backup is available and ready to be restored.
The omnisci the database will not be automatically renamed to the new default name heavyai.This will be done manually and it's documented in the upgrade steps.
All the dumps created with the dump command on Omnisci cannot be restored after the database is upgraded to this version.
The following table describes the changes to environment variables, storage locations, and filenames in Release 6.0 compared to Release 5.x. Except where noted, revised storage subfolders, symlinks for old folder names, and filenames are created automatically on server start.
Change descriptions in bold require user intervention.
| Description | Omnisci 5.x | HEAVY.AI 6.0 |
|---|
The order of these instructions is significant. To avoid problems, follow the order of the instruction provided and don't skip any step.
This upgrade procedure is assuming that you are using the default storage location for both Omnisci and HEAVY.AI.
Stop all containers running Omnisci services.
In a terminal window, get the Docker container IDs:
You should see an output similar to the following. The first entry is the container ID. In this example, it is 9e01e520c30c:
Stop the HEAVY.AI Docker container. For example:
Backup the Omnisci data directory (typically /var/lib/omnisci).
Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme.
Create a new configuration file for heavydb changing the data parameter to point to the renamed data directory.
Rename the Omnisci license file (EE and FREE only).
Download and run the 6.0 version of the HEAVY.AI Docker image.
Select the tab depending on the Edition (Enterprise, Free, or Open Source) and execution Device (GPU or CPU) you are upgrading.
Check that Docker is up and running using a docker ps command:
You should see output similar to the following:
Using the new container ID rename the default omnisci database to heavyai:
Check that everything is running as expected.
To upgrade an existing system installed with package managers or tarball. The commands upgrade HEAVY.AI in place without disturbing your configuration or stored data.
Stop the Omnisci services.
Backup the Omnisci data directory (typically /var/lib/omnisci).
Create a user named heavyai who will be the owner of the HEAVY.AI software and data on the filesystem.
Set a password for the user. It'll need when sudo-ing.
Login with the newly created user
Rename the Omnisci data directory to reflect the HEAVY.AI naming scheme and change the ownership to heavyai user.
Check that everything is in order and the semaphore directory is created
All the directories must belong to heavyai user and the directory catalogs would be present
Rename the license file. (EE and FREE only)
Please follow all the steps in the Installation and Configuration until Intialization step.
Log in with the heavyai user and be sure that the heavyai services are stopped.
Create a new configuration file for heavydb changing the data parameter to point to the /var/lib/heavyai/storage directory and the frontend to the new install directory.
All the settings of the upgraded database will be moved to the new configuration file.
Now we have to complete the database migration.
Remove the semaphore directory.
To complete the upgrade, start the HEAVY.AI servers.
Check if the database migrated, running this command and checking for the Rebrand migration complete message.
Rename the default omnisci database to heavyai.
Run the command using an administrative user (typically admin) with his password (default HyperInteractive)
Restart the database service and check that everything is running as expected.
After all the checks confirmed that the upgraded system is stable, clean up the system to remove the Omnisci install and relative system configuration. Remove permanently the configuration of the services.
Remove the installed software.
Delete the YUM or APT repositories.
: Install NVIDIA drivers with CUDA toolkit from NVIDIA Website
: Install NVIDIA drivers via .run file using the NVIDIA Website
: Install NVIDIA drivers using APT package manager
CUDA is a parallel computing platform and application programming interface (API) model. It uses a CUDA-enabled graphics processing unit (GPU) for general-purpose processing. The CUDA platform provides direct access to the GPU virtual instruction set and parallel computation elements. For more information on CUDA unrelated to installing HEAVY.AI, see .
Open and select the desired CUDA Toolkit version to install.
Install the CUDA package for your platform and operating system according to the instructions on the NVIDIA website ().
Please check that the driver's version you are downloading meets the HEAVI.AI
Be careful when choosing the driver version to install. Ensure that your GPU's model is supported and that meets the HEAVI.AI
For more information about troubleshooting Vulkan, see the section.
If you installed NVIDIA drivers using above, the CUDA toolkit is already installed; you may proceed to the verification step below.
For more information, see C++ .
Follow these instructions to install a headless JDK and configure an environment variable with a path to the library. The “headless” Java Development Kit does not provide support for keyboard, mouse, or display systems. It has fewer dependencies and is best suited for a server host. For more information, see .
To install software using DNF's package manager, you can utilize DNF's package management capabilities to search for and then install the desired software. This method provides a convenient and efficient way to manage software installations and dependencies on your system.
Installing via a tarball involves obtaining a compressed archive file (tarball) from the software's official source or repository. After downloading the tarball, you would need to extract its contents and follow the installation instructions provided by the software developers. This method allows for manual installation and customization of the software.
See for details.
The first step to activate the system is starting HeavyDB and the Web Server service that Heavy Immerse needs.
For more information, see .
If you are on Enterprise or Free Edition, you need to validate your HEAVY.AI instance with your license key. You can skip this section if you are using Open Source Edition.
Copy your license key from the registration email message. If you have not received your license key, contact your Sales Representative or register for your 30-day trial .
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Pointmap using Heavy Immerse.
For more information on Docker installation, see the .
Install NVIDIA driver and Cuda Toolkit using
See also the note regarding the in Optimizing Performance.
For more information, see .
If you are on Enterprise or Free Edition, you need to validate your HEAVY.AI instance using your license key. You must skip this section if you are on Open Source Edition
Copy your license key of Enterprise or Free Edition from the registration email message. If you don't have a license and you want to evaluate HEAVY.AI in an enterprise environment, contact your Sales Representative or register for your 30-day trial of Enterprise Edition . If you need a Free License you can get one .
To verify that everything is working, load some sample data, perform a heavysql query, and generate a Scatter Plot or a Bubble Chart using Heavy Immerse
If you receive an error message stating that the connection is not private, follow the prompts onscreen to click through to the unsecured website. To secure your site, see .
For more information on Heavy Immerse features, see .
For more information, see .
Follow these instructions to connect to your instance using SSH from MacOS or Linux. For information on connecting from Windows, see .
For more information, see .
See also the note regarding the in Optimizing Performance.
Download and Install the latest version following the install documentation for your Operative System and
Install the HEAVY.AI software following all the instructions for your Operative System. and .
Port | Service | Use |
6273 | heavy_web_server | Used to access Heavy Immerse. |
6274 | heavydb tcp | Used by connectors (heavyai, omnisql, odbc, and jdbc) to access the more efficient Thrift API. |
6276 | heavy_web_server | Used to access the HTTP/JSON thrift API. |
6278 | heavydb http | Used to directly access the HTTP/binary thrift API, without having to proxy through heavy_web_server. Recommended for debugging use only. |
$OMNISCI_STORAGE | $HEAVYAI_BASE |
/var/lib/omnisci | /var/lib/heavyai |
Environmental variable for storage location | $OMNISCI_STORAGE | $HEAVYAI_BASE |
Default location for $HEAVYAI_BASE / $OMNISCI_STORAGE | /var/lib/omnisci | /var/lib/heavyai |
Fixed location for Docker $HEAVYAI_BASE / $OMNISCI_STORAGE | /omnisci-storage | /var/lib/heavyai |
The folder containing catalogs for $HEAVYAI_BASE / $OMNISCI_STORAGE | data/ | storage/ |
Storage subfolder - data | data/mapd_data | storage/data |
Storage subfolder - catalog | data/mapd_catalogs | storage/catalogs |
Storage subfolder - import | data/mapd_import | storage/import |
Storage subfolder - export | data/mapd_export | storage/export |
Storage subfolder - logs | data/mapd_log | storage/log |
Server INFO logs | omnisci_server.INFO | heavydb.INFO |
Server ERROR logs | omnisci_server.ERROR | heavydb.ERROR |
Server WARNING logs | omnisci_server.WARNING | heavydb.WARNING |
Web Server ACCESS logs | omnisci_web_server.ACCESS | heavy_web_server.ACCESS |
Web Server ALL logs | omnisci_web_server.ALL | heavy_web_server.ALL |
Install directory | /omnisci (Docker) /opt/omnisci (bare metal) | /opt/heavyai/ (Docker and bare metal) |
Binary file - core server (located in install directory) | bin/omnsici_server | bin/heavydb |
Binary file - web server (located in install directory) | bin/omnisci_web_server | bin/heavy_web_server |
Binary file - command- line SQL utility | bin/omnisql | bin/heavysql |
Binary file - JDBC jar | bin/omnisci-jdbc-5.10.2-SNAPSHOT.jar | bin/heavydb-jdbc-6.0.0-SNAPSHOT.jar |
Binary file - Utilities (SqlImporter) jar | bin/omnisci-utility-5.10.2-SNAPSHOT.jar | bin/heavydb-utility-6.0.0-SNAPSHOT.jar |
HEAVY.AI Server service (for bare metal install) | omnisci_server | heavydb |
HEAVY.AI Web Server service (for bare metal install) | omnisci_web_server | heavy_web_server |
Default configuration file | omnisci.conf | heavy.conf |
OmniSci less then 5.5 | HEAVY.AI 7.0 | Upgrade to 5.5 --> --> 7.0 |
OmniSci 5.5-5.10 | HEAVY.AI 7.0 | Upgrade to --> 7.0 |
HEAVY.AI 6.0-6.4 | HEAVY.AI 7.0 | Upgrade to 7.0 |
