ODBC Data Wrapper Reference
ODBC HeavyConnect and import is currently a beta feature.
This topic provides setup and configuration details and information about supported data types when using ODBC HeavyConnect or import. This reference also describes ODBC data wrapper assumptions and supported mappings of HeavyDB column types to ODBC column types, and provides RDMS-specific notes.
To HeavyConnect to or import from an external RDMS using the ODBC data wrapper, make sure that an appropriate ODBC driver is installed for the RDMS you want to access. RDMS vendors websites usually provide ODBC driver download and configuration instructions, and in certain cases, ODBC drivers are managed by third-party vendors that provide these instructions.
Two key configuration files need to be set up when using ODBC HeavyConnect or import:
odbcinst.inifile specifies a list of installed drivers, along with required configuration for those drivers (the path to the driver binary). The following example is an
odbcinst.iniconfiguration file for an instance with installed drivers for PostGreSQL, Redshift, and Snowflake:
Description=PostgreSQL ODBC driver
Description=Redshift ODBC driver
Description=Snowflake ODBC Driver
Use an intuitive name for the drivers, such as the official name of the RDMS, so that users can easily know the driver names to use in connection strings.
odbc.inifile contains configuration for database-specific connections. Each connection configuration is identified by a data source name (DSN). The following example is an
odbc.inifile with configuration for the default
postgresdatabase on a local instance of PostgreSQL running on port 5432:
[ODBC Data Sources]
Description=Local default PostgreSQL database
/etc/directory, or in the home directory for cases where the server process is started under a specific user account.
For docker deployments of HEAVY.AI, ODBC driver installations can be managed in a Dockerfile that uses the HEAVY.AI Docker image as a base. For example, you can create a docker image with installed PostGreSQL, Redshift, and Snowflake ODBC drivers using a Dockerfile with the following content.
Always consult the license and terms and conditions of the ODBC drivers you install. The examples here are provided for illustrative purposes only.
# Install PostGreSQL ODBC driver.
# The Snowflake ODBC driver depends on unixodbc.
RUN apt-get update && apt-get install -y odbc-postgresql unixodbc
# Install Redshift ODBC driver.
RUN wget https://s3.amazonaws.com/redshift-downloads/drivers/odbc/126.96.36.1990/AmazonRedshiftODBC-64-bit-188.8.131.520-1.x86_64.deb
RUN dpkg -i ./AmazonRedshiftODBC-64-bit-184.108.40.2060-1.x86_64.deb
RUN rm ./AmazonRedshiftODBC-64-bit-220.127.116.110-1.x86_64.deb
# Install Snowflake ODBC driver.
RUN wget https://sfc-repo.snowflakecomputing.com/odbc/linux/latest/snowflake-odbc-2.25.2.x86_64.deb
RUN dpkg -i ./snowflake-odbc-2.25.2.x86_64.deb
RUN rm ./snowflake-odbc-2.25.2.x86_64.deb
You can then build a new image with installed ODBC drivers using the Dockerfile.
odbc.iniconfiguration files should be bound to
/etc/odbc.ini, respectively, when running the container.
- The remote database management system supports the MIN, MAX, and LENGTH SQL functions. These functions are used internally for optimizations and validations.
- The combination of the query specified in the foreign table’s SQL_SELECT __ option and the __ SQL___ORDER_BY option is guaranteed to return a result set in the same order every time the query is executed.
- Queries for geo columns are always returned as WKT strings. This implies that either geo columns are stored as text columns containing WKT strings, geo columns are cast to text (for example, using a function like ST_AsText) in the SQL_SELECT query, or the remote database management system, by default, projects geo columns as WKT strings.
* PostgreSQL requires setting the option
BoolsAsChar=falseto represent boolean columns as SQL_BIT.
Array data types currently are not supported.
See https://docs.microsoft.com/en-us/sql/odbc/reference/appendixes/sql-data-types for more details about ODBC data types, and consult the specific RDMS ODBC driver documentation for details about how the RDMS data types map to the ODBC data types.
- Accessing BOOLEAN data in PostgreSQL requires the
BoolsAsChar=falseoption to be added to the connection string or DSN configuration.