Release Notes
Release notes for currently supported releases
For release notes for releases that are no longer supported, as well as links to documentation for those releases, see Archived Release Notes.
As with any software upgrade, it is important to back up your data before you upgrade HEAVY.AI. In addition, we recommend testing new releases before deploying in a production environment.
For assistance during the upgrade process, contact HEAVY.AI Support by logging a request through the HEAVY.AI Support Portal.
IMPORTANT - In HeavyDB Release 7.x.x, the “render groups” mechanism, part of the previous implementation of polygon rendering, has been removed. When you upgrade to HeavyDB Release 7.x.x, all existing tables that have a POLYGON or MULTIPOLYGON geo column are automatically migrated to remove a hidden column containing "render groups" metadata.
This operation is performed on all tables in all catalogs at first startup, and the results are recorded in the INFO log.
Once a table has been migrated in this manner, it is not backwards-compatible with earlier versions of HeavyDB. If you revert to an earlier version, the table may appear to have missing columns and behavior will be undefined. Attempting to query or render the POLYGON or MULTIPOLYGON data with the earlier version may fail or cause a server crash.
As always, HEAVY.AI strongly recommends that all databases be backed up, or at the very least, dumps are made of tables with POLYGON or MULTIPOLYGON columns using the existing HeavyDB version, before upgrading to HeavyDB Release 7.x.x.
Dumps of POLYGON and MULTIPOLYGON tables made with earlier versions can still be restored into HeavyDB Release 7.x.x. The superfluous metadata is automatically discarded. However, dumps of POLYGON and MULTIPOLYGON tables made with HeavyDB Release 7.x.x are not backwards-compatible with earlier versions.
This applies only to tables with POLYGON or MULTIPOLYGON columns. Tables that contain other geo column types (POINT, LINESTRING, etc.), or only non-geo column types, do not require migration and remain backwards-compatible with earlier relea
For Ubuntu installations, install libncurses5 with the following command:
sudo apt install libncurses5- Adds an EXPLAIN CALCITE DETAILED command that displays more details about referenced columns in the query plan.
- Improved logging around system memory utilization for each query.
- Adds an option to SQLImporter for disabling logging of connection strings.
- Adds a "gpu-code-cache-max-size-in-bytes" server configuration parameter for limiting the amount of memory that can be used by the GPU code cache.
- Improves column name representation in Parquet validation error messages.
- Fixes a parser error that could occur for queries containing a NOT ILIKE clause.
- Fixes a multiplication overflow error that could occur when retrying queries on CPU.
- Fixes an issue where table dumps do not preserve quoted column names.
- Fixes a "cannot start a transaction within a transaction" error that could occur in certain cases.
- Fixes a crash that could occur for certain query patterns involving division by COUNT aggregation functions.
- Removes a warning that is displayed on server startup when HeavyIQ is not configured.
- Removes spurious warnings for CURSOR type checks when there are both cursor and scalar overloads for a user-defined table function.
- Adds hit testing support for custom measures that reference multiple tables.
- Fixes SAML authentication regression in 7.1.0
- Fixes chart export regression in 7.1.0
- Exposes new geo overlaps function ST_INTERSECTSBOX for very fast bounding box intersection detections.
- Adds support for the max_reject COPY FROM option when importing raster files. This ensures that imports from large multi-file raster datasets continue after minor errors, but provides adjustable notification upon major ones.
- Adds a new ST_AsBinary function that returns the Well-Known Binary (WKB) representation of geometry values. This highly-efficient format is used by postGIS newer versions of Geopandas.
- Adds a new ST_AsText function that returns the Well-Known Text (WKT) representation of geometry values. This is less efficient than WKB but compatible even with nonspatial databases.
- Adds support for loading geometry values using the load_table_binary_arrow Thrift API.
- New version of HeavyAI python library with direct Geopandas support.
- New version of rbc-project with geo column support allowing extensions which input or output any geometric type.
- New JAROWINKLER_SIMILARITY string operator for fuzzy matching between string columns and values. This is a case insensitive measure including edit transitions and (slightly) sensitive to white space.
- New LEVENSHTEIN_DISTANCE string operator for fuzzy matching between string columns and values. This is case insensitive and represents the number of edits needed to make two strings identical. An “edit” is defined by either an insertion of a character, a deletion of a character, or a replacement of a character.
- Extends the ALTER COLUMN TYPE command to support string dictionary encoding size reduction.
- Improves the error message returned when out of bound values are inserted into FLOAT and DOUBLE columns.
- Adds a "watchdog-max-projected-rows-per-device" server configuration parameter and query hint that determines the maximum number of rows that can be projected by each GPU and CPU device.
- Adds a "preflight-count-query-threshold" server configuration parameter and query hint that determines the threshold at which the preflight count query optimization should be executed.
- Optimizes memory utilization for projection queries on instances with multiple GPUs.
- Support for PCA models and PCA_PROJECT operator.
- Support SHOW MODEL FEATURE DETAILS to show per-feature info for models, including regression coefficients and variable importance scores, if applicable.
- Support for TRAIN_FRACTION option to specify proportion of the input data to a CREATE MODEL statement that should be trained on.
- Support creation of models with only categorical predictors.
- Enable categorical and numeric predictors to be specified in any order for CREATE MODEL statements and subsequent inference operations.
- Enable Torch table functions (requires client to specify libtorch.so).
- Add tf_torch_raster_object_detect for raster object detections (requires client to specify libtorch.so and provide trained model in torchscript format).
- Allow Array literals as arguments to scalar UDFs
- Support table function (UDTF) output row sizes up to 16 trillion rows
- Adds support for Column<TextEncodingNone> and ColumnList<TextEncodingNone> table function inputs and outputs.
- SQL projections now are sized per GPU/CPU core instead of globally, meaning that projections are more memory efficient as a function of the number of GPUs/CPU threads used for a query. In particular, this means that various forms of in-situ rendering, for example, non-grouped pointmaps, renders can scale to N GPUs more points or use N GPUs less memory, depending on the configuration.
- Better parallelize construction of metadata for subquery results for improved performance
- Enables result set caching for queries with LIMIT clauses.
- Enables the bounding box intersection optimization for certain spatial join operators and geometry types by default.
- Fix potential crash when concatenating strings with the output of a UDF.
- Fixes an issue where deleted rows with malformed data can prevent ALTER COLUMN TYPE command execution.
- Fixes an error that could occur when parsing odbcinst.ini configuration files containing only one installed driver entry.
- Fixes a table data corruption issue that could occur when the server crashes multiple times while executing write queries.
- Fixes a crash that could occur when attempting to do a union of a string dictionary encoded text column and a none encoded text column.
- Fixes a crash that could occur when the output of a table function is used as an argument to the strtok_to_array function.
- Fixes a crash that could occur for queries involving projections of both geometry columns and geometry function expressions.
- Fixes an issue where wrong results could be returned when the output of the DATE_TRUNC function is used as an argument to the count distinct function.
- Fixes an issue where an error occurs if the COUNT_IF function is used in an arithmetic expression.
- Fixes a crash that could occur when the WIDTH_BUCKET function is called with decimal columns.
- Fixes an issue where the WIDTH_BUCKET function could return wrong results when called with decimal values close to the upper and lower boundary values.
- Fixes a crash that could occur for queries with redundant projection steps in the query plan.
- Fixes a crash that could occur on multi-gpu systems while handling an out of GPU memory error.
- Zoom to filters, setting map bounding box to extent of current filter set.
- Image preview in map chart popups where image URLs are present.
- Fixed error thrown by choropleth chart on polygon hover.
- Adds support for nested window function expressions.
- Adds support for exception propagation from table functions.
- Fixes a crash that could occur when accessing 8-bit or 16-bit string dictionary encoded text columns on ODBC backed foreign tables.
- Fixes unexpected GPU execution and memory allocations that could occur when executing sort queries with the CPU mode query hint.
- Fixes an issue that could occur when inserting empty strings for geometry columns.
- Fixes an issue that could occur when out of bounds fragment sizes are specified on table creation.
- Fixes an issue where system dashboards could contain unexpected cached data.
- Fixes a crash that could occur when executing aggregate functions over the result of join operations on scalar subqueries.
- Fixes a server hang that could occur when GPU code compilation errors occur for user-defined table functions.
- Fixes a data race that could occur when logging query plan cache size.
- Add support for rendering 1D “terrain” cross-section overlays.
- Rewrite 2D cross-section mesh generation as a table function.
- Further improvements to system state logging when a render out of memory error occurs, and move it to the ERROR log for guaranteed visibility.
- Enable auto-clear-render-mem by default for any render-vega call taking < 10 seconds.
- Render requests with 0 width or height could lead to a CHECK failure in encodePNG. Invalid image sizes now throw a non-fatal error during vega parsing.
- Visualize terrain at the base of atmospheric cross sections in the Cross Section chart with the new Base Terrain chart layer type.
- Fixed local timezone issue with Chart Animation using cross filter replay.
- Improves instrumentation around CPU and GPU memory utilization and certain crash scenarios.
- Fixes a crash that could occur for GPU executed join queries on dictionary encoded text columns with NULL values.
- Improve instrumentation and logging related to gpu memory utilization, particularly with polygon rendering, as well as command timeout issues
- Fix a potential segfault when a Vulkan device lost error occurs
IMPORTANT - In HeavyDB Release 7.0, the “render groups” mechanism, part of the previous implementation of polygon rendering, has been removed. When you upgrade to HeavyDB Release 7.0, all existing tables that have a POLYGON or MULTIPOLYGON geo column are automatically migrated to remove a hidden column containing "render groups" metadata.
This operation is performed on all tables in all catalogs at first startup, and the results are recorded in the INFO log.
Once a table has been migrated in this manner, it is not backwards-compatible with earlier versions of HeavyDB. If you revert to an earlier version, the table may appear to have missing columns and behavior will be undefined. Attempting to query or render the POLYGON or MULTIPOLYGON data with the earlier version may fail or cause a server crash.
As always, HEAVY.AI strongly recommends that all databases be backed up, or at the very least, dumps are made of tables with POLYGON or MULTIPOLYGON columns using the existing HeavyDB version, before upgrading to HeavyDB Release 7.0.
Dumps of POLYGON and MULTIPOLYGON tables made with earlier versions can still be restored into HeavyDB Release 7.0. The superfluous metadata is automatically discarded. However, dumps of POLYGON and MULTIPOLYGON tables made with HeavyDB Release 7.0 are not backwards-compatible with earlier versions.
This applies only to tables with POLYGON or MULTIPOLYGON columns. Tables that contain other geo column types (POINT, LINESTRING, etc.), or only non-geo column types, do not require migration and remain backwards-compatible with earlier releases.
For Ubuntu installations, install libncurses5 with the following command:
sudo apt install libncurses5- Adds new Executor Resource Manager enabling parallel CPU and CPU-GPU query execution, and support for CPU execution on data inputs larger than fit in memory.
- Adds HeavyML, a suite of machine learning capabilities accessible directly in SQL, including support for linear regression, random forest, gradient boosted trees, and decision tree regression models, and KMeans and DBScan clustering methods. (BETA)
- Adds HeavyConnect support for MULTIPOINT and MULTILINESTRING columns.
- Adds ALTER COLUMN TYPE support for text columns.
- Adds a REASSIGN ALL OWNED command that allows for object ownership change across all databases.
- Adds an option for validating POLYGON and MULTIPOLYGON columns when importing using the COPY FROM command or when using HeavyConnect.
- Adds support for CONDITIONAL_CHANGE_EVENT window function.
- Adds support for automatic casting of table function CURSOR arguments.
- Adds support for Column<GeoMultiPolygon>, Column<GeoMultiLineString>, and Column<GeoMultiPoint> table function inputs and outputs.
- Adds support for none encoded text column, geometry column, and array column projections from the right table in left join queries.
- Adds support for literal text scalar subqueries.
- Adds support for ST_X and ST_Y function output cast to text.
- Improves concurrent execution of DDL and SHOW commands.
- Improves error messaging for when the storage directory is missing.
- Optimizes memory utilization for auto-vacuuming after delete queries.
- Fixes an issue where the root user could be deleted in certain cases.
- Fixes an issue where staging directories for S3 import could remain when imports failed.
- Fixes a crash that could occur when accessing the "tables" system table on instances containing tables with many columns.
- Fixes a crash that could occur when accessing CSV and regex parsed file foreign tables that previously errored out during cache recovery.
- Fixes an issue where dumping table foreign tables would produce an empty table.
- Fixes an intermittent crash that could occur when accessing CSV and regex parsed file foreign tables that are backed by large files.
- Fixes a "Ran out of slots in the query output buffer" exception that could occur when using stale cached cardinality values.
- Fixes an issue where user defined table functions are erroneously categorized as ambiguous.
- Fixes an error that could occur when a group by clause includes an alias that matches a column name.
- Fixes a crash that could occur on GPUs with the Pascal architecture when executing join queries with case expression projections.
- Fixes a crash that could occur when using the LAG_IN_FRAME window function.
- Fixes a crash that could occur when projecting geospatial columns from the tf_raster_contour_polygons table function.
- Fixes an issue that could occur when calling window functions on encoded date columns.
- Fixes a crash that could occur when the coalesce function is called with geospatial or array columns.
- Fixes a crash that could occur when projecting case expressions with geospatial or array columns.
- Fixes a crash that could occur due to rounding error when using the WIDTH_BUCKET function.
- Fixes a crash that could occur in certain cases where left join queries are executed on GPU.
- Fixes a crash that could occur for queries with joins on encoded date columns.
- Fixes a crash that could occur when using the SAMPLE function on a geospatial column.
- Fixes a crash that could occur for table functions with cursor arguments that specify no field type.
- Fixes an issue where automatic casting does not work correctly for table function calls with ColumnList input arguments.
- Fixes an issue where table function argument types are not correctly inferred when arithmetic operations are applied.
- Fixes an intermittent crash that could occur for join queries due to a race condition when changing hash table layouts.
- Fixes an out of CPU memory error that could occur when executing a query with a count distinct function call on a high cardinality column.
- Fixes a crash that could occur when running a HeavyDB instance in read-only mode after previously executing write queries on tables.
- Fixes an issue where the auto-vacuuming process does not immediately evict chunks that were pulled in for vacuuming.
- Fixes a crash that could occur in certain cases when HeavyConnect is used with Parquet files containing null string values.
- Fixes potentially inaccurate calculation of vertical attenuation from antenna patterns in HeavyRF.
- Add support for rendering a 1d cross-section as a line
- Package the Vulkan loader libVulkan1 alongside heavydb
- Fix a device lost error that could occur with complex polygon renders
- Data source Joins as a new custom data source type. (BETA)
- Adds improved query performance defaults for the Contour Chart.
- Adds access to new control panel to users with role "immerse_control_panel", even if the user is not a superuser.
- Adds custom naming of map layers.
- Adds custom map layer limit option using flag “ui/max_map_layers” which can be set explicitly (defaults to 8) or to -1 to remove the limit.
- Renames role from “immerse_trial_mode” to “immerse_export_disabled” and renames corresponding flag from “ui/enable_trial_mode” to “ui/user_export_disabled”.
- Various minor UI fixes and polishing.
- Fixes an issue where changing parameter value causes Choropleth popup to lose selected popup columns.
- Fixes an issue where changing parameter value causes Pointmap to lose selected popup columns.
- Fixes an issue where building a Skew-T chart results in a blank browser page.
- Fixes an issue where Skew-T chart did not display wind barbs.
- Fixes an issue with default date and time formatting.
- Fixes an issue where setting flag "ui/enable_map_exports" to false unexpectedly disabled table chart export.
- Fixes an issue with date filter presets.
- Fixes an issue where filters "Does Not Contain" or "Does not equal" did not work on Crosslinked Columns.
- Fixes an issue where charts were not redrawing to show the current bounding box filter set by the Linemap chart.
- Adds support for literal text scalar subqueries.
- Fixes a crash that could occur due to rounding error when using the WIDTH_BUCKET function.
- Fixes a crash that could occur for queries with joins on encoded date columns.
- Fixes a crash that could occur when running a HeavyDB instance in read-only mode after previously executing write queries on tables.
- Fixes an issue where the auto-vacuuming process does not immediately evict chunks that were pulled in for vacuuming.
- Fixed issue where Skew-T chart would not render when nulls were used in selected data.
- Fixed issue where wind barbs were not visible on Skew-T chart.
- Added feature flag
ui/session_create_timeoutwith a default value of 10000 (10 seconds) for modifying login request timeout.
- Adds the HeavyDB server configuration parameter
enable-foreign-table-scheduled-refreshfor enabling or disabling automated foreign table scheduled refreshes..
- Fixes a crash that could occur when S3 CSV-backed foreign tables with append refreshes are refreshed multiple times.
- Fixes a crash that could occur when foreign tables with geospatial columns are refreshed after cache evictions.
- Fixes a crash that could occur when querying foreign tables backed by Parquet files with empty row groups.
- Fixes an error that could occur when select queries used in ODBC foreign tables reference case sensitive column names.
- Fixes a crash that could occur when CSV backed foreign tables with geospatial columns are refreshed without updates to the underlying CSV files.
- Fixes a crash that could occur in heavysql when executing the \detect command with geospatial files.
- Fixes a casting error that could occur when executing left join queries.
- Fixes a crash that could occur when accessing the disk cache on HeavyDB servers with the read-only configuration parameter enabled.
- Fixes an error that could occur when executing queries that project geospatial columns.
- Fixes a crash that could occur when executing the EXTRACT function with the ISODOW
date_partparameter on GPUs. - Fixes an error that could occur when importing CSV or Parquet files with text columns containing more than 32,767 characters into HeavyDB NONE ENCODED text columns.
- Fixes a Vulkan Device Lost error that could occur when rendering complex polygon data with thousands of polygons in a single pixel.
- Optimizes result set buffer allocations for CPU group by queries.
- Enables trimming of white spaces in quoted fields during CSV file imports, when both the
trim_spacesandquotedoptions are set.
- Fixes an error that could occur when importing CSV files with quoted fields that are surrounded by white spaces.
- Fixes a crash that could occur when tables are reordered for range join queries.
- Fixes a crash that could occur for join queries with intermediate projections.
- Fixes a crash that could occur for queries with geospatial join predicate functions that use literal parameters.
- Fixes an issue where queries could intermittently and incorrectly return error responses.
- Fixes an issue where queries could return incorrect results when filter push-down through joins is enabled.
- Fixes a crash that could occur for queries with join predicates that compare string dictionary encoded and nonencoded text columns.
- Fixes an issue where hash table optimizations could ignore the
max-cacheable-hashtable-size-bytesandhashtable-cache-total-bytesserver configuration parameters. - Fixes an issue where sharded table join queries that are executed on multiple GPUs could return incorrect results.
- Fixes a crash that could occur when sharded table join queries are executed on multiple GPUs with the
from-table-reorderingserver configuration parameter enabled.
- Multilayer support for Contour and Windbarb charts.
- Support custom SQL measures in Contour charts.
- Restrict export from Heavy Immerse by enabling trial mode (feature flag:
ui/enable_trial_mode). Trial mode enables a super user to restrict export capabilities for users who have theimmerse_trial_moderole.
- Allow MULTILINESTRING to be used in selectors for Linemap charts.
- Allow MULTILINESTRING to be used in Immerse SQL Editor.
This release features general availability of data connectors for PostGreSQL, beta Immerse connectors for Snowflake and Redshift, and SQL support for Google BigQuery and Hive (beta). These managed data connections let you use HEAVY.AI as an acceleration platform wherever your source data may live. Scheduling and automated caching ensure that fast analytics are always running on the latest available data.
Immerse features four new chart types: Contour, Cross-section, Wind barb, and Skew-t. While especially useful for atmospheric and geotechnical data visualization, Contour and Cross-section also have more general application.
Major improvements for time series analysis have been added. This includes an Immerse user interface for time series, and a large number of SQL window function additions and performance enhancements.
The release also includes two major architectural improvements:
- The ability to perform cross-database queries, both in SQL and in Immerse, increasing flexibility across the board. For example, you can now easily build an Immerse dashboard showing system usage combined with business data. You might also make a read-only database of data shared across a set of users.
- Render queries no longer block other GPU queries. In many use cases, renders can be significantly slower than other common queries. This should result in significant performance gains, particularly in map-heavy dashboards.
- Adds support for cross database SELECT, UPDATE, and DELETE queries.
- Support for MODE SQL aggregate.
- Add support for strtok_to_array.
- Support for ST_NumGeometries().
- Support ST_TRANSFORM applied to literal geo types.
- Enhanced query tracing ensures all child operations for a query_id are properly logged with that ID.
- Adds support for BigQuery and Hive HeavyConnect and import.
- Adds support for table restore from S3 archive files.
- Improves integer column type detection in Snowflake import/HeavyConnect data preview.
- Adds HeavyConnect and import support for Parquet required scalar fields.
- Improves import status error message when an invalid request is made.
- Support POINT, LINESTRING, and POLYGON input and output types in table functions.
- Support default values for scalar table function arguments.
- Add tf_raster_contour table function to generate contours given x, y, and z arguments. This function is exposed in Immerse, but has additional capabilities available in SQL, such as supporting floating point contour intervals.
- Return file path and file name from tf_point_cloud_metadata table function.
- Previous length limit of 32K characters per values for none-encoded text columns has been lifted, and now none-encoded text values can be up to 2^31 - 1 characters (approximately 2.1billion characters).
- Support array column outputs from table functions.
- Add TEXT ENCODING DICT and Array<TEXT ENCODING DICT> type support for runtime functions/UDFs.
- Allow transient TEXT ENCODING DICT column inputs into table functions.
- Support COUNT_IF function.
- Support SUM_IF function.
- Support NTH_VALUE window function.
- Support NTH_VALUE_IN_FRAME window function.
- Support FIRST_VALUE_IN_FRAME and LAST_VALUE_IN_FRAME window functions.
- Support CONDITIONAL_TRUE_EVENT.
- Support ForwardFill and BackwardFill window functions to fill in missing (null) values based on previous non-null values in window.
- Fixes an issue where databases with duplicate names but different capitalization could be created.
- Fixes an issue where raster imports could fail due to inconsistent band names.
- Fixes an issue that could occur when DUMP/RESTORE commands were executed concurrently.
- Fixes an issue where certain session updates do not occur when licenses are updated.
- Fixes an issue where import/HeavyConnect data preview could return unsupported decimal types.
- Fixes an issue where import/HeavyConnect data preview for PostgreSQL queries involving variable length columns could result in an error.
- Fixes an issue where NULL elements in array columns with the NOT NULL constraint were not projected correctly.
- Fixes a crash that could occur in certain scenarios where UPDATE and DELETE queries contain subqueries.
- Fixes an issue where ingesting ODBC unsigned SQL_BIGINT into HeavyDB BIGINT columns using HeavyConnect or import could result in storage of incorrect data.
- Fixes a crash that could occur in distributed configurations, when switching databases and accessing log based system tables with rolled off logs.
- Fixes an error that occurred when importing Parquet files that did not contain statistics metadata.
- Ensure query hint is propagated to subqueries.
- Fix crash that could occur when LAG_IN_FRAME or LEAD_IN_FRAME were missing order-by or frame clause.
- Fix bug where LAST_VALUE window function could return wrong results.
- Fix issue where “Cannot use fast path for COUNT DISTINCT” could be reported from a count distinct operation.
- Various bug fixes for support of VALUES() clause.
- Improve handling of generic input expressions for window aggregate functions.
- Fix bug where COUNT(*) and COUNT(1) over window frame could cause crash.
- Fix wrong coordinate used for origin_y_bin in tf_raster_graph_shortest_slope_weighted_path.
- Speed up table function binding in cases with no ColumnList arguments.
- Support arrays of transient encoded strings into table functions.
Render queries no longer block parallel execution queue for other queries.
- The Immerse PostgreSQL connector is now generally available, and is joined by public betas of Redshift and Snowflake.
- New chart types:
- Contour chart. Contours can be applied to any geo point data, but are especially useful when applied to smoothly-varying pressure and elevation data. They can help reveal general patterns even in noisy primary data. Contours can be based on any point data, including that from regular raster grids like a temperature surface, or from sparse points like LiDAR data.
- Cross-section chart. As the name suggests, this allows a new view on 2.5D or 3D datasets, where a selected data dimension is plotted on the vertical axis for a slice of geographic data. In addition to looking in profile at parts of the atmosphere in weather modeling, this can also be used to look at geological sections below terrain.
- Representing vector force fields takes a step forward with the Wind barb plot. Wind barbs are multidimensional symbols which convey at a glance both strength and direction.
- Skew-T is a highly specialized multidimensional chart used primarily by meteorologists. Skew-Ts are heavily used in weather modeling and can help predict, for example, where thunderstorms or dry lightning are likely to occur.
- Initial support for window functions in Immerse, enabling time lag analysis in charts. For example, you can now plot month-over-month or quarter-over-quarter sales or web traffic volume.
- For categorical data, in addition to supporting aggregations based on the number of unique values, MODE is now supported. This supports the creation of groups based on the most-common value.
- Fixed an issue where a restarted server can potentially deadlock if the first two queries are executed at the same time and use different executors.
- Fixed an issue where COUNT DISTINCT or APPROX_COUNT_DISTINCT, when run on a CASE statement that outputs literal strings, could cause a crash.
- Fixes a crash when using COUNT() or COUNT(1) with the window function, i.e., COUNT(*) OVER (PARTITION BY x).
- Fixes an incorrect result when using a date column as a partition key, like SUM(x) OVER (PARTITION BY DATE_COL).
- Improves the performance of window functions when a literal expression is used as one of the input expressions of window functions like LAG(x, 1).
- Improves query execution preparation phase by preventing redundant processing of the same nodes, especially when a complex input query is evaluated.
- Fixes geometry type checking for range join operator that could cause a crash in some cases.
- Resolves a query that may return an incorrect result when it has many projection expressions (for example, more than 50 8-byte output expressions) when using a window function expression.
- Fixes an issue where the Resultset recycler ignores the server configuration size metrics.
- Fixes a race condition where multiple catalogs could be created on initialization, resulting in possible deadlocks, server hangs, increased memory pressure, and slow performance.
- Fixes a crash encountered during some SQL queries when the read-only setting was enabled.
- Fixes an issue in
tf_raster_graph_shortest_slope_weighted_pathtable function that would lead some inputs to be incorrectly rejected.
In Release 6.2.0, Heavy Immerse adds animation and a control panel system. HeavyConnect now includes connectors for Redshift, Snowflake, and PostGIS. The SQL system is extended with support for casting and time-based window functions. GeoSQL gets direct LiDAR import, multipoints, and multilinestrings, as well as graph network algorithms. Other enhancements include performance improvements and reduced memory requirements across the product.
- TRY_CAST support for string to numeric, timestamp, date, and time casts.
- Implicit and explicit CAST support for numeric, timestamp, date, and time to TEXT type.
- CAST support from Timestamp(0|3|6|9) types to Time(0) type.
- Concat (||) operator now supports multiple nonliteral inputs.
- JSON_VALUE operator to extract fields from JSON string columns.
- BASE64_ENCODE and BASE64_DECODE operators for BASE64 encoding/decoding of string columns.
- POSITION operator to extract index of search string from strings.
- Add hash-based count distinct operator to better handle case of sparse columns.
- Support MULTILINESTRING OGC geospatial type.
- Support MULTIPOINT OGC geospatial type.
- Support ST_NumGeometries.
- Support ST_ConvexHull and ST_ConcaveHull.
- Improved table reordering to maximize invocation of accelerated geo joins.
- Support ST_POINT, ST_TRANSFORM and ST_SETSRID as expressions for probing columns in point-to-point distance joins.
- Support accelerated overlaps hash join for ST_DWITHIN clause comparing two POINT columns.
- Support for POLYGON to MULTIPOLYGON promotion in SQLImporter.
- RANGE window function FRAME support for Time, Date, and Timestamp types.
- Support LEAD_IN_FRAME / LAG_IN_FRAME window functions that compute LEAD / LAG in reference to a window frame.
- Add TextEncodingNone support for scalar UDF and extension functions.
- Support array inputs and outputs to table functions.
- Support literal interval types for UDTFs.
- Add support for table functions range annotations for literal inputs
- Make max CPU threads configurable via a startup flag.
- Support array types for Arrow/select_ipc endpoints.
- Add support for query hint to control dynamic watchdog.
- Add query hint to control Cuda block and grid size for query.
- Adds an
echo alloption to heavysql that prints all executed commands and queries. - Improved decimal precision error messages during table creation.
- Add support for file roll offs to HeavyConnect local and S3 file use cases.
- Add HeavyConnect support for non-AWS S3-compatible endpoints.
LiDAR
- Add
tf_point_cloud_metadatatable function to read metadata from one or more LiDAR/point cloud files, optionally filtered by a bounding box. - Add
tf_load_point_cloudtable function to load data from one or more LiDAR/point cloud files, optionally filtered by bounding box and optionally cached in memory for subsequent queries.
Graph and Path Functions
- Add
tf_graph_shortest_pathtable function to compute shortest edge-weighted path between two points in a graph constructed from an input edge list - Add
tf_graph_shortest_paths_distancestable function to compute the shortest edge-weighted distances between a starting point and all other points in a graph constructed from an input edge list. - Add
tf_grid_graph_shortest_slope_weighted_pathtable function to compute the shortest slope-weighted path between two points along rasterized data.
Enhanced Spatial Aggregations
- Support configurable aggregation types for
tf_geo_rasterizeandtf_geo_rasterize_slopetable functions, allowing for AVG, MIN, MAX, SUM, and COUNT aggregations. - Support two-pass gaussian blur aggregation post-processing for
tf_geo_rasterizeandtf_geo_rasterize_slopetable functions.
RF Propagation Extension Improvements
- Add dynamic ray splitting to
tf_rf_prop_max_signaltable function for improved performance and terrain coverage. - Add variant of
tf_rf_prop_max_signaltable function that takes per-RF source/tower transmission power (watts) and frequency (MHz). - Add variant of
generate_seriestable function that generates series of timestamps between a start and end timestamp at specified time intervals.
- ST_Centroid now automatically picks up SRID of underlying geometry.
- Fixed a crash that occurred when ST_DISTANCE had an ST_POINT input for its hash table probe column.
- Fixed an issue where a query hint would not propagate to a subquery.
- Improved overloaded table function type deduction eliminates type mismatches when table function outputs are used downstream.
- Properly handle cases of RF sources outside of terrain bounding box for
tf_rf_prop_max_signal. - Fixed an issue where specification of unsupported GEOMETRY column type during table creation could lead to a crash.
- Fixed a crash that could occur due to execution of concurrent create and drop table commands.
- Fixed a crash that could occur when accessing the Dashboards system table.
- Fixed a crash that could occur as a result of type mismatches in ITAS queries.
- Fixed an issue that could occur due to band name sanitization during raster imports.
- Fixed a memory leak that could occur when dropping temporary tables.
- Fixed a crash that could occur due to concurrent execution of a select query and long-running write query on the same table.
- Disables render group assignment by default.
- Supports rendering of MULTILINESTRING geometries.
- Memory footprint required for compositing renders on multi-GPU systems is significantly reduced. Any multi-GPU system will see improvements, but is most noticeable on systems with 4 or more GPUs. For example, rendering a 1400 x 1400 image results in ~450mb of memory saved when using 8 GPUs for a query. Multi-gpu system configurations should be able to set the
res-gpu-memconfiguration flag value lower as a result, freeing memory for other subsystems. - Adds INFO logging of peak render memory usage for the lifetime of the server process. The render memory logged is peak render query output buffer size (controlled with the
render-mem-bytesconfiguration flag) and peak render buffer usage (controlled with theres-gpu-memconfiguration flag). These peaks are logged in the INFO log on server shutdown, when GPU memory is cleared viaclear_gpu_memoryendpoint, or when a new peak is reached. These logged peaks can be useful to adjust therender-mem-bytesandres-gpu-memconfiguration flags to improve memory utilization by avoiding reserving memory that might go unused. Examples of the log messages:- When a new peak
render-mem-bytesis reached:New peak render buffer usage (render-mem-bytes):37206200 of 1000000000 - When a new peak
res-gpu-memis reached:New peak render memory usage (res-gpu-mem): 166033024 - Peaks logged on server shutdown or on
clear_gpu_memory:Render memory peak utilization:Query result buffer (render-mem-bytes): 37206200 of 1000000000 Images and buffers (res-gpu_mem): 660330240 Total allocated: 1660330240
- Fixed an issue the occurred when trying to hit-test a multiline SQL expression.
- Dashboard and chart image export
- Crossfilter replay
- Improved popup support in the base 3D chart
- New Multilayer CPU rendered Geo charts: Pointmap, Linemap, and Choropleth (Beta)
- Control Panel (Beta)
- Redshift, Snowflake, and PostGIS HeavyConnect support (Beta)
- Skew-T chart (Beta)
- Support for limiting the number of charts in a dashboard through the
ui/limit_charts_per_dashboardfeature flag. The default value is 0 (no limit).
- Fixed duplicate column names importer error.
- Various bug fixes and user-interface improvements.
Last modified 17d ago