Release notes for unsupported releases
Archived Release Notes for v6.x
Adds support for POLYGON to MULTIPOLYGON promotion in the load table Thrift APIs and SQLImporter.
Fixes an issue that caused an intermittent KafkaImporter crash on CentOS 7.9.
Fixes an issue that cause incorrect results in multiple aggregation of date columns that include COUNT DISTINCT.
Adds support for limiting the number of charts in a dashboard through the ui/limit_charts_per_dashboard feature flag. The default value is 0 (no limit).
Adds a new set of log-based (request_logs, server_logs, web_server_logs, and web_server_access_logs) system tables.
Adds a new Request Logs and Monitoring dashboard.
Adds a new SHOW CREATE SERVER command, which displays the create server DDL for a specified foreign server.
Adds support for non-super-user execution of SHOW CREATE TABLE on views.
Adds a new ALTER SESSION SET EXECUTOR_DEVICE command, which updates the type of executor device (CPU or GPU) for the current session.
Adds a new ALTER SESSION SET CURRENT_DATABASE command, which updates the connected database for the current session.
Adds a new ALTER DATABASE OWNER TO command, which allows super users to change the owner of a database.
Extends the INSERT command to support inserting multiple rows at once/batch insert.
Add support for default values on shard key columns.
Add initial support for Window function framing, including support for BETWEEN ROWS clause for all numeric and date/time types, and BETWEEN RANGE clause for numeric types.
Enable group-by push down for UNION ALL such that group-by and aggregate operations applied to the output of UNION ALL are evaluated on the UNION ALL inputs, improving performance.
Add support for LCASE (alias for LOWER), UCASE (alias for UPPER), LEFT, and RIGHT string functions
Adds a new trim_spaces option for delimited file import.
(BETA) Adds data import/COPY FROM support from Relational Database Management Systems and Data Warehouses using the Open Database Connectivity (ODBC) interface.
Initial support for CUDA streams to parallelize GPU computation and memory transfers.
Increase per-GPU projection limit with watchdog enabled from 32M to 128M rows to take advantage of improvements in large projection support in recent releases.
Add new SHOW FUNCTIONS and SHOW FUNCTIONS DETAILS commands to show registered compile-time UDFs and extension functions in the system and their arguments, and SHOW RUNTIME FUNCTIONS [DETAILS] and SHOW RUNTIME TABLE FUNCTIONS [DETAILS] to show user-defined runtime functions/table functions.
Support timestamp inputs and outputs for table functions.
Advanced Analytics
Add tf_compute_dwell_times table function, that given a query input input with entity keys and timestamps, and parameters specifying the minimum session time, minimum number of session records, and max inactive seconds, outputs all unique sessions found in the data with the duration of the session (dwell time).
Add tf_feature_self_similarity table function, that given a query input of entity keys/IDs, a set of feature columns, and a metric column, scores each pair of entities based on their similarity, computed as the cosine similarity of the feature column(s) between each entity pair, which can optionally TF/IDF weighted.
Add tf_feature_similarity table function, that given a query input of entity keys, feature columns, and a metric column, and a second query input specifying a search vector of feature columns and metric, computes the similarity of each entity in the first input to the search vector based on their similarity, computed as the cosine similarity of the feature column(s) for each entity with the feature column(s) for the search vector, which can optionally be TF/IDF weighted.
Fixed an issue where some join queries on ODBC-backed foreign tables can return empty result sets for the first query.
Fixed an issue where append refreshes on foreign tables backed by delimited or regex-parsed files ignore file-path filter and sort options.
Fixed a crash that can occur when very large dates are specified for the refresh_start_date_time foreign table option.
Fixed a crash that can occur when a foreign table’s data source is updated within a refresh window.
Fixed an issue where databases owned by deleted user accounts are not visible, and adds a restriction that prevents dropping users who own databases.
Fixed an issue where joins on string dictionary-encoded columns would hit spurious none-encoded string translation
Fixed issue with certain UNION ALL query patterns, such as UNION ALL containing logical values.
Disabled KEY_FOR_STRING for UNNEST operations on string dictionary-encoded columns, to prevent a crash.
Fixed an issue where logged stats for raster imports could overflow.
Fixed an issue where joins on synthetic tables (for example, created with a VALUES statement or table function without an underlying table) could crash.
Fixed an issue where require checks used on string dictionary inputs to a table function could crash.
Fixed a crash and/or wrong query results that can occur when a decimal literal is used in a nested query.
Fixed a potential crash when attempting to auto-retry a render immediately after an OutOfGpuMemory exception is thrown. This crash can occur only if the render-oom-retry-threshold configuration option is set.
Fixed a regression where polygons with transparent colors are rendered opaque.
Corrects an issue with point/symbol rendering with explicitly Vega projections where the projection was not being updated when panned/zoomed if the query did not change.
Significant improvements in hit-testing consistency and stability when rendering queries with subqueries, window functions, or table functions.
Font size controls.
Borders and Zebra Striping in Table charts.
Justify content in Table charts.
Customization polygon border control.
Allow measure date formatting for table charts.
Extend y-axis on Vega combo charts to end at the next whole value past the highest data point.
Add layer visibility toggle to kebab dropdown on multi-layer raster charts.
Made unsaved changes modal less aggressive.
Custom Source Table Functions Browser
Don’t show unsaved warning modal after adding default filter set.
(BETA) PostgreSQL connector.
Allows maxBounds to be set in servers.json.
Toggle dashboard unsaved when updating annotations.
Dashboard save state behavior fixes.
Table Chart order by group keys when present.
Use key_for_string when ordering by known dictionary measures/dimensions.
Add default formatting for date/time on table chart.
Add admin feature flag to hide key manager.
Customizable polygon border color and existing border bug fixes.
Cannot append to table using PostgreSQL connector.
Building a raster chart with the layer visibility toggle feature flag enabled causes a crash.
Support for fast string functions on dictionary-encoded text columns (the default), including LOWER, UPPER, INITCAP, TRIM/LTRIM/RTRIM, LPAD/RPAD, REVERSE, REPEAT, SUBSTRING/SUBSTR, REPLACE, OVERLAY, SPLIT_PART, REGEXP_REPLACE, REGEXP_SUBSTR, AND CONCAT (||). The output of these expressions can be chained, grouped-by, and used in both the left and right side of join predicates.
Support for fast string equality/inequality operations without the previous requirement of watchdog disablement when the two columns do not share dictionaries.
Support for fast case statements with multiple text column inputs that do not share dictionary-encoded strings.
Support for ENCODE_TEXT to encode none-encoded strings, which can then be grouped on and manipulated like dictionary-encoded strings. This operator is not intended for interactive use at scale but instead for ELT-like scenarios. Use the new server flag watchdog-none-encoded-string-translation-limit to set the upper cardinality allowed for such operations (1,000,000 by default).
Support for UNION ALL is enabled by default, and now works across string columns that do not share dictionaries with significantly better performance than in the previous release.
Window functions now support expressions in the PARTITION BY and ORDER clauses.
Support for subqueries in CASE statement clauses.
SHOW USER DETAILS is changed to only list those users with access to the currently-selected database. Previously, all users on the HeavyDB instance would be listed; this is still available to superusers with SHOW ALL USER DETAILS.
10X improvements in initial join performance (including geo joins) through faster, parallelized hash table construction, removing redundant inter-thread hash table computation.
Improved join ordering to avoid loop joins in certain scenarios.
Parallel compilation of queries as well as inter-executor generated code increases concurrency and throughput in common, Immerse-driven scenarios by up to 20%. Also decreases latency for a single user interacting with dashboards or issuing SQL queries in a way that required new plans to be code-generated.
New result set recycler allows query substeps (expensive in subqueries) can be cached using SQL hints ( /*+ keep_result */), dramatically improving performance where the subquery is reused across multiple queries (for example, in Immerse) and only outer steps of the query vary.
The default for the header option of COPY TO to a CSV/TSV file has been changed from 'false' to 'true'.
Faster dictionary map in StringDictionaryProxy, accelerating various string operations involving transient entries.
Arrow execution endpoints now use multiple executors and can run concurrently like queries issued to the Thrift endpoints.
Addition of sparse dictionary output capability for Arrow queries, which automatically creates a subset of a string dictionary to send via Arrow when it detects that it is faster than sending the full, unfiltered dictionary. This provides orders-of-magnitude better server- and client-side performance and scalability for common cases where large dictionary-encoded text columns are filtered or top-k sorted such that only a small subset of dictionary entries are needed in the result set.
ST_INTERSECTS now can operate directly on top of compressed (the default) coordinates, leading to 2-3X increase in speed.
New table function framework allows for both system and user-defined table functions. Table functions can run on both CPU and GPU and are designed for efficient, scalable execution of custom algorithms in-situ on data that might be hard or impossible to implement in SQL.
Support for generate_series table function (similar to Postgres) for easy and fast integer series generation, particularly useful for left joins against binned tables to fill in gaps, whether for visualization or downstream operations like window functions, and generate_random_strings for generation of string columns of a user-defined size and cardinality.
Support for geo_rasterize and geo_rasterize_slope table functions to efficiently bin vector data into gap-free bins, with the optional ability to fill in null values, apply box blur, and compute slope and aspect ratios
Initial support for HeavyRF, a module that allows for real-time, ray-based computation of signal propagation, taking inputs of both terrain data and real or hypothetical signal sources.
Beta support for Python-defined scalar (row-level) and tabular User Defined Functions (UDFs and UDTFs), using the RBC library to translate Numba python code into LLVM IR that is JITed into query execution code for fast, scalable, custom user-defined capabilities.
Complete redesign and rewrite of Parquet import to one that is more robust, efficient, and performant.
Adds support for import from regex parsed files on either the server file system or S3 using the COPY FROM command.
The geo and parquet WITH options of COPY FROM have been deprecated and replaced by source_type. Using the deprecated syntax generates the following:
Deprecation Warning: COPY FROM WITH (geo='true') is deprecated. Use WITH (source_type='geo_file') instead.
Update any scripts you have to replace the deprecated syntax with the new syntax. For more information, see CSV/TSV Import.
(BETA) Adds support for import from RDMS/data warehouses using the COPY FROM command.
Adds system table support.
A new default information_schema database contains 10 new system tables that provide information regarding CPU/GPU memory utilization, storage space utilization, database objects, and database object permissions.
New system dashboards that enable intuitive visualization of system resource utilization and user roles and permissions.
Support for Zarr and NetCDF raster file import.
You can now import raster files with ground control points geospatial references.
Support for file path filtering, globbing, and sorting when importing geo and raster files.
Improved error messaging when attempting to save a dashboard that uses a duplicate dashboard name.
Support for connections to delimited files on either the server file system or S3. S3 support includes an option to use the S3 Select API, which provides better performance but with limitations on supported column types.
Support for connections to Parquet files on either the server file system or S3. HeavyConnect leverages Parquet metadata to provide efficient data access and row group-level filter push down.
Parquet column type coercion. Convert Parquet column types to more memory-efficient HeavyDB column types for use cases that guarantee no loss of information.
Connections to regex parsed files on either the server file system or S3. This enables you to query unstructured text files, such as logs, by specifying regular expression patterns that extract components of the text files into table columns.
(BETA) Support for connections to Relational Database Management Systems and Data Warehouses, leveraging the Open Database Connectivity (ODBC) interface to provide seamless access to data.
(BETA) ODBC column type coercion. Use HeavyConnect to convert ODBC column types to more memory-efficient HeavyDB column types for use cases that guarantee no loss of information.
Support for scheduled data refreshes. Specify a start date time and interval at which connected data gets refreshed.
Adds support for disk level caching. By default, data fetched by HeavyConnect are cached at the disk level in addition to normal CPU/GPU level caching. This provides better overall query performance for network based connections, such as S3, and systems with limited CPU/GPU memory capacity. Disk cache size and level can be set through HeavyDB server configuration.
Adds support for file path filtering, globbing, and sorting for Parquet, delimited, and regex parsed file use cases.
Complete redesign and rewrite of the Parquet detect_column_types Thrift API. The Parquet detect/data preview feature is now more robust, efficient, and performant.
Change to query interrupt mechanism allowing certain classes of queries such as loop joins to be easily and quickly interrupted.
Fixed crash that could occur with joins on predicates that had functions on the left hand side expression, i.e. geoToH3.
Fix crash that could occur with Arrow queries that did not return results.
Avoid building metadata for empty result sets.
Fixes a crash that can occur when executing queries on GPU that involve a baseline group by and variable length column projections.
Fixes some table query concurrency bottlenecks. Previously, queries such as INSERT, TRUNCATE, and DROP TABLE required system wide locks to execute and would therefore block execution of other unrelated queries. These kinds of queries can now be executed concurrently.
Fixes a crash that can occur on server restart when the disk cache is enabled and tables with cached data are deleted.
Fixes a crash that can occur when the max_rows table option is altered for an empty table.
Fixes an issue in the JDBC driver where tables from multiple databases are listed even when a single database is specified.
Fixes an issue where raster POINT column type import would incorrectly throw an exception.
Fixes a crash that can occur when restoring a dump for a table with previously deleted columns.
Updates the export COPY TO command to include headers by default.
Removes the file_type parameter from the create_table Thrift API. This parameter was not used.
Fixes a crash that can occur when executing SQL commands containing comments.
Fixed the setting for default database (DEFAULT_DB) being ignored in a SAML login for a user who already exists.
The OpenGL renderer driver has been fully removed as of this release. Vulkan is the only available driver and enables a more modern, flexible API. As a result, the renderer-use-vulkan-driver program option has been removed. Remove any references to that program option from your configuration files. For more on the move to the Vulkan driver, see Vulkan Renderer.
A novel polygon rendering algorithm is now used as the default when rendering polygons. This algorithm does no triangulation nor does it require “render groups” (a hidden column to assist the old polygon rendering algorithm). However, the render groups column is still added on import as a fallback. See Importing Geospatial Data for more on render group deprecation.
You can now hit-test certain render queries with subqueries more effectively. For example, if the subquery is only used for filter predicates, renders should now be sped up and hit-testing more flexible.
Render times are now being logged correctly (“render_vega-COMPLETED nonce:2 Total Execution: (ms), Total Render: (ms)”). The execution time and render time were incorrectly logged as 0 in Releases 5.9 and 5.10
Fixes a regression introduced in Release 5.10.0 when hit-testing an Immerse cohort-generated query. The hit-test would result in an error such as the following: “Cannot find column in hit-test cache for query …”
Resolves a crash when trying to hit-test render queries with window functions or cursorless table functions.
Fixes an issue where a multi-layer, multi-GPU render with a poly or line mark as the first layer can result in ghosting artifacts if the query associated with that layer resulted in 0 rows.
Fixes an issue when switching between a density accumulation scale with an auto-computed range (via min/max/+-1stStdDev/+-2ndStdDev) to a scale with an explicitly defined range. In this case, the explicit case was not reflected.
Removes a legacy constraint that prevented you from rendering a query that referenced one or more tables with more than one polygon/multipolygon column.
Improved speed of server interface using the Thrift binary protocol.
Data Manager has been redesigned to support HeavyConnect via S3, server file uploads, and expanded raster file support.
Introduced the new Gauge chart type.
Introduced a Welcome Panel and Help Center menu.
Rebranded interface for HEAVY.AI. Updated styles for the default dark and light themes.
Added option to toggle the legend on the New Combo chart.
Added configuration option for setting the default chart type.
Added configuration option for hiding specified chart types.
Added auto-selection of geo columns and measures on geo chart types.
Adjusted maximum bins for larger Top-N groups.
Added support for cross-domain configuration without SSL.
BETA: Added filter support for global custom expressions.
BETA: Introduced the new iframe chart type.
BETA: Introduced Arrow transport protocol for a limited number of chart types.
Fixed various minor UI and performance issues.
Fixed parameter creation from dashboard title in Safari browser.
Fixed displaying of the Jupyter logo when integration is unavailable.
Archived release notes for release 4.x
4.8.0 | 4.7.1 | 4.7.0 | 4.6.1 | 4.6.0 | 4.5.0 | 4.4.2 | 4.4.1 | 4.4.0 | 4.3.0 | 4.2.0 | 4.1.3 | 4.1.1 | 4.1.0 | 4.0.2 | 4.0.1 | 4.0.0
OmniSci release 4.8.1 contains several fixes.
Fixed Issues
Fixed an issue that occurred when creating a table with an invalid shared dictionary column.
Resolved an issue where a CSV import with a row delimiter inside a quoted string would cause a premature line break, resulting in incorrect imported values for all subsequent rows.
Resolved an issue starting OmniSciDB inside a Docker image on legacy versions of Docker (affects version 18.06 and prior).
Fixed an issue affecting importing data from AWS S3 when the import statement was malformed.
Fixed an issue where importing geospatial data using the SQLImporter would fail after importing 10,000 records.
Fixed an issue affecting multi-step distributed queries over empty tables.
OmniSci release 4.8.0 contains several new features and fixes.
Major Changes and Known Limitations
Jupyter notebook integration is an experimental feature. See Jupyter Integration.
Thrift methods get_frontend_view, get_frontend_views, create_frontend_view, delete_frontend_view, get_table_descriptor, and get_row_descriptor are deprecated and no longer function. If you use one of these methods, you receive an error message describing how to migrate to the new API calls.
Arrow version upgraded to 0.13.0.
Minimum supported Nvidia CUDA driver is now 410.48.
Significant additional controls available in the logging framework. See Logs and Monitoring. Note that the individual file names for the logs have changed from <service_name>.<machine_name>.<severity>.<timestamp> to <service_name>.<severity>.<timestamp>.log.
When running an encrypted server configuration in Docker containers there is a current limitation that the aggregator omnisci_server and the omnisci_web_server must be running in the same Docker container and they cannot be started with the startomnisci script, which is the default. The use of a Docker COMMAND is required to start the services. For example:
For more information, contact OmniSci support.
New Features and Improvements
Added support to log user's roles information during login event.
Added support to allow communication among distributed system components to use encrypted binary connections.
Added OPTIMIZE support for sharded tables, allowing administrators to create more optimal schemas.
Added support for ST_Point geo constructor.
Added new flag is_super (whether current session is super user) to session info returned from Thrift get_session_info.
Improved performance of some high-cardinality group-by queries.
Improved performance of ST_Contains by adjusting the bounding box heuristic.
Improved error messages for unsupported function/type combinations.
Fixed Issues
Fixed issue with ODBC connector not working with PowerBI.
Fixed segfault during COPY FROM direct to leaf node on distributed setups.
Fixed issue with Check Failure related to a query on a distributed sharded configuration.
Improved signal handling and shutdown procedures for OmniSciDB server.
Fixed an issue in geo join due to table join reordering.
Fixed an issue where a non-Admin user could set a license key.
Fixed an error resulting from running the same Vega render twice in quick succession.
Fixed issue with a program error from a query that casts a TIMESTAMP(0) to TIMESTAMP(3).
Added support for more graceful shutdown of OmniSciDB when receiving OS signals.
Fixed issue with web server incorrectly switching to SAML mode if a session ID cookie is present.
Fixed issue for server program errors on GROUP BY queries with nullable SMALLINT and TINYINT keys.
Fixed issue where empty string as group name causes server program error.
Fixed issue where the startomnisci script no longer works on CentOS.
Fixed issue where ROUND_TO_DIGIT(x, y) (all data types) resulted in a server program error.
Fixed edge case where Log10(NULL)/ Log(NULL)/LN(NULL)/EXP(NULL) was being computed and resulted in a number.
Fixed issue where rendering join queries with multiple rowids produced the wrong results.
Fixed issue where \detect did not appear in \h listing in omnisql.
Fixed issue where requesting Arrow results on CPU (via select_ipc) results in the query executing on CPU, rather than just the resultset being located in CPU memory.
Fixed issue to workaround where cuDF does not recognize arrow::date32 type, using arrow::date64 for gpu buffers.
Fixed issue where query returns different results on distributed and single-node.
Fixed issue where hit testing on Line map with color measure selection throws an error.
Fixed issue with a render resulting in CHECK failure from a Choropleth Vega query.
Exception for a query and subquery with differing limits.
Major Changes
Line charts are deprecated in 4.8. While existing Line charts continue to function normally on dashboards, new edits to those charts cannot be saved. OmniSci recommends upgrading Line charts to Combo charts to take advantage of the latest feature improvements.
Over the coming releases, OmniSci will stop developing new Immerse features for Internet Explorer. We strongly recommend all customers switch to a non-IE browser to experience the latest Immerse features.
New Features and Improvements
Immerse now directly integrates with JupyterLab. You can send SQL queries to Jupyter from SQL Editor, or access it directly from top level navigation. See Jupyter Integration.
Combo chart supports focus zoom. Moving the mouse wheel up and down zooms the chart in/out, while holding an Option or Shift key while scrolling pans the chart from right to left. Clicking the Reset button on hover resets the chart to the original timeframe.
You can visualize Immerse in a new Dark Mode theme, accessible through account settings.
Immerse supports importing and visualizing millisecond timestamp data.
Immerse lets you duplicate charts within dashboards.
Immerse lets you export and import dashboard metadata within Immerse.
Immerse now pops up a modal when you are using a non-WebGL supported browser, informing you of the reduced experience.
+ and - buttons appear on geospatial charts as zoom controls.
Offline maps now include a Minimalist base map layer at the country level.
Custom base map layers are now defaulted first for easier selection.
Fixed Issues
Updated combo chart's dynamic top 5 colors to consistently show the five series with the most data volume.
Fixed issue where pop-ups on Choropleth were failing to appear for views created from multiple data sources.
Fixed custom colors resulting in an add/remove loop in Combo chart, and added a toggle to control whether the chart is in dynamic or fixed mode.
Incorrect global filters now error gracefully rather than remain part of the active filters list.
Fixed issue where Choropleth was representing the Other category always as grey; now the Other category correctly shows the selected color.
Fixed issue where error messages were showing up during dashboard duplication.
Fixed issue where key manager shows key issuer instead of client name/company.
Fixed issue where case-different, same text columns were causing a "table does not exist" error.
Fixed issue where mapd-connector running under Node.js was incorrectly returning null for some functions.
Fixed issue causing blank page when Add Chart button is pressed during dashboard load.
Fixed issue where raster chart gaps showed when a Combo chart was present on the dashboard.
Fixed issue where x-axis scale on a Combo chart is incorrect when the relative date filter is applied in addition to a second measure.
Fixed issue where Bar chart was incorrectly applying formatting to color measures and dimensions.
Fixed issue where filter label and dropdown menu were unconnected during the scroll action.
Fixed issue where Combo chart binning label was incorrect.
Fixed issue where certain custom colors were not discoverable in the Combo chart search dropdown.
OmniSci release 4.7.1 contains several fixes.
Fixed an issue where setting a dictionary encoded 16 or dictionary encoded 8 column as the shard key could cause a program error during data import.
Fixed an issue where OmniSci allowed insertion to a table using a select query where the precision of column did not match, resulting in an incorrect result. Columns must match exactly for insertion from a select query.
Fixed an issue where packed colors were broken with SPIR-V.
Fixed an issue where a literal projection coupled with an aggregate query could cause a program error.
Fixed an issue where adding an ORDER BY clause to a query executed in distributed mode could return 0 results, even if the query without the ORDER BY returned results.
Fixed an issue to allow time-related columns (TIMESTAMP, DATE, TIME) to be used as shard keys for sharded tables.
Added support for INSERT statements on sharded tables in distributed mode.
Fixed an issue that might occur with a join between two date columns where the inner table is empty.
Fixed an issue where decimal fixed encoding columns failed to load when using a load_table_binary_columnar endpoint.
OmniSci Immerse
Fixed an issue causing an increased margin on the right side of geospatial charts.
Fixed an issue causing errors during dashboard load.
OmniSci release 4.7.0 contains several new features and fixes.
See also the OmniSci Blog.
OmniSci Core has been renamed to OmniSciDB throughout the product and in the documentation.
Thrift methods get_frontend_view and create_frontend_view are deprecated.
Default user and database names are changed. Note and update your scripts accordingly:
The default username has been changed from mapd to admin. All new data directories created with initdb create a superuser with the username admin. Existing usernames are not migrated.
The default username for all utilities is now admin. Existing users must override the default username in connection strings / command-line arguments of utilities (for example, omnisql --user mapd).
ALTER USER <> RENAME TO <> is now supported, allowing you to rename the existing mapd user to admin if you do not want to use username overrides in your applications.
After rename, any sessions connected to the renamed database are invalidated and need to log in again.
The default database has changed from mapd to omnisci. All new data directories create a default database named omnisci instead of mapd.
The default database for all utilities is now named omnisci. Similar to the username change, you can override the default database: for example, omnisql --db mapd for existing installations.
ALTER DATABASE <old_db> RENAME TO <new_db> is now supported, allowing you to rename an existing mapd database to omnisci for convenience.
After rename, any sessions connected to the renamed database are invalidated and must log in again.
OmniSci 4.7 fixes an issue with OPTIMIZE TABLE WITH VACUUM, which previously could cause an incorrect entry in the storage for the optimized table. If you have used OPTIMIZE TABLE WITH VACUUM, you should export and then re-import or reload your data.
OmniSci 4.7 is the last version to support NVIDIA driver version 386. Future releases will require NVIDIA driver version 410 or higher.
Any blank passwords (that is, nothing) are reset to a random 72-character string. This is required to prevent any accidental blank passwords enabled in the system. Previous versions automatically created blank passwords when logging on with LDAP or SAML, which creates a conflict with the new fallback to local login support.
ODBC usage requires the license on the connecting server to be Enterprise enabled.
\clear_gpu and \clear_cpu now require superuser privileges to run.
Users that have access to a database through a role are now returned as part of a get_users call for a specific database. More users are now visible in shared Immerse dashboard dialogues.
Apache Calcite version is now 1.19.
For geojoin-configured Choropleth charts, Immerse now generates a new poly render query that no longer uses the server poly cache. The poly cache may be deprecated completely in a future release. The change includes better distribution of poly data across all available GPUs, causing an overall memory footprint improvement.
TEXT ENCODING DICT(n) columns cannot be encoded as less than 32 if they are to be used as a SHARD KEY on a sharded table. This restriction will be addressed in a later release.
New Features
OmniSciDB now supports INSERT TABLE AS SELECT (ITAS) on both single node and distributed, including sharding support.
Joins on bucketed date/time are now bucket-size aware and more memory-efficient.
OmniSciDB now supports Window functions. Empty Window function partitions are currently not supported; that is, all Window functions must include a PARTITION BY key in the OVER clause. Also, OmniSci recommends that you use Window functions over aggregate query outputs at this time (instead of raw data). Future releases aim to improve scale and performance and remove these restrictions; see the documentation for details on what is supported.
The default username is now admin.
OmniSci DB now supports ALTER USER RENAME, which allows you to rename users.
OmniSciDB now supports Domain-based Authentication, allowing for multiple login modes (SAML, LDAP, and local) depending on user ID. This allows you to create OmniSci admin accounts outside of LDAP and SAML. The server first checks for SAML and then for LDAP. You configure this in the server configuration file. In addition, the server now allows authorization to fall back to local OmniSci authorization if LDAP or SAML (or both) are enabled, but fail. The --allow-local-auth-fallback controls this behavior, and is set to false by default.
OmniSciDB now supports Distributed Polygon Rendering, allowing for much greater scale in rendering. For Enterprise customers running in Distributed mode, the Choropleth chart in Immerse now supports backend rendering by default.
ODBC allows for cookies on http and https connections; relevant only if a load balancer is used.
OPTIMIZE TABLE now recomputes metadata for dictionary-encoded string columns, supporting updates and deletes on dictionary-encoded string data.
Performance improvements:
(Experimental) Sorting columns at import time (a user operation) can result in improved performance by up to 3.5x on query patterns that involve filters on those columns.
Improvements in distributed query performance involving joins. Distributed execution mode requires that your tables are structured correctly for optimal query execution. Immerse queries are formulated to work in Distributed mode, but customers using OmniSciDB with custom applications need to ensure that any table on the inside of a join is either replicated or the join relation is on a sharded column in both tables, unless both sides of the join relation are the result of an intermediate query step.
View optimization is now on by default. OmnisciDB optimizes all view queries through RA rewrite. This should result in a noticeable reduction in the number of steps required by many view queries.
Improved performance in distributed mode from more efficient result set serialization.
Improved performance of multistep queries in which subsequent steps are projection nodes.
Improved performance of filter queries in which the filtered column comes from the inner table in an inner join relation.
Added support for SQL WITH syntax in poly render queries.
Added support for IF NOT EXISTS and IF EXISTS to database DROP and CREATE statements, respectively.
Added new logging messages for monitoring query execution in Distributed mode.
Fixed Issues
Fixed SAML-configured login that made regular access to other utilities awkward or restrictive.
Improved performance by eliminating runtime loads of fixed-length array sizes that slow down POINT geo functions.
Added cookie support to ODBC driver.
Fixed a render error in Vega runtime that dumped the entire FRAGMENT SHADER code in the log.
Fixed an issue where a scalar sub-query returned multiple rows exception was thrown while validating scalar sub-queries.
Fixed an issue where update storage layer erroneously used std::numeric_limits< double >::min() when initializing the max range of a nonnegative number.
Fixed a program error resulting from a multi-level join query.
Fixed a program error on hash tables with more than 2B entries.
Fixed an issue where a cast from BOOLEAN to BOOLEAN was not supported, resulting in an error.
Fixed a program error resulting from tables with a negative epoch.
Fixed an issue where the importer did not skip rows with incorrectly sized fixed-length arrays.
Fixed an issue where an exception resulted when \clear_gpu was run after any render.
License configuration parameter now allows quotes.
Fixed an issue resulting in incorrect results on CPU for group by query with CASE.
Fixed a program error that resulted from a COPY FROM on Parquet with POINT geo type.
Fixed a render error when going from a non-in-situ query to a projection query that uses multiple GPUs.
Fixed a render error when updating from min/max -> stddev density accum renders.
Fixed an issue where shard count requested with CREATE was different from what was reported with \d in Distributed mode.
Fixed an issue where lack of rowid in projection queries negatively impacted Vega property updates.
Fixed program error when shard count exists, but shard key does not.
Fixed an issue with distributed line rendering not respecting Vega line format with scalar columns.
Fixed an issue involving joins on a value or null. For example, a.x = b.x or a.x is null and b.x is null.
Fixed a regression with multilayer render on Distributed systems using Vega xforms and aggregation/projection queries.
Fixed importer program error resulting from shapefile with null geo features.
Fixed program error during SAMPLE on a geo join in nondistributed mode.
Fixed program error with GROUP BY with BIGINT and INT columns.
Fixed server program error resulting from CASE WHEN/THEN string test select join with INT, FLOAT, and SMALLINT columns caused a server check and program error.
Fixed an issue where a user could not be dropped if the username contained a period.
Fixed program error with geo function filters under certain circumstances.
Fixed an issue with JOIN WITH subquery not returning results in a Distributed environment.
Fixed an issue where a non-superuser user was not included in get_users Thrift call when the user has ACCESS/SELECT database rights.
Setting a license key through Immerse or omnisql now works in HA mode.
Invalid SQL casts now produce more meaningful error messages.
Fixed a program error involving render query hit testing in Distributed mode.
Fixed a Vega bug involving dictionary-encoded string columns from an aggregate target in an ordinal scale.
Improved performance of projecting queries with geospatial targets in Distributed mode.
Calcite server no longer has a program error if the user passes in an empty string.
Known Limitations
Distributed Poly rendering on Choropleth charts introduced in 4.7 has the following known limitations (Enterprise Edition):
With the server-side poly cache no longer being used, some render performance issues with geojoin configured Choropleth charts can occur. If the dimension table for the geojoin (that is, the poly table) has a large number of rows, you might see some performance degradation in release 4.7. Internally, tables <100K polygons tend to perform equally to the cache, 100K-500K rows sees mild degradation, and >500K rows can see more substantial degradation. Performance is ultimately dependent on the total number of vertices (not rows).
Currently, neither Immerse nor the server can be configured to use the poly cache. This limitation impacts Immerse users only. Custom clients doing server-side polygon rendering using join queries can continue using existing Vega code and use the poly cache. OmniSci recommends that you switch to the new query format; instructions for doing so will be included in a future release. If you prefer to make these changes beforehand, contact OmniSci support.
A known issue with Distributed poly rendering and replicated poly tables can result in overdraw of the polygons from each leaf in projection queries. This might not impact the final render if the polygons are rendered fully opaque. If they are rendered with transparency, the final render is less transparent than expected due to overdraw.
Line chart in Immerse is scheduled for deprecation in a future release, to be replaced by the Combination Bar/Line (Combo) chart.
For geospatial charts, changes in release 4.7 introduce an increased margin on the right side. This is scheduled to be fixed in a future release. As a workaround, resize the window to remove the margin.
You must upgrade to Windows 10 1903 and Edge 44.18362 to use Edge with OmniSci. If you choose not to upgrade, you can use an alternate browser with OmniSci.
New Features
Visual Data Fusion supports plotting data from multiple data sources on the same Combo chart, which also enables cross-filtering with other charts. This was previously available only on geospatial charts.
Filtering on an aggregated value is supported in Pointmap and Scatter Plot charts.
Data import status provides time estimation and information on import success.
Server-rendered Choropleth charts are supported in Distributed mode.
Data import popovers provide help text on file formats available for import.
Binning limits on Line and Combo charts are increased from 400 bins to 1000 bins. Note: When binning granularity is set to auto mode on a Line or Combo chart created prior to version 4.7, the reported bin granularity on the chart (for example, Hour) might not match the actual bin value (for example, Day). Workaround: Recreate the chart in version 4.7.
Fixed Issues
Fixed an issue for applying a global timestamp filter to a Combo chart where the displayed dimension is not of TIMESTAMP data type.
Fixed an issue where user did not appear in list for sharing despite having database permissions.
Fixed issues with Combo chart related to setting min/max values on the x and y axes.
Fixed an issue where the locked icon ona Combo chart disappeared, despite having a fixed value set on the axis.
Fixed an issue where the top bin of binned data was missing values due to rounding discrepancies.
Fixed an issue where Combo chart did not bin correctly on cross-filter.
Fixed an issue where a new license could not be pasted in after an previouis license expired.
Fixed an issue in Bar chart where sorting by measure did not update the chart.
Fixed an issue to better address auto-binning in Combo chart.
Fixed an issue in Stacked Bar chart for setting custom colors.
Fixed an issue in Line chart for setting an incorrect aggregation due to color dimension.
Fixed an issue in Combo chart for automatic axis labels.
Fixed issues with Choropleth chart error handling.
Fixed chart-switching error issues in Pointmaps.
Removed Append Data button for users without INSERT permission.
Fixed x-axis label issue in Stacked Bar charts.
Fixed tooltip sorting in Combo charts so that the ordering is consistent.
Removed default brush on focus chart after a range chart brush is set.
Fixed an issue causing an error state for Append Data using Internet Explorer 11.
OmniSci release 4.6.1 contains several fixes.
Fixed Issues
Fixed an issue where a grouped pointmap with multiple "SAMPLE" measures could cause a program error.
Fixed an issue where a JOIN on the result of an unnest operation caused a program error.
Fixed an issue where QueryMemoryInitializer was not passed safely to the JOIN loop builder.
Fixed an issue where queries retried on CPU with subqueries that had already been executed would cause a program error.
Removed buffer size limitations between cluster aggregator and leaves.
Fixed an issue where sql_execute_df would throw an exception when encountering a BOOL column.
Restored SQL execute timings to std_log output.
Fixed an issue with intermittent server program errors with CUDA error code 201 in egl compositor.
Improved error resolution when an error is thrown during the build of a poly cache.
Fixed an issue with COPY TO behavior when executing in HA mode.
OmniSci release 4.6.0 includes several fixes and performance improvements.
See also the OmniSci Blog.
Syscatalog/catalog separation has changed directory layout. System catalog/system database is now called omnisci_system_catalog and is in a separate folder in catalogs. The default database is still named mapd, but it is a standalone database.
The --enable-access-priv-check flag has been removed. Access privilege verification is now required and enabled by default for OmniSci version 4.6.0 and higher.
CTAS (CREATE TABLE AS SELECT) is now supported on distributed installations.
In release 4.6.0, avoid using CTAS to create tables with very large row counts (for example, in excess of 10 million rows). In such cases, the leaf-aggregate communication via Thrift imposes size limits (2GB), which OmniSci is working to address.
OmniSci supports updates on variable length columns, specifically arrays and strings with encoding NONE.
Added partition='replicated' option for COPY FROM command on geo tables [specified via option WITH (geo = 'true')].
Further foundational improvements in efficient columnar output, allowing for better memory efficiency with larger result sets. This is behind a configuration flag and will be turned on by default in a future release.
Added support for importing from gdb(Esri geodatabase) format.
Added support for importing from Parquet files, from both local and S3, including auto-detect capabilities on omnisql. Immerse will support import of Parquet files in a future release.
Improved string dictionary performance from use of rk_hash fingerprints. This is behind a configuration flag and will be turned on by default in a future release.
Added support for custom S3 endpoints, allowing for non-AWS S3-compatible storage systems to be used as input sources.
Added support for Single Sign-on using Okta (SAML 2.0 Identity provider) for authentication.
There is a known limitation with users set up via Okta, where such users cannot be superusers. OmniSci will address this in an upcoming release.
\l command to list users' database access is now available to non-Superusers.
Added support for PKI-based authentication starting with the ODBC API.
Users can now be assigned a default database as part of CREATE or ALTER USER command options.
SQLImporter now supports HTTPS and HTTP besides the binary protocol.
Lifted limit on dictionary entries from 1B to 2B, allowing for larger string dictionaries.
Logging format now standardized to allow for better import of OmniSci logs in a delimited file format (some minimal post-processing is required).
Improved performance of between 25-35% on string dictionary reloads from better thread utilization.
String dictionary server status now visible as part of \status command in omnisql.
Added support for specifying TIMESTAMP precisions, including TIMESTAMP(3) - milliseconds, TIMESTAMP(6) - microseconds and TIMESTAMP(9) - nanoseconds. Immerse will support this feature in a future release.
Added support for CARDINALITY for array columns (returns the number of elements — for example, the length of an array column).
Added support to allow variable length array columns to be set to NULL.
Added early, experimental support for scalar user-defined extension functions written in C++/CUDA. These kernels can now be compiled on server startup and visible to the server runtime immediately. Please refer to the relevant OmniSci documentation for details on how to use this capability.
Added Thrift support to list the databases that can be accessed by the currently logged in user. Also added support for a user, once logged in, to switch between these databases without needing to login again.
Added support to allow for dashboards to be shared via roles.
Eliminated per-fragment overhead for non-lazy GPU projections, allowing larger result sets for rendering and other use cases.
Fixed Issues
Fixed issues with non-existent source in dashboard metadata which was causing migration to fail - this is now logged to enable better diagnosis of the scenario.
Fixed an issue where an empty array query resulted in a server program error under certain circumstances.
Fixed an issue where the Calcite server alone failed, resulting in an unusable but running OmniSci Server. Now, should the Calcite server fail, OmniSci Server does not allow new connections. The failure is captured in the logs, and OmniSci Server is stopped so that the systemd utility can restart both the OmniSci and Calcite servers.
Fixed an issue where binary load from arrow load for DATE resulted in incorrect data.
Fixed an issue where an INTERVAL ADD to timestamp resulted in a program error.
Fixed a case where a large number of noisy error messages were generated in the logs from auto-completion hint failures.
Fixed issues with spurious and potentially confusing "Connection refused" Thrift messages on server startup.
Fixed an issue with inability to export a dashboard from admin account, when admin is not the owner of the dashboard.
Fixed an issue where OmniSci HA node goes out of sync after program errors during execution of Kafka message instructions.
Fixed an issue with ALTER TABLE RENAME on multipolygon columns.
Fixed an issue where querying views with joins resulted in a watchdog error.
Fixed an issue where dropping a database resulted in recycled database IDs that caused unexpected behavior in the catalog.
Fixed program error on selecting Geo column from Outer Join.
Fixed program error resulting from certain SELECT * queries.
Fixed case where GRANT SELECT VIEW on database resulted in Update privilege.
Fixed cases where CTAS (CREATE TABLE AS SELECT) involving SELECT *, or CASE statements resulted in a program error.
Fixed an issue with version 4.4.2 not correctly updating buffer shapes on map unless map boundaries change.
Fixed a rendering regression related to log/sqrt/pow scale types in Vega.
Fixed an issue where Optimize Table command resulted in extra disk space being consumed on each run.
Fixed an issue with ODBC sessions stopping on reconnect. ODBC sessions now continue upon successful reconnect to omnisci_server.
Fixed an issue regarding incorrect text array length is shown in the new table after running CTAS.
Fixed an issue where SELECT queries failed after UPDATE on a newly added text column added via ALTER TABLE ADD COLUMN.
Fixed an issue with visibility of s3 credentials in history file.
Fixed a threading issue in CTAS that caused server failure.
Fixed an issue related to LDAP that affected omnisci_server users who had not yet been set up in the system.
Fixed an issue where multiple SAMPLE invocations in a single query resulted in inconsistent results.
Fixed an issue with program error on updating copy_params and re-detecting CSV.
Fixed an issue with approx_count_distinct causing system failure on GROUP BY with more than one group.
Fixed Issues
Fixed an issue where ST_ functions on a geo column used in a color measure were not parsing correctly in render request.
Fixed an issue with incorrect renders on distributed systems involving Vega transforms with aggregate queries.
New Features
Immerse supports integration with OKTA/SAML for Single Sign-On based authentication.
Tables can be deleted directly from Data Manager.
Data can be added to any table via the Append functionality in Data Manager.
All rows within a table can be removed via the Delete All Rows functionality in Data Manager.
Users can denote when a file has a header row during data import, using the Includes Header Row option within Import Settings.
Added in situ poly rendering for backend-rendered choropleths, greatly enhancing performance at scale.
Added sampling to server-rendered Choropleths.
Added crossfiltering and bounding box filter to server-rendered Choropleths.
Max number of groups that can be visualized in Bar and Bubble charts increased from 100 to 500.
Fixed Issues
Fixed an issue where clause sampling caused overflow with greater than 3.4 billion records.
Fixed an issue where brush was not aligned with bin increment in Combo chart.
Fixed an issue where Amazon S3 import could not be performed using a full link URL.
Turned off ability to rotate MapBox base maps to improve usability.
Import Data button is hidden on Data Manager screen if user lacks privileges to import data.
Fixed inconsistencies with timezones; all dates shown in Immerse are displayed as UTC.
Fixed an issue where the OmniSci logo appears before being overwritten by servers.json configurations.
Fixed an issue where Choropleth pop-up does not disappear when user zooms in on the map.
Fixed an issue in Bar chart where sort by selection was saved but not applied when switching color measures and aggregates.
Fixed an issue in Stacked Bar chart where custom colors selected in the color palette were not respected on the chart display.
Fixed an issue with Stacked Bar where axis label renaming was not persisting.
Fixed an issue in Combo chart where the chart editor right panel would become unresponsive after certain settings were configured.
Fixed an issue that prevented a user from changing measure formatting without clearing formatting first.
Fixed an issue with incorrect default formatting.
Fixed an issue in chart editor where popup remained active after required measure/dimension had been selected.
Fixed issues in Table chart, Histogram, and Heatmap that prevented date formatter from showing up correctly.
OmniSci release 4.5.0 includes several fixes and performance improvements.
With release 4.5.0 and higher, OmniSci now requires that all configuration flags used at startup match a flag on the OmniSci server. If any flag is misspelled or invalid, the server does not start. This change helps to ensure that all settings are intentional and will not have unexpected impact on performance or data integrity.
Licenses are now required for OmniSci Enterprise Edition. Servers start but are not functional without a valid license file installed.
OmniSci configuration files are now strictly validated on startup. Servers do not start with an unrecognized option in the configuration file, and an error message is logged to stderr containing the unrecognized option.
Default port numbers for OmniSci services have been changed. In addition, the names of key services are now prefixed with omnisci_, changing from mapd_. Symlinks from the old names are provided, but will be removed in future versions. Please note these changes and upgrade any scripts/workflows that use the old names.
Due to a system-related change in the behavior of DROP/TRUNCATE, data files are not deleted from disk, but tombstoned and identifiable by the DELETE_ME prefix on the filesystem for up to five minutes after you issue the command.
(Applicable to distributed installs only): StringDictionaryServer now lazy loadsdictionaries on request. Consequently, the first request to a dictionary-encoded column of a large table can be slow for very large tables with high-cardinality string dictionary columns.
The --gpu and --cpu flags are replaced with the --cpu-only flag for OmniSci Server.
Community Edition has been deprecated and replaced with Enterprise Trial. See https://www.omnisci.com/platform/downloads/ for details.
Renamed key components from MapD to OmniSci. Specifically:
mapd_server is now omnisci_server.
mapd_web_server is now omnisci_web_server.
mapdql is now omnisql.
String Dictionary Server is now omnisci_sd_server.
JDBC connection strings now are jdbc:omnisci:localhost:6273:<dbname> and the JDBC driver class changes to com.omnisci.jdbc.OmniSciDriver.
Better memory handling through improved estimation of query GPU memory requirement. Allow-cpu-retry is now turned on by default, which allows queries in certain low-GPU-memory situations to be identified and executed on CPU automatically.
OmniSci Core now supports inserts of array literals containing NULLs.
OmniSci Core now supports importing csv files with arrays containing NULLs.
Changed default port from 9092 to 6273 to avoid conflicts with other well known services running on those ports.
DECIMAL / NUMERIC fields can now be downcast to different scale/precision.
Dictionary size limit now increased to ~2.15Billion entries.
Deleting tables using the DROP command will immediately remove the table from the catalog but asynchronously remove data from disk.
Added interval support for DATEPART/DATEADD operations.
DATEPART - weekday, quarterday
DATEADD - week, weekday, decade
Added ability to recompute table metadata on demand using the OPTIMIZE command.
Fixed Issues and Enhancements
omnisci.conf is now validated at server start. This is to prevent any erroneous configuration settings taking effect. If erroneous config parameters are found, the server fails to start and throws an error.
Fixed issues when geo data is imported (for example from PostGIS) via JDBC.
Replaced --gpu and --cpu flags with a --cpu-only flag for OmniSci Server.
Fixed an issue where geo files could not be appended to tables created through Immerse geo file import process.
Fixed program error caused by selecting SAMPLE on a geo data type column.
Fixed an issue with ODBC not reconnecting on a server restart.
Fixed certain cases where DATE was handled incorrectly by SQLImporter.
Fixed an issue where session IDs were being logged in certain cases.
Fixed an issue where Importer did not estimate number of rows correctly.
Fixed an issue with JDBC handling of the type of a geo column.
Fixed an issue where JDBC needs to filter DB information from Table Priv Call.
Fixed an issue that prevented sharding on a date column.
Fixed an issue where projecting a varlen target inside SAMPLE in distributed mode could corrupt subsequent columns.
Fixed a typo in table schema output from the \d command in omnisql.
Fixed an issue projecting varlen columns inside a top k query (for example, SELECT str, x FROM proj_top ORDER BY x DESC LIMIT 1;).
Resolved a program error when running a subquery projecting dictionary encoded strings inside an IN expression.
Fixed an issue where out-of-bound integer ranges fed to timestamp column erroneously get turned into null sentinels.
Conditionally remove ST_CONTAINS filter when computing the filtered count for a geo join.
Improved accuracy of ST_CONTAINS on certain complex MULTIPOLYGON shapes.
Fixed Issues and Enhancements
Fixed an issue with Vega Scale parse failing if range values are whole numbers.
Fixed an issue with GLLayoutBuffer exception related to rendering the same Vega in two successive render_vega calls.
Fixed intermittent error with renders using Percent Accumulation Scale.
Enhancements to messages related to Vega parse errors.
New Features
Enterprise edition trial (see https://www.omnisci.com/platform/downloads/ for details).
Add support for lasso filter on line map chart.
Added clarity to formatting options and created a new option to represent billions as B.
Geo Heatmap colors rendered that do not fall in scheme selected now display in grey rather than purple.
Sort null measure values to the end by default for grouped chart queries (crossfilter and Stacked Bar charts).
Fixed Issues and Enhancements
Fixed an issue with session connection being lost on timeout.
Fixed ability for data tables to be switched via the search interface in Data Manager.
Fixed an issue where exported CSV does not match chart or omnisql data.
Fixed circle, polygon and lasso filters for Linemap.
Fixed cross-filtering for Number chart.
Fixed an issue with losing color when clicking on column in Stacked Bar chart.
Fixed an issue with overflow of bars when switching to Stacked Bar chart.
Fixed issues with non-functioning backend-rendered multi-layer map and backend-rendered multi-layer Scatter Plot examples.
Fixed an issue where Stacked Bar chart was not maintaining a filter on dashboard load.
Fixed an issue with HeatMap chart related to the assignment of default green colors for 0 values. Values that are 0 are assigned the mid-range green color.
Fixed an issue in Chart Editor where disabling NULL values after a dimension drag and drop generated a query causing a 502 error.
Fixed an issue with Immerse failing to load correctly if there was no servers.json file.
Fixed an issue with disappearing dashboard header in certain cases.
OmniSci release 4.4.2 includes several fixes and performance improvements.
Fixed Issues and Enhancements
Improved error logging when the mapd_server release is older than the server's data directory.
Added support for Arrow DATE values using small date buffers.
Fixed Issues and Enhancements
Fixed an issue where using conditional expressions (CASE WHEN condition THEN result ELSE default END) in a custom SQL dimension would cause a chart error.
OmniSci release 4.4.1 includes new features as well as several fixes and performance improvements. It comprises the 4.4.0 release version and a fixed regression.
New Features
Added support for the ST_Disjoint, ST_DWithin, and ST_DFullyWithin geospatial functions.
Added support for get_session_info() Thrift API.
DATE datatype is compressed. Dates now have two smaller encoding sizes: 4 bytes (default) and 2 bytes. Existing date columns continue to work as before.
Added support for SAML authentication with Okta (Enterprise Edition only).
Fixed Issues and Enhancements
Usernames can now include dashes.
Attempting an ST_Contains join with a geospatial string literal argument no longer causes a program error.
User session IDs are no longer potentially exposed in OmniSci Core log files.
The hit-test cache limit has been removed and should eliminate the following error: "Cannot retrieve results for query <query>. The results have not been cached for hit-testing."
Fixed database consistency errors that could occur if the underlying filesystem ran out of disk space.
Fixed a synchronization issue for overflow/underflow checks on NVIDIA Volta and Turing GPUs.
Improved performance on String Dictionary import for multiple String Dictionary-encoded columns.
Intermediate ResultSets are now automatically compressed before being shipped between the aggregator and leaf servers. This allows ResultSets over 2 GB in uncompressed size to be passed around the cluster and can result in performance improvements due to better utilization of network bandwidth.
Fixed an issue where concurrent updates/deletes in import operations could cause table corruption.
Fixed a corner case where BOOL, TINYINT, or SMALLINT columns could result in invalid data during import.
Fixed an issue with rowids projected in CTAS that caused an error in table creation, because rowid is a reserved column name.
Added support for AM/PM designation in timestamps.
Fixed an issue where sharded tables with geospatial columns could cause a program error during insert or projection queries.
Improved performance for queries using 32-bit timestamps.
Improved memory usage during hash joins.
Fixed an issue in which hash joins on Dictionary-encoded strings, where one of the columns was empty, could cause a program error.
Deprecated/removed the following obsolete flags: --disable-rendering, --db-convert, fast-strcmp, and multi-subquery-exec.
New Feature
Support is added for pct/blend accumulation rendering modes in distributed configurations.
Fixed Issues and Enhancements
Multiple point/symbol layers using Vega projections no longer potentially render a blank image.
In distributed configurations, a multilayer rendering issue occurred in which an aggregate query and a projection query, each referencing Vega transforms, could cause a program error. This has been fixed.
Using Vega transforms on line/poly queries with empty data no longer causes an error; it now produces an empty image, as expected.
Fixed Issues and Enhancements
Improved error tracking to allow better triage and resolution.
Fractional numbers are now supported in global filters in Immerse.
Fixed an error that occurred when switching chart types with binned dimensions.
Deprecated: The Export Chart button has been removed from Text charts.
SAMPLE aggregate is added as an option for all chart measures in conjunction with GROUP BY.
Fixed ambiguous-column error for identically named columns on geo joins.
Fixed issues with importing of tables with geospatial columns.
Resolved issue with modifications to Custom Colors on Combo charts with color dimensions were reflected in the right pane of the UI, but not the chart palette, which caused incorrect rendering.
Updated license uploading for Community Edition.
OmniSci release 4.3.0 consists of a number of new features as well as several fixes and performance improvements outlined below.
New Features
Decimal compression: Decimals now automatically use the smallest possible encoding for the specified precision/scale.
Hash joins now support null left-hand side joined to not null right-hand side and vice-versa.
Hash joins allow different integer types between the left-hand side and right-hand side.
Fixed Issues
Ensure MAX_ROWS is an integer literal and properly reject non-integer values.
Fixed an issue where CTAS did not properly parse date_trunc functions and geometry casts.
Fixed an issue where DROP DATABASE did not remove all table data from disk.
Fixed an issue where the minus sign was being counted as part of decimal precision.
Improved error handling in distributed mode.
JDBC now properly generates boolean string literals when setBoolean is called.
Properly validate DECIMAL type ranges and check for overflows.
Resolved an issue where DB owners may not have access privileges on new databases by default.
Resolved a system failure that occured for unsupported DDL statements.
Resolved a system failure that occured if a database were created then dropped before a user connected to it.
Resolved a system failure that occured when serializing multiple geospatial types in distributed mode.
Resolved a system failure when clearing GPU memory from mapdql when no GPUs are present.
Resolved a misleading error message no dashboards were returned while listing dashboards in mapdql.
Resolved a problem with geo append when existing table has uncompressed coords
Resolved an issue where ALTER TABLE on a view caused a system failure.
Resolved an issue where an invalid geospatial literal could cause a system failure.
Resolved an issue where approx_count_distinct could send too much data and/or cause a system failure during serialization in distributed mode.
Resolved an issue where deleted users were not totally purged after being dropped.
Resolved an issue where hash joins on distributed would run incorrectly if one of the tables was not replicated or both tables were not sharded.
Resolved an issue where object privilege migration could be incomplete.
Resolved an issue where precision errors could occur during queries involving aggregates.
Resolved an issue where session timeouts could cause a system failure in distributed mode.
Resolved an issue where the roles display for a user could be incorrect.
Resolved an issue where using privilege APIs could cause a system failure if object level privileges were disabled.
Resolved precision errors in ST_Distance and ST_Intersect for some geospatial types.
Resolved various issues in automatic FROM clause table reodering.
The last row of a CSV file is no longer ignored if the CSV file does not have a trailing new line.
New Feature
Added support for in-situ line rendering (i.e. without a copy) for projection queries (LINESTRING columns only)
Fixed Issues
Properly handle rows where all columns are NULL in the Vega transforms API.Resolved an issue handling NULLs in Vega scales.Resolved an issue when multiple sessions ran the same polygon hit testing query.Resolved an issue where Vega projections would fail to be applied.
New Feature
Enabled in-situ line rendering for linemap.
Fixed Issues and Enhancements
Added schema validation to prevent an edge case where dashboards could become corrupt and unloadable.
Fixed an issue with linemap hover popups on single node instances.
Fixed an issue where decimal-baded global filters returned incorrect results.
Fixed an issue where the Y-axis measure in the stacked bar chart could be missing a name label.
Fixed an issue with categorical color measures in choropleth that could result in a hang.
Fixed an issue where formatting dates on combo chart could result in a hang.
Added code to ensure measure aggtypes can be updated on linemap.
Removed an error thrown when changing choropleth color dimensions.
Fixed an error in linemap where geo column names other than mapd_geo or omnisci_geo could cause errors.
Improved handling of null data on piechart.
Adjusted linemap hit testing behavior to match pointmap.
Added code to prevent an issue where dashboard state was not preserved when switching to SQL Editor.
Fixed an issue with custom SQL measures in linemap.
Improved the legend for min/max numerical color scale for linemap.
Optimized viewport clipping for rectangular/non-complex shapes.
Added code to ensure measure aggtypes can be updated on linemap.
Improved handling of color measures for linemap.
Improved linejoin drawing techniques for linemap.
Prevented linemap from showing in rendering-disabled environments.
Added code to ensure adding and deleting custom colors on combo propagates to the chart.
Improved error logging and persistence for SQLLogger.
Enabled in-situ line rendering for linemap.
Fixed timezone issues for certain timestamps set to midnight (00:00:00).
OmniSci release 4.2 consists of a number of new features as well as several fixes and performance improvements outlined below.
New Features
Added top-level ACCESS privilege to check if a user logging in has any object permissions at all, and restrict Immerse capabilities accordingly.
This is a backward incompatible change. OmniSci recommends that you back up your existing database before installing OmniSci 4.2.
Added GRANT ALL privilege on database to allow for better management of object privileges
Added https support now available for major APIs - JDBC, Thrift, pymapd, KafkaImporter, StreamImporter APIs and mapdql
Added ST_MaxDistance allowing users to compute the maximum distance between a POINT and a LINESTRING
Added ST_Distance support for MULTIPOLYGONS
Added ST_Intersects support for MULTIPOLYGONS
Fixed Issues
Fixed an issue with simple literal additions returning incorrect results
INSERT now works correctly with text[] column types
SAMPLE now handled correctly on array types
Faster imports of dictionary-encoded string data - upto 3x performance improvement, depending on the data
OmniSci now uses LLVM 7.0
Catalog properly loaded even if a user logs into the non-default database first
Roles can now GRANT to other roles, rather than just individual users - this fixes an existing issue with dashboard sharing to/from roles
get_all_roles_for_user() thrift call now works for roles as well
OmniSci now supports Arrow 0.10.0 and 0.11.0
db_object_privs thrift call now supports user privileges
GRANT now supports granting multiple privileges to multiple grantees in a single command allowing for better privilege management
has_object_privilege thrift call now supports recursively returning privileges for a given user, which identifies not just their directly granted privileges but also those granted via roles
Projection queries now support columnar outputs, significantly improving the efficiency of memory management for query results.
ALTER TABLE now enabled for non-super users
Fixed incorrect timestamp precision encoding, including handling of leading zeros
Fixed edge case issues with decimal formatting for display
SAMPLE now supports varlen arguments in distributed mode with GROUP BY
Fixed an issue with long-running first load queries disallowing Immerse logins
Fixed an issue with array projections needed for improved GeoSpatial support in a distributed installation
Improvements to geo import robustness
GeoSpatial queries now supported properly in distributed mode
Fixed issues with POLYGON/MULTIPOLYGON handling within ST_Intersects
All default GeoSpatial tables now use omnisci_ as the prefix. Please update any scripts that use these tables accordingly
The default geo column name is now omnisci_geo (from mapd_geo). Please update any scripts accordingly
Fixed certain edge cases in ST_Contains(POLYGON, LINESTRING) that caused incorrect results
ST_Contains extended to support ST_Contains(MULTIPOLYGON, LINESTRING)
Several improvements in CTAS robustness - CTAS now supports most major types including GeoSpatial types (NOTE - currently these improvements are restricted to a single node. This restriction will be removed in an upcoming release)
OmniSci now uses GDAL 2.3.2 (geo import library)
Import reports line number of bad geo in CSV files
Bulk inserts for geo data now supported via JDBC API
New Features
Added support for TINYINT in vega transforms
Added support for rendering buffers larger than 4GB
Added distributed line-rendering support.
Fixed Issues
Fixed an issue preventing Polygon rendering if the first render query returned empty results
Allow line/poly rendering even if render-mem-bytes is set to zero
Symbol sizing in vega now supports distance units (meters). Subsequent releases will integrate this into Immerse geo charts.
Prevent renderer system failure in certain cases on shutdown via SIGINT
Fixed an issue with switching density accumulation from max/min to stddev
Fixed issues with apparently incorrect render memory allocation on multiple devices
Fixed system failure caused with COPY/TRUNCATE/DROP statements executed after clear_gpu_memory is run with rendering enabled
New Features
Combo and Stacked bar charts support percentage (relative) views
Added support to show/hide SQL Editor in a configurable manner. See Roles and Privileges for configuration information
Added Linemap Geo Chart, allowing for LINESTRING data to be displayed and used in Geo joins as well. This chart type works with multi-layer geo charts
Fixed Issues
Several improvements in performance, usability and robustness of the Stacked Bar chart
Stacked Bar chart now supports CSV export
Multiple improvements and fixes to chart legends on multilayer Geo charts
Multiple improvements to Combo Chart related to data loading indicators and behavior
Fixed minor issues with navigation icon alignment on Chart Editor
Fixed an issue causing overrides of user-specified column names in Table Importer
Raster charts no longer fail to render correctly if a dimension is added after omnisci_geo measure is added
Fixed an issue with lineStyle being lost upon save/apply actions in certain cases on the Combo chart
Fixed edge case issues with handling negative values in Combo and Stacked Bar charts
Table importer: support for DOUBLE types and also fixed handling of TIME types
Removed 'atlas' mapbox style (no longer valid) from servers.json
The 4.1.3 release is primarily focused on rebranding MapD to OmniSci.
Fixed an issue where Core might experience a system failure when consuming autocomplete data.
Fixed a performance issue applying a relative date range global filter to a combo chart.
Version 4.1.2 was a limited release with minor fixes and performance enhancements.
New users no longer have default access to client tools. To provide new users with access to Immerse and client tools, you need to grant them the ACCESS privilege.
Added geospatial operator/function support:
ST_Within (inverse of ST_Contains).
Cartesian and Geodesic ST_Length.
Cartesian and Geodesic ST_Perimeter for POLYGON and MULTIPOLYGON.
Cartesian ST_Area for POLYGON, MULTIPOLYGON.
See Using Geospatial Functions.
Improved performance by reducing fixed overheads during JIT Query compilation.
Support for both fixed and variable length arrays. See Datatypes.
The following issues are resolved in MapD 4.1.1.
MapD Core
The name of the JDBC driver has changed and is now versioned. It takes the form mapdjdbc-<release-number>.jar, where <release-number> changes in sync with each release. For example, the current JDBC driver file name is mapdjdbc-4.1.1.jar.
Fixed typos in description for configuration flags.
Emails can now be used as usernames.
JDBC now supports geometry types, timestamps on SELECT queries.
Fixed invalid HA configurations in certain cases, due to quotes in configuration parameters.
Fixed system failure on fixed length array import in distributed mode.
Fixed system failure on ALTER/DROP table.
Fixed an issue where duplicate CASE statements in WHERE filter conditions compared with an OR operator caused a segfault.
Improved connection safety on MapD session creation.
clear_gpu and clear_cpu now work correctly in distributed mode.
mapd_server no longer becomes unresponsive when making too many connections in distributed mode.
Improved query times for expressions with a large number of IN operators.
Allow configurable expiry of sessions - inactive sessions can be expired in a configurable manner, and extremely long running sessions can be controlled with a session timeout.
Fixed issues with edge case inaccuracies in ST_Contains.
Fixed system failure related to edge case on revoking <username> role from <username> when <username> logs in.
Fixed errors with certain geo import edge cases.
Fixed system failure on subsequent SELECT queries after ALTER TABLE ADD COLUMN was used to add a geo column.
Fixed inability to append to existing geo table. Geo append is now faster with large tables.
Allow dashes in role names.
Fixed an issue with distributed metadata updates.
Fixed invalid credential in HA connection processing.
Fixed system failure on temporary tables with geo columns.
Rename LAST_SAMPLE to SAMPLE. LAST_SAMPLE is deprecated.
Fixed bug in mapdql preventing proper handling of decimal arrays.
Add idle_session_duration for invalidating inactive sessions.
Add ability to obfuscate session ID in MapD server logs.
Fixed mapd_server system failure when using a subquery in a where clause that references a view.
S3 import fixes:
Fix a constraint that s3 importer could detect/import at most 999 files.
Fixed a bug where S3 importer needed to download all files in an archive before it could detect columns.
Migration history now uses correct table name.
Fixed header skipping on detection with header=true.
Better handling of NULL values/EMPTY geometries. Rows imported into geo tables with NULL/EMPTY geometry values in geo fields now cause entire row to be skipped during import.
Added ACCESS privilege to allow or deny login access for accounts with no privileges.
Rendering
Fixed run-time shader compile failure in Symbol shader when using packed colors.
Fixed mapd_server system failure while trying to render extremely large polygon table.
The following are features and improvements in MapD 4.1.0.
Core
Ability to synchronize user roles from LDAP (Enterprise Edition only)
ALTER TABLE ADD COLUMN now supported.
GRANT/REVOKE ALL ON DATABASE now supported.
Improved performance on Nvidia Pascal and Volta architectures on single column low-cardinality GROUP BY with COUNT(*) by using GPU shared memory.
Calcite version upgraded to 1.16.
Immerse
Stacked Bar Chart released.
Several performance/stability improvements on Combo chart.
Import UI - Ability to specify tables as replicated for distributed deployments.
Additional user interface customization options (see documentation for details).
Added auto-zoom to data bounds on geo heatmap chart.
The following issues are resolved in MapD 4.1.0.
MapD Core
Removed hidden physical columns from select * expansion for geospatial types.
Fixed mapdql system failure when running `\d` on a view with geospatial columns.
MapD Core Enterprise Edition
Fixed an issue where projecting float columns could return an incorrect value in distributed mode.
Fixed an issue where case statement with strings caused system failure in distributed mode.
Fixed an issue where none encoded string could system failure the aggregator in distributed mode.
Fixed an issue where swap table order on a distributed join with one replicated table could give incorrect results.
Fixed an issue where sharing dashboards with users but not roles did not work.
SuperUser role from LDAP did not automatically have superuser privileges granted.
Allow Grant privileges to a role that is corresponding to LDAP role
Fixed a segfault issue on `LAST_SAMPLE` when serializing resultset on distributed.
Login fails when LDAP properties are defined in mapd.conf.
Fixes to handling of table reordering for joins in a distributed setting.
Immerse
Removed double group-by in backend-rendered choropleth, improving performance
Now supports text types in backend-rendered choropleth chart color measure.
Added autozoom on creation of geo heatmap.
Fixed an issue where column header names revert to `column_1`, `column_2` etc. upon import after changing a CSV import setting.
Removed/suppressed additional options in 4.1 'About Immerse' dropdown.
Fixed an issue where dashboard Save button was not showing in all cases.
Fixed an issue where brushed range chart on the area line combo chart was unable to render a focus chart with extracted timestamps.
Fixed an issue where auto-selecting mapd_geo ignored the dimension column for a join.
Can now set a global filter on datasets containing arrays.
Fixed an issue where combo chart focus and range brushes were out of sync.
Fixed an issue on combo chart where the x-axis was misaligned when the x-axis dimension was a numeric datatype.
Immerse now supports TINYINT datatype.
Fixed an issue on Internet Explorer where the EULA accept button did not appear until the window was resized.
Fixed an issue where the global filter icon was not activated for a custom measure filter.
Fixed an issue where switching from Choropleth to Geo Heatmap threw an error and the Geo Join component was not clearing.
Fixed an issue where the dimension was not applied to Scatter/Pointmap if dimensions were selected after measurements.
Fixed an issue on combo chart where users were unable to set a limit on the x axis.
Fixed an issue where the scatter chart did not re-plot points after a dimension is removed.
Fixed an issue on combo chart where binning did not change for the smaller value when setting a date filter.
Fixed an issue where the Export Dashboard filter was not working.
Fixed mapd-connector to address poor query performance on large array columns.
Fixed an issue where the horizontal axis lock did not hold when switching from line to histogram chart.
Fixed an issue where turning off binning for large data sets could cause the browser to hang.
Fixed an issue to combo charts where they were unable to render the range chart after placing a restrictive dimension filter.
Fixed an issue where turning off binning on geo table could cause Immerse to stop.
Improved performance of zoom/pan on line charts.
Fixed an issue where the incorrect key symbol was displayed in the color palette after editing the line style.
Fixed an infinite error loop when switching from choropleth to table chart.
Fixed auto bin highlighting.
Now supports mixed content: http web server, https backend.
Fixed some combo chart interface inconsistencies.
The following issues are fixed in version 4.0.2.
Migration Fixes
Fixed transaction within transaction exception when upgrading to MapD version 4.0 or later.
Rendering Fixes
Properly handle dictionary encoded strings inside a CASE statement when used as input to a Vega transformation.
MapD Immerse Fixes
Fixed latitude bounds sanitization on Mapbox chart render/zoom.
Other Fixes
Improved the stability of LAST_SAMPLE on dictionary encoded string columns
Fixed incorrect result for certain queries on dictionary encoded strings in a distributed cluster.
Fixed an issue where GROUP BY on a Boolean column caused a system failure.
Properly handle projecting TINYINT columns in a distributed cluster.
The following issues are fixed in version 4.0.1.
Migration Fixes
Sqlite3 Error: database is no longer locked after migration.
Fixed a permission migration issue when the owner of the database is not a superuser.
Fixed an issue that caused a system failure after copying geo data from csv in distributed cluster.
Fixed an issue regarding migration of non-superuser dashboards from pre-4.0.
Rendering Fixes
Fixed an issue where aggxform is unable to access the alias map when no results are returned.
Fixed Invalid dimensions for texture array exception when rendering a geo pointmap in a distributed environment.
Fixed an issue with aggregating and compositing a geospatial point render.
Other Fixes
Fixed an issue that caused a segfault in UPDATE, DELETE, DROP, and CREATE stress test.
Fixed an issue that caused a segfault if no rows are passed to load_table.
Fixed an issue with COPY FROM with ARRAY.
Fixed an issue where network geo import from SSL-requiring locations failed on Ubuntu/Docker.
REGEXP now falls back to CPU when required rather than report errors.
Added DATE, TIME, TIMESTAMP support for pymapd and all Arrow consumers.
Volta performance has been improved.
Fixed an issue that caused a system failure with LAST_SAMPLE for certain high cardinality group bys.
Fixed a CHECK fail in reduceSingleRow() when using LAST_SAMPLE().
Fixed inconsistency while building role map.
Fixed an issue granting CREATE dashboard via role.
Fixed
SMALLINT null values are now handled correctly.
Encryption in core logs is improved; AWS access keys and secrets are no longer logged in clear text.
Fixed an issue for accessing a shard with a negative shard key.
Fixed an issue where table <tablename> must be replicated error message was inadvertently shown for replicated tables on a distributed join.
Fixed an issue with "lazy" projections (those not in an aggregate or filter) on sharded joins.
Properly report SHARD_COUNT and PARTITIONS in the DDL generated by the \d command in mapdql.
Privileges for secondary databases are now working correctly.
Hash joins between two sharded tables now work correctly.
Projection queries on GPU use significantly less memory for lazy projections (for example, select a, b, c from table where x).
New
Added four new aggregate transform modes to the Vega transform data block: distinct, countdistinct, median, and quantile.
Support for new geospatial types for line and polygon rendering.
Support for new geospatial types for point rendering using geospatial accessors.
Improved precision for Vega spatial projections.
Robust POLYGON/MULTIPOLYGON import and rendering. System no longer requires polygon triangulation, so all poly geometry, whether or not it is degenerative, can be imported and rendered without issue.
POLYGON/MULTIPOLYGON holes are now supported.
New procedural rendering technique for symbols significantly improves symbol render times.
Fixed
Various filtering issues have been fixed.
SMALLINT null values are now handled correctly.
DECIMAL supported in Vega transform operations.
Better handling of queries that return empty results.
Better stability in distributed mode when a query fails just prior to rendering.
More robust poly hit-testing support.
Appropriately handles NULLs, empty results, and decimal types with Vega transforms.
New
Shared Dashboards let you share your dashboard configurations with specific users or user roles.
Measure Formatting lets you format number and date values on the X-axis.
You can run Immerse in a completely disconnected environment.
Backend-rendered Choropleth with geospatial support has been added.
Import preview now detects all geo column types.
Vega transforms have been added to Choropleth, Pointmap, and Heatmap charts.
You can now import compressed archives of geospatial files (such as .shp files).
Saved Dashboards now have persistent URLs.
Fixed
Various filtering issues have been fixed, including relative time and date filtering.
In Internet Explorer, labels for Bar, Pie, and Bubble chart types are no longer dropped.
Under certain conditions, Immerse would not show data when custom date filters were applied. This has been fixed.
Global clear filter action now clears all table chart filters.
Issues with the appearance of tooltips in stacked bar charts have been resolved.
Zoom on Scatter Plot charts now works as expected.
See also the OmniSci Blog.
Before installing MapD 4.0, MapD recommends that you back up your database. Security features are now enabled by default. Changes in the database to support security features do not allow reversion to an earlier release.
All tables that include polygon information must be re-imported for version 4.0.
The minimum supported NVIDIA driver version for MapD 4.0 is 384.81. Use the `nvidia-smi` utility included with the drivers to see the current active driver version. If you need assistance upgrading the driver, contact MapD support at support@omnisci.com.
For the complete list of limitations in this release, see Known Issues.
UPDATE and DELETE SQL commands let you change values or delete rows from your MapD database.
Added object permissions model for Database, Table, View, and Dashboard object types.
Role-based access control allows you to specify who can access different views.
Countries, US States, and US Counties tables are now automatically created and imported with initdb.
Enhanced Geospatial Support lets you import and manipulate geospatial object data.
Added support for TINYINT data type.
Added support for fragment skipping on join queries with the inner-join-skip-fragments command line flag.
Add support for LAST_SAMPLE aggregate.
Array literal syntax is now supported in insert statements. For example:
Fixed
SMALLINT null values are now handled correctly.
Encryption in core logs is improved; AWS access keys and secrets are no longer logged in clear text.
Fixed an issue for accessing a shard with a negative shard key.
Fixed an issue where table <tablename> must be replicated error message was inadvertently shown for replicated tables on a distributed join.
Fixed an issue with "lazy" projections (those not in an aggregate or filter) on sharded joins.
Properly report SHARD_COUNT and PARTITIONS in the DDL generated by the \d command in mapdql.
Privileges for secondary databases are now working correctly.
Hash joins between two sharded tables now work correctly.
Projection queries on GPU use significantly less memory for lazy projections (for example, select a, b, c from table where x).
New
Added four new aggregate transform modes to the Vega transform data block: distinct, countdistinct, median, and quantile.
Support for new geospatial types for line and polygon rendering.
Support for new geospatial types for point rendering using geospatial accessors.
Improved precision for Vega spatial projections.
Robust POLYGON/MULTIPOLYGON import and rendering. System no longer requires polygon triangulation, so all poly geometry, whether or not it is degenerative, can be imported and rendered without issue.
POLYGON/MULTIPOLYGON holes are now supported.
New procedural rendering technique for symbols significantly improves symbol render times.
Fixed
Various filtering issues have been fixed.
SMALLINT null values are now handled correctly.
DECIMAL supported in Vega transform operations.
Better handling of queries that return empty results.
Better stability in distributed mode when a query fails just prior to rendering.
More robust poly hit-testing support.
Appropriately handles NULLs, empty results, and decimal types with Vega transforms.
New
Shared Dashboards let you share your dashboard configurations with specific users or user roles.
Measure Formatting lets you format number and date values on the X-axis.
You can run Immerse in a completely disconnected environment.
Backend-rendered Choropleth with geospatial support has been added.
Import preview now detects all geo column types.
Vega transforms have been added to Choropleth, Pointmap, and Heatmap charts.
You can now import compressed archives of geospatial files (such as .shp files).
Saved Dashboards now have persistent URLs.
Fixed
Various filtering issues have been fixed, including relative time and date filtering.
In Internet Explorer, labels for Bar, Pie, and Bubble chart types are no longer dropped.
Under certain conditions, Immerse would not show data when custom date filters were applied. This has been fixed.
Global clear filter action now clears all table chart filters.
Issues with the appearance of tooltips in stacked bar charts have been resolved.
Zoom on Scatter Plot charts now works as expected.
Archived release notes for release 3.x
3.6.1 | 3.6.0 | 3.5.0 | 3.4.0 | 3.3.1 | 3.3.0 | 3.2.4 | 3.2.3 | 3.2.2 | 3.2.1 | 3.2.0 | 3.1.3 | 3.1.2 | 3.1.1 | 3.1.0.1 | 3.1.0 | 3.0.0
New
Data detection to files imported from S3
Fixed
Multiple fixes and performance improvements to S3 Import to support MapD Cloud
Improved handling of networked file systems, including EFS
Fix memory leak with certain style of select query
Fixed
Improved multi-GPU/multilayer rendering compositing
New
New user interface for loading data to MapD from Amazon S3
Fixed
Fixed an issue with MultiPolygon geoJSON parsing bounds (Choropleth).
New
Enable sharding for (left-deep) outer joins
Added bulkUpdate support for JDBC to support Pentaho
Significant performance improvement on string comparisons from using std::sort
Fixed
Several fixes to auto-completion
Fix to avoid dictionary collisions in multiple databases
export_dashboard works with spaces in dashboard name
Multiple fixes to significantly improve accuracy and performance on APPROX COUNT DISTINCT, for large cardinalities
Fix issue for filter pushdown into expression range calculation when filter results in empty set
Fix broken view from changed underlying tables
ROLLBACK on connection failure with Dictionary server
Revised TColumn->Datum conversion to fix slow selects of array columns
New
Supports spatial projections in Vega.
Supports running MapD Core in CPU-only mode (with CUDA disabled) with backend rendering enabled if a GPU is available. Requires OpenGL 4.6 or higher.
Add opacity, fillOpacity, and strokeOpacity properties to marks so that opacity can be controlled independently of the fillColor and strokeColor literals.
New
A new Bar/Line combo chart supports multiple measures. This is currently a BETA feature.
You can export chart data as a CSV file for all frontend-rendered charts.
Fixed
Filtering on timechart with histogram hangs Immerse.
Chart dimension/measurements requirement and logo disappears when hovering.
Text partially cut off in color Legend Item field when hovering.
An Immerse error occurs when dashboard data takes longer to refresh than the auto-refresh time period set in the dashboard.
Scatter plots could render two charts on top of one another.
New
S3 Import: COPY FROM now supports directly loading data files (including compressed formats supported in release 3.4) from S3 buckets.
KafkaImporter: MapD can now read delimited rows from Kafka topics using the efficient binary columnar load path.
StreamImporter: New utility with improved streaming reads for data load, using binary columnar load path.
Outer join performance improvements.
Autocomplete support in mapdql based on new get_completion_hints Thrift call
Fixed
Multiple issues related to race conditions causing metadata corruption during simultaneous load and query of capped collections
Multiple JDBC-related issues, including missing checkpoints on bulk distributed loads and reporting correct JDBC versions
Edge case of filter qual pushdown causing negative cardinality estimation
Enabled use of hardware accelerated atomics for double-precision aggregates on Volta GPUs
Better exception handling of arrays in shapefile metadata
Fixed exception thrown on DROP...IF NOT EXISTS
Better error reporting if string dictionary paths not found at startup
startmapd now passes --config to mapd_web_server
mapdql: fix LineNoise memory leak
mapdql: Various error handling improvements
New
Significant performance improvements in rendering datasets with large numbers of polygons
Fixed
Better handling of bad data in shapefiles
New
SQL Editor now includes autocomplete and syntax highlighting.
Scatter Plot and Geo Pointmap charts now support aggregations.
Immerse now allows adding additional Mapbox basemap styles and third party map tiles
Community Edition EULA is updated.
Fixed
Skinny histogram charts when using date or time extracts.
Immerse hangs when editing a chart, with the error message “Cannot read property timeBin of null”.
Y-Axis has repeating numbers.
Range chart disappears on clearing color dimension.
Unable to add second layer in multilayer chart on Firefox.
Unable to load chart after adding second multilayer in Geo Heatmap chart on Firefox.
SUM aggregation for measures in Geo Heatmap chart should not return COUNT.
Changing map style before selecting a source prevents selecting source and breaks the Dashboard button.
Extract binning does not work on timestamp.
Empty popups in Pointmap chart.
Dashboards with large number of filters generate bad sql and fail to reload
New
COPY FROM command supports input files compressed with tar, zip, rar, gzip, bzip2, and tgz.
DATE_TRUNC and EXTRACT of YEAR, QUARTER and MONTH are 3-5x faster executing on GPU.
Loading of dictionary-encoded data is 15-30% faster.
“Top K” queries using LIMIT k OFFSET n with k+n < 1000 are optimized to run using k+n memory per GPU and in linear time; not yet enabled for distributed queries.
Allow simple predicates to be used in calculation of expression ranges. This allows for faster GROUP BY when binning by timestamp or numeric expressions if a narrow filter is applied on the same column.
Allow simple predicates with simple casts to be used for skipping fragments.
New initialize “-i” file option is added to SQLImporter.
Add get_completion_hints thrift call to enable friendlier SQL clients.
Add deallocate_df thrift call to deallocate an unused data frame.
Add thrift calls (and mapdql support) for get_tables, get_views, and get_physical_tables.
Speed up t and d in mapdql by requesting only the required metadata.
Add –help-advanced option to mapd_server.
Add thrift calls to set/get table epochs by name.
Fixed
Address issue where LDAP parameters cause mapd_web_server to fail to start by allowing passing LDAP parameters enclosed in double quotes.
Pushdown simple predicates containing casts of TIMESTAMP and INTs into expression range and fragment skip calculations.
Fix issues with operand nullability for time comparisons.
Fix crash on outer cross join.
Add thrift and mapdql commands for get_hardware_info (missing in mapdql help).
Fix possibly nondeterministic result of left outer join.
Sink boolean expression in a projection to its reference in the condition.
Start distributed loads at random leaf to better balance over all nodes.
Allow export/import of Immerse dashboards with spaces in the name.
Change default pagesize from 1M to 2M, add version_num to table metadata.
Recognize and use shared encoding dictionary with IN subqueries, resulting in better performance when inner and outer query use the same dictionary.
Support DECIMAL and NUMERIC for load_table_binary_columnar; improve errors
Fix equi-join on dictionary-encoded columns without shared dictionaries.
Upgrade to JDBC 4.0.
Fix processing of spaces on JDBC inserts.
Return correct count of inserted rows in JDBC executeBatch.
Allow URL format in JDBC driver spec as jdbc:mapd://<hostname:port:db:protocol>.
Fix JDBC DatabaseMetaData to return MapD version, set JDBC driver version.
Fix conversion BOOLEAN to LONG for better Hive support in SQLImporter.
Fix handling of NULL doubles in SQLImporter.
New
Multi-thread filling poly and aggregate query render giveing a 2-4x performance improvement.
Performance of line rendering has been improved by 25% - 50%
Performance of polygon rendering is significantly faster for large data sets
New command-line option “render-poly-cache-bytes” is added to MapDServer (default 300M).
Allow threshold scale to be used with symbol shape property in Vega render spec.
Fixed
Rendering deals more gracefully with malformed polygons. If simple triangulation fails, a proxy geom will be used in its place. It will first try using a Ramer-Douglas-Peucker simplification algorithm, next will will try to generate a convex hull, and throw an error only if this also fails.
Addresses issue when the only difference between successive render_vega queries is a change between projection and aggregation.
Address issue where Rendering a poly table for the first time with a filtered query crashes server.
Mostly address GL_FRAMEBUFFER_UNSUPPORTED error seen on some older Cuda drivers on Centos.
Fix issue when a multi-GPU non-accumulation render is followed by a single-GPU accumulation render.
Always cache non-in-situ renders for get_result_row_for_pixel hit testing.
New
Dashboards can reference multiple data sources.
Geo charts can incorporate multiple layers.
Dashboards can be configured to auto-refresh on a set time interval.
New menu items in Community Edition have been added under Help to request a trial license and manage trial license keys.
Four new Base Map Themes have been added: Odyssey, Vintage, Decimal, and North Star.
Shape selection now is allowed in point maps and scatterplot charts.
A new Top Level menu - Data Manager - has been added.
Fixed
Immerse now allows highlighting and copying a field in a table chart without inadvertently applying a filter.
Text Widget can now save text on Internet Explorer 11.
New
Add initialize file option to SQLImporter
Add thrift apis and mapdql comands for reporting on role/user privileges
Fixed
Fix handling of NULL in loading doubles with SQLImporter
Add export/import dashboard commands to mapdql help
Fix precision for output of decimal to file
Fixed nulls first/last sorting for distributed
Rollback failed table copy from
Reduce checkpoints to once per load, locking table to limit to one concurrent update
Apply reverse distribution to logical arguments
Add optional error percentage argument to APPROX_COUNT_DISTINCT
Move SQL importer to binary columnar format for better performance
Upgrade SQL parser to Calcite 1.14
Dependencies
Now requiring Java Runtime Environment (JRE) version 1.8 or higher
Fixed
Fix to reduce GL_FRAMEBUFFER_UNSUPPORTED when running older CUDA drivers on CentOS
Fix to render thread shutdown
Fix to error code on framebuffer error
Address mark property update issue when the only thing changing between render calls is sql and render property
Add multijoin and left deep inner join to table scan for rendering
New
Add nulls sort first/last to table sort
Fixed
Fix for exception raised when switching from geoheatmap to other chart type
Fix to row and pie chart to set minimum number of groups to one instead of zero
Allow SQL editor to accept SQL statements starting using WITH table expression
New
Improve support for left-deep joins to allow joins of large numbers of tables
Automatically reorder tables in a query by row count
Allow encoding dictionaries to be shared among multiple columns in the same table
Add new mapdql commands to export/import Immerse dashboard definitions
Remove the limit on the number of simultaneous connections
Fixed
Allow object level privileges against multiple databases, still under trial flag
Throw an error if two operands of IN operator don’t have matching types
Allow empty IN operands
Add nested loop code to code generation
Initial pass at internal catalog functions for viewing object privileges
Fix jdbc to not ignore substitution parameter at the end of a query
Add at_most_n parameter to thrift sql_execute endpoint
Fix to join of a column against an expression
Fix for possible CPU heap corruption after memory starvation
Improve error messages on file operation failure
New
Run all render commands on a dedicated thread, removing connection limit
Add line mark rendering type
New
Additional GTM tagging has been included
Improved error reporting on failure to load a dashboard
User logout moved to dropdown by navigation bar, count widget moved under dashboard header
Fixed
In SQL Editor, fixed issue with scrolling
New
Add Thrift call load_binary_table_arrow for Arrow columnar import
Fixed
Fix loading of DECIMAL string with leading decimal point
Create object security tables if feature is enabled
Fix to checkpoint creation on epoch-based table rollback
Read fragment metadata in parallel
Parallelize cold start reload of string dictionary
Fix table rollback by epoch
Fix DECIMAL import for values between -1 and 0
New
Support rendering of aggregate and projection queries for distributed configuration
Enable rendering where number of render GPU’s is less than query execution GPUs
Fixed
Fix logging of rendering calls
New
For Community Edition, added link to the [Community Forum](https://community.mapd.com) under Help
Fixed
Fixed an issue where a line chart filter could not be removed
Fixed an issue with old crossfilter instance preventing opening a new dashboard
Fixed an issue where exceptions/errors prevented histogram and line charts to reload
Fixed an issue with multi-series line chart filter icon not disappearing
Fixed an issue creating a numerical binned multi-series line chart
Fixed an issue where reverting edits on a line chart resulted in error
New
Add support for object privileges, not enabled by default
Add Thrift call for load_table_binary_columnar
Update EULA text
Fixed
Add basic lock protection for drop and truncate table
Fix potential incorrect result for filtered outer join queries
Fix to improve precision in decimal arithmetic
Optimize queries with combinations of equality and null
Fix possible crash on sort of empty result
Fix to bulk import related to decimal scale
Round when reducing decimal scale on import
Write temporary install files to temp directory
Save sample data sets to temp directory during install to reduce required privs
Fix to command line options while inserting sample data
Fix to limit pushdown with offset
Allow case insensitive boolean literals on import
Fix range checking for fixed width encoded strings
Fix possible segfault on outer non-equijoin on string
Fix for distributed query with grouped sort and offset
Fix for null handling in top k queries on a single-fragment table
Fix for self-join query
Fix for possible error on query with empty ranges of type double
Fix join on fixed encoding dictionary columns with nulls
Enable HA in build of MapD Enterprise Edition
Fixed
Fix for multi-layer rendering in distributed configurations
Improve support for ordinal scales when domain count exceeds range count
Fix for floating point issues with heat map hex binning
Fix for multi-layer rendering with 1 or more accumulation layers
New
Update EULA text
Added Google Tag Manager for product usage tracking
New
Equi-joins between expressions and columns are now supported
Joins on time and date are now supported
Non-grouped projected top-k queries of form order by col limit n are now executed more efficiently
Performance improvement for grouped top-k queries on sharded tables
Fixed
Fix for projection query with order by a float column not used in a filter
Fix for null pointer exception in PreparedStatement (https://github.com/mapd/mapd-core/pull/53)
Fixes for conditions of potential arithmetic overflow
Fix for segfault on natural join, or join explicitly including rowid
Gracefully return exception on scalar subquery returning text
Fix for failure to execute join query against an empty table
Fix for failure to execute a query with parenthesis around ilike condition
Fix to enable some cases of reordering joined tables improve performance
New
Symbols now work with multi-layer rendering
Add support for GPU rendering with EGL on IBM Power
Fixed
Fix blend accumulator type mismatch bug
Issue resolved for potential crash with geo-heatmap render
Various vega symbol update issues
New
New chart type : Text Widget
Alias for custom measures and custom dimensions
Geo heatmap (available on single node only)
Fixed
Range chart is now OFF by default
“Back to Dashboard” link is now aligned on browser resize
New
allow group by floating point column, even if watchdog is enabled
allow one-to-many join on multiple columns
Fixed
fix for overflow detection for fixed encoding dictionary entry creation
disallow multi-node export, for now
fix for subquery with a CASE statement inside a derived table returning a non-grouped column
explicitly disallow LIMIT 0
fix for query with sort on a dictionary encoded field containing null values
add progressive backoff to mapdql connect retries and limit retries
New
enable multi-sampling for smoother rendering
Fixed
Use correct cuda context index during EGL compositing
Fixing active gl context bug during distributed rendering
New
Allow switching between donut and pie chart styles for pie chart
Top level Help has one new menu item for Immerse tutorials
Declining EULA in community edition disconnects user
Default aggregation on time unit measures is MAX
Fixed
Fixed spinner on incorrect custom dimension expression
Fix for switching series dimension in a line chart or histogram to reflect new dimension
Number chart no longer exposes show range chart toggle
Fixed isue with histogram range bar width
Multi-series range chart colors are no longer reset when range chart is toggled off/on
New
Tables now can be sharded by a numeric or dictionary encoded column
Dictionaries for encoded columns can be shared across multiple tables
Multi-column join is allowed provided combination of columns yields a unique join key
Tables can be joined on columns of type BIGINT, provided the column yields a unique join key
Join now allows the one-to-many case, with uniqueness in the outer table
TRUNCATE TABLE command is now supported
MapD startup has an optional parameter for maximum error for APPROX_COUNT_DISTINCT
Data import via COPY now reports on specific location of bad data
COPY TO now supports relative path names
MapD Server now defaults to ignore divide by zero
Fixed
Max string length of 32767 is now enforced
Trailing semicolon is automatically removed from SQL or SQL inside a view
Incorrect result fixed for selecting with a filter from a join view
Incorrect result fixed with LEFT JOIN resulting in a cross join
Fix issue with precision of APPROX_COUNT_DISTINCT when running on CPU
Fix issue with join on empty inner table
Fix issue with database.table syntax
Fix for possible race condition in distributed queries causing query hang
New
Improved anti-aliasing using multi-sampling, most noticable stroking polys or symbols < 2 pixels wide
Allow dictionary-encoded string columns from geo polygon tables to be used as domains in ordinal scales
Added set of new extension functions as helpers for hexagonal headmap binning and rendering
Symbols now can be used in accumulation rendering
Fixed
Symbols that are both filled and stroked now render in the appropriate oder
Fix issue when using values from a dictionary encoded string column in an ordinal scale when rendering a GROUP BY query
Fix for crash when trying to render a GROUP BY query also sorted with ORDER BY
New
Top level Help has one additional menu item that displays core version, immerse version and whether backend rendering is enabled.
Dimension lock axes on histogram and line charts.
Range charts for histograms
Stacked histogram charts
Fixed
Issue with line chart date fields don’t reflect filter dates after clearing filter
Issue where brushing on initialized chart (Range/Line/Histogram) will extend brush filter beyond what is selected
Issue where multiple range charts and focus charts created when changing chart type
Issue where range chart with multi-series changes color of range chart when brushing range
Issue with Time entry box for line chart not applying time filter
Bug where Line chart does not cancel changes when switching from binning to extract and clicking cancel
Multi-series line chart loses color when going to and from SQL editor
Uncaught TypeError - Cannot read property ‘dimensions’ of undefined when creating a line chart
New
Improved support for INTERVAL data type, and added functions for TIMESTAMPADD, TIMESTAMPDIFF, DATEADD, DATEDIFF and DATEPART. For example the following two queries can now be run, which would give equivalent results:
Fixed
NULL now does not match other values (e.g. another NULL) in hash joins
Crash in cpu mode when a hash join inner table column is all NULLs
Crash when LIKE or REGEXP_LIKE pattern is not a literal
Problem with loading through aggregator in multi-node setups when table contains non-encoded strings or arrays
Fixed
Bug when using dict-encoded string column as color when rendering a GROUP BY query
Crash when using a heatmap query with a LIMIT
New
SQL Editor Reference panel for tables and columns
Fixed
Issue with Number chart not removing old charts when changing aggregation type
New
Division by zero behavior is now configurable via flag null-div-by-zero. New installs using systemd will default to true in their config files, meaning that division by zero returns null; existing installs will default to false, preserving the existing behavior of throwing a division by zero exception.
round_to_digit(x,y) function now supported. Rounds x to y digits.
Dictionary encoded columns may now be specified for the sql_execute_gpudf endpoint
Fast versions of statistics functions are now available. Note that these functions might overflow; use single precision float instead of double precision. STDDEV_FLOAT, STDDEV_POP_FLOAT, STDDEV_SAMP_FLOAT, VARIANCE_FLOAT, VAR_POP_FLOAT, VAR_SAMP_FLOAT, CORRELATION_FLOAT, CORR_FLOAT, COVAR_POP_FLOAT, COVAR_SAMP_FLOAT
Avoid overflows for most literal arithmetic expressions
Fixed
Crash with certain join queries when inner table is empty
Incorrect results on order by non-encoded string with filter applied on that column
Incorrect detection of overflow when multiplying decimals with negative numbers
Dictionary folders are now dropped in multi-node setups when tables are dropped
SUM and AVG overflows for tables with more than 4B rows
Crash on grouped COUNT(DISTINCT) in multi-node setups
Crash on join query when column contains all nulls
New
Vega API now supports ability to render non-projection ( GROUP BY ) queries
Added new ExtensionFunction rect_pixel_bin as a helper for rectangular heatmap binning
New symbol vega mark type. Supports circle, square, diamond, cross, triangles, and hexagons
Added support for a new percent accumulation type, which allows a region to be colored based on one category’s percent of all results at that location.
Fixed
Fix vega update issue when mark property references new data and nothing else changes.
New
Added Custom SQL Dimension feature, allowing SQL to be used to form dimensions
STDDEV now available in UI as an aggregate type
Immerse now displays End User License Agreement when product is accessed for first time from Community Edition
Fixed
Issue with pointmap and scatterplot charts, where chart image is larger than the chart widget size
Issue with widgets shifting on dashboard after changing browser window size then restoring to original size
Issue when using custom measure on Scatterplot chart
Values between -1 and 1 now display in a typical decimal format rather than with “m” abbreviation
Incorrect behavior for points auto-sizing and manual sizing (pointmap and scatterplot)
New
Multi-column hash joins now work provided the dimension table has unique composite keys
Loop join is now automatically allowed for inner tables with up to 1,000 rows
Improved performance for LIKE/ILIKE operations on dictionary-encoded columns
Improved performance for COUNT(DISTINCT) and APPROX_COUNT_DISTINCT for low cardinalities
Enhanced SQLimporter compatibility and performance with PostgreSQL
StreamInserter now offers —-quoted flag which, when set to true, removes any double-quotes in the output
Fixed
Due to an issue with the current release of glibc on many operating systems, we have moved away from a JNI interface to Calcite to avoid segfaults
Better support for batch JDBC insert (statements are no longer case sensitive)
Fix for possible race condition when using popups on a rendered chart in a distributed system
Import improvements around column detection and reject row summary
Improved error messages around user management commands
Eliminated overflow when comparing integers with high-precision decimals
Now recording rejected rows properly in case of Overflow
Now dropping string dictionaries for DROP TABLE in distributed case
Issue with AVG(float) when number of records is >2B
No notes this release
New
Chart y-axis height can now be capped using a control on the axis, for example to exclude outliers from view. Data is not filtered from the dataset, but merely excluded from view.
X and Y axes for dc charts can now have their labels edited, allowing for friendly names
SQL editor beta now offers a command-line style interface with query history
Fixed
No longer incorrectly showing non-encoded strings as available Dimension columns
This point release addresses an issue introduced in recent version of glibc. JNI fails with a segfault after the operating system had been updated. This release fixes that issue by removing the JNI usage.
MapD Core for single-node usage has been open sourced under the Apache 2.0 license (https://github.com/mapd/mapd-core/blob/master/LICENSE.md). Code available at https://github.com/mapd/mapd-core . EULA at https://www.mapd.com/legal/
New
Initial API support for GOAI consortium (http://gpuopenanalytics.com/). New Thrift API endpoint (sql_execute_gpudf) creates a query resultset on the GPU in Arrow format and returns a handle. Other GPU kernels can work directly on the data via the returned handle, reducing the data movement normally associated with staged operations.
Improved initial GPU memory load times by more fully utilizing available PCIE bandwidth
Now supporting COVAR_POP, COVAR_SAMP, CORR
Performance for certain complex subqueries, for large IN subqueries for multi-node environments, and for MIN on a nullable float or int
Float performance improvement on AVG/SUM/MIN/MAX aggregates
Multi-way JOIN queries now use memory more efficiently
Fixed
Issue where sorting GROUP BY and AVG through view of a JOIN query returns results in non-sorted order
Problem where in some cases --allow-cpu-retry flag would not fall back to CPU
Now correctly returning list of columns for views
Crash on CREATE TABLE AS SELECT when SELECT statement returns no rows
Incorrect results when doing MAX on a float in some cases
Issue with accuracy of results with drawn polygon filters
AVG and SUM on float now supported on Kepler GPUs
Allow execution on GPUs for queries larger than can fit in GPU memory when --enable-watchdog flag is set to false
Further overflow detection for decimal type
Improved precision of numeric on initial load
Reduce data transfer to GPU memory when not required
COUNT(DISTINCT) on all NULL now returns correct (empty) result
New
Default rendering now done through EGL. No longer dependent on Xorg
Multi-layered points/polygons can now be rendered in multi-node environments
Now using blosc/zstd compression for passing image data between nodes
Accumulation rendering now supported in multi-node environments
Fixed
Incorrect darkening of color for images with transparent colors composited on multiple GPUs/nodes
Issue with re-initializing counts for accumulation rendering when rendering a new frame
Issue with layers incorrectly persisting from one frame to another when number of layers changes from frame to frame
Problem with EGL compositing in multi-node environments
Error on accumulation rendering when using POW scale
Errors with removal of polygons from cache
Bugs when rendering polygons against empty query results
Issue when coloring by dictionary encoded string when string is in inner table of a joined query
Accumulation rendering crash when data from query doesn’t span the compositor’s gpu
Now not throwing error when fill/stroke is turned off between successive render frames
Fixed shader compile issue when updating the data reference for a packed color vega property
New
Pointmap basemap themes can now be changed
Raster charts can now render with a density color gradient (represents the number of points stacked at each pixel)
Chart axis labels can now be edited
Fixed
Various issues with the scatter chart
This version introduces a change in the Thrift API which breaks compatibility between 3.0 servers/clients and older servers/clients. Users with custom Thrift clients will need to re-run Thrift to generate the update client libraries.
The deprecated v.1 Immerse is no longer available as of this release. Attempts to access v.1 dashboard links will lead to a 404 message.
To use MapD Core Rendering Engine, CentOS and Redhat users must upgrade to CUDA driver version 375.51 or later.
The --disable-rendering flag has been changed to be --rendering=false
New
Native distributed scale-out capability, allowing deployment of the MapD Core database across a cluster of GPU machines
Native ODBC support, enabling MapD Core to be connected to 3rd party clients
VARIANCE STDDEV_POP`` and ``STDDEV_SAMPLE
Support for more groups in multi-column GROUP BY queries (40m - 60m groups)
Up to 10x performance improvement for IN subqueries with many values. Additional small performance improvement for IN queries with at most 3 values.
Ability to have 3+ way JOIN on a conjunction of equality expressions when each of them is one-to-one
Now offering ability to specify commonly used columns to be loaded from disk at database startup time
Supporting HH:MM time format for file import
Fixed
Issue where HAVING IS NOT NULL wasn’t applied to date extraction/truncation expressions
Issue with 3-way JOIN using star schema
Fix import correctness issues when rows are rejected
Issue where non-grouped queries returned incorrect number instead of NULL when no rows match for MAX / MIN aggregates
Issue with potential corruption during overlapping parallel loads to single table
Issue where column metadata for VIEW/s would show the term “Explanation” instead of proper metadata
New
Rendering now uses EGL instead of GLX, removing the Xorg requirement
Fixed
Crash when rendering JOIN queries
New
Dimension axis now dynamically updates during cross-filtering to maintain fine granularity as data ranges get smaller
Fixed
Issue with Line/Histogram chart where brushing range would extend filter beyond what is selected
Chart spinners, used to indicate network wait time, are now less intrusive and appear/disappear more quickly
Restricting arrays from being selectable as dimensions/measures when not available, and visually indicating them more clearly
Certain problems with handling NULL
Certain problems with shape selector tools
This sitemap link is for the benefit of the search crawler.
Archived release notes for Release 5.x
5.10.2 | 5.10.1 | 5.10.0 | 5.9.0 | 5.8.1 | 5.8.0 | 5.7.1 | 5.7.0 | 5.6.4 | 5.6.3 | 5.6.2 | 5.6.1 | 5.6.0 | 5.5.6 | 5.5.5 | 5.5.2 | 5.5.1 | 5.5.0 | 5.4.2 | 5.4.1 | 5.4.0 | 5.3.2 | 5.3.1 | 5.3.0 | 5.2.2 | 5.2.1 | 5.1.2 | 5.1.1 | 5.1.0 | 5.0.1 | 5.0.0
The COPY TO command now exports time, date, and timestamp data types in ISO 8601 format. Previously, date and timestamp data types were exported as unix epochs.
Fixed crash that could occur when window function partitions were sorted on multiple columns.
Fixed crash that could occur when in-situ queries had query step punted to CPU.
Fixed crash that could occur when columnar output is forced for queries projecting variable length types, either by command-line flag or query hint.
Fixes hit-testing issues in single-node elated to rendering SQL with aggregate subqueries. Such queries could be generated using the cohort feature in Immerse or using custom dimensions. Previously, you could see errors such as the following in map charts:
Cannot find column <column name> in hit-test cache for query <sql query>.
Most errors of that form should be resolved in single-node.
Fixed New Combo Chart legend appearing over chart bars.
Fixed 'Failed to Load Tables' error when navigating to the data manager.
Fixed issue where projections with more than 1M rows of geo and other variable-length types could crash the server when –columnar-large-projections is enabled.
Fixed issue with rendering results of certain subqueries when running distributed.
Fixed data manager error occuring when global custom expressions beta feature is enabled.
Upgrade log4j to 2.17.1 to address CVE-2021-44832.
The SAML Entitlement feature is deprecated and may be removed in a future release. Row-level security (RLS), with a role specified through SAML, provides an improved replacement.
Row-level security (RLS): Administrators can use new commands CREATE POLICY, SHOW POLICIES, and DROP POLICY to apply security filtering to queries run as a user or with a role.
Significantly more performant, parallelized window functions, executing up to 10X faster than in Release 5.9.
Automatic use of columnar output (instead of the default row-wise output) for large projections, particularly benefitting window functions, subqueries, and table function calls that returned large numbers of rows, lowering query times by 5-10X in some cases. The threshold defaults to 1M rows, but can be modified with thecolumnar-large-projections-threshold flag (and turned off with columnar-large-projections=false).
UNION ALL now works with tables that do not share dictionary columns and supports query patterns such as grouped inputs. Previously only projections were supported.
Large IN subqueries are now significantly faster and scalable by rewriting the associated query plans as decorrelated joins.
Added support for full set of ST_TRANSFORM SRIDs supported by geos/proj4 library.
Significant ST_DWithin speedups involving LINESTRING types via an optimization to clip the checked region of the LINESTRING to just the portion that can possibly fall within the specified distance of the other geometry.
Removes a previous restriction where null values were not allowed for text array columns.
Adds ANY_VALUE as canonical SQL alias for existing SAMPLE aggregate operator.
Support for full set of ST_TRANSFORM SRIDs supported by geos/proj4 library.
High performance UTM transforms (ST_TRANSFORM) available to and from web mercator and geographic coordinate systems.
Significant ST_DWithin speedups involving LINESTRING types via an optimization to clip the checked region of the LINESTRING to just the portion that can possibly fall within the specified distance of the other geometry.
Adds support for import from dozens of image and raster file types, such as jpeg, png, geotiff and ESRI grid, including remote files.
Adds support for numerous vector GIS files (100+ formats supported by current GDAL release).
Adds support for multidimensional array import from GRIB/GRIB2 formats common in science and meteorology.
When importing GRIB/GRIB2 raster files, the default multithreaded import behavior is unstable due to a bug in the GDAL library; this issue will be addressed in the next release. You must use the WITH option threads=1 to force single-threaded operation and avoid a server crash. This restriction does not apply to any other raster formats.
Adds support for selected layers or bands within multilayer files.
Adds a new flag to SQLImporter that allows narrowing of integer values.
Improves point import speed regardless of source.
Extended file metadata and column detection in omnisql to include additional formats.
Fixes an issue in the ODBC driver where some SQL statements containing double quotes would result in an error.
Fixes an issue where decimal value imports could fail, even when the specified number of maximum rejections has not been reached.
Fixes a crash that occurs when the max rows option is set for an empty table.
Fixes a potential ghosting artifact when rendering consecutive accumulation renders in a multi-gpu system using the OpenGL driver.
Introduces dashboard-level Named Custom SQL.
Improves Table Chart export to access all data represented by a table chart.
Adds overridable CSS file (override.css) for custom CSS in embedded implementations.
Panning support in the New Combo chart with alt+scroll.
New color picker user interface and made other UI improvements.
Alerts for unsaved dashboard changes.
Geo-joined Choropleth charts now spatially cross-filter the fact table on zoom and pan.
Expanded functionality of the Iframe API (Beta).
Introduces optional higher data throughput Arrow Transport (Beta).
Expanded functionality for Embedded UI Customization (Beta).
Adds database-level Global Custom Expressions (Beta).
Adds cohort support for crosslinking (Beta).
Fixed application of Configuration UI changes to discretion charts’ legends.
Fixed custom SQL expression support for custom data sources.
Fixed various cases for upgrading legacy charts to the New Combo chart.
Fixed various minor UI issues.
Adds IF EXISTS support to DROP USER and DROP ROLE statements.
Significant speedup for POINT and fixed-length array imports and CTAS/ITAS--generally 5-20X faster.
Fixes an issue where null values and empty arrays were not imported correctly for text array columns.
Correctly account for compressed size of 8-bit and 16-bit dictionary encoded text columns and all geo type columns to avoid overly aggressive punting of queries to CPU when columns of these types are used in a query.
Prevent overflow of columns with widths less than 4 bytes when columnar projection used in a query, which could lead to a crash.
Improves the performance of density accumulation stats gathering when using min/max/1stStdDev/2ndStdDev ranges, up to 50x speedups in some instances.
The PNG encoding step of a render request is no longer a blocking step, thereby improving render concurrency.
Fixes a potential timeout in the Vulkan renderer in a multi-GPU server configuration if GPU contention caused by many queries/renders executing at the same time exists. The timeout can manifest as a VulkanDeviceLost error that can lock up renders for a minute before the renderer can recover.
Fixes a minor memory leak when caching parsed Vega JSONs.
Fixes a possible ghosting artifact in multilayer/multi-GPU renders when individual layers use different subsets of available GPUs when rendered.
BETA - Adds custom expressions to table columns.
BETA - Adds Crosslink feature with Crosslink Panel UI.
BETA - Adds Custom SQL Source support and Custom SQL Source Manager.
Adds support to hide deprecated chart types from add/edit chart menu.
Improved speed of user role verification.
Fixes minor issues in the Parameter manager UI.
Fixes New Combo chart binning migration.
Fixes issues in the global side navigation.
Fixes fetching a cohort count when a custom SQL filter contains a parameter.
Fixes Vega chart color-utils to always fetch latest color palettes.
Fixes filter removal after changing and resetting the default value if you use a SQL custom dimension as a parameter.
Fixes minor UI styling issues.
Fixes custom source selection in Vega source selector.
Fixes cross-filter support with geo-joined bounding boxes.
Fixes resetting of parameter definitions when canceling from inside the chart editor view.
Fixes new cases of chart title helper support.
Fixes New Combo Bar label automatic formatting.
Added a new SHOW ROLES command for viewing directly assigned and effective user roles.
Fixed a regression in Release 5.8.0 where purging idle render sessions can result in server crashes with the OpenGL renderer.
Fixes a regression in Release 5.8.0 where poly renders can result in segfaults.
User role checks that block dashboard load and login are completed much more quickly.
Release 5.8 officially enables the new Vulkan backend renderer as the default, replacing the OpenGL renderer. For more information on Vulkan, reasons for the change, and troubleshooting, see Vulkan Renderer.
The legacy OpenGL backend renderer is still available as a fallback, but will be deprecated and removed in subsequent releases, so it is advisable to use the legacy OpenGL renderer only if you have a blocking issue with Vulkan. To disable the Vulkan renderer and enable OpenGL, add the following flag to your server configuration:
renderer-use-vulkan-driver=false
In distributed clusters, the configuration file for each node must have this parameter set to use OpenGL.
Releases 5.7 and higher require the installation of a Vulkan API loader library to support the Vulkan renderer. This library is required regardless of whether you are using the Vulkan renderer or not. If you are upgrading from Release 5.7, you have already installed this library and no further action is required.
If you see an error similar to the following when trying to start a renderer-enabled Release 5.8 server for the first time, you need to install the Vulkan API loader:
error while loading shared libraries: libvulkan.so.1: cannot open shared object file: No such file or directory
To install the Vulkan API loader:
On CentOS: sudo yum install vulkan
On Ubuntu: sudo apt install libvulkan1
For a summary of how to install the Vulkan loader on other various Linux distributions, see https://linuxconfig.org/install-and-test-vulkan-on-linux.
For other troubleshooting issues, see Vulkan Renderer.
Parallel executors now on by default (with default --num-executors=2)
Spatial joins between point types using ST_Distance type are now accelerated using overlaps join hash framework, with increased in speed up to 100x.
Window functions can run on empty partitions and can operate over tables with multiple fragments and shards.
Queries that need to run a query step on CPU due to memory pressure or compatibility reasons now execute only that individual step on CPU. Previously, the full query was restarted and all steps ran on CPU.
Support WIDTH_BUCKET operator for easier numeric binning.
Natively support ST_Transform to/from all UTM zones and EPSG:4326 (Lon/Lat) and EPSG:900913 (Web Mercator).
Support provided for file path regex filter and for file path sort order when running the COPY FROM command.
New ALTER SYSTEM CLEAR commands enable clearing CPU or GPU memory.
Error messages for DUMP/RESTORE commands are improved.
Validations and error messages are improved when specifying default column values.
More robust handling is added for long decimal strings during import.
Non-superusers can view and interrupt their own queries.
Rewrote query plans where certain aggregates are performed on the same expression that is grouped-by (that is, COUNT DISTINCT column where column is being grouped) to improve performance for these queries
Fixed an issue where window functions with a preceding filter would pull all of a table’s columns into memory, even if unused by the query.
Disabled window functions for updates because it could crash the server. Targeted to be fixed and enabled in a future release.
Various fixes related to aggregate window functions.
Improvements to auto-casting logic for extension functions, user-defined row functions, and table functions.
Avoid unnecessary translation between dictionary-encoded text columns for certain classes of hash joins when the inner and outer tables’ join columns share the same dictionary, thereby improving performance.
Fixed crash that could occur when performing a join on multifragment input when the hash table is built on GPU but the join is executed on CPU.
Various bug fixes and performance improvements for runtime query interrupt.
Fixed broken Immerse dashboard import in omnisql.
The default renderer is now Vulkan. The legacy OpenGL renderer can still be used as a fallback. See the notice at the top of these release notes for more information.
Improved the memory footprint/performance of the multi-GPU compositor by doing overlapped, tiled transfers from render GPUs to the compositor GPU. The more GPUs on the node, the bigger potential gains.
Added extra logging in the event of a timeout or Vulkan device lost error.
Migrated numeric binning in Immerse to use the new WIDTH_BUCKET SQL operator, significantly improving binning/histogram performance in some cases.
BETA: Added a new global side navigation.
BETA: Added support for 3D terrain in 3D scatter chart.
BETA: Support for hiding Immerse UI elements in iframed app usage is added.
BETA: Crossfilter referencing in custom filters is supported.
Fixed an issue where the parameters dropdown would not autocomplete.
Fixed incorrect file headings in chart map export.
Fixed multilayer legend value for Choropleth chart with geo heat map.
Fixed display of long dashboard titles.
Fixed dashboard duplication permission case.
Fixed hit-test error on a custom measure.
Fixed SQL syntax error on Pointmap chart with custom SQL group-by dimension.
Fixed rendering of Scatter Plot and Pointmap charts after hovering over points.
Fixed various minor UI bugs affecting overall usability.
Fixed SAML login to a specified database based on login URL with database name.
Fixed role retrieval for usernames with @ and other symbols ($ & + , ; : / = ?).
Query Capabilities
Added support for default column values. When creating tables or adding new columns to existing tables, you can now specify column default values. This works in both SQL and with Thrift APIs.
Added support for APPROX_QUANTILE, with performance and functionality similar to APPROX_MEDIAN, which is APPROX_QUANTILE called with a 50% quantile argument.
Per-kernel interrupt performance significantly improved, now on by default. Queries can be interrupted using Ctrl + C in omnisql, or by calling the interrupt API.
Performance
The Arrow data frame API now differentiates between where the query is run and where the Arrow result is requested. For example, a query that must run on CPU can be pushed to GPU for Arrow IPC sharing, and a query that would normally run on GPU can still run on GPU, even if the user requests the Arrow buffer in CPU memory via IPC sharing.
Improved performance of high-cardinality group-by queries. Large cohorts in Immerse should show substantial performance improvement.
Parallel executors now in public beta (set with --num-executors flag). In future versions of OmniSci, the number of executors will be set to 2 by default.
Administrative
Added a new REASSIGN OWNED command, which changes ownership of database objects (tables, views, dashboards, etc.) from a user or set of users in the current database to a different user.
Added a new SHOW USER DETAILS command for introspecting user information.
Error messages have been updated to remove redundancy and otherwise extraneous content.
Geospatial
Added support to ST_Transform for transformations from 900913 (web mercator) to 4326 (lon/lat).
ST_Contains and ST_Intersect joins and filters can now run fully accelerated on dynamically constructed points using ST_Point(lon, lat).
Geospatial projection support for points is now more robust, faster, and widely supported. For example, ST_SetSRID(ST_Point(id,id),4326) is now supported in projections.
Major enhancements to ODBC driver, including support for Geospatial types.
Added FlatGeoBuf import/export support, which is about 3x faster than shapefile and 8x faster than geojson.
Improved handling of geospatial columns in intermediate results and temporary tables. Previously, a "columnar conversion not supported" error would be thrown in joins involving multi-fragment geospatial tables. Now, the geospatial column can be zipped up, allowing joins and similar operations to proceed.
Enhanced AWS Permissions and Session Management
Added support for using IAM roles or server permissions when importing data from AWS S3. Admins can enable the use of IAM roles when running on an EC2 instance. Credentials can also be configured on the server either through AWS environment variables or credential files. Enable this option with the allow-s3-server-privileges server configuration.
Added support for AWS session tokens through omnisql and Thrift import APIs.
Improved messaging related to an error that can occur when the sample data insert script downloads data to a path outside the server data directory.
Fixed an issue where the server can encounter an error on startup due to a pre 4.0 release migration bug.
Fixed a race condition that can occur when SELECT queries and auto-vacuuming execute concurrently.
Projections without limits now should not re-compile if query literals change, significantly increasing performance
Arrow over-the-wire query requests were always being executed on CPU, now they will run on default device type (and can be overridden by the user with a query hint)
Fix crash that could occur with empty Arrow result sets
omnisql now returns error status codes if a single command is called and the command fails. This allows omnisql to be more easily embedded into scripts with error checking.
Resolved an issue where join table reordering could fail for some geojoins, resulting in an error message or crash.
Resolved a crash that occurred when inserting a NULL value into a geospatial column in distributed mode.
Various bug fixes and performance improvements for geospatial types in Insert Into As Select / Create Table As Select queries in distributed mode.
Resolved a crash that occurred when updating a BOOLEAN array column.
Resolved null handling issues when grouping by BIGINT columns.
Resolved an issue which prevented CURRENT_TIME() or NOW() from being used in aggregate queries.
Resolved an issue involving CASE statements with string literals in one of the case branches, where the returned results could be incorrect.
Comparison between full array and indexed array columns will throw an appropriate error. Previously, SQL with such a comparison could cause the server to crash.
Resolved an issue where NULLS LAST in an ORDER BY clause could cause an incorrect ordering with respect to sign (e.g. negatives before positives).
Resolved an issue where CTAS/ITAS queries with small returns in the SELECT statement could enter an infinite loop in distributed mode.
Resolved an issue preventing the owner of a database from dropping that database.
Release 5.7 includes the official beta release of the new Vulkan-backed renderer, which will replace the current OpenGL renderer. For more information, see Vulkan and the reasons we are making the switch, see Vulkan Renderer (Beta).
Because Vulkan will be the default renderer in Release 5.8, OmniSci strongly recommends using it now. The OpenGL renderer will be deprecated and removed in subsequent releases. The Vulkan renderer is reliable and stable, and switching to Vulkan now can help reveal any unforeseen issues. Finding such issues early, while an easy fallback exists, ensures a smooth, less risky transition.
To enable the beta Vulkan renderer, set the renderer-use-vulkan-driver configuration parameter to true.
In distributed clusters, the configuration file for each node must have the parameter set.
The Vulkan library is required for Release 5.7, regardless of which renderer you use. If you do not install the renderer, you will see the following error when trying to start a renderer-enabled server for the first time:
error while loading shared libraries: libvulkan.so.1: cannot open shared object file: No such file or directory
If you receive this error, you need to install the Vulkan API loader:
CentOS: sudo yum install vulkan
Ubuntu: sudo apt install libvulkan1
For a summary of how to install the Vulkan loader on various Linux distributions, see: https://linuxconfig.org/install-and-test-vulkan-on-linux.
Significant speedups in large polygon renders across multiple GPUs. The larger the table, and the more GPUs, the bigger the speed increase.
Memory footprint improvements in compositing and anti-aliasing components.
3-4x speedup with procedural rendering (i.e. “symbol” vega mark type)
Better pipelining of compressed geo columns and projection math; 2x memory footprint improvement in use cases
Added a new Airplane symbol shape for Vega that can be used in symbol/legacy symbol marks.
Significant enhancements to Immerse parameters.
Dashboard parameter widgets:
Side-panel parameter controllers can be added to dashboards.
Supports sophisticated custom dashboards with no code required.
Parameter value display across the product:
Chart titles and chart axes labels
Dashboard titles
Tooltips and Legends
Convenience methods to use parameters within:
Chart column selectors
Simple filters
Quick filters
Enhanced parameter management:
Show/hide hidden parameters
Improved user interface for tracking parameters usage
Allow parameter usage in Demo mode
Improved parameter selection autocomplete in all chart types
Improvements in Geo charts:
Ability to set top color category in Pointmaps.
Improved gradient selection in Pointmaps
Support Zoom Level in Map Chart `Zoom to` Field
High-precision, higher-performance lasso tool:
Now uses an ST_Contains filter expression for drawn polygons instead of expensive inside-triangle expressions.
Employs a dynamic level-of-detail drawing algorithm that automatically adjusts the resolution of parts of a polygon/circle that are in view so that it matches the results of the ST_Contains filter exactly for map charts.
3D Pointmap chart (Beta release).
Embed HTML in a text chart (Beta release).
Updated mapd-connector with latest version of Thrift.
Unique category support for larger unique category sets.
Apply numeric filters using parameters.
Fixed display of parameters on new dashboards after a session timeout.
Fixed Choropleth hit test when hovering over a polygon.
Fixed display of boolean values in the input field of a global filter and dropdown menu.
Fixed parameter support in Choropleth charts with a geo join.
Fixed reversed axes labels on New Combo chart.
Fixed BIGINT column support in cohort builder.
Fixed display of parameter value in filter component input.
Fixed boolean setting of column value parameters.
Upgrade charts with column parameters set as dimension.
Fixed errant session clearing for non-autologin instances.
Fixed missing support for map-move crossfilter feature flags in Combo chart.
Fixed imperial formatting in popups.
Fixed a multilayer popup visibility when hovering over a line or point.
Various minor UI improvements and bug fixes.
OmniSci release 5.6.4 includes a fixed issue.
Resolved an issue where cleaning up an expired session could cause the server to crash.
OmniSci release 5.6.3 includes a new feature and an improvement.
ITAS and CTAS queries are now protected from entering an infinite loop when small numbers of records are returned.
Support for encrypted credentials in connectors is decoupled from authentication through Immerse.
OmniSci release 5.6.3 includes a new feature and an improvement.
ITAS and CTAS queries are now protected from entering an infinite loop when small numbers of records are returned.
Support for encrypted credentials in connectors is decoupled from authentication through Immerse.
OmniSci release 5.6.2 includes new features and improvements and fixes several issues.
Added a new 'airplane' symbol shape.
When using INSERT to insert floats into INT columns, the numbers are rounded instead of truncated; now consistent with COPY behavior.
Fixed an issue where UDFs created at server start time with the -udf flag were not generating code for the GPU.
Resolved an issue where certain queries with a smaller number of groups ( < 20,000) would fail with a “Ran out of slots in the query output buffer” message.
Reduced incidence of “Ran out of slots in the query output buffer” errors with large group-bys.
Improved READ-ONLY configuration to never write to underlying storage.
Resolved an issue where a left join could get incorrect results if IS NOT NULL was part of the WHERE clause for the query.
Resolved sort order with aggregated results (negatives before positive) when NULLS LAST is specified.
Fixed an issue causing a crash when ITAS/CTAS populates a NULL point geometry object.
Resolved and issue where COPY FROM was restricting all other access in a distributed setup
Improved AVG handling when NULL values are used.
Fixed a random segmentation fault that occurred when running render unit tests.
Immerse now accepts a set of encrypted credentials for secured authentication of a custom application.
Improved support for creating parameters on a new dashboard.
Resolved an issue where the record count on a zoomed map would reset after editing.
Fixed return key support for input of column type parameters.
Fixed the resetting of column and default value after a parameter is created.
Fixed caching of the number chart with a chart level filter.
Resolved issue related to support of Booleans for column value parameters.
Fixed an issue affecting null column values in quantitative color palette scales for Heat charts.
Batch upgrading of charts includes additional edge cases.
Resolved issue related to support for custom color dimensions in New Combo charts.
OmniSci release 5.6.1 includes a new feature and fixes several issues.
Fixed an issue where a rename of a user with ALTER USER USER RENAME TO USER_NEW could cause issues later when trying to restart the server, depending on role membership.
Fixed a system crash that occurred during an INSERT TABLE AS SELECT command on a table that does not exist.
Fixed an issue where polygons from a table which had been appended using SQLImporter might not render. Added command-line option --noPolyRenderGroups/-nprg to disable render group assignment, to match the available WITH option on other import workflows.
When basemap sources are not available--for example, if the user is on a private network with no external access--the user is set to offline mode and Immerse uses the lightweight map source bundled with Immerse.
Fixed an issue to make sure that Combo charts correctly plot on autobinning when the start-of-week flag is set to Saturday or Sunday.
Duplication of charts with an empty filter is now allowed.
OmniSci release 5.6.0 contains several new features and improvements and fixes a number of issues.
Allowed import and export paths are now enforced. Confirm that any import or export paths are included via the configuration parameters allowed-export-paths and allowed-import-paths.
Loading data now more strictly enforces validation or numeric inputs. Previously, an out-of-range integer would “wrap” and be ingested as incorrect values. In Release 5.6, numerics are validated and if validation fails, the row is rejected.
String dictionaries do not load if you exceed their capacities; previously, they inserted null for the string.
Utility jar mapd-1.0-SNAPSHOT-jar-with-dependencies.jar is renamed. Scripts using SQLimporter need to reference the new .jar omnisci-utility-5.6.0.jar.
Significant speedups for point-in-polygon joins (ST_CONTAINS, ST_INTERSECTS) via optimized algorithms and range join hash framework; on by default for point-in-polygon joins.
Approximate median function support through the new approx_median aggregate function. Currently works only in nondistributed deployments and executes on CPU.
Query interrupt is now possible when the query is outside of core kernel execution (for example, copy/import, CTAS, ITAS, query data fetch, result reduction). On by default and controlled by new flag enable-non-kernel-time-query-interrupt.
Query interrupt during core kernel execution has been optimized so that enabling it with the existing enable-runtime-query-interrupt flag should have little to no impact on query performance; actual impact depends on the query pattern.
Added partial column specification for INSERT and INSERT FROM SELECT, such that data can be inserted and appended to a subset of columns in a table. NULL values are added to columns not specified in the statements.
Updated the CREATE TABLE command max_rollback_epochs option with a default value of 3. This reduces the number of epochs/data versions that are stored. When combined with new space reclamation on delete, optimizes space utilization. Setting the option with a high value stores more epochs and reduces data compaction and space reclamation. To update older tables, use the ALTER TABLE <table> SET MAX_ROLLBACK_EPOCHS=<value> command.
Added automatic metadata updates and vacuuming. On UPDATE and DELETE queries, OmniSciDB updates metadata, vacuums, and re-uses space as specified. Automatic metadata updates are turned on by default. See enable-auto-metadata-update in Configuration Parameters. Automatic vacuuming on delete occurs when deletes or updates on variable length columns exceed the vacuum threshold. For more information, see vacuum-min-selectivity in Configuration Parameters.
To ensure optimal space usage, ensure the table’s MAX_ROLLBACK_EPOCHS option is set to 3 or lower. This is automatically set by default for the CREATE TABLE command on new tables created in release 5.6 and higher.
For existing databases, OmniSci recommends using ALTER TABLE <table> SET MAX_ROLLBACK_EPOCHS=3 to cap the space usage, and OPTIMIZE TABLE [<table>] WITH (VACUUM='true'); to reclaim space. This primes the system for efficient space management.
Added RENAME TABLE command to enable one or more tables to be renamed at the same time.
OmniSciDB now enforces permitted filepaths for import/export operations by default. The default import and export paths ({data directory}/map_import and {data directory}/map_export respectively) are allowed by default, so no change is required when doing an import/export from Immerse or for sample data import on server startup. However, when using other commands with user-provided paths (such as COPY FROM) provided paths must be under an allowed root path. See the Configuration Parameters for more information.
NOTE: For distributed systems, allowed paths only need to be in the configuration file on the aggregator. However, for consistency and future compatibility, OmniSci recommends that the allowed paths also be added to the leaf omnisci.conf file.
Significantly improved OmniSciDB startup time. Removed extra metadata seeks/reads during startup. Systems with high epoch counts or large column counts will have marked improvements.
Validation of numeric inputs is more strictly enforced on load. Previously an out-of-range integer would “wrap” and be ingested as some incorrect value. Numeric-typed columns are now validated, and if validation fails, the row is rejected.
String dictionary reaching capacity during import now throws an error and stops the load instead of inserting nulls in the dictionary-encoded column.
Significant performance improvements for load into high-cardinality dictionary-encoded text columns via optimized dictionary hash table resizing.
Improved load performance on systems with many cores. Default max number of threads to use is capped at 32, instead of 2X the number of cores, which can cause contention on systems with many cores.
Improved the performance of left join queries with filters by searching for left-hand side filters that can be executed before the join, potentially reducing the cardinality of the join.
Support date format %m/%d/%y with 2-digit year on data import.
Improved the performance of common multicolumn sort patterns.
Chunksize default increased to 2 GB to allow for more records per fragment, particularly when variable-length columns are present (none-encoded strings, arrays, and geospatial types). This allows certain query patterns, such as large joins that might previously span multiple fragments, to run faster.
Entitlements (introduce simple row level security via SAML attributes).
Accept session-token when using S3 import, WITH option is s3-session-token.
Upgrade to calcite 1.25.
Improved the accuracy of ST_Contains and ST_Intersects at small geospatial scales.
Allow Narrowing casts in SQLImporter.
Upgrade to sqlite 3.34.
Allow SAML and LDAP to provide authentication without having to also provide authorization.
Utility jar renamed and versioned: omnisci-utility-5.6.0-SNAPSHOT.jar.
Added support for CURRENT_TIMESTAMP, CURRENT_DATE and CURRENT_TIME functions
Improved READ-ONLY support.
UDTFs now allow for a ColumnList argument type, which allows a variable number of arguments to be used for inputs.
Improved the granularity of the session lock, allowing local users to access the system even if third-party authentication providers have failed or are experiencing high latency.
Many bug and performance improvements.
Fixed issue causing a crash when UDTFs were composed.
Fixed an issue where definition of CPU UDTFs could lead to all subsequent queries running on CPU.
Resolved an issue where selecting from a view with a self join could cause a crash.
Improved the ability of the parser to handle complex UPDATE and DELETE queries.
Added human-friendly error messages to UPDATE and DELETE queries.
Resolved an issue where specifying warmup queries would cause the aggregator to crash in distributed mode.
In distributed mode, specify warmup queries for the aggregator and all leaf nodes to work correctly.
Resolved an issue where an equijoin between array columns could crash. Now, an attempt to join throws an exception if the join is not supported.
Resolved an issue where window functions in a subquery could cause a crash.
Resolved an issue where window function aggregates could crash if used with filters.
Improved sorting performance of and resolved a number of stability and correctness issues.
Resolved an issue in distributed mode where subqueries over replicated tables could return duplicate results.
Resolved several potential deadlocks between the rendering engine and the query engine when multiple queries or types of queries are in process.
Fixed an issue where SHOW CREATE TABLE would show the wrong table name after a table with shared dictionaries was renamed.
Resolved an issue where a COPY TO command with a subquery could fail.
Improved the quality of density accumulation rendering by incorporating a technique that uses 64-bit atomic operations instead of 32-bit when calculating standard deviation. This allows for a greater range of colors when using statistical measures to color the densities. The 32-bit technique was prone to uncaught overflows that resulted in a much more clamped color range.
Performance improvement when using a Mercator projection explicitly for marks using the Vega projection property.
Multi-GPU compositor performance and memory footprint is improved through better pipelining and reduced copy operations.
New Arrow stock symbol shape can be used in symbol/legacy symbol marks.
Line and polygon stroke rendering improved by 30-40% by simplifying the math to build the stroke geometry.
Significant improvements to the quality of point/symbol rendering, achieved by improvements to procedural anti-aliasing technique and to render sample capturing when drawing procedural symbols.
Polygon geometry imported via the SQLImporter path can now be rendered.
Fixed a bug where decimals are not properly updated in vega scales after an empty render.
Fixed a rare CUDA misaligned address crash when rendering multilayer vega with points, geo, and xforms in consecutive vegas.
Resolved a small host memory leak in the renderer.
Fixed a polygon render issue where fully transparent polygons can inadvertently affect the renders of visible neighbors.
Fixed a crash when attempting to render a line mark layer with opacity explicitly set to 0.
Fixed a potential race condition when DROP or CREATE is run in between renders or while render is in flight. The race condition could result in a crash when executing the get_result_row_for_pixel endpoint.
Corrected an issue with multilayer rendering with multiple density accumulation layers where all density layers incorrectly applied the color palette from a single layer.
Fixed a crash that occured when rendering a line mark in a multi-GPU-configured server/leaf when the number of GPUs required to render the lines jumps from one to more-than-one GPU when rendering consecutive vegas.
Corrected the stroke rendering of ellipse symbols.
Corrected possible skew in symbol rendering when using the angle property.
Custom SQL dimensions, measures, and filters can now be parameterized, enabling more flexible and powerful analysis:
Support for column, attribute, numeric, and custom parameter types
New Parameter Manager where parameters are created, modified, and deleted.
Parameter panel on the dashboard supports dynamic edits of the parameter values, and the ability to link/unlink parameters across dashboard tabs.
All charts have been upgraded with the new custom SQL modal, which contains better SQL validation and ability to inject parameters and columns into the SQL.
Chart-level error handling and recovery.
Median added as a supported aggregation type across all charts and aggregate filters.
A new angle measure has been added to Pointmap and Scatter charts, allowing orientation data to be visualized with wedge and arrow icons.
When you mouse over a point, line, or poly on Pointmap, Choropleth, or Linemap charts, a popup box appears, where you can copy the popup information to your clipboard. If the popup includes a URL, you can click the URL to open the web page.
The Other category can be toggled off from the categorical color legend on map charts.
Immerse charts can now be configured with a Saturday or Sunday start of week by using the servers.json ui/start_of_week flag. Supported start of week days include Saturday, Sunday, and Monday (default behavior).
Zoom and panning performance has been significantly improved on New Combo charts.
A new confirmation modal has been added to the Reset Dashboard Settings action to help prevent accidental loss of work
On multilayer charts, unchecking layers from the master layer view now persists those changes and prevents unchecked layers from being displayed.
Fixed issue on charts where click-based cross-filters from binned time dimensions were applied incorrectly.
Fixed issue on Table charts where the chart did not highlight the row correctly after filtering on a boolean value.
Fixed issues where Map and Stacked Bar chart data did not refresh correctly with the dashboard set to auto-refresh.
Fixed issue in the global filter panel where the calendar picker was hidden behind charts when the dashboard was scrolled down.
Fixed issue on map charts where color legends were not displaying correctly for multilayer charts.
Fixed issue in New Combo charts where brush filters were removed when filter sets were switched.
Resolved issue in New Combo charts where the range and focus charts were out of sync when there were multiple sources on the chart.
Fixed issue in New Combo charts where newly created columns were not available for data selection.
Fixed issue on new combo charts where x-axis labels were not updating correctly when sorting time columns.
On New Combo charts, brush filters are now set correctly on duplicated charts.
Fixed issues on Table charts where formatting was applied incorrectly to custom measures and date dimensions.
Resolved issue on New Combo charts where bars with certain custom SQL measures in percentage mode did not render correctly.
OmniSci release 5.5.6 fixes an issue.
License key input removes extraneous characters from pasted text.
OmniSci release 5.5.5 includes a new feature and fixes an issue.
Align license management for Enterprise Edition and Free Edition.
Resolved sign-on issues after idle session timeout expiry.
OmniSci release 5.5.2 includes new features and fixes several issues.
SQLImporter allows text-based WKT or WKB fields to be imported into an appropriate geo column.
A new command-line option, log-user-id, enables logging internal numeric user IDs instead of textual user names.
OmniSciDB Arrow distribution is upgraded to v2.0.0.
Resolved an issue where concurrent CTAS runs could deadlock.
Resolved an issue where replacing an existing dashboard could cause the server to crash.
Corrected behavior of multicolumn sort with NULL values.
Resolved an issue when using ANY with a decimal array.
Resolve an issue when loading BIGINT data into an ARRAY.
Resolved an issue when using OFFSET with no LIMIT.
Fixed incorrect left join-to-inner join optimization rule.
Improved handling of CASE statement literals when dictionaries are used.
Improved overflow checking.
Fixed issues for persistence of deleted rows after OPTIMIZE.
Improved locking for CONNECT to be less blocking.
Fix a multilayer Vega render bug in which two consecutive renders with minimal changes in the code (for example, a change in layer ordering for) can result in a cryptic exception, but the Vega code is otherwise legal.
OmniSci release 5.5.1 includes a new feature and fixes several issues.
Resolved an issue where newly created databases are unusable after server restart.
Fixed an issue that could cause a crash during UPDATE or DELETE queries.
Resolved an issue where UPDATE queries with a join would run indefinitely.
Resolved an issue where a GPU-enabled version of OmniSciDB without GPUs present failed to gracefully fall back to CPU mode, resulting in a crash.
You can now configure max legend width on charts from the Settings UI panel. You can save the configuration per-dashboard or database-wide.
Fixed an issue where cohort filters were not applied correctly to New Combo charts.
OmniSci release 5.5.0 contains several new features and improvements and fixes a number of issues.
OmniSci 5.5 includes a significant change to the OmniSciDB storage layout. The migration from the storage layout prior to Release 5.5 is transparent, but is not backwards compatible. (OmniSci in general does not guarantee backwards compatibility.) Back up your OmniSciDB data directory before upgrading. If you are an OmniSci Enterprise customer, you can contact Support for help.
VALIDATE command now detects and identifies out-of-sync epochs.
Added support for concurrent SELECT and UPDATE/DELETE queries for single-node installations.
New SHOW TABLE DETAILS command displays storage-related information for a table.
Adds rendering support for CPU-only configured query execution.
CPU-only query execution can be CUDA-disabled, or CUDA-enabled with the cpu-only configuration parameter set to true.
In either scenario, rendering support requires at least 1 server-side GPU.
iGPU support is not guaranteed because of Linux driver incompatibilities with an OpenGL driver.
User-defined function (UDF) support for NONE ENCODED strings.
Aggregates in user-defined table function (UDTF) output are supported.
Multi-table inputs into UDTFs is supported.
JDBC and in binary endpoints load_table, load_table_binary, load_table_binary_columnar, and load_table_binary_arrow now support out-of-order and column subset inserts.
Added support for geospatial transformation to SRID 4326 on CSV import.
Added support for the ST_Centroid geometry constructor.
Added support for importing geospatial data in WKB format from CSV files.
Improved support for distributed mode and sharded tables to prevent data truncation.
Improved support for query interrupt and query interrupt flag added. Query interrupt enables superusers to show a list of pending queries, and allows users to interrupt their own queries. This is currently considered beta functionality and off by default.
Added MAX_ROLLBACK_EPOCHS support, which limits the number of rollback states kept on disk. Useful for deployments with small rolling updates and for managing table size on disks with heavy insert/update patterns.
To run OmniSciDB in GPU mode, OmniSci Release 5.5 requires NVIDIA driver version 450.xx or higher.
Resolved a crash occurring on ST_DWithin when providing distance argument from another column in the same row.
Fixed an issue where loop joins with large inner tables could crash.
Resolved precision issues with date addition functions.
Resolved issues that could occur when appending empty polygons to a table.
Improved handling of hash tables near the entry count limit.
Added a parameter to DATE_TRUNC to allow weeks to start on either Sunday or Monday.
Resolved issues importing geospatial data from PostgreSQL using the SQL Importer utility.
Fixed a security issue in the LDAP connector.
The --help and --version server commands no longer require a data directory to exist before printing output.
Annotations supported on new combo charts for different dimension/measure combinations.
Annotations can be edited, moved, and resized; font size can be configured via the UI Settings panel.
A top-level dashboard toggle controls visibility of annotations on the dashboard.
New Combo chart improvements:
Dashboard and chart level in-place migration for legacy stacked bar, bar, histogram, and combo charts to the New Combo chart.
Chart zoom
Toggle for All Others on the legend when a group by dimension is present.
Toggle for the default null option on the group by dimension
Column formatting in map pop-ups, including the ability to toggle the pop-up on/off without removing any columns.
For exact match chart-level filters, a default All option that allows you to toggle off the filter from the dashboard view.
For table charts with dimensions, a Null Measures toggle allows null measure values to be displayed.
Performance improvements for map charts.
You can set margins between charts using the UI Settings panel. The default margin is 16 pixels.
You can set min/max manually on heat charts.
Opacity can be set for single layer map charts.
Matching for global string filters can be case-sensitive.
Filter panel defaults can be set along two dimensions: open/closed, and simple/advanced.
Added an instance-level flag (restricted_viewing) that prevents users from adding new charts or modifying existing charts if they are in a “view only” experience (VIEW but not EDIT privileges on the dashboard).
Databases in the database switching list are now sorted alphabetically.
Added the OmniSci-provided “Minimalist” base map as a default to all charts, regardless of online/offline status.
Added the ability to configure default and max number of points on Pointmap charts.
Added the user role “restricted_sharing” that prevents users with that role from being able to share dashboards.
Tabs are on by default; information about configuring the default experience is here.
Fixed Issues
Fixed issue where the filter panel was scrolling off page by making it always present when opened.
Fixed issue in geo export for multilayer charts where only the top layer was available for export.
Dates on map pop-ups are now displayed in UTC instead of local time.
Duplicated charts now are the same size as the original chart.
Static and topN values now migrate correctly from Histogram and legacy Combo charts to New Combo charts.
Fixed performance issues on the New Combo chart.
The range chart on New Combo chart now displays the full data range.
Fixed issue in New Combo chart, when configured as a line chart, where the default y-axis value was 0 instead of the data min.
The number of groups from legacy bar charts now migrate correctly to the New Combo chart.
New Combo charts now update correctly with streaming data.
Fixed issue in New Combo charts where the collapsed legend was not saving correctly.
Fixed issues with tabs causing crashes.
The backend error message for missing tables is now passed correctly to the corresponding Immerse error modal.
Fixed an issue in New Combo charts where long axis titles caused chart render issues.
Brush filters in New Combo charts can now be applied in stacked/percentage mode.
Fixed issue in new combo chart where the legend tooltip did not render in Chrome and Firefox browsers.
Binning options in New Combo charts now correctly accept non-integer values.
Fixed an issue in New Combo charts where modifying the range filter in one filter set applied the change to other filter sets.
Resolved an issue New Combo charts where the default axis title was not set for custom dimensions.
Legend options in New Combo charts now have sufficient contrast in Light Mode.
Fixed multiple issues related to dashboard resize that caused charts to move around unexpectedly on the dashboard.
Resolved an issue where saved dashboard views did not clear correctly between user sessions using the same browser.
Fixed an issue where switching databases in the Chrome browser caused Immerse to pull an incorrect dashboards list.
Fixed an issue where Jupyter access disappeared on dashboard refresh.
Tabs can now be renamed when using the Safari browser.
Fixed an issue in New Combo charts where a gap remained after a pinned legend was removed.
In New Combo charts, source and data selection are now retained when switching between other charts and the New Combo chart.
A display issue in Pointmaps that cut off the last digit for the number of points input box is resolved.
Fixed an issue in multi-layer charts where chart-specific filters were incorrectly applied across layers sharing the same source instead of applied for the layer that the filter was created on.
For access to OmniSci release 5.4.2, contact support@omnisci.com.
OmniSci release 5.4.2 includes some new features and improvements and fixes a number of issues.
Improved performance of large group-by queries over dictionary encoded strings when filters are present. The filters are applied to the group by reducing the size of the output.
Added memory protection for APPROX_COUNT_DISTINCT queries. The configuration parameter bitmap-memory-limit allows you to increase or decrease the maximum amount of memory allocated for APPROX_COUNT_DISTINCT bitmaps per execution kernel (thread or GPU). Previously, this allocation was not capped. The default cap is 8 GB, but can be increased for high-memory systems.
Added support for PowerBI as an OmniSciDB client.
Resolved an issue where a query with LIMIT 0 in distributed mode could return results.
Resolved an issue where singleton inserts or updates/deletes on sharded tables in distributed mode could result in truncation of the table.
Fixed an issue where join queries could return incorrect results with NVIDIA driver 455. OmniSci continues to recommend driver versions 418.39 to 450 for OmniSci release 5.4.x.
Fixed a crash on unsupported, multi-table self joins.
Resolved an issue where an entire table could be projected for SELECT * LIMIT n queries, where n is small. OmniSciDB attempts to project only the first n rows, unless deletes without vacuum have occurred.
Resolved several issues with multistep queries in distributed mode. Stability and performance of multistep queries, multistep queries with subqueries, and multistep subqueries have been improved.
Fixed an issue with sharded tables in distributed mode where INSERT INTO SELECT queries did not persist changes to disk.
Resolved an issue where the colors of a symbol render (via a Pointmap in Immerse) can be incorrect if it is immediately preceded by a Heatmap render.
Fixed issue where users with the CREATE DASHBOARD privilege could not save their dashboards.
OmniSci release 5.4.1 fixes a number of issues.
Resolved an issue where queries with extra whitespace returned an error. Whitespace is now trimmed automatically by OmniSciDB.
Resolved an issue where SHOW CREATE TABLE showed an incorrect SHARD_COUNT for sharded table DDL in distributed mode.
Fixed an issue where left joins could be converted to inner joins erroneously when joining multiple subqueries.
Resolved an issue where dashboard owner updates were not properly persisted after server shutdown.
Fixed an issue where global filters were not applying correctly to old Combo charts.
Fixed and issue where chart-level filters could not be saved.
Resolved an issue where layers were not displaying correctly for multilayer map export.
Resoved an issue in linked pan/zoom where turning off the feature flag did not disable the feature for previously linked charts.
OmniSci release 5.4.0 contains several new features and improvements and fixes a number of issues.
Improved performance of the CSV importer by up to 50%. String import, geospatial import via CSV, and importing wide varlen rows are all faster.
Performance improvements for update and delete queries.
Improved the performance of filtered group by queries on large cardinality string columns. This is particularly relevant for Immerse-generated cohort queries on dictionary-encoded string columns.
Added initial compilation support for NVIDIA Ampere GPUs.
Added support for Alteryx integration via the Alteryx Lua connector.
Added support for Oracle Access Manager as an identity provider for SAML authentication.
Added support for exporting geo data in GeoJSON format.
Added a new SQL function, SAMPLE_RATIO, which takes a proportion between 0 and 1 as an input argument and filters out rows to obtain a sampling of a dataset.
Upgraded OmniSciDB Arrow distribution to v1.0.0
Can now disable the origin lookup for inbound API requests. On busy systems with slow or throttled networks, DNS lookups can cause bottlenecks as multiple users attempt to access the database. Use the configuration flag log-user-origin.
Resolved an issue where nested joins within a view could crash.
Array fields in shapefiles are now imported correctly.
JDBC driver now returns a valid update count. Currently, OmniSciDB does not report the number of rows updated, so the update count is -1.
Resolve correctness issues in date/time extract functions.
Resolved an issue where a Boolean column in an IN expression could crash the server.
Fixed an issue where listing the roles for the default OmniSciDB user fails. This is resolved only for default users named admin. If your default user is still mapd, you can rename the user to admin by using ALTER USER RENAME TO.
Resolved an issue where a sort with a limit over sharded or empty data could crash.
Resolved an issue where some left joins were incorrectly rewritten to inner joins.
Resolved an issue where a rendering hit test against a sharded table could crash.
Fixed an issue with dump/restore of nonencoded geospatial columns. Previously, archives of nonencoded geospatial columns did not restore correctly. Note that the archives must be re-created.
OmniSQL \d now shows geo column encoding.
Perform bulk actions on the dashboards list, including export, share, and delete.
Apply filters to the dashboards list based on owner, source, last modified date, shared status, and dashboard name. A new count widget displays the number of filtered dashboards.
Dashboard list filters can be saved as views and then duplicated, renamed, and deleted. Users default to the last filter view they used.
The UI Settings panel offers a variety of new chart customizations:
Change font size, boldness, and contrast across chart titles and labels
Customize axis label truncation
Modify and add new color palettes
Save UI settings as the database-wide default
Dashboards now have tabs, which can be duplicated and reordered. A Play button makes it easy to cycle through tabs.
Choropleth, Linemap, and Pointmap charts now support chart-level data export in GeoJSON and CSV formats.
The min/max axis values for Scatterplot charts can be configured manually.
The SQL Editor now accepts Vega JSON requests and displays image results in the editor window.
A lat/long viewer is displayed when hovering over map charts.
Added the # of Groups functionality to the New Combo chart, providing a consistent experience with other bar charts.
Added the option to allow zooming on map and combo charts only when the SHIFT key is held down, to prevent accidental zoom actions.
Many of these features can be implemented and customized using feature flags.
Fixed an issue in old Combo charts where charts did not load correctly with updated data.
Resolved an issue where invalid filters caused dashboards to load incorrectly.
Fixed an issue in New Combo charts where line marks were displayed as area marks when bar marks were present.
Crowded display of date dimensions in the New Combo chart is fixed.
Improved padding on New Combo chart, allowing the range chart to be displayed in a more compact way.
Fixed an issue in New Combo charts where global filters were not correctly copied over during chart duplication.
Fixed an issue in New Combo charts where injecting the column name in the custom SQL window repeatedly inserted the column name.
OmniSci release 5.3.2 contains new features and improvements and fixes a number of issues.
New Features and Improvements
Added LEFT JOIN support for accelerated geojoins using the overlaps join.
Improved performance of projection queries with OFFSET.
Improved performance of OmniSciDB function calls (e.g. geometric functions) by eliding null checks and eliminating runtime casts.
Added support for implicit casting during insert into the table as select.
Improved error messages and logging. In particular for casting errors, union all type mismatch errors, and create table as select / insert into as select logging all.
During Create table as select, dictionaries between the source and target tables will automatically be shared. This improves the performance of both the CTAS query and of any queries using both tables. Append WITH (USE_SHARED_DICTIONARIES=FALSE) at th end of the CTAS query to disable this option.
Reduced connection timeout errors by introducing keepalive on API calls that use a client with a timeout (intended to replace manual management of individual service timeouts). keepalive is controllable using the calcite-service-keepalive and leaf-with-keepalive command line options.
Resolved an issue where a case statement in a subquery could crash the server with the error Check failed: generation_ >= 0 (-1 >= 0).
Fixed an issue where an update on a temporary table could result in no rows being updated after a delete had occurred.
Overhauled date_trunc and extract runtime, improving performance and resolving correctness issues with dates before January 1, 1970.
Resolved correctness issues with TIMESTAMPDIFF and DATEADD.
Resolved an issue where queries could crash the server in a CUDA/Render enabled build running in cpu-only mode. CPU-only builds do not suffer the same problem.
Resolved an issue where SELECT * LIMIT N (for any N) on a table with deleted rows would pull the entire table into memory.
Fixed an issue where a validate query preceding a SQL query could cause the error Ran out of slots in the query output buffer during SQL query execution.
Resolved a crash on group by queries with large output buffers.
Fixed Issues
Fixed issue where New Combo chart could not be turned off via feature flag.
Fixed issue where certain string values were handled incorrectly in Pointmap coloring.
OmniSci release 5.3.1 contains new features and improvements and fixes a number of issues.
New Features and Improvements
GPU memory usage is improved across OmniSciDB and OmniSci Render. Previously, large amounts of reserved GPU memory (res-gpu-mem) were required to prevent out-of-memory errors when rendering large line or polygon datasets. With more buffer sharing between the database and rendering engine, you can reduce the res-gpu-mem value without increasing out-of-memory errors.
Added a query hint framework, allowing you to control query execution parameters in the query string. You can set CPU execution mode by adding /*+ cpu_mode */ after the SELECT keyword in a SELECT query.
Performance is significantly improved when sorting on COUNT DISTINCT and APPROX_COUNT_DISTINCT aggregates.
Performance is improved in query parse, particularly for small-cardinality group-by queries. Additional performance improvements for multicolumn and large-cardinality group-by queries using baseline hash.
GPU and CPU buffer pool allocation (slab) size are configurable as a runtime parameter, and the default for GPU buffer pool is increased from 2GB to 4GB for more efficient GPU memory usage.
GPU JIT warmup is added to OmniSciDB at server start. As a result, OmniSciDB verifies the correct CUDA version and generates initial code for the GPU at startup instead of when the first query is executed. First query times on GPU after startup have been reduced. The server does not accept connections during JIT warmup, but a message is printed in the logs before and after warmup completes.
Access to import/export paths to the server using the COPY command can be restricted. Administrators can restrict access to root paths used in COPY FROM and COPY TO commands. Currently, all paths are allowed by default.
Added the SQL placeholder CURRENT_USER for accessing the username of the current user in a SQL query.
Added support for the ALL PRIVILEGES alias when granting object privileges.
Resolved an issue where queries with STDDEV inside a CASE statement could fail with the error message, “Aggregate function STDDEV not supported.”
Fixed an issue where projecting NULL could crash the server.
Resolved an issue where TIMESTAMPDIFF could return incorrect results for dates earlier than January 1st, 1970.
Resolved correctness issues with the EXTRACT function on date, time, and timestamp values. Extract week could return incorrect results, and dates before January 1st, 1970 could also return incorrect results.
Resolved an issue where getString() on an array returned via JDBC would return an out-of-bounds exception.
Improved handling of cast and filter operations on high-precision timestamps. Reduced incidence of overflow/underflow errors when casting or filtering between timestamps of different precision.
Resolved an issue where OmniSci Server could exit without an error code when encountering some errors (for example, too many open files during execution of an import query).
Resolved an issue where user input was not effectively sanitized.
New extension functions reg_hex_horiz_pixel_bin_packed, reg_hex_vert_pixel_bin_packed, and rect_pixel_bin_packed replace reg_hex_horiz_pixel_bin_x/y, reg_hex_vert_pixel_bin_x/y, and rect_pixel_bin_x/y, respectively. The packed versions return an image coordinate packed into a 32-bit INT, eliminating the need for separate x/y functions and reducing redundancy. Query times are improved up to 2x.
Fixed an intermittent CHECK on a multi-GPU server when rendering a query where the results are distributed to a sparse subset of available GPUs.
Addressed incorrect renders on multi-GPU servers when the offset-device-by-table-id config flag is enabled. This flag is now enabled by default.
Fixes an exception safety issue in a multi-GPU server when an out-of-memory exception is thrown in the compositor before resource initialization is complete. This rare scenario would have resulted in a segfault.
Fixes a CHECK when rendering polygon/multipolygon columns that were imported without an SRID and no encoding defined, usually via CSV when the SRID is not implied.
Added the ability to build cumulative and percentage distributions in the New Combo chart for non-binned and categorical dimensions.
Fixed issue where binning min/max values were not preserved on table and pie charts.
Added a feature flag for the SELECT * query snippet in the SQL Editor.
Fixed bugs in New Combo charts related to top-N, binning, and NULL handling.
Fixed issue in Table charts where certain measures produced an invalid date filter in Firefox browser.
OmniSci release 5.3.0 contains several new features and improvements and fixes a number of issues.
The render-poly-cache-bytes flag has been removed and must be deleted from configuration files or other start scripts; otherwise, the omnisci_server binary fails to start.
Clearing the NVIDIA code cache after upgrading OmniSciDB is a best practice. To clear the nvidia code cache, remove the .nv directory in the home directory of the user running OmniSciDB. If you are running in a Docker container, the code cache is automatically cleared when a new container is launched in the default configuration.
Previously, date_trunc(decade,...) truncated to the start of the “1”s year of the decade. For example, DATE_TRUNC('decade', CAST('1969-12-31 23:59:59' AS TIMESTAMP)) would return `1961-01-01 00:00:00`). Now, DATE_TRUNC(decade,...) returns the start of the “0”s year; for example, `1960-01-01 00:00:00`. This change was made for consistency with other SQL systems. To start decades on the “1”s, use the DATEADD function.
New Features and Improvements
Array support for the Array type over JDBC.
Added support for SELECT DISTINCT in UNION ALL. UNION ALL is currently prerelease and must be explicitly enabled in the server configuration file.
Added a query type return to the sql_execute endpoint, allowing API users to determine if a query altered date or a table or was a read-only query.
Added support for joins on DECIMAL types.
Added support for retrieving metadata for a given statement after running SQLPrepare in ODBC. For example, after calling SQLPrepare, the SQLColAttribute will return metadata for columns involved in the query.
Performance improvements on CUDA GPUs, particularly Volta and Turing. OmniSci now generates PTX code targeted at specific GPU architectures, resulting in increased performance for aggregate workloads. Performance of summation on double precision types has increased by 10x in some instances.
Increased use of CUDA shared memory, resulting in additional performance increases on aggregate queries in the presence of a group by.
sql_validate now returns column descriptors for each projected column in the order in which columns appear in the select clause. Previously, column descriptors could be in any order.
Fixed Issues
Resolved an issue where RESTORE TABLE could be exploited to gain shell access on the running OmniSci server.
Resolved an issue where incorrect GPU memory requirement estimates for JOIN queries could result in a query that would run on GPU being transferred unnecessarily to CPU.
Reduced memory fragmentation during select queries, particularly with approx_count_distinct aggregates. OmniSci recommends dropping caches or restarting the system after upgrading to OmniSci 5.3.0 to make more large pages available for the server.
Resolved an issue where a comparison of geometry types could crash. Most comparisons of geometry types are not yet supported.
Resolved various crashes when sorting, projecting, or otherwise interacting with geospatial columns. These crashes resulted from an unsupported operation being performed on geospatial data, and now result in error messages.
Fixed a crash on Boolean comparisons where the Boolean column was declared NOT NULL.
Resolved an issue where DISTINCT * in a subquery was not respected, resulting in an extra rowid being inserted into the subquery result and skewing the overall distribution in the query. Now, rowid is not present in a SELECT * statement unless explicitly projected.
Resolved an issue where a syntax error in a SQL statement could cause the aggregator node to crash in a distributed setup.
Resolved an issue where a multicolumn group-by window function could return incorrect results.
Resolved an issue where the date_trunc function could return incorrect results for dates before the Unix epoch, January 1st 1970.
Resolved an issue where dictionary encoded string columns using a shared dictionary would incorrectly have an encoding displayed in SHOW CREATE TABLE;` or `\d`. Now, DDL from a table with columns using a shared dictionary can be copied and pasted directly to create a new, identical table.
Resolved an issue where multiple filter expressions using OR could cause a crash if the same column was used on the left side of the expression with different casts.
Added logging for OmniSciDB server shutdowns due to exceeding the maximum number of available open files on the system.
Resolved an issue where `INSERT INTO` commands sent from the ODBC driver would fail.
Fixed an issue where a series of nested queries grouping on float columns could give incorrect results.
Fixed an issue where importing geojson files with embedded dates could truncate the date epoch to 0 or NULL.
Resolved an issue where ST_Contains queries using the overlaps join framework could crash or return incorrect results.
Resolved an issue where importing from a parquet file with multiple row groups could return incorrect results.
Resolved an issue in the ODBC driver where incorrect data could be returned for double precision types.
Resolved an issue where geospatial joins could return incorrect results.
Resolved inconsistencies in argument support for date/time functions. Now, all date/time functions should support literal arguments with or without quotes (for example, `date_part(day, …` and `date_part(‘day’, …)`
Resolved a small memory leak in the OmniSciDB SQL parser.
Fixed Issues
Resolved an issue where the server could crash when a user disconnected if the users render session had timed out.
Resolved a crash where switching between different point renders on the same data could cause a crash if hit testing is disabled.
New Features and Improvements
New Combo chart with increased configurability and power:
New chart configurations like row/column, stacked/percentage bar, stacked/percentage line and area, line/bar combination.
For numerical base dimensions, measures can be configured to display as cumulative and percentage distributions.
A new “Top N” experience for charts with a group-by dimension, allowing you to select dimension values dynamically or manually (including a new “Add all” option), as well as the ability to configure sorting logic for Top N.
Chart legends can be pinned to the right or superimposed on the chart.
The visibility of individual group by series can be toggled on or off from the chart legend
Chart orientation can be configured between vertical and horizontal.
The min and max of chart axes can be independently locked. Existing bar and combo charts can “duplicate as New Combo chart” at the dashboard level.
Chart-specific filters across all chart types, which can be toggled off when not in use.
For chart-specific filters that require an exact match on the filter value, you can use “quick filters” to access those filters directly on the chart.
A simplified filter panel, with two modes: the existing “Advanced” mode, and a new “Simple mode” focused on simplifying the consumption and interactivity of filters.
An updated filter sets experience targeted at simplifying options and offering greater visibility across filter sets.
Added sorting by column value for dashboards on the dashboards list.
On multi-layer charts, visibility of individual map layers can be set by zoom level.
Different map charts can be synced together for pan and zoom actions, regardless of data source.
The New Combo chart, linked pan and zoom, and default filter panel can be configured using feature flags.
Resolved issue with global filters adding escapes to string values with underscores or percent marks.
Resolved issue in histogram and combo charts where binning min and max values changed after saving and reopening the chart.
Resolved issue in the SQL Editor where newly created tables did not appear on the tables list.
Resolved issue in pointmap where Boolean values could not be selected for the color parameter.
Resolved issue where invalid config did not cause the web server to fail in a visible way.
Resolved issue where switching database while viewing a dashboard did not allow the user to open a dashboard in the new database.
Resolved issue with database switching where user permissions were set incorrectly.
Resolve dissue in the filter panel where time-based filters were displaying local time vs. UTC.
Resolved issue where cohort creation errored out on dashboards containing geo-joined multilayer charts.
Fixed minor UX and display issues, specific to the dark and light mode themes.
OmniSci release 5.2.2 includes a new feature and fixes two issues.
New Feature
Added support for geospatial columns in temporary tables.
Fixed Issues
Resolved an issue in which a render request and a table modify operation (for example, COPY FROM, DROP TABLE, TRUNCATE TABLE), dispatched simultaneously, could cause a lockup.
Fixed an incorrect result for queries that sort based on the result of an aggregate in which the aggregate type is not NULL. This typically occurred with an explicit non-NULL filter in the WHERE clause on the input to the aggregate.
OmniSci release 5.2.1 fixes an issue.
Fixed Issue
Resolves an issue where OmniSciDB fails to respond correctly to operating system signals in some contexts.
OmniSci release 5.2.0 contains several new features and improvements and fixes a number of issues.
New Features and Improvements
CUDA requirement is updated to version 10.1. OmniSciDB now requires NVIDIA driver v418.39 or later.
The UDF compiler now requires Clang++ v9.0.
Updated embedded Apache Arrow to support v0.16.
Added INT8 support to Arrow result converter.
Multi-fragment hash join iterator added to improve performance on multi-fragment inner joins and geospatial joins.
Added NULL support for geospatial types, including in ALTER TABLE ADD COLUMN.
Improved performance for query interrupt (currently disabled by default), allowing you to interrupt a query in omnisql (using Ctrl+C) and JDBC (using the Cancel endpoint).
Added SQL SHOW commands:
SHOW TABLES -- Show all tables the user currently has access to in the logged-in database.
SHOW DATABASES -- Show all databases the user can log in to.
SHOW CREATE TABLE -- Show the DDL CREATE TABLE command for a given table.
SHOW USER SESSIONS -- Show all persisted sessions on the server (superuser only). Expired sessions are still shown; they are removed from session state on login attempt.
Added support for the ALTER TABLE DROP COLUMN command.
Updates and deletes are now supported on temporary tables.
Added additional UDF command-line options for systems using nondefault Clang++ installation locations.
Added support for quoted table names in DDL and DML commands. Use quotes if the table name contains special characters or matches a reserved word.
Added support for window functions in distributed mode.
JDBC driver updates:
Supports the cancel method, requesting that the database cancel the current request. The enable-runtime-query-interrupt flag must be set to TRUE in OmniSciDB configuration.
Handles JDBC function escape syntax for the fn keyword.
Supports the getMetaData endpoint.
Implements setSchema as a null operation (OmniSciDB currently does not support schemas) instead of throwing a "method not implemented" error.
Added support for EXTRACT(DATEEPOCH from m), where m is a TIMESTAMP column.
omnisql now returns a nonzero exit code if the last executed query failed to complete (threw an exception).
Improved logging of connection failures when using LDAP, SAML, or other external authentication providers.
Added support for COPY TO in distributed mode.
Added support for importing compressed parquet files (gzip).
Various performance improvements, particularly for join queries, projection queries with order by and/or limit, queries with scalar subqueries, and multicolumn group-by queries.
Fixed Issues
Resolved a series of locking sequence errors, which could cause deadlocks when multiple users were interacting with the database and rendering engine at the same time.
Improved the ability of omnisci_server to detect a failure in the Calcite service, resulting in automatic restart of omnisci_server when managed by systemd.
Resolved an issue where projection queries running in columnar mode could crash the server.
Resolved an issue where queries could fail or crash the server with “error code 8”. Those queries are now gracefully retried or aborted, and a correct error message is now displayed (for example, "not enough GPU memory to execute the query".)
The geospatial importer is now more tolerant of unsupported types. Instead of causing the import to abort, unsupported types in a shapefile are skipped and a message is logged.
OmniSciDB now returns an error when running the OPTIMIZE command on a view. OPTIMIZE is not supported on views.
Resolved a crash on binary operations between TINYINT and DECIMAL/NUMERIC types.
Resolved a crash when attempting to import to a table with TINYINT columns using StreamImporter or StreamInserter.
Resolved an issue where a LogicalValues query (query over VALUES with no table in the FROM clause) with a sort could crash the server.
Resolved an issue where SQLImporter failed to import geo datatypes.
Disabled ALTER TABLE ADD COLUMN on tables with sort_column. This previously caused the server to crash.
Improved handling of special characters in SQL queries, including quotes.
Resolved issues where join query results were not updated correctly with table data changes after an UPDATE, DELETE, TRUNCATE, or ALTER command.
Resolved an issue where SQL CASE statements with Boolean arguments failed to implicitly cast correctly, resulting in a cast failed error message.
Resolved an issue where multistep queries with a sort could crash the server. These queries are typically dispatched for choropleth charts in Immerse.
Fixed an issue where a join on a literal or values table could crash the server.
Resolved an issue where users created prior to the can_login property being added could not log in.
Resolved an issue where some API calls would not properly reset a users idle session duration, resulting in premature idle session timeouts.
Fixed an issue where a string dictionary overflow during CREATE TABLE AS SELECT or INSERT INTO AS SELECT could crash the server. Now, an exception is thrown and the query aborts.
New Features and Improvements
Polygon rendering in cached mode has been deprecated. For migration steps for custom clients, see Migrating Vega Code to Dynamic Poly Rendering.
Increased default render-mem-bytes config setting from 500MB to 1GB.
Added a new config option, max-concurrent-render-sessions, to set the maximum number of concurrent render session caches. Default value is 500.
Fixed Issue
Resolved a hit-test issue when trying to retrieve data from a column that uses LIKE/ILIKE in an expression.
New Features and Improvements
SQL Editor is updated:
SQL query formatting is now available, which inserts whitespace for additional readability.
You can run a selected portion of the SQL query. Selecting text transforms the “Run query” button to “Run selected”.
The table browser supports filtering on column type.
From the table browser, clicking on a table or column name shows options for inserting “query snippets,” or a shortcut to common queries that provide additional information about the table and its columns.
The query history tab lists previously run queries from the active session, and clicking a query displays previously fetched results. Click the insert icon to insert the query back into the input window to re-run it.
You can press tab for autocomplete on queries and resize the results window.
Using multiple databases in Immerse has been enhanced:
Dashboard URLs are now unique across databases.
You can access additional databases from the account dropdown in Immerse, either in the current browser tab or as a new tab.
You can access multiple databases at the same time, as long as the user sessions are active.
Load time for the dashboards list is reduced significantly.
Count of records can now be used as filtering criteria during cohort creation.
Error handling is improved.
Custom logos can now be configured separately for light and dark themes.
Logos can be configured to deep-link to a specific URL.
Fixed Issues
Fixed formatting issue where values above 100M were rounded to two significant digits; additional precision is added to the default number formats.
Fixed issue on the histogram chart where the last bin was missing. When you log in one browser tab, you are logged out across all other browser tabs.
Added tooltips in column section to make reading long column names easier.
Fixed issue with custom measures created with CASE statements not rendering correctly in choropleths.
Fixed issue where shapefile imports failed on files with uppercase extensions.
Fixed issue where appending data errored out when import settings were changed.
In heatmaps, dates are now sorted correctly--by time--instead of alphabetically.
Upgrading from HTTP to HTTPS now works correctly.
OmniSci release 5.1.2 contains new features and several fixed issues.
New Features and Improvements
Added support for SQL VALUES syntax, allowing literals to be used inline in SQL query expressions.
Added calcite-service-timeout to the OmniSciDB server configuration parameters, allowing the timeout between OmniSciDB and the Calcite process to be increased for high utilization systems. See calcite-service-timeout on the Configuration Parameters page.
Fixed Issues
Resolved an issue where multiple, concurrent shapefile imports from Immerse could fail.
Resolved an issue where concurrent imports and render queries could cause a crash.
Fixed ALTER TABLE ADD COLUMN in the presence of sharded tables. Previously, the new column was not added to all shards.
Removed a dependency on nvcc being available in the user’s PATH when compiling C/C++ UDFs. CUDA is still required, but nvcc does not need to be accessible to OmniSciDB.
Removed GPL-licensed com.unboundid.unboundid-ldapsdk package that had been included in mapd.jar.
Fixed Issue
Added support for transient string dictionary columns in ordinal scales.
New Features and Improvements
Added the ability for polygon borders to be the same color as the polygon fill color, rather than always defaulting to white.
Added the ability to display a list of databases to choose from at login, configured via servers.json. See login_panel_databases.
OmniSci release 5.1.1 contains one significant fixed issue.
Fixed Issue
Resolves an issue with OmniSciDB 5.1.0 on several cloud service provider marketplaces.
OmniSci release 5.1.0 contains many new features and fixes.
New Features and Improvements
Added an option to the web server which allows users to disable TLS certificate verification on startup. While this is not recommended for production deployments, services running in unencrypted mode in a container require this option to properly start using the startomnisci script. The web server backendUrl parameter now also respects https, if configured in the OmniSciDB config.
Enhanced OmniSciDB log files to include inbound client IP address and protocol for all endpoints and additional logging for import queries when CUDA contexts fail to be created.
Improved error handling for View queries. Errors now display the view DDL and instructions for resolving the underlying issue that prevents the view from being executed.
Improved support for EXISTS and NOT EXISTS subqueries.
Added support for LINESTRING, POLYGON, and MULTIPOLYGON in user-defined functions.
Improved logging of SAML and LDAP roles and associated actions when authenticating with an identity provider. Logs are written to the INFO log when a user establishes a new session.
Added support for the can_login user attribute. When set to false, the user is unable to log in to any database.
Improved performance for multi-column GROUP BY queries, as well as single column group by queries with high cardinality. Performance improvement varies depending on data volume and available hardware, but most use cases can expect a 1.5 to 2x performance increase over OmniSciDB 5.0.
CREATE TABLE AS SELECT queries now support an optional IF NOT EXISTS clause. For example:
Improved the resiliency of query parse errors, including allowing for more special characters inside quoted strings.
Added support for UPDATE via join with a subquery in the WHERE clause.
Added support for DELETE via join queries.
Added support for CREATE TEMPORARY TABLE.
Added support for JDBC authentication using a PKI certificate.
Added support for signed SSL certificates with a non-system certificate authority (-ssl-trust-ca).
Fixed Issues
Resolved an issue where embedded tabs or new lines could cause a crash during query parse.
Resolved an issue where MIN/MAX/SUM over a NOT NULL column could return the NULL sentinel representation instead of NULL when no rows passed the given filter.
Resolved an issue where SSL keystore passwords were visible in the system process list when OmniSciDB was running.
Resolved an issue where adding a NULL to a date or timestamp value incorrectly resulted in an addition. Adding a NULL to a date or timestamp now generates an exception.
Suppressed extraneous log messages when the query completion hints system was activated (typically either by hitting the tab key in `omnisql` or by using the SQL Editor in Immerse).
Fixed an issue where the GRANT and REVOKE command for dashboard privileges failed when running in distributed mode.
Resolved various issues when updating variable length datatypes (arrays) containing NULL values.
Resolved an issue where overflow / underflow errors were not detected during import or cast operations on TIMESTAMP types.
Improved the stability of JOIN queries in distributed mode.
Resolved an issue where INSERT INTO TABLE AS SELECT using geospatial columns could incorrectly throw a "column encoding sizes do not match" error.
Resolved an issue where connections made to the OmniSciDB HTTP(S) endpoint would be refused when running warmup queries.
Resolved an issue where the log_severity configuration file option failed to read quoted values.
Resolved an issue where taking the absolute value of a TINYINT column type using the ABS() function could crash the server.
Improved type checking when comparing geospatial types. Now, all geospatial types are required to have the same SRID, if specified.
Fixed Issues
Resolved an issue where hit-testing results might be incorrect when rendering cached polygons.
Resolved an issue where hit-testing an accumulation render in distributed mode would return an empty result when using multi-gpu-configured leaves.
Resolved an issue where users attempting to render the result of a view were required to have access to the underlying table. Now, only access to the view is required for render query execution.
Resolved an issue where accumulation renders could crash in distributed mode when the results are relatively unbalanced across the leaves.
Resolved an issue where an incorrect buffer layout error could be thrown incorrectly when using Vega transforms in successive renders.
New Features and Improvements
Log-ins are fully sessionized and persist across page refreshes.
Pie chart now supports “All Others” and percentage labels.
Cohorts can now be built with aggregation-based filters.
New filter sets can be created through duplicating existing filter sets.
Dashboard URLs now link to individual filter sets.
The top level "toggle off" filters option now supports toggling those filters back on.
A 5-second refresh rate has been added as the lowest refresh setting for dashboards.
Added ability to increase the size of bubbles on the Bubble chart, and to be able to manually configure the MIN and MAX size of bubbles when there is a size measure.
Improved display and messaging of error modals.
Added a label for the current database in the account settings dropdown.
Fixed Issues
Fixed error handling resulting in popup loop due to incorrect Custom Measure syntax, on choropleth rendering, and during specific cases of chart type switching.
Fixed error handling where there was no error information presented when a dashboard name was duplicated.
Fixed an issue where after adding a color dimension on one source, the user is unable to change other sources to the bar style.
Fixed an issue where changes to global filters were not picked up by the Save button.
Fixed an issue where there was a shape filter delay on the zoom in/out button action.
Fixed an issue where Bar chart inverse filters did not correctly account for NULLs.
Fixed an issue where Bar and Table charts were showing errors when switching to charts with a color dimension.
Fixed an issue in Bubble chart where switching to an extract unit caused Immerse to crash.
Fixed issues related to dark and light theme font color inconsistencies.
Fixed an issue where chart filters on extracted timestamps appeared incorrectly in the filter panel when time series charts were brushed.
Fixed a formatting issue causing x-axis labels to only show two significant digits.
Fixed an issue where certain filters from charts were not showing up in the filter panel.
Fixed an issue where filtering on a measures only Table chart caused Immerse to be in an error state.
OmniSci release 5.0.1 contains two critical fixes.
Fixed an issue where CREATE TABLE AS SELECT or INSERT INTO TABLE AS SELECT could return incorrect results in distributed mode.
Fixed an issue where projection queries over sharded tables could return incorrect results in GPU mode.
OmniSci release 5.0 contains many new features and fixes.
New Features and Improvements
OmniSciDB now supports certain forms of correlated subqueries.
OmniSciDB now supports update via subquery, to allow for updating a table based on calculations performed on another table.
Added support for binary dump/restore of database tables.
Added support for compile-time registered user-defined functions in C++, and experimental support for runtime user-defined SQL functions and table functions in Python via the Remote Backend Compiler (see omnisci_examples folder within the OmniSci JupyterLab instance for example UDF/UDTFs).
Multistep queries that generate large, intermediate result sets now execute up to 2.5 times faster by leveraging new JIT code generator for reductions and optimized columnarization of intermediate query results.
Added support for LIMIT 0.
Added SAML support for auth0 and ping, and Oracle Access Manager.
Added support for window functions inside complex expressions.
Standardized encrypted transport options across ODBC, JDBC, SQLImporter, and omnisql.
Added an option to log LLVM IR or NVIDIA PTX to a log file per query-step using --log-channels.
Added support for geospatial POINT construction from literal or computed values.
Added support for SQL operators IS TRUE and IS NOT TRUE.
Added support for SQL operators EXISTS and NOT EXISTS.
Added support for geospatial types to StreamImporter.
Added support for updates through joins.
Docker editions now support Ubuntu 18.04.
Generalized performance and optimization improvements.
Fixed Issues
Aggregating empty results in distributed mode could cause a crash.
The Apache Arrow interface (sql_execute_df) properly handles empty results.
Fixed an issue where join queries could give no results in distributed mode, even if the tables were properly sharded or replicated.
Fixed an issue where time zones could be parsed incorrectly on import of date/time/timestamp types.
Improved clarity of the get_db_objects_for_grantee() (Thrift) / \object_privileges (omnisql) APIs and resolved an issue where all privileges were not always included.
Fixed an issue where Calcite could crash on malformed queries.
Resolved issues where aggregate functions could overflow, returning an incorrect result.
Resolved an issue where the S3 importer could crash when attempting to import from a SSL encrypted OmniSciDB server.
Resolved an issue where small deviations in image appearance could occur during anti aliasing.
Resolved an issue where adding a column to a sharded table could corrupt the table.
Resolved an issue where commas in log messages could cause the logs to be improperly parsed by OmniSciDB or third-party utilities.
Resolved an issue where the ARRAY constructor could crash in the presence of a sort or limit.
Fixed an issue importing MULTIPOLYGON columns from PostGIS via SQL Importer.
Resolved an issue where NULL was not handled correctly in AND/OR operations.
Resolved an issue where creating a new table via CTAS would fail with a parse error if the table name contained the word with.
Resolved an issue where CTAS would not properly apply the LIMIT or OFFSET if present in the query.
Resolved an issue where DECIMAL columns could overflow when used in a CTAS query.
Fixed a crash that could occur when a user reauthenticates after session timeout.
Fixed a crash when filtering geospatial columns by IS NULL/IS NOT NULL. (Geospatial nullability is not yet supported.)
Resolved an issue where compressed POINTs could be mishandled during result set iteration.
Addressed known crashes in aggregators with variable-length arguments, including various geometry types and nonencoded strings.
Fixed an issue during OmniSci ODBC Driver DSN Setup, where the “Ignore Invalid Certificates (not recommended)” field is unavailable in the dialog box.
Handle more gracefully a Calcite exception error that caused the server to abort.
Fixed a crash occurring when a prior query result was passed through a SQL function using a cast.
Resolved a crash that could occur when running aggregate functions on geospatial types in distributed mode.
Fixed an issue where binary operations on TINYINT types could crash.
LDAP and SAML role synchronization is now case insensitive, matching OmniSciDB internal representation.
Fixed an issue where combining columnar support with a shared table under ORDER BY or LIMIT could return an incorrect result.
Fixed an issue where a long query would time out a session, invalidating the operation or series of operations that required a longer execution time.
Resolved an issue where a table could be created with a specified shard_key but without a shard_count.
Fixed an issue where the AVG aggregate function is not properly cast when passing floating-point data.
Fixed an issue where SELECT queries with no filter and a LIMIT could return incorrect results on a table where one or more rows have been deleted.
Fixed an issue where a JOIN may not return correct results on a distributed setup.
Resolved issue where embedded new line characters could cause CTAS or ITAS queries to be rejected.
New Features and Improvements
SPIRV is now the default shader IR, and the --enable-spirv flag has been removed. Fallback to disable SPIRV is supported automatically.
The angle (rotation in degrees & radians) property is now available in Vega symbol marks.
The wedge property is now available in Vega symbol shapes.
Fixed Issues
Fixed a choropleth rendering issue when a FLOAT column is used to drive the color.
Fixed a data underflow issue for integer inputs for quantitative/qualitative scales.
Addressed possible validation issue when a floating-point column is used as an input for scales with explicit integral values defined for the domain.
New Features and Improvements
New filter panel that delivers major improvements, including:
Filter Sets providing access to different sets of filters in one dashboard.
The ability to toggle filters on and off, in addition to removing them.
The ability to visualize and modify cross-filters directly from the filter panel.
The ability to visualize from which charts cross-filters are generated.
The ability to filter columns by type, making it easier to find the correct column for a filter.
A custom filter builder that gives visual access to table columns for greater context and easier generation of filters
The ability to filter a text column on whether it starts with or end withs a certain pattern.
The ability to multi-select patterns on which to filter a text column.
Cohort building and analysis capabilities, including:
The ability to use global and cross-filters to interactively build cohorts of interest.
The ability to apply a cohort as a dashboard filter, either within the existing filter set or in a new filter set.
Data Catalog, a repository of datasets that users can use to enhance existing analyses.
Frontend-rendered choropleths now support the selection of base map layers.
Duplicate queries in charts were significantly reduced for performance improvements.
Map scales have been added to map charts.
Dark mode is the default Immerse theme.
Fixed Issues
Fixed issue in multilayer map charts where prior filters were being kept on the dashboard, even after the layer was removed.
Fixed an issue where color legend min/max inputs were not retained.
Fixed an issue where table name changes were not saved in the Update Settings flow.
Fixed experience in combo chart where a chart with all negative values had its axis inverted to positive; now the values stay in the original orientation.
Fixed an issue in map chart where resizing the chart would create an asymmetric resize animation.
Fixed an issue in dashboards search where results do not display correctly if only one character is used for the search.
Fixed an issue where geo join columns can be used in custom color measures across both the fact and geo join tables.
Fixed an issue in scatter chart where the circle filter goes out of container after clicking the cancel button.
Fixed an issue where global filters errored with date inputs that returned no results.
Fixed an issue for combo chart where secondary measure formatting was incorrect.
Fixed an issue where linemap size measure had incorrect range values.
Fixed an issue where “OmniSci” appears in browser tab before being overwritten via whitelabeling.
Fixed an issue where line chart series were not fully migrated to combo chart.
Fixed an issue where Oracle Access Manager deep links were working incorrectly.
Fixed an issue in combo chart where bars switched to the secondary axis overlap the bars in the primary axis.
Fixed an issue where the streets Immerse theme appeared layered on top of points.
Fixed a lag in shape filters.
Fixed an issue where invalid custom SQL filters would create a dashboard-level error state.
Fixed an issue where number charts were not updating with the rest of the dashboard.
Fixed how web links in text charts were incorrectly deep-linking outside the product.
Fixed an issue where custom filters containing OR statements were incorrectly applied to queries sent to OmniSciDB from Immerse.
Fixed an issue where pointmap values would disappear after the map theme was changed.
Fixed an issue where changing chart type from line to combo would remove locked x-axis values.
Immerse chart type selections scroll in a width-constrained browser.
Import/export dashboard metadata icons have been updated for improved clarity.