Release Notes
Looking for documentation from a previous release?
5.x
4.x
3.x
As with any software upgrade, it is important that you back up your data before you upgrade OmniSci. Each release introduces efficiencies that are not necessarily compatible with earlier releases of the platform. OmniSci is never expected to be backward compatible.
For assistance during the upgrade process, contact OmniSci support at [email protected] before you upgrade your system.
OmniSci release 5.6.4 includes a fixed issue.
- Resolved an issue where cleaning up an expired session could cause the server to crash.
OmniSci release 5.6.3 includes a new feature and an improvement.
- ITAS and CTAS queries are now protected from entering an infinite loop when small numbers of records are returned.
- Support for encrypted credentials in connectors is decoupled from authentication through Immerse.
OmniSci release 5.6.3 includes a new feature and an improvement.
- ITAS and CTAS queries are now protected from entering an infinite loop when small numbers of records are returned.
- Support for encrypted credentials in connectors is decoupled from authentication through Immerse.
OmniSci release 5.6.2 includes new features and improvements and fixes several issues.
- Added a new 'airplane' symbol shape.
- When using INSERT to insert floats into INT columns, the numbers are rounded instead of truncated; now consistent with COPY behavior.
- Fixed an issue where UDFs created at server start time with the
-udfflag were not generating code for the GPU. - Resolved an issue where certain queries with a smaller number of groups ( < 20,000) would fail with a “Ran out of slots in the query output buffer” message.
- Reduced incidence of “Ran out of slots in the query output buffer” errors with large group-bys.
- Improved READ-ONLY configuration to never write to underlying storage.
- Resolved an issue where a left join could get incorrect results if IS NOT NULL was part of the WHERE clause for the query.
- Resolved sort order with aggregated results (negatives before positive) when NULLS LAST is specified.
- Fixed an issue causing a crash when ITAS/CTAS populates a NULL point geometry object.
- Resolved and issue where COPY FROM was restricting all other access in a distributed setup
- Improved AVG handling when NULL values are used.
- Fixed a random segmentation fault that occurred when running render unit tests.
- Immerse now accepts a set of encrypted credentials for secured authentication of a custom application.
- Improved support for creating parameters on a new dashboard.
- Resolved an issue where the record count on a zoomed map would reset after editing.
- Fixed return key support for input of column type parameters.
- Fixed the resetting of column and default value after a parameter is created.
- Fixed caching of the number chart with a chart level filter.
- Resolved issue related to support of Booleans for column value parameters.
- Fixed an issue affecting null column values in quantitative color palette scales for Heat charts.
- Batch upgrading of charts includes additional edge cases.
- Resolved issue related to support for custom color dimensions in New Combo charts.
OmniSci release 5.6.1 includes a new feature and fixes several issues.
- Fixed an issue where a rename of a user with ALTER USER USER RENAME TO USER_NEW could cause issues later when trying to restart the server, depending on role membership.
- Fixed a system crash that occurred during an INSERT TABLE AS SELECT command on a table that does not exist.
- Fixed an issue where polygons from a table which had been appended using SQLImporter might not render. Added command-line option
--noPolyRenderGroups/-nprgto disable render group assignment, to match the available WITH option on other import workflows..
- When basemap sources are not available--for example, if the user is on a private network with no external access--the user is set to offline mode and Immerse uses the lightweight map source bundled with Immerse.
- Fixed an issue to make sure that Combo charts correctly plot on autobinning when the start-of-week flag is set to Saturday or Sunday.
- Duplication of charts with an empty filter is now allowed..
OmniSci release 5.6.0 contains several new features and improvements and fixes a number of issues.
- Allowed import and export paths are now enforced. Confirm that any import or export paths are included via the configuration parameters
allowed-export-pathsandallowed-import-paths. - Loading data now more strictly enforces validation or numeric inputs. Previously, an out-of-range integer would “wrap” and be ingested as incorrect values. In Release 5.6, numerics are validated and if validation fails, the row is rejected.
- String dictionaries do not load if you exceed their capacities; previously, they inserted null for the string.
- Utility jar
mapd-1.0-SNAPSHOT-jar-with-dependencies.jaris renamed. Scripts using SQLimporter need to reference the new .jaromnisci-utility-5.6.0.jar.
- Significant speedups for point-in-polygon joins (ST_CONTAINS, ST_INTERSECTS) via optimized algorithms and range join hash framework; on by default for point-in-polygon joins.
- Approximate median function support through the new
approx_median aggregatefunction. Currently works only in nondistributed deployments and executes on CPU. - Query interrupt is now possible when the query is outside of core kernel execution (for example, copy/import, CTAS, ITAS, query data fetch, result reduction). On by default and controlled by new flag
enable-non-kernel-time-query-interrupt. - Query interrupt during core kernel execution has been optimized so that enabling it with the existing
enable-runtime-query-interruptflag should have little to no impact on query performance; actual impact depends on the query pattern. - Added partial column specification for INSERT and INSERT FROM SELECT, such that data can be inserted and appended to a subset of columns in a table. NULL values are added to columns not specified in the statements.
- Updated the CREATE TABLE command
max_rollback_epochsoption with a default value of 3. This reduces the number of epochs/data versions that are stored. When combined with new space reclamation on delete, optimizes space utilization. Setting the option with a high value stores more epochs and reduces data compaction and space reclamation. To update older tables, use theALTER TABLE <table> SET MAX_ROLLBACK_EPOCHS=<value>command. - Added automatic metadata updates and vacuuming. On UPDATE and DELETE queries, OmniSciDB updates metadata, vacuums, and re-uses space as specified. Automatic metadata updates are turned on by default. See
enable-auto-metadata-updatein Configuration Parameters. Automatic vacuuming on delete occurs when deletes or updates on variable length columns exceed the vacuum threshold. For more information, seevacuum-min-selectivityin Configuration Parameters.
To ensure optimal space usage, ensure the table’s MAX_ROLLBACK_EPOCHS option is set to 3 or lower. This is automatically set by default for the CREATE TABLE command on new tables created in release 5.6 and higher.
For existing databases, OmniSci recommends using
ALTER TABLE <table> SET MAX_ROLLBACK_EPOCHS=3 to cap the space usage, and OPTIMIZE TABLE [<table>] WITH (VACUUM='true'); to reclaim space. This primes the system for efficient space management.- OmniSciDB now enforces permitted filepaths for import/export operations by default. The default import and export paths (
{data directory}/map_importand{data directory}/map_exportrespectively) are allowed by default, so no change is required when doing an import/export from Immerse or for sample data import on server startup. However, when using other commands with user-provided paths (such as COPY FROM) provided paths must be under an allowed root path. See the Configuration Parameters for more information.NOTE: For distributed systems, allowed paths only need to be in the configuration file on the aggregator. However, for consistency and future compatibility, OmniSci recommends that the allowed paths also be added to the leaf omnisci.conf file. - Significantly improved OmniSciDB startup time. Removed extra metadata seeks/reads during startup. Systems with high epoch counts or large column counts will have marked improvements.
- Validation of numeric inputs is more strictly enforced on load. Previously an out-of-range integer would “wrap” and be ingested as some incorrect value. Numeric-typed columns are now validated, and if validation fails, the row is rejected.
- String dictionary reaching capacity during import now throws an error and stops the load instead of inserting nulls in the dictionary-encoded column.
- Significant performance improvements for load into high-cardinality dictionary-encoded text columns via optimized dictionary hash table resizing.
- Improved load performance on systems with many cores. Default max number of threads to use is capped at 32, instead of 2X the number of cores, which can cause contention on systems with many cores.
- Improved the performance of left join queries with filters by searching for left-hand side filters that can be executed before the join, potentially reducing the cardinality of the join.
- Support date format
%m/%d/%ywith 2-digit year on data import. - Improved the performance of common multicolumn sort patterns.
- Chunksize default increased to 2 GB to allow for more records per fragment, particularly when variable-length columns are present (none-encoded strings, arrays, and geospatial types). This allows certain query patterns, such as large joins that might previously span multiple fragments, to run faster.
- Entitlements (introduce simple row level security via SAML attributes).
- Accept session-token when using S3 import, WITH option is s3-session-token.
- Upgrade to calcite 1.25.
- Improved the accuracy of ST_Contains and ST_Intersects at small geospatial scales.
- Allow Narrowing casts in SQLImporter.
- Upgrade to sqlite 3.34.
- Allow SAML and LDAP to provide authentication without having to also provide authorization.
- Utility jar renamed and versioned: omnisci-utility-5.6.0-SNAPSHOT.jar.
- Added support for CURRENT_TIMESTAMP, CURRENT_DATE and CURRENT_TIME functions
- Improved READ-ONLY support.
- UDTFs now allow for a ColumnList argument type, which allows a variable number of arguments to be used for inputs.
- Improved the granularity of the session lock, allowing local users to access the system even if third-party authentication providers have failed or are experiencing high latency.
- Many bug and performance improvements.
- Fixed issue causing a crash when UDTFs were composed.
- Fixed an issue where definition of CPU UDTFs could lead to all subsequent queries running on CPU.
- Resolved an issue where selecting from a view with a self join could cause a crash.
- Improved the ability of the parser to handle complex UPDATE and DELETE queries.
- Added human-friendly error messages to UPDATE and DELETE queries.
- Resolved an issue where specifying warmup queries would cause the aggregator to crash in distributed mode.
In distributed mode, specify warmup queries for the aggregator and all leaf nodes to work correctly.
- Resolved an issue where an equijoin between array columns could crash. Now, an attempt to join throws an exception if the join is not supported.
- Resolved an issue where window functions in a subquery could cause a crash.
- Resolved an issue where window function aggregates could crash if used with filters.
- Improved sorting performance of and resolved a number of stability and correctness issues.
- Resolved an issue in distributed mode where subqueries over replicated tables could return duplicate results.
- Resolved several potential deadlocks between the rendering engine and the query engine when multiple queries or types of queries are in process.
- Fixed an issue where SHOW CREATE TABLE would show the wrong table name after a table with shared dictionaries was renamed.
- Resolved an issue where a COPY TO command with a subquery could fail.
- Improved the quality of density accumulation rendering by incorporating a technique that uses 64-bit atomic operations instead of 32-bit when calculating standard deviation. This allows for a greater range of colors when using statistical measures to color the densities. The 32-bit technique was prone to uncaught overflows that resulted in a much more clamped color range.
- Performance improvement when using a Mercator projection explicitly for marks using the Vega projection property.
- Multi-GPU compositor performance and memory footprint is improved through better pipelining and reduced copy operations.
- New Arrow stock symbol shape can be used in symbol/legacy symbol marks.
- Line and polygon stroke rendering improved by 30-40% by simplifying the math to build the stroke geometry.
- Significant improvements to the quality of point/symbol rendering, achieved by improvements to procedural anti-aliasing technique and to render sample capturing when drawing procedural symbols.
- Polygon geometry imported via the SQLImporter path can now be rendered.
- Fixed a bug where decimals are not properly updated in vega scales after an empty render.
- Fixed a rare CUDA misaligned address crash when rendering multilayer vega with points, geo, and xforms in consecutive vegas.
- Resolved a small host memory leak in the renderer.
- Fixed a polygon render issue where fully transparent polygons can inadvertently affect the renders of visible neighbors.
- Fixed a crash when attempting to render a line mark layer with opacity explicitly set to 0.
- Fixed a potential race condition when DROP or CREATE is run in between renders or while render is in flight. The race condition could result in a crash when executing the get_result_row_for_pixel endpoint.
- Corrected an issue with multilayer rendering with multiple density accumulation layers where all density layers incorrectly applied the color palette from a single layer.
- Fixed a crash that occured when rendering a line mark in a multi-GPU-configured server/leaf when the number of GPUs required to render the lines jumps from one to more-than-one GPU when rendering consecutive vegas.
- Corrected the stroke rendering of ellipse symbols.
- Corrected possible skew in symbol rendering when using the angle property.
- Custom SQL dimensions, measures, and filters can now be parameterized, enabling more flexible and powerful analysis:
- Support for column, attribute, numeric, and custom parameter types
- New Parameter Manager where parameters are created, modified, and deleted.
- Parameter panel on the dashboard supports dynamic edits of the parameter values, and the ability to link/unlink parameters across dashboard tabs.
- All charts have been upgraded with the new custom SQL modal, which contains better SQL validation and ability to inject parameters and columns into the SQL.
- Chart-level error handling and recovery.
- Median added as a supported aggregation type across all charts and aggregate filters.
- A new angle measure has been added to Pointmap and Scatter charts, allowing orientation data to be visualized with wedge and arrow icons.
- When you mouse over a point, line, or poly on Pointmap, Choropleth, or Linemap charts, a popup box appears, where you can copy the popup information to your clipboard. If the popup includes a URL, you can click the URL to open the web page.
- The Other category can be toggled off from the categorical color legend on map charts.
- Immerse charts can now be configured with a Saturday or Sunday start of week by using the servers.json
ui/start_of_weekflag. Supported start of week days include Saturday, Sunday, and Monday (default behavior). - Zoom and panning performance has been significantly improved on New Combo charts.
- A new confirmation modal has been added to the Reset Dashboard Settings action to help prevent accidental loss of work
- On multilayer charts, unchecking layers from the master layer view now persists those changes and prevents unchecked layers from being displayed.
- Fixed issue on charts where click-based cross-filters from binned time dimensions were applied incorrectly.
- Fixed issue on Table charts where the chart did not highlight the row correctly after filtering on a boolean value.
- Fixed issues where Map and Stacked Bar chart data did not refresh correctly with the dashboard set to auto-refresh.
- Fixed issue in the global filter panel where the calendar picker was hidden behind charts when the dashboard was scrolled down.
- Fixed issue on map charts where color legends were not displaying correctly for multilayer charts.
- Fixed issue in New Combo charts where brush filters were removed when filter sets were switched.
- Resolved issue in New Combo charts where the range and focus charts were out of sync when there were multiple sources on the chart.
- Fixed issue in New Combo charts where newly created columns were not available for data selection.
- Fixed issue on new combo charts where x-axis labels were not updating correctly when sorting time columns.
- On New Combo charts, brush filters are now set correctly on duplicated charts.
- Fixed issues on Table charts where formatting was applied incorrectly to custom measures and date dimensions.
- Resolved issue on New Combo charts where bars with certain custom SQL measures in percentage mode did not render correctly.
OmniSci release 5.5.6 fixes an issue.
- License key input removes extraneous characters from pasted text.
OmniSci release 5.5.5 includes a new feature and fixes an issue.
- Align license management for Enterprise Edition and Free Edition.
- Resolved sign-on issues after idle session timeout expiry.
OmniSci release 5.5.2 includes new features and fixes several issues.
- SQLImporter allows text-based WKT or WKB fields to be imported into an appropriate geo column.
- A new command-line option, log-user-id, enables logging internal numeric user IDs instead of textual user names.
- OmniSciDB Arrow distribution is upgraded to v2.0.0.
- Resolved an issue where concurrent CTAS runs could deadlock.
- Resolved an issue where replacing an existing dashboard could cause the server to crash.
- Corrected behavior of multicolumn sort with NULL values.
- Resolved an issue when using ANY with a decimal array.
- Resolve an issue when loading BIGINT data into an ARRAY.
- Resolved an issue when using OFFSET with no LIMIT.
- Fixed incorrect left join-to-inner join optimization rule.
- Improved handling of CASE statement literals when dictionaries are used.
- Improved overflow checking.
- Fixed issues for persistence of deleted rows after OPTIMIZE.
- Improved locking for CONNECT to be less blocking.
- Fix a multilayer Vega render bug in which two consecutive renders with minimal changes in the code (for example, a change in layer ordering for) can result in a cryptic exception, but the Vega code is otherwise legal.
OmniSci release 5.5.1 includes a new feature and fixes several issues.
- Resolved an issue where newly created databases are unusable after server restart.
- Fixed an issue that could cause a crash during UPDATE or DELETE queries.
- Resolved an issue where UPDATE queries with a join would run indefinitely.
- Resolved an issue where a GPU-enabled version of OmniSciDB without GPUs present failed to gracefully fall back to CPU mode, resulting in a crash.
- You can now configure max legend width on charts from the Settings UI panel. You can save the configuration per-dashboard or database-wide.
- Fixed an issue where cohort filters were not applied correctly to New Combo charts.
OmniSci release 5.5.0 contains several new features and improvements and fixes a number of issues.
OmniSci 5.5 includes a significant change to the OmniSciDB storage layout. The migration from the storage layout prior to Release 5.5 is transparent, but is not backwards compatible. (OmniSci in general does not guarantee backwards compatibility.) Back up your OmniSciDB data directory before upgrading. If you are an OmniSci Enterprise customer, you can contact Support for help.
- VALIDATE command now detects and identifies out-of-sync epochs.
- Added support for concurrent SELECT and UPDATE/DELETE queries for single-node installations.
- New SHOW TABLE DETAILS command displays storage-related information for a table.
- Adds rendering support for CPU-only configured query execution.
- CPU-only query execution can be CUDA-disabled, or CUDA-enabled with the
cpu-onlyconfiguration parameter set totrue. - In either scenario, rendering support requires at least 1 server-side GPU.
- iGPU support is not guaranteed because of Linux driver incompatibilities with an OpenGL driver.
- User-defined function (UDF) support for NONE ENCODED strings.
- Aggregates in user-defined table function (UDTF) output are supported.
- Multi-table inputs into UDTFs is supported.
- JDBC and in binary endpoints load_table, load_table_binary, load_table_binary_columnar, and load_table_binary_arrow now support out-of-order and column subset inserts.
- Added support for geospatial transformation to SRID 4326 on CSV import.
- Added support for the ST_Centroid geometry constructor.
- Added support for importing geospatial data in WKB format from CSV files.
- Improved support for distributed mode and sharded tables to prevent data truncation.
- Improved support for query interrupt and query interrupt flag added. Query interrupt enables superusers to show a list of pending queries, and allows users to interrupt their own queries. This is currently considered beta functionality and off by default.
- Added MAX_ROLLBACK_EPOCHS support, which limits the number of rollback states kept on disk. Useful for deployments with small rolling updates and for managing table size on disks with heavy insert/update patterns.
To run OmniSciDB in GPU mode, OmniSci Release 5.5 requires NVIDIA driver version 450.xx or higher.
- Resolved a crash occurring on ST_DWithin when providing distance argument from another column in the same row.
- Fixed an issue where loop joins with large inner tables could crash.
- Resolved precision issues with date addition functions.
- Resolved issues that could occur when appending empty polygons to a table.
- Improved handling of hash tables near the entry count limit.
- Added a parameter to DATE_TRUNC to allow weeks to start on either Sunday or Monday.
- Resolved issues importing geospatial data from PostgreSQL using the SQL Importer utility.
- Fixed a security issue in the LDAP connector.
- The --help and --version server commands no longer require a data directory to exist before printing output.
- Annotations supported on new combo charts for different dimension/measure combinations.
- Annotations can be edited, moved, and resized; font size can be configured via the UI Settings panel.
- A top-level dashboard toggle controls visibility of annotations on the dashboard.
- New Combo chart improvements:
- Dashboard and chart level in-place migration for legacy stacked bar, bar, histogram, and combo charts to the New Combo chart.
- Chart zoom
- Toggle for All Others on the legend when a group by dimension is present.
- Toggle for the default null option on the group by dimension
- Column formatting in map pop-ups, including the ability to toggle the pop-up on/off without removing any columns.
- For exact match chart-level filters, a default All option that allows you to toggle off the filter from the dashboard view.
- For table charts with dimensions, a Null Measures toggle allows null measure values to be displayed.
- Performance improvements for map charts.
- You can set margins between charts using the UI Settings panel. The default margin is 16 pixels.
- You can set min/max manually on heat charts.
- Opacity can be set for single layer map charts.
- Matching for global string filters can be case-sensitive.
- Filter panel defaults can be set along two dimensions: open/closed, and simple/advanced.
- Added an instance-level flag (restricted_viewing) that prevents users from adding new charts or modifying existing charts if they are in a “view only” experience (VIEW but not EDIT privileges on the dashboard).
- Databases in the database switching list are now sorted alphabetically.
- Added the OmniSci-provided “Minimalist” base map as a default to all charts, regardless of online/offline status.
- Added the ability to configure default and max number of points on Pointmap charts.
- Added the user role “restricted_sharing” that prevents users with that role from being able to share dashboards.
Fixed Issues
- Fixed issue where the filter panel was scrolling off page by making it always present when opened.
- Fixed issue in geo export for multilayer charts where only the top layer was available for export.
- Dates on map pop-ups are now displayed in UTC instead of local time.
- Duplicated charts now are the same size as the original chart.
- Static and topN values now migrate correctly from Histogram and legacy Combo charts to New Combo charts.
- Fixed performance issues on the New Combo chart.
- The range chart on New Combo chart now displays the full data range.
- Fixed issue in New Combo chart, when configured as a line chart, where the default y-axis value was 0 instead of the data min.
- The number of groups from legacy bar charts now migrate correctly to the New Combo chart.
- New Combo charts now update correctly with streaming data.
- Fixed issue in New Combo charts where the collapsed legend was not saving correctly.
- Fixed issues with tabs causing crashes.
- The backend error message for missing tables is now passed correctly to the corresponding Immerse error modal.
- Fixed an issue in New Combo charts where long axis titles caused chart render issues.
- Brush filters in New Combo charts can now be applied in stacked/percentage mode.
- Fixed issue in new combo chart where the legend tooltip did not render in Chrome and Firefox browsers.
- Binning options in New Combo charts now correctly accept non-integer values.
- Fixed an issue in New Combo charts where modifying the range filter in one filter set applied the change to other filter sets.
- Resolved an issue New Combo charts where the default axis title was not set for custom dimensions.
- Legend options in New Combo charts now have sufficient contrast in Light Mode.
- Fixed multiple issues related to dashboard resize that caused charts to move around unexpectedly on the dashboard.
- Resolved an issue where saved dashboard views did not clear correctly between user sessions using the same browser.
- Fixed an issue where switching databases in the Chrome browser caused Immerse to pull an incorrect dashboards list.
- Fixed an issue where Jupyter access disappeared on dashboard refresh.
- Tabs can now be renamed when using the Safari browser.
- Fixed an issue in New Combo charts where a gap remained after a pinned legend was removed.
- In New Combo charts, source and data selection are now retained when switching between other charts and the New Combo chart.
- A display issue in Pointmaps that cut off the last digit for the number of points input box is resolved.
- Fixed an issue in multi-layer charts where chart-specific filters were incorrectly applied across layers sharing the same source instead of applied for the layer that the filter was created on.
OmniSci release 5.4.2 includes some new features and improvements and fixes a number of issues.
- Improved performance of large group-by queries over dictionary encoded strings when filters are present. The filters are applied to the group by reducing the size of the output.
- Added memory protection for APPROX_COUNT_DISTINCT queries. The configuration parameter
bitmap-memory-limitallows you to increase or decrease the maximum amount of memory allocated for APPROX_COUNT_DISTINCT bitmaps per execution kernel (thread or GPU). Previously, this allocation was not capped. The default cap is 8 GB, but can be increased for high-memory systems. - Added support for PowerBI as an OmniSciDB client.
- Resolved an issue where a query with LIMIT 0 in distributed mode could return results.
- Resolved an issue where singleton inserts or updates/deletes on sharded tables in distributed mode could result in truncation of the table.
- Fixed an issue where join queries could return incorrect results with NVIDIA driver 455. OmniSci continues to recommend driver versions 418.39 to 450 for OmniSci release 5.4.x.
- Fixed a crash on unsupported, multi-table self joins.
- Resolved an issue where an entire table could be projected for SELECT * LIMIT n queries, where n is small. OmniSciDB attempts to project only the first n rows, unless deletes without vacuum have occurred.
- Resolved several issues with multistep queries in distributed mode. Stability and performance of multistep queries, multistep queries with subqueries, and multistep subqueries have been improved.
- Fixed an issue with sharded tables in distributed mode where INSERT INTO SELECT queries did not persist changes to disk.
- Resolved an issue where the colors of a symbol render (via a Pointmap in Immerse) can be incorrect if it is immediately preceded by a Heatmap render.
- Fixed issue where users with the CREATE DASHBOARD privilege could not save their dashboards.
OmniSci release 5.4.1 fixes a number of issues.
- Resolved an issue where queries with extra whitespace returned an error. Whitespace is now trimmed automatically by OmniSciDB.
- Resolved an issue where SHOW CREATE TABLE showed an incorrect SHARD_COUNT for sharded table DDL in distributed mode.
- Fixed an issue where left joins could be converted to inner joins erroneously when joining multiple subqueries.
- Resolved an issue where dashboard owner updates were not properly persisted after server shutdown.
- Fixed an issue where global filters were not applying correctly to old Combo charts.
- Fixed and issue where chart-level filters could not be saved.
- Resolved an issue where layers were not displaying correctly for multilayer map export.
- Resoved an issue in linked pan/zoom where turning off the feature flag did not disable the feature for previously linked charts.
OmniSci release 5.4.0 contains several new features and improvements and fixes a number of issues.
- Improved performance of the CSV importer by up to 50%. String import, geospatial import via CSV, and importing wide varlen rows are all faster.
- Performance improvements for update and delete queries.
- Improved the performance of filtered group by queries on large cardinality string columns. This is particularly relevant for Immerse-generated cohort queries on dictionary-encoded string columns.
- Added initial compilation support for NVIDIA Ampere GPUs.
- Added support for Alteryx integration via the Alteryx Lua connector.
- Added support for Oracle Access Manager as an identity provider for SAML authentication.
- Added a new SQL function, SAMPLE_RATIO, which takes a proportion between 0 and 1 as an input argument and filters out rows to obtain a sampling of a dataset.
- Upgraded OmniSciDB Arrow distribution to v1.0.0
- Can now disable the origin lookup for inbound API requests. On busy systems with slow or throttled networks, DNS lookups can cause bottlenecks as multiple users attempt to access the database. Use the configuration flag
log-user-origin. - Resolved an issue where nested joins within a view could crash.
- Array fields in shapefiles are now imported correctly.
- JDBC driver now returns a valid update count. Currently, OmniSciDB does not report the number of rows updated, so the update count is -1.
- Resolve correctness issues in date/time
extractfunctions. - Resolved an issue where a Boolean column in an IN expression could crash the server.
- Fixed an issue where listing the roles for the default OmniSciDB user fails. This is resolved only for default users named admin. If your default user is still mapd, you can rename the user to admin by using ALTER USER RENAME TO.
- Resolved an issue where a sort with a limit over sharded or empty data could crash.
- Resolved an issue where some left joins were incorrectly rewritten to inner joins.
- Resolved an issue where a rendering hit test against a sharded table could crash.
- Fixed an issue with dump/restore of nonencoded geospatial columns. Previously, archives of nonencoded geospatial columns did not restore correctly. Note that the archives must be re-created.
- OmniSQL \d now shows geo column encoding.
- Apply filters to the dashboards list based on owner, source, last modified date, shared status, and dashboard name. A new count widget displays the number of filtered dashboards.
- Dashboard list filters can be saved as views and then duplicated, renamed, and deleted. Users default to the last filter view they used.
- Change font size, boldness, and contrast across chart titles and labels
- Customize axis label truncation
- Modify and add new color palettes
- Save UI settings as the database-wide default
- Dashboards now have tabs, which can be duplicated and reordered. A Play button makes it easy to cycle through tabs.
- Choropleth, Linemap, and Pointmap charts now support chart-level data export in GeoJSON and CSV formats.
- The min/max axis values for Scatterplot charts can be configured manually.
- A lat/long viewer is displayed when hovering over map charts.
- Added the # of Groups functionality to the New Combo chart, providing a consistent experience with other bar charts.
- Added the option to allow zooming on map and combo charts only when the SHIFT key is held down, to prevent accidental zoom actions.
- Fixed an issue in old Combo charts where charts did not load correctly with updated data.
- Resolved an issue where invalid filters caused dashboards to load incorrectly.
- Fixed an issue in New Combo charts where line marks were displayed as area marks when bar marks were present.
- Crowded display of date dimensions in the New Combo chart is fixed.
- Improved padding on New Combo chart, allowing the range chart to be displayed in a more compact way.
- Fixed an issue in New Combo charts where global filters were not correctly copied over during chart duplication.
- Fixed an issue in New Combo charts where injecting the column name in the custom SQL window repeatedly inserted the column name.
OmniSci release 5.3.2 contains new features and improvements and fixes a number of issues.
New Features and Improvements
- Added
LEFT JOINsupport for accelerated geojoins using the overlaps join. - Improved performance of projection queries with
OFFSET. - Improved performance of OmniSciDB function calls (e.g. geometric functions) by eliding null checks and eliminating runtime casts.
- Added support for implicit casting during insert into the table as
select. - Improved error messages and logging. In particular for casting errors, union all type mismatch errors, and create table as select / insert into as select logging all.
- During
Create table as select, dictionaries between the source and target tables will automatically be shared. This improves the performance of both the CTAS query and of any queries using both tables. AppendWITH (USE_SHARED_DICTIONARIES=FALSE)at th end of the CTAS query to disable this option. - Reduced connection timeout errors by introducing
keepaliveon API calls that use a client with a timeout (intended to replace manual management of individual service timeouts).keepaliveis controllable using thecalcite-service-keepaliveandleaf-with-keepalivecommand line options.
- Resolved an issue where a case statement in a subquery could crash the server with the error
Check failed: generation_ >= 0 (-1 >= 0). - Fixed an issue where an update on a temporary table could result in no rows being updated after a delete had occurred.
- Overhauled
date_truncandextractruntime, improving performance and resolving correctness issues with dates before January 1, 1970. - Resolved correctness issues with
TIMESTAMPDIFFandDATEADD. - Resolved an issue where queries could crash the server in a CUDA/Render enabled build running in
cpu-onlymode. CPU-only builds do not suffer the same problem. - Resolved an issue where
SELECT * LIMIT N(for any N) on a table with deleted rows would pull the entire table into memory. - Fixed an issue where a validate query preceding a SQL query could cause the error
Ran out of slots in the query output bufferduring SQL query execution. - Resolved a crash on group by queries with large output buffers.
Fixed Issues
- Fixed issue where New Combo chart could not be turned off via feature flag.
- Fixed issue where certain string values were handled incorrectly in Pointmap coloring.
OmniSci release 5.3.1 contains new features and improvements and fixes a number of issues.
New Features and Improvements
- GPU memory usage is improved across OmniSciDB and OmniSci Render. Previously, large amounts of reserved GPU memory (
res-gpu-mem) were required to prevent out-of-memory errors when rendering large line or polygon datasets. With more buffer sharing between the database and rendering engine, you can reduce theres-gpu-memvalue without increasing out-of-memory errors. - Added a query hint framework, allowing you to control query execution parameters in the query string. You can set CPU execution mode by adding
/*+ cpu_mode */after the SELECT keyword in a SELECT query. - Performance is significantly improved when sorting on COUNT DISTINCT and APPROX_COUNT_DISTINCT aggregates.
- Performance is improved in query parse, particularly for small-cardinality group-by queries. Additional performance improvements for multicolumn and large-cardinality group-by queries using baseline hash.
- GPU and CPU buffer pool allocation (slab) size are configurable as a runtime parameter, and the default for GPU buffer pool is increased from 2GB to 4GB for more efficient GPU memory usage.
- GPU JIT warmup is added to OmniSciDB at server start. As a result, OmniSciDB verifies the correct CUDA version and generates initial code for the GPU at startup instead of when the first query is executed. First query times on GPU after startup have been reduced. The server does not accept connections during JIT warmup, but a message is printed in the logs before and after warmup completes.
- Access to import/export paths to the server using the COPY command can be restricted. Administrators can restrict access to root paths used in COPY FROM and COPY TO commands. Currently, all paths are allowed by default.
- Added the SQL placeholder CURRENT_USER for accessing the username of the current user in a SQL query.
- Added support for the ALL PRIVILEGES alias when granting object privileges.
- Resolved an issue where queries with STDDEV inside a CASE statement could fail with the error message, “Aggregate function STDDEV not supported.”
- Fixed an issue where projecting NULL could crash the server.
- Resolved an issue where TIMESTAMPDIFF could return incorrect results for dates earlier than January 1st, 1970.
- Resolved correctness issues with the EXTRACT function on date, time, and timestamp values. Extract week could return incorrect results, and dates before January 1st, 1970 could also return incorrect results.
- Resolved an issue where
getString()on an array returned via JDBC would return an out-of-bounds exception. - Improved handling of cast and filter operations on high-precision timestamps. Reduced incidence of overflow/underflow errors when casting or filtering between timestamps of different precision.
- Resolved an issue where OmniSci Server could exit without an error code when encountering some errors (for example, too many open files during execution of an import query).
- Resolved an issue where user input was not effectively sanitized.
- New extension functions
reg_hex_horiz_pixel_bin_packed,reg_hex_vert_pixel_bin_packed, andrect_pixel_bin_packedreplacereg_hex_horiz_pixel_bin_x/y,reg_hex_vert_pixel_bin_x/y, andrect_pixel_bin_x/y, respectively. The packed versions return an image coordinate packed into a 32-bit INT, eliminating the need for separate x/y functions and reducing redundancy. Query times are improved up to 2x.
- Fixed an intermittent CHECK on a multi-GPU server when rendering a query where the results are distributed to a sparse subset of available GPUs.
- Addressed incorrect renders on multi-GPU servers when the
offset-device-by-table-idconfig flag is enabled. This flag is now enabled by default. - Fixes an exception safety issue in a multi-GPU server when an out-of-memory exception is thrown in the compositor before resource initialization is complete. This rare scenario would have resulted in a segfault.
- Fixes a CHECK when rendering polygon/multipolygon columns that were imported without an SRID and no encoding defined, usually via CSV when the SRID is not implied.
- Added the ability to build cumulative and percentage distributions in the New Combo chart for non-binned and categorical dimensions.