Archived release notes for release 4.x
OmniSci release 4.8.1 contains several fixes.
- Fixed an issue that occurred when creating a table with an invalid shared dictionary column.
- Resolved an issue where a CSV import with a row delimiter inside a quoted string would cause a premature line break, resulting in incorrect imported values for all subsequent rows.
- Resolved an issue starting OmniSciDB inside a Docker image on legacy versions of Docker (affects version 18.06 and prior).
- Fixed an issue affecting importing data from AWS S3 when the import statement was malformed.
- Fixed an issue where importing geospatial data using the SQLImporter would fail after importing 10,000 records.
- Fixed an issue affecting multi-step distributed queries over empty tables.
OmniSci release 4.8.0 contains several new features and fixes.
Major Changes and Known Limitations
- Thrift methods
get_row_descriptorare deprecated and no longer function. If you use one of these methods, you receive an error message describing how to migrate to the new API calls.
- Arrow version upgraded to 0.13.0.
- Minimum supported Nvidia CUDA driver is now 410.48.
- Significant additional controls available in the logging framework. See Logs and Monitoring. Note that the individual file names for the logs have changed from <service_name>.<machine_name>.<severity>.<timestamp> to <service_name>.<severity>.<timestamp>.log.
- When running an encrypted server configuration in Docker containers there is a current limitation that the aggregator omnisci_server and the omnisci_web_server must be running in the same Docker container and they cannot be started with the
startomnisciscript, which is the default. The use of a Docker COMMAND is required to start the services. For example:command: |/bin/bash -c "/omnisci/bin/omnisci_web_server --config /omnisci-storage/omnisci.conf & /omnisci/bin/omnisci_server --config /omnisci-storage/omnisci.confFor more information, contact OmniSci support.
New Features and Improvements
- Added support to log user's roles information during login event.
- Added support to allow communication among distributed system components to use encrypted binary connections.
- Added OPTIMIZE support for sharded tables, allowing administrators to create more optimal schemas.
- Added support for ST_Point geo constructor.
- Added new flag is_super (whether current session is super user) to session info returned from Thrift get_session_info.
- Improved performance of some high-cardinality group-by queries.
- Improved performance of ST_Contains by adjusting the bounding box heuristic.
- Improved error messages for unsupported function/type combinations.
- Fixed issue with ODBC connector not working with PowerBI.
- Fixed segfault during COPY FROM direct to leaf node on distributed setups.
- Fixed issue with Check Failure related to a query on a distributed sharded configuration.
- Improved signal handling and shutdown procedures for OmniSciDB server.
- Fixed an issue in geo join due to table join reordering.
- Fixed an issue where a non-Admin user could set a license key.
- Fixed an error resulting from running the same Vega render twice in quick succession.
- Fixed issue with a program error from a query that casts a TIMESTAMP(0) to TIMESTAMP(3).
- Added support for more graceful shutdown of OmniSciDB when receiving OS signals.
- Fixed issue with web server incorrectly switching to SAML mode if a session ID cookie is present.
- Fixed issue for server program errors on GROUP BY queries with nullable SMALLINT and TINYINT keys.
- Fixed issue where empty string as group name causes server program error.
- Fixed issue where the startomnisci script no longer works on CentOS.
- Fixed issue where ROUND_TO_DIGIT(x, y) (all data types) resulted in a server program error.
- Fixed edge case where Log10(NULL)/ Log(NULL)/LN(NULL)/EXP(NULL) was being computed and resulted in a number.
- Fixed issue where rendering join queries with multiple rowids produced the wrong results.
- Fixed issue where \detect did not appear in \h listing in omnisql.
- Fixed issue where requesting Arrow results on CPU (via select_ipc) results in the query executing on CPU, rather than just the resultset being located in CPU memory.
- Fixed issue to workaround where cuDF does not recognize arrow::date32 type, using arrow::date64 for gpu buffers.
- Fixed issue where query returns different results on distributed and single-node.
- Fixed issue where hit testing on Line map with color measure selection throws an error.
- Fixed issue with a render resulting in CHECK failure from a Choropleth Vega query.
- Exception for a query and subquery with differing limits.
- Line charts are deprecated in 4.8. While existing Line charts continue to function normally on dashboards, new edits to those charts cannot be saved. OmniSci recommends upgrading Line charts to Combo charts to take advantage of the latest feature improvements.
- Over the coming releases, OmniSci will stop developing new Immerse features for Internet Explorer. We strongly recommend all customers switch to a non-IE browser to experience the latest Immerse features.
New Features and Improvements
- Immerse now directly integrates with JupyterLab. You can send SQL queries to Jupyter from SQL Editor, or access it directly from top level navigation. See Jupyter Integration.
- Combo chart supports focus zoom. Moving the mouse wheel up and down zooms the chart in/out, while holding an Option or Shift key while scrolling pans the chart from right to left. Clicking the Reset button on hover resets the chart to the original timeframe.
- You can visualize Immerse in a new Dark Mode theme, accessible through account settings.
- Immerse supports importing and visualizing millisecond timestamp data.
- Immerse lets you duplicate charts within dashboards.
- Immerse lets you export and import dashboard metadata within Immerse.
- Immerse now pops up a modal when you are using a non-WebGL supported browser, informing you of the reduced experience.
- + and - buttons appear on geospatial charts as zoom controls.
- Offline maps now include a Minimalist base map layer at the country level.
- Custom base map layers are now defaulted first for easier selection.
- Updated combo chart's dynamic top 5 colors to consistently show the five series with the most data volume.
- Fixed issue where pop-ups on Choropleth were failing to appear for views created from multiple data sources.
- Fixed custom colors resulting in an add/remove loop in Combo chart, and added a toggle to control whether the chart is in dynamic or fixed mode.
- Incorrect global filters now error gracefully rather than remain part of the active filters list.
- Fixed issue where Choropleth was representing the Other category always as grey; now the Other category correctly shows the selected color.
- Fixed issue where error messages were showing up during dashboard duplication.
- Fixed issue where key manager shows key issuer instead of client name/company.
- Fixed issue where case-different, same text columns were causing a "table does not exist" error.
- Fixed issue where mapd-connector running under Node.js was incorrectly returning null for some functions.
- Fixed issue causing blank page when Add Chart button is pressed during dashboard load.
- Fixed issue where raster chart gaps showed when a Combo chart was present on the dashboard.
- Fixed issue where x-axis scale on a Combo chart is incorrect when the relative date filter is applied in addition to a second measure.
- Fixed issue where Bar chart was incorrectly applying formatting to color measures and dimensions.
- Fixed issue where filter label and dropdown menu were unconnected during the scroll action.
- Fixed issue where Combo chart binning label was incorrect.
- Fixed issue where certain custom colors were not discoverable in the Combo chart search dropdown.
OmniSci release 4.7.1 contains several fixes.
Fixed an issue where setting a dictionary encoded 16 or dictionary encoded 8 column as the shard key could cause a program error during data import.
Fixed an issue where OmniSci allowed insertion to a table using a select query where the precision of column did not match, resulting in an incorrect result. Columns must match exactly for insertion from a select query.
Fixed an issue where packed colors were broken with SPIR-V.
Fixed an issue where a literal projection coupled with an aggregate query could cause a program error.
Fixed an issue where adding an ORDER BY clause to a query executed in distributed mode could return 0 results, even if the query without the ORDER BY returned results.
Fixed an issue to allow time-related columns (TIMESTAMP, DATE, TIME) to be used as shard keys for sharded tables.
Added support for INSERT statements on sharded tables in distributed mode.
Fixed an issue that might occur with a join between two date columns where the inner table is empty.
Fixed an issue where decimal fixed encoding columns failed to load when using a load_table_binary_columnar endpoint.
- OmniSci Immerse
- Fixed an issue causing an increased margin on the right side of geospatial charts.
- Fixed an issue causing errors during dashboard load.
OmniSci release 4.7.0 contains several new features and fixes.
- OmniSci Core has been renamed to OmniSciDB throughout the product and in the documentation.
- Thrift methods
- Default user and database names are changed. Note and update your scripts accordingly:
- The default username has been changed from mapd to admin. All new data directories created with
initdbcreate a superuser with the username admin. Existing usernames are not migrated.
- The default username for all utilities is now admin. Existing users must override the default username in connection strings / command-line arguments of utilities (for example,
omnisql --user mapd).
ALTER USER <> RENAME TO <>is now supported, allowing you to rename the existing mapd user to admin if you do not want to use username overrides in your applications.
- After rename, any sessions connected to the renamed database are invalidated and need to log in again.
- The default database has changed from mapd to omnisci. All new data directories create a default database named omnisci instead of mapd.
- The default database for all utilities is now named omnisci. Similar to the username change, you can override the default database: for example,
omnisql --db mapdfor existing installations.
ALTER DATABASE <old_db> RENAME TO <new_db>is now supported, allowing you to rename an existing mapd database to omnisci for convenience.
- After rename, any sessions connected to the renamed database are invalidated and must log in again.
- OmniSci 4.7 fixes an issue with
OPTIMIZE TABLE WITH VACUUM, which previously could cause an incorrect entry in the storage for the optimized table. If you have used
OPTIMIZE TABLE WITH VACUUM, you should export and then re-import or reload your data.
- OmniSci 4.7 is the last version to support NVIDIA driver version 386. Future releases will require NVIDIA driver version 410 or higher.
- Any blank passwords (that is, nothing) are reset to a random 72-character string. This is required to prevent any accidental blank passwords enabled in the system. Previous versions automatically created blank passwords when logging on with LDAP or SAML, which creates a conflict with the new fallback to local login support.
- ODBC usage requires the license on the connecting server to be Enterprise enabled.
- \clear_gpu and \clear_cpu now require superuser privileges to run.
- Users that have access to a database through a role are now returned as part of a
get_userscall for a specific database. More users are now visible in shared Immerse dashboard dialogues.
- Apache Calcite version is now 1.19.
- For geojoin-configured Choropleth charts, Immerse now generates a new poly render query that no longer uses the server poly cache. The poly cache may be deprecated completely in a future release. The change includes better distribution of poly data across all available GPUs, causing an overall memory footprint improvement.
TEXT ENCODING DICT(n)columns cannot be encoded as less than 32 if they are to be used as a SHARD KEY on a sharded table. This restriction will be addressed in a later release.
- OmniSciDB now supports
INSERT TABLE AS SELECT(ITAS) on both single node and distributed, including sharding support.
- Joins on bucketed date/time are now bucket-size aware and more memory-efficient.
- OmniSciDB now supports Window functions. Empty Window function partitions are currently not supported; that is, all Window functions must include a
PARTITION BYkey in the
OVERclause. Also, OmniSci recommends that you use Window functions over aggregate query outputs at this time (instead of raw data). Future releases aim to improve scale and performance and remove these restrictions; see the documentation for details on what is supported.
- The default username is now admin.
- OmniSci DB now supports
ALTER USER RENAME, which allows you to rename users.
- OmniSciDB now supports Domain-based Authentication, allowing for multiple login modes (SAML, LDAP, and local) depending on user ID. This allows you to create OmniSci admin accounts outside of LDAP and SAML. The server first checks for SAML and then for LDAP. You configure this in the server configuration file. In addition, the server now allows authorization to fall back to local OmniSci authorization if LDAP or SAML (or both) are enabled, but fail. The
--allow-local-auth-fallbackcontrols this behavior, and is set to false by default.
- OmniSciDB now supports Distributed Polygon Rendering, allowing for much greater scale in rendering. For Enterprise customers running in Distributed mode, the Choropleth chart in Immerse now supports backend rendering by default.
- ODBC allows for cookies on http and https connections; relevant only if a load balancer is used.
OPTIMIZE TABLEnow recomputes metadata for dictionary-encoded string columns, supporting updates and deletes on dictionary-encoded string data.
- Performance improvements:
- (Experimental) Sorting columns at import time (a user operation) can result in improved performance by up to 3.5x on query patterns that involve filters on those columns.
- Improvements in distributed query performance involving joins. Distributed execution mode requires that your tables are structured correctly for optimal query execution. Immerse queries are formulated to work in Distributed mode, but customers using OmniSciDB with custom applications need to ensure that any table on the inside of a join is either replicated or the join relation is on a sharded column in both tables, unless both sides of the join relation are the result of an intermediate query step.
- View optimization is now on by default. OmnisciDB optimizes all view queries through RA rewrite. This should result in a noticeable reduction in the number of steps required by many view queries.
- Improved performance in distributed mode from more efficient result set serialization.
- Improved performance of multistep queries in which subsequent steps are projection nodes.
- Improved performance of filter queries in which the filtered column comes from the inner table in an inner join relation.
- Added support for
SQL WITHsyntax in poly render queries.
- Added support for
IF NOT EXISTSand
IF EXISTSto database
- Added new logging messages for monitoring query execution in Distributed mode.
- Fixed SAML-configured login that made regular access to other utilities awkward or restrictive.
- Improved performance by eliminating runtime loads of fixed-length array sizes that slow down POINT geo functions.
- Added cookie support to ODBC driver.
- Fixed a render error in Vega runtime that dumped the entire FRAGMENT SHADER code in the log.
- Fixed an issue where a scalar sub-query returned multiple rows exception was thrown while validating scalar sub-queries.
- Fixed an issue where update storage layer erroneously used
std::numeric_limits< double >::min()when initializing the max range of a nonnegative number.
- Fixed a program error resulting from a multi-level join query.
- Fixed a program error on hash tables with more than 2B entries.
- Fixed an issue where a cast from BOOLEAN to BOOLEAN was not supported, resulting in an error.
- Fixed a program error resulting from tables with a negative epoch.
- Fixed an issue where the importer did not skip rows with incorrectly sized fixed-length arrays.
- Fixed an issue where an exception resulted when \clear_gpu was run after any render.
- License configuration parameter now allows quotes.
- Fixed an issue resulting in incorrect results on CPU for group by query with CASE.
- Fixed a program error that resulted from a
COPY FROMon Parquet with
- Fixed a render error when going from a non-in-situ query to a projection query that uses multiple GPUs.
- Fixed a render error when updating from min/max -> stddev density accum renders.
- Fixed an issue where shard count requested with
CREATEwas different from what was reported with
\din Distributed mode.
- Fixed an issue where lack of rowid in projection queries negatively impacted Vega property updates.
- Fixed program error when shard count exists, but shard key does not.
- Fixed an issue with distributed line rendering not respecting Vega line format with scalar columns.
- Fixed an issue involving joins on a value or null. For example, a.x = b.x or a.x is null and b.x is null.
- Fixed a regression with multilayer render on Distributed systems using Vega xforms and aggregation/projection queries.
- Fixed importer program error resulting from shapefile with null geo features.
- Fixed program error during
SAMPLEon a geo join in nondistributed mode.
- Fixed program error with
GROUP BYwith BIGINT and INT columns.
- Fixed server program error resulting from
CASE WHEN/THENstring test select join with INT, FLOAT, and SMALLINT columns caused a server check and program error.
- Fixed an issue where a user could not be dropped if the username contained a period.
- Fixed program error with geo function filters under certain circumstances.
- Fixed an issue with JOIN WITH subquery not returning results in a Distributed environment.
- Fixed an issue where a non-superuser user was not included in
get_usersThrift call when the user has ACCESS/SELECT database rights.
- Setting a license key through Immerse or omnisql now works in HA mode.
- Invalid SQL casts now produce more meaningful error messages.
- Fixed a program error involving render query hit testing in Distributed mode.
- Fixed a Vega bug involving dictionary-encoded string columns from an aggregate target in an ordinal scale.
- Improved performance of projecting queries with geospatial targets in Distributed mode.
- Calcite server no longer has a program error if the user passes in an empty string.
- Distributed Poly rendering on Choropleth charts introduced in 4.7 has the following known limitations (Enterprise Edition):
- With the server-side poly cache no longer being used, some render performance issues with geojoin configured Choropleth charts can occur. If the dimension table for the geojoin (that is, the poly table) has a large number of rows, you might see some performance degradation in release 4.7. Internally, tables <100K polygons tend to perform equally to the cache, 100K-500K rows sees mild degradation, and >500K rows can see more substantial degradation. Performance is ultimately dependent on the total number of vertices (not rows).
- Currently, neither Immerse nor the server can be configured to use the poly cache. This limitation impacts Immerse users only. Custom clients doing server-side polygon rendering using join queries can continue using existing Vega code and use the poly cache. OmniSci recommends that you switch to the new query format; instructions for doing so will be included in a future release. If you prefer to make these changes beforehand, contact OmniSci support.
- A known issue with Distributed poly rendering and replicated poly tables can result in overdraw of the polygons from each leaf in projection queries. This might not impact the final render if the polygons are rendered fully opaque. If they are rendered with transparency, the final render is less transparent than expected due to overdraw.
- Line chart in Immerse is scheduled for deprecation in a future release, to be replaced by the Combination Bar/Line (Combo) chart.
- For geospatial charts, changes in release 4.7 introduce an increased margin on the right side. This is scheduled to be fixed in a future release. As a workaround, resize the window to remove the margin.
- You must upgrade to Windows 10 1903 and Edge 44.18362 to use Edge with OmniSci. If you choose not to upgrade, you can use an alternate browser with OmniSci.
- Visual Data Fusion supports plotting data from multiple data sources on the same Combo chart, which also enables cross-filtering with other charts. This was previously available only on geospatial charts.
- Filtering on an aggregated value is supported in Pointmap and Scatter Plot charts.
- Data import status provides time estimation and information on import success.
- Server-rendered Choropleth charts are supported in Distributed mode.
- Data import popovers provide help text on file formats available for import.
- Binning limits on Line and Combo charts are increased from 400 bins to 1000 bins. Note: When binning granularity is set to auto mode on a Line or Combo chart created prior to version 4.7, the reported bin granularity on the chart (for example, Hour) might not match the actual bin value (for example, Day). Workaround: Recreate the chart in version 4.7.
- Fixed an issue for applying a global timestamp filter to a Combo chart where the displayed dimension is not of TIMESTAMP data type.
- Fixed an issue where user did not appear in list for sharing despite having database permissions.
- Fixed issues with Combo chart related to setting min/max values on the x and y axes.
- Fixed an issue where the locked icon ona Combo chart disappeared, despite having a fixed value set on the axis.
- Fixed an issue where the top bin of binned data was missing values due to rounding discrepancies.
- Fixed an issue where Combo chart did not bin correctly on cross-filter.
- Fixed an issue where a new license could not be pasted in after an previouis license expired.
- Fixed an issue in Bar chart where sorting by measure did not update the chart.
- Fixed an issue to better address auto-binning in Combo chart.
- Fixed an issue in Stacked Bar chart for setting custom colors.
- Fixed an issue in Line chart for setting an incorrect aggregation due to color dimension.
- Fixed an issue in Combo chart for automatic axis labels.
- Fixed issues with Choropleth chart error handling.
- Fixed chart-switching error issues in Pointmaps.
- Removed Append Data button for users without INSERT permission.
- Fixed x-axis label issue in Stacked Bar charts.
- Fixed tooltip sorting in Combo charts so that the ordering is consistent.
- Removed default brush on focus chart after a range chart brush is set.
- Fixed an issue causing an error state for Append Data using Internet Explorer 11.
OmniSci release 4.6.1 contains several fixes.
- Fixed an issue where a grouped pointmap with multiple "SAMPLE" measures could cause a program error.
- Fixed an issue where a JOIN on the result of an unnest operation caused a program error.
- Fixed an issue where QueryMemoryInitializer was not passed safely to the JOIN loop builder.
- Fixed an issue where queries retried on CPU with subqueries that had already been executed would cause a program error.
- Removed buffer size limitations between cluster aggregator and leaves.
- Fixed an issue where
sql_execute_dfwould throw an exception when encountering a BOOL column.
- Restored SQL execute timings to std_log output.
- Fixed an issue with intermittent server program errors with CUDA error code 201 in egl compositor.
- Improved error resolution when an error is thrown during the build of a poly cache.
- Fixed an issue with COPY TO behavior when executing in HA mode.
OmniSci release 4.6.0 includes several fixes and performance improvements.
- Syscatalog/catalog separation has changed directory layout. System catalog/system database is now called omnisci_system_catalog and is in a separate folder in catalogs. The default database is still named mapd, but it is a standalone database.
--enable-access-priv-checkflag has been removed. Access privilege verification is now required and enabled by default for OmniSci version 4.6.0 and higher.
- CTAS (CREATE TABLE AS SELECT) is now supported on distributed installations.
In release 4.6.0, avoid using CTAS to create tables with very large row counts (for example, in excess of 10 million rows). In such cases, the leaf-aggregate communication via Thrift imposes size limits (2GB), which OmniSci is working to address.
- OmniSci supports updates on variable length columns, specifically arrays and strings with encoding NONE.
partition='replicated'option for COPY FROM command on geo tables [specified via option WITH (geo = 'true')].
- Further foundational improvements in efficient columnar output, allowing for better memory efficiency with larger result sets. This is behind a configuration flag and will be turned on by default in a future release.
- Added support for importing from gdb(Esri geodatabase) format.
- Added support for importing from Parquet files, from both local and S3, including auto-detect capabilities on omnisql. Immerse will support import of Parquet files in a future release.
- Improved string dictionary performance from use of rk_hash fingerprints. This is behind a configuration flag and will be turned on by default in a future release.
- Added support for custom S3 endpoints, allowing for non-AWS S3-compatible storage systems to be used as input sources.
- Added support for Single Sign-on using Okta (SAML 2.0 Identity provider) for authentication.
There is a known limitation with users set up via Okta, where such users cannot be superusers. OmniSci will address this in an upcoming release.
\lcommand to list users' database access is now available to non-Superusers.
- Added support for PKI-based authentication starting with the ODBC API.
- Users can now be assigned a default database as part of CREATE or ALTER USER command options.
- SQLImporter now supports HTTPS and HTTP besides the binary protocol.
- Lifted limit on dictionary entries from 1B to 2B, allowing for larger string dictionaries.
- Logging format now standardized to allow for better import of OmniSci logs in a delimited file format (some minimal post-processing is required).
- Improved performance of between 25-35% on string dictionary reloads from better thread utilization.
- String dictionary server status now visible as part of
\statuscommand in omnisql.
- Added support for specifying TIMESTAMP precisions, including TIMESTAMP(3) - milliseconds, TIMESTAMP(6) - microseconds and TIMESTAMP(9) - nanoseconds. Immerse will support this feature in a future release.
- Added support for CARDINALITY for array columns (returns the number of elements — for example, the length of an array column).
- Added support to allow variable length array columns to be set to NULL.
- Added early, experimental support for scalar user-defined extension functions written in C++/CUDA. These kernels can now be compiled on server startup and visible to the server runtime immediately. Please refer to the relevant OmniSci documentation for details on how to use this capability.
- Added Thrift support to list the databases that can be accessed by the currently logged in user. Also added support for a user, once logged in, to switch between these databases without needing to login again.
- Added support to allow for dashboards to be shared via roles.
- Eliminated per-fragment overhead for non-lazy GPU projections, allowing larger result sets for rendering and other use cases.
- Fixed issues with non-existent source in dashboard metadata which was causing migration to fail - this is now logged to enable better diagnosis of the scenario.
- Fixed an issue where an empty array query resulted in a server program error under certain circumstances.
- Fixed an issue where the Calcite server alone failed, resulting in an unusable but running OmniSci Server. Now, should the Calcite server fail, OmniSci Server does not allow new connections. The failure is captured in the logs, and OmniSci Server is stopped so that the
systemdutility can restart both the OmniSci and Calcite servers.
- Fixed an issue where binary load from arrow load for DATE resulted in incorrect data.
- Fixed an issue where an INTERVAL ADD to timestamp resulted in a program error.
- Fixed a case where a large number of noisy error messages were generated in the logs from auto-completion hint failures.
- Fixed issues with spurious and potentially confusing "Connection refused" Thrift messages on server startup.
- Fixed an issue with inability to export a dashboard from admin account, when admin is not the owner of the dashboard.
- Fixed an issue where OmniSci HA node goes out of sync after program errors during execution of Kafka message instructions.
- Fixed an issue with ALTER TABLE RENAME on multipolygon columns.
- Fixed an issue where querying views with joins resulted in a watchdog error.
- Fixed an issue where dropping a database resulted in recycled database IDs that caused unexpected behavior in the catalog.
- Fixed program error on selecting Geo column from Outer Join.
- Fixed program error resulting from certain SELECT * queries.
- Fixed case where GRANT SELECT VIEW on database resulted in Update privilege.
- Fixed cases where CTAS (CREATE TABLE AS SELECT) involving SELECT *, or CASE statements resulted in a program error.
- Fixed an issue with version 4.4.2 not correctly updating buffer shapes on map unless map boundaries change.
- Fixed a rendering regression related to log/sqrt/pow scale types in Vega.
- Fixed an issue where Optimize Table command resulted in extra disk space being consumed on each run.
- Fixed an issue with ODBC sessions stopping on reconnect. ODBC sessions now continue upon successful reconnect to omnisci_server.
- Fixed an issue regarding incorrect text array length is shown in the new table after running CTAS.
- Fixed an issue where SELECT queries failed after UPDATE on a newly added text column added via ALTER TABLE ADD COLUMN.
- Fixed an issue with visibility of s3 credentials in history file.
- Fixed a threading issue in CTAS that caused server failure.
- Fixed an issue related to LDAP that affected omnisci_server users who had not yet been set up in the system.
- Fixed an issue where multiple SAMPLE invocations in a single query resulted in inconsistent results.
- Fixed an issue with program error on updating copy_params and re-detecting CSV.
- Fixed an issue with approx_count_distinct causing system failure on GROUP BY with more than one group.
- Fixed an issue where ST_ functions on a geo column used in a color measure were not parsing correctly in render request.
- Fixed an issue with incorrect renders on distributed systems involving Vega transforms with aggregate queries.
- Immerse supports integration with OKTA/SAML for Single Sign-On based authentication.
- Tables can be deleted directly from Data Manager.
- Data can be added to any table via the Append functionality in Data Manager.
- All rows within a table can be removed via the Delete All Rows functionality in Data Manager.
- Users can denote when a file has a header row during data import, using the Includes Header Row option within Import Settings.
- Added in situ poly rendering for backend-rendered choropleths, greatly enhancing performance at scale.
- Added sampling to server-rendered Choropleths.
- Added crossfiltering and bounding box filter to server-rendered Choropleths.
- Max number of groups that can be visualized in Bar and Bubble charts increased from 100 to 500.
- Fixed an issue where clause sampling caused overflow with greater than 3.4 billion records.
- Fixed an issue where brush was not aligned with bin increment in Combo chart.
- Fixed an issue where Amazon S3 import could not be performed using a full link URL.
- Turned off ability to rotate MapBox base maps to improve usability.
- Import Data button is hidden on Data Manager screen if user lacks privileges to import data.
- Fixed inconsistencies with timezones; all dates shown in Immerse are displayed as UTC.
- Fixed an issue where the OmniSci logo appears before being overwritten by servers.json configurations.
- Fixed an issue where Choropleth pop-up does not disappear when user zooms in on the map.
- Fixed an issue in Bar chart where sort by selection was saved but not applied when switching color measures and aggregates.
- Fixed an issue in Stacked Bar chart where custom colors selected in the color palette were not respected on the chart display.
- Fixed an issue with Stacked Bar where axis label renaming was not persisting.
- Fixed an issue in Combo chart where the chart editor right panel would become unresponsive after certain settings were configured.
- Fixed an issue that prevented a user from changing measure formatting without clearing formatting first.
- Fixed an issue with incorrect default formatting.
- Fixed an issue in chart editor where popup remained active after required measure/dimension had been selected.
- Fixed issues in Table chart, Histogram, and Heatmap that prevented date formatter from showing up correctly.
OmniSci release 4.5.0 includes several fixes and performance improvements.
- With release 4.5.0 and higher, OmniSci now requires that all configuration flags used at startup match a flag on the OmniSci server. If any flag is misspelled or invalid, the server does not start. This change helps to ensure that all settings are intentional and will not have unexpected impact on performance or data integrity.
- Licenses are now required for OmniSci Enterprise Edition. Servers start but are not functional without a valid license file installed.
- OmniSci configuration files are now strictly validated on startup. Servers do not start with an unrecognized option in the configuration file, and an error message is logged to stderr containing the unrecognized option.
- Default port numbers for OmniSci services have been changed. In addition, the names of key services are now prefixed with omnisci_, changing from mapd_. Symlinks from the old names are provided, but will be removed in future versions. Please note these changes and upgrade any scripts/workflows that use the old names.
Due to a system-related change in the behavior of DROP/TRUNCATE, data files are not deleted from disk, but tombstoned and identifiable by the DELETE_ME prefix on the filesystem for up to five minutes after you issue the command.
(Applicable to distributed installs only): StringDictionaryServer now lazy loadsdictionaries on request. Consequently, the first request to a dictionary-encoded column of a large table can be slow for very large tables with high-cardinality string dictionary columns.
--cpuflags are replaced with the
--cpu-onlyflag for OmniSci Server.
- Community Edition has been deprecated and replaced with Enterprise Trial. See https://www.omnisci.com/platform/downloads/ for details.
- Renamed key components from MapD to OmniSci. Specifically:
- mapd_server is now omnisci_server.
- mapd_web_server is now omnisci_web_server.
- mapdql is now omnisql.
- String Dictionary Server is now omnisci_sd_server.
- JDBC connection strings now are jdbc:omnisci:localhost:6273:<dbname> and the JDBC driver class changes to com.omnisci.jdbc.OmniSciDriver.
- Better memory handling through improved estimation of query GPU memory requirement. Allow-cpu-retry is now turned on by default, which allows queries in certain low-GPU-memory situations to be identified and executed on CPU automatically.
- OmniSci Core now supports inserts of array literals containing NULLs.
- OmniSci Core now supports importing csv files with arrays containing NULLs.
- Changed default port from 9092 to 6273 to avoid conflicts with other well known services running on those ports.
- DECIMAL / NUMERIC fields can now be downcast to different scale/precision.
- Dictionary size limit now increased to ~2.15Billion entries.
- Deleting tables using the DROP command will immediately remove the table from the catalog but asynchronously remove data from disk.
- Added interval support for DATEPART/DATEADD operations.
DATEPART- weekday, quarterday
DATEADD- week, weekday, decade
- Added ability to recompute table metadata on demand using the OPTIMIZE command.
Fixed Issues and Enhancements
- omnisci.conf is now validated at server start. This is to prevent any erroneous configuration settings taking effect. If erroneous config parameters are found, the server fails to start and throws an error.
- Fixed issues when geo data is imported (for example from PostGIS) via JDBC.
--cpuflags with a
--cpu-onlyflag for OmniSci Server.
- Fixed an issue where geo files could not be appended to tables created through Immerse geo file import process.
- Fixed program error caused by selecting SAMPLE on a geo data type column.
- Fixed an issue with ODBC not reconnecting on a server restart.
- Fixed certain cases where DATE was handled incorrectly by SQLImporter.
- Fixed an issue where session IDs were being logged in certain cases.
- Fixed an issue where Importer did not estimate number of rows correctly.
- Fixed an issue with JDBC handling of the type of a geo column.
- Fixed an issue where JDBC needs to filter DB information from Table Priv Call.
- Fixed an issue that prevented sharding on a date column.
- Fixed an issue where projecting a varlen target inside SAMPLE in distributed mode could corrupt subsequent columns.
- Fixed a typo in table schema output from the
\dcommand in omnisql.
- Fixed an issue projecting varlen columns inside a top k query (for example,
SELECT str, x FROM proj_top ORDER BY x DESC LIMIT 1;).
- Resolved a program error when running a subquery projecting dictionary encoded strings inside an
- Fixed an issue where out-of-bound integer ranges fed to timestamp column erroneously get turned into null sentinels.
- Conditionally remove
ST_CONTAINSfilter when computing the filtered count for a geo join.
- Improved accuracy of
ST_CONTAINSon certain complex MULTIPOLYGON shapes.
Fixed Issues and Enhancements
- Fixed an issue with Vega Scale parse failing if range values are whole numbers.
- Fixed an issue with GLLayoutBuffer exception related to rendering the same Vega in two successive
- Fixed intermittent error with renders using Percent Accumulation Scale.
- Enhancements to messages related to Vega parse errors.
- Add support for lasso filter on line map chart.
- Added clarity to formatting options and created a new option to represent billions as B.
- Geo Heatmap colors rendered that do not fall in scheme selected now display in grey rather than purple.
- Sort null measure values to the end by default for grouped chart queries (crossfilter and Stacked Bar charts).
Fixed Issues and Enhancements
- Fixed an issue with session connection being lost on timeout.
- Fixed ability for data tables to be switched via the search interface in Data Manager.
- Fixed an issue where exported CSV does not match chart or omnisql data.
- Fixed circle, polygon and lasso filters for Linemap.
- Fixed cross-filtering for Number chart.
- Fixed an issue with losing color when clicking on column in Stacked Bar chart.
- Fixed an issue with overflow of bars when switching to Stacked Bar chart.
- Fixed issues with non-functioning backend-rendered multi-layer map and backend-rendered multi-layer Scatter Plot examples.
- Fixed an issue where Stacked Bar chart was not maintaining a filter on dashboard load.
- Fixed an issue with HeatMap chart related to the assignment of default green colors for 0 values. Values that are 0 are assigned the mid-range green color.
- Fixed an issue in Chart Editor where disabling NULL values after a dimension drag and drop generated a query causing a 502 error.
- Fixed an issue with Immerse failing to load correctly if there was no servers.json file.
- Fixed an issue with disappearing dashboard header in certain cases.
OmniSci release 4.4.2 includes several fixes and performance improvements.
Fixed Issues and Enhancements
- Improved error logging when the
mapd_serverrelease is older than the server's data directory.
- Added support for Arrow DATE values using small date buffers.
Fixed Issues and Enhancements
- Fixed an issue where using conditional expressions (
CASE WHEN condition THEN result ELSE default END) in a custom SQL dimension would cause a chart error.
OmniSci release 4.4.1 includes new features as well as several fixes and performance improvements. It comprises the 4.4.0 release version and a fixed regression.
- Added support for the
- Added support for
- DATE datatype is compressed. Dates now have two smaller encoding sizes: 4 bytes (default) and 2 bytes. Existing date columns continue to work as before.
- Added support for SAML authentication with Okta (Enterprise Edition only).
Fixed Issues and Enhancements
- Usernames can now include dashes.
- Attempting an ST_Contains join with a geospatial string literal argument no longer causes a program error.
- User session IDs are no longer potentially exposed in OmniSci Core log files.
- The hit-test cache limit has been removed and should eliminate the following error: "Cannot retrieve results for query <query>. The results have not been cached for hit-testing."
- Fixed database consistency errors that could occur if the underlying filesystem ran out of disk space.
- Fixed a synchronization issue for overflow/underflow checks on NVIDIA Volta and Turing GPUs.
- Improved performance on String Dictionary import for multiple String Dictionary-encoded columns.
- Intermediate ResultSets are now automatically compressed before being shipped between the aggregator and leaf servers. This allows ResultSets over 2 GB in uncompressed size to be passed around the cluster and can result in performance improvements due to better utilization of network bandwidth.
- Fixed an issue where concurrent updates/deletes in import operations could cause table corruption.
- Fixed a corner case where BOOL, TINYINT, or SMALLINT columns could result in invalid data during import.
- Fixed an issue with rowids projected in CTAS that caused an error in table creation, because rowid is a reserved column name.
- Added support for AM/PM designation in timestamps.
- Fixed an issue where sharded tables with geospatial columns could cause a program error during insert or projection queries.
- Improved performance for queries using 32-bit timestamps.
- Improved memory usage during hash joins.
- Fixed an issue in which hash joins on Dictionary-encoded strings, where one of the columns was empty, could cause a program error.
- Deprecated/removed the following obsolete flags:
- Support is added for pct/blend accumulation rendering modes in distributed configurations.
Fixed Issues and Enhancements
- Multiple point/symbol layers using Vega projections no longer potentially render a blank image.
- In distributed configurations, a multilayer rendering issue occurred in which an aggregate query and a projection query, each referencing Vega transforms, could cause a program error. This has been fixed.
- Using Vega transforms on line/poly queries with empty data no longer causes an error; it now produces an empty image, as expected.
Fixed Issues and Enhancements
- Improved error tracking to allow better triage and resolution.
- Fractional numbers are now supported in global filters in Immerse.
- Fixed an error that occurred when switching chart types with binned dimensions.
- Deprecated: The Export Chart button has been removed from Text charts.
- SAMPLE aggregate is added as an option for all chart measures in conjunction with GROUP BY.
- Fixed ambiguous-column error for identically named columns on geo joins.
- Fixed issues with importing of tables with geospatial columns.
- Resolved issue with modifications to Custom Colors on Combo charts with color dimensions were reflected in the right pane of the UI, but not the chart palette, which caused incorrect rendering.
- Updated license uploading for Community Edition.
OmniSci release 4.3.0 consists of a number of new features as well as several fixes and performance improvements outlined below.
- Decimal compression: Decimals now automatically use the smallest possible encoding for the specified precision/scale.
- Hash joins now support null left-hand side joined to not null right-hand side and vice-versa.
- Hash joins allow different integer types between the left-hand side and right-hand side.
- Ensure MAX_ROWS is an integer literal and properly reject non-integer values.
- Fixed an issue where CTAS did not properly parse date_trunc functions and geometry casts.
- Fixed an issue where DROP DATABASE did not remove all table data from disk.
- Fixed an issue where the minus sign was being counted as part of decimal precision.
- Improved error handling in distributed mode.
- JDBC now properly generates boolean string literals when
- Properly validate DECIMAL type ranges and check for overflows.
- Resolved an issue where DB owners may not have access privileges on new databases by default.
- Resolved a system failure that occured for unsupported DDL statements.
- Resolved a system failure that occured if a database were created then dropped before a user connected to it.
- Resolved a system failure that occured when serializing multiple geospatial types in distributed mode.
- Resolved a system failure when clearing GPU memory from mapdql when no GPUs are present.
- Resolved a misleading error message no dashboards were returned while listing dashboards in mapdql.
- Resolved a problem with geo append when existing table has uncompressed coords
- Resolved an issue where ALTER TABLE on a view caused a system failure.
- Resolved an issue where an invalid geospatial literal could cause a system failure.
- Resolved an issue where approx_count_distinct could send too much data and/or cause a system failure during serialization in distributed mode.
- Resolved an issue where deleted users were not totally purged after being dropped.
- Resolved an issue where hash joins on distributed would run incorrectly if one of the tables was not replicated or both tables were not sharded.
- Resolved an issue where object privilege migration could be incomplete.
- Resolved an issue where precision errors could occur during queries involving aggregates.
- Resolved an issue where session timeouts could cause a system failure in distributed mode.
- Resolved an issue where the roles display for a user could be incorrect.
- Resolved an issue where using privilege APIs could cause a system failure if object level privileges were disabled.
- Resolved precision errors in ST_Distance and ST_Intersect for some geospatial types.
- Resolved various issues in automatic FROM clause table reodering.
- The last row of a CSV file is no longer ignored if the CSV file does not have a trailing new line.
- Added support for in-situ line rendering (i.e. without a copy) for projection queries (LINESTRING columns only)
- Properly handle rows where all columns are NULL in the Vega transforms API.Resolved an issue handling NULLs in Vega scales.Resolved an issue when multiple sessions ran the same polygon hit testing query.Resolved an issue where Vega projections would fail to be applied.
- Enabled in-situ line rendering for linemap.
Fixed Issues and Enhancements
- Added schema validation to prevent an edge case where dashboards could become corrupt and unloadable.
- Fixed an issue with linemap hover popups on single node instances.
- Fixed an issue where decimal-baded global filters returned incorrect results.
- Fixed an issue where the Y-axis measure in the stacked bar chart could be missing a name label.
- Fixed an issue with categorical color measures in choropleth that could result in a hang.
- Fixed an issue where formatting dates on combo chart could result in a hang.
- Added code to ensure measure aggtypes can be updated on linemap.
- Removed an error thrown when changing choropleth color dimensions.
- Fixed an error in linemap where geo column names other than mapd_geo or omnisci_geo could cause errors.
- Improved handling of null data on piechart.
- Adjusted linemap hit testing behavior to match pointmap.
- Added code to prevent an issue where dashboard state was not preserved when switching to SQL Editor.
- Fixed an issue with custom SQL measures in linemap.
- Improved the legend for min/max numerical color scale for linemap.
- Optimized viewport clipping for rectangular/non-complex shapes.
- Added code to ensure measure aggtypes can be updated on linemap.
- Improved handling of color measures for linemap.