Welcome to OmniSci Documentation
Learn how to use Immerse to gain new insights to your data with fast, responsive graphics and SQL queries.
Ship traffic analysis
Local data import preview
Learn how to extend OmniSci with an integrated data science foundation and custom charts and interfaces. Contribute to the OmniSci Core Open Source project.
Pyomnisci interface
- Significant speedup for POINT and fixed-length array imports and CTAS/ITAS, generally 5-20X faster.
- The PNG encoding step of a render request is no longer a blocking step, providing improvement to render concurrency.
- Adds support to hide legacy chart types from add/edit chart menu in preparation for future deprecation (defaults to off).
- BETA - Adds custom expressions to table columns, allowing for reusable custom dimensions and measures within a single dashboard (defaults to off).
- BETA - Adds Crosslink feature with Crosslink Panel UI, allowing crossfilters to fire across different data sources within the same dashboard (defaults to off).
- BETA - Adds Custom SQL Source support and Custom SQL Source Manager, allowing the creation of a data source as a SQL statement (defaults to off)
- Parallel execution framework is on by default. Running with multiple executors allows parts of query evaluation, such as code generation and intermediate reductions, to be executed concurrently. Currently available for single-node deployments.
- Spatial joins between geospatial point types using the ST_Distance operator are accelerated using the overlaps hash join framework, with speedups up to 100x compared to Release 5.7.1.
- Significant performance gains for many query patterns through optimization of query code generation, particularly benefitting CPU queries.
- Window functions can now be executed without a partition clause being specified (to signify a partition encompassing all rows in the table).
- Window functions can now execute over tables with multiple fragments and/or shards.
- Native support for ST_Transform between all UTM Zones and EPSG:4326 (Lon/Lat) and EPSG:900913 (Web Mercator).
- ST_Equals support for geospatial columns.
- Support for the ANSI SQL WIDTH_BUCKET operator for easier and more performant numeric binning, now also used in Immerse for all numeric histogram visualizations
- The Vulkan backend renderer is now enabled by default. The legacy OpenGL renderer is still available as a fallback if there are blocking issues with Vulkan. You can disable the Vulkan renderer using the
renderer-use-vulkan-driver=falseconfiguration flag.- Vulkan provides improved performance, memory efficiency, and concurrency.
- You are likely to see some performance and memory footprint improvements with Vulkan in Release 5.8, most significantly in multi-GPU systems.
- Support for file path regex filter and sort order when executing the COPY FROM command.
- New ALTER SYSTEM CLEAR commands that enable clearing CPU or GPU memory from Immerse SQL Editor or any other SQL client.
- Extensive enhancements to Immerse support for parameters. Parameters can now be used in chart column selectors, chart filters, chart titles, global filters, and dashboard titles. Dashboards can have parameter widgets embedded on them, side-by-side with charts. Parameter values are visible in chart axes/labels, legends, and tooltips, and you can toggle parameter visibility.
- In Immerse Pointmap charts, you can specify which color-by attribute always render on top, which is useful for highlight anomalies in data.
- Significantly faster and more accurate "lasso" tool filters geospatial data on Immerse Pointmap charts, leveraging native geospatial intersection operations.
- Immerse 3D Pointmap chart and HTML support in text charts are available as a beta feature.
- Airplane symbol shape has been added as a built-in mark type for the Vega rendering API.
- Vega symbol and multi-GPU polygon renders have been made significantly faster.
- User-interrupt of query kernels is now on by default. Queries can be interrupted using Ctrl + C in
omnisql, or by calling the interrupt API. - Parallel executors is in public beta (set with
--num-executorsflag). - Support for APPROX_QUANTILE aggregate.
- Support for default column values when creating a table and across all append endpoints, including
COPY TO,INSERT INTO TABLE SELECT,INSERT, and binary load APIs. - Faster and more robust ability to return result sets in Apache Arrow format when queried from a remote client (i.e. non-IPC).
- More performant and robust high-cardinality group-by queries.
- ODBC driver now supports Geospatial data types.
- Custom SQL dimensions, measures, and filters can now be parameterized in Immerse, enabling more flexible and powerful scenario analysis, projections, and comparison use cases.
- New angle measure added to Pointmap and Scatter charts, allowing orientation data to be visualized with wedge and arrow icons.
- Custom SQL modal with validation and column name display now enabled across all charts in Immerse.
- Significantly faster point-in-polygon joins through a new range join hash framework.
- Approximate Median function support.
- INSERT and INSERT FROM SELECT now support specification of a subset of columns.
- Automatic metadata updates and vacuuming for optimizing space usage.
- Significantly improved OmniSciDB startup time, as well as a number of significant load and performance improvements.
- Improvements to line and polygon stroke rendering and point/symbol rendering.
- Ability to set annotations on New Combo charts for different dimension/measure combinations.
- New ‘Arrow-over-the-wire’ capability to deliver result sets in Apache Arrow format, with ~3x performance improvement over Thrift-based result set serialization.
- Support for concurrent SELECT and UPDATE/DELETE queries for single-node installations
- Initial OmniSci Render support for CPU-only query execution ("Query on CPU, render on GPU"), allowing for a wider set of deployment infrastructure choices.
- Cap metadata stored on previous states of a table by using MAX_ROLLBACK_EPOCHS, improving performance for streaming and small batch load use cases and modulating table size on disk
- Added initial compilation support for NVIDIA Ampere GPUs.
- Improved performance for UPDATE and DELETE queries.
- Improved the performance of filtered group-by queries on large-cardinality string columns**.**
- Added SQL function SAMPLE_RATIO, which takes a proportion between 0 and 1 as an input argument and filters rows to obtain a sampling of a dataset.
- Added support for exporting geo data in GeoJSON format.
- Dashboard filter functionality is expanded, and filters can be saved as views.
- You can perform bulk actions on the dashboard list.
- New UI Setting panel in Immerse for customizing charts.
- Tabbed dashboards.
- SQL Editor now handles Vega JSON requests.
- New Combo chart type in Immerse provides increased configurability and flexibility.
- Immerse chart-specific filters and quick filters add increased flexibility and speed.
- Updated Immerse Filter panel provides a Simple mode and Advanced mode for viewing and creating filters.
- On multilayer charts, layer visibility can be set by zoom level.
- Different map charts can be synced together for pan and zoom actions, regardless of data source.
- Array support for the Array type over JDBC.
- SELECT DISTINCT in UNION ALL is supported. (UNION ALL is prerelease and must be explicitly enabled.
- Support for joins on DECIMAL types.
- Performance improvements on CUDA GPUs, particularly Volta and Turing.
- NULL support for geospatial types, including in ALTER TABLE ADD COLUMN.
- Ability to perform updates and deletes on temporary tables.
- Updates to JDBC driver, including escape syntax handling for the fn keyword and added support to get table metadata.
- Notable performance improvements, particularly for join queries, projection queries with order by and/or limit, queries with scalar subqueries, and multicolumn group-by queries.
- Query interrupt capability improved to allow canceling long-running queries, also supports JDBC now.
- Database switching from within Immerse, as well as dashboard URLs that contain the database name.
- Over 50% reduction in load times for the dashboards list initial load and search.
- Cohort builder now supports count (# records) in aggregate filter.
- Improved error handling and more meaningful error messages.
- Custom logos can now be configured separately for light and dark themes.
- Logos can be configured to deep-link to a specific URL.
- Added support for UPDATE via JOIN with a subquery in the WHERE clause.
- Improved performance for multi-column GROUP BY queries, as well as single column GROUP BY queries with high cardinality. Performance improvement varies depending on data volume and available hardware, but most use cases can expect a 1.5 to 2x performance increase over OmniSciDB 5.0.
- Improved support for EXISTS and NOT EXISTS subqueries.
- Added support for LINESTRING, POLYGON, and MULTIPOLYGON in user defined functions.
- Immerse log-ins are fully sessionized and persist across page refreshes.
- New filter sets can be created through duplicating existing filter sets.
- The new filter panel in Immerse enables the ability to toggle filters on and off, and introduces Filter Sets to provide quick access to different sets of filters in one dashboard.
- Immerse now supports using global and cross-filters to interactively build cohorts of interest, and the ability to apply a cohort as a dashboard filter, either within the existing filter set or in a new filter set.
- Data Catalog, located within Data Import, is a repository of datasets that users can use to enhance existing analyses.
- To see these new features in action, please watch this video from Converge 2019, where Rachel Wang demonstrates how you can use them.
- Added support for binary dump and restore of database tables.
- Added support for compile-time registered user-defined functions in C++, and experimental support for runtime user-defined SQL functions and table functions in Python via the Remote Backend Compiler.
- Support for some forms of correlated subqueries.
- Support for update via subquery, to allow for updating a table based on calculations performed on another table.
- Multistep queries that generate large, intermediate result sets now execute up to 2.5x faster by leveraging new JIT code generator for reductions and optimized columnarization of intermediate query results.
- Frontend-rendered choropleths now support the selection of base map layers.
Last modified 1yr ago