Release 3.x
Archived release notes for release 3.x
New
- Data detection to files imported from S3
Fixed
- Multiple fixes and performance improvements to S3 Import to support MapD Cloud
- Improved handling of networked file systems, including EFS
- Fix memory leak with certain style of select query
Fixed
- Improved multi-GPU/multilayer rendering compositing
New
- New user interface for loading data to MapD from Amazon S3
Fixed
- Fixed an issue with MultiPolygon geoJSON parsing bounds (Choropleth).
New
- Enable sharding for (left-deep) outer joins
- Added bulkUpdate support for JDBC to support Pentaho
- Significant performance improvement on string comparisons from using std::sort
Fixed
- Several fixes to auto-completion
- Fix to avoid dictionary collisions in multiple databases
- export_dashboard works with spaces in dashboard name
- Multiple fixes to significantly improve accuracy and performance on APPROX COUNT DISTINCT, for large cardinalities
- Fix issue for filter pushdown into expression range calculation when filter results in empty set
- Fix broken view from changed underlying tables
- ROLLBACK on connection failure with Dictionary server
- Revised TColumn->Datum conversion to fix slow selects of array columns
New
- Supports spatial projections in Vega.
- Supports running MapD Core in CPU-only mode (with CUDA disabled) with backend rendering enabled if a GPU is available. Requires OpenGL 4.6 or higher.
- Add
opacity,fillOpacity, andstrokeOpacityproperties to marks so that opacity can be controlled independently of thefillColorandstrokeColorliterals.
New
- A new Bar/Line combo chart supports multiple measures. This is currently a BETA feature.
- You can export chart data as a CSV file for all frontend-rendered charts.
Fixed
- Filtering on timechart with histogram hangs Immerse.
- Chart dimension/measurements requirement and logo disappears when hovering.
- Text partially cut off in color Legend Item field when hovering.
- An Immerse error occurs when dashboard data takes longer to refresh than the auto-refresh time period set in the dashboard.
- Scatter plots could render two charts on top of one another.
New
- S3 Import:
COPY FROMnow supports directly loading data files (including compressed formats supported in release 3.4) from S3 buckets. KafkaImporter: MapD can now read delimited rows from Kafka topics using the efficient binary columnar load path.StreamImporter: New utility with improved streaming reads for data load, using binary columnar load path.- Outer join performance improvements.
- Autocomplete support in
mapdqlbased on newget_completion_hintsThrift call
Fixed
- Multiple issues related to race conditions causing metadata corruption during simultaneous load and query of capped collections
- Multiple JDBC-related issues, including missing checkpoints on bulk distributed loads and reporting correct JDBC versions
- Edge case of filter qual pushdown causing negative cardinality estimation
- Enabled use of hardware accelerated atomics for double-precision aggregates on Volta GPUs
- Better exception handling of arrays in shapefile metadata
- Fixed exception thrown on
DROP...IF NOT EXISTS - Better error reporting if string dictionary paths not found at startup
startmapdnow passes--configtomapd_web_server- mapdql: fix LineNoise memory leak
- mapdql: Various error handling improvements
New
- Significant performance improvements in rendering datasets with large numbers of polygons
Fixed
- Better handling of bad data in shapefiles
New
- SQL Editor now includes autocomplete and syntax highlighting.
- Scatter Plot and Geo Pointmap charts now support aggregations.
- Immerse now allows adding additional Mapbox basemap styles and third party map tiles
- Community Edition EULA is updated.
Fixed
- Skinny histogram charts when using date or time extracts.
- Immerse hangs when editing a chart, with the error message “Cannot read property timeBin of null”.
- Y-Axis has repeating numbers.
- Range chart disappears on clearing color dimension.
- Unable to add second layer in multilayer chart on Firefox.
- Unable to load chart after adding second multilayer in Geo Heatmap chart on Firefox.
SUMaggregation for measures in Geo Heatmap chart should not returnCOUNT.- Changing map style before selecting a source prevents selecting source and breaks the Dashboard button.
- Extract binning does not work on timestamp.
- Empty popups in Pointmap chart.
- Dashboards with large number of filters generate bad sql and fail to reload
New
- COPY FROM command supports input files compressed with tar, zip, rar, gzip, bzip2, and tgz.
- DATE_TRUNC and EXTRACT of YEAR, QUARTER and MONTH are 3-5x faster executing on GPU.
- Loading of dictionary-encoded data is 15-30% faster.
- “Top K” queries using LIMIT k OFFSET n with k+n < 1000 are optimized to run using k+n memory per GPU and in linear time; not yet enabled for distributed queries.
- Allow simple predicates to be used in calculation of expression ranges. This allows for faster GROUP BY when binning by timestamp or numeric expressions if a narrow filter is applied on the same column.
- Allow simple predicates with simple casts to be used for skipping fragments.
- New initialize “-i” file option is added to SQLImporter.
- Add get_completion_hints thrift call to enable friendlier SQL clients.
- Add deallocate_df thrift call to deallocate an unused data frame.
- Add thrift calls (and mapdql support) for get_tables, get_views, and get_physical_tables.
- Speed up t and d in mapdql by requesting only the required metadata.
- Add –help-advanced option to mapd_server.
- Add thrift calls to set/get table epochs by name.
Fixed
- Address issue where LDAP parameters cause mapd_web_server to fail to start by allowing passing LDAP parameters enclosed in double quotes.
- Pushdown simple predicates containing casts of TIMESTAMP and INTs into expression range and fragment skip calculations.
- Fix issues with operand nullability for time comparisons.
- Fix crash on outer cross join.
- Add thrift and mapdql commands for get_hardware_info (missing in mapdql help).
- Fix possibly nondeterministic result of left outer join.
- Sink boolean expression in a projection to its reference in the condition.
- Start distributed loads at random leaf to better balance over all nodes.
- Allow export/import of Immerse dashboards with spaces in the name.
- Change default pagesize from 1M to 2M, add version_num to table metadata.
- Recognize and use shared encoding dictionary with IN subqueries, resulting in better performance when inner and outer query use the same dictionary.
- Support DECIMAL and NUMERIC for load_table_binary_columnar; improve errors
- Fix equi-join on dictionary-encoded columns without shared dictionaries.
- Upgrade to JDBC 4.0.
- Fix processing of spaces on JDBC inserts.
- Return correct count of inserted rows in JDBC executeBatch.
- Fix JDBC DatabaseMetaData to return MapD version, set JDBC driver version.
- Fix conversion BOOLEAN to LONG for better Hive support in SQLImporter.
- Fix handling of NULL doubles in SQLImporter.
New
- Multi-thread filling poly and aggregate query render giveing a 2-4x performance improvement.
- Performance of line rendering has been improved by 25% - 50%
- Performance of polygon rendering is significantly faster for large data sets
- New command-line option “render-poly-cache-bytes” is added to MapDServer (default 300M).
- Allow threshold scale to be used with symbol shape property in Vega render spec.
Fixed
- Rendering deals more gracefully with malformed polygons. If simple triangulation fails, a proxy geom will be used in its place. It will first try using a Ramer-Douglas-Peucker simplification algorithm, next will will try to generate a convex hull, and throw an error only if this also fails.
- Addresses issue when the only difference between successive render_vega queries is a change between projection and aggregation.
- Address issue where Rendering a poly table for the first time with a filtered query crashes server.
- Mostly address GL_FRAMEBUFFER_UNSUPPORTED error seen on some older Cuda drivers on Centos.
- Fix issue when a multi-GPU non-accumulation render is followed by a single-GPU accumulation render.
- Always cache non-in-situ renders for get_result_row_for_pixel hit testing.
New
- Dashboards can reference multiple data sources.
- Geo charts can incorporate multiple layers.
- Dashboards can be configured to auto-refresh on a set time interval.
- New menu items in Community Edition have been added under Help to request a trial license and manage trial license keys.
- Four new Base Map Themes have been added: Odyssey, Vintage, Decimal, and North Star.
- Shape selection now is allowed in point maps and scatterplot charts.
- A new Top Level menu - Data Manager - has been added.
Fixed
- Immerse now allows highlighting and copying a field in a table chart without inadvertently applying a filter.
- Text Widget can now save text on Internet Explorer 11.
New
- Add initialize file option to SQLImporter
- Add thrift apis and mapdql comands for reporting on role/user privileges
Fixed
- Fix handling of NULL in loading doubles with SQLImporter
- Add export/import dashboard commands to mapdql help
- Fix precision for output of decimal to file
- Fixed nulls first/last sorting for distributed
- Rollback failed table copy from
- Reduce checkpoints to once per load, locking table to limit to one concurrent update
- Apply reverse distribution to logical arguments
- Add optional error percentage argument to APPROX_COUNT_DISTINCT
- Move SQL importer to binary columnar format for better performance
- Upgrade SQL parser to Calcite 1.14
Dependencies
- Now requiring Java Runtime Environment (JRE) version 1.8 or higher
Fixed
- Fix to reduce GL_FRAMEBUFFER_UNSUPPORTED when running older CUDA drivers on CentOS
- Fix to render thread shutdown
- Fix to error code on framebuffer error
- Address mark property update issue when the only thing changing between render calls is sql and render property
- Add multijoin and left deep inner join to table scan for rendering
New
- Add nulls sort first/last to table sort
Fixed
- Fix for exception raised when switching from geoheatmap to other chart type
- Fix to row and pie chart to set minimum number of groups to one instead of zero
- Allow SQL editor to accept SQL statements starting using WITH table expression
New
- Improve support for left-deep joins to allow joins of large numbers of tables
- Automatically reorder tables in a query by row count
- Allow encoding dictionaries to be shared among multiple columns in the same table
- Add new mapdql commands to export/import Immerse dashboard definitions
- Remove the limit on the number of simultaneous connections
Fixed
- Allow object level privileges against multiple databases, still under trial flag
- Throw an error if two operands of IN operator don’t have matching types
- Allow empty IN operands
- Add nested loop code to code generation
- Initial pass at internal catalog functions for viewing object privileges
- Fix jdbc to not ignore substitution parameter at the end of a query
- Add at_most_n parameter to thrift sql_execute endpoint
- Fix to join of a column against an expression
- Fix for possible CPU heap corruption after memory starvation
- Improve error messages on file operation failure
New
- Run all render commands on a dedicated thread, removing connection limit
- Add line mark rendering type
New
- Additional GTM tagging has been included
- Improved error reporting on failure to load a dashboard
- User logout moved to dropdown by navigation bar, count widget moved under dashboard header
Fixed
- In SQL Editor, fixed issue with scrolling
New
- Add Thrift call load_binary_table_arrow for Arrow columnar import
Fixed
- Fix loading of DECIMAL string with leading decimal point
- Create object security tables if feature is enabled
- Fix to checkpoint creation on epoch-based table rollback
- Read fragment metadata in parallel
- Parallelize cold start reload of string dictionary
- Fix table rollback by epoch
- Fix DECIMAL import for values between -1 and 0
New
- Support rendering of aggregate and projection queries for distributed configuration
- Enable rendering where number of render GPU’s is less than query execution GPUs
Fixed
- Fix logging of rendering calls
New
Fixed
- Fixed an issue where a line chart filter could not be removed
- Fixed an issue with old crossfilter instance preventing opening a new dashboard
- Fixed an issue where exceptions/errors prevented histogram and line charts to reload
- Fixed an issue with multi-series line chart filter icon not disappearing
- Fixed an issue creating a numerical binned multi-series line chart
- Fixed an issue where reverting edits on a line chart resulted in error
New
- Add support for object privileges, not enabled by default
- Add Thrift call for load_table_binary_columnar
- Update EULA text
Fixed
- Add basic lock protection for drop and truncate table
- Fix potential incorrect result for filtered outer join queries
- Fix to improve precision in decimal arithmetic
- Optimize queries with combinations of equality and null
- Fix possible crash on sort of empty result
- Fix to bulk import related to decimal scale
- Round when reducing decimal scale on import
- Write temporary install files to temp directory
- Save sample data sets to temp directory during install to reduce required privs
- Fix to command line options while inserting sample data
- Fix to limit pushdown with offset
- Allow case insensitive boolean literals on import
- Fix range checking for fixed width encoded strings
- Fix possible segfault on outer non-equijoin on string
- Fix for distributed query with grouped sort and offset
- Fix for null handling in top k queries on a single-fragment table
- Fix for self-join query
- Fix for possible error on query with empty ranges of type double
- Fix join on fixed encoding dictionary columns with nulls
- Enable HA in build of MapD Enterprise Edition
Fixed
- Fix for multi-layer rendering in distributed configurations
- Improve support for ordinal scales when domain count exceeds range count
- Fix for floating point issues with heat map hex binning
- Fix for multi-layer rendering with 1 or more accumulation layers
New
- Update EULA text
- Added Google Tag Manager for product usage tracking
New
- Equi-joins between expressions and columns are now supported
- Joins on time and date are now supported
- Non-grouped projected top-k queries of form
order by col limit nare now executed more efficiently - Performance improvement for grouped top-k queries on sharded tables
Fixed
- Fix for projection query with order by a float column not used in a filter
- Fixes for conditions of potential arithmetic overflow
- Fix for segfault on natural join, or join explicitly including rowid
- Gracefully return exception on scalar subquery returning text
- Fix for failure to execute join query against an empty table
- Fix for failure to execute a query with parenthesis around ilike condition
- Fix to enable some cases of reordering joined tables improve performance
New
- Symbols now work with multi-layer rendering
- Add support for GPU rendering with EGL on IBM Power
Fixed
- Fix blend accumulator type mismatch bug
- Issue resolved for potential crash with geo-heatmap render
- Various vega symbol update issues
New
- New chart type : Text Widget
- Alias for custom measures and custom dimensions
- Geo heatmap (available on single node only)
Fixed
- Range chart is now OFF by default
- “Back to Dashboard” link is now aligned on browser resize
New
- allow group by floating point column, even if watchdog is enabled
- allow one-to-many join on multiple columns
Fixed
- fix for overflow detection for fixed encoding dictionary entry creation
- disallow multi-node export, for now
- fix for subquery with a CASE statement inside a derived table returning a non-grouped column
- explicitly disallow LIMIT 0
- fix for query with sort on a dictionary encoded field containing null values
- add progressive backoff to mapdql connect retries and limit retries
New
- enable multi-sampling for smoother rendering
Fixed
- Use correct cuda context index during EGL compositing
- Fixing active gl context bug during distributed rendering
New
- Allow switching between donut and pie chart styles for pie chart
- Top level Help has one new menu item for Immerse tutorials
- Declining EULA in community edition disconnects user
- Default aggregation on time unit measures is MAX
Fixed
- Fixed spinner on incorrect custom dimension expression
- Fix for switching series dimension in a line chart or histogram to reflect new dimension
- Number chart no longer exposes show range chart toggle
- Fixed isue with histogram range bar width
- Multi-series range chart colors are no longer reset when range chart is toggled off/on
New
- Tables now can be sharded by a numeric or dictionary encoded column
- Dictionaries for encoded columns can be shared across multiple tables
- Multi-column join is allowed provided combination of columns yields a unique join key
- Tables can be joined on columns of type BIGINT, provided the column yields a unique join key
- Join now allows the one-to-many case, with uniqueness in the outer table
- TRUNCATE TABLE command is now supported
- MapD startup has an optional parameter for maximum error for APPROX_COUNT_DISTINCT
- Data import via COPY now reports on specific location of bad data
- COPY TO now supports relative path names
- MapD Server now defaults to ignore divide by zero
Fixed
- Max string length of 32767 is now enforced
- Trailing semicolon is automatically removed from SQL or SQL inside a view
- Incorrect result fixed for selecting with a filter from a join view
- Incorrect result fixed with LEFT JOIN resulting in a cross join
- Fix issue with precision of APPROX_COUNT_DISTINCT when running on CPU
- Fix issue with join on empty inner table
- Fix issue with database.table syntax
- Fix for possible race condition in distributed queries causing query hang
New
- Improved anti-aliasing using multi-sampling, most noticable stroking polys or symbols < 2 pixels wide
- Allow dictionary-encoded string columns from geo polygon tables to be used as domains in ordinal scales
- Added set of new extension functions as helpers for hexagonal headmap binning and rendering
- Symbols now can be used in accumulation rendering
Fixed
- Symbols that are both filled and stroked now render in the appropriate oder
- Fix issue when using values from a dictionary encoded string column in an ordinal scale when rendering a GROUP BY query
- Fix for crash when trying to render a GROUP BY query also sorted with ORDER BY
New
- Top level Help has one additional menu item that displays core version, immerse version and whether backend rendering is enabled.
- Dimension lock axes on histogram and line charts.
- Range charts for histograms
- Stacked histogram charts
Fixed
- Issue with line chart date fields don’t reflect filter dates after clearing filter
- Issue where brushing on initialized chart (Range/Line/Histogram) will extend brush filter beyond what is selected
- Issue where multiple range charts and focus charts created when changing chart type
- Issue where range chart with multi-series changes color of range chart when brushing range
- Issue with Time entry box for line chart not applying time filter
- Bug where Line chart does not cancel changes when switching from binning to extract and clicking cancel
- Multi-series line chart loses color when going to and from SQL editor
- Uncaught TypeError - Cannot read property ‘dimensions’ of undefined when creating a line chart
New
- Improved support for
INTERVALdata type, and added functions forTIMESTAMPADD,TIMESTAMPDIFF,DATEADD,DATEDIFFandDATEPART. For example the following two queries can now be run, which would give equivalent results:
SELECT TIMESTAMP '2016-02-28 1:23:45' + INTERVAL '24' HOUR
SELECT TIMESTAMPADD(HOUR, 24, TIMESTAMP '2016-02-28 1:23:45')
Fixed
- NULL now does not match other values (e.g. another NULL) in hash joins
- Crash in cpu mode when a hash join inner table column is all NULLs
- Crash when
LIKEorREGEXP_LIKEpattern is not a literal - Problem with loading through aggregator in multi-node setups when table contains non-encoded strings or arrays
Fixed
- Bug when using dict-encoded string column as color when rendering a
GROUP BYquery - Crash when using a heatmap query with a
LIMIT
New
- SQL Editor Reference panel for tables and columns
Fixed
- Issue with Number chart not removing old charts when changing aggregation type
New
- Division by zero behavior is now configurable via flag
null-div-by-zero. New installs usingsystemdwill default totruein their config files, meaning that division by zero returnsnull; existing installs will default tofalse, preserving the existing behavior of throwing adivision by zeroexception. round_to_digit(x,y)function now supported. Roundsxtoydigits.- Dictionary encoded columns may now be specified for the
sql_execute_gpudfendpoint - Fast versions of statistics functions are now available. Note that these functions might overflow; use single precision float instead of double precision.
STDDEV_FLOAT,STDDEV_POP_FLOAT,STDDEV_SAMP_FLOAT,VARIANCE_FLOAT,VAR_POP_FLOAT,VAR_SAMP_FLOAT,CORRELATION_FLOAT,CORR_FLOAT,COVAR_POP_FLOAT,COVAR_SAMP_FLOAT - Avoid overflows for most literal arithmetic expressions
Fixed
- Crash with certain join queries when inner table is empty
- Incorrect results on order by non-encoded string with filter applied on that column
- Incorrect detection of overflow when multiplying decimals with negative numbers
- Dictionary folders are now dropped in multi-node setups when tables are dropped
SUMandAVGoverflows for tables with more than 4B rows- Crash on grouped
COUNT(DISTINCT)in multi-node setups - Crash on join query when column contains all nulls
New
- Vega API now supports ability to render non-projection (
GROUP BY) queries - Added new ExtensionFunction
rect_pixel_binas a helper for rectangular heatmap binning - New
symbolvega mark type. Supports circle, square, diamond, cross, triangles, and hexagons - Added support for a new percent accumulation type, which allows a region to be colored based on one category’s percent of all results at that location.
Fixed
- Fix vega update issue when mark property references new data and nothing else changes.
New
- Added Custom SQL Dimension feature, allowing SQL to be used to form dimensions
STDDEVnow available in UI as an aggregate type- Immerse now displays End User License Agreement when product is accessed for first time from Community Edition
Fixed
- Issue with pointmap and scatterplot charts, where chart image is larger than the chart widget size
- Issue with widgets shifting on dashboard after changing browser window size then restoring to original size
- Issue when using custom measure on Scatterplot chart
- Values between -1 and 1 now display in a typical decimal format rather than with “m” abbreviation
- Incorrect behavior for points auto-sizing and manual sizing (pointmap and scatterplot)
New
- Multi-column hash joins now work provided the dimension table has unique composite keys
- Loop join is now automatically allowed for inner tables with up to 1,000 rows
- Improved performance for
LIKE/ILIKEoperations on dictionary-encoded columns - Improved performance for
COUNT(DISTINCT)andAPPROX_COUNT_DISTINCTfor low cardinalities - Enhanced SQLimporter compatibility and performance with PostgreSQL
- StreamInserter now offers
—-quotedflag which, when set to true, removes any double-quotes in the output
Fixed
- Due to an issue with the current release of glibc on many operating systems, we have moved away from a JNI interface to Calcite to avoid segfaults
- Better support for batch JDBC insert (statements are no longer case sensitive)
- Fix for possible race condition when using popups on a rendered chart in a distributed system
- Import improvements around column detection and reject row summary
- Improved error messages around user management commands
- Eliminated overflow when comparing integers with high-precision decimals
- Now recording rejected rows properly in case of Overflow
- Now dropping string dictionaries for
DROP TABLEin distributed case - Issue with
AVG(float)when number of records is >2B
- No notes this release
New
- Chart y-axis height can now be capped using a control on the axis, for example to exclude outliers from view. Data is not filtered from the dataset, but merely excluded from view.
- X and Y axes for dc charts can now have their labels edited, allowing for friendly names
- SQL editor beta now offers a command-line style interface with query history
Fixed
- No longer incorrectly showing non-encoded strings as available Dimension columns
This point release addresses an issue introduced in recent version of glibc. JNI fails with a segfault after the operating system had been updated. This release fixes that issue by removing the JNI usage.
MapD Core for single-node usage has been open sourced under the Apache 2.0 license (https://github.com/mapd/mapd-core/blob/master/LICENSE.md). Code available at https://github.com/mapd/mapd-core . EULA at https://www.mapd.com/legal/
New
- Initial API support for GOAI consortium (http://gpuopenanalytics.com/). New Thrift API endpoint (
sql_execute_gpudf) creates a query resultset on the GPU in Arrow format and returns a handle. Other GPU kernels can work directly on the data via the returned handle, reducing the data movement normally associated with staged operations. - Improved initial GPU memory load times by more fully utilizing available PCIE bandwidth
- Now supporting
COVAR_POP,COVAR_SAMP,CORR - Performance for certain complex subqueries, for large
INsubqueries for multi-node environments, and forMINon a nullable float or int - Float performance improvement on
AVG/SUM/MIN/MAXaggregates - Multi-way
JOINqueries now use memory more efficiently
Fixed
- Issue where sorting
GROUP BYandAVGthrough view of aJOINquery returns results in non-sorted order - Problem where in some cases
--allow-cpu-retryflag would not fall back to CPU - Now correctly returning list of columns for views
- Crash on
CREATE TABLE AS SELECTwhenSELECTstatement returns no rows - Incorrect results when doing
MAXon a float in some cases - Issue with accuracy of results with drawn polygon filters
AVGandSUMon float now supported on Kepler GPUs- Allow execution on GPUs for queries larger than can fit in GPU memory when
--enable-watchdogflag is set to false - Further overflow detection for decimal type
- Improved precision of numeric on initial load
- Reduce data transfer to GPU memory when not required
COUNT(DISTINCT)on allNULLnow returns correct (empty) result
New
- Default rendering now done through EGL. No longer dependent on Xorg
- Multi-layered points/polygons can now be rendered in multi-node environments
- Now using blosc/zstd compression for passing image data between nodes
- Accumulation rendering now supported in multi-node environments
Fixed
- Incorrect darkening of color for images with transparent colors composited on multiple GPUs/nodes
- Issue with re-initializing counts for accumulation rendering when rendering a new frame
- Issue with layers incorrectly persisting from one frame to another when number of layers changes from frame to frame
- Problem with EGL compositing in multi-node environments
- Error on accumulation rendering when using
POWscale - Errors with removal of polygons from cache
- Bugs when rendering polygons against empty query results
- Issue when coloring by dictionary encoded string when string is in inner table of a joined query
- Accumulation rendering crash when data from query doesn’t span the compositor’s gpu
- Now not throwing error when fill/stroke is turned off between successive render frames
- Fixed shader compile issue when updating the data reference for a packed color vega property