HEAVY.AI Docs
v8.1.0
v8.1.0
  • Welcome to HEAVY.AI Documentation
  • Overview
    • Overview
    • Release Notes
  • Installation and Configuration
    • System Requirements
      • Hardware Reference
      • Software Requirements
      • Licensing
    • Installation
      • Free Version
      • Installing on Docker
        • HEAVY.AI Installation using Docker on Ubuntu
      • Installing on Ubuntu
        • HEAVY.AI Installation on Ubuntu
        • Install NVIDIA Drivers and Vulkan on Ubuntu
      • Installing on Rocky Linux / RHEL
        • HEAVY.AI Installation on RHEL
        • Install NVIDIA Drivers and Vulkan on Rocky Linux and RHEL
      • Getting Started on AWS
      • Getting Started on GCP
      • Getting Started on Azure
      • Getting Started on Kubernetes (BETA)
      • Upgrading
        • Upgrading HEAVY.AI
        • Upgrading from Omnisci to HEAVY.AI 6.0
        • CUDA Compatibility Drivers
      • Uninstalling
      • Ports
    • Services and Utilities
      • Using Services
      • Using Utilities
    • Executor Resource Manager
    • Configuration Parameters
      • Overview
      • Configuration Parameters for HeavyDB
      • Configuration Parameters for HEAVY.AI Web Server
      • Configuration Parameters for HeavyIQ
    • Security
      • Roles and Privileges
        • Column-Level Security
      • Connecting Using SAML
      • Implementing a Secure Binary Interface
      • Encrypted Credentials in Custom Applications
      • LDAP Integration
    • Distributed Configuration
  • Loading and Exporting Data
    • Supported Data Sources
      • Kafka
      • Using HeavyImmerse Data Manager
      • Importing Geospatial Data
    • Command Line
      • Loading Data with SQL
      • Exporting Data
  • SQL
    • Data Definition (DDL)
      • Datatypes
      • Users and Databases
      • Tables
      • System Tables
      • Views
      • Policies
      • Comment
    • Data Manipulation (DML)
      • SQL Capabilities
        • ALTER SESSION SET
        • ALTER SYSTEM CLEAR
        • DELETE
        • EXPLAIN
        • INSERT
        • KILL QUERY
        • LIKELY/UNLIKELY
        • SELECT
        • SHOW
        • UPDATE
        • Arrays
        • Logical Operators and Conditional and Subquery Expressions
        • Table Expression and Join Support
        • Type Casts
      • Geospatial Capabilities
        • Uber H3 Hexagonal Modeling
      • Functions and Operators
      • System Table Functions
        • generate_random_strings
        • generate_series
        • tf_compute_dwell_times
        • tf_feature_self_similarity
        • tf_feature_similarity
        • tf_geo_rasterize
        • tf_geo_rasterize_slope
        • tf_graph_shortest_path
        • tf_graph_shortest_paths_distances
        • tf_load_point_cloud
        • tf_mandelbrot*
        • tf_point_cloud_metadata
        • tf_raster_contour_lines; tf_raster_contour_polygons
        • tf_raster_graph_shortest_slope_weighted_path
        • tf_rf_prop_max_signal (Directional Antennas)
        • ts_rf_prop_max_signal (Isotropic Antennas)
        • tf_rf_prop
      • Window Functions
      • Reserved Words
      • SQL Extensions
      • HeavyIQ LLM_TRANSFORM
  • HeavyImmerse
    • Introduction to HeavyImmerse
    • Admin Portal
    • Control Panel
    • Working with Dashboards
      • Dashboard List
      • Creating a Dashboard
      • Configuring a Dashboard
      • Duplicating and Sharing Dashboards
    • Measures and Dimensions
    • Using Parameters
    • Using Filters
    • Using Cross-link
    • Chart Animation
    • Multilayer Charts
    • SQL Editor
    • Customization
    • Joins (Beta)
    • Chart Types
      • Overview
      • Bubble
      • Choropleth
      • Combo
      • Contour
      • Cross-Section
      • Gauge
      • Geo Heatmap
      • Heatmap
      • Linemap
      • Number
      • Pie
      • Pointmap
      • Scatter Plot
      • Skew-T
      • Table
      • Text Widget
      • Wind Barb
    • Deprecated Charts
      • Bar
      • Combo - Original
      • Histogram
      • Line
      • Stacked Bar
    • HeavyIQ SQL Notebook
  • HEAVYIQ Conversational Analytics
    • HeavyIQ Overview
      • HeavyIQ Guidance
  • HeavyRF
    • Introduction to HeavyRF
    • Getting Started
    • HeavyRF Table Functions
  • HeavyConnect
    • HeavyConnect Release Overview
    • Getting Started
    • Best Practices
    • Examples
    • Command Reference
    • Parquet Data Wrapper Reference
    • ODBC Data Wrapper Reference
    • Raster Data Wrapper Reference
  • HeavyML (BETA)
    • HeavyML Overview
    • Clustering Algorithms
    • Regression Algorithms
      • Linear Regression
      • Random Forest Regression
      • Decision Tree Regression
      • Gradient Boosting Tree Regression
    • Principal Components Analysis
  • Python / Data Science
    • Data Science Foundation
    • JupyterLab Installation and Configuration
    • Using HEAVY.AI with JupyterLab
    • Python User-Defined Functions (UDFs) with the Remote Backend Compiler (RBC)
      • Installation
      • Registering and Using a Function
      • User-Defined Table Functions
      • RBC UDF/UDTF Example Notebooks
      • General UDF/UDTF Tutorial Notebooks
      • RBC API Reference
    • Ibis
    • Interactive Data Exploration with Altair
    • Additional Examples
      • Forecasting with HEAVY.AI and Prophet
  • APIs and Interfaces
    • Overview
    • heavysql
    • Thrift
    • JDBC
    • ODBC
    • Vega
      • Vega Tutorials
        • Vega at a Glance
        • Getting Started with Vega
        • Getting More from Your Data
        • Creating More Advanced Charts
        • Using Polys Marks Type
        • Vega Accumulator
        • Using Transform Aggregation
        • Improving Rendering with SQL Extensions
      • Vega Reference Overview
        • data Property
        • projections Property
        • scales Property
        • marks Property
      • Migration
        • Migrating Vega Code to Dynamic Poly Rendering
      • Try Vega
    • RJDBC
    • SQuirreL SQL
    • heavyai-connector
  • Tutorials and Demos
    • Loading Data
    • Using Heavy Immerse
    • Hello World
    • Creating a Kafka Streaming Application
    • Getting Started with Open Source
    • Try Vega
  • Troubleshooting and Special Topics
    • FAQs
    • Troubleshooting
    • Vulkan Renderer
    • Optimizing
    • Known Issues and Limitations
    • Logs and Monitoring
    • Archived Release Notes
      • Release 6.x
      • Release 5.x
      • Release 4.x
      • Release 3.x
Powered by GitBook
On this page
  • Am I Using the Poly Cache?
  • Migrating Your Code
  • Example 1 - Simple projection query
Export as PDF
  1. APIs and Interfaces
  2. Vega
  3. Migration

Migrating Vega Code to Dynamic Poly Rendering

In Release 5.2, the polygon cache for rendering was deprecated and will be completely removed in a subsequent release. Any poly cache rendering in your code must be reworked to use dynamic poly rending. This topic describes how to migrate poly cache code to dynamic, cacheless rendering.

Caching poly buffers has two main drawbacks, both of which have a significant impact on memory:

  • The cache cannot span multiple GPUs.

  • The entire table is cached, regardless of the filter in the query.

In contrast, dynamic poly rendering can utilize all available GPUs and only uses the data that passes any filters.

To move to dynamic poly rendering, determine if you are using the poly cache, and then adjust your code if needed.

HEAVY.AI strongly recommends that you also remove the render-poly-cache-bytes option from your server configuration file, if used. This will help prevent startup warnings or errors in subsequent releases of HEAVY.AI.

Am I Using the Poly Cache?

It may not be immediately obvious if you are using poly cache rendering because no flag is used to enable it. Instead, poly cache rendering is enabled according to the SQL code used in a poly-formatted data block of Vega code. If the query ultimately projects or results in a POLYGON/MULTIPOLYGON column, the cache is not used and no code changes are requried.

However, if the query does not reference a POLYGON/MULTIPOLYGON column, but projects a rowid column, then poly caching is in use.

The following Vega code has a poly query that uses the cache system:

...,
"data": [ 
    {
      "name": "polys", 
      "format": "polys", 
      "sql": "SELECT rowid from zipcodes WHERE ST_XMax(heavyai_geo) > -160.0 AND ST_XMin(heavyai_geo) <
              160.0 AND ST_YMax(heavyai_geo) > -60.0 AND ST_YMin(heavyai_geo) < 60.0"
     }
   ], 
...

The sql property of the polys data block, which uses "format": "polys", projects a rowid column. This activates poly caching, even though the geo column (heavyai_geo in this case) is used in the filter.

To convert this Vega code to dynamic, cacheless rendering, change the SQL query:

"sql": "SELECT heavyai_geo from zipcodes WHERE ST_XMax(heavyai_geo) > -160.0 AND ST_XMin(heavyai_geo) < 
        160.0 AND ST_YMax(heavyai_geo) > -60.0 AND ST_YMin(heavyai_geo) < 60.0"

Alternatively, you can check the INFO logs to see if poly caching is used. Look for a LOG statement similar to the following:

Caching <number> bytes (<number> for vbo/ibo) for poly query: <sql> on gpu <number>
Poly cache on gpu <number> now using <number> of <number> bytes.

or

Cannot cache <number> bytes (<number> for vbo/ibo) for poly query: <sql> on gpu <number>. 
There is currently <number> of <number> total bytes used in the poly cache.

If you find either of these, then a poly cache render query is used.

Migrating Your Code

To migrate from poly cache to dynamic poly rendering, you output a POLYGON/MULTIPOLYGON column in the SQL query of a poly-formatted data block of your Vega code. In the following examples, the heavyai_geo column is a MULTIPOLYGON.

Example 1 - Simple projection query

Cached (only rowid is output):

..., 
"data": [
     {
       "name": "polys",
       "format": "polys",
       "sql": "SELECT rowid from zipcodes WHERE ST_XMax(heavyai_geo) > -160.0 AND ST_XMin(heavyai_geo) < 
               160.0 AND ST_YMax(heavyai_geo) > -60.0 AND ST_YMin(heavyai_geo) < 60.0"
     }
   ],
 ...

Dynamic (heavyai_geo is now projected, along with rowid):

..., 
"data": [
     {
       "name": "polys",
       "format": "polys",
       "sql": "SELECT heavyai_geo, rowid from zipcodes WHERE ST_XMax(heavyai_geo) > -160.0 AND ST_XMin(heavyai_geo) < 
               160.0 AND ST_YMax(heavyai_geo) > -60.0 AND ST_YMin(heavyai_geo) < 60.0"
     }
   ], 
...

Example 2 - Join query using a WITH subquery

Cached (no geo column is projected in the outer query of the join):

...,
"data": [ 
     {
       "name": "choropleth",    
       "format": "polys",
       "sql": "WITH colors AS (SELECT pickup_building as key0, avg(tip_amount/fare_amount) as color 
               FROM taxi_factual_closestbuilding WHERE (fare_amount > 0 AND tip_amount >= 0 AND tip_amount/fare_amount<1) 
               GROUP BY key0) SELECT nyc_buildings.rowid, colors.key0 as key0, colors.color as color 
               FROM colors, nyc_buildings WHERE (colors.key0 = nyc_buildings.bld_id) AND 
               (ST_XMax(nyc_buildings.heavyai_geo) >= -74.06993970987278 AND 
               ST_XMin(nyc_buildings.heavyai_geo) <= -73.87446838332451 AND 
               ST_YMax(nyc_buildings.heavyai_geo) >= 40.64111330399987 AND 
               ST_YMin(nyc_buildings.heavyai_geo) <= 40.87934187208219)"
     }
   ],
...

Dynamic (the geo column is projected in place of rowid):

...,
"data": [
     {
       "name": "backendChoropleth",
       "format": "polys",
       "sql": "WITH colors AS (SELECT pickup_building as key0, avg(tip_amount/fare_amount) as color 
               FROM taxi_factual_closestbuilding WHERE (fare_amount > 0 AND tip_amount >= 0 AND tip_amount/fare_amount<1) 
               GROUP BY key0) SELECT nyc_buildings.heavyai_geo as heavyai_geo, colors.key0 as key0, colors.color as color 
               FROM colors, nyc_buildings WHERE (colors.key0 = nyc_buildings.bld_id) AND 
               (ST_XMax(nyc_buildings.heavyai_geo) >= -74.06993970987278 AND 
               ST_XMin(nyc_buildings.heavyai_geo) <= -73.87446838332451 AND 
               ST_YMax(nyc_buildings.heavyai_geo) >= 40.64111330399987 AND 
               ST_YMin(nyc_buildings.heavyai_geo) <= 40.87934187208219)"
     }
   ], 
...
PreviousMigrationNextTry Vega

You can keep rowid in this query and use it for later hit-testing. It's presence does not affect dynamic rendering. See for an example.

the simple projection query in Example 1