Use the Vega data property to specify the visualization data sources by providing an array of one or more data definitions. A data definition must be an object identified by a unique name, which can be referenced in other areas of the specification. Data can be statically defined inline ("values":), can reference columns from a database table using a SQL statement ("SQL":), or can be loaded from an existing data set ("source":).
JSON format:
The data specification has the following properties:
Load discrete x- and y column values using the values database table type:
Use the sql database table type to load latitude and longitude coordinates from the tweets_data database table:
Use the source type to use the data set defined in the sql data section and perform aggregation transforms:
The name property uniquely identifies a data set, and is used for reference by other Vega properties, such as the Marks property.
The format property indicates that data preprocessing is needed before rendering the query result. If this property is not specified, data is assumed to be in row-oriented JSON format.
This property is required for Polys and Lines mark types. The property has one of two forms:
The "short form", where format is a single string, which must be either polys or lines. This form is used for all polygon rendering, and for fast ‘in-situ’ rendering of LINESTRING data.
The "long form", where format is an object containing other properties, as follows:
For lines, each row in the query corresponds to a single line.
This lines format example of interleaved data renders ten lines, all of the same length.
In this lines format example of sequential data, x only stores points corresponding to the x coordinate and y only stores points corresponding to the y coordinate. Make sure that columns only contain a single coordinate if using multiple columns in sequential layout.
The following example shows a fast "in-situ" LINESTRING format:
The following example shows a polys format:
The database table source property key-value pair specifies the location of the data and defines how the data is loaded:
Transforms process a data stream to calculate new aggregated statistic fields and derive new data streams from them. Currently, transforms are specified only as part of a source data definition. Transforms are defined as an array of specific transform types that are executed in sequential order. Each element of the array must be an object and must contain a type property. Currently, two transform types are supported: aggregate and formula.
See Tutorial: Using Transforms for more detailed examples.
If true, automatically adds rowid column(s) to the SQL statement where appropriate, enabling the data block for hit-testing using the get_result_row_for_pixel endpoint.
If false, the data block is not automatically hit-test enabled, and any later get_result_row_for_pixel calls return empty hit-test results.
If the enableHitTesting property is not present, the following legacy behavior is used as the default:
If the SQL statement represents a projection query, hit-testing is enabled if a rowid column is explicitly projected.
If the SQL statement represents an aggregate query, hit-testing is always enabled.
This legacy behavior will likely be deprecated and removed in an upcoming version of OmniSci. At that point, the enableHitTesting property will be required for activating hit-test support for the data.
Property
Data Type
Required
Description
string
X
User-assigned database table name.
string/object
How the data are parsed. polys and lines are the only supported format mark types and are for rendering purposes only. Use the single string "short form" for polygon and simple linestring renders. Use the JSON object "long form" to provide more information for rendering more complex line types.
string
Data source:
values: Embedded, static data values defined inline as JSON.
source: Name of an existing Vega data set to use as this data set’s source. Use in combination with a transform pipeline to derive new data. You can source only one existing data set.
sql: A SQL query that loads the data.
string
An array of transforms to perform on the input data. The output of the transform pipeline then becomes the value of this data set. Currently, can only be used with source data set types.
boolean
If true, automatically adds rowid column(s) to the SQL statement, which is required for hit-testing using the get_result_row_for_pixel endpoint.
Format Property
Description
type
coords
Applies to type: lines.
Specifies x and y arrays, which must both be the same size.
This permits column extraction pertaining to line rendering and place them in a rendering buffer. The coords property also dictates the ordering of points in the line.
Separate x- and y-array columns are also supported.
layout
(optional) Applies to type: lines.
Specifies how vertices are packed in the vertices column. All arrays must have the same layout:
interleaved: (default) All elements corresponding to a single vertex are ordered in adjacent pairs. For example, x0, y0, x1, y1, x2, y2.
sequential: All elements of the same axis are adjacent. For example, x0, x1, x2, y0, y1, y2.
Key
Value
Description
source
String
Data is loaded from an existing data set.
sql
SQL statement
Data is loaded using a SQL statement.
You can use extention functions to convert distance in meters from a coordinate or point to a pixel size, and determine if a coordinate or point is located within a view defined by latitude and longitude. For more information, see OmniSci SQL Extensions.
values
JSON data
Data is loaded from static, key-value pair data definitions.
Type
Description and Properties
aggregate
Performs aggregation operations on input data columns to calculate new aggregated statistic fields and derive new data streams from them. The following properties are required:
fields: An array of strings referencing columns from the sourced data table.
ops: An array of keyword strings and objects indicating the predefined operation to perform. For objects, the type property is required to name the type of the aggregation function. Supported operators:
count: The total count of data objects in the group.
countdistinct: The number of distinct values in an input data column; operates only on numeric or dictionary-encoded string columns.
distinct: An array of distinct values from an input data column; operates only on numeric or dictionary-encoded string columns.
max: The maximum field value.
mean / average / avg: The mean (average) field value.
median: The median of an input data column; operates only on numeric columns.
min: The minimum field value.
missing: The count of field values that are null or undefined.
quantile: An array of quantile separators; see https://en.wikipedia.org/wiki/Quantile. Operates only on numeric columns:
numQuantiles: The number of contiguous intervals to create; returns the separators for the intervals. The number of separators equals numQuantiles - 1. Range: 1-100. Default: 4
includeExtrema: Whether to include min and max values (extrema) in the resulting separator array. When true, the resulting array size is numQuantiles + 1. Values: true or false. Default: false
sum: The sum of field values.
stddev: The sample standard deviation of field values.
stddevp: The population standard deviation of field values.
valid: The count of field values that are not null nor undefined.
variance: The sample variance of field values.
variancep: The population variance of field values.
as: An array of strings used as output names of the operations for later reference.
formula
Evaluates a user-defined expression. The following properties are required:
expr: An expression string to be evaluated. Expressions currently support these operators and functions.
as: A string used as an output name for later reference.
Note: Currently, expressions can only be performed against outputs (as values) from prior aggregate transforms.
A Vega specification is a JSON-formatted structure that describes a visualization, which can be sent to the back end for rendering. This document introduces the the Vega specification syntax and provides links to topics that provide more details about each Vega property.
For examples of using Vega, see Tutorials. You can also see and edit examples in Try Vega.
The Vega specification includes properties for describing the source data, mapping the data to the visualization area, and visual encoding. The root Vega specification supported by OmniSci has the following JSON structure and top-level properties:
Property names are case-sensitive.
Property values are typed.
Unsupported properties are ignored by the rendering engine.
Property
Type
Description
width and height
unsigned integer
Visualization area width and height, in pixels. Both properties are required. Example: Set the viewing area width to 384 pixels and the height to 564 pixels:
array
Source data. The Vega data model uses tabular data, similar to a spreadsheet. Organized in rows with any number of named columns. JSON format:
array
Projection data. Maps longitude and latitude data to projected x and y coordinates.
JSON format:
array
Data-to-visualization area mapping. Maps visually encoded data values to pixel positions with attributes, such as color. JSON format:
array
Geometric primitive used to visually encode data. JSON format:
Marks visually encode data using geometric primitives.
General JSON format:
A Vega marks specification includes the following properties:
Each marks property is associated with the specified data property.
Marks are rendered in marks property array order.
Marks property values can be constants or as data references. You can use the scales property to transform marks property values to the visualization area.
Apply the x and y scales to the x and y database table columns to scale the data to the visualization area width and height. For example:
Marks must include a type property that specifies the geometric primitive to use to render the data.
Specify x and y coordinate values using either constants, or domain and range values of a data reference. If the from property is not specified, the x and y properties fields must be constants.
Define a point with size, color, and opacity:
Associate the points geometric primitive with tweets data items.
Specifying the data format property as lines causes the rendering engine to assume a lines database table layout and to extract line-related columns from the table.
Specify x and y coordinate values using either constants, or domain and range values of a data reference. If the from property is not specified, the x and y properties fields must be constants.
The polys type renders data as a polygon. When the data format property is polys, the rendering engine assumes a polys database table layout and extracts the poly-related columns from the table. A polys database table layout implies that the first data column is the vertex x- and y-positions. The vertices are interleaved x and y values, such that vertex[0] = vert0.x, vertex[1] = vert0.y, vertex[2] = vert1.x, and vertex[3] = vert1.y, for example. The next three positions of a polys database table are the triangulated indices, and line loop and drawing information for unpacking multiple, associated polygons that can be packed as a single data item.
The symbol marks type renders data as one of the supported shapes.
Currently, in symbol mark types, strokes are not visible beneath other marks, regardless of opacity settings.
Specify x and y coordinate values using either constants or domain and range values of a data reference. If the from property is not specified, the x and y properties fields must be specified using constant values.
symbol ExamplesThe following example defines symbol mark types including fill, stroke, and general opacity properties:
The from field specifies the input database table to use.
Example
Use the tweets database table for marks input data.
If from is not specified, the data source is implicitly a single point with the value defined in the points properties.
The properties property specifies type-dependent visual encoding that define the position and appearance of mark instances. The property value is specified using one of the Value Reference options.
Typically, a single mark instance is generated per input data element, except for polys, which uses multiple data elements to represent a line or area shape.
The following table describes the various marks properties and lists the types for which the property is valid.
A value reference describes how to specify marks properties values. The value can be a constant or data object reference:
Examples:
Statically set the point fillColor and size.
For the x marks property, apply the x scale transform to the implicit x-coordinate data column.
A field reference is either a string literal or an object. For object values, the following properties are supported:
Typically, color values are specified as a single RGB color value. To specify specific color fields or use a different color space, use one of the following color value reference formats:
Examples
Set the red and blue channels of an RGB color as constants, and uses a scale transform to determine the green channel:
Use the rgb color space for the color field:
The transform object specifies any Vega projections to be applied to the mark. Each transform is specified as a key:value pair in the transform object:
The value references an existing Vega object by name.
For example, the following transform references the projection my_mercator_projection defined in the top-level Vega projections property.
Currently, the only supported transform is projection.
Property
Data Type
Required
Description
string
X
Graphical marks type or shape:
points
lines
polys
symbol
object
Database table associated with the marks.
object
X
Visual encoding rules. Valid properties depend on marks type.
object
Transforms applied to a mark.
Marks Type
Description
points``
Render marks as points.
lines``
Render marks as lines.
polys``
Render marks as a polygon.
symbol``
Render marks as a shape.
Data Source Field
Data Type
Description
data
string
Name of the data source. The data name must be defined in the data property.
Property
Data Type
Valid Primitive Types
Description
angle
number
symbol
Amount of rotation about the defined center of rotation. The center of rotation depends on the properties that specify the symbol location:
x and y: Lower-left corner.
x and yc: Left center.
xc and y: Bottom center.
xc and yc: Center.
Must be a numerical constant or a scale that provides numerical values.
In the following example, the triangle-down symbol is rotated 30 degrees about the downward point:
angleUnit
string
symbol
Optional. Unit of measure for the rotation of a symbol around the center of rotation, defined in angle. Either degrees (default) or radians.
fillColor
color
points, polys, symbol
Fill color. Must be a scale/data reference, a string, or a color represented by a 32-bit integer or unsigned integer. See Color Value Reference.
fillOpacity
number
points, polys, symbol
The fill opacity, from transparent (0) to opaque (1). If used with opacity, the values are multiplied together to determine final opacity.
height
number
symbol
Mark height, in pixels.
lineJoin
string
line, polys, symbol
Line join method:
bevel - Extension of a line end
miter - Clipped extension of a line end
round - Semi-circle at a line end
miterLimit
number
line, polys, symbol
The miter limit at which to bevel a line join, in pixels.
Must be a positive number. Default = 10.0
opacity
number
all
The line opacity as a whole, from transparent (0) to opaque (1). If used with fillOpacity (points, polys, symbol) or strokeOpacity (lines), the values are multiplied together to determine final opacity.
shape
string
symbol
Shape name:
circle
cross
diamond
hexagon-horiz
hexagon-vert
square
triangle-down
triangle-left
triangle-right
triangle-up
wedge
size
number
points
Graphical primitive size, in pixels. Must be a scale/data reference or a number.
stroke
color
symbol
Stroke color.
strokeColor
color
line, polys
Stroke color. Must be a scale/data reference, a string, or a color represented by a 32-bit integer or unsigned integer. See Color Value Reference.
Default color = white
strokeOpacity
number
line, polys, symbol
Stroke opacity, from transparent (0) to opaque (1). If used with opacity, the values are multiplied together to determine final opacity.
strokeWidth
number
line, polys, symbol
Stroke width, in pixels. Must be a scale/data reference or a number.
width
number
symbol
Mark width, in pixels.
x
number
all
Primary x-coordinate, in pixels. Must be a scale/data reference for polys, or a scale/data reference or a number for points, lines, or symbol. See Value Reference.
x2
number
symbol
Secondary x-coordinate, in pixels. See Value Reference.
xc
number
symbol
Center x-coordinate, in pixels. Incompatible with x and x2. See Value Reference.
y
number
all
Primary y-coordinate, in pixels. Must be a scale/data reference for polys, or a scale/data reference or a number for points, lines, or symbol. See Value Reference.
y2
number
symbol
Secondary y-coordinate, in pixels. See Value Reference.
yc
number
symbol
Center y-coordinate, in pixels. Incompatible with y and y2. See Value Reference.
z
number
points, symbol
Primary depth-coordinate, in pixels. Must be a scale/data reference or a number. See Value Reference.
Name
Type Description
value
Any
Constant value. If field is specified, value is ignored.
field
Perform a lookup on the current data value. The marks from property determines the source data table and the field name must be a column defined in the data property.
scale
Name of a scale transform to apply to the mark. If the input is an object, it indicates a field value from which to dynamically look up the scale name and follows the Field Reference format.
Property
Type
Description
Property Name
FieldRef
Perform a lookup on the property name. This is the default operation when a field reference is a string.
Property Value Field
Data Type
Description
field
string
Name of the attribute from the data: sql field.
colorSpace
string
Space in which the color is defined:
Hue-Chroma-Luminance color space. See HCL color space.
Use r, g, and b property names.
Hue, saturation, and lightness color space. See HSL and HSV color space.
Use h, s, and l property names.
Lab color space. A perceptual color space with distances based on human color judgments. The L dimension represents luminance, the A dimension represents green-red opposition and the B dimension represents blue-yellow opposition. See Lab color space.
Use l, a, and b property names.
RGB color space. A version of LAB, which uses polar coordinates for the AB plane. See RGB color space.
Use h, c, and l property names.
Vega projections map longitude and latitude data to projected x and y coordinates. When working with geospatial data in OmniSci, you can use projections to define geographic points and regions.
General projections property JSON format:
When you specify a projection, you must reference it in the Marks Property using the transform object. For example, if you define the projection my_mercator_projection:
you then reference it as follows:
The projections specification has the following properties:
Use Vega projection projection alongside array columns:
The Vega scales property maps visually encoded data values to pixel positions with attributes, such as color. See the D3 scales documentation for additional background information about scales.
General scales property JSON format:
The scales specification is one or more arrays with the following properties:
Note: As a general rule, limit the total number of domain and range values used to a maximum of 1000. Exceeding this limit can cause an error.
Define two scales, x and y. For the x scale, linearly transform input data values between -100 and 999 to the visualization area width. For the y scale, linearly transform input data values between 0 and 500 to the visualization area height. The width and height range values are pre-defined literals that reference the width and height properties.
The name property uniquely identifies the scale for reference by other properties.
The type property specifies how to transform the input, domain data to output, range visual values. Vega supports the following transforms, categorized by quantitative, discrete, and discretizing scales:
The domain field specifies the domain of input data values. For quantitative data, this can take the form of a two-element array.
Specify minimum and maximum input values.
For ordinal or categorical data, the domain can be an array of valid input values.
Specify valid input data languages.
Scale range specifies the set of visual values. For numeric values, the range can take the form of a two-element array with minimum and maximum values. For ordinal or quantized data, the range can be an array of desired output values, which are mapped to elements in the specified domain.
Scale ranges can be specified in the following ways:
As an array of static values: "range": [0, 500] or "range": ['a', 'b', 'c'].
Using pre-defined literals: "range": "width" or "range": "height".
Specify a color scale that quantizes input values between 0 and 100 among five visual output colors.
Scale ranges can accept width and height string literals that map to the Width and Height Properties.
Specify a y scale that linearly maps input values between 0 and 500 to the height of the visualization area.
The default scales property specifies the output value to use when the input domain value does not map to the range.
The default property is not applicable to the threshold scale type, which maps domain values outside of the range to either the lowest or highest range value.
The accumulator property enables you to identify regional density of data in a layer of a backend render and apply pixel coloring based on the accumulation mode that you have defined. Each data point is rendered individually, providing an accurate representation of data distribution in a spatial setting.
Apply a density accumulator to a linear scale named pointcolor:
The color at a pixel is determined by normalizing per-pixel aggregated counts and using that value in the scale function to calculate a color. Normalization is performed according to the required minDensityCnt and maxDensityCnt properties. After normalization, minDensityCnt == 0 and maxDensityCnt == 1.
minDensityCnt and maxDensityCnt can have explicit integer values or use one of the following keywords to compute statistical information about per-pixel counts: min, max, -1stStdDev, -2ndStdDev, 1stStdDev, 2ndStdDev.
For more detailed examples of using accumulators, see Tutorial: Vega Accumulator.
Property
Data Type
Required
Description
name
string
X
User-assigned name of the projection.
type
string
X
Projection type. Currently supported types:
mercator: Mercator map projection.
bounds
object
Specifies the longitude and latitude bounding box for the projection. Default values:
x: [-180.0, 180.0]
y: [-85.0, 85.0]
Property Field
Data Type
Required
Description
name<code></code>
string
X
User-defined scale name.
type<code></code>
string
Scale type, which specifies the domain-to-range transform:
linear: Quantitative, continuous scale that preserves proportion among data items.
log: Quantitative scale that applies a logarithmic transform to the data.
ordinal: Discrete domain and range scale.
pow: Quantitative scale that applies an exponential transform to the input data.
quantize: Quantitative, discrete scale that divides input data into segments.
sqrt: Quantitative scale that applies an square root transform to the input data.
threshold: Discrete scale that maps arbitrary domain subsets to discrete range values.
domain<code></code>
array
Domain. Array of input interval data values.
range<code></code>
string or array
Range. Array of output interval visual data values.
default<code></code>
number
Default output value to use when domain value does not map to range value.
accumulator<code></code>
string
Accumulation rendering type:
blend: Blends colors by category. Works only for discrete output scales (ordinal, quantize, and threshold).
density: Performs count aggregation per pixel and applies the supplied color based on the normalization of the per-pixel aggregated counts over a specified range. The range is determined by the required minDensityCnt and maxDensityCnt properties. minDensityCnt and maxDensityCnt can be explicit integer values or one of the following keywords that automatically compute statistical information about the per-pixel counts:
min
max
-1stStdDev
-2ndStdDev
1stStdDev
2ndStdDev
pct: Apply a color range based on percentage accumulation for a specific category.
nullValue
number
Specify the output value to use when the input value is null.
Type
Description
Additional Information
linear
Preserves proportional differences, where range value y can be expressed as a linear function of the domain value x: y = mx + b.
log
Applies a logarithmic transform to the input domain value before the output range value is computed. The mapping to the range value y can be expressed as a logarithmic function of the domain value x: y = m log(x) + b.
As log(0) = -∞, a log scale domain must be strictly-positive or strictly-negative. The domain must not include or cross zero. A log scale with a positive domain has a well-defined behavior for positive values. A log scale with a negative domain has a well-defined behavior for negative values. For a negative domain, input and output values are implicitly multiplied by -1. The behavior of the scale is undefined if you compute a negative value for a log scale with a positive domain, and vice versa.
log scale values must be positive. Default = base 10.
pow
Applies an exponential transform to the input domain value before the output range value is computed. Range value y can be expressed as a polynomial function of the domain value x: y = mx^k + b, where k is the exponent. Power scales also support negative domain values, and input value and resulting output value are then multiplied by -1.
Default exponent = 1.
sqrt
A shorthand for power scales with an exponent of 0.5, indicating a square root transform.
sqrt scale values must be positive.
Type
Description
Resource
ordinal
Applies a discrete domain-to-range transform, and functions as a lookup table from a domain value to a range value.
Specify a default value for domain values that do not map to a range.
Type
Description
Resource
quantize
Divides input domain values into uniform segments based on the number of values in, or the cardinality of, the output range, where range value y can be expressed as a quantized linear function of the domain value x: y = m round(x) + b.
threshold
Maps arbitrary, non-uniform subsets of the domain to discrete range values. The input domain is continuous but divided into slices based on a set of domain threshold values. The range must have N+1 elements, where N is the number of domain threshold boundaries.
Value
Description
width
A spatial range that is the value of t``width``.
height
A spatial range that is the value of height. The direction of the range, top-to-bottom or bottom-to-top, is determined by to the scale type.
Mode
Description
density
Perform count aggregation per pixel and define a color for a pixel by normalizing the count and applying a color to it based on a color scale.
You can activate density accumulation for any scale that takes as input a continuous domain (linear, sqrt, pow, log, threshold scales) and outputs a color range. The range is determined by the required minDensityCnt and maxDensityCnt properties. minDensityCnt and maxDensityCnt can be explicit integer values or one of the following keywords that automatically compute statistical information about the per-pixel counts:
min
max
-1stStdDev
-2ndStdDev
1stStdDev
2ndStdDev
Note: Domain values of density accumulators must be between 0 and 1 inclusive.
blend
Blend by category (ultimately an ordinal scale). You can provide a color to a category and blend those colors to show the density of the distinct categorical values at a pixel.
pct
For a specific category, apply color based on the percentage of the category in a region.