Interactive Data Exploration with Altair

See the Altair open source project documentation to learn more and stay up to date.
The HEAVY.AI data science foundation includes the Altair visualization library. An overview of Altair from the project website:
Altair is a declarative statistical visualization library for Python, based on Vega and Vega-Lite, and the source is available on GitHub.
With Altair, you can spend more time understanding your data and its meaning. Altair’s API is simple, friendly and consistent and built on top of the powerful Vega-Lite visualization grammar. This elegant simplicity produces beautiful and effective visualizations with a minimal amount of code.
Altair and Ibis
Although Altair is typically used with smaller, local datasets, HEAVY.AI has integrated it with Ibis (and this integration itself is open-source). This combination allows interactive visualization over extremely large datasets consisting of billions of data points, all with minimal Python code.
In addition, Altair supports composable visualization, which allows for more than just local data exploration on small datasets when combined with Ibis. Because Ibis can support multiple storage backends, you can, for example, create charts that cover more than one (remote) data source at a time. 
Examples
The following examples highlight the capabilities of Altair and ibis together with HEAVY.AI.
JupyterLab version 2.0 or higher is required for the following examples.
First, install ibis-vega-transform, which in turn installs Altair and Ibis. 
1
pip install ibis-vega-transform
2
jupyter labextension install ibis-vega-transform
Copied!
Simple Example
The following minimal example of Ibis and Altair together starts with a simple pandas dataframe.
1
import altair as alt
2
import ibis
3
import ibis_vega_transform
4
import pandas as pd
5
​
6
source = pd.DataFrame({
7
    'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],
8
    'b': [28, 55, 43, 91, 81, 53, 19, 87, 52]
9
})
10
​
11
connection = ibis.pandas.connect({'source': source })
12
table = connection.table('source')
13
​
14
alt.Chart(table).mark_bar().encode(
15
    x='a',
16
    y='b'
17
)
Copied!
This produces an image like this.
You can use Altair directly with pandas, without using Ibis (see the Altair documentation). This example shows how Ibis can support pandas itself as a backend in addition to the SQL backends Ibis supports.
Adding Interactivity
Next, let's use Altair with a more scalable Ibis backend. This example uses HeavyDB, but you can try this with other Ibis backends supported via the ibis-vega-transform project that bridges Altair to Ibis.
This example connects to a public HEAVY.AI server, but you can use any HEAVY.AI server you have access to.
1
conn = ibis.heavyai.connect(
2
    host='metis.mapd.com', user='admin', password='HyperInteractive',
3
    port=443, database='heavyai', protocol= 'https'
4
)
5
​
6
t = conn.table('flights_donotmodify')
Copied!
Here is a chart definition using Altair (and Vega/Vega-Lite) interactivity to parametrize the chart. Unlike a static pandas dataframe shown earlier, this uses an Ibis expression.
1
heavyai_cli = ibis.heavyai.connect(
2
    host='metis.mapd.com', user='admin', password='HyperInteractive',
3
    port=443, database='heavyai', protocol= 'https'
4
)
5
​
6
t = conn.table('flights_donotmodify')
Copied!
Next, let's create a simple Altair chart. This chart groups the list of airlines by the number of records (i.e flights) in this dataset. Doing so should produce a bar chart like the earlier example, but the difference here, is that we're connected to an HEAVY.AI backend rather than using a local pandas dataframe. 
In the background, the ibis expression t[t.carrier_name]) is translated into a SQL query, and the results are rendered as a chart directly - no SQL knowledge required!
1
c = alt.Chart(t[t.carrier_name]).mark_bar().encode(
2
    x='carrier_name',
3
    y='count()'
4
)
Copied!
Let's create a more interesting chart beyond a simple bar chart - in this case an Altair heatmap.
1
delay_by_month = alt.Chart(t[t.flight_dayofmonth, t.flight_month, t.depdelay]).mark_rect().encode(
2
    x='flight_dayofmonth:O',
3
    y='flight_month:O',
4
    color='average(depdelay)',
5
    tooltip=['average(depdelay):Q']
6
)
7
delay_by_month
Copied!
This should create a chart like this, where hovering over the cells shows an interactive tooltip
Adding More Interactivity
Altair provides many ways to add interactivity to charts. Actions like selection and brush filters can provide more dynamic data visualizations in Altair, that allow you to explore data in a far richer manner, beyond creating static charts.
1
#The next 2 lines create a selection slider to drive a parametrized Ibis expression
2
slider = alt.binding_range(name='Month', min=1, max=12, step=1)
3
select_month = alt.selection_single(fields=['flight_month'],
4
                                    bind=slider, init={'flight_month': 1})
5
​
6
#Note how this uses an Ibis expression for the chart data source
7
alt.Chart(t[t.flight_dayofmonth, t.depdelay, t.flight_month]).mark_line().encode(
8
    x='flight_dayofmonth:O',
9
    y='average(depdelay)'
10
).add_selection(
11
    select_month
12
).transform_filter(
13
    select_month
14
)
15
​
Copied!
This creates an interactive chart that is parametrized by the slider. Moving the slider changes the selected month and updates the chart. Unlike working with a static, local dataset, you are now running SQL queries against HeavyDB each time the slide value changes. 
You can see this in the logs, in the final query generated:
1
"SELECT ""flight_dayofmonth"", avg(""depdelay"") AS average_depdelay
2
FROM (
3
  SELECT ""flight_dayofmonth"", ""depdelay"", ""flight_month""
4
  FROM flights_2008_7M
5
  WHERE ""flight_month"" = 3.0 #this is from the slider value
6
) t0
7
GROUP BY flight_dayofmonth"T
Copied!
Crossfiltering
You can build sophisticated chart combinations that combine several Altair capabilities with Ibis to create a cross-filtered visualization, like in Heavy Immerse. In this example, every data source is an Ibis expression that generates SQL queries to a HEAVY.AI backend. A total of five queries are generated and executed to create the cross-filtered visualization.
1
states = alt.selection_multi(fields=['origin_state'])
2
airlines = alt.selection_multi(fields=['carrier_name'])
3
​
4
DEBOUNCE_MS = 50
5
​
6
dates = alt.selection_interval(
7
    fields=['dep_timestamp'],
8
    encodings=['x'],
9
    on=f'[mousedown, window:mouseup] > window:mousemove!{{0, {DEBOUNCE_MS}}}',
10
    translate=f'[mousedown, window:mouseup] > window:mousemove!{{0, {DEBOUNCE_MS}}}',
11
    zoom=False
12
)
13
​
14
HEIGHT = 750
15
WIDTH = 1000
16
​
17
count_filter = alt.Chart(
18
    t[t.dep_timestamp, t.depdelay, t.origin_state, t.carrier_name],
19
    title="Selected Rows"
20
).transform_filter(
21
    airlines
22
).transform_filter(
23
    dates
24
).transform_filter(
25
    states
26
).mark_text().encode(
27
    text='count()'
28
)
29
​
30
count_total = alt.Chart(
31
    t,
32
    title="Total Rows"
33
).mark_text().encode(
34
    text='count()'
35
)
36
​
37
flights_by_state = alt.Chart(
38
    t[t.origin_state, t.carrier_name, t.dep_timestamp],
39
    title="Total Number of Flights by State"
40
).transform_filter(
41
    airlines
42
).transform_filter(
43
    dates
44
).mark_bar().encode(
45
    x='count()',
46
    y=alt.Y('origin_state', sort=alt.Sort(encoding='x', order='descending')),
47
    color=alt.condition(states, alt.ColorValue("steelblue"), alt.ColorValue("grey"))
48
).add_selection(
49
    states
50
).properties(
51
    height= 2 * HEIGHT / 3,
52
    width=WIDTH / 2
53
) + alt.Chart(
54
    t[t.origin_state, t.carrier_name, t.dep_timestamp],
55
).transform_filter(
56
    airlines
57
).transform_filter(
58
    dates
59
).mark_text(dx=20).encode(
60
    x='count()',
61
    y=alt.Y('origin_state', sort=alt.Sort(encoding='x', order='descending')),
62
    text='count()'
63
).properties(
64
    height= 2 * HEIGHT / 3.25,
65
    width=WIDTH / 2
66
)
67
​
68
carrier_delay = alt.Chart(
69
    t[t.depdelay, t.arrdelay, t.carrier_name, t.origin_state, t.dep_timestamp],
70
    title="Carrier Departure Delay by Arrival Delay (Minutes)"
71
).transform_filter(
72
    states
73
).transform_filter(
74
    dates
75
).transform_aggregate(
76
    depdelay='mean(depdelay)',
77
    arrdelay='mean(arrdelay)',
78
    groupby=["carrier_name"]
79
).mark_point(filled=True, size=200).encode(
80
    x='depdelay',
81
    y='arrdelay',
82
    color=alt.condition(airlines, alt.ColorValue("steelblue"), alt.ColorValue("grey")),
83
    tooltip=['carrier_name', 'depdelay', 'arrdelay']
84
).add_selection(
85
    airlines
86
).properties(
87
    height=2 * HEIGHT / 3.25,
88
    width=WIDTH / 2
89
) + alt.Chart(
90
    t[t.depdelay, t.arrdelay, t.carrier_name, t.origin_state, t.dep_timestamp],
91
).transform_filter(
92
    states
93
).transform_filter(
94
    dates
95
).transform_aggregate(
96
    depdelay='mean(depdelay)',
97
    arrdelay='mean(arrdelay)',
98
    groupby=["carrier_name"]
99
).mark_text().encode(
100
    x='depdelay',
101
    y='arrdelay',
102
    text='carrier_name',
103
).properties(
104
    height=2 * HEIGHT / 3.25,
105
    width=WIDTH / 2
106
)
107
​
108
time = alt.Chart(
109
    t[t.dep_timestamp, t.depdelay, t.origin_state, t.carrier_name],
110
    title='Number of Flights by Departure Time'
111
).transform_filter(
112
    'datum.dep_timestamp != null'
113
).transform_filter(
114
    airlines
115
).transform_filter(
116
    states
117
).mark_line().encode(
118
    alt.X(
119
        'yearmonthdate(dep_timestamp):T',
120
    ),
121
    alt.Y(
122
        'count():Q',
123
        scale=alt.Scale(zero=False)
124
    )
125
).add_selection(
126
    dates
127
).properties(
128
    height=HEIGHT / 3,
129
    width=WIDTH + 50
130
)
131
​
132
(
133
    (count_filter | count_total) & (flights_by_state | carrier_delay) & time
134
).configure_axis(
135
    grid=False
136
).configure_view(
137
    strokeOpacity=0
138
).properties(padding=50)
Copied!
This generates the following Altair visualization, which leverages composable charting and provides greater interactivity with enhanced selections powered by dynamic data loading via Ibis.
Geospatial Visualization
Altair and Ibis can also be used to visualize geospatial data. Altair supports multiple geospatial visualizations and can accept GeoPandas dataframes as input. Some Ibis backends, including HEAVY.AI, support spatial operations, which output to GeoPandas dataframes. By combining the two, you can create map-based visualizations.
Exploring Further
You can combine Ibis and Altair inside JupyterLab. By defining multiple Ibis backend connections with Ibis, you can create complex interactive visualizations that span multiple data sources, all without moving data into local memory. This allows greater flexibility and productivity in data exploration. 