Introduction to Data Navigator
Data Navigator is a library for making data structures navigable using a keyboard, screen reader, or other navigational input device or modality. This page is dedicated to documentation for how to apply Data Navigator practically. For details about what Data Navigator is, see Data Navigator's landing page. For more technical, research-oriented details (motivation, concepts, system design), see our paper on Data Navigator.
Getting started
First, let's install Data Navigator and get it into a project:
# to install dn into a project, use npm, yarn, or equivalent
npm install data-navigator
// and then you may use it in a .js or .ts file, like so:
import { default as dataNavigator } from "data-navigator";
// whole ecosystem
console.log("dataNavigator", dataNavigator);
// one module in the ecosystem
console.log("dataNavigator.rendering", dataNavigator.rendering);
<!-- you can also import as an HTML script tag (module or not): -->
<script type="module">
// pay attention to the version! the latest may be higher than this example
import dataNavigator from "https://cdn.jsdelivr.net/npm/data-navigator@2.2.0/dist/index.mjs";
console.log(dataNavigator);
</script>
Example, starting dataset
For our first example, we will be using a super simple chart that only has 4 data points, like so:
const data = [
{
fruit: "apple",
store: "a",
cost: 3
},
{
fruit: "banana",
store: "a",
cost: 0.75
},
{
fruit: "apple",
store: "b",
cost: 2.75
},
{
fruit: "banana",
store: "b",
cost: 1.25
},
]
Example, starting visualization
We're going to use BokehJS to create a bar chart of this data. You can use any method of producing a data visualization (Bokeh is not necessary). We're going to make sure our method works with whatever you choose. As long as you can get a chart rendered into HTML, whether that is a png loaded in an image, or using a visualization library, this demo should work for you.
As a note: Bokeh is a bit tricky when it comes to getting Data Navigator's
rendering
module to work correctly, but we will discuss
that in a later section and go over some options.
So here is a BokehJS stacked bar chart, using the above data:
Now, Bokeh renders charts as <canvas>
elements. That means it is just
pixels! This is perfect for us to demonstrate Data Navigator. Try to use a screen reader (or
even just tabbing with your keyboard) on that chart. It isn't helpful, is it? (Apologies to
screen reader users, as the inaccessibility of that visualization is part of the point.) As
an aside, for anyone interested in details about Bokeh and accessibility, we have published
our
comprehensive accessibility evaluation of Bokeh's ecosystem
online (this was in partnership with Quansight and Anaconda folks and part of a separate
project from Data Navigator).
Building basics
Data Navigator has 3 modularized subsystems: structure
, input
, and
rendering
. We will go over an introduction to each of these below.
Structure
Structure in Data Navigator are the bones, so to speak, of an accessible navigation experience. We want to specify which elements matter and which elements can be navigated to and from other elements. This is ultimately a graph data structure: it has nodes and edges (nodes are sometimes also referred to as vertices, while edges are sometimes called links). Structure, therefore, is about relationships between things!
So first, let's just make a list structure using Data Navigator. We will do this "manually" for now, just to demonstrate all the pieces of the library and how they work.
Nodes
First thing to do is to create a structure
variable that will hold nodes
that represent our data. Our nodes will be stored in an object, for fast lookup. For
now, we will just create 1 node per datum:
import { default as dataNavigator } from "data-navigator";
// nodes need to have the following properties: an id and
// a reference to (or copy of) the datum it represents
let structure = {
nodes: {
_0: {
id: "_0",
data: {
fruit: "apple",
store: "a",
cost: 3
},
},
_1: {
id: "_1",
data: {
fruit: "banana",
store: "a",
cost: 0.75
},
},
_2: {
id: "_2",
data: {
fruit: "apple",
store: "b",
cost: 2.75
},
},
_3: {
id: "_3",
data: {
fruit: "banana",
store: "b",
cost: 1.25
},
}
}
}
While this isn't going to be helpful yet, we can visualize our 4 nodes that don't have any edges below: (keep in mind that this visualization is not part of Data Navigator, but just used to help illustrate our structure as we go)
Edges
So for our next step, we need to create edges and add references to those edges in our nodes. Now our structure looks like the following:
import { default as dataNavigator } from "data-navigator";
// each of these nodes now have a property, edges, that hold
// an array containing strings of ids, which reference edges in the structure.
let structure = {
nodes: {
_0: {
id: "_0",
data: {
fruit: "apple",
store: "a",
cost: 3
},
edges: ["_0-_1"]
},
_1: {
id: "_1",
data: {
fruit: "banana",
store: "a",
cost: 0.75
},
edges: ["_0-_1", "_1-_2"]
},
_2: {
id: "_2",
data: {
fruit: "apple",
store: "b",
cost: 2.75
},
edges: ["_1-_2", "_2-_3"]
},
_3: {
id: "_3",
data: {
fruit: "banana",
store: "b",
cost: 1.25
},
edges: ["_2-_3"]
}
},
edges: {
"_0-_1" : {
source: "_0",
target: "_1"
},
"_1-_2" : {
source: "_1",
target: "_2"
},
"_2-_3" : {
source: "_2",
target: "_3"
}
}
}
So now we can visualize our graph with nodes and edges, as a list:
Input
Now that we have a structure, we need to be able to navigate it! So let's set a few more
pieces up to make this possible. First, we set navigationRules
in our
structure
. We will enable navigation right and left through the structure,
as well as give users a way to exit the structure (TAB is one way to exit
without using Data Navigator, but having a library-provided way via ESC is
good too).
Adding navigation rules
import { default as dataNavigator } from "data-navigator";
let structure = {
nodes: {
...
},
edges: {
...
},
navigationRules: {
left: { key: "ArrowLeft", direction: "source" }, // moves backward when pressing ArrowLeft on the keyboard
right: { key: "ArrowRight", direction: "target" }, // moves forward when pressing ArrowRight on the keyboard
exit: { key: "Escape", direction: "target" } // exits the structure when pressing Escape on the keyboard
}
};
Now that we have rules set up, we want to add our
"left"
and
"right"
rules to each existing edge.
import { default as dataNavigator } from "data-navigator";
let structure = {
nodes: {
...
},
edges: {
"_0-_1" : {
source: "_0",
target: "_1",
navigationRules: ["left", "right"]
},
"_1-_2" : {
source: "_1",
target: "_2",
navigationRules: ["left", "right"]
},
"_2-_3" : {
source: "_2",
target: "_3",
navigationRules: ["left", "right"]
}
},
navigationRules: {
left: { key: "ArrowLeft", direction: "source" }, // moves backward when pressing ArrowLeft on the keyboard
right: { key: "ArrowRight", direction: "target" }, // moves forward when pressing ArrowRight on the keyboard
exit: { key: "Escape", direction: "target" } // exits the structure when pressing Escape on the keyboard
}
}
Generic edges
Our next step is to create a new edge for nodes to call the
"exit"
rule. We will give this edge the id
"any-exit"
and add it to every node.
import { default as dataNavigator } from "data-navigator";
let structure = {
nodes: {
_0: {
id: "_0",
data: {
fruit: "apple",
store: "a",
cost: 3
},
edges: ["_0-_1", "any-exit"]
},
_1: {
id: "_1",
data: {
fruit: "banana",
store: "a",
cost: 0.75
},
edges: ["_0-_1", "_1-_2", "any-exit"]
},
_2: {
id: "_2",
data: {
fruit: "apple",
store: "b",
cost: 2.75
},
edges: ["_1-_2", "_2-_3", "any-exit"]
},
_3: {
id: "_3",
data: {
fruit: "banana",
store: "b",
cost: 1.25
},
edges: ["_2-_3", "any-exit"]
}
},
edges: {
...,
"any-exit": {
source: (_d, c) => c,
target: () => {
exit();
return "";
},
navigationRules: ["exit"]
}
}
}
Even though "any-exit"
is a single edge in our code
that belongs to every node, we will represent it in our schema graph as a disconnected
node (since navigating to it actually will exit the graph).
Our "any-exit"
edge is a
generic
edge. When applied to a node, the source
is a function, instead of an id.
The node's function will always return itself as the source id. For the
target
, it is also a function instead of an id string. The target function
will return a an empty string (anything falsey will suffice) but before it does, run an
exit()
function. The exit
function is what matters about this generic edge. We will create it below, as well.
import { default as dataNavigator } from "data-navigator";
let structure = {
nodes: {
...
},
edges: {
...,
"any-exit": {
source: (_d, c) => c,
target: () => {
exit();
return "";
},
navigationRules: ["exit"]
}
}
}
const exit = () => {
// we use our rendering api to show the exit element (which we scaffold later)
// rendering.exitElement.style.display = 'block';
input.focus(exitPoint);
previous = current;
current = null;
// we use our rendering api to remove the previous element we no longer need
// rendering.remove(previous);
}
Scaffolding our input handler
Let's finish scaffolding our input
handler. This module's primary job is to
take a location (as a node id) combined with input command (that corresponds to a
navigation rule) and convert that into a new location (a new node id). Data Navigator's
input
handles movement across the structure
. So at a bare
minimum, we want to help the user get in and out of our structure.
import { default as dataNavigator } from "data-navigator";
let structure = {
...
}
// first, we create an "entry" point (the first node navigated to)
// The below pattern navigates to whatever node is first in our structure
const entryPoint = structure.nodes[Object.keys(structure.nodes)[0]].id || structure.nodes[Object.keys(structure.nodes)[0]].nodeId;
// let's make the string we used for our exit function reference a variable instead
// we will use this below in our input variable too
const exitPoint = "exit";
// when we exit, there is no generic method to exit, so we call focus on the exit node
const exit = () => {
rendering.exitElement.style.display = 'block';
input.focus(exitPoint);
previous = current;
current = null;
rendering.remove(previous);
}
const input = dataNavigator.input({
structure,
navigationRules: structure.navigationRules,
entryPoint,
exitPoint
})
// when we enter, our input handler knows where our entry point is, so it can take us into the structure
const enter = () => {
const nextNode = input.enter();
if (nextNode) {
initiateLifecycle(nextNode);
}
};
Before we move on, there is a function I've created called
initiateLifecycle
inside our
enter
function. Let's just make an empty
function right now, and then we can finish it in the next section when we go over
rendering
.
const initiateLifecycle = nextNode => {
// contents to be added shortly
};
Additionally, our exit()
event
moves the user's focus to an element with the id
"exit"
. As of right now, we don't have an element like that in our HTML. So, now we're done
setting up our input
handling. Let's use Data Navigator's
rendering
module to actually put interactive elements in place.
Rendering
This is where the magic comes together. First, let's set up our HTML. Below is a pattern
that I like to use. There's a <div>
that holds 2 elements: 1.
currently shown, which is where we will render our visualization (the bokeh chart) and
2. eventually a <div>
we will render using our
rendering
module. The chart is placed before (aka under) the
elements we will add for accessibility, which is how they will appear on top.
Adding our HTML + CSS skeleton
<div id="dn-root-chart" class="wrapper">
<div id="chart"></div>
<!-- our Data Navigator elements will render here later, after the chart
<div id="dn-wrapper-chart" role="application" aria-label="Data navigation structure" aria-activedescendant="" class="dn-wrapper" style="width: 100%;">
<button id="dn-entry-button-chart" class="dn-entry-button">Enter navigation area</button>
<figure role="figure" id="_0" class="dn-node dn-test-class" tabindex="0" style="width: 0px; height: 0px; left: 0px; top: 0px;">
<div role="image" class="dn-node-text" aria-label="fruit: apple. store: a. cost: 3. Data point."></div>
</figure>
</div> -->
</div>
.dn-root {
position: relative;
}
.dn-wrapper {
position: absolute;
top: 0px;
left: 0px;
}
.dn-node {
position: absolute;
padding: 0px;
margin: 0px;
overflow: visible;
border: 2px solid white;
outline: #000000 solid 1px;
}
.dn-node:focus {
border: 2px solid white;
outline: #000000 solid 3px;
}
.dn-node-text {
width: 100%;
pointer-events: none;
}
Using the rendering api
Now, we can use Data Navigator's rendering function. The renderer needs
elementData
, which ideally contains information about everything that we
want to render. It is fine to pass in our nodes to this, however they will be missing
spatial information and semantics (which we cover after this). Additionally, we have to
specify what our root id is (which we just created in our HTML, above), and also specify
whether we want the renderer to create an exit element and an entry button for us.
let structure = {
...
}
const entryPoint = structure.nodes[Object.keys(structure.nodes)[0]].id || structure.nodes[Object.keys(structure.nodes)[0]].nodeId;
const exitPoint = "exit";
const exit = () => {
rendering.exitElement.style.display = 'block';
input.focus(exitPoint);
previous = current;
current = null;
rendering.remove(previous);
}
const input = dataNavigator.input({
...
})
// when we enter, our input handler knows where our entry point is, so it can take us into the structure
const enter = () => {
const nextNode = input.enter();
if (nextNode) {
initiateLifecycle(nextNode);
}
};
// this id is used by all other ids in the structure
const id = "chart";
// our renderer
const rendering = dataNavigator.rendering({
elementData: structure.nodes, // these become rendered HTML elements
defaults: {
cssClass: 'dn-test-class' // a class applied to every node, could be anything
},
suffixId: 'data-navigator-schema-' + id, // this is a suffix id added to other ids, to help uniqueness
root: {
id: 'dn-root-' + id, // this is the id given to the structure's root in our HTML
cssClass: '', // we can add a class, if we want
width: '100%', // this helps dn stretch to fit the chart already in the HTML wrapper
},
entryButton: {
include: true, // this adds a button to the UI, which conveniently enters the structure
callbacks: {
click: () => {
enter(); // our button just runs the enter function we made earlier
}
}
},
exitElement: {
include: true // we create an exit element, for convenience
}
});
rendering.initialize(); // this actually initializes our renderer
Adding semantics
We have one more step to do, in order for our rendering engine to be able to render: we need to generate data that helps us actually create DOM elements. As we mentioned earlier, Bokeh currently creates pixels, which have no information and are only made interactive to mouse input. So we need to make elements that have semantics. Let's prep our semantics, using our data:
// first, we are going to add a new import to our project
// this is imported from data navigator's utility functions
import { describeNode} from './node_modules/data-navigator/dist/src/utilities.js'
const addRenderingProperties = nodes => {
// we want to loop over all of our nodes:
Object.keys(nodes).forEach(k => {
let node = nodes[k];
// our rendering engine looks for a "renderId", we will just use our id
if (!node.renderId) {
node.renderId = node.id;
}
// this is where we add our semantics
node.semantics = {
label: describeNode(node.data, {})
};
});
};
addRenderingProperties(structure.nodes);
Building a minimalist lifecycle
With our rendering
engine set up, we can revisit our
initiateLifecycle
function and finish it. Our
lifecycle is hand-baked, which we will be using vanilla JavaScript for. While our little
lifecycle handler can
scale outrageously well
(since we will only ever render 1 element at a time), some ecosystems, such as React,
don't play nice when you use JavaScript to mess with the DOM. For a deep dive into
integrating Data Navigator into a React visualization ecosystem, check out
our contribution to Adobe's React Spectrum Charts library.
In the lifecycle, we do a few things: first, we create an element to render. Then, we
add event listeners for keydown, blur, and focus. Then we focus the new element, and
update our variables that keep track of
current
and previous
node ids. Lastly, we delete the previous
element.
// first, we create 2 variables for keeping track of things
let current = null;
let previous = null;
const initiateLifecycle = nextNode => {
// we make a node to turn into an element
const renderedNode = rendering.render({
renderId: nextNode.renderId,
datum: nextNode
});
// we add event listeners
renderedNode.addEventListener('keydown', e => {
// input has a keydown validator
const direction = input.keydownValidator(e);
if (direction) {
e.preventDefault();
move(direction); // we need to add this function still
}
});
renderedNode.addEventListener('focus', _e => {
// if we want a tooltip, this is when we would show it
});
renderedNode.addEventListener('blur', _e => {
// if we have a tooltip, we hide it here
});
// focus the new element, using the renderId for it
input.focus(nextNode.renderId);
// set state variables
previous = current;
current = nextNode.id;
// delete the old element
rendering.remove(previous);
};
Enabling movement
We have a renderer that enables us to enter our structure using the provided button and exit the structure using the escape key. Now, let's enable movement (the left and right part):
// the final piece:
const move = direction => {
const nextNode = input.move(current, direction);
if (nextNode) {
initiateLifecycle(nextNode);
}
};
Woohoo, we technically have an accessible, navigable chart! You can try this below. On the left is the chart. On the right, in our graph, we show a summary of the navigation state.
Instructions for using: (it's always good to provide interaction instructions)
Expected input | Navigation result |
---|---|
Activate "Enter navigation area" button. | Enter the visualization |
ESC key. | Exit the visualization |
← (left arrow key). | Left: Backward through data |
→ (right arrow key). | Right: Forward through data |
Trouble with focus indication
We have nearly finished our first, navigable data structure. But what is still missing? A visual focus indicator on the chart as we navigate.
Notice how our graph schema above provides a visual indication of where the focus is at, during navigation? The indicator is a thick black stroke on the node that is currently focused. Well, normally this graph schema isn't shown; we hand-baked this to help explain what Data Navigator is doing, for our docs. When using Data Navigator in the wild, there'd be no way for a user to know where they are at visually, using our code so far.
Providing a visual indication of navigation is important for accessibility. Many people who use screen readers still have some degree of sight and many people who use other navigational assistive technologies (such as a sip and puff device) also have sight. So while a screen reader user who is completely blind might not notice any downsides to the experience so far, many others will have no idea where they are currently navigating.
There is a focus indicator that appears on the bar chart, however it is small and
in the wrong location. It is a tiny black box in the upper left of our visualization
(you can see it once you enter the chart). Our final step is to put that indicator in
the right location and at the right size. We do that by setting the
spatialProperties
for our rendered node's data. An example of a perfect
render node's data might look like the following:
We are missing spatialProperties
_0: {
id: "_1",
data: {
"fruit": "banana",
"store": "a",
"cost": 0.75
},
edges: [
"_0-_1",
"_1-_2",
"any-exit"
],
renderId: "_1",
semantics: {
label: "fruit: banana. store: a. cost: 0.75. Data point."
},
// this is the important bit we are missing:
spatialProperties: {
height: 33.545448303222656,
width: 110.3,
x: 32,
y: 103.7727279663086
},
}
In our data visualization, those are the actual coordinates we would need to have in
order to show a focus indicator on the chart. The problem? This data isn't available to
us unless we export the chart as an SVG and run some code to extract coordinates from
the SVG file (which is exactly how I got those
spatialProperties
above). That won't work in practice.
But try navigating into this chart below and pressing → (right arrow) to move
to the second data point. You'll notice that a focus indicator appears where we want it,
using the correct spatialProperties
.
Making a focus indicator work for Bokeh
So what do we do? How can we fix this? Unfortunately for Bokeh, the library would have to pass up coordinates and geometry data to the frontend or enable some kind of way to query this information and generate it on-demand. As far as I know, this isn't possible.
We have a few options forward, but none are ideal. However, for a visualization this
small, we have a solution that is pretty good. What we will do is wrap our Data
Navigator element around the entire chart and then programmatically render a visible
outline on the appropriate chart elements in
canvas
using BokehJS.
The basic pattern here has one downside: if a user zooms in very far (which is common for folks who are low vision), their screen is expected to move to the location of the element that is in focus. For a small enough chart, this is fine. But if we had a full-screen data visualization (which is actually helpful for some low vision folks), then they would have no way of knowing where their focus actually is without panning and scrolling the screen with every key press. (This is considered an accessibility failure in both WCAG and Chartability.)
When it comes to our focus indicator problem (and getting element coordinates on-demand): Most data visualization libraries will not help us solve this problem. Getting a focus indicator to work correctly is quite difficult. This is why it is so important for tool-makers, like those at Bokeh, to commit to accessibility rather than expecting user-developers to make visualizations accessible. Ideally, Bokeh makes charts navigable by default. Or, at the very least, Bokeh allows an interface where element information can be accessed as-needed.
Focusing the whole chart but drawing a small indicator
So, our final step is to add spatialProperties
to our elements and then
also add a function to highlight our Bokeh chart elements when our Data Navigator
elements receive focus. Every node will always be at position 0,0 and have a height and
width that match the visualization's height and width.
import { describeNode} from './node_modules/data-navigator/dist/src/utilities.js'
// the first thing we want to do is create some variables (or use ones)
// ideally width and height are known and used to render the visualization
// however, you can also just query the element to get its current size
// or even use "100%" - since these will be css values in our renderer
const width = 300;
const height = 300;
const addRenderingProperties = nodes => {
// we want to loop over all of our nodes:
Object.keys(nodes).forEach(k => {
let node = nodes[k];
if (!node.renderId) {
node.renderId = node.id;
}
node.semantics = {
label: describeNode(node.data, {})
};
// all of our elements will start at 0,0 and be full width/height
node.spatialProperties = {
x: 0,
y: 0,
width,
height
}
});
};
addRenderingProperties(structure.nodes);
Now we just add a function that programmatically adds an additional bar to the chart when we focus, but as an outline. (This is Bokeh-specific, so keep that in mind.) First, we prep some data that just makes it more convenient to calculate what Bokeh needs in order to render outlines, then we dynamically send in our outline data to our plotting interface for our Bokeh chart. I'm sure there are more elegant ways to do this, but since BokehJS is not my area of expertise, I'm essentially deleting and redrawing the chart with a visible focus
// first, we prep some data that let's us dynamically send in what bokeh needs to draw
const interactiveData = {
data: [
[[3, 2.75],[0, 0]],
[[3.75, 4], [3, 2.75]],
],
indices: {
fruit: {
apple: 0,
banana: 1
},
store: {
a: 0,
b: 1
}
}
}
...
const initiateLifecycle = nextNode => {
const renderedNode = rendering.render({
...
});
renderedNode.addEventListener('keydown', e => {
...
});
renderedNode.addEventListener('blur', _e => {
...
});
renderedNode.addEventListener('focus', _e => {
// we use the data from our interaction, aka nextNode, to pass focus info to bokeh
const i = interactiveData.indices.fruit[nextNode.data.fruit];
const d = interactiveData.data[i]
const target = interactiveData.indices.store[nextNode.data.store];
const line_color = target ? ['none', '#000000'] : ['#000000', 'none']
document.getElementById("wrappedIndicatorChart").innerHTML = "";
plot('wrappedIndicatorChart',{
top: d[0],
bottom: d[1],
line_color
})
});
...
};
And since we are providing our own focus indicator inside the chart, we can hide the one that is visible through css.
.dn-node {
position: absolute;
padding: 0px;
margin: 0px;
overflow: visible;
border: 1px solid white;
outline: #00000000 solid 1px;
}
.dn-node:focus {
border: 1px solid white;
outline: #313131 solid 1px;
}
Deeper concepts and designs
(This section is under construction!)
Planned headings below:
Improving structure
Using the dimensions API
Tree-like structure
Intersecting tree structure
Improving input
Text input
Voice input
Gesture input
Providing instructions
Enabling re-mapping
Improving rendering
Mobile-friendly strategy
Pre-rendering vs on-demand
Visual debugging
Examples
(This section is under construction!)
More examples to follow, later.