Data Navigator demo

What is Data Navigator?

Data Navigator enables designers and developers to render a semantic, navigable structure on top of any graphic. This structure can be used by a massive variety of different input modalities.

If you want to learn more or get started, you can

Read our accessible HTML paper on Data Navigator
Check out the code repo for Data Navigator on GitHub
Go and install Data Navigator from NPM

Below is an introductory video, explaining a basic overview of what Data Navigator does, why we made it, and what we hope our system is used for:

Demo

In this demo, you can use a bunch of different input modalities to navigate the data structure of a png image of a chart.

The core 3 subsystems within Data Navigator are Structure, Input, and Rendering:

The structure is bespoke but mostly follows common patterns.
You can use a lot of different inputs!
The chart itself is just a png image. The rendered elements (when you navigate) are semantic HTML with custom visuals.

Overall instructions and commands

You will need to "enter" Data Navigator's structure before you can begin navigating.

The details of every input command are outlined below, however note that each input modality specifies these commands in different ways (see each section for more details):

View commands:

Command	Result
Enter	Enter the interface. (You must enter the interface before using any other command.)
Exit	Exit the interface.
Right	Move right (across teams).
Left	Move left (across teams).
Up	Move up (across legend items or between title/legend/axes).
Down	Move down (across legend items or between title/legend/axes).
Child	Drill into the x axis, legend, or childmost elements.
Parent	Drill out toward the x axis.
Legend	Drill out toward the legend.
Undo	Move to a previous position.

Keyboard, screen reader, touch, and mouse-drag

Show section

View keyboard and desktop screen reader controls:

Command	Expected input
Enter	Click the "Enter navigation area" button.
Exit	`ESC` key.
Right	`→` (right arrow key).
Left	`←` (left arrow key).
Up	`↑` (up arrow key).
Down	`↓` (down arrow key).
Child	`ENTER` key.
Parent	`BACKSPACE` key.
Legend	`L` key.
Undo	`.` (period) key.

View mobile screen reader, touch, and mouse-drag controls:

Command	Expected input
Enter	Long press and release on the chart area or click the "Enter navigation area" button.
Exit	Long press and release on the chart area (if you have entered already).
Right	→ Press and swipe right.
Left	← Press and swipe left.
Up	↑ Press and swipe up.
Down	↓ Press and swipe down.
Child	↘ Press and swipe down and to the right.
Parent	↖ Press and swipe up and to the left.
Legend	↗ Press and swipe up and to the right.
Undo	↙ Press and swipe up and to the right.

Text and speech

Show section

View text and speech controls:

All of the commands from the previous section (Overall Commands) can simply be typed into the text input or spoken using the button. For example:

Command	Expected input
Enter	"Enter" "enter" and "EnTeR" are all valid.

Issue a text command:

Speak a single-word command:

Gesture

Show section

The gesture control model is heavy, so we only load it if you want to try it. Click "Load Gesture Model" to load it and then activate the model with "Open Webcam" (it will need access to your camera).

View gesture controls:

Close your hand to set the gesture center (marked by an 👊 emoji on the camera space). When ready, you can open your hand or point in a direction to move. Close your hand again to set the gesture center for a new gesture (and repeat).

The model is slow and isn't very smart, so you will need to exaggerate your movement distances and hold your gesture for about a second. It fails often (apologies).

Command	Expected input
Enter	🖐 Open your hand, facing the camera.
Exit	🖐 Open your hand, facing the camera (once already entered).
Right	👉 Point right of your gesture center.
Left	👈 Point left of your gesture center.
Up	☝ Point above your gesture center.
Down	👇 Point below your gesture center.
Child	👇 👉 Point below and to the right of your gesture center.
Parent	☝ 👈 Point above and to the left of your gesture center.
Legend	☝ 👉 Point above and to the right of your gesture center.
Undo	👇 👈 Point below and to the left of your gesture center.

👊

Major trophies for some English teams. Stacked bar chart.

Why make Data Navigator?

Modern data visualization accessibility faces 3 challenges in design and development that we wanted to help practitioners and researchers tackle:

Navigable structure is hard to build for data visualizations. Structure is important for understanding and usability but is often ignored.
Only mouse input is treated well (with sporadic support for touch or screen reader input). Many other input modalities are unaddressed!
Visualizations are often rendered as semantic-less SVG or raster (pngs, canvas, etc). If semantics are added at all they end up using low-level SVG, which is often not appropriate. Semantics help understanding and add functional interactivity.

Advantages of Data Navigator

Data Navigator uses a graph-based infrastructure, comprised of nodes and edges. This allows us to do two really interesting things: create almost any other possible structure (list, hierarchy, spatial, network, etc) and prioritize direct relationships among data points. This underlying infrastructure is different from HTML's, which prioritizes hierarchies and not direct relationships.

It might seem like anarchy to design a low-level building block like this, but this is actually philosophically more empowering: designers and developers can add the structure and order that make the most sense for what they are doing, rather than try to make everything fit into a hierarchy.

Data Navigator allows designers and developers to express both rich and unique structures in ways that can handle making an entire library of charts more accessible or even take on unaddressed or bespoke visualizations.

And navigation of a structure can also be built to fit: Data Navigator abstracts navigation rules into namespaces, like "commands." Commands can be entirely customized to suit the needs of the structure they support. The advantage of this abstraction is that Data Navigator can be made to work with nearly any input modality as long as input is validated and converted into an existing command.

And lastly, Data Navigator can create an accessible rendering layer (using semantic HTML) on top of any existing visuals (or no visuals at all). This approach lets designers and developers fully control how navigation looks and feels for both screen readers and other input modalities without relying on whatever was used to render the original visualization (SVG, png, canvas, etc). This means that interactive elements in a visualization can be real "button" elements and not an SVG rectangle made from scratch to emulate a button.