Inclusive Data for 1.3 Billion: Designing Accessible Visualizations

Speaker 1 [00:07]

Thank you. Hi, everyone. Really nice to see so many folks interested in talking about data visualization and accessibility. I'm Pavitra. I am an Indian woman with curly black hair, neck length curly black hair, and I'm wearing a black blouse today and glasses, which is new. I don't usually wear glasses. All right. So let's start with this magic number, 1.3 billion, where does it come from? These are the number of people who experience a significant disability in the world, and to put this number in context, because everything makes sense in context, this is one in six people in the world. So that's a lot of people, right? And out of these many, folks who use screen readers have something like this, like they experience something like this when they use a screen reader for accessing a data visualization of the US election results back in 2020. So, I'm going to play this video. Image. Image. Image. Image. Image. One face. Image. Logo other. Image. So you get the point, right? Not great. So, yeah, that is why we work on all of this. Accessibility, in general, it is giving everyone equal access, which includes disabled people, and it's important to mention disabled people to include them in the accessibility conversations. And I think if you're in this room, you already care about accessibility a little bit, but if you were to pitch this to someone who maybe doesn't care as much as you do, you can use some more things. But before that, yeah, disability, I do want to just start off with a lot of different types of disability. we have physical, cognitive and learning, hearing, speech, neurological, but in this talk specifically and related to data visualization, we'll be focusing on these three, physical, sensory, and cognitive. And now we can come to, yeah, talking to your, yeah, seniors. So disability in general, we think of it as somewhat permanent, but it is not. It can also be situational and temporary, which is sometimes you can't see your screen when you're sitting in sunlight, So, high contrast websites help there. When you create for high Zoom level experiences, you can see things on your phone everywhere. So, when you speak with other folks, accessibility is important for everyone and makes things better for everyone. And it's also now a legal requirement in a lot of places. European Accessibility Act, you can look this up, comes into effect this June and requires a lot of, not just websites, but also products and services is to be made accessible. All right, so we've talked about accessibility. Now coming to data visualization. And data visualization is interesting because of just the number of users, right? Since this morning, how many of you saw a plot in a doc? Yeah, exactly. That's today. And there's so many users already. So we have a lot of different places where data visualization is used. And they can also be different types of visualizations, which adds complexity. It can be interactive and static, right? So we see it in websites, in papers, research papers. These are more images. In Jupyter notebooks, we do data analysis here. We are at a PyData conference, which is more, you can see both interactive and static, but again, a different medium of accessing the same visuals. And lastly, also in logs and monitoring reports, these are almost always presented in data visualization formats and are real time and constantly moving. So thinking of accessibility in all of these spaces, it's hard, there's a lot of work involved. So in this talk, we'll be scratching the surface, but I'm just prefacing with there's a lot more you can learn about and need to do. I don't need water, I'm just gonna try. Okay, so we've had an intro, accessibility, data visualization. Now let's talk about how we create these tools. I'm pretty, I'm assuming, how many folks here create visualizations? Yeah, that's why you're here, all right, awesome. And this next section is for all of you, right? Rather, all of us. So I'm sure you'll be using a library to create these visualizations, and we'll talk about libraries a little more later, but what can we do as just people using these libraries to create plots? There are lots of options, but something most libraries have in common is a way, like, they give you customization options. You can tweak things in your plot when you create them. So on this page, I have Bokeh, Matplotlib, Plotly, Seaborn. We call it Terran Streamlit, some of the more popular ones. But there are a lot, lot, lot more options. But again, most of them give you customization options. So let's talk about how we can start creating accessible plots. First, let's start with a plot. Before we talk about accessibility, we need to make sure it's a usable plot. So a complete plot, start with a good plot before we move on to accessibility. So now here we have information being presented to us. Oh, something I wanted to talk actually earlier before we move on. The tweet I showed you, right, with the screen reader, that is by Sarah Fossheim. They are an accessibility expert. And I've heard them say this and a lot of other experts in the field. Just by creating a visual, we are making the raw data more accessible. So just by creating plots, we're already working in accessibility. So the next steps, so we're already there. We're like, we've taken the first few steps. So this is like just making it even better. And I thought that was a nice note to start this section. All right, so let's continue. OK, this is a complete plot. A lot of folks can make sense of it, but not in grayscale. Right? So that's where we start by adding texture. This is something we can all do. most libraries should offer you some features to do this, regardless of your plot type. You can add hatching if you have areas and such. So we've added, just not using color alone to communicate information is one of the first things we can do to make things more accessible. And this also is not super optimal, because these lines are really thin. So again, folks might need to zoom in to see things. So we increase the line width. So this is already looking much better, I hope, objectively to more people here. So far, so good. However, this is, again, communicating information in only one medium, which is visual, right? We'll get to that. Just because I don't have speaker notes, I'm mirroring my screen for the audio parts. There are going to be some places I need to hop back and forth. OK, so actually, let's go back here. All of these plots are created with Bokeh. And just so that the next bit makes sense. So far, so good. Bokeh is an interactive data visualization library. So by default, it creates interactive plots. So this, let's actually go with this one. So this is the same plot, but rendered in HTML, which is interactive. And it gives us some plotting tools that lets us zoom in and out and pan to different areas. This is helpful when you have a lot of data and need to share more complex information. However, in this case, this is a static plot. I don't even have hover tips, hover tool tips. So this is actually, with interactions, a user can end up in places that don't make sense, or don't, like right now, there's nothing in this plot that's making sense. So it's best if we just not allow interactions for this particular use case. So that is what we're doing in the next slide here, where just stopping interactions, you can just save the plot, ask for help, that's it. This is the plot, the final version. Now, let's come to the most optimal version that we've created so far, but it's still visual. And that is where I want to present now. If it's only presented in a visual format, you can't make sense of it otherwise, which is where text comes in. So any visualization we create, it will be really nice if all of us started having text descriptions just as one more medium in which the same information is conveyed. But writing alternative texts, we've seen this mostly with images. like when we embed images in websites and such, it's already hard, and with data visualization, the complexity it adds, it's even harder. So I like this framework of writing alternative texts and descriptions. So this is going to be fully comprehensive, mainly for alternative text. Captions can have slightly less information, and same with descriptions, that's more adding on to the plot itself. So the next few slides heads up is going to be very text heavy, just because they're going to actually write the alternative text for a plot like this, okay? So this framework, we start by just explaining what is on the chart, like the title, axes, the bare bones. Then you go on to showing the stat itself, like what is being plotted. Then you want more complex trends, like more things you're perceiving. And lastly, contextualizing all of this, like, why do we see these trends according to you as the writer of this alternative description, right? Okay. So, let's start with this first one, elemental and encoded. We start with a multi-line plot titled bachelor's degrees in engineering and business earned by women from 1970 to 2011. That plots the number of degrees earned each year. So, that's the main plot. I'm going to be reading through all of this because this is a talk about accessibility, so if there are folks watching it, they do also need to know what's happening here. Okay. So, next. That's the plot. Number of degrees are plotted on the y-axis ranging from 0 to 50, and years are plotted on the x-axis in 10-year increments. The title already has the years, so we can just say so much. Am I going too fast? All good? Thank you. Thank you. All right, so next is the main trend being shown in this plot, which is degrees earned in business are overall higher than in engineering, and the gap increases drastically, like around 1976 to 1984, and then remains consistent, like there's just the gap, which is true. And then lastly, this is more specific trend. So in the engineering degrees line, it starts at zero, reaches 10 around 1980, then increases to around 18 by 2002, slight dip, but the number of degrees stay between 15 and 20 until 2011. Business degrees start around 10 and rapidly increase to around 46 by 1984. It then gradually decreases, sorry, reaches 50 by 2002, followed by a slight dip to 48 around 2011. So with just this text, you can still form a mental model without the plot of what this is conveying and all the important information is conveyed. One thing we've missed here is the contextual info that needs to be added on top of this. Why is there this dip in the first place? Why is there a gap? But this is sample data. That's why I haven't added it, but you would if you were creating a plot. All right, so with that, the big text-heavy portion is done. We can go back to our regulars. So this framework was helpful, I think. Did you find it helpful? Show of hands. Have you seen it before? Okay, so frameworks are helpful for that reason. They give us a clear checklist or like guidelines to follow. And but previously, with our line width and colors and stuff, we just went off of vibes. But we do have standards for everything, which is helpful. And the most important one you would have seen around is the Web Content Accessibility Guidelines, the WCAG standards used by a lot of organization and governments around the world. So they define exact contrast ratios and the kind of interactions you need to have to make all web content accessible, not just data visualization, which can be used as a really good starting point. And I'll get to specifically this in a little bit for DataWiz. And specifically to DataWiz, we have Chattability, which is another set of heuristics almost building on top of WCAG, but specifically for data visualization, like putting the WCAG concepts in a data visualization context. And folks who are familiar with WCAG, which is the Web Content Accessibility Guidelines, would have seen the first four principles here, perceivable, operable, understandable, and robust. So just for folks who are not aware, we have a lot of these rules, right, like guidelines for small, small things that you can check your plot against and improve it if it fails. They are just categorized into these categories. So perceivable would be something like make sure your website or data visualization is perceived, can be perceived by anyone regardless of their disability. So this is where color contrast comes in as well as the alternative text we saw. So you're making that plot perceivable to them. Operable would be something that they can use like assistive technology to use or interact with the plot, so it's not just one medium. All of these small things, they just fall under categories. Compromising a system and flexible are things that chartability introduces as a composite of the first four in some way. So flexible would be a perceivable and operable and robust in some senses. So it's just a more extensive list of heuristics. With that, now we move into the tooling part Because, well, all of us create visualization with tools, and we can write alt text, we can write descriptions. We probably have some ways to do it. If you are sharing an image of the plot somewhere, you can write alt text. But what if it is in a dashboard or in a logging software, which is built with another different application? The tools need to provide some way for you to even give alt text in so that it's then used on the final product or application. So that is why we work on tools. and at the tooling level, and also just thinking of if our defaults are more accessible, like the default plots you make, just a lot more plots in the world will be more accessible, right? Like regardless of if you as the creator have put in the effort to do so or not. All right. So coming to tools, this talk, for the rest of it, we're going to focus on Bokeh because we are doing some of this work for Bokeh. And Bokeh is in a special place because Bokeh is interactive data visualization library. You can write Python code, and it converts it to JavaScript and then presents it for web browsers. It's almost aimed for web browsers and interactive database for web browsers. So this is why we chose Bokeh. We can leverage things like WCAG, standards created for the web, to pull things into data visualization. All right. Sorry. How are we doing on time? Oh, awesome. I'm way ahead. This is great. All right. So, where do we... Now that we have moved on to tooling, and you saw the slide. So, we can talk about audits. But this is just a good place to start when you think about, okay, I want to make accessibility improvements at a larger scale, because otherwise you don't know where to start. Let's start by defining what the issues are. Let's find out what the issues are, prioritize them, and then we can go on to solving them. So an audit is a great place to start. We're just evaluating the current state of things and getting the issues down. And chartability comes in and is helpful, again, because there is a really helpful workbook that we can use that I'm going to actually switch to. This is the chartability. Let's the workbook. I thought I had it open. Okay, so that is the workbook, and not only does it define and illustrate how you can perform an audit, it gives us very specific tests we can do, and we're going to do a couple just together. So as we said, the tests are divided into, or categorized with those categories we saw earlier. So perceivable is, let's start with with a contrast test. Is this visible? That should be better. Okay. So, jointability suggests geometries and text size must have over 3 to 1 contrast against the background, and regular text must have 4.5 to 1. So, let's try to use this with the same tool that they suggest, the contrast checker, on the bokeh plot. I'm going to move it here. Let's look at text first, which was 4.5, like the big text. So I'm going to pull this up. Let's look at the large text. That is a pass, because it's 9.7. Let's look at some other text around here. I think most of this, because it's the same color. should be good. It's all the same. Right? Almost. Now, let's look at the geometric elements, which should have 3 to... Over 3. 3 to 1. So, I'm going to pick it again. These are the plot tools that come with all the plots. So, this fails because it's 2.45. Right? So, this is a part of all bokeh plots, and it is, at least by a standard, not as accessible. So we note this as an issue. Let's improve contrast over plot tools, and then move on to the next test. We did a perceivable one. Let's move to an operable test. Yes. Let's... Yeah. Interaction modality has one input type. So this is interactive elements. This is really good. Chartability includes not just starting plotting libraries, but also dynamic and interactive libraries, interactive plots, and tools by default. So the description suggests if a chart is interactive with a mouse, it must also be interactive with a keyboard. So you want to be able to access things using tab and arrow keys. Focusing should mirror hovering. So focusing would be a keyboard focus, which we'll look at just in a little bit. And then the chart must also be tested with a touch device and screen reader. We'll do screen reader because laptop is not a touch device, and this is going to be a bit with the phone. So let's do that. I'm going to close this one. I hope this is big enough. I'm just going to tab through. We're already in this window. The whole plot is selected. And then we immediately hop to that area. So nothing in the plot so far. and it's just going to cycle through and that's it um yeah we don't have support for keyboard i mean we knew this at in bokeh but it's like nice to have a visual like a good reference point as well so clearly not we need to work on keyboard navigation i'm also going to open my screen reader welcome to voiceover voiceover on arc bokeh plot window bokeh engineering business bokeh 3 Okay, I'm just going to move through it. Image. Engineering. Image. Business. Visited. Link. Bokeh 3. 7. Pan. Empty group. Box zoom. Empty group. So although I can get to the interactive tool, I can't engage with it like that. This is the zoom, right? With my mouse, I can click on it. I can actually zoom in and out of the plot. And this is still interactive. We didn't disable it in this specific plot. So, there are two different ways. Like, not only do I have to get to it, I need to also be able to interact with it the same way and more consistently as I would do with a mouse. This just goes on. So, I'm just actually... We'll zoom. Empty group. Save. Reset. Click the question mark to end of group. And that's it. We're out of the plot. Not as accessible. So, that is just an example of... VoiceOver off. But... Press Command-F5 to turn on VoiceOver. Now you know, on a Mac. Press command F5 to turn on voiceover. Oh, come on. All right. Thank you. Coming back, so this, we have almost, I think, 50 or so tests you can perform to get a full audit. This makes sense when you're doing a full project about accessibility, and I've sat down to do this for your tool. Of course, not for every single plot, but if you want to, go for it. But it definitely makes sense at the tooling level, because, yeah, just the number of people you impact is a lot more. And then let's go back to the slides for the rest. Okay, so, yes, a handy tool. Go check it out. And now I'm going to call out the author of Chartability, creator of Chartability, Frank, is now a collaborator, which is awesome. And we're done. Oh, that's the demos. He worked with us to do a full audit for Bokeh, which is available at bokeh-a11y-audit.readthedocs.io. So please check it out. This is, as far as we know, the only fully available audit of a data visualization library available in open source, I guess, and available for reference for more people. This is usually done at higher organizations, and the audit is, of course, behind. It's private. So yeah, really happy to share this, check it out. So as you, because Frank worked on this directly, the creator of Chartability and also an accessibility expert researcher, we have a nice reference for other tools, right? Like you can, even if you're doing it yourself, all that Chartability workbook writes down the tests, has a lot of descriptions, has examples as well of like what a good, or what a passing test will look like in any situation, but we also have how you can document it. And this summarization, this is just from the observer's perspective, as an independent observer. Next, it was our job as the Bokeh team to distill it into a roadmap and think of items that we can directly convert into Bokeh issues. Because there are some things you also can't do, because open source libraries have their own historical reasons for doing things. So we can't just do everything, but that's kind of why we need that translation layer. Okay. So, yeah, after doing that, what did we find? I'm just going to jot down, like, a few, rather share just a few high-level topics that came out. But I guess everything else will be on GitHub at some point. So let's just go through these quickly. First was design. We did see that the contrast ratios were off. They were off at, like, various places. And And this is only the default tool, but the hover interaction, the focus state interaction, all of these need to have different, not just contrast ratios, but also sizing to be accessible. So there are a lot of design changes that need to happen at the default plot level. So this involves all the spacing and interaction, how the interactions are defined, both with mouse movements, but also keyboard navigation. So overall, just working on some newish design items. Our keyboard navigation, of course. So we're gonna start by just adding foundational capability to navigate through different parts. So we definitely want to access interactive elements with just keyboard and when you tap through things or arrow key through things. The keyboard focus and tab stops need to be just complete and match what Chattability requires, and the order also is important. The way you go through a plot needs to be in a logical format. This is closely tied to assistive tech and screen readers. Here, we do want to access almost all parts of the plot, but you don't need to read through things, but there needs to be some communication back to a screen reader that these were the title, this was the title, these were the axes, what this plot is conveying. Navigation order is important here again. And lastly, any changes, especially in an interactive plot, need to be communicated back. So when you zoom, you need feedback saying, hey, zoom happened. This is now what is visible on your screen now. Some more miscellaneous things were customization. So when you have user set preferences at the browser level or at the OS level, they need to be reflected back into the plot. Accessing data. This is a big one that we just couldn't go through more, But giving access to raw data, maybe in a tabular format, is better because, rather, is good overall. Because they can use that to, there is just more tech that can, assistive tech has more features to access tables and read through tables than right now, database. The interactions need to be consistent. So this is just making sure the way interactions are done, like just with the keyboard and just with the mouse, are also aligned. We don't just want interactions. They need to be in sync and have similar interactions. Semantic elements, this is just a good one to call out. Whenever possible, especially with browser-focused data visualization, it's nice to use just raw HTML elements when you can. We can't do that in Bokeh for various reasons, because raw HTML elements also have some accessibility features, and screen readers have more support for those. So we need to enable it on the Bokeh side separately. And lastly, responsiveness. Just add. This includes reflow a little bit, making sure overall things don't break at super high zoom levels or needing to move things around and such. And this is like, I kept it at the end just because this came out as super important. Documentation, as plotting libraries, this is plotting library documentation becomes a source of truth for plot creators as well. So libraries are in a good position to share best practices with plot creators. So making sure all of our examples in your own documentation, the gallery examples are all accessible, but you also have just some specific guidelines. And especially in interactive plots like Bokeh, it was also important to think about the consumers of the plot, the readers, telling them what to expect and how to engage, especially with with all the interactive items, because there is just so much that you can do with Bokeh, like all of these plot icons. Like I showed you that I can zoom, is there something? OK. That we can zoom, but it's not super obvious immediately until you play with it. But it should be, because there are just creators on one side, but there are a lot more folks who will access the plot. All right, I think with that, I did want to just leave it on a happy note, because I've just talked about problems and problems and problems. This is the same setup for something whose tweet we saw earlier, but this is just a better plot that they found. Map of election results. Use arrow keys to move around the map. Prep Idaho results. So we can get there. It's just going to take some work. D-Trump has represented 549,126. 64% J-Biden, then 286,355. 33% last updated Wyoming result. Well, it just goes on, but yeah, we do have screener access on this particular plot from a different news site, I think. So, yeah, with that, I am going to just shout out the whole team working on this. I'm presenting this talk, but, yeah, there's a crew behind it. We have Frank, who is an author of Chartability and worked on the audit with us. We have a few folks at Quonsight Labs who are leading this work. We have someone here, Smira, sitting in the audience, who is working on this as well. We have folks at Anaconda, Philip and Mateus. They're also part of the Bokeh core team. And they don't just work on Bokeh. They work on a lot more data visualization libraries, if you've used Panel and HVPlot, the Hall-of-Wears Toolkit. And lastly, just the Bokeh team. They're awesome. Just doing all of this work is great. I'm really happy we get to do it. One more shout-out is all of this work right now. We have a two-year grant from the Chan Zuckerberg Initiative to improve accessibility on Bokeh. And that's why we are able to do all of this work can do an audit and share this with you. So yeah, it wouldn't be possible without that support. And thanks. Do we have any questions? Yeah.

Speaker 2 [29:55]

Yeah, we have a lot more time. If there are any questions, please feel free to add it in Slido. Feel free to add your questions. So yeah, we just got a couple of questions. I'll go through one by one. So the first question is, accessibility is better for everyone. Do you have figures on how much money industry spent last year to accomplish legal rules?

Speaker 1 [30:22]

I don't I yeah I don't know I'm thinking of if I know a specific source where you can find that information yeah I don't know about these specific numbers like I know they're working on it I know like Sarah Fossheim like whose tweets I showed you they work a lot with organizations or on this but I don't know the numbers sorry

Speaker 2 [30:43]

okay so the next question is so if you can't implement all or like every accommodation in your visualization due to time constraints which one is your kinds of uh should you prioritize first

Speaker 1 [30:58]

Oh, yeah. I did have some slides about prioritization. I just thought it would be too Bokeh-specific to not include it. It depends, is the short answer. So, right now, like, at least, let's talk for Bokeh and, like, you can extend it to your use case. Like, of course, like, creating that basic plot that you... As just plot creators, you can do a lot. So, like, just enabling those is, of course, the first step. At the library level, we started with foundational access, so there is foundational support for keyboard navigation and screen reader support, which is just what we did not have. So just being able to just go through all of these elements is important. And then we can work on the navigation order and accessing things. For prioritization, yeah, we start with, like, I think just the first few slides that we had, right? Like, the description you convey by text, just making sure you can also get it with the screen reader, and, like, that was the first priority. What else? Theming is also a priority. Like, just making sure any custom, just being able to change themes, like, I'll take a step back. being very Bokeh-specific because we're working on it right now, but the reason it depends is there are different... There's a lot of just technical debt and historical reasons why things were made, right? Semantic elements might be a higher priority for you, because then screen readers can directly access that semantic element, but we can't do that in Bokeh, so it's not a priority for us. So it depends a lot on just your core team and things you are able to do also skill set of folks working on it. Not everyone has the right skill set to work on all of these. So prioritization is just very tricky to do.

Speaker 2 [32:57]

okay so here like one more question so what is your opinion on the usage of example chart GPT or something to describe visualizations would this work in most simple use cases like Wikipedia statista etc

Speaker 1 [33:12]

Glad you asked. Yes, actually, I think before, say, a few weeks ago, my answer would have been no. But we've been experimenting with some things, and it's actually quite good, at least for the basic plots. Even if you just share an image, we were able to get really good text description of things. So the next step is we're experimenting with if the tool had access to the raw data, we can create even better text descriptions. like it was already doing like a fair enough job. So I think there's potential there is my answer right now.

Speaker 2 [33:48]

Great. It's not a question or something, but I'll end up with this. Great presentation. Thank you for your work in accessibility. I think it is important work for a lot of people. With that, I'll thank Pavitra on behalf of everyone. Thank you, Pavitra. Thank you.