Dash: Interactive Data Visualization Web Apps with no Javascript

Speaker 1 [00:02]

So yes, thank you very much. As I said, my name is Dom Weldon. I'm going to be talking about DASH, or Interactive Data Visualizations Without JavaScript, or kind of an alternative title for this presentation I think would be, so what you can, can't, should, and also probably shouldn't do with DASH, which is a library from Plotly. And so over the course of this talk I want to kind of explore a little bit about why you might want to use Dash and then kind of give a very basic introduction to it, not an in-depth sort of deep dive tutorial, because obviously that's what the documentation is for. But I do want to also provide some introductory examples and then discuss some of the things that we've learned at my company, Decision Lab, from using Dash across different teams. And also, if you're interested in some of the points that I kind of talk about later on and want to get more involved in Dash or have any kind of advice or interesting points to discuss, please do come and approach me. I'll be at the conference today and also at the drinks this evening. I should also, just before we continue, add a very quick disclaimer. That disclaimer is that I'm not an expert on the Dash project. I'm not an author in this particular library or anything like that. This really is a talk about my personal experience and the experience of my company using Dash rather than a sales pitch for it from the open source community. You might ask, if I'm not an expert, why am I here? Also, more to the point, my background is as a full-stack developer. I'm a JavaScript developer, I work in Python, and for my sins I used to work in PHP as well. It's a bit weird to be using Python to write JavaScript. And so, I want to explain how it is that I came to be using Dash on a regular basis at work. And so, I work for this company, Decision Lab. We're a mathematical modeling consultancy based in central London in the UK. And I should just say very quickly before we carry on, if you are following the latest in the Brexit debacle, I'm sorry. I haven't got a clue what's going on either, and it really is not good. But it's lovely. I should also want to say thank you to the organizers of this conference and to everyone who's come here from all over Europe. Events like this across different countries and within the whole of Europe are really quite special. And so more of this, less of that, please. Anyway, so Decision Lab, we are, as I said, a mathematical modeling consultancy. We have expertise in machine learning, data science, optimization, and simulation. We work mostly in defense and engineering, intelligence, public bodies, various other kind of different markets, and we're a small company and our primary interest is in unusual bits of data, interesting problems, and things that we haven't necessarily done before. So one of our kind of main selling points as well is that we don't just take someone's data and write a report. We build tools using data. And some of these tools, for example, this was a project recently where we We took satellite imagery and then analyzed it to look for deforestation to detect illegal gold mining in the Amazonian rainforest, providing tools to the Colombian government. And at the moment, we're also deploying on board these things here. This is our deployment environment for AWS, for Azure. We're deploying on a Type 45 destroyer for the Royal Navy with a predictive maintenance system based on neural network machine learning. So obviously we're a company of data scientists and people like myself, software engineers as well. But one of the problems or things that we notice is definitely this kind of two cultures to Python. And I'm sure, in fact, it's evident in this conference today. This is a joint conference between PyCon Germany and PyData Berlin. You can see there's a sort of like data scientist sort of ecosystem in Python. And there's also the kind of software engineer sort of ecosystem, maybe focusing on the web or sort of MicroPython and other kind of frameworks. And so it's not quite as bad as the kind of C.S. Snow essay with the two cultures and the arts and humanities back in the 1950s and so on. But there is definitely, in my opinion, that kind of, you know, there's sort of two cultures and two sort of backgrounds to your typical person working in Python. And they have different priorities and different approaches, say. And it's really important that these two communities collaborate with one another. I mean, hence why everyone's here today, right? But generally, if you're working to a deadline and you're on something that might be commercially sensitive as well, you actually kind of want to minimize often the number of new technologies that, say, a data scientist has to learn, or this might be someone who's just recently joined your company or just fresh out of their PhD or something. You want to minimize the new non-data technologies that they need to learn in order to be able to take a project on. Because I really want my data scientists to be taking a lead, developing a proof of concept or an alpha, talking to a client, and doing interesting things with data. But their background probably isn't in web development, and it's probably not really what excites them or what they come into work for in the morning. And so that's how we discovered Dash. And this is the project by Plotly. And the selling point is that this is a team of data scientists or people who make data science tools that allow you to build websites using pure Python and not having to learn JavaScript. It's an open source project. There are some paid consultancy options as well that you can use. And it recently, it's quite a young project, but it did recently hit version one, I think, in June, which was a big deal in the Dash community. The selling point of Dash, like I said, is that it allows you to build an interactive website without having to use JavaScript. That's a good thing sometimes, but not always. In the rest of this talk, what I want to go over is just explain how Dash works fundamentally, what it does, give some examples of what you can do really quickly with Dash, and also discuss some of the, like I said before, the tips and good practices that we've learned as a company. But I'll also cover this point here, which is when you want to stop using Dash and build a more involved or like a proper sort of website and some of the kind of considerations that you might want to think about when you're deciding to start a project in Dash. So first things first, pip install Dash. It's very, very straightforward. You can simply install it via pip and then use it as a standard kind of Python script. A hello world example is up here. So if we have a look here, we're importing these two libraries, dash and dash HTML components, which I'll discuss in a sec. You create an app just like you might do if you're familiar with Flask. And actually under the hood, dash here is creating a Flask application for you. You define a layout. And so here we've got a very simple layout, just one div or division tag. And then we've got a heading tag in there as well, which is just going to have some text in it. And then we literally just run that script or run that server. And now this is where I have to think across different web browsers, I'm afraid. But if you bear with me, you can see very, very simply that we have created a Dash application that's the equivalent of our Hello World. And that is actually generated using that code. and I've put a little logo on there using some fancy CSS because there's a default that will look for a CSS file for you by default and we've got it running up on my laptop now and actually there is a Github repository on my Github so just go to github.weldon and you can find these repositories that have all of the code examples that I'm using today plus some more and you can run them in a little docker container and it just opens up all the different examples on different ports. So that's our hello world. And sorry, I'm going to have to squeeze this over here and then, all right. And so that's our hello world, but that's not an interactive web page, is it? But we'll come to exactly how to make it interactive in just a moment. Under the hood or what Dash is doing for us is basically just two libraries that talk to one another. And so one of those libraries is this dash HTML components, which we imported up at the top before. And that is just a wrapper around lots of different React components, which we'll come to in a second. So this is just a Python wrapper around a React component, which itself is wrapping an HTML tag. And so this library allows us to kind of define HTML layouts using just Python. And then there's the actual Dash application, which manages the relationships between these particular components, and it serves the application for us locally or on our server, wherever we want using Flask. Like I said before, though, the hello world script that just displays some HTML for us isn't particularly interesting. We want to have interactive web pages that allow our users to do something a bit beyond reading just some text. And so before I explain how this works and show an example in Dash, I thought it'd probably be quite useful, especially for people who might not have a background in web development, to explain how this would work in plain old Java script. So first off, we're going to have to define a layout and write some HTML. And here we create some DOM nodes. And I should say as well, just at the top here, this isn't DOM as in me. I'm a web developer that happens to be called DOM. DOM actually stands for the document object model. So this isn't vanity when I start talking about the DOM. This is just the lingo, I'm afraid. So we create these three DOM nodes up here. And as you can see, we've got this paragraph tag, this span, and this input. And the way that we would represent this as a tree is here. We've just got a paragraph and then the span. You can see that's there. The span with a particular ID that's donated by that hashtag there. And likewise, the input there that's donated by that ID and basically all I want to do is create a script that rather than saying hello world or hello Berlin whatever allows a user to type their name in and it will say hello to that particular user and so what I need to do is take the value that's put into that input so that's a text field there and then set it as the HTML that's going to be inside of my spam and but rather Rather than doing that just once when the page loads, I need to write a JavaScript application that's going to do that continually. Whenever it changes, it's going to update the DOM and do something to make sure that the page is up-to-date. The two technologies that are involved here, the main one is JavaScript, which allows us to manipulate me or manipulate the DOM, and then React, which is a JavaScript library that allows us to rather than write a full javascript application and code in pure javascript and implement everything ourselves react allows us to do this declaratively so we just define what we want to happen and then react takes care of the rest for us and so ultimately um the piece of code which is going to say hello to our particular user and display it in the front end and put it in that spam and is going to be a bit of javascript that looks something like this The exact syntax here isn't especially important, but the important point is that there's going to be some JavaScript that runs inside a user's browser that updates the DOM for us and kind of implements the behavior that we want. Meanwhile, though, if we start looking at how to do this inside of Dash, Dash is a Python library and all of the logic that we write and all of the code that comes to Dash or that we put in Dash is Python code. And so the interactions between our different DOM elements or the different dash components that we put inside of our app are going to be written in Python. And that Python is going to live on our server. And so that means that every time any input changes or we have any kind of interactivity on the front end of our website, we're going to have to make a call back to the server where our Python script lives and run something there. So we're constantly crossing a network to have these sort of interactive web pages. So, if we take a very brief example here, we rewrite the JavaScript app that we've just discussed and we put it into Dash here, so you can see I'm just defining that layout again using the Dash HTML components. The input component though, so that's our text box, actually belongs to a third library called Dash Core Components that we have to import as well. So we've got our input here, and then I'm creating this callback, and that's literally just a Python callable, so usually a function, and I'm defining the output and the input to this function. So whenever any of these inputs change, this function is going to be called, and it's going to create some output here. And the output of this particular function, so what it's returning, in this case a string, is going to come and live inside of this H1 up here. and as I say, it's going to run any time that this particular input here changes. So it's a little bit like an Excel field or something where you're making a reference to some other cell, and whenever you update that cell, then other cells related to it in the formula are going to update as well. And so now if we – let me pull this web browser over, I'm afraid, and we now go to our second example. So here you can see I'm running this app locally now. I've set up this sort of interactive example. And now if I want to, without dropping that pointer, I can type style 6. And you can see that here Dash has created an interactive web page for us. We've written everything using Python. There's no JavaScript at all, but we've suddenly got an interactive web page that we can start to play with. So, as I said before, every change that's going to happen on that website, or every time that I push the key on my laptop just then to change the text that's in that heading, my browser made a call to the server, which in this case is also on my laptop, but if was on the internet it would be over a bigger network, for example. To run that piece of Python that's just that very basic string interpolation there, using the kind of f-string syntax that was new in 3.6. And this causes a huge sort of performance loss, and actually in Berlin I'm not sure that this really carries much weight anymore because I tried one of those electric scooters last night and they're really, really powerful. So actually, yes, I'm not sure which is React and which is Dash here, but there's a huge performance loss in terms of the interactivity of your web page. But that's generally okay. I'm building a really basic proof-of-concept app here, and I want it to be made by a data scientist. It's not going to be a huge production load application. It might be for 10, 20, 100 users in a specialist area, people who are looking at particular data sets or want to have some results from a machine learning model, say. It doesn't have to be super, super performant in the way that Airbnb, say, might be. And so it's good. And my data scientists can take the lead and really start to build things with it. And so if you want to think about what can you actually do in Dash, how far can you go? Well, the answer is very far, actually. I mean, Dash is, like I say in its name, is geared towards building dashboards above anything else. But you can actually add some really quite interesting sort of transactional kind of activities or build transactional web pages, which I'll show in a minute as well. And so just to give a kind of very brief whirlwind tour of some of the different functions of Dash and features of what you can do, I'm going to show three examples. One is just going to be listing essentially a data frame, but making it interactive. Another one is going to display a graph, because obviously Plotly are very well known for plotting. And then thirdly, we're going to explain what I mean by this kind of transactional approach or transactional request before. And like I say, all of the examples are in this GitHub repository here, so if you want to take a look in more detail, then head over there. So first off, the displaying a data frame, I took the Titanic data set, so this is the list of all of the passengers on the Titanic and you can see here and this might look like quite a lot of code but actually it's just a lot of it's the way it's formatted so we have this data table here which you can see and then I've created this drop-down as well so this is just my layout I've got a data table and a drop-down and I've got this callback which just filters my the data that goes into my data table based on the value of that particular drop-down and so now if we come back to our previous example you can see here I've got the the data set that was in my data frame and this is all using Python there's no JavaScript involved here and if I wanted to show you say just the female passengers as I feel some sex you can see here I've got a list of just the female passengers run the mail and then likewise I want to go to all sexes you can see I can update that. All of this is happening just in Python. There's no JavaScript involved here. No one's had to learn any new front-end technologies. You could do this in a G-Python notebook if you wanted to. There's no special web thing here. It's just a data science library. Let's see if I just move this back. Moving on. You can see here this callback with the decorator. That's what makes it interactive. A graph, that's often one of the things that we really need to present in a data science report of some kind. And again, it might look like a lot of code, but again, it's how it's formatted. There's this dash core component, and they actually have lots of different types of graph built in. In this case, I'm just building a histogram. I've split up my data set. So I'm displaying the number of or the frequency of passengers on board the Titanic by the first letter of their first name, which was a fun little miscomprehension to write if you take a look inside of that GitHub repository so here I'm literally just defining I'm going to have a graph, here's my data putting some labels on and all that kind of thing and then again I hit run and again you could do this in a GPython notebook really quite easily and you can see I've got a web page now with a full histogram just displayed using pure Python and I could make that interactive I could put all sorts of other different interactions and kind of build up a lot of complexity in my React application or in my Dash application. And then finally, this transactional example. And this is something that I think is really powerful if you start using Dash a lot inside of your company because rather than just displaying information on the web page, often you want to have the ability to change that information, maybe to leave a comment or to, if you're classifying images, say you want to say yes or no is this an example of deforestation or not and so here I'm creating a very classic web development tutorial which is a to-do list so here I've just got an unordered list that I'm going to populate with my tasks I've got this input which is going to take some text from the user and I've got button here and so that every time that buttons clicked I want to do something and so then I define a callback on actually well here I'm mimicking having a database say so in this example I've just literally put a list inside of memory and I've got this this function that will give me a list of lists items here so this is an HTML tag and then this callback again the important thing here is that and whenever that buttons clicked and so buttons have a property which is the number of times they've been clicked so whenever that goes up this code here is going to run and I'm pulling out the value from my from my inputs and my text field and then I'm able to add that or append that to my list of tasks and so again move this over you can see a do list to do list so we could add any class we want in here and we've created a just a very basic to-do application there and the important thing is that okay this is loaded inside of memory but if I were to refresh the page now for example and it's still there because I haven't shut down my server and you can put that into a database like a MongoDB or a sequel database or something like that and you can actually start to create interactive tools where you don't just display data but you actually you're able to edit it as well also make changes to your state. You can also build your own Dash components, so if you are a JavaScript developer or you have one on your team and you want to extend the functionality of Dash in a particular direction that isn't covered yet, you can create your own component and use it however you want. But as a slight criticism of that, I would say that the actual API here is slightly limited, and access to some advanced JavaScript features like being able to use Redux, for example, is um complicated at best sometimes so but that being said you can actually go very very far with dash and where a example from a project we used recently and this is actually a slightly older um iteration of this project but this is a machine learning um tool so in the back end here there's a whole machine learning pipeline that takes images from satellites and looks for deforestation and then decides on each instance of that deforestation, how likely is it that it's down to illegal mining which we're trying to detect for the Colombian government. This is a tool that's used by law enforcement on a trial basis for a particular algorithm and you can see we've got all sorts of thresholds and filters and they can approve or reject particular hypotheses and so on. This isn't like a really slick web application necessarily like you know, Facebook or Google or something. But it was built by two data scientists with no web experience over the course of a couple of months as they were iterating on their model. And so when we want to kind of take this further and actually take it up to another TRL or, you know, kind of deploy it as a bigger tool, we'll obviously change this around. But in order for rapid prototyping on small teams, this was a brilliant project for us in that we were really able to advance things very, very quickly here. And the only slightly complicated part of this was the mapping component here, which we built in-house and are looking to open source as well. So just some final parting tips for when you start to build larger Dash applications. The tips are really the same as building any larger software project or larger data science project, and that is to kind of religiously organize your code and split your callbacks and your layouts into different files, different files, say, and start to run things as modules and really think about the logic and the organization of your code. So a typical Dash example we have here, and this isn't covered in the Dash documentation but more of a convention that we've developed at DecisionLab, we'll have for every particular component or layout or route that we build inside of Dash, we have the callbacks and layouts filed and then these are hierarchical and nested. We've also begun developing an internal framework for Dash. Moving away from just writing single Dash scripts to actually declaratively defining our routes and our particular endpoints in Dash and the callables that we want to use. That allows us to, for example, automate loading data frames and implement things like dependency injections. If you want a connection to a database, we provide the data scientist with a connection to the database rather than have to worry about connection strings and secret handling and all that kind of thing inside of a data science project. So if this is something that you think might be interesting, then please let us know because we are looking to open source this in the future and we are always looking for volunteers to help and provide details to that particular framework. But just before we finish, I think a really important thing to discuss is actually when you want to stop using Dash and start building a more advanced or a more involved, I'll use inverted commas here, proper web application. Just to kind of summarize, Dash is really good for rapid UI development by non-specialists. Data scientists can get an application up and running without specialist knowledge really, really quickly, and the applications they build are actually very informed as well. They don't have to hand over work to, say, a software engineer or a front-end developer who isn't as interested in the data science aspect as they are in the UI development aspect. But at the same time, Dash presents quite a few problems because it's really rapid UI development by non-specialists. And so if you're going to start a project that you think might eventually lead to building a production web application, maybe you should get a JavaScript engineer on board really quickly because otherwise you're going to create some technical debt. If you're eventually going to have to build something in JavaScript, Sometimes biting the bullet is better than, you know, continuing down the line. And likewise, this idea that UI development is informed, well, you have to ask who's informing it. You know, it might be very informed by a data scientist, but actually, would you not be better doing a lot of user research here and getting some UX consultants in to see what's the tool that you actually want to build and how are your users going to be using it in the end? And also what you can do in Dash using these components is, of course, somewhat limited as well. So Dash is a great tool for facilitating rapid development, especially with data-driven dashboard interfaces and so on. Actually investing a little bit of time into it and getting on board with it as a company say really can allow you to go very, very far and produce some really, really cool tools. But I'd say the take-home message is that actually front-end developers will still have a job. Dash is not here to replace that. So thank you very much. Thank you very much. Thank you very much. Maybe we have time for one or two questions. Thank you for your talk. I'm using Dash myself at work. One question is, do you have any experience with custom design? I think you mentioned quickly CSS, but maybe you have a bootstrap template or something you want to apply. Have you worked with that, and what is your experience? Yes, we do. So as an example, actually, of the – this isn't really the design, but an example of how we kind of use our internal sort of framework. Often you find that you don't want to have a data application that has just one graph or one display or something like that. And so we try and take the design decisions away from people doing the data science as much as possible and style everything using kind of consistent CSS and but one of the good things that we well good practices we try and develop is because we have them we have a framework that essentially you use a decorator say this is my layout this is my callback and then we kind of assemble a dash application under the hood and it means that we can actually do things like implementing routing and navigation and that kind of stuff and without the data scientists having to worry about that and so here for example I've got the same component repeated twice and that's being built and that's kind of being repeated twice and handled by our framework. And then we have internal CSS tooling that creates banners and sort of corporate identities and stuff. So yeah, manage it with CSS, but also abstract out your scripts, I would say, to make it more repeatable. We're using DAS as well. You have talked about rapid development. Do you have any experience using this in a more production environment? I mean, we've had quite a few problems with Flash being embedded in DAS and not the other way around, especially around deployment. Is that something you can comment about? Yes, it can be a real pain, for example, if you want to have an API plus a user interface and maybe some Swagger documentation all living on the same Flask application. Yeah, it is a real pain. And the way that we kind of solve that is that we don't, we wouldn't necessarily use Dash in something that we would consider to be like a proper production environment, say a board like a destroyer or something. We really keep this to like our proof of concepts and kind of our alphas as a sort of development tool. And then as we want to build something that's a bit bigger or more involved, we have front-end engineers that would start to take that over so yeah, we use this for rapid templating and prototyping and building proof of concepts, alphas, rather than betas or actual kind of proper products that we would deploy Yeah, thank you very much, it was very insightful unfortunately we don't have time for questions anymore, so please feel free to approach Dom maybe here or in the foyer later Let's give an applause to our speaker today.