Create interactive Jupyter websites with JupyterLite

Speaker 1 [00:04]

Thank you, everyone, for coming. So, yeah, my name is Jeremy, and I'm a core Jupyter developer. And over the past, I think, two years already, we've been working on a new distribution of Jupyter called JupyterLite. And this is what we'll be talking about today. And the goal will be to show you how you can use Jupyterite and how you can create your own Jupyterite deployment so you can create applications and make them available to your users. So we're going to start with a few slides first, and then we're going to jump into the workshop. So if you were here last year at the conference, there was a talk about JupyterLite. That was more of a presentation about the new tool. So I invite you to check it out. It's on YouTube. And there we cover a bit more, like, the differences and also the history of JupyterLite. So you might wonder, like, in JupyterLite, there is the word light, like, how is it different compared to normal Jupyter? So the main difference, if you use it from, for example, from a GitHub pages deployment, is that it's very, very fast to start. So you can just get to it very quickly, get to the interface very quickly. And that is because JupyterLite deployments can be created using a JupyterLite CLI tool, which is kind of like a static site generator. So if you have used the JCL before, Ruby-based static site generator, you can think of JupyterLite as something similar. So it's just like you have a command available in your terminal, you run it, and it generates HTML, CSS and JavaScript files, and also we will see later your content, for example notebooks. And what that means is, if it's just static files, you don't need an application server anymore to serve your Jupyter, and an application server is usually in Jupyter world is based on Python, so you need first a server and then Python, which can be sometimes tricky to get. You don't need a command line for using the tool. For building the tool, we still need to go to the terminal. And probably one of the best things about this is that your users, they really don't need to deal with installing Python locally or installing other packages locally, especially on Windows, MacOS, Linux, all of these different platforms can sometimes create frictions when you want to, for example, do a tutorial or teach something via Jupyter. So again, like the main advantage of Jupyter Lite is the fact that it's very easy to deploy. So there is documentation there if you want to check it out. What that means is that in practice you only need HTTP server like something very simple if you used the HTTP dot server module in Python. We will see also that later You can just start it with Python minus M HTTP dot server and then you serve your files like this Okay before I continue this is interactive As the name suggests so if you have any question don't hesitate and we can we can take them as we as we go no problem so um where can you host tryptolite pretty much anywhere where you can get a http server like a simple http server so github pages is a very good candidate we have many examples running on github pages and the same for gitlab gitlab pages read the docs even binder if you want and also more like those platforms that you see more in the front-end world usually versal or netlify also works there and of course there are many many others so jupyterlight is not that new it's not like a crazy new thing it's more like building on top of everything that already exists so the interface is based on jupyterlab components which means that they were reassembled together to produce something else. So there are a few functionalities in JupyterLite that are not available in JupyterLite. For example, a debugger. Maybe that will be a case later. But the idea is you take those building blocks, you put them together, and you create this new application. And the important thing is that the front end, so the UI that you get when you use JupyterLite, It's still the same, and it communicates the same with the kernels, and we will see that the main difference with the kernels is that they don't run on a server because there is no application server anymore, but they run in a browser, and they still communicate with the front end using the Jupyter protocol, and the Jupyter protocol is really this central piece like at the center of Jupyter that you would find everywhere, but you never see it as a user, but it's really this thing that makes compatibility between notebooks, interfaces, and also kernels possible. It's the standard pretty much, protocol standard. And yeah, you can use visualization libraries you're familiar with, matplotlib, bqplot, and so on, and with JupyterLite, so as we said, the kernels run in a browser, and that is made possible using a browser technology called WebAssembly. So WebAssembly is the fourth standard in the web browser world. So you have HTML, JavaScript, CSS, and now we have WebAssembly. WebAssembly allows you to take a program written in another language, for example, C, C++, even Rust, and compile it to something that the web browser can understand. And the good thing is, Python, the default implementation of Python is called CPython, and CPython is implemented in C, and C can be compiled to WebAssembly. So that's what people have been doing, especially with the Pyodide project, they were the ones kind of starting all of this. So that means that now we can have Python running in a web browser, thanks to this. So when it comes to interactive computing, which is what we'll be talking about with TryptoLite, we have two Python kernels, so one is called PyDyde and the other one is called Xoos Python, so they take different approaches, we'll see that later, but it's much the same in the end. It's just taking Python and combining it to WebAssembly so you can use it in the browser. And they both support, like we said before, like the visualizations libraries, and more. So one thing we've been starting to see with this, and this is something that potentially can interest you at the end if you when we finish to the tutorial is that once you have a Jupyter light deployment out there you can pretty much embed it very easily in other places in other websites so here is just an example of how you could embed a small Python console in your blog, and this is using a component in JupyterLab that is already there, it's been there for many years, it's the code console, it just gives you a console, that's it, but you can already do quite a bit of things with this. But one of the major adopters is the numpy.org website, so if you go there, you will find something that looks the same, and you can play around with numpy in a browser, so that's pretty convenient to try things out. And it's the same for SymPy now, they also adopted it, and we've seen that across many other popular packages in the PyData stack, so everybody is starting to look, oh, I want the same, so they decided to do the same, so it's been growing a bit fast like this. Another use case is the actual Jupyter website itself. So it used to be using a service called MyBinder. I don't know if you know it. This is like it takes your repo and looks at the dependencies in the requirements of text or environment.yaml and creates a Docker image and then it pushes it to a cloud and then it starts a server, like a normal Jupyter server, and then you get access to it via your browser. So that's really great if you want to be able to reproduce stuff using this. But we were also using it on the Jupyter website just to demo the JupyterLab and Jupyter Notebook interfaces. And now this has been switched to using JupyterLite, and that helped a lot when it comes to compute resources because at some point, maybe last year or so, the Binder project had some issues with funding I think it's okay now but they were looking in a way to cut the cost and we are we deployed this thing very quickly on github pages and really had to reduce okay I don't have it here but it was something like 30% or so in terms of traffic and also resources used on on the cloud so yeah here the trade off is you exchange compute resources for static assets, so arguably your bandwidth is going to increase a little bit more because you have more files to send to your users over the wire. Another way you can embed it is if you have a docs, so that's also similar to what we saw before, many packages have started to use JupyterLite to embed a notebook or code console in their docs. So again, the docs is where you go find information for how you use a package, and it's great if you can actually try it there directly. So main advantages are that it's easy to deploy, it's just a bunch of static files, it's simple for users. They don't need to install Python or any other packages locally. For educational use cases, especially when you want to do only like very simple things or cover the basics of Python, this is very great. And also, one last thing is once you have your website with everything, all of the assets it's very easy to just take it and put it somewhere like archive it or put it on usb stick and you can think of it as like a time capsule so the good thing about the web is that it's all governed by by standards so it's very unlikely that javascript html and css and web assembly are going away anytime soon so if you have some research and you want to kind of make sure that it's going to work again in maybe 10 or 15 years then once you have this and your set of static files you can pretty much just say okay it's done and then you can just package it or put somewhere and then maybe open it later and it will work just like websites created ten years ago, they still work today. So that's a really big strength. But it also has some limitations, so it's always a game of tradeoffs in the end. So you can't do very heavy workloads, so you won't be able to just use gigabytes of data because your browser is going to crash before. You can't use GPUs yet, maybe it will be possible in the future, let's see. One thing about packages is that many packages are also not available, and we don't even know if they will be at some point, if it makes even sense to have them on a browser. So then you work with some kind of subset of Python in a browser, you also don't have multi-processing right now. I think it's going to be the case later, but as of today, it's not easily possible to enable that or even just raw sockets. And since your execution environment is now in a browser, you need to bring that to your browser from somewhere. So you have have increased data over the network, so if you look at the web browser, you can access the developer tools, you will see that more data is transferred over the wire. So that's the trade-off. Fortunately, the browsers are able to cache data quite aggressively sometimes, but that that can already help to avoid downloading the same files over and over again. So in this tutorial, I'm going to show how you can make your own. So pretty much you can just host it yourself on your GitHub or anywhere else if you want. And there is a demo repository. I think we're going to start with that just to be sure. And then we're going to check how you can add files, notebooks, extensions, other packages, how you can customize, how you can build a small application using GPU widgets, and how you can export all of that as a static dashboard. So yeah. And if you want to go to this URL. So that is the script we'll be following on read the docs, and I think I posted it on Discord already, so you can also find it there. Yeah. Should be good. Yeah. So if you have any questions, on the intro slides, yes? Yeah, so we will see it later, but maybe I can already give some answers here. Maybe I will repeat the question. So the question was how do you know if a package is available, right, for the browser? So in this tutorial, we're going to use remember when I said there were two Python kernels right now, one called Pyline and one called Xoos Python, we're going to go with Xoos Python, which is very similar in the end, but the difference is that Xoos Python reuses the whole conda forge infrastructure, so if you're familiar, who here is familiar with conda? Okay, that's good. Maybe half of the room. So CondaForge is a decentralized way of working with packages and even packaging Python packages. So it's not like just a couple of people who handle the thousands of packages coming from everywhere. It's more like each maintainer has a responsibility for their own package, and there is a lot of CI going on, a lot of bots, and that makes it much easier to work with this and publish packages to a contact channel called CondaForge, and we'll see it again later, but just to give you an idea, so if you want to see if a package is available, you have to check in two places. So if you would be just using a normal Jupyter, you could just check on CondaForge or PyPI, But here, since we are in the browser, not everything is available. So the first thing you can check is, for example, let's say iPad widgets. So iPad widgets is this thing that lets you create controls. So I use this prefix.dev website because it's very, very easy and very sleek, but you can also go to anaconda.org and also search from there. So we see that it's there, so it's fine. And it's a no-arch package. means it's not tied to a specific architecture so that means that this one can be installed whether it will be working or not is another story because it might depend on other packages which are maybe not you know maybe they don't work in the browser or not so so it's still hard to know whether something is available and the other place we need to look into is this channel called EmscriptenForge. It's like CondaForge, but for packages that need to be compiled in some way before. So if you have NumPy, for example, if you want to use NumPy, you will find it here. It's here. It's available. Because it has C extensions, they need to be compiled to Web assembly. And for that, we use a tool called Emscripten. If you were at the web assembly presentation on Monday, I think Antonio talked a little bit about Emscripten. So without going too much into the details, it's a bit like GCC, you know, but for the web and also for servers, but in our case, it's not really the topic. But you could use WebAssembly somewhere else than just in in the browser, but yeah, let's forget that for now. So yeah, so you have to check there and then also on the other page. But if you are using Pyodide, you can also use the micro PIP helper, which is in Pyodide, but maybe we'll have time to check that as well at the end. And you can try to install and see if it works and then import and if it does, then we are good. But since there is an issue about this in the Jupyterlight repo, I think we will put at some point a small tool to make that much easier, because otherwise people maybe don't know where to look unless they know these websites, for example. Any other questions? Yes? in the first session sometimes I send a notebook and send them to Google Colab, right? Yeah. So I would like to know the technical and practical differences between this and Google Colab. Yeah. And the second question is, when I give them a link, they access a notebook, and the state of the notebook will be not good. Do they block the browser or they can download the copy, or how does it work? Is it shared between some of them? OK, yeah, we will see all of this. So the first question is, how is it different compared to Google Colab? So Google Colab lets you use notebooks in a browser as well. But on the back end, they still use a Jupyter server or something similar. I'm not exactly sure if they used the raw Jupyter server from the open source project, but it has to be something like this. and they also use a kernel on the server, and they give you access to GPUs, so they are like, you know, it's a bit more heavy, but you have more resources, and it's also something you cannot host yourself. So if you wanted to use it also offline, with Jupyterlight you can, but with Google Colab, it's going to be more tricky. And about the content management, So for now, you can create a JupyterLite deployment with content already. So it's already available to your users. It will just be distributed as a static file. And then when they open the interface, when they open the notebook, when they edit it, it will be saved in the browser storage. So there is a component in a browser called indexdb. It's like a database in your browser, and JupyterLite uses this to store the notebooks and make sure they are not lost. But all of this is scoped by domain name. So if you work, if you have a JupyterLite deployment on this domain, it's a combination of domain and port, your notebook will be available only if you access other JupyterLite hosted under the same domain. And that is for security reasons, right, for baked in the browser. But if you switch domains, then they won't see the same notebooks. And from the interface, you can also download the notebooks if you want. But again, like since JupyterLite is based on JupyterLab and JupyterLab is a set of extensions, you can also create your own extension and store the content somewhere else. So it's modular enough. It requires a bit of work. It requires web development skills, because you need to integrate that with a TypeScript extension. But I think some people have already done that. They want to store that somewhere else, like on S3, for example, or another place on the cloud. So they dissociate the content from the actual interface and site. All right. So let's see. Yeah, I think we have an hour or so, so that should be enough. So what we're going to try to do, let's see if we're going to the end or not, is to build something like this. So this is JupyterLite. It's running on GitHub pages. This is a demo I did before. We use widgets. paste a list of URLs with GPS coordinates. I'm going to show a map, pretty much just an app where you can show your pictures, and then you click on that, and then you can just hit the save button. It's going to download the HTML for the map, and then you can open it and see it. So you can see that as an app for your users who want, for example, to just fiddle with maps. But you can imagine anything else. So if you have any other thing you would like to do and work with, feel free. You just need to swap the notebook with your own and, of course, fix the dependencies if you use other dependencies. So you are free to do that if you want. So we're going to pick some parts of the existing docs. So the docs on JupyterLite already cover how you can get started. So there is a getting started section here. You can use something that is already available. Or you can deploy your own on GitHub pages. And then the thing we saw with the report on the website and so on. And then you have more focused topics, focused guides on different topics. So here how you can configure it, add content and so on. So this page is pretty much just a kind of summary of all of this, and we're going to do it live. So we see, like, all of the steps and what it means. Okay. So we have two ways to proceed here, and it's going to be a matter of preferences. So I'm going to first show how we can use the demo template, which is the kind of recommended way if you want to make something on GitHub pages. And then we're going to continue with the actual JupyterLite CLI, like I mentioned before, just to show you what happens when you use it. Because in the end, this is what is used on the GitHub template anyway. So it's also good to know how the CLI works. So you can more easily debug it in case you have issues. So if you go to this repo here, this is called Exus Python demo. everybody have a GitHub account? Okay. I guess that is good. If you don't, don't worry. If you want to do it later, you can also just follow the tutorial later. Should be fine. This is Foxus Python kernel. There's another one for the Pyodide kernel, but we're going to use this one today. So if you go here, use this template create a new repository this is going to create a new repository under your account or you can choose an org if you are also part of an org let's see we can do a back on the jupiter light tutorial we leave it as public because otherwise deploying to github pages is going to be more difficult unless you are a pro subscription I think and that's it so you can click on create afterwards so it generates the repo it's pretty much just copying the files that's it one thing you need to do and that is also mentioned in the get started on the docs I go to the settings and then pages and here you need to to select GitHub Actions. So it's going to use GitHub Actions to deploy to GitHub Pages. Before it was to deploy to GitHub Pages, you had to set up a gh-pages branch, but this is kind of the old way of doing it. Now they are switching to using GitHub Actions, which is still in beta for now, but maybe it will become the default at some point. This is what the template uses. Now, so if you go back to your repo, once it's generated, you can see that first there is the CI running, CI is continuous integration, it's going to run some scripts to run the command. So let's check that first, just so you know where it is. So here, there is a GitHub action, a GitHub workflow to deploy the JupyterLite website. And it does a bunch of setup, and that's implementation details. And then here, it calls the JupyterLite CLI by doing a JupyterLite build. But don't worry about this, we'll see that later. And then when it's done, it pushes to the GitHub pages environment here using the GitHub actions for deploying to GitHub pages. So this is just what it does for deploying. So it's just a bunch of commands to run and GitHub actions to use. So while it's doing its thing, we can look at the rest of the repo. So the content folder is where you can put your content and the demo here is the thing I showed you on the video just before. We get back to that afterwards. This is not the one, but never mind. This is just a demo on a book. And then you have environment and build environment. So if you're familiar with CondaForge, you'll recognize those. Build environment is your, as the name suggests, the dependencies for building a website. So we have to keep in mind that there are two environments, one for building a website and one for executing the code in a browser, okay? So one for building the website is this one, and we see that we have the dependencies on the JupyterLite CLI, and the other one for running the code is this one, and this is where we see we have a dependency on the iPad widget, for example, okay. So let's go back. Yeah, here we have the green checkbox. That means what you can do, actually go here and click on use your GitHub pages website. Save. So now it's going to be available here. And you open it on a new tab. I don't know why it's not HTTPS. So it's using my domain name here because I mapped it to GitHub pages, but for you maybe it will just be something on GitHub.io, which is also fine. Yeah. Yeah, sure. So the last step was how do you get from this repo to the actual GitHub pages deployment? So the GitHub UI provides a convenient way to do it. If you click on this gear here, then there is this checkbox, and it's going to generate it. If it's not there for some reasons, you can also easily construct it yourself. So it's going to be your username, and then .github.io, and then slash the name of the repo. Yeah. And redirects to the Jupyterlite. And there you go. Just a couple of seconds, okay, minutes for building the website, and you have it there. So yeah. So that's how you can get started. Next we will see how to use the JupyterLite CLI. It's going to require a couple of dependencies on your machine. So in case it doesn't work for you or you just don't want to do it or whatever, you can also use the repo directly. So again to show let's say you want to add a dependency here, let's say you want to add numpy. So we go to the environment.yaml, which is the one for the browser. You go here. And I will recommend creating a branch. For example, add dependency. And then here you can say add numpy. And then you propose change. It's going to commit your change in a new branch. And then GitHub automatically asks if you want to create a pull request, so you can do this. You can say yes and create a pull request. So that's a way for you to work on your JupyterLite website without even touching the terminal. However, it's going to be a bit, you have to be patient because you need to wait for the CI to build the thing for you. So you can see it here. The checks are available here. And once this check is done, it produces an artifact on GitHub actions that you can download and double check. So that's a good way to work with this. Unfortunately, there is not a way for now to get GitHub pages previews automatically from pull requests. So that would be great. Unfortunately, GitHub pages only works on specific branches. But maybe at some point they will add something like this. And that would be super useful because then you could just work from your repo and get previews without doing anything locally. There are already ways to do it. If you use read the docs or they give you live previews or even live previews URLs that you can double check before merging your changes. So while it's doing this, we're going to jump into the CLI, so I will go a bit more in depth here just so you get an idea of how it works under the hood. So if you want to use the CLI, there is also a guide on the docs later if you want to get back to that at some point. So for that, you need a terminal, and in our case, we will be using a package management tool called Mamba, or to be more precise, MicroMamba. To confuse you even more, Mamba is a bit like, it's like Conda pretty much. It's a drop-in replacement. It was created to speed Conda up, speed things up when it comes to solving the environment. And now Conda is even using parts of Mamba in their Conda CLI tool. So with MicroMamba, it's easy because it's just a binary, so you can just install it like this. And then you get it. Yeah. You can also do it with Python like this if you want. However, for the Python kernel, since it's using CondaForge, to fetch the packages, you still need to have Conda at some point. So you still need to install Conda. But we can see later how to use the Pyodide package, which is not which doesn't require using conda so I'm going to do this now so it doesn't take too much time yes it's that's probably because you choose to create a private repo yeah yeah Yeah, so the question is, need to update to Github Pro to deploy on Github pages? Yes, if you have a private repo, no if you have a public repo. So here, so if you don't have MicroMaba, I still recommend you to check it out. There are blogs here, it's linked. What we're going to do to create a virtual environment, to install our build dependencies, so that will be JupyterLite and other build dependencies, and again, this is separate from the execution environment, which is in a browser, and it's going to be managed separately, to keep things well separated. Okay, so mamba activate, what was it called already? Yeah, Jupyterite tutorial. Now if you do member list, now you get a bunch of dependencies. So if we go back to the next step, yeah, no, this one is from this, right? Yeah. It's like I mentioned in the beginning, it's just a summary of the others, and I'm just linking you to the docs in case you want to go more in detail afterwards. But for now, we're just going to follow this script. So I'm going to do this one in the meantime. So here we installed the CLI and also another dependencies that will be useful for indexing the content. So it's available on the site. So we use memba to install, you can also use conda, okay, and you can double check that JupyterLite CLI is available. Yes? Can we use the build environment YAML file to install? Which file? I see there is a build environment YAML file, can we just... Yes, you can use that, yeah. Yeah, absolutely. It's a good practice actually to do this like this, to install it in an environment. Here I'm just doing it step by step to go through all of the dependencies we need. But you're right, it's better to keep it in this file here. You just put it. Actually all of them are already here, the one I'm going to install. But once you have in your virtual environment, if you don't want to use mamba, you could also use pip. It's available on PyPI. You just need to install it like this with JupyterLite core, and then the brackets will lab or you can install JupyterLab on the side just because it's needed for indexing the content and also a bunch of other extensions we will see later. Yeah. Okay. So once you have this, you can do a JupyterLite init. And if you do ls, okay, let's do this, you get two files. One is a hidden file, it's for storing the previously executed steps, so you don't need to care too much about this, and the output is the default name of the output folder. And this is pretty much, that's it, that's your site, everything is in the output folder. So let's see if we do, yeah, if you do a tree, you see that you have notebooks, HTML files, CSS files, JavaScript, of course, and a bunch of other things. So yeah. Once you have this, you can actually just serve it. So you can use the JupyterLite serve command, or you can use also the Python-M HTTP server like we talked about before a bit bigger yeah so let's just use this for for simplicity and then you go to localhost okay i think it's my cache yeah okay let's put that back into light So, I see that I have a notebook from a previous try, because it's still under the same localhost domain. Let me see, first, no, yeah, okay. So cache issues, that was a good demonstration of cache issues, so since I already did that before my machine under the localhost 8000 port, my browser remembered that I had already static assets and it was showing a kernel on the interface, which shouldn't be the case right now because we haven't added one. So here we are currently at this step, let's see, where you get only something very dry, There is nothing really you can do that much, just the bare application. We are going to change this. Now you need a kernel, so we're going to use Duxus Python kernel. You can use pip to install it, unfortunately, I don't think it's available on ContaForge right now, but it will be eventually. So once you have the Jupyter, the thing you need to do is to build the website again. So it's always this step is always required, a bit like when you build your website with Jackal, you do a Jackal build, they also have a serve command, I think, which rebuilds automatically. So that's, again, that's only when you want to create your assets, once they are distributed on GitHub pages, you don't need to deal with this anymore. So let's do it. So we see that it triggered some member thing there, and that is because we need to pack the environment that will be running in the browser, and that is done using microMamba. We could also be done using Mamba or Conda. We haven't added any dependencies yet, but we'll see how to do it after. But it still creates the files that the Python runtime would be needed in the browser. So we already have the server running here, so we can go back to this. And now we have a kernel showing up in the interface. So you can click on that. And like we saw in the beginning, in the presentation, it can take a little while to start up. So the icon here was, had a different state for a while, I don't know if you noticed. That is because you need to start Python in a browser and also download all of the files can take a while still. But then once it's there, it's available, you can do some whatever you want, however However if you want to use different packages, they might not be there yet and we're going to add them right after. So first we're going to add some content. So working with notebooks is important with a Jupyter deployment. So if you don't have one around in a notebook, you can use your own, otherwise there is this PyTudes repo which is quite interesting, lots of examples of how to use notebooks. We're going to download it, and I'm going to put it in notebooks folder, and save. We are here. Stela. You got someone asking online. Okay. Is this the right tool to create an interactive Python app when all the coding is hidden? We see at the end. At the end, you can create a static dashboard. So you can use notebooks, indeed, to do this. Otherwise, you can use Pyodide directly or even PyScript. is also a tool to make Python apps without showing the code. But at the end, yeah, there should be enough time. We'll see how you can actually go from the notebook from the static dashboard. OK, so maybe let's spin things up a bit. So now we have a notebooks folder here. If I list, I have a notebook in there. What you need to do is rebuild again, build, but this time specifying the contents. So that means there is no application server, remember. So if your browser asks for a file, how can we know if there is a file or not? So Drupalite at build time indexes the files and makes them available via a metadata file so you can know be aware of what files you have available so this is part of the build step if you specify the notebooks I'm going to refresh the page and now I have this Sudoku notebook available here this one is living in my browser if so if you do that you won't have it unless you create the same name but it's in the index.db like we saw before, but now we can open this one and we can work with it. Okay. So JupyterLite can reuse almost, I would say most of the JupyterLab extensions, some of them will not work because they need more components or maybe they are not suited for the browser, but simple ones or maybe more advanced ones can also work. So you can see how we can add some of them. So there is this execute time extension that you can add to your site. So it's again, you installed it in your environment or you add it to your build environment.yaml Yeah, it's here. So that's the JupyterLab extension, but JupyterLite is able to reuse the same extension. I just need to build it again. We're going to keep the contents flag for now so we don't lose our notebook we added earlier. Okay. Okay. By the way, for those using the GitHub, directly GitHub interface, so all of this you can do by going to your branch and you can edit the environment. If it's a Python dependency, you add it on the pip. If it's a conda dependency, under the dependency here. And once you have a checkbox here. You can actually check your built website from the summary. It's a bit complicated, but yeah, that's the GitHub Actions UI. And then you click here, and it's going to download the tarball. You see it's quite heavy because it has dependencies. And then you can run your server, your HTTP server from there, yeah. My question, are you planning to support or are you supporting Bokeh? Yeah, I think, I'm not sure if Bokeh works, but Panel does, I'm not sure if, I would say it should, it should work because, if I remember correctly, they have a version based on iPad widgets and API widgets are also available in Jupyterlite. So I would say it should, and if it's not, then maybe it's just a matter of time. What's the name of the widget? IPyWidgets, yeah. OK. OK, so back to our site. So now we have the extension. And you can see there is this new thing here. And that is provided by the third party extension. That's not in part of Drupal app by default. It's something we added extra. And that's the one crucial point about the whole Drupal ecosystem is you can configure it very deeply if you want. And there are many extensions like these ones. And this one gives you like the time it took to execute a set. So it could be useful to have on your website. So one other thing we can check is how to add a theme. So here you have, by default, a dark theme and a light theme, but you can also install one, and I think this is this one here. You can install one that is a third-party theme. Let's say the night theme, for example, which is quite cool. Okay, this is because I didn't save the notebook. It might take a while for the browser to pick up the changes. Don't you need to rebuild? You're right. Yeah. That was good. Actually, just need to rebuild to forget. Yeah. So when you build a site, it copies the extension files, which are also just JavaScript files, static files, to the build directory, in our case, the output. So we're going a bit in depth here. You don't need to exactly know all of this, but in the end, they end up here. So under the extensions folder, you see there is the night theme now. Let's refresh. And now we have the new theme. There are just a lot of things you can add to a website, so we're going to see another one is how you can add a new display language. So this is all English right now, but let's say you want to add French support to your or JupyterLite, which is quite useful, actually, because there are some people using JupyterLite in France for education, and they have been using the translation package that we're installing right now to have everything in French. So yeah, let's pick up the question, maybe, while it's building. No, there's no question. OK, that's good. Well, you've got a technical question, although I think you mentioned about it already. When trying to rebuild my page, it fail saying I need Mamba or Conda. Is it not possible to run JupyterLite with virtual environment? Yes. So if it's someone from remote, maybe they can check the very bottom of this page here, the tutorial. There is extra. You can use the Pyodide kernel instead. So you can skip the Xoos Python, which requires Mamba to build your environment, and use Pyodide instead. And then I will refer to the docs directly. that will be a bit easier to follow remotely, I think. Then you can use Pyodide to install packages. It would be a bit different right now. So in future, we hope there will be some kind of convergence. I think this is going to happen at some point between the two kernels, but for now, this is how things are. So it's good also to state them just so that you know about that. But I would recommend using this if you don't want to deal with Mamba locally or use the repo directly like this, and then it's available on the GitHub actions here. Okay, so let's go back here, and yeah, in settings now, you have an option to switch to French. So this is what students see sometimes in, okay, there is maybe a cache issue, yeah. Yeah, in France, when they use JupyterLite for their introductory class on how to learn Python, for example. So let's go back to English. Where were we? Now we go back to here, to the next step is how you can install extra packages. So here there's a little note reminding you that we're using Duxus Python kernel, which is based on the conda forge, pretty much, and Emscripten forge. So it means that you have an environment.yaml file locally. So here I'm going to show you how to do that. So for now, there is no environment.yaml file on my machine because I started from scratch. If you start from the repo, there should be one already in the files here. So I'm going to create one. You see that we specify two channels that might be a bit unfamiliar to you. They are the ones, especially the mscriptenforge is the one listing the packages that were built for WebAssembly so that we can reuse them in the environment that will be running in a browser. Yeah. I think we just in Folium actually no arch packages, so they can be used without mscriptenforge, but later if we add more, let's say, packages like pandas on NumPy, mscriptenforge will be needed. this, and then another build. So I'm going to scroll up again, and you see that in the build, the build step is now installing Folium, which is dependency, and also iPad widgets. So for me it says cache because I already did it before, otherwise it would be fetching it from the Internet. That's it, and again, I'll refresh. Let's create a new notebook. Yeah, this is the indicator I mentioned before. Usually on normal Jupyter, it's much faster. Here in a browser, it takes a bit more time. Let's see if that works. Okay, maybe it's not. Browser. I think there is even a code we can copy-paste, yes, this, okay, so yeah, you saw before that sometimes you can have some errors, this is likely due to browser caching the previous assets so that we ask for folio but it's not there yet, just need to be patient and refresh again afterwards. Question? I have a question. I'm following this with the GitHub workflow. Yes. And you mentioned a file where you should actually configure it to include this. Yes, it would be environment.yaml. And I don't have the environment file. Did you start from the the Xoos Python template? Or the demo template? The demo is based on Pyodide. Sorry for confusing you, but this is how it is right now. Yeah. Yeah. The Xoos Python demo is the one with the environment.yaml. Yes. So if you want to restart with the other one, that would be easier. Otherwise you think you can just continue from your repo and add also the Jupyterite suspython dependency in your build environment, but I think it would be simpler to start again with GitHub, generate a new project, yeah. Yes? And if you're doing it with the CLI, where is the environment, you know? At the top level, yeah, where you will be running your build command, so... Where the underscore outputs... Yes, yeah. But you know that all of this, this is just a default. This is also configurable, so you could, for example, output the folder somewhere else and also use an environment.yml file that is not named environment.yml, but you need to specify extra CLI parameter, I don't want to confuse you too much about this right now, but it's possible if you want. So we have our packages available, the versions here. And like we saw already before, so maybe we're going to go a bit quicker on that now, is that M script and Forge is where packages are pulled from, so they work in a browser compared to WebAssembly. And the main difference compared to Pyodide is that Pyodide, as of today, they still bundle whole distribution together, so they are like a distribution with NumPy, Pandas, and a bunch of other scientific packages all together. But it becomes a bit tricky to use if you want to have more control on the versions you want to use of the packages, and also which ones you want. So maybe you don't want SymPy in your deployment. But unfortunately with PyDyde, it's there by default. So it's two different approaches, but there's been some work to make the two kind of talk to each other at some point. So at some point, we'll probably see PyDyde being able to reuse M7Forge packages as well, so it's a bit easier to compose environment like you would be doing on a normal Jupyter. This note is about how to find if a package is available. That's what we discussed today a bit earlier. And if you want to know more about it, there's always talks. There is also this configuration file. So lots of config files. But this one lets you put your CLI parameter, like we content, for example, here. If we don't want to repeat this all the time, or if you want to have a central place where where you have all your configuration set for your JupyterLite website, you can also create this file and then place the content here. So there is the doc on the structure of this file and where to put the arguments. Can be very useful once you start to have something more customized than what we're going to do. So we don't have to have like huge CLI common lines with lots of parameters. So once you have a file there and you run the build, the build step would pick up the file and do the same as we saw before. There is also this file, the Jupyter, sorry, there is a typo, it's not underscore, it's hyphen. So I'm going to show you that. So this is for runtime, runtime configuration. So the other one was for build time and this one is for runtime. when you reload the page pretty much here we're going to just change the name of the of the uh of the app uh we still need the build because the build is going to take that file and put it in the output directory let me do a check time okay Okay. Yeah, you see that there is a JupyterLite JSON file here. If we go back to our test demo, we see here it's written about JupyterLite, but now if we refresh, now it says JupyterLite tutorial because that's the name we gave in app name. There are a bunch of other things you can configure there, including enabling real-time collaboration. So there is support, that is a bit buggy right now for RTC in JupyterLite, it's using WebRTC. So you have, you can imagine having people collaborate on notebooks, deployed on static HTTP server without setting up a service. You still need a way for them to talk to each other over WebRTC, but this is possible. But yeah, if you want to try, be careful because it's still a bit experimental and it's not working really well. All right. So I think we're going to stop here for the actual JupyterLite interface, like the JupyterLab. You should also know that if you are more of a classic notebook user, JupyterLite includes a notebook view, like a notebook application, similar to the classic notebook, except that it's not the classic notebook, so if you are curious about this, you should come to my second talk right after, that will be in another room, we will be talking about the future of the Jupyter notebook, and pretty much what you see here. But some people like that interface, especially for teaching, because it's just a notebook, And yeah, you don't have any distractions, and people just like to work with that. So it's also possible, it's included in JupyterLite, and this is based on JupyterLab components. Okay. So now we're going to talk about how to create static dashboards from your notebooks. So the video I saw, like, show you that the small applications we wanted to make. So now we're going to call to that. So who here has used Voila already before? Okay, it's not that much. It's just maybe a quarter, maybe not even. So Voila is like, it's a Jupyter project as well. It doesn't have Jupyter in the name, but it's still a Jupyter project. And the main idea with Voila is, let me see if there is a screencast. I think there is one here. Yes, it's just loading. So you take your notebook, let's say you worked on analysis on app or something like that, and you really don't want to rewrite it in other language or in Python, or you don't have time for that, let's say. With Voila, you just run the Voila CLI and it starts the Drupal server, and it's going to show you this, your app, there you go. In a new tab, in a browser tab, with just the output so that's what there was a question earlier about this like how do you do this uh voila lets you do that yeah but voila itself still requires a jupiter server notice python application server in jupiter there's nothing like that so i'm going to show you something still very new by new i mean i think it's was created maybe two months ago or so it's still under active development so yourself with that. But we're going to use it already because you can already do some interesting things. And, you know, bugs are going to be fixed anyway at some point. So the new tool is called Voici. So if you know French, you will know why it's called like this. And it lets you generate those static dashboards so that you can serve them statically again like on GitHub pages, and it's all going to be the same as Jupyterite with Python running in a browser. So it's only available on PyPI for now. So if you follow on the GitHub repo, it's going to be a bit more tricky because now you're going to need to use the Voicy CRI to build the site. So you will have to go to the deploy GitHub action workflow and update it. So I think there is, I put some notes on that here, here. So if you are running remotely or you want to check that later, check out this note here. Okay. So now that we have WASI installed, build, we're going to do a WASI build. And here I've specified the other apps so they are available as well, otherwise it will will only be the generated notebooks for your application. But if we specify the lab and retro apps via the CLI, you will get access to the interfaces like this. For now, it's just for convenience for us. Okay, so it's still using the output by default. Okay, so here it looks like it didn't generate the files And I, this is something you can face at some points if you have an issue with the files that you think they should be built, but they are not. It's likely a cache issue. So in that case, and I think I left a note about this. Yeah, if you encounter any issue and want to start from scratch, you can actually even delete the output because it's just generated, right? So you can just go back to a clean state of your repo. and then run the build command again. So we're going to see if that works this time. Okay. Okay, maybe not. So let's see if I do it again. Okay, so we might have an issue here. I'm going to, let's see. So normally when you build a Voici, yeah, okay, that was my previous cache. You will get a WSE endpoint here, so you can access the kind of Walla-like pages in a browser. But here, I think we have an issue with the builds normally. There should be a tree folder here, so maybe let me check if I can do a more cleanup. So I'm going to remove that. we don't need this. Otherwise, I would just start from a blank state just to be sure. Maybe pass this. So sometimes force will bypass some of the cache steps. So by default, the build is caching some files at different steps. Just to make sure that you don't do the same again and again. Okay. Looks like it doesn't want to. So we're going to do it differently. demo code not available today maybe. So what we can do instead is I'm going to show you first how you can configure your Voici dashboard. So the good thing is that in Voila you can configure, for example, a template and a theme. And in Voici, it's also possible. For that, you just need to install the template if you want to use a different template. So there is this material template. And if you use it in Voila and also in Voici, you will get something like this. The template is pretty much just a set of HTML and CSS files. And it will give a different look to your application that is being built. So, for example, like this one. If you want to know more about this, it's based on the Jinja2 template system, so then you can do a lot of very advanced things using that. And then you can also choose a different theme if you want. So for now, there are the light theme and also the dark theme, but there should be also support for other JupyterLabs themes, like we saw before, So night theme, for example. And there are things you can control using these two CLI flags. So either you pass the template or the theme. Or you can also control it using the theme query string parameter. And all of this is also possible to put, like we saw before, the build time config file. You can specify them under a new key here because it's a kind of WALA configuration. And you can specify the theme you want to use and also the dark theme, yeah. So what yes, the template is modifying the HTML of the host page. So if you just export a notebook to HTML with a template, it's going to be just a bunch of cells without the code on the page. But with the template, you can potentially add more, like, you can beautify your page pretty much, right? If you can add CSS, you can make the page look pretty. You can add a logo of a company, for example, if you want to generate a dashboard at the very top, like this one, you can have a logo of someone else. I think we have time to show this quickly. So they are defined like that. So let's say what a template base. So this is what a template looks like. Can be a bit difficult to grasp, but it's all pretty much just Jinja. And if you've used NP convert, it's also using this system. So again, here we are into the Jupyter world where components can be reused from many different places. good example of how the stack and the stack is being used to generate a static web page. And you see CSS and also you have this idea of blocks that you can inherit from other templates. Like I said, it can be complicated to understand at first, but the Jinja experts, they know how to deal with this. But hopefully if you want to do something small, it's just a matter of just looking at one of the existing templates and then copying and then replacing it with what you want. Yeah. So somewhere in that file appears your logo, where it's in the image. Yeah. So question about the logo, where is it? I don't know if it's in this one, it could be. Is it SVG logo? Okay, that's just a spinner. So yeah, I don't remember exactly where it is, but it's probably somewhere there. Yeah, so if you search on the whole code base, maybe it would be easier, right? So now we've seen that. So I'm going to show you what we wanted to make. So this small video I mentioned before about the app we want to make, like the one displaying a map and so on, so we can remove this one, and we're going to switch back to JupyterLite for now. First I'm going to potentially remove any previous sites that were generated. So here, we're just putting Voicene aside for now. And let's go back to lab. Okay. So I'm executing this. This says... So we want to make a small application that lets users paste a bunch of UIs with a latitude and longitude, and we show them on a map using iPad widgets for the text input and also the buttons. As usual, just shift enter everything. Here I pasted an example so it's a bit faster to have a look. You can paste it here, and then click on Show, and it shows you the folio map. This is again all in the browser, except for, of course, the images that are fetched from the internet, but you could have them hosted somewhere else. Then you click on them, and they show up. Click on Save. I can actually put them somewhere in the download here, save, and then they end up here. And this is pretty much something you could give to your users, like if they want to create their own gallery or whatever, just make this little app in a notebook and then expose it. They can use it and they just leave home with their map.html file and then they are happy. they could, for example, email it to some other people. It's actually also saved in the files here by default, so you can also have a look at it from here. But a Jupyter Notebook interface, or JupyterLab interface like this is, can be a bit complicated to put in front of users sometimes, which is why we have Wallah, which lets you just take the notebook and make it as a web app, pretty much. And since I was having issues locally, and there is only ten minutes left, I'm going to show you the end result. So you're going to have to trust me on this one. So here, if you go to deploy, I can show you what happens. It's not the correct one. We have too many repos. This is the one. I'm going to post that on Discord afterwards, in case you want just the final thing, like everything configured correctly, you can just look at that. It started from the template, as you can see here, generated from the template. And it's pretty much just the same as what we did locally. I wanted to do it locally first just to get an idea of what it means to build a Jupyter 8 website. But fortunately, yeah, that's the one I want to show. Deploy. voici here we have the command to build the voici app and that's pretty much what's changed between the JupyterLite build and the voici build that's it okay and GitHub pages so let's this is deployed on GitHub pages so if I click on this It's redirecting to lab by default, but here if you choose YC3, yeah. We have the material template that the one is configured to use material template. This is what it looks like when you choose your notebooks. So that's the tree view of your notebooks, and then you click on that. We also set up the dark theme, and then you see it looks the same as Voila. But again, this is GitHub pages, things, there is no server, so there is no Voila running, it's just everything happening in a browser. So it can take a while to load. That's one of the major drawbacks at the moment. Yes, question. Yes? Yes? Yeah, so I'm trying to grasp the difference between Voici and Voila. Yep. And so is it that the Voici would be basically the, what is it, the Jupyter lab server-less version? Using if you want. Yeah, or any static website. Or any other static HTTP server. Yeah, OK. What was the project that didn't get the two comparisons? What did you compare? We were comparing Voila to Voici. Voila to Voici. OK, so it will be easier from here. So where are the links? OK, so first, let's finish with this. Yeah, it says Voila on the page, but it's not Voila, it's Voici, but still like Voila. Yeah, we like to make things complicated. No, I'm kidding, it's not true. Actually, we tried to do the opposite, but it's harder than it looks. So here you can do this. Well, it was also written there. So it's the same as what we had in Notebook, but now it's rendered as HTML5, so it says .html here. So, that would have been the output I would have gotten if I had done it locally as well with the Voicy build. But it's here, it's directly on GitHub pages, and I click on show, and it's going to show the map. Might take a while, I think, maybe, yeah, didn't check. I think the Wi-Fi was pretty good, actually, so, yeah, let's see. Yeah, and you get the same thing here, and you can save and save the map just like we saw before. So you can think of maybe internally at your company, you want to make a static dashboard and it's very easy to host. You don't want to put it on Kubernetes, for example, then you can just do something like this. Of course, as we saw in the beginning, there are limitations. You can't do everything in the browser at the moment. So it's always a matter of trade-off and the use case. To go back to the question first, and then there is another one. Voila against Voici. So as a reminder, Voila is just like like Jupyter Notebook or JupyterLab needs a Python server to run somewhere. So you have to, for example, start the EC2 machine or put it on Heroku or something like that or on your own server. And then it's going to take requests from this Python tornado-based server and run nbconvert on the fly to generate a dashboard like this one. With Voicy, it's different. It follows the approach of JupyterLite. So it's based on Jupyterite actually, it just extends it. Everything happens at build time, so this is the difference. You have to first generate your dashboards at build time that you want to distribute. We'll be looking later on how you can actually do that on the fly as well. Should be possible but it's not there yet. But everything will be happening at build time and you get a set of static files, a set of HTML files And then you can just take them and host them wherever you want. Yeah. Two questions, I think there was one there first. Maybe, yeah. Perhaps you already answered, but if you would you could analyze, is it possible to directly render the notebook in a parallel cut, instead of the local server? Yeah. Yeah, unfortunately not, and that's what I was, yeah, kind of trying to say just before, it's like you need to do it at build time, so outside of JupyterLite for now, but we'll be looking into being able to do it inside JupyterLite in the future, yeah, which is tricky. It could be tricky because it has to happen in a browser if you want to do this. That means, like, for now it's based on mpconvert, so that means also running mpconvert in a browser. It can be tricky, So like over the past year or so, we've done so many pull requests to change some of the existing packages so they don't have hard dependencies on many packages that don't work in a browser so they can be reused. So it's always this kind of work that needs to happen first so that you have something a bit slimmer that can be used in a browser. So I think in theory it's possible. But there is nothing like that at the moment. But yeah, hopefully soon. Yeah. Yes, a Python script you mean, a Python module, yeah. Yeah, so JupyterLite and also Voici can read from the files here. So if you create a file, actually this Python file. Yeah, you can pretty much say my function or whatever, return whatever, 42, and then you can import that module from here, or you can, yeah. Yeah you see that the files you get there are the ones that correspond to this, and Wasee is also aware of that, so there is some kind of file system mounting happening in So you can, the kernels are aware of the files you create there, and if you add a Python module next to your notebook, yeah, it will also be available, and you can also import it like this. Yeah. Fetch. Yes, you can do that, you can, yeah, everything pretty much happens at the CI or at the build time, yeah. Okay. So that was the end result. If you go back to this page at the very bottom, there are links of how you can do that. And there is also this template, just like the other one. But this one is for Voicy. So if you are just interested in making a Voicy app, like the last thing we saw, you can just start from that repo directly if you want. It's still based on JupyterLite. It's very similar. There are just a few things that changed, like I mentioned, like the CLI. And there's even a screencast that shows you what it looks like. And you can also use the pilot kernel, so in some situations it would be better to use a pilot kernel at the moment because you can install packages at runtime. And maybe some packages are available there and are not available in M2M4. Like I said, it's still a work in progress in some way. Python in the browser is getting better and better, but it's still early. And as always, yeah, if you have any questions, repos. only to open issues or discussions, and feel free to check the docs as well. And I think we are done. And another question. Can you also use it with poetry? I think you could use poetry, yeah. You can, well, not for the SUSE Python kernel, unfortunately. It's still really pretty much based on the Conda ecosystem. So you could use poetry to manage your build dependencies in your build environment, but I don't think it's going to work in your browser environment. One more. Yes. Yeah. You can use the fetch browser method. It's available also in the Python, at the Python layer, and you can fetch files from the internet. There are a few examples in the docs on how to do that. So if you go here, working with contents, yeah, you can even automate this even better, and you can even, okay, this is not about pulling a module directly, but you can even open notebooks directly on the fly using this kind of configuration. There are a bunch of guides there that show you different things. And if you think of another use case, don't hesitate to open an issue and we can add it to the docs or we can document that so it's more. I was thinking about several people Yeah. Yeah, you can Yeah, you can, I think you can just fetch them, and then you can save them on disk, and I say disk like this because it's not really disk, it's the virtual file system in a browser, and then read them from the notebook, yes. Or do it at build time, like you just fetch everything at build time, and then make them available via the contents. Yeah. And the last question. When you start to read, you have problems due to the size of what you report there? Yeah. So the more dependencies you add, the bigger you get, unfortunately, that's the drawback. So here again, there's a difference between Pyodide and the kernel, the kernel is going to get everything when you start the kernel, but at least you have everything ready after. In Pyodide, they're doing it lazily, so dependencies are fetched when needed, so it's also nice because then you don't download everything at once. So maybe there will be, I hope there will be a convergence at some point between the so that we have the best of both worlds. But for now, there is kind of split like that, yeah. So, yeah, network traffic can be heavy, still. That's the, yeah. Yeah, like I said, yeah, the browser might crash, and also there are other limitations on the size of the files you can store. So, I think it's 15 megabytes, maybe. something like that, I think it depends maybe on the browsers, but like index.zp doesn't allow you to store something bigger than 50 megabytes maybe, but it's enough if you have small data, but that's the kind of limitations you can hit, yeah. So I think we are out of time, thank you all for coming, and if you still have questions, maybe not right now because I need to jump on another talk, but afterwards you can just find me, no problem, yeah.