Python 3.11 in the Web Browser - A Journey Keynote

Speaker 1 [00:05]

That's the hardest part of my keynote, getting the output done. So hi, I'm Christian. I work for Red Hat as a principal software engineer in security engineering. I'm an all-time Python developer, as Alexander pointed out before, a core developer. I'm also engaged in the diversity inclusion working group, and I'm also part of the security team. With this one, so this presentation, I'm going to do something different. wanted to explore a new part. Actually, I'm very happy exploring and going out. So, this is my speaker collection at home. This is behind my desk. So, I love to engage with the community, explore, like, new countries, and I highly recommend that. The Python community is very diverse. So, last time I gave a keynote before a big audience, before my keynote, I got a Bollywood style welcome dance, like 20 minutes in India, fantastic. So, to share just one more personal information, but please keep that in the room, I hope there's no recording, I have one confession to make, actually, I have no clue how browsers work. I'm a back end security developer, I have no clue how front ends work, JavaScript, well, never learned that. And it's a good language. And I just started to learn WebAssembly a couple of months ago. I'm really new. So just to revise the actual title, so we're talking about CPython, not just Python, CPython. And WebAssembly, I'll also talk about browsers but more about WebAssembly, and this is really a leading-edge thing. So when Alex and I discussed the keynote, he came up with a journey, it's very omnis, can mean anything, means we're at the very beginning. How would you actually run Python in the browser? There are multiple ways. The easiest one is probably run your Python on the server and just pipe the output via a web socket. Or maybe you just rewrite Python in JavaScript. You can also do something like transpiling, so converting Python code to Java code at build time and then run the Java code. Or something like browser plug-ins we had in the past, but all the browsers have deprecated the API because we had something like Adobe Flash, which was a series of security bugs. But the fun part is now WebAssembly and Emscripten and Pyodide. So WebAssembly is an interesting platform. So here's a talk by a friend of mine, it takes like two minutes to figure out how to compile CPython to WebAssembly if you know how to do it, but it takes very, very long to get something very simple like reading from the keyboard and printing out to the user. It took like 28 minutes in the talk to explain how that works. So last month, last year, in November, Ethan and I started to hack on Python in the browser in CPython, the core. And this was like the first attempt, end of November, like we were hacking for a week or two on that, and it got it actually working. Well, I was still struggling with that output, which is actually lots of success. So this is a panic by the Python interpreter. You see on the right, like, HTML. do something in the browser, and it can't import the encoding model. The encoding package is the first thing where Python leaves the C space and tries to import something from the file system, and that failed for me, which was lots of progress, actually, because most of Python was already bootstrapping and running, except the encoding parts and the rest, but, well. Just a couple of weeks ago, we got here, we have now a working REPL that's committed in C Python main, so what's going to be 3.11, and we can actually run all our unit tests on the Node.js, so we can compile Python with the JavaScript bindings and run that in Node.js and have all the tests passing, except we skip a lot of tests that don't work, but okay. Let's say effort by lots of people. So Ethan and Brett from CPython, Alan helped me with the build system, Katie did the web UI, the people helped out, I got lots of patches from them integrated into CPython, the mscripten, SDK people, especially Sam, we had some early adopters, Paul and Trey will show something, and people who probably don't know they contributed, especially Lynn Clark. If you want to know more about WebAssembly, I highly recommend her talks. She gave a lot of talk about WebAssembly and Vasi and similar topics over the last couple of years. She's fantastic. So how does WebAssembly work? Let's dive a bit into WebAssembly compilers and operating systems. So most of you think that browsers have JavaScript as their main language, which is apparently no longer true. most, basically all modern web browsers these days, support two main programming languages, JavaScript and WebAssembly. But why did the vendors like Mozilla, Microsoft, Apple, Google and others develop WebAssembly? Well, they had the same issue as we as Python developers. Python is a very dynamic language, JavaScript too, so they had a hard time optimizing their code, while we, the Python community, can use something like C extensions and native code binaries, they can't do that, because browsers work on lots of operating systems, lots of CPUs, and they're also sandboxed, and sandboxing C code is, well, it's very hard. So they came up with the idea to define a new language that was optimised for the web. And optimised even so far, these days you have game engines like Unity and Unreal working in the web browser natively. It's fantastic, without any plug-in. So it just uses standard web browser APIs. And not just web browsers. You have also Node.js, and you have runtime that don't use any JavaScript, like small, like 19 megabytes of Rust binary, like where you can run special kinds of WebAssembly. So how do you compile that? Let's quick look how compilers work on a very, very high level. So compilers do something like they take human code. Yep, C is considered for humans. And turn that into native code. Something can run on, like, CPUs. They do it typically in two steps. There's a front end that creates some intermediate representation from different languages. And then you can do some optimization on that. And then you have back ends that convert this ER to the actual machine code for your x86-64 CPU, or your ARM CPU, or whatever. So they separate their concerns. They're specialists who work on machine code and CPUs, specialists who work on C, C++, Go, Rust, and whatever. That's how a typical compiler works. So what it does on the front and the back end, so if you take a simple function, this is a probably know that from school math, That's a slope and line with an offset, so A multiplied by X plus B. In C code, it looks like that, so you have to give him some types, but the rest looks very much like Python code, so probably everybody can read that. To convert that to machine code, it gets a bit more complicated depending on the machine. So there's an example like for your standard CPU, Most of you probably have x86-64, which uses some multiplication addition using SSE registers. If you look at the Raspberry Pi, it has a VMLA operation, which is multiply and add, so they can do both in one machine instruction. Then they have to move some stuff around to different registers. Mainframe computers do something differently, so it's hard to read, hard to understand. I don't understand that much, just a little bit. And also how to port. Every computer has different instruction set architecture. But there's a simpler way. So much general approach for that is a stack machine. Stack machine is like a stack of cards where you can only look on the top. You can add cards to the top and remove cards to the top. You push and pop. To implement any kind of primitives, you can only do that on the stack. So you want to do like add. You pop two values from the top of the stack. an operation and push the result back. Same for adding and multiplying. You need some way to actually get these things on the stack, so if you have a function, I'm just hand-wavy here, don't want to get too much details, you need to load, like, things from locals, like function arguments into the stack, and then you can program your stack very easily. You load two values, do multiply, it removes two values, pushes one back, load the last value, the b, add it, and you're done. And the last value on the stack is your function result. So you run it in Python code, you get 17. It's much easier to understand if you don't know how registers work. You probably guessed why I showed you that, because if you look at WebAssembly, it should do the same. That's how you would write WebAssembly in text. They have a text format and a binary format. And that's WebAssembly. And not just WebAssembly, if you disassemble Python code, it looks very much the same, because we also use the stack machine. And in 3.11, we're going to have some optimizations that we actually have operations that can do fast operations on floats. That's one of the things that are going to be added or has been added for the faster Python effort. If you want a two-minute introduction to stack machines, I recommend David Beasley's keynote for PyCon India 2019, where he wrote a WebAssembly browser interpreter and runtime in 45 minutes. I'll have links later on the slides. So back to compilers. So we can do something like get from the R to WebAssembly. Just consider WebAssembly as another kind of machine code. we need a bit more. We need also to look into how we're doing stuff with the world around. So we need to, like, talk to the operating system. Like, we want to load a file from disk. So we use in Python, you open and you have some disk somewhere, and, well, in Python, you go through several layers. Eventually, you open call. Well, that one can't directly talk to the hardware because you have, like, a wall between that. That's the operating system that does permission checks and provide facilities for that. So what you typically do, you talk to a Lipsy implementation which goes from open to an open add function to a syscall that then does some magic and goes into kernel space and the kernel space has a table to handle the syscalls, goes to a virtual file system called file system, some block devices, SATA, PCIe, whatever, to your disk, or to NFS, or to Sambar, or whatever. But how do you do that in a browser? And how do you do that in a way that works everywhere? Because every operating system, even every Linux platform, have like this in Cisco, numbers. And browsers are sandboxed, so you don't want your random web page like read your private key or your emails. I think the simplest approach for that is what everybody does. We make a standard, create an abstraction layer, call it an API, and let somebody else figure that out. That's what WebAssembly does. So WebAssembly itself has no clue how to open files, it just delegates that to the runtime, so the people like the browser. bit more complicated because actually it delegates it to JavaScript, so if you compile Python to the web browser, you have like a kernel-like JavaScript wrapper that provides kernel syscall interfaces which then runs inside the browser, the browser does the rest for you. Where you have WASI, then the WASM runtime will take care of that. Elegant. Very easy. It looks a bit like that, so if you read the JavaScript code, there's like a table for syscalls, and you create the web assembly with this table information, run it, and then have some more JavaScript code that provides, like, different virtual file systems. For memory, you can use actual files or IDB if you have a persistent file system in the browser. That's how it works. And I think we need to just add here, I'll need that later, is something called a target triplet. So we have a writing system, we have your CPU, we may have different lipc, like gdipc or a muzzle, and we sometimes have vendor specific changes, and this we typically interpret and represent in a triple, which can be two to four values, it's a bit confusing, but yeah, it's like CPU, unknown means just anything, they don't care, Linux, GNU is GDBC, et cetera, et cetera, et cetera, there's Windows, there's 64 Apple Darwin, that's OS X on an M1 chip, and so on and so on. And we have the same for WebAssembly and for WASI. So we have WASM, the CPU, and we have the operating system, which is either Emscripten, which we use in the browser, or WASI. Just to summarize, here what we have, we have WebAssembly can be just viewed as its machine independent machine code, you typically as an end user interact with WebM, that's the binary representation, you can also for debugging or just for playing around use what, that's the WebAssembly text representation. It's designed to be portable and very fast, so you can get close to native instruction speed. The browser does very interesting things to speed that up. WebAssembly has only, like, four primitive data types. It has two kinds of floats and two kinds of integers, and that's it. Even something like a string is internally represented as something that looks a bit like a pointer and the length of a value to a memory array, because in Wasm you even don't have memory. Memory has to be right as an array inside your runtime. There are just about 200 instructions, it's a very small instruction set, so operations like add and multiply, but also shifting, and some operations to do flow operations, like if the value is greater than zero, then jump to some other instruction. There are also optional instructions, like SEMD, so vector operations, which are interesting for people like NumPy, they can speed up numeric operations, other features are also optional, and you have, again, you have these imports, the exports, and the memory. The imports are things that need to be provided by the runtime, like opening a file or handling memory, and the exports are things that the browser can call from the WebAssembly model, And that's WebAssembly, so not that complicated, actually, but, yeah. So next up, the Python build system. So what would we do to get Python built on the WebAssembly? So if you ever have built Python, you probably run something like Configure and Make. So on Linux, on Windows, it's totally different. So you run configure, you get a bunch of output, total gibberish, and you also now see again what I marked here blue, the platform triplets, so I'm building on x64 PC, so PC doesn't matter, can be unknown, can be something else, on Linux with g2pc. And after that, I run make, run compiler, compile something to machine code, and eventually gets the Python decreter, and then runs setupPy. So it runs dist.hills to build the extension models. So the config.script, the first step we do, it's using autotools and m4, does feature detection. So it just runs the compiler, looks for files, looks for libraries, et cetera. So you see some examples on the right. For example, it looks like for a header file, which defines APIs. It looks if the GPC provides P threads, Sigmas function. It looks for real-time clock information for very fast and very precise clock. Or it does some WebAssembly specific changes. So it looks for the system, which, for example, could be Emscript, and then it knows the binary that comes out at the end as a JS extension. So it detects all this information, creates a header file, pyconfig-h, to create a makefile.prefile, which is the first stage of the makefile. Then we pass additional information for models, so these are extension models, so like standard library C extensions, we finally create the makefile. the make file, then compile the rest of Python. So we run make, new make, mostly. First thing we do, we compile a very, very minimal core. It has just the basic data types, and it can open files, but it can't import anything, because we have the import system in CPython, it's written in Python, but we can't import the import library without having the import library in place, So, what we do, we take the import library Python file, compile it to Py C, convert the Py C file into a header file and embed that in the C code. So we have like Py C byte code inside your C code somehow. And that's what freeze model does, the first step. And then there's something new after we have the import lib working, we can import some libraries in Python. Not all of the standard library, but a small subset, but you can actually do something useful with Python now. So we create a minimal bootstrap Python interpreter and we deep freeze some parts of the standard library. That means we embed something like the OS model or OS path inside the binary which makes startup faster. There's a new thing in 3.11. After that, we can also compile the rest of the built-in models. Then we get the Python interpreter at the end, and finally we can run the Python interpreter to build all the shared extension that is not linked into the main binary but all linked and imported then on demand. These are usually big things like my SL bindings are external. Yeah. So if you want to cross-compile with something called amp-scriptum, which is the SDK that compiled C code for the browser. We also need some information. So we need to know what's the build system. This is something that obviously confuses me, too, about build and target host. It's why I tried to make it more clear by underlining that. The build system is a system I'm building on. So my laptop is a Linux 64 computer. I want to target 1132 unknown. That's I need to have a cross compiler, so something that takes C code and compile that for a CPU that I can't run on my computer. This is a very cool concept. So I can compile code for a CPU I can't execute on my hardware. That is a very old feature, but also used to bootstrap new hardware. The MCC internally is a Python script, a very large Python script, which runs CLang, LLVM, a tool called Binary N that's part of the WebAssembly tool chain, and Node.js internally because it also compiles JavaScript code. We need a sysroot, this is just a change root which contains header files and libraries which are already precompiled for M script, so like libc, we need that, it's in there. we have two wrapper scripts that just set some end bars and automate some annoying steps, so mconfigure and make, so you do like mmake, make, or mconfigure, configure, and yeah. Well there's a small problem. We have a freeze model and we have run Python, but we can't run that because that's running on a different CPU architecture which you can't execute. And now comes in the things that we need to fix to make Python work on the web browser. We need to figure out how we can actually freeze things, so free the import library and build all the extension models. And this was the first step that Erland and I did to get CPython working in the browser and do the compiling step. So Emscripten and Pyodide had that figured out before with some manual steps and patching. You want to have that more automated and in a way that you can integrate into CPython. So we changed a lot. The third thing, we had to teach the whole system about WESM32. They didn't know about that. They could only compile to some other hardware. We introduced a new option to specify a build Python. So we built Python like two times. I built a standard Python for my own operating system, which then is used by the other parts to compile the cross-compiling. Because, as you may know, if you have PyC faults, they're specific to Python version. You can't take PyC faults from 3.10, import them 3.11, and even doing 3.11. So faster Python, the effort by Guido and Mark Shannon, they changed the bytecode like every couple of days instruction set. So we had to build a Python interpreter to bootstrap the whole process. We also moved lots of detection of standard library models inside the configure script and now have a tooling to build almost everything, like three models are still missing, to build them just for makefile, so we don't have to run Python to build shared extension models. So some improvements like config cache, it caches all the information we detect because a config around with M script takes like five minutes or four minutes and it's very annoying if you develop Python, you don't want to take four minutes to rebuild all your tools, so this one shrinks it down to like half a minute or so. Out of creed building is an interesting concept I'll explain a bit later and also a target. So we target different kinds, flavors of M script and we can run it in a browser or we can run it inside, like, Node.js. And so this is how it looks like, the new things we added to detecting models, so this information is now embedded in the make file. Probably not a lot of information of the users of Python, but if you're an embedder or distributor of Python, yeah, it makes your life much easier. So let's build Python for the browser. So the first thing you have to do, clone the repo, create two directories internally, output directories. And then we first stage we have to build the build Python interpreter so we go inside the build directory and then run the configure file from the root of the checkout. And what it does, it keeps the checkout clean and puts all your new generated file, like the output and the binaries inside your directory, and you get a Python interpreter at the end. Then you change into your other directory, run this very complicated command, which I explain in a minute, and run make, and then you're good. So what this command does, you see here, so we specify and use the bulk interpreter from And the other directory, we also have a file that overrides some of the detection because some of the detection does not work correctly, so we have to tell Python, no, this function do not exist, or it will fail. We define, okay, I'm building on Linux, and I'm targeting WebAssembly, and I'm targeting WebAssembly in the browser, so it does not compile in, like, threading support or Node.js, rawfs, which doesn't work in the browser. In the end, you get a couple of files, you get the actual webm file and the JavaScript file that's the operators, wrappers, you also get a worker thing that has to do with how we run the HTML file, which is the console, and the Python.data file that's your standard library compressed into one single file, and that's it. Or you want to add simpler, I have a build container and a couple of scripts, check the build container out, it's very large, like one gigabyte of download, and run that. Oh, and one other thing, you can't run that from the file system. You have to run a web browser, a web server, sorry. You have to run a web server, because the browser doesn't let you load WebAssembly code from the file system for security reasons. Or you can go to our repo and just download the assets. They're pre-built every day, I think, or you just go to any of these URLs. It's Nathan's output and my backup output in case something breaks. And then you have Python in the browser. I'll show that. So we have demos at the end. I don't want to switch between displays too often. Yeah. So. Cool. So what we actually, that's it. I'm not finished yet, I'm not finished yet, I'm not finished yet, so it's too easy, the demo is coming in the end. So I'll just quickly show things that we actually did to fix CPython, so we had like 64 requests, There's some very annoying parts. This was one that took like two days to figure that out. A function signature mismatch. So don't want to get too much detail. So this was like two days' work for this small change. You can read that up on the ticket. So WebAssembly is an interesting platform with several interesting properties how it works. We also fixed a lot of bugs in the MScripten package, it turns out that CPython is a very good test suite for POSIX compliance. So time things were very strange, like crash, segfaults, fun stuff. And all the interesting stuff, so I had to write on JavaScript code that looks like kernel code and write the test for that in C. So, cool, now we have Python working in the browser in Node.js, so what actually works? Well, not much. So if you run it in the browser, you don't have any processes, you don't have threading, you have threading in Node.js, you don't have file systems, you don't have PyPI packages, you don't have signals, you can't do anything with the web, that's also, we don't know how that works yet, but another project is called Pyodi, these are the first people who actually created CPython for the web browser. They have figured out much more for some of what they do instead of writing their own socket library and having this socket emulation. So in Emscripten, you can do something like you pretend to be a socket, which internally gets wrapped in the WebSocket API, which then gets handled by the browser, so you can do raw TCP. And then in Python, you can layer SSL on top of that and your own HTTP library on top of that. Well, what they actually did, so what do browsers best? HTTP requests. So they just wrap the web fetch API and, well, that's easy. So that's the Pyrolight project. That works. So some questions I get very often The most frequent question I get is can you actually now use JavaScript into a Python to build your web frameworks? It does not work yet. So we don't know how to access the DOM from Python yet. Also it's surprisingly small. So if you don't want that compressed, it's just four and a half megabytes. It's actually good. It's not perfect, but it's still a lot of data you have to transfer if you're mobile. It's surprisingly fast, so we haven't done any optimizations yet, but it's like depending on which kind of workload you do, it's like one and a half to three times slower, except if you use lots of memory, then it crashes. We also don't know how we're going to support that, so in CPython, we don't have the resource and knowledge to support WebAssembly and browsers, because we are like C hackers, not WebAssembly hackers. So our idea for that, or my idea for that, is to just build the foundation and then have the Pyodide project to actually build useful bits that the community can put on, which, for example, the NumPy SciPy people did with JupyterLite, yeah, and Pyodide. This is the project that we took lots of ideas from and worked heavily. In 3.11, currently, the last release, there are two patches missing and then Pyodide would be compiling to WebAssembly and the browser without digital patches. For 3.11 final, I want to have it in a place where Pyodide can just build on top and we take care of all the build system issues and they focus on providing high-level interfaces and the useful bits so people can build on top of our apps. So why did we do that? So what kind of things will we get from WebAssembly? So what's the future? So some ideas before I get into demos and the question and answers. I think the most interesting part for lots of people is going to be education. So if you're a teacher, if you're like an instructor and teach Python to beginners or to kids, the hardest bit is to install Python, so you need to tell them how to install that on Linux or on Windows, you need to tell them how to build and get your dependencies, you may get permissions issues, if they only have a tablet with Android or iOS, it's hard. But having that in the browser is super convenient. We also can do something like executable documentation and executable examples. I'm not sure how many of you know Trey Hannah. He's an educator, teacher, and has these Python muscles. So he created a paste bin where you can paste Python code, click the run code button, share your code, and then the people can actually look at your paste and run it in the browser and just execute your examples. Something that we're considering in CPython is to have like the doc string, so we have like documentation examples in documentation, have that runnable, so you can just have people run the examples, modify the examples, run it again, get the output, and just explore Python on that. What I hinted before, Python on mobile, there are like ways to install packages on mobile with Python, but just having the browser would enable people to immediately download like a tailored distribution with some examples in the browser. And not just simple examples, there's actually something called JupyterLite, which Jeremy Tudor, I hope I didn't butcher your name too much, I think it's French, the Jupyter community has JupyterLite, which is a Jupyter with a Python kernel completely running in WebAssembly in the browser. Lots of cool things. So, with the talk yesterday, I recommend to check that out. Other things you can do, like, there's something called Electron, which runs, like, Visual Studio Code, or there's VS Code Dev or GitHub Dev, so web IDEs, or IDEs that can only run in TypeScript or Node.js. They don't have Python yet, but they could just have a button to start Python Shell, and you have a Python Shell, and you can develop your Python code and execute it in your browser without installing anything. You can even do something like debugging, how that works. So you can actually debug CPython codes with a JavaScript debugger that's in Chrome now. Step through the code, step through the stack, and just explore how it works. That's fun. The mscript and the SDK we're using, they have some very interesting libraries here. They have something called WebGPU, or like SDL, the single direct media layer. Maybe you can PyTorch in the browser without installing anything. Or maybe PyGame? Yes. Yes. So this is one of the first people who adopted CPython. This is a hack of Pygame and CPython from the latest Git running like Pong or another game I'll show later. You can maybe see me losing totally on stage. So fun. So Pygame works in the browser. So kids can develop games in the browser, run them in the browser, explore Python, learn on Python without having to install anything on the computer. Totally different approach is WebAssembly system interface. Anybody running like web services or like Lambda similar functions. So WASI is something you have sandbox secure isolated processes that are limited what they can do and what they can access. They're very fast to start and stop and they're designed to do edge computing. So pushing your workload from the data center into smaller data centers closer to the customer, even like mobile antennas and self-driving cars. And we're working on that to get that also done. And something I'm just amusing about, I'm not sure it's actually possible to do that, is the Python binary extension hell. So if you have C extension like NumPy, you need to build that for a lot of platforms. And these are just three platforms, three different kind of CPUs and three Python versions. So could we do something like compile C extension to WebAssembly, have a WebAssembly runtime in C Python, and have like one binary that works on every operating system, every CPU in every Python version. Maybe? I don't know. Well, you can do that a bit already. There's a Python bindings where you can run WebAssembly, import a model, run your WebAssembly either from the code or you load the file, and it works. So, Python WebAssembly, lots of opportunities, Lots of work to do. The first thing we're going to do in CPython now is we will have a build bot that verifies the WebAssembly build every day. And we're going to consider, I will present it in two weeks of the Python Language Summit and Salt Lake City at PyCon, how much support we can get from, like, PSF and the other core developers to make that stable. Yeah. Cool. So you may want to take a picture of that. That's a lot of resources and slides for links and videos. I will put that on the slide, and on the right side, there will be a slide deck. And now we can go to questions and answers while I do some demoing of some features. Let me switch over to a different slide. So, Alexander, questions and answers? Do you have any questions? I hope so. Probably. Okay. Keep that. So. Okay. That works. Perfect. So WebAssembly, what you can do, just some show of, so 3.11, so 3.11, 3.11, that's the wrong 3.11 but still 3.11. That's Windows 3.11 running in a dust box implemented in WebAssembly running inside to the Internet. One second. Running inside a virtual reality environment. So, yeah. So, let me start my first browser I've ever used in my life on a GUI. So, this is now running Netscape. This is the grand, grand, grand of Mozilla inside a virtual reality environment inside a dust box in WebAssembly on Firefox. That's not my work. So I'm totally innocent there. But this is the work we've actually done. So... Oh, Internetworks. Hi. So this is Python in the web browser. you can start to ask me questions if you have anything sure hello

Speaker 2 [40:22]

Hello. Hello.

Speaker 1 [40:24]

You can't move those.

Speaker 2 [40:24]

So, the first question is, why should we run Python in a browser? Yeah. . Could you motivate some of the use cases that are unlocked by this capability? How do we mitigate security concerns?

Speaker 1 [40:38]

So, security concerns is their web browsers, and web browsers are designed to be sandboxed and secure, so the browser vendors have heavily invested into security, and even so, if you run WebAssembly, so that's why you need to run a web browser with special headers to run WebAssembly, because you need to use some operations like set a policy for cross origin border something, I'm not a developer, but you have to tell the browser that the actual window can't communicate to other domains, so the browser does a lot of things. For some of the motivations, I'll probably mention them before education, just you start something in the browser and have it working, you can reuse code in the browser and talk to others, do prototyping.

Speaker 2 [41:32]

Hey, next question. How was cross-compiling WebAssembly different from standard cross-compiling into an ARM target? You could also not execute the compile Python.

Speaker 1 [41:46]

It's the same, actually. So we had cross-compiling in the past, so that's Pygame. Cross-compiling in the past, it was broken because it was never tested. So but internally it is the same problem you have. If you cross-compile to ARM, you also need to build a build Python interpreter now with the new system and have to byte compile your import library, for example. So the same problem we had. The only difference is that if you build for ARM, then you get a binary you can execute on ARM, but if you build for C Python for the browser, you get a JavaScript file that you can run on the browser, and the embedded standard library, because built from the browser can't access the file system, so you have to bundle, and it's called preloading. You preload the center library compressed into one single file inside the browser. That's why it's like five or six files, and you have also this worker thing, web worker. So with the console I showed before, that one, so it does not run Python directly in the main browser thread, it uses something called a web worker, which you can't, no, you can't show that here on this browser, you can't show the yellow browser. It runs it like in the browser thread, and that's also the other difference.

Speaker 2 [43:19]

Next question is, does it mean I can use NumPy library in Node?

Speaker 1 [43:26]

That was a good one.

Speaker 2 [43:27]

Very cool, that's very good, that was quick

Speaker 1 [43:31]

It's probably not easy to use it, but in theory, it's possible, yeah. So it's in there. So if you have Node.js, but you would run a Python data breacher and then figure out how to talk from Node to the Python data breacher. That becomes a bit complicated because, again, the very beginning of my talk, it took Katie 28 minutes to explain how to do something like std in, std out from browsers, same with no jazz. But if you write enough tooling, yeah, it should work. Cool.

Speaker 2 [44:05]

Can this in the future also be used to solve the packaging problem of Python? Will I be able to ship my service via WebAssembly instead of a container?

Speaker 1 [44:16]

Yes, yes. There's a statement by one of the directors from Docker who's told that if they had WebAssembly like when they started Docker Inc. they would have gone a totally different approach and just go for WebAssembly instead of for containers. Because you get both the works on every platform for free and you get the security property sandboxing for It's still a lot of work to get it working, but, yeah, it's in there, it's baked in. And the design is so that it's security first, while with containers, it's always a bit complicated to reason about the policies and which syscalls you allow and how you do filtering and maybe it's a Linux. It's easier to reason about that with WebSimply.

Speaker 2 [45:09]

If we cannot replace

Speaker 1 [45:10]

replace.

Speaker 2 [45:11]

JavaScript with Python and the main advantages is accessibility are there are there is easier ways to achieve that I'm just reading the questions

Speaker 1 [45:23]

So we can't replace JavaScript with Python yet. It may be possible in the future, but probably also not advised for all use cases. So what you have to do, you have to download at least a couple of megabytes of data, like the Python standard library and the Python interpreter. They can't be shared across, like, origins, so there's security policies that even if If you download the same URL from two different sites, the browser will download it two times so that one side can't do fingerprinting on the other side for security reasons. So it's a lot of data transfer. It's a lot of memory consumed, and performance wise, it's native JavaScript directly doing DOM operations is in my gut feeling probably more efficient than going through the additional layers of C Python call and this back and forth communication.

Speaker 2 [46:24]

Last question. Does your effort help in any way with cross-compiling normal C extension modules?

Speaker 1 [46:31]

There have been people, the Conrad people are working on mscript and forge, I think it was a lightning talk yesterday.

Speaker 2 [46:42]

I didn't make it yesterday.

Speaker 1 [46:44]

I didn't make it yesterday. I didn't make it either. So about mscript and forge. So the non-py people, the conda people, the conda forge is working on that because they are also interested. So they approached me two days ago. So talk to up the stage? You, yeah. Show your face. Yeah. Probably want to talk to you. Thank you.

Speaker 2 [47:09]

So please welcome Jeremy on stage.

Speaker 1 [47:18]

This is your expert yeah, just a quick word that it was indeed Tosun during the demo yesterday during the lightning talk so if you want to know more about it, just Find him somewhere. I think he yeah right here On the right side left for you. Yeah, you can come So you can ask more people questions not just me because I'm a single part of it. Hey, I'm that guy. Yeah To your face or show you mask

Speaker 3 [47:48]

So yeah, we can cross-compile stuff now quite convenient with the whole CondaForge cross-compiling capabilities, so we didn't have to fix that much or change that much, but now it works more or less the same as if you compile for your machine, so you just run Python pip install and that's it. So that works. And also you can access the DOM from Python.

Speaker 1 [48:22]

I have no clue.

Speaker 3 [48:24]

So, in Emscripten, you can access JavaScript, and, well, from Python, you can wrap the calls to Emscripten, and then you have it. The fundamental problem is rather that Python is more synchronous, and JavaScript is more asynchronous. And you have the problem that you're blocking the main thread and so on, so I think it's more playing around to have websites purely in Python than anything in production, but it's still fun. Thank you.

Speaker 1 [48:59]

I think I've ordered us over time. No, we're basically.

Speaker 2 [49:01]

We're basically on time. Thank you very much, Christian. Thanks, everyone else working on this. Thanks for your work.