Roll for Architecture: DungeonPy – A D&D Companion as Server + Thin Clients

Speaker 1 [00:06]

To begin with, just a basic admin thing. If you have any questions, please ask in the talks.pycon.de. Then it's easier to do questions in the end. And now let's welcome Francesco and give him a big round of applause.

Speaker 2 [00:30]

familiar faces and unfamiliar faces here. I hear a bit of a, is it fine, the audio? It's okay. So, welcome. I'm Francesco. I am very obviously Italian, and this comes with consequences, like I just usually walk around, etc. I promise I stay here, but you might not suspect from this talk, but I'm also an astroparticle physicist, even if I try to transition into industry and on to other things that I am I am a proud nerd is anybody offended by nerd nerdiness or something it happened in the past okay great so yes I'm a nerd and as a good nerd I like coding and I like role-playing yeah so is anybody here not aware in this case D&D means Dungeons and Dragons. Okay, so yeah, D&D, the most famous fantasy role-playing game. We call it role-playing game because, you know, there are a matter of roles. There is one player that assumes the role of the storyteller, the world builder, have full access to rules, to to the world, basically, and there are the other people that are the players, each of them impersonates a character, so to say. And this is not really acting, but it's something that has a lot of similarities. And the net effect is that you get to know your pals pretty deeply, because you share a lot in this interaction. But why would I come here to tell you, no, to talk about D&D? The point is that, if you remember 2020, that was a weird year, and basically sitting together became kind of less mainstream around the table, and so the friend told me, you know, you're the nerd, figure it out, make it work, and I started thinking, okay, and the answer is, well, basically, it's talk, right? It's what you see today. So, okay, I see people started wondering whether there's going to be any fight during this. Yes, it starts here. So, I thought about it and I found my ansatz, which is, tabletop replay games are secretly distributed systems. Think about it. There is one canonical state, right, made of things that you can actually describe, dungeon, the map, the creatures, the players, secret doors, whatever, and so on. There are many asymmetrical clients, each of them has its own take on the world, the player and the character, right? And no player is the same as any other player. And trust me, I played with twins. They're all different. The third part is lossy links. Not only, I'm not talking about network, not only. The point is that players are untrustworthy. They miss information, they misremember, they misinterpret, and in general, their detection span is horrifying on average. And four, we have a strict access control that needs to be enforced by the Dungeon Master because it needs to deliver tailored, custom-made truth to the clients. So you strip all of this from the fantasy wrapping. What you have is typically the description of any other application needs to serve multi-client service, right? And then I said, okay, Python can work to describe this kind of, to find a solution like that. So what do I need to Okay, first of all, the first thing I need is a tracker. The tracker basically is a sorted list where all the characters are sorted by the initiative value, which is basically a value that tells you how quick they are to reply. So the higher the value of the initiative, the first two act, and they act in turns until the slowest one, and you start a new round and so on and so forth. This is an evolving situation, so you want to keep track of many things, and you want to anyone, for instance, who is down, who is wounded, when are the spats expiring. So Adventure Master really needs something like that. So it's easy to do in Python, super easy. For instance, you can do it with the wonderful PySimple GUI, which is super easy to use, has a very steep learning curve. This is just an example. We were going to put it later as well. Super easy to use, and I mean, I didn't know it in 2020, but now I'm addicted because you can do many different things with it. I had to divide the tracker in three different slides because else you wouldn't be able to see it, but in general I'm not talking about the ND today, sorry, we can do it later if you want, but not now. It can do many, many different things. For instance, it can handle private chats, which are kind of vital in this situation because you want to deliver partial truth and to have secret communication with the players. Anyway, the other thing that you need to have, of course, is a battle map. And yeah, it's beautiful, isn't it? When I started, I didn't think it would be so nice with so little effort. I would have settled for less. Everything is clickable, movable, placeable, lights, light sources, objects, creatures, whatever. And this, I used Pygame. So this baby is a bit more complicated to start with, in a sense. The learning curve is slightly less deep but in the end it lets you deliver with a kind of a moderate amount of effort and i was super happy with this i mean so put it together by game the button map and by simple ue as a tracker i mean that's what you need as a dungeon master but we are still talking about my machine okay so how what do you want to do for instance you want to click on a map and you want to select it on the tracker click a character on the map and you want to character select on the tracker. Or the other way around. Select it on the tracker, on the table, and you want it active on the map. How do you do that? Simple TCP messages. You click on the map, it sends a message like, Arial selected, or active, or nothing is selected, which is basically the entire protocol we have. And the tracker says, okay, acknowledge, oh yes, it's selected. Something like that. Super easy, super quick, kind of, you know, kind the, I would say, basics. Now, it's only my machine and it's already, I already see a few very important, almost critical questions. First of all, if the tracker can instantiate something, like I click on the tracker and it sends a message to the map, or I click on the map and it sends a message to the tracker, and they are sharing some kind of list, for instance a list of combatants, who owns the truth? Because they don't own it together, it's not synchronization, right? Imagine doing it, exporting it to a number of clients around on the network, this becomes just crazy. This cannot be, right? And this is a problem that needs to be solved. Secondly, in general, if you want to use, like basic or whatever, the frameworks are definitely not thread-safe. You don't want a background thread to write directly on the widget, or in the end it will corrupt. Among other things, imagine that we have a race condition among different clients. So no. Another thing is that, of course, you click on a tracker, sends a message for a selection to the map, the map acknowledges, and then sends a message back, oh, I selected this character. Back to the tracker. The tracker is not trained to say, yeah, yeah, I know. It was me, Mario. No, it just sends a message back. And you want to stop this. So this is an easy-to-solve program in Python. But at the same time, this is an issue of you need to handle external events. Anyway, nothing of this is a D&D problem. These are all problems on any multi-environmental app, multi-client app, right? OK. Before getting to the server, what is a state? Now, we had two Jewish sharing a list, but I mean, that's not broadcastable to many clients. So you need to write a state. And my choice was, for instance, to instantiate every combatant as a data class. So you just can use toDict, fromDict, safe. You can think of the map as a grid of code. And then you can just dump everything, and not only the combatant and the map, but also the light sources, all the information, all the indexes of the activity on a JSON. And that's your safe game, as a matter of fact. That's your state. That's everything you need to know is there. It also allows you to handle all saves, because it's so easy, the structure, it's so linear, that whenever you load an old save that you created before actually implementing the light source in your code, on load, you write a function that checks the save, fills all the information it might be missing, because it was missing those kind of implementation, and that's it. You can use it. So simplicity, that's what I need. Don't forget that I did this because I wanted to play with my friends. got out of hand so okay it's ready to go on a server uh my policy is to use an authoritarian server which basically is a server that um only accepts uh when the client asks for permission not the server decides so a client asks can i move somewhere there no the intent is processed, and it checks. First of all, is it valid? If it's not valid, it's a silent rejection. Let's not talk about it. If it's valid, go on. Then it applies the move on the server, and then it broadcasts back to the client, as it should be for a multi-client app. This is an extremely simple structure, right? But this is true. Normally, that's what happens. And all the conflicts between the clients disappear magically by construction because the intents do not touch anything. They're just sent as an intent to the server that needs to queue them somehow. But no more conflict or race condition between the clients in this situation, right? One message at a time. How do you do it? With something like this little creature here, we know what regimen value. So here there is an unsafe way to update a widget in a safe way. The unsaved way is you have a GUI that sends a background thread that updates directly the window with a message. And this is madness because all the clients want to do that at the same time. And if any of them think that they can update a widget at the same time, it's going to corrupt them, of course, because they keep on updating and updating and updating, not at the same time, not directly on the widget, on the save game, for instance, or on the tracker. The best way is when the message gets to a queue system. So they write the event value on a queue system, the queue system then is handled by something which is able to handle cool routines, like I use an async IO, on a main loop, and then one by one, slowly, reads all the socket messages and applies them. This is the safe way to do it, but you don't let background thread just wander around your code because it's going to kill it. That's all about the client. What can the server send? The server sends, in my case, two kind of very easy messages. One is an event. The event is a very simple targeted message, very cheap, very small, but it only tells you what changed like ariel has been wounded or another character is blind or that door the door was toggled open or closed whatever just what happened right you don't need to broadcast the entire game all the time and ideally you have a series of event and everything is synchronized so all the players all the clients get the same event and they have the same state ideally but you know life is not so easy so from time to time i also periodically send the snapshot the entire state so if there is an event missing for whatever reason for a client the client is readjusted to the snapshot and everybody is in sync again and then we start again updating only event event event event snapshot so everything is in sync uh it works pretty well i mean maybe it's very basic for some of you but this is a lot of lessons that i learned just by trying to do it from scratch Okay, this is the best part of it. I'm very proud of this part. Imagine that the world state, the canonical world state, is on the server and the server decides to broadcast it to all the clients. Now, how would you broadcast it? Well, the Dungeon Master gets everything of the state because there is no reason to hide something from Dungeon Master. They need to know everything. But the clients, no, There is no need to give all the information to the clients, right? Because there are things that the characters shouldn't know. For instance, they explore only a few rooms in dungeon. There is no need to tell them what's in the box they did not explore. There are invisible creatures that they're not supposed to see. Let's not broadcast the position of those creatures. If you can do that, it adds a lot to the fun because then you can obtain personalized view on the clients depending on the character, okay? And can we do that? Yes, with something like this beauty here. Efficiency, too nerdy? No? Fine. Efficiency through simplicity. Imagine, basically, we cycle through the combatant and we append them, but we filter out, for instance, is it hidden? We filter out. Is it invisible? We filter out. Unless you can actually see invisible pictures. And then we append it and we send it. So, you will not know what you're not supposed to know. And the same with the tiles, right? Did you already explore them? Sure, let's include them. let's include them. If you do not explore them, you're not gonna get them. What can this get us? This. I mean, I'm so happy with this. On the left you have the Dungeon Master view, or the same screenshot, and on the right you have the player view, specifically the player on the carpet, in this case, okay? Now, I don't know if you can actually see, but two characters, two creatures here are actually invisible, they are transparent. On this screen it's not super easy, like this one, on the the other one on the carpet is half transparent and the top left is half transparent. So that means they're invisible and indeed the player doesn't see them. Okay, no idea. Also there are rooms that the Dungeon Master sees but the player has no idea. The only thing he can see actually is glancing through the open door so you can see how more or less how the line of sight works for the exploration in this kind of framework. But that's it. Now all the information he doesn't need to know, he doesn't have them. You can also highlight something on the other room as as a Lentil master, like there is a sound from here, and what they will get on the top, highlight tiles on the very north behind the door, but he has no idea what's in there. Maybe just the noise you can describe it, but you can play a lot with it, and it adds a lot to the fun. And this is a literal game changer, because sometimes a player knows what a character does, right, and this is something you cannot avoid at the table. For instance, a player sees an invisible creature, You say this, and other players realize that they have to pretend they don't know it, but they know. And this creates a separation between the player and the character. And this damages the immersion to the experience. And, you know, here the player only knows what the character knows. And it's super fun when you play on Teamspeak, on Zoom, and you see, like, a player talking like a character. Like, careful, the creature is coming. Which creature? You don't see the creature? Where is the creature? It's super fun if this just cannot happen on the table. I mean they won't give you much more, but this you can only have So the only one question is left and is what does the client do during the mid-ish the client? Wants to track for instance a creature on the map He drugs it then sends the indent the intent to the server the server deliberates But in the meantime, what does the client do? it can do two things apply the move or not apply the move while the client decides and the idea of applying the move here i call the optimistic update before the server actually validates it means that if the server then blocks the move you just roll back and first of all it feels very snappy and the the the the experience is kind of a kind of jarring sort of thing and also the rollback system is not so trivial when you have a lot of events that at some point get rejected altogether and you have to roll back and it can create problems so for the players typically i assume um control update policy so you only will see the move when you click on your client after the server has validated it the typical latency of this is 0.1 seconds even less you're not gonna it's not gonna make a difference on the experience on setup like this um but uh Yeah, so for the dungeon master, I just said, yeah, yeah, you can just update because there is no reason to assume that the server would reject a dungeon master. So, testing my secret joy. I typically test everything, of course, as anybody of us does. So I test my units as just pure state transition. For instance, I have Ariel, I wound Ariel, I take her HP to zero. does she get the state unconscious and something like that. I simulate a simple transition and see if the transition gives me the output that I want. I have like hundreds of tests like that. For the protocol it's a bit more complicated because you need to simulate the client and then you need to go really into the details of the web socket connection and stuff like that and lots of tests which are more or less the same thing but I like to think of them as a different category because the simulations are a bit more complicated. You can have a look at that later if you would. While for the rendering, running, implementing the focal wall and in general for everything visual, usually I leave it to playtesting. I have a few tests, but nothing beat the UNI for these kind of things. So, yeah. So, four major lessons learned. First of all, don't let background thread touch anything because they don't know what they're doing. If you let the background thread touch something without funneling them to one validation point, you're gonna corrupt your widget, your safe state, everything. This is the first major that I learned. Two, if you want the projection function, which is the thing that masks the information, delivers it, tailoring it to every client, be sure that it's a pure function don't try to reshape it so you can do two things at the same time or can you modify a bit so you can do other thing that you know because they're gonna corrupt it and for instance if you include two different clients on the same instance you're gonna break the privacy for instance because it can be proved no and but there are many other things you can you can mess up if you if you if you start having the project function somewhere else number three don't just don't let background trend go around. Use queuing system. Write event, wait for validation, wait for broadcast and so like that. Normally the UI framework has this. They don't just touch anything, right? It's like a standard. And the fourth thing is I personally think that using snapshots and events mixed with the right amount of a frequency of snapshots is the way to go for something so easy as a save game for just DICs and JSONs. It was wonderful and never had troubles even of a remote. It was just perfect. And it never had troubles after a proper pre-testing, so to say. So first of all, I started this project because they asked me, and that's how I did it. In general, a personal project is perfect when you want to learn something, because the stakes are very low. So even if you fail, nothing happens. But when you want to experiment, it's when you learn, right? And that's why you can start with a personal project, and you really go wide, and that's why you really learn. The complexity is the same as any other production task that you might take upon you. So even if you try to take a shortcut, it's going to come back and bite you back. And at the same time, it's your project, it's your baby. So you want to fix everything properly and to see it growing safe and sound and strong. So I cannot tell you how much I learned when I was discovering this thing, when I was developing this thing as a side activity. So I would say, wish me good luck on my job hunting as a design test, you have a lovely audience and that's it on my side. This is my GitHub, you can see if you want to see how it's the projection logic, which is the nicer view, you can check the server API, if you want to see how I handle the more complicated tests, you can have a test server map.

Speaker 1 [22:34]

Do we have any questions? I didn't see in the board room. Do we have any questions? Anyone? Then we can just give him an applause, and we'll be done.