Using machine learning for Level Generation in Snake (video-game)

Speaker 1 [00:03]

Hello everyone. So, this is my first tutorial and also first time here. Thank you. And so, this was a little experiment that I made. And so, the idea is that we're gonna, like a couple months ago in a tutorial, they used snake as an example. And while I was playing, I noticed that some really funny situations appeared on snakes, such as like the apple right in front of the snake or things like that. And at that time, I thought about maybe we can use some regression model or something to do this much better than than this. And so, as you can see here, a lot of examples like this would happen. After having the situation referred to me a couple of times, I thought about a way of choosing the location of the apple in sort of a smarter way. And I wanted this model taking into account the snake state or position or direction to be able to predict the next location of the apple. And this way we could avoid situations like this one here. And also, we can actually compare scores between players and things like that because they are not affected by this random variable that is where is the apple going to be next um and so but we have some problems because this is not a conventional regression problem one says not really a ground true for the apple there are several places that could be a good place to put this apple um and so once that some apples might have a good location and some others might not we need a way to be able to tell what are the good locations or not. And by good location I mean that the apple was put in a place in which the player has to do some moves and not just let the snake keep going straight. And so for this we need to find a way to score each individual apple like I was saying and then we want to use this score to select what were the best placements of the apple. So to get this score, we're going to create several metrics to be able to evaluate the quality of a placement. And so, we have to think these metrics in a way that will allow us to be able to distinguish situations like the one here to situations that force the player to take an action. And so, after selecting what are the best plays and not, we are going to use this data. But there's still a problem, which is, like I was saying, for each snake state, each observation is not necessarily unique, right? So there might be several good places to put an apple. And so for this, we need to generate some more apple locations. And this will also allow us to, besides having more data to train our model, we can also have some, imply some randomness to the final model, which we also want because we don't want it to always act the same way when the snake is in the same position. And so after this, after having a dataset to train it, I'm just going to go through some difficulties and problems that I found doing cross-validation to train and test our model. And so this is going to basically, the tutorial is going to have two parts. So the first part where we're going to create a game law. The first part where we're going to create a game log where we can, while playing, we can actually have a CSV or a TSV in this case with all of our placements. So where the apple was placed and what was the snake state at that point. And so, for this first part, I had to pick a game, an implementation of Snake. And I'm sorry for the code of this. The game I picked is not the most clean one, but it was the only implementation I found that was actually playable and not just a really simple Snake game. So for this first part where we're going to create a game log, this game log is going to be a file where we write all the apple locations and the snake state when the apple is placed. But because we have to find a way of measuring what is the quality of our apple, some of the metrics we are going to use, we need to calculate them during the game. And so this was one of the problems I found while doing this. So, the problem is that, like, so when we are for each apple that the snake catches, we need to say what was the snake position when the apple was placed in the screen. And then that snake is going to move to catch this apple. And as it moves, we want to save this snake's position and then wait for the player to catch the apple, and while it's trying to catch the apple we're going to measure these metrics. And so we have to save the snake state at this point, and then when it catches the apple then we will save this state. And so for this game log I end up using just a TSV file because it's much easier to just dump lists which include snake position and apple position and all of that because of the commas into a file. And so this, sorry, and so these two time seconds and moves to Apple were the two metrics that I thought about to measure during gameplay. And I will talk a bit more about them later. And so for this first part I'm gonna go through the game and some... just to show you some difficulties that I add and how did I end up solving it? Sorry. And so this is our game. Like I said, I mean, I'm sorry, the code is not the cleanest. I did some refactoring, but still, it's not the best one. And so the idea here, and so we start the game. We do all these inits, which most of it is game-related things. And so here I created these variables to save the snake state when it catches the apple. And then while the plane is trying to catch that apple, we're going to measure the time it's taking it to catch the apple and measure how many moves it has to do. And once he catches that apple, then we will save these positions into a file. And so the things I choose to save about the snake was just pretty simple things, just the position, direction, and the score at that point. And then we are going to use these variables later for features and training our models and all of that. And so, this is just the normal loop of the game. So, here we're basically looping for each frame of the game. Here, just like some checking collisions with the snake, with the walls and with the snake itself. And so, here, when the snake catches the apple was where I had most of the work. And so, once the game starts... Let me actually just show you the game for... Okay, so this is the game. It's a pretty simple game. Just so that you can see it. And so here the idea is that once the game starts, as you could see, it waits for the player to press the first key to then start to actually do the game, start the game. And so here, where we're detecting all the key presses events. So here is where, once the player presses the first key, which is when the first apple is placed, we're going to, at that point, we don't know what is the snake direction because the key, no key was pressed yet. And so for the first apple in the game, we're going to say that the direction of the snake was 00. And then we're going to write that event to our file. And then the score also, which obviously starts as zero. And we just saved this start event to a file, which includes the snake's position and the apple, the first placement of the apple position. And then for every apple that the player catches, we are also going to write the event. But so as I was saying before, because we want to measure time and moves to the apple, this write event happens before actually saving the snake position, because this snake position will only be saved when the player catches the next apple. And so here also when we catch the apple I set the moves to zero. And then here just add plus one to the move whenever a key is pressed. Besides this in the game there are some config files where you can pick the screen size, if we want to use a model or not to generate the apples, and the name of our regressor, the file of the the pico file, and then a file name with just the game log with all these events that I was talking about before. Okay, so here is just where I actually write the header of our TSV and that's pretty much it. Another thing I had to do here, it was actually loading the model, which I'm just doing here. And then after loading this model, I will use it in the Apple here, where I will actually, whenever a new apple is generated, I compute the features using the snake position, snake reaction and game score. I will talk about what these features are later and then set the position of the snake for the predicted value of our model. So here, I don't know if everyone is following along, I also have a PDF here with this notebook, I forgot to mention, if you guys have some problems setting up environments or things like that. After creating this log, we're going to move to the second part, where we're going to use data from these logs that we just created to select the dataset, train and test our models. So, let me just restart this. So, I don't know if anyone has any questions about this first part where I explain the concept. So for this second part we're gonna open our file and so this is just a helper function that I created. So here I just opened that file that we created and like I said I had to use a TSV because of the commas in the list, I was having some problems with it. And then after that, we're going to check all the places in the TSV that are lists, which is the apple position, the snake position, the apple position in the snake direction. And we're going to transform them back to lists so that we can use them during our notebook. There's also another field that for other information during the game. And also I end up converting this to array because it's easier to do with NumPy. And I also added this field time to apple. It's basically how long the player takes to go from one apple to the other. Because here during the game, what I'm actually measuring, it's just a timestamp that starts in the beginning of the game and it keeps growing as the game goes. And at the end of the game, I set this back to zero. So, So because it's increasing, I have to subtract the previous column with the next column to get this last time to apple value. And so here I just do a shift in the column and get this time to apple. we're going to use later to measure how good our apples were. And so going back to the notebook. Okay, so here I'm just displaying all of these fields to make sure that our conversions all worked and everything. And so here we have the data from the game that I was talking about. And so, here we started by a status. So, when the game starts, we write the first row with the time at zero, like I was saying, and then this time keeps increasing as we go through the game and as the snake catches apples. And so, here we basically... So, we have the score of the Snape here, the moves it takes you to go to that apple, and then the apple position that was placed at that stage. And this is a snake position which we can see it's an array with more than one position because it's a position per block of the snake. And this is the snake direction. So, in all of these positions and directions, the axis is as it is here. So, we have the y-axis going inverted compared to the normal representation of an axis. And that's why I plotted it this way, to be able to have the same reference as I did in-game. Okay, so here I also created a helper method, which is called PlotSnake, which it's basically the function that I did all of these images with. And so, this helper function can end up being quite useful, because it allowed me to do like quick debugging, it allowed me to watch several placements of the app at the same time. It also helped me to do simple debugging and experimenting with the types of scenarios that I have for features, for all of those things. And so for this case, for the visualization and also to help during the debugging phases, I added this previous apple position so we can see what was the apple that the snake is catching and also we could see what is the next apple that the snake has to use. And so here we can see some examples of the snake. And so the numbers are, so this is the first apple that the snake catches, as we can see because the snake is close to it, and then the next one is here. So this is the zero one, it's the previous apple position that I'm setting here. Also this, so there's a square, there is a dark green, this square is supposed to be the snake's mouth, just as a point of reference. And so, as I said, now that we have a dataset with all of these apple placements, the idea is to think about metrics that allow us to measure the quality of each apple location. But these metrics have to take into account the snake's state, which by snake's state I mean it's the snake's direction and position. And so these metrics are a little bit not so easy to think about because it's hard to find a metric that defines this in a good way. And so, what I ended up doing was to create several metrics. And as we'll see, depending on how we weight these metrics or which metrics we use, this will influence how our apples will be placed, as it makes sense. And so, when designing these metrics, I take into account the metrics that work the better. So I had to identify the models that work the better, sorry. Okay, so when think about these metrics, I had to think about that they have to be simple because I tried some harder things and in the end, the model just couldn't figure out those connections, those relations between the snake state and what was the apple position. And so, the metrics that I ended up implementing was, as I was saying, so the time to apple that this time to apple has to be measured during the game, right? So, it's the time that basically takes for the snake to go from one apple to the other. And the point of this metric was that basically the longer it takes to get to an apple, the harder for the player it was, or the more action the player had to take. And this is not always necessarily true, as I noticed, but as the game, as the snake keeps growing, we see that parts of the body of the snake get in the way of of the apple and so you have to sort of like go around the snake itself and it's this particular observations of placing the apple that we are interested about for this particular metric. And so then also thought about the moves to apple which it's basically how many moves it took the player to get to the next apple. And this is a pretty simple metric. So, basically, the more moves you have to do, the harder it was. And therefore, there was a better apple than some other apple that took less time. Besides that, I also use the apple distance. So, it's basically the distance between the snake's mouth and the apple. And this is actually one of the first metrics that I thought about. And the idea of these metrics is to penalize apples that were too close to the snake. Even though that's not necessarily always true, that apples that are close to the snake might be bad, but the idea was that other metrics could detect that. And so, that's why I ended up doing all of these metrics. Another metric that I used was the angle between the snake's direction and the apple. So, the higher this angle is, basically. So, as we can see here, if the snake is going this way and if an apple is at this angle here, it means that you will have to turn to actually get to this apple. And so, the idea was to try to also identify these situations with this metric here. So, after that, this metric didn't work as well, but I ended up leaving it here also. So, So the idea was that the number of times that, so if we look at a straight line from the snake's mouth to the apple, the idea was that if you have an intersection between the snake and the apple, so in this case, you're the snake's body, it means that you can't go directly to this apple. and therefore it means that you have to go around the snake itself to try to get to that apple. Okay, so these were the five metrics that I used in the end. And so here we're going to calculate these metrics. And so, they happen here in the end of our data frame. And so, next I'm going to just walk you through how I implemented these metrics. Okay, so this was the function that I was running in the notebook. And so, here I start by getting the position of the snake's mouth. And then, because I will use that in several metrics. And so, the idea here for the Apple distance is pretty easy. I just created a function that computes distance that I learned before that Python 8 already has. So, Python 2.8 already has. But it's a pretty easy function. It just computes the difference and then squares it and sums it. And then for the angle, the way I did it was that I first off calculated the angle between the snake mouth and the, so the snake mouth and the X axis. So basically think about like a vector then just as x that's zero and then as you go around you get from zero to 360 in this angle. And so after having this angle I calculated the angle between the f and the apple which here is represented by this vector here And then after that, I subtract these two angles to get the final angle. Here I have to divide to get the module, the rest of the division to normalize it from zero to 360. And so for this case here, what I'm doing is that I'm basically normalizing the angle. So in this case here, we're not interested in the side of the angle. So I don't care if it's to this side or to that side. I'm just interested about the absolute value of this angle. So if this angle here, if the apple was here, this angle would be the maximum value for this metric. So, after that, I calculated the number of intersections with snake and for that I used this function here that I also created that uses Shapely to basically create the snake, create a line, so create this line here between the snake's mouth and the apple, and then another line that is the snake itself, and these two lines, it's pretty easy to calculate the intersection with Shapely, and there's just a direct function that does that and then we get this final value that it's basically the number of intersections that we have between the snake and the apple. And so in some cases this was also a line when actually these two lines, so if this line is aligned with the snake this can actually happen to be a line which in that case I just ignore it because that tends to really happen a lot of times to be honest. So after that I also create another metric that in the end I am not using which was this one here so So it was basically the number of sides in the apple that are touching the snake. So in this case, it would just be one because this side here, the snake is in front of it. And the idea was to try to detect cases where the apple is placed in the middle of the snake, where we would have three cases of, in this case, you would have three sides of the apple. And that means that the apple is placed on top of the snake, and it will make it harder for the player to, in the end, get it. Okay, I think it has metrics, that's all. And so now, after calculating them, we can see some cases for these metrics where the player actually... where these types of plays happen. So, here I try to show the highest cases for each one of the metrics where these metrics are actually useful. And so, as you can see year, the time to Apple works really well for cases when the snake is quite big, because that means it will take you a lot more time to go around the snake. And even though these cases here, the distance is pretty close, for example, we can still see that there's still good placements because it's a pretty long snake and makes it that you have to go around it to get this final apple. So here it's the number of moves, which does not necessarily mean that it's a good play. But as we can see here, it also coincides with a time to weapon for one of the cases. And the other cases are also... This one is actually not super good, but this is why I end up doing several metrics. And so for the apple distance, I thought about this metric more for the beginning of the game where the snake is not the only thing that you can do to make it harder is place it away from the player. And so here it's the direction of the snake and so in this case here we can see that the apple was placed here and this angle that I was talking about is maximum when you put it on the back of the snake, which also makes you have to go around. And so most of these cases are cases in which the apple is directly behind the player. And so here for the number of intersections. As we can see, some of the cases are also good placements. None of them are placements that are directly in front of the snake or things like that. If you look at the distribution of these metrics, we can see that everything looks good. We have the distance that is sort of a normal distribution, which makes sense because we're placing these apples randomly. And then we have moves to apple, which it's also a pretty good distribution. There's a mistake here from the int. And there's some place here that the player actually had to take more moves than most of the normal cases. Also the intersections are as expected. So decreasing and there's no more than two intersections with the snake also. And the direction as we can see goes from 0 to 180 because I don't care about the side of this angle. So I just care what is the angle, like how much back the snake has to go to get this apple. The time is also very similar to the moves, which makes sense also. And so now that we have these metrics that I talked about, the idea is to and create a final score with all these metrics that will allow us to select the observations that we want to use to train. And this score can also be used to compare performance between different models and the random process, for example, to see if our final model will be better than just placing the apple randomly. And so to compute this score, we are basically going to calculate a weighted average of the metrics that we define. So this allows us to be able to pick different weights and to be able to select if we want behaviors that are more towards a certain metrics or towards another metrics. And so one of the problems that I found here is that each one of these metrics has completely different scales, as we can see here. And so if we use their value directly, they're basically going to have different weights, right? So metrics that have higher values like angles, for example, are going to have different contributions. So I decided to, for each one of these metrics, rank them, and then instead of using the value directly, I rank these metrics and basically end up with a percentile. And each one of these percentiles are going to be weighted and then averaged. And this way we are able to remove the influence between metrics and actually get the same weight per metric. So to calculate this score, I started, so here I'm displaying the metric columns so that we can see how can we do our weights. And so, the idea here, like I was saying, because I had to rank them, the idea here is that we start with our data frame and we select just the columns that we want to use to score. And then here, I basically rank them, each one of them, and then divide by the maximum rent to get the final percentile of that metric. As you can see here, I also tried just normalizing them with minimax, but it didn't work as well. And after that, we're just going to average them with the weights I talked about, and then return the final data frame with the score. So, here I'm just, because later we're going to need to know what were the columns that were used specifically for, that was used specifically for scoring. And so, here I'm just checking if the weight was different than zero, then we will use this value later. And so here we can see how these scores are. So as we can see they are from 0 to 1, which makes sense once we're just using percentiles. And we can see that we also have a good distribution of these scores. So now once that we have this course, we need to use this course to filter our original data and then get a final training set. So, here I start by creating a test set, which we have to do now because we don't want to have any influence from our test set to pick our scores. And so here we can select different values. So the higher this score is, as you can see, the less samples we're gonna have, which also makes sense. And so if this score is too low, we're just gonna get a lot of bad apples that won't allow us to, in the end, train our model. And so after this, after creating our training set, we have to make sure that we have observations in all the spectrums of the game score because we don't want our model to just be good when the score is higher than 20 or whatever. So for that I just plot the distribution of our scores and then as you can see here we have a range in all the scores which allows our model to know cases for all the scores that we use. And so as I I said before, we also need to, so once now that we have already a training set, the idea here, it's to, for each state of the snake, we need to generate more than one, just one case, just one apple, because first off, we don't want our model to overfit for those cases. And it's actually what happens because there's not just one case when the apple is played. And so besides that, I also, because we're doing all these metrics and all the filtering, and so we don't want to generate apples that have worse metrics than the original ones. And so, here for each snake state, we are basically gonna generate, try to generate a number of apples. And so, here I start by, here I'm just calculating what is the range, because there's cases in which the apple is in a corner. And for those cases, we need to reduce the range in which we're generating apples because we don't want apples to be out of the screen. And so here, I calculate the limits for X and Y with this limit. So, we have a limit of 50, 50 by 50 to generate apples around the original one. After this, I just stack both X and Y after generating the random locations that I will then add to the original one. So here I just generated a value maximum between minus 50 and 50. and so here we're just going to add the random positions that we just created to our original apple position and then here we're just generating a dataset that has the same snake position the same direction and everything but in the end it has one it has a different apple And so after that, we are going to compute the metrics for these samples and this is where I use the valid columns that I filtered and that I calculated above to know actually what were the metrics that were used so that I can just filter those metrics. And so here all I do is just see what are in this new apples data frame. I see what are the, for the valid columns, I see which ones are basically highest, are bigger or equal to my original case. So here we are just testing this and as you can see, sometimes we don't even get any apples, sometimes we get. So because they're random and then I'm filtering the metrics, each column can generate a different number of apples. So after this, so this function will be used later during cross-validation and training. So So, in the next section, I'm going to talk about the features that I use to be able to do this. And so, in this case, because our model has to react to the snake position and direction, we need features that reflect this. And so, for this, we use directly the snake direction, just as dummy, once we only have four possible cases, which is left, right, up, and down. And then the same for the snake's direction. But for the snake's position, once the size keeps going as the game progresses and our feature size is always to be the same, we need to find a way to normalize this array of positions to a constant size. And so to do this, I basically consider the line, as we can see here, that starts from the apple, starts from the end of the snake and goes towards the end. And then we want to normalize this number of positions that the snake has, in this case, it's 11, to only 5. So considering this line, after that I'm gonna equally sample this line in the end points that I want to reduce the snake to, as we can see here. And so we end up having this array with always five positions that describes the snake beginning, end and other points during the snake. So So besides that, I just did quite simple features, just like min and max and min of the snake location, which in the end didn't help much. And so it was just basically that the most useful feature was by far the line one and also the direction. And so here we're calculating our features. we can see here. So, because we have x and y coordinates, this makes it that we have a position zero and a position zero y and x. And as I said, the normalized five positions, which when the snake, for example, only has one position, and we are only going to get of these values are going to be the same because we just have one position, right? And so, to calculate the features, I tried to keep... So, for these features, we have to declare it in the game because we need to calculate everything during the game also. All these features need to be calculated during the game. And then I just import it from the Jupyter Notebook. And so, I also try to be careful here doing these features once they're running inside the game to not use pandas and just try to use NumPy, even though that it's a pre-laid game. I try to be careful with that. And so, for these features, we have this function that computes all the features. And so, we start by normalizing the position like I said. And for this case, I end up using just a linspace of NumPy that interpolates the snake's position in this line space and makes that, in the end, we're just going to have endpoints for the position. So, when it's one, as I said, I just return that point five times. Here, I also did some position statistics. So, this function just receives a list of functions and computes those functions. and saves them in a list to be able to use during the game. I also used this function to get the dummies for the direction that we need both during gameplay and for our notebook. And so these are the statistics that I did, just min, max, and average. But most of these things are already in the normalized position, so in the end it wasn't super useful. And so after normalizing the position, I just get the statistics, right? I have some auxiliar vars to be able to return the feature names for my notebook. And because I didn't want to have features in two places, I decided to create everything here to be able to call it from both sides and not have any problems with the features. And so after this, I do the dummy for the direction with this function here, which just checks. Once there's only four directions, the direction vector will always have one other number that is zero, or both of them will be zero when the game is starting, which is this state that is stopped. But because I end up not using stop, because it means that we just need four, and when we're stopped, none of them is true. And so for features, I end up just using these four cases here. And so, after that, I just concatenate all the features and return them as an array. If we're calling this function inside the game, excuse me, if we're calling this function from the notebook, then I have to get all the all the feature names where I do that here for the position in the range of the points you have, for the statistics too, for the snake direction with all the dummy letters also and then I just do a dictionary with all the feature values that we can use in the notebook. So, after then, we're going to try to train our model with our features. And so, here, during the only problem that I had, it was that I wanted to use cross validation, but because we are generating plays that are exactly the same snake position in where the apple just changes. And even then, the apple is still close to the other one. So, we have to be really careful in cross-validation to not leak these snake positions from the training set to the test set. And so, because of that, I ended up using just GridSearchCV from SkitLearn, and I implemented my own folds and also my own metrics, because in this case, we're trying to predict two points, X and Y. And SkitLearn just assumes that they're two independent predictions when they're not, because we're talking about a location. So, for this, I use the distance between the points to have the final error for cross-validation. And so, here, to implement the folds manually, it's quite easy with Skitlearn. You just need a list with the indexes of each fold. and what is the training set and what is the test set, basically. And so, here's where we're going to use the function to generate new apples. And so, we start by creating three folds with kfold. And then, because we want to make sure that there's no leaking between test set and train set, we have to generate the apples only with the train, only with the train set. And for that, we just use the indexes directly to pass this to the function that generates more apples. And then we have a final data frame with all the generated apples. And then we have to update the indexes for our training fold. And so, this is what I'm doing next. So, I just basically get the size of the training set. And then I will use this size to create a range that I will add to the training set. And I'm just basically... So, I'm basically appending the generated apples in the end of the data set and then setting the indexes in the train set for the end of the data set. This takes a little bit to run. So after then, so here I end up using compute all features this way with the apply because Because we have to use this function also during the game. So this function has to compute a line at a time. And so I end up just doing an apply and then stacking all the arrays of this feature to get my X training set and also my ground true. As we can see here, this matches these two sizes here. And as you can see here, even though we are trying six times per apple, that doesn't mean that we're going to get six times more apples. We have about three, which makes sense because all of them are not going to have the good metrics to be able to go for the training set. And so after that, we pick our regressor. I tried a couple more when random forests end up working best, even though there's not a lot of data. And so here for a cross-validation, like I was saying, I just created a grid with the parameters that I want to test. And I had to make my own score, which has to return a final value. But because the skit learn sends all of the predictions at once, and this function was only computing one point, I ended up just creating a lambda to feed to the makeScore and then create a final score. So this grid gets the folds we created above, also gets our score and the model that we want to train basically and also the parameters of course. GridSearch returns all the results in a pretty confusing way with this object, so I end up creating a nice visualization just in pandas to be able to tell how things are going. And so, we can see that it's not a super stable training because our parameters are varying a lot and the test set is not decreasing a lot from parameter to parameter, which probably means we don't have enough data because I would probably have to play snake for quite a while to have enough data to train here. But this was still the classifier that I got the best results with. And so after this, we're going to calculate our training error. And so for this, we shouldn't use the same data set that we use for cross-validation because we have used the same sample to generate apples more than once because this sample was present in different training folds. And so here, this is what I'm doing. So I basically generate all the apples again for the first selected data set that I have and then create a total training set. that is this selected data set plus my generated data. And so here I'm doing the same thing again, calculating my features and stacking them, my Y. And then here I'm also using sample weights. These sample weights, they reflect directly the score that I was using because I was trying to basically make certain plays more important than others but in the end the results didn't really help much so I ended up removing it. Here I'm also setting them, I'm increasing the number of estimators because this is our final estimator so just to get a bit more accuracy. So, for our training error, finally, we get an 89%... 89 average distance between predictions, which, comparing with our test score from trained auditions, shows that we're probably overfitting a little bit. And so, which shows, once again, that we needed more data to, in the end, have an actual good model. And so here we can take a look at our distributions. And as we can see, some cases of distance are actually being quite good, but some of them are actually not that good. And the end result wasn't like super great. It works, but it's not the best. So here for the features importance. So as you can see, the normalized positions of the snakehead are the most important ones. And also some directions because of the metric that measures the direction compared to the snake direction. And yeah, most of the other features weren't really that important. So here we can see the snakes plot with the predictions, which zero is the original one and the predictions are one. And so, here we can see a little bit that the model tends to shift our predictions a bit to the middle here, because it's safer and the error will be eventually lower, which shows that we probably need more features to try to have a better result in the end. So here, so now I'm just gonna save the model and making sure that I get the same error and then I can load it. And so now we can actually use this model to play our game. And so here, we can see that because of our location, the model tends to put the apple a lot in the back of the snake, which is an interesting behavior. And the reason for that is that's probably the only feature that the model can pick on to, in the end, be able to have a better performance. So as for testing, testing was an interesting case because some metrics are calculated in game and some others are not. So ideally you should be testing it in the actual game by playing and then making sure that your metrics are higher than, all of your metrics are higher than the random process. But because that really difficult development, because you have to play every time you want to test the model, I just created a test set with the last games that I created. And with that test set, I check the metrics that don't depend on the game and just use those metrics as my final result. And so here, because for this test set, it also doesn't make sense to use the same metric we're using in cross-validation, because here what we actually want to compare is not how close to the final apple you were, but more how good the apple is compared to the metrics and not comparing to the random process that we were doing before, like we did doing cross-validation. And so for this, I'm just computing the features for my test set and then computing the metrics instead of the distance this time. And so here we can see the metrics for our predictions. And here we can see the metrics regionally. And so we have moves to Apple and time to Apple, which should be the same because we in actual play. And then we have the distance and these three ones that are a little bit higher in our predictions compared to the previous one. And so just as a final challenge, I'm not sure if we still have time for this. I'd like you to try to actual play, and then get some observations, and then try to get your own model. But probably because you only have 100 plays, it'll be even harder to train your model. So maybe if you increase the number of tries and decrease your score function, you can actually get a model that does something in the end. And that's it. Thank you.