AutoGluon: AutoML for Tabular, Multimodal and Time Series Data

Speaker 1 [00:02]

Hey, hey everyone. Mic check. Is everything okay? Like with the voice? All right, great. And thanks, Antonia, for the introduction. That was about half the talk, so we'll focus the rest of our time into like the nitty gritty of the library. So first of all, welcome everyone to our session on AutoGluon. AutoGluon is our AutoML library that provides a variety of data modalities for you to work with. Myself, Janer, and Alexander will be presenting the session. My name is Janer. I work at AWS. I'm a senior applied scientist. I've been with the company for about four years now and in the general data science space for about a decade. Alexander, my colleague, is also an applied scientist at AWS. And together, Together he got his PhD recently from the Technical University of Munich and together we work on the forecasting time series features within the general AutoGluon framework. So let's start. So I'm going to start with basically describing ML in a nutshell and try to frame the problem of AutoML there, but this is ML in a nutshell before ChatGPT basically, but what you might encounter in any ML problem or in the general workflow of a data scientist is you have data, so that data can come in multiple modalities that might be images, text, tabular data, and time series data perhaps, and you have a specific task that you want to address. For example, if it's tabular data, you might be interested in regression, classification. If it's images, it could be image classification, et cetera. And finally, what you really want on the third pillar here is you want a machine that can make predictions, like a test time on data it has never seen before. And hopefully those predictions are accurate predictions, they are basically usable, they arrive in like good time etc and in like that second pillar that does not at all do any justice to the amount of work that is required is the real work of like most some of the work of a data scientist which is pre-processing that data trying out different models trying out different approaches for solving that task and finally you know selecting one model or one approach that can give you the best performance at test time. And AutoML, in a nutshell, is trying to take that second pillar and trying to automate as much of that away as possible so as to make the lives of data scientists much easier and get to the most accurate models with maybe as little as three lines of code and the least amount of human involvement as possible. And AutoGluon does just that. AutoGluon is an AutoML library and framework that aims primarily to democratize machine learning so as to empower the next millions of data scientists. And what it does for you is, in a nutshell, it builds accurate models for a variety of tasks with image data, text data, time series data, or tabular data, with as little as three lines of code, hiding away all of that complexity that we just talked about in the second pillar. And it is open source. It's available on GitHub. We'll share the links. And it is also very easy to install. It's available on PyPI, so you can just pip install autoglue on and get started with with Autoglouan. Autoglouan is used in over 200 Amazon projects internally, so it powers bits and pieces of our external-facing AI services. Those you might have heard of include, for example, SageMaker, if you've ever used that. It also is used internally by Amazon.com teams in a variety of tasks for example and supply chain tasks etc and not only that it is also adopted externally by 50 companies at least that we know of brand names including things ones like Intel, Nvidia, IBM and Autoglue on a variety of tasks that it was designed to work on it is the state of the art in terms of performance or like performance per training time performance for inference time time, and it was featured as the opening keynote of the AutoML conference in 2022. So I will first start this talk by describing what it can do for tabular data and trying to give you a sense of how easy it is to use and what it does under the hood, and then Alexander will take over for the more interesting stuff that includes images, text, and time series, and time series is the one that's really closest to our hearts. And to be able to do that, let me first start by introducing the problem. The problem is, well, it's the first problem that you see in data science school, like if you ever went there, like on day one, right? So it's like you have a table of data, and it includes some columns. Those columns are your features. And particularly if you're interested in supervised learning, That's most of what we're interested in in autoglouan. You have one particular column that is of interest. That column is your label, so to speak. And if that label is real valued, you're trying to solve a regression problem, trying to predict that column in terms of the others. And if that feature is categorical, you're trying to solve a classification problem. And Autoglouan, we promised it can do that in as little as two or three lines of code. Trying to solve that tabular problem would be as easy as just calling the tabular predictor object, and that's really the only class name that you have to know about if you're trying to address tabular problems. You give it the name of the label column that you want it to expect from the data you're going to give it. You point it to the data. You tell it vaguely what to do, like focus on quality here, like try to get me the best, most accurate models possible. You specify the time limit. We're going to go into the details later. And really, it just takes an hour to train the best it can, the best classification or regression models, depending on the task it can for you. And you You can then take that predictor object, you can save it, and then you call it back and you can use the predict method on it at test time. How do we do that? How do we hide away all of the complexity of building models, selecting models, optimizing models, and giving the best performance? To be able to describe that, I should first really go into a bit of history here. So how these models or how AutoGluon came to be, the real inspiration of AutoGluon is actually maybe like if you've ever competed in machine learning competitions like on platforms like Kaggle, if you did that five years ago, maybe it was easy to call whatever was the state-of-the-art classification model, for example, back then, for example, your random forest, random forests were pretty popular. This might be more than five years ago, or like XGBoost, and you could have gotten like a really good, let's say, ranking, and that was like the good old days of Kaggle. But really soon after that, what happened in like the Kaggle or machine learning competition space is people started building these automated pipelines that just generate like hosts of plethora of features, train like, you know, just swaths of models, like different models, and then just build those models on top of one another, stack them on top of each other into these huge monstrous architectures and get the best performance. This was actually also the case that in the Netflix competition, if anyone remembers that. So in the Netflix competition, Netflix went out and said, can somebody please point us to a good algorithm for doing recommendations? And the best thing that came back was something that the Netflix engineers said, OK, we accept that that is performing very well, but this is a monster. If we went out and tried to implement this, it would take us ages. And Autoglouan, what it tries to do is it tries to replicate as much of that competition winning logic, but in a completely automated manner, in a completely hiding away all of the complexity of both training such architectures and also deploying them in test time with just a very simple, very narrow, very concise interface for its users. And that philosophy also reflects in the competition results, of course, so in many competitions where like autogluon is tried and just like with either no or very little tuning, it can come to very competitive rankings. And as Alexander will talk about, not only in tabular tasks but also in multimodal tasks. And not only that, in academic benchmarks as well, this is the AutoML benchmark, it's a very comprehensive comparison of different AutoML frameworks, so other frameworks that are trying to solve this problem of, like, get the best model with minimal human intervention in the best possible time, it categorically outperforms all of the other frameworks, some some of which are commercially available. It does this not only on average, but I could say categorically across different tasks, so binary classification, multiclass classification, and regression problems, and also for different time windows, so when all of the frameworks are given one hour to train, also for when All of the frameworks are given, four hours to train, et cetera. And the paper is available in archive, and I would recommend it as also a good introductory reading to the AutoML space if you're interested in an academic view of things. And to be able to describe how that happens, I already alluded to competition beating monsters. But allow me to go into a bit more detail at an intuitive level on how that happens. So if you went to any AutoML conference, even like last year, or mentioned the word AutoML to people working in this space, their perception of that concept would be equivalent almost to this thing called the cash problem. there in parentheses, which is like the combined algorithm selection and hyperparameter tuning model problem, which basically just says, let's say you're trying to solve regression, you have these very different models, let's say those in scikit-learn, one of them is like linear regression, another one is like support vector regression, etc., etc., and what you want to do is, given a specific amount of time, you want to go to each one model, you want to tune their hyperparameters, maybe you sort of search the best regularization hyperparameter in one, you try to search for, if you're doing a neural network regression, for example, you try to search for the best neural network architecture in one, and try to come up with the one single best model that wins overall, that you expect it to generalize the best at prediction time. Autoglouan does that, but it also does a few things on top. So first, it has some, let's say, built-in data pre-processing, data pre-processing that you might find in some of those like competitive machine learning pipelines. It has emphasis on modern deep learning techniques so like it always will include some sort of deep learning even if it's working on tabular or time series data to get like squeeze out the best performance for you from your data and it can like completely hides away the complexity of doing that right so you don't have to Like, even if you have a GPU, it will sort of try to find it and use it for you. It has lots and lots of tricks and optimizations that go into, like, how to select models and what type of problems and, like, which models to prefer. It has a lot of presets, as we will talk about, so you can sort of maybe not being at all an expert user, you can nudge it in the direction of, like, this is the type of models that I want to use, and this is the type of performance that I want to get out of this library. But most importantly, there is number two, and, like, I ask myself why you put it there. Number two is ensembling via multi-layer stacking. So it's that idea of building monsters, like, with models upon models upon models, instead of relying on the single best model, can we come up with the best ensemble of models that won't really penalize us all that much at test time in terms of like how long it takes to predict, but it really gives a huge performance boost in terms of accuracy and like the task at hand. And the key idea of ensembling, maybe going a bit into that, is that simply many are better than few, right? So if you have two models and those two models make their own mistakes, so to speak, so they have their own inductive biases and they look at the same data set but they fail in different ways, those two models or three models can come, like they can vote within each other, so to speak, and come to a consensus and that consensus will will usually, in some cases, always be better than any one individual model, even if those models are worse in isolation. And when you're trying to solve this cache problem, you're trying to select the best algorithm, the thing is that you have gone and spent all that compute in trying to find that best model. So you've done your linear regression in our previous example, you've done your support vector regression, you've done your neural network. So why throw all of those away at test time instead of just taking some of them and packaging them and ensembling them into a new model? And that's really what the key idea of ensembling is. And in multilayer stack ensembling, the idea is that you get your input data, you try to – not there, it appears – you have trained all of the different models, your linear regressor, your support vector regressor, your neural network, et cetera. And what you do on top is you get your original data, think a data frame, you concatenate, you join in the predictions of the layer one models, and you use that new data to train a new cohort of models. However, of course, the key trick here to enable better generalization is that this new data is out of fold. This is like a new data set that was held out from the first, let's say, core to, like, the first layer of models. And this is really, like, all there is to it. This is what, at least, Autoglow and Tabular taps heavily into instead of, like, very, very involved hyperparameter optimization algorithms, things like Bayesian optimization, let's say, multifidelity methods, things like that, if you've ever heard, that are more, like, you know, classical, let's say, AutoNL. And of course, for all of that to work, you need a good model zoo, and AutoGlue offers a good variety. So like all of the things that you can come across at Kaggle, maybe we try to put them in there and try to maintain the complexity of making those things play nice together for you, including things like XGBoost, like GBM, CatBoost, neural network models that are based on PyTorch that are all in there and that will hopefully come and start working together when you do the pip install on a clone. And we try to hide away all of the complexity behind this very intuitive API where now I can go into more detail. You just tell it what you're trying to do and label class essentially does that. You say, okay, give me a predictor object. this is really one of the few objects that you need to know about if you're using the library, and you're trying to predict a column named class, you point it to the data set, that data set might be a string, like a file name, a URL, can be a pandas data frame, can be a variety of things, and you tell it presets. So presets is, you're trying to nudge the library, so we offer like a bunch of these presets you're trying to nudge the library into selecting like a strategy really so best quality means you want the the most accurate models no matter what but you could have said like for example fast inference so you you're trying to tell it to okay you know maybe sort of I have a little bit of give in in terms of accuracy, but please give me something that will work fast at test time. And the time limit is like how much you want it to train. Here it's like one hour in terms of seconds, you're saying, okay, I want you to do whatever you do in the timeframe of an hour. And that's really all there is to it. It will train a host of different models that we work on all the time to pre-select like the roster of models that will work best for you and hopefully give you something that works like state-of-the-art performance. Okay, now is the time I hand over to Alex for the rest of the talk. Thanks a lot.

Speaker 2 [20:54]

So as we have seen with the tabular predictor, we can do extremely well in different tabular prediction problems like classification, regression, but of course in many practical applications, the data that we have does not really like perfectly fit into this format of just a table that has numbers and categories in it. Very often we have to work with images or maybe text data, or even like in some quite challenging applications, we have the combinations of all these data modalities together where you can have both potentially multiple images, various text fields, in addition to the tabular data we have seen before, like numbers and categories. To make this a bit more specific, we can have a look at the pet finder computation that took place in Kaggle, where the goal was basically to predict when pets would get adopted. So for each pet, you had a picture, so an image, also some tabular metadata describing this image, and the goal was to predict some score, which tells us how quickly this pet is going to get adopted, and this would help the shelters to better help pets find their new homes. And here you see we have the combination of all these different data types in a prediction problem. As a bit more of a practical-minded example, we can think of some online store where you have a huge catalogue of products, and there each product also has all of these different features. We have multiple photos of the product, we have a text field which is a description and the title and also have different numerical and categorical attributes like price, color, maybe model, etc. And one question you might be interested in asking is whether this product is actually a duplicate of something else we have in the dataset. So let's say when a new seller comes to the platform, tries to sell a new product, you want to say is what they're trying to sell something we already have in our catalog or is it something new that we should list as a new item in the category. And of course these are just some examples. I'm sure in applications that you're working on, you have encountered some other problems where you have to deal with this multimodal data where all the different data modalities are present. And Autoglone can also help you solve problems like this. Again, to work with this data, we again have to somehow coerce it into a tabular format, but now this table you see is a bit more general than what China has showed us before, where now besides the numerical and categorical data types, we also have some other data types like images and text. For example, for images, we can have paths to images that are stored on your computer and for text it's just strings that we have there. For some other applications, like if you have to do object detection, you want to find different objects and images, you can also have bounding boxes as a column here in the data frame. Or if you want to do name entity recognition in text, you can have these same entities also as another attribute. And once you can convert the data into this format, you can again throw it into autoglown and get the best possible prediction in a limited amount of time. For this, there is a multimodal module of autoglone where we have another object called multimodal predictor which takes care of various tasks associated with such multimodal data like images, text, etc. As you see, the API here is very similar to the tabular predictor, just a few lines of code fitting, predicting, and this solves the problem. And under the hood, what happens here is we wrap various foundation models coming from libraries like TIM, MMDetection, CLIP, Hug & Face Transformers, and combine their predictions. So we do multi-modal fusion of these various models, we train special task-specific heads that allow us to fine-tune these models specifically to your task. So say even if you don't have much experience working with such complex models yourself, you can just put your data in this format, throw the multi-modal predictor at it, and it will take care of all these complex fine-tuning jobs for you. We can also use these models mentioned in here in a different way and combine them with the tabular predictor. So let's say if you have some classification problem where you have to classify some images, one thing you can do is you can use these foundation models to get embeddings, which is basically converting each image into some array with some fixed dimensionality. And then with these embeddings that we have obtained from the models, we can again put them into autoglow and tabular and train the tabular ensemble we have seen before. So essentially what this does, it converts the multi-modal problem into a tabular problem where we can use all the classical tools for tabular data, and then just uses the tabular predictor to solve the problems that you have. In terms of support for various problem types that we have for multi-modal data, these are, as I already mentioned, the classical things like classification and regression, but now for combinations of images, text, and tabular data. but also some more interesting things that are more specific to this multimodal data. So one such example is named entity recognition, where let's say you have various documents with text and you might be interested in finding some entities mentioned in there. Let's say there is some company name, you want to see whether this company name is mentioned in various text documents. And this way multimodal predictor with the problem type NER for named entity recognition can do this for you. You can also do problems that are more image-specific, such as object detection, where you want to find various objects located in an image. So let's say for self-driving, you might want to detect pedestrians, you might want to detect other cars, or stop signs, or things like that. This is covered by the object detection problem type. And finally, what I mentioned as an example in the beginning with this object matching, this is what we call here multimodal matching. the question is where you have different objects represented by images, text, tabular data, you want to answer the question, are these two objects the same? So given a bunch of images, text, descriptions, and tabular data for one object and for the other, you want to ask, is this the same object or is it something else? Or are they different? So this way the multimodal predictor will just give you some score that tells you how similar the two things are. In terms of the specifics as to how this whole thing works, like I said, we use various foundation models provided by these libraries, and depending on the type of the data, so depending on the modality, whether it's images, text, or just tabular data, we pass it to the respective models that can process this data. These models are some things like EfficientNet or ResNet, if you know, like for computer vision applications, transformer models, or even multi-modal models that takes multiple inputs such as clip, and then for the multi-modal predictor, once we pass them into these models, we can then do fine-tuning. So depending on the application, let's say if you want to do classification, it passes data into the models and then trains an additional layer on top which helps you classify based on the inputs coming from these models. So essentially taking care of all this complexity of managing various models coming from the different libraries with the simple API of the multimodal predictor. The final thing that I want to talk about today is another application supported by Autoglone, which is time series forecasting. This is again quite different from tabular data, quite different from multimodal, and Autoglone can help you deal with this as well. So time series is essentially some set of measurements that you make at regular intervals. say you're some energy company and you want to, you can measure like the energy consumption of some household every hour, and you can plot it, like here you see this curve, so for every hour we have the measurement, and they're usually interested in asking the question what will happen in the future, how will this develop over time. For an energy company it might be interesting, for example, to plan its capacities for electricity, let's say if you expect a big spike in consumption, you need to make sure that you have the available capacity in your network. This also doesn't have to be in electricity consumption, it can be, for example, demand forecasting in retail, where you have to stock your inventory to make sure that if there are more sales coming, you have enough of the units in stock to fulfill this demand. And all these different questions of where you have some quantities changing over time, you can frame them as time-serious forecasting problems. The way we are usually interested in solving them as practice is not by just giving you a single prediction for every time step into the future, so I don't want to say the energy consumption tomorrow is going to be 150, because that's not all we need to make these decisions in the end downstream. What we really care about is the range of possible outcomes. So maybe we want the model to tell us, okay, there is a 90% chance that the demand for this product tomorrow is going to be 150 and 170 units, and this way we're actually able to play in our inventory where we know, okay, we are kind of expected to be within these reasonable bounds, and this is what we use for the downstream predictions. And mathematically, this corresponds to predicting the quantiles of the distribution, which essentially means the model just gives you some number and say, so here we see the 10% quantile, the model says there's a 10% chance that the demand for this product is going to be below number in here, and you also predict the 90% quantile, so there's a 90% chance the demand is going to be below this number in here, and in total, this is kind of the range that we can expect to see for our time series. And this is exactly the kind of problem that Autoglow on time series is designed to do, is to generate such probabilistic forecasts for time series data. Another important aspect when doing time series forecasting is that we don't usually only have the one time series that we are trying to predict, so let's say we don't only have the sales for one product that we are trying to predict, we actually have a lot of other additional information. For example, we can have some other things that change over time, let's say we know that maybe we know if we have some promotions on different items, we know if there will be some holidays or weekends, and as you can imagine, these can all affect, say, your sales or other properties of the time series, and these can all be combined together to generate the most accurate forecasts. So in this figure you see on top the original time series that we are trying to predict, we have some other time series that we only know in the past, which can for example be demand for a related product in a different category that we have observed historically, and there can also be some other time-varying covariates, which is maybe like an indicator whether we have a promotion on this item or not, that we also, because we control it, we also know what its value is going to be in the future. And all of these we can combine together, describing them as covariates. And finally, when working with these time series data sets, we usually don't just have a single time series we want to predict, but we have many time series that we are trying to predict simultaneously. So let's say for demand forecasting, we don't just want to predict the demand for a single product, we want to predict demand for all the products in our catalogue, which can be thousands of time series, each corresponding to a separate product. And here for each time series, we can also associate some metadata, like name of the category, price, or maybe some other features that help us generate the forecast. For example, if we know that some products are all sold in the same location, we might want to give the model the capability to generate similar forecasts to the products sold in the same store that is located in the same state. We can feed all these types of data into Outer Glow On, But combining them again into a data frame which we now call time series data frame Where the idea is to stack all these various time series into one big table Where for each row we have some indicator for the time series So here you see we have the indicator for product A and we also have one column describing the timestamps telling us when Telling us when the time series file was recorded. We have the target time series, which is the maybe the demand for the product that we have at each specific day and And we can also have some additional covariates. In this case, we have the weekend, which allows the model to determine whether, which allows the model to use this information in the prediction as well. And once we pack all of our data into this time series data frame, we can just call autoglow on the time series predictor in this case, and it will generate a probabilistic forecast for us. As you see, the API is also quite simple compared to what we have seen before. We just have to specify the prediction length, telling how many steps into the future we want to predict. So let's say if I have daily time series data, this way I will tell the predictor, please generate a forecast for the next 30 days for each time series in my data set. We call fit, we call predict as usual, and now predictions will contain the probabilistic forecasts for all the time series in my original training data. So you see what's a bit different in time series world is that we don't really have this train validation split, train test split, we only have training data, and then when we generate the predictions, these contain the future values of the time series that we had in the training set. And internally, the time series module combines various forecasting approaches, starting from simple statistical models like ARIMA, that essentially capture simple patterns in the data, like seasonality, trends, maybe some auto-regressive structure. These are usually very fast to fit, but might not be super accurate, So to compensate for that, Autoglone also feeds deep learning models from the Glow-in-TS library, such as DeepAR. These usually take a bit longer to train, but can give you more accurate predictions, and also do quite well when combining these various covariates and static features we have seen before. And of course, we can use the integration with Autoglone Tabular to use the different tabular models, like CatBoost and LightGBM. And as usual in the Autoglone way, we do an assembling of all these models to generate the best possible model that will give you the most accurate forecast. I want to quickly jump over a few things here that are not really specific to the time series model, but what I show you is that essentially in OuterGloan, even though the basic API is very simple, where you only need a few lines of code to define your problem and generate predictions, there is also lots of room for configuration, and you can really tailor it to your specific application, and you can tailor it to your specific application and make sure that it generates the best result for you if you have something that is very non-standard and is not covered by the presets or the defaults. So for example, for time series you can say which quantile levels you want to predict, you can say which metrics should be used to measure your forecast, so you can say with this choice you would say I really care about the uncertainty of my forecast, with some other choices you could say I only care about having a single accurate prediction for each time step, which is like a point forecast. point forecast and also when training the models you can as usual use the presets in all these different modules but you can also use autoglone as like a very simple wrapper about around other libraries so let's say if you wanted to train statistical models deep learning models tree based models for forecasting usually you would have to implement the train validation strategy yourself some evaluation pipeline various data pre-processing by hand but this way with autoglone you can also just say I only want to train these models and autoglone with automatically train these models, evaluate them for you, and basically simplify your work as maybe some more advanced user who doesn't need the presets but wants to do something custom themselves. Yeah, and since we're close on time, so there are also some other modules offering other functionality in Autoglone that I haven't covered here, which is more like additional functionality that you might be interested in using. And finally, just to say, like, if you are interested in learning more about Autoglone, please visit the website. We have lots of information in there. We have the documentation. We have various tutorials, quick start guides, installation instructions. Since it's an open source package, you can visit it on GitHub. You can also, of course, you're also, of course, welcome to contribute if you would like to add some new features or tell us about something that you found in the package, and you can just install it with Vip and try it out. Thank you.

Speaker 3 [37:07]

Thank you very much for this great presentation. So we have quite some questions on Slido. The first one would be, stakeholders often explicitly ask for explainability. To what extent and how does Autogluon enable that?

Speaker 1 [37:26]

So I know we had at one point some explainability features in the tabular module. It's not like a primary focus of the library. So it's more of an accuracy type of, like, that's what we optimize for, I guess. But, like, one could always, as Alexander just talked about, like, by sort of focusing the selection of models to only those that are very interpretable, etc., one could sort of nudge the library towards that direction. However, we do not have out-of-the-box support for explainability, for example, in time series or multimodal problems as far as I'm concerned.

Speaker 3 [38:14]

How would you deploy a model after training?

Speaker 2 [38:19]

is one of the cases where other modules of Autoglone can be helpful. For example, Autoglone Cloud allows you to both train the models in the cloud and also deploy them in there. So you will just get an endpoint where you can send your data and get the predictions back. This is, I guess, the recommended way of doing this, but you can of course also, because it's open source, you can deploy it on your own infrastructure. And we also offer Docker containers, AWS, that you can use for that. Or you can, you know, since it's an open source library, just install it yourself and use it in your own infrastructure.

Speaker 3 [38:50]

What work does AutoGluon do in the feature pre-processing stage? For example, fill the missing values and such.

Speaker 2 [38:58]

This is specific for every model, so this is like one of the ideas of autoglone is to generate predictions that are as diverse as possible across different models to make sure we get the good ones. So for example, for tabular data, this is all model specific, so for each model we try different pre-processing. If you want, you can override those defaults, but by default we are trying to try out many different strategies to make sure we get the right thing.

Speaker 3 [39:23]

Comparing to PyCaret as also an AutoML platform, what are the pros and cons of this one?

Speaker 2 [39:31]

I'm not really familiar with PyCaret, unfortunately.

Speaker 3 [39:38]

Does the time series forecasting also work for relative days, i.g. day one, day two, etc?

Speaker 2 [39:47]

Yeah, I mean, so the generate the forecast contains predictions for multiple time steps. So if you say you only I Mean, I guess you can just generate the prediction lengths to be let's say if you have to need to have forecast for up to 10 days You can either use one predictor and then only look at the forecast for a single day or you can train multiple multiple predictors with different prediction lengths This is the standard place

Speaker 3 [40:10]

Are the sub-models extractable for further tuning?

Speaker 2 [40:14]

Yes, but since we mostly rely on samples it might not be like you might not be only interested in a single sub module But in principle this can be done. Yes

Speaker 3 [40:26]

You mentioned the focus on deep learning methods how many samples it needs for robust solution and in a seasonality based time series for example

Speaker 2 [40:39]

So I guess based on some benchmarking that we did ourselves for time series, for the smaller time series, usually deep learning models don't give you that much of an advantage over the simple ones. So I think in that setting, if you have little data, you would mostly use the statistical models for forecasting. But Autoglom will automatically determine which one should be used. But what I want to mention, we don't really support cold start forecasting, so if you only have the time series for one or two time steps, this is one case where Autoglom is not there yet to support it, but this is something we consider adding in the future.

Speaker 3 [41:12]

How do you tackle backtest overfitting for the time series case?

Speaker 1 [41:19]

I think that's a really good question. So one thing we do is we enable multi-window backtesting. So you can actually, that's one of the configuration parameters that you could have put into the time series predictor objects and saying, during fit, you say, I actually want you to not only focus on one window and try to predict that and model select based on that, do that on multiple windows by iteratively fitting models.

Speaker 3 [41:51]

Is it possible to do hierarchical forecasting with autogluon?

Speaker 2 [41:56]

Not yet

Speaker 3 [41:59]

Okay, why why don't you use profit for time series?

Speaker 1 [42:03]

So, that's another part of, I think, AutoGlo on history. There was a time that we did include it in the model zoo, however, our experience with profit has been that the difficulty of maintaining it, especially during inference and deploying it to cross-platform, it's just like the difficulty of doing that doesn't justify the accuracy benefits. similar benefits from other statistical models.

Speaker 3 [42:34]

Is it possible to parametrize a forecasting horizon for the feature or lack creation?

Speaker 2 [42:42]

Yes, so as I mentioned here you can configure all the models that we have in the forecasting zoo So you can provide those type of parameters and say how exactly they should be configured and which lags they should use

Speaker 1 [42:55]

So I'd like to be clear. This is like a model specific choice, right? So not all models are designed

Speaker 3 [43:05]

So there are many more questions, actually. I bet you can approach those who and ask the questions afterwards. So again, thank you very much for a great presentation.