The evolution of Feature Stores

Speaker 1 [00:07]

much. Thank you very much for joining me, everyone. I know it's always very difficult to have a presentation very late in the afternoon because everyone's very tired and you just had a very big lunch and you just want to go home. I'm going to try to make this as short as possible. What I really want to talk about today is really the evolution of feature stores. If you've been in software engineering for some time, there's a bunch of things that you've seen evolve time over time over time which is really just a consistent thing with our industries right things we invent things but before we invent them they already existed in some form before so what I really want to talk about today really is what features how feature stuff started what they are today and potentially what they could look like in a couple years and just before before I get I get started just a quick show of hands how many people here have like feature stores running in your company, whether you built it yourself or you're using the vendor? Oh, cool. Cool, cool, cool. That's really nice. So before I get started, let me just introduce myself. My name is Ola. I have about eight years of experience in software and machine learning. I'm currently the head of the machine learning platform in Zalando, where I basically just build tools to make the life of machine learning engineers and researchers and data scientists easier. Outside work, I have a lot of interest in low-resource NLP because I come from Africa where there's not a lot of focused on languages that I speak locally. I also have a lot of interest in model quantization, compliance, the AI act, a lot of interest in hemoplasms because I'm also building one at work. I have interest in the future stars. And outside work, you most likely see me playing video games. I'm currently playing The Rise of Ronin, which is a pretty good game if you've not played it. To really just streamline the talk today, I'm really just going to be splitting the talk into basically three parts. Arguably four, but three main parts. The first thing we really talk about today is what a feature store is. Then we look at the first phase of feature stores, how they started. Then we'll talk about what the current phase is. Then we'll look at potentially what they could become in a couple of months, a couple of years, with the ever-changing state of our industries. What I'm going to try not to do today is recommend a feature store. We all know the good ones, we all know the bad ones. I'm not going to call anybody, but I'm not going to try recommending a feature store today. Let's get started. I think the very first question that comes up about what a feature store is really is if you go out there and you look at all of these feature store providers, each one of them has a different definition of what they think a feature store is. But ultimately, when you look at all of this definition, it converges to a bunch of keywords that I think are very important to what we consider feature stores to be. So you have your feature store and then you need to do some transformation somewhere. You need to store the data after doing the transformation. You need to make this data easy for discoverability in your company. You need to be able to share these features with your teams. You need to be able to serve these features to applications that need them. You also need to be able to reuse this feature in whatever way you want to use it. So when I think about all of the different definitions of feature store up there, these are the main keywords that actually do stand out to me. But the first argument I'm going to make in this presentation today is that these different definitions of feature stores, while they are all really correct, they are kind of a more modern definition of what feature stores are. When really the concept started, this is really what feature stores were. Right. So our first argument today is how exactly Feature Store started. And for us to really understand how Feature Store started, we really need to just take a step back. And a couple of years ago, let's say back in 2003, we had the rise of distributed caching, right? That was when Memcached came around. We noticed companies started getting more data and they didn't get to serve more users. and the solution they had to that was, oh, hey, let's cache some of the frequently requested data so we won't have to hit our database every single time. So between 2003 and 2009, we saw the rise of distributed caching. That was really basically just a bunch of key value store that had the data in memory and they gave you the option of doing this in a distributed fashion. So that was like the first major service that really took part in the creation of what we call feature stores today. Then with this, we also had the rise of big data processing. When Google released their MapReduce papers and we saw the rise of tools like Hadoop, we saw the rise of tools like Kafka, Tom, Flink, all of these tools came between 2006 and 2014, but they were all very pivotal in what really became the concept of feature stores today. And the way to really picture this, to see how this became a feature source today, is how data was being processed by a lot of the companies back then. So let's assume you have a data lake somewhere where you have a batch data or streaming data from your data lake. You need to be able to do a transformation for this data lake, right? The cool thing about this is that back in 2006 through 2014, we already had a lot of these big data processing tools. We had Kafka, we had Spark. You have a lot of options on how to process your data. So this problem isn't really new. There are already solutions to solve them. So it made sense to really just use the existing solutions for the transformation part of your data pipeline. Then you needed to store the data after transforming it somewhere. Pretty much, you could store the data wherever you want to store it, because there's not a lot of issues with that. You could get your typical MySQL database and just store your data there after transforming it. Then the next phase in that process usually was, oh, hey, after storing my transformed data, I need to be able to dump it somewhere for my data scientists or my machine learning engineers who want to train models with it. And then the final step of this is I want to be able to serve some of this transformed data to live applications that really depend on it. So if you look at this flow, like the entirety of this flow, this is basically what a feature store is in its most basic form. You have a data lake, you do this transformation, you store the data, and then you do some training. If you need to do training there, you also serve it. So if you think about the way this started a couple years ago to this point, a lot of the things you need to do here, some tools already existed that did this. For the transformation bits, we have the Spark, we have Kafka. For the serving bits was where things became a bit tricky because for a lot of the real-time needs that you have for your feature store, you want to be able to serve very fast, right? Because usually a user will come to your platform. Let's say you're an ad platform. Someone comes to your platform. You get the customer's ID, and you want to be able to get a result to the customer as fast as possible. And this is where really the concept of distributed caching became a huge thing in feature stores. So we have the idea that if you want to be able to sort of serve applications in like real time, you could decide to use a service like Redis as your caching layer, right? Or you could decide to use whatever service you want that can serve as, you could decide to use KeyDB that could serve as your caching layer that will be able to serve all of the requests for you very fast. So if you look at the entirety of the rectangle, every single tool there where things already existed in one form or another before, but we never really called them feature stores back when the need for this became huge. And the need for this became really big around, I would say, 2016, 2017, when a lot more companies were, oh, hey, we can actually do machine learning, and we have this huge data set somewhere. We need to be able to process this data in very large scale, or we need to be able to process streaming data coming from some other application somewhere. We need to be able to serve this data to our live applications running somewhere. And this was really, in my opinion, what birthed features does. This needs to actually be able to sort of serve your machine learning applications real time in your application lifecycle. So, this is the first phase of what I would argue is really the feature stores. So, the second phase, and I got this through a very good diagram from the Hopsworks website, is, you know, this is what feature stores are now. They're huge. Like, so many moving parts, so many cool things are happening within the feature stores right now. You still have your normal data source from either a patch or a stream process, but now you need to think about backfilling, right? So now I have a feature store with some data offline and I need to be able to, oh, hey, I got this new data and I need to update this for a specific time and point. So I need to think about the new thing you need to think about now is how do I backfill this data? But then you also have to think about how do I potentially make this sort of reusable in terms of pipelines, right? So the first thing that really stands out here is you see your feature pipeline where you could process data either on demand whenever you need it, or you could process model-independent data where you don't really need this for an application, you just need to process the data. You do the processing and you store the result in your feature store. Again, this bit, we already had it in the first phase, which was basically just the transformation stage. What is different or what is really new about this stage is really the need to sort of backfill on request as data comes in. Right. The next stage that also becomes very interesting is how do I actually get data out of my feature store, right? So to do this, you have a bunch of use cases. You could decide that, oh, I need to create a training pipeline that I could reuse over and over and over again. So you set up a training pipeline in your feature store and you dump data from your feature store as you need it into the training pipeline. You do your model training. You train the model. You store your model in the model registry. And then there's a second use case here again where you need to sort of do some predictions with the data in the feature store. It's pretty much the same thing. You could do this either on demand or you could do this as you need it to. What is interesting about this to me is there's a lot of similarities here from the first phase into this current phase. And the biggest similarities you can point out here, the first one being for the inference where you still need to serve live applications really fast, you still need this to be as fast as possible. So caching still comes in play here. You still need to be able to cache the request because it still comes in. You want to be able to give them the result as fast as possible. The second thing that's quite similar to the training pipeline, to the first phase rather, is the training pipeline. You want to be able to sort of materialize your feature data into a training pipeline, and you can run the training pipeline as many times as you think is necessary for your workflow. So if you think about essentially where we started from, which is basically this, right, and you think about where we are now, there's quite a lot going on right now in the feature store space as a whole, but there's still, like I said, a whole lot of similarities from where we started from, which is also one of my favourite things about the industry as a whole, right? We just keep building and building on top of what we already have right now. So when you move on and you think about the concept of the very first feature store that really had all of this tooling in place, you know, the first thing that comes to my mind that I could remember that was, like, the standard then was Uber's Palette, the feature store. This is, like, a perfect example of everything we've looked at so far from the phase zero to the phase one. You can see the different toolings there, right? You can see... I'm trying to see if I can just... Yes, cool. So we're going to look at this from the offline and then the online use case, right? You have your data lake. You get some data from... Oh, actually, does this work? Okay, it's just gone. You get some data from the data lake. You do some prep with the data, which is really just your normal ETL for transformation for the data. Then you can store this in the feature store. You can store this in the offline feature store somewhere. In this case, they decided to use the high feature store and then once you store your data in the high feature store you can export this into a batch training process right, you do your batch training and then you can store the result of the training job in the model somewhere, in the model registry somewhere and then you can serve this for live applications that depend on it. The second use case which we've also discussed earlier is you have the features in your high feature store and then you could do a batch prediction here for use cases where you need to predict things as a batch or for use cases where you need to predict things in the live format. So again, the very interesting thing about this to me is there is sort of this consistent need to just, oh, okay, this is where we are now, and then there is the upgrade, right? And what also sort of comes out as interesting to me in all of these different phases of what we've seen as the feature store is really the difference between the difference between the first phase, which is the phase zero and the phase one, right? And the major differences I could see there in my experience with feature store is that there's like more needs for visibility across the feature store right now. As your company grows, you need to be able to sort of, I need to see what sort of features that Team A is processing, so if Team B needs the same features, they don't have to recompute everything from scratch again. They just use the existing features. There is more requirements for point-in-time updates and point-in-time debugging. If something goes wrong, I need to be able to figure out exactly at what point this thing went wrong. I need to be able to get the data for it. I need to be able to debug this. These were a lot of the features that the initial phase did not really have, because back then again, it wasn't a feature store. It was just a bunch of tools that you put together, and then the result was what you would call a feature store. There's also the growing need to avoid duplication. I think I might have used the wrong link, because I still have a bunch of things there, but there is the growing need to avoid duplication. This is a huge problem for me as the machine learning platform in Zonando because we have a lot of use cases where people just do things and they're isolated in their workflow and usually someone comes from somewhere and says, hey, has anyone tried this before? Has anyone tried to solve this problem before? And usually we're not able to answer that question for our engineers. Oh, hey, use this feature or build this model with this specific use case. But then part of the problem we're trying to solve with feature stores right now is if you have if you do a transformation or if your data goes through an ETL process, there should be a very clear lineage of everything that happened from the very beginning to the very end of the process. From where you got the data from, from how you process the data, every single step there should be available in a visual lineage that someone can see and say, this is useful to me, I can use it or do whatever I need to do with this information here. There's also, which is really the last slide, I try to keep this as short as possible so we can all have a very early, you know, evening. I think that the last bit here, which, you know, is very interesting to me and, you know, it's becoming much more interesting to companies that have a certain user base is that you need to make things very, very clear to everyone. You have internal compliance for your company. You have compliance on the organisational level. You have compliance on the business unit level. You have compliance centrally in EU right now. And one of the bigger issues we're trying to solve right now is someone builds a model somewhere in some isolated workflow and they want to be able to ask the question of, OK, this data set that we've targeted has some problems. What are all of the models that we're able to build with it? And one of the best ways for us to be able to answer this question, like I said earlier, is being able to provide the lineage for every single workflow that goes through a data set. So what that means is you use a data set for an ETL somewhere, whether it's offline or whether it's online, there is a lineage there that someone can go, take a look at, and say, okay, this data set had some issue. I use it at this point. I no longer need to use this. I need to report this to someone somewhere to take care of this for me so that's basically what I think have been the three faces of FutureStore so far it's a very short talk thank you very much for joining me and if you have questions, I want a lot of questions so we can just use the rest of the time to talk and take as many questions as possible

Speaker 2 [19:18]

Thank you very much, Ola. Actually, we have a couple of questions for you. Which one is better in your opinion, Bait versus open sources future store, and why?

Speaker 1 [19:28]

I'm sorry is that vendor-based or open-source? I think the person who asked yeah, I think I answered in my very first slide This this bit I said, I'm not gonna recommend a feature store So, you know, I don't want to put myself in trouble, but obviously there's a very good one out there But there's also a very bad one out there. I'm just not gonna call names I'm sorry if you work in the bad one I'm really sorry, but, you know.

Speaker 2 [20:00]

Yeah, okay. So another question. What are obvious sign that a company team needs a future store?

Speaker 1 [20:08]

That's a very good question. I think the biggest need I've seen for future stores, at least in my experience, is to reduce duplication. And that's a huge problem we have across BOD. Imagine you have teams across departments and then departments across business units. And a lot of these teams work with the same data of very similar data sets. And what you want to try and do is like, okay, if someone has computed some values from a data set, you want to be able to make this available to everyone. So at the points where you feel like people are doing transformations of the same data over and over and over again, it feels like a good time for you to actually rethink and say, okay, I think we need to be able to, we need a feature store here to reduce the amount of duplication that we actually have in our process. So I think baseline, that's pretty much the most important thing that I think feature stores actually bring into the workflow for you. Besides that, there are a whole bunch of other use cases that feature stores would be perfect for in any organization setup. But I think the effort duplication is really the biggest issue I can think about here.

Speaker 2 [21:26]

I think it's also related when not to use a future store.

Speaker 1 [21:31]

that's also a very good question I think I would say the only time I can tell you not to consider using a feature store is if you don't have the data set capacity for it because ultimately you really need a feature store if you feel like you have a very large data science, you have a large number of data scientists or teams working together and they need to be able to share things between each other so if it's a very small company of say five ten people yeah a feature store might be useful for you but it could also be an overkill right so i think the the big decider here in my perspective would really be um the company size is um where would decide if you really need a feature store or not because you know like you know in software generally every new dependency that you introduce is an overhead cost that you need to actually maintain and if you're not able to maintain that and what is the what is the point of having a feature stop so yeah

Speaker 2 [22:34]

So we still have a lot of questions, so we'll take one, and Ola can answer the rest maybe after the presentation. Hopefully it's not a long one. I'm still confused about the difference between a future store and a DP. Could you summarize what are the key differences?

Speaker 1 [22:52]

That's a good question, and it's also very similar to a question that I get asked often, which is really, what is the main difference between a feature store and a data lake? That's a question I get quite often. And I think, obviously, the first and the most obvious, I would say, difference between both of them here is that for the feature store, it's a purposely built service for your machine learning services. Databases, a lot of them are not purposely built for, you know, doing machine learning workflows, right? So your feature store is primarily built for, oh, hey, I have this large amount of data somewhere. I need to be able to transform this, put this in an application somewhere. This is like purposely built for that, right? There are, like, other differences. A good example I can think about here, right, as a difference between a feature store and the database is a lot of feature stores, at least a lot of modern feature stores that I've seen have this point-in-time debugging capability in there where you can just go in and say, okay, what exactly was the point of the timestamp or the point of information that I used to train a model, to predict a model, and you can use that to debug your entire workflow. You can replicate that with some databases out there, right? But not a lot of databases have this built in them, right? So, I think that's a huge win for me personally, for a lot of the use cases that we have. Internally, being able to just sit down and just debug things with a feature store is a huge use case for us. I think one other one that is very important to me also is sort of lineages, right? Some databases have lineage by default. A lot of databases don't. But one of the things that feature stores really make easy for you is to be able to track things across board, right? Which, you know, is a really good feature to have as you grow in sizes, because then you can easily monitor your workflows and monitor how things are going within your team or your organizations.

Speaker 2 [25:04]

Another question, how do you make sure that other teams know about your futures?

Speaker 1 [25:11]

That's where the discoverability bit comes in. And really, there are a bunch of answers to this. For one, it depends on sort of the feature store vendor that you go with. Some feature store vendors have this inbuilt where you can just search across the whole organizational setup. You can see all the features that are available in your feature store. You can search per feature group. You can search by the feature view if you create a feature view. So it depends on the kind of feature store that you have internally. What we do, it's really interesting, is we try to sort of align everything with the concept of application. So you have an application that is serving the model or serving something somewhere, and you can view all the components that are tied to the application. You can see the machine learning models. you can see the features, you can see the Kubernetes services, you can see the database services, all tied to one application. So what that means is if you know the application, you can find everything relating to the application, and with that, you can discover all the features that are related to it. So that's the way we set up to just make discoverability easy for everyone at CrossPod.

Speaker 2 [26:30]

In which phase is your company right now and is there any recommendation you would do for others who want to start with a future store?

Speaker 1 [26:38]

I think we're currently in the first phase, which is really like this very huge... It looks huge, but it's really not huge once you just sit down and go through it. This is like the more common phase that you see out there for feature store vendors or for open-source feature stores. Sorry, did the question ask me to recommend? Yeah. Okay, no. We already said that. I'm not recommending, but, you know. Yeah, I think it's difficult to recommend, because the feature store you use really depends on your needs, right? So I can't tell you this would work for you, because I don't know what your requirements are, so I don't want to put myself in that kind of situation, but yeah.

Speaker 2 [27:23]

Last question. Assuming data machine learning pipelines within a data lake and that machine learning is nothing else than complex ETL, why should one use future store instead of using persist layer in DL?

Speaker 1 [27:38]

Sorry, I didn't get the beginning.

Speaker 2 [27:38]

Sorry. Assuming that data machine learning's pipelines within a data lake and that machine learning is nothing else than complex ETL, why should one use a future store instead of using processed layer?

Speaker 1 [27:59]

It's always difficult to answer the question of why you should use a feature store, right? I can't tell you why you should use a feature store. I can tell you why we use a feature store, which is what really I've tried to do through this presentation today. I think data lakes are purposely built for, you know, data access and, you know, governance and all of those things. The feature stores are more purposely built for, oh, I need to access a feature as quick as possible. I need to be able to export the data and do a training job with it. And that's a perfect use case for feature stores. You can do the same with a data link, you can do the same with a database, you can do the same with a bunch of tools out there, but it just feels like, it just makes sense that there are some tools that are like purposely built for some features, and the feature stores just make sense for a lot of the use cases that we have out there. Yeah, that's it.

Speaker 2 [28:49]

Thanks a lot, Ola, and thank you for being here also.