Secure ML: Automated Security Best Practices in Machine Learning

Speaker 1 [00:03]

Yeah, so as mentioned, today we're going to dive into a very interesting topic, which is basically on machine learning security. So we're going to talk a little bit about that. Just a bit about myself. My name is Alejandro Saucedo. I'm Engineering Director at Selden Technologies. It's a machine learning deployment and monitoring startup. We have one of the most popular open source projects for machine learning deployment in Kubernetes. I'm Chief Scientist at the Institute for Ethical AI, which is a research center that focuses on developing frameworks for responsible ML and governing member at large at the ACM, which is sort of a private body for computer scientists. And yeah, so as I mentioned a couple of times as people were coming in, some of the resources for the talk, you can find them in this repo. So this is going to be quite a practical talk. So I want to not just show you a bunch of slides, but give you some practical examples. And the way that I wanted to do that is to show you about and tell you about security in machine learning with some practical examples of machine learning gone wrong. And what are some of the potential challenges that may be introduced, may introduce security flaws at each stage of the machine learning lifecycle. So the repo contains some float ML examples that we're going to try to cover in this session. It also has a link to the slides and link to other relevant resources if you want further reading. So let's talk about security challenges in machine learning. The interesting thing about security in machine learning is that it's not as defined and explored as you would expect, similar to what we have seen in software engineering, in DevOps. There's a lot of practical frameworks that can be adopted, as well as some understanding of what are the main potential security vulnerabilities at different stages. And there's a lot of resources on machine learning applied to security, things like malware detection or spam filters, et cetera, et cetera. But what about security in machine learning? So in this session, we're going to be taking a similar approach that was taken in those other fields, try to identify the key challenges of security in machine learning, but also acknowledging that it's impossible to make systems completely without vulnerabilities or completely unhackable, right? But what we can do is we can mitigate undesired outcomes, and we can identify those critical or high potential vulnerabilities that actually need to be addressed. As opposed to, you know, try the impossible, which is like making a system completely, fully out of vulnerabilities, which would be impossible. And the solutions will not just be technical, and they will be technical in nature, but ultimately they will still rely on humans and process. right? So that's the key thing. And we're going to be presenting it under the umbrella of this concept of ML SecOps. So there's already a concept of SecOps, security operations, as well as DevSecOps. So this is ultimately kind of like a consortium of all of those different components. So identifying the best practices in the intersection of machine learning infrastructure, DevOps, developer operations, and automation, as well as security policies. So that's kind of an angle that we're going to be taking today because this is a python conference we're going to be actually covering a little bit more of the python and we're going to be covering a little bit less of the infrastructure but as i'm going to reference later i have other deep dives on you know different audiences like kubecon which is going to be more on the infrastructure side so there is also a lot of content that we can rely on right there's there's a lot of great research and great resources that have already done the hard work of identifying some potential areas of relevance, specifically in the software world, right? So this is the OWASP, which is the open web sort of like project for security, and they have this very famous sort of resource, which is the top 10 vulnerabilities that they release pretty much every year. And, you know, I'm not going to go in detail right now, but you will see that this top 10 vulnerabilities, we're going to come back to them, and we're going to see which are the flawed machine learning vulnerabilities that we can map into this OWASP, like the OWASP ones. And today's talk is not going to be covering the whole spectrum of machine learning, right? Like, we're going to be just talking about machine learning security. So, it has to be a little bit kind of, like, succinct, and we have to, like, you know, look over a lot of the more complex parts. But what we are going to be doing under the hood is we're going to be taking a machine learning model, and, you know, hopefully, like, a lot of you already have exposure to some kind like machine learning training even if it's like the hello worlds like the iris classifier which we're going to be using um and we're going to be deploying this model so actually taking the the model that has been trained as its binary i'm exposing it and putting it as a running microservice as a service that listens to requests and provides responses the request would be the prediction input the response is the output so so we're going to actually like you know identify what are the potential flaws at the different parts of this machine learning life cycle, so the training, the deployment, the packaging, the monitoring, et cetera, et cetera. If you want to get a little bit more detail, so this is the TLDR, this talk is going to be TLDR on the machine learning deployment, if you do want to actually delve into what is happening, there's talks linked in these slides that you can use, so that you see that is not like just all hand wavy and magical. There's some resources behind this. And what we're going to be doing is we're going to first train this machine learning model and we're going to deploy it using this tool called Selden Core. This is the tool that allows us to convert our binaries or our Python code into a fully fledged microservice, our machine learning model into a fully fledged microservice. So let's actually do that. So let's take a scikit-learn model with the IRIS data set, right? So that's basically, you know, the usual hello world of machine learning. We're going to train a logistic regression model. We're going to fit that logistic regression model, and we're going to test. So we run a prediction. The prediction is an input of a flower, an iris, and it predicts what class it will fall under. We're going to then actually take this model that we've trained, and we're going to persist that model, because we need to deploy it, right? Somehow we need to say, let's save the model, and then let's put it inside of the microservice. So we're going to actually pickle this model. We're going to see what's inside. We can see that there's actually the logistic regression class, so this is how Python knows how to recreate it. Let's actually copy it into a Google bucket, S3, right? Just a remote bucket that then we can deploy using Selden, right? So we now have deployed this model. The way that it works is there is a container that runs in Kubernetes, that hand wavy magic. The container itself pulls the artifact and serves it, right? That's all we need to know. It's a microservice with an endpoint that if we send a request, it gives us a response. The same thing that we saw on our predict function we are seeing here great we have a model deployed easy simple and we're ready to go uh you know home because the work is done right high accuracy model is running in production you know that's that's where the life cycle of the model finishes unfortunately that's not the case right and we know that with the different stages of the machine learning life cycle what we have done is you know process the data train the model persist the model deploy the model and serve it right so the challenge is now we need to ask the question what are the security what are the areas that actually are at high risk of potential security vulnerabilities and the only places that are at high risk of security vulnerability are the parts in red right so basically everything right and we're going to actually talk a little bit about that we're going to talk a little bit about different parts of security risk in machine learning we're to talk about the risks in loading model artefacts, on people getting access to the model or the endpoints of the model, the challenges of the dependencies of the artefact and the servers, the vulnerabilities in your code of your actual model, the vulnerabilities of the images, and then some honourable mentions because I only have 25 minutes and I cannot cover all of the stuff. Even if I had 45 minutes I wouldn't be able to cover all of this stuff because it's just so much. So the first part. Who loves pickles? We all love pickles. We know what pickles are. Pickles are the way in which Python is able to dump a binary and reload it in a different environment. You know who else loves pickles? Machine learning practitioners. They love pickles. They use pickles like there's no tomorrow. And there's a lot of best practices that are now moving away into more sort of like robust serialization methods, but we're going to see how this actually becomes a challenge with the security areas of pickles. So let's actually load that model that we just trained. So this was basically stored as a pickle. That's what we deployed. We're going to just reload it into Python memory. We now reload it into this model safe. We run the prediction. Model safe still works. Now we're going to do an injection into this pickle. We're going to take that function that Python runs to understand how to load the pickle, which is this reduce function. We're going to then inject it into that model itself. This could be run somewhere else, I don't know where, it could be run without the data scientist knowing. The model still behaves correctly, you can still run inference, but the only challenge is that when you actually save that model, what you save is actually something different. So what are we saving here? We're saving this rubbish, whatever it is, for showing this is just going to copy your local environment and put it in a file called pond.txt, right, and we're just going to show that this actually can run anything, right, like what I can show is you can actually pull the Kubernetes secrets and store them in a file, or you can pull the Kubernetes secrets and send them into a remote server, you can do whatever you want, you can serve like a SSH session that you can connect, you can do quite a lot of things, so we're going to now actually copy that artifact, we're going to deploy that artifact into that unsafe model, so now it's running, right, there's nothing wrong because it just loaded it into memory but if we not look at that container we can see that the pond.txt file was created if we actually load it locally right we can see that there's no pond.txt file locally if we load that model locally we can see that now it creates that pond.txt file right so so this is just basically saying like you know the pond.txt doesn't like you know kill your computer but you can actually do whatever you want so that actually shows some challenges and the problem is like what i said a lot of machine learning practitioners and a lot of like very very uh you know high profile frameworks use this right so uh major frameworks like scikit-learn pytorch they actually have especially scikit-learn they actually tell you this is the way that you actually save a model right and the challenge is that there's also exploration okay how do we how do we solve this right do we have do we create some sort of like super intelligent super smart code scan python scan pickle scan libraries that just like check for malicious code and the actual recommendation is no right when it comes to pickles and when it comes to just you know running code even if it's containers it it comes into a trust or discard mechanism right you either have a mechanism in which you can ensure that the the artifacts that you're that you're retrieving are legit uh you know irrespective of whether they are you know pickles or or not i mean of course if you can avoid using pickles that's even better because they are dependent on the python environment and there are you know tons of other issues that pickles bring that you know are well beyond just security but there are some considerations to make sure that it is more of that trust and discard perspective right as opposed to trying to bring in some sort of like although there could be some value of exploring some sort of code scans there are there are some frameworks that that do provide that which are pretty interesting and i do recommend checking out but that is just some key consideration and as as we go through the talk we're going to be mentioning some of these touch points throughout the CICD pipelines, your ETL pipelines, your data science human workflows that actually benefit from having this trust or discard mechanisms. You need to make sure that there is a trust by design process within the different phases of your machine learning system. And maybe this doesn't make sense when you're just like a team of data scientists who are responsible for the serving of your models. This may not make sense if you're just one data scientist who's responsible of just their Jupyter notebook and their serving of their model. But this makes sense when you have, like, multiple business units with data scientists that produce models that then are, you know, connected into pipelines through machine learning engineers and then actually monitored by DevOps teams, right? So that's where it kind of, like, starts making sense for that. Now, second part. This is the kind of, like, you know, sexier part of the machine learning security, the one that, like, everybody talks about, but, you know, we don't see as much in the world. and this is what can you do when you have access to the model endpoint or artifact and this is where the concept of adversarial security comes in so adversarial security is the ability to be able to take a model and try to ask the question how can i fool this model and how can i fool this model whilst fooling a human as well right so from this example these are some set of images for the c410 classifier that classifies you know one of the the 10 images or actually I think it may be one of the slightly larger ones data sets but yeah so you can see that like these are pictures of a horse of a truck and a dog and after being run through an adversarial sort of example to create an adversarial example you can see that the picture of a horse with you know undetectable human eye you wouldn't be able to know what actually makes this suddenly a dog right or that truck an automobile or that dog a horse right so this is basically the ways in which you can actually modify an example in order for you to be able to provide that capability of fooling a model as well as a human that looks at it and this is it is interesting because ultimately this means that if you just give access to the end point of a machine learning model without actually the respective you know monitoring or logging to understand that there may be some malicious intent to reverse engineer in some way or another the behavior the internal behavior of the model then you may end up in a situation where yeah people can exploit that right and I have you know again an example that shows basically hands-on how you can create this adversarial examples but yeah for the you know perspective of time I'm not gonna be able to dive into it but you will have you know more than more than more than and the ability to dive into it yourself. So, the third thing is to just emphasize that once you actually are able to detect that you need to guard your potential model for adversarial attacks, you can introduce things like adversarial detectors. So we actually at CELDON have a library called Alibi Detect that provides some state-of-the-art algorithms where you can leverage some of these adversarial detection mechanisms. So it allows you to run this detector in front of your model such that whenever a prediction comes in, you can actually ask the question, is it an adversarial slash malicious example? The challenge is that if somebody actually has access to the model artifacts itself, they may be able to find examples that can fool not only the model, but also the detector. And the challenge is also, once somebody has access to the model, there are some challenges of not just being able to fool the model itself but to be able to extract insights from the model itself so this is an interesting piece of research that talks about extracting training data from large language models so and there are some some machine learning models that actually require having the data you know with with the model itself to be able to like perform the inference so that those are just some some things uh that you have to take into consideration you know if you if you uh perhaps break this kind of like best practice in some of the parts of your pipeline you may be exposed to some of these potential challenges. Okay, number three, dependencies. So the challenge is Python. Python provides a lot of nuances when it comes to dependencies. I'm sure anyone that has maintained a Python library for more than a year would know about this. One day you go home and everything works correctly, the next day you come back and your CI tests are failing, you try to figure out why, and like a fourth level dependency got updated, And the third level dependency package didn't pin it, even though you pinned all of your dependencies. And welcome to pip, right? So the challenge with the Python dependency management is that this is not something that out of the box you're gonna be able to lock. And you do need to make sure that this is a security consideration, because there can be a potential third, fourth, fifth, 20th level dependency that may have been updated with some malicious malware that may affect you. And we have seen a lot of examples, which I'm gonna talk a little bit about, of how they are provided. So here I have some examples where we can show some of the dependencies, how like with PipDevTree, you can see that scikit-learn actually pulls some third level dependencies. You can use other tools like poetry. I don't know, who here has heard about poetry? Raise your hands. Nice, okay, you're all on the good practice. Yeah, so poetry basically creates a poetry.log file, which is almost equivalent, but not really as doing a pip freeze into a requirements.txt file. But yeah, so basically this allows you to create a complete sort of log of your Python environment. Disclaimer, this is just your Python environment, right? This is not your entire Linux non-Python dependencies, which are the thing that we're gonna cover in a bit. So yeah, and this is challenging because we have actually seen well-intended actions that actually have raised some ethical questions, but that actually reflect the introduction of code that will potentially affect any development library that depends on that specific library. So this is not something that is completely unheard of, that never happens. So it's something to actually be conscious of. So I think that's one thing to mention. And I think, yeah, I have one more thing to mention. So there are some libraries that are also kind of like improving kind of like the infrastructure. So MLflow, you know, there's actually some really great talks at this conference on MLflow. We've actually been doing some collaboration with our Python machine learning server called MLserver. So we actually tackle some of those dependency hell challenges through the introduction of CondaPack, where we actually are able to like dump kind of like a freeze of the environment that then the actual microservice can load it. And then using stuff like Poetry so that we can actually have a complete reproducible environment. The thing with CondaPack is that it's actually, you know, with Conda you have more than just Python, right? Like you can install Java. I mean, for good or for bad, right? I mean, for some challenges. And there are other things that you can leverage for dependency scans. So you can actually use things like, you know, open source frameworks like SafetyCheck, which actually checks all of your dependencies, validates whether there are dependencies that have vulnerabilities, dependencies that are actually potentially too, you know, old, et cetera, et cetera. So, yeah. So, I mean, I think, you know, you'll have enough time to cover that on your own site. Code scanning is also important, making sure that you don't have, like, bad practices on your code itself. We know that there are some potential code vulnerabilities that have now been removed in frameworks like TensorFlow, right, like, they actually had, like, import from YAML, we know, I mean, if you remember, there was a massive vulnerability with YAML, with the load unsafe, that, you know, could be exploited, and any framework that had that, like, was actually reminiscent of that, and ultimately code scans allow you to, like, avoid those things, and, you know, you can check the code scan example, which basically shows you some, like, potential vulnerability areas, and, you know, one thing to just like bear in mind is that Jupyter notebooks are no different right that's still code until you know once they grow to a certain level so that's that's something to take into consideration image scanning another important thing you know making sure that you scan your containers for CVS and update the internal dependencies and some honorable mentions making sure that you actually have your encryption for data artifacts at rest and in transit authentication authorization and infrastructure hiring there's some really good resources that you can check out there. Again, we talked about the OWASP standards. These are the flood machine learning equivalents. You know, I'd love to go into a bit more detail, but you know, you will actually like be able to see it in more detail. So this is, you know, also kind of like as a bit of a pun, the FML equivalent. And you know, I work hard to, you know, get that acronym to work. But yeah, so this is actually quite interesting. And we're going to be working on standardizing them a bit further. So yeah, so the anatomy of production ML, we've been able to like talk about the training, the CI, CD, the deployment, we've been able to also like mention some of the different personas, the data scientists, the machine learning engineer, DevOps engineer, and, you know, some further resources that you will be able to find on the repo, and then as a reminder, the reasons for the talk are on the ethical ML slash FML security, you know, repository, where you'll find all of this, right, you'll be able to run it yourself, you'll be able to try it, and if you don't like something you'll be able to submit a PR because open source is great and with that thank you very much and I think we are on just on time hopefully yeah

Speaker 2 [22:04]

Yeah, we are even awesome Yeah, thanks you were even a bit faster, so again, please feel free to ask questions I Don't see one yet Do you think that the security in ml is reflected in concern or that the customer is willing to pay for or what is your opinion on that

Speaker 1 [22:39]

Yeah, that's a great question um so I would say to specifically your question Yes, absolutely like they are like you know security is is their last name right, but they're clueless Absolutely clueless, and I just know that because putting together this talk Emphasize how little resources there is there's a lot about machine learning for security, but not about security Machine secure machine learning so yeah

Speaker 2 [23:07]

Then there's a question from Stefan. Have you ever run into problems with fixed and conflicting package pinning?

Speaker 1 [23:14]

Have I run into an issue of like sort of configuration management of dependency pinning?

Speaker 2 [23:20]

I believe that's the case.

Speaker 1 [23:21]

Yes, yes, yes. Oh, absolutely. Yeah, yeah. I mean, you could hear the pain in my voice when I was talking about that. We actually had like, we've had several, we've actually had several, like, you know, the one I remember from the top of my head is like, I remember Azure updated their framework. So like, you know, three, like, there's like 20 major versions above. And that just broke like, yeah, half of the libraries out there, because tons of them actually had that dependency for some weird reason. And it was like, yeah, like a third, fifth level dependency.

Speaker 2 [23:21]

Yes, yes, yes.

Speaker 1 [23:50]

And we've also run into this issue when addressing CVEs, right? Like you try to like pin something and then, you know, a second level, third level may have some issues as well. So that's why we just moved to poetry.

Speaker 2 [24:01]

Okay, great, and then there's another question could you elaborate on how you may be able to detect an adversarial attack with Alibi?

Speaker 1 [24:08]

yeah for sure yeah yeah well so so there's actually like a an end-to-end example a jupiter notebook that guides you with a c410 data set um so probably that would that would cover it in more in more depth but the way that i would say it is that the adversarial detector would actually try to get um the you know input of the that is coming into the model and try to like see how feasible that is for actually going into the model itself based on the the probabilities of the output so that would actually like work in tandem with the model um so it's quite an interesting area area research so I recommend check it out

Speaker 2 [24:41]

Okay, great. Are there other questions? I think then thank you for your time. Thank you for the amazing talk. Awesome.

Speaker 1 [24:49]

Thank you.