Composable AI: Building Next-Gen AI Agents with MCP

Speaker 1 [00:07]

Hey, welcome everyone. Great to see so many people are interested in this topic, especially since we already had a talk about MCP yesterday by Julian Beck. He covered a little bit the protocol and how to set up servers. Today we will focus more on the why than on the how. We will discover how composable AI helps developers and companies build smarter systems and learn a little bit about what MCP is and how it lets you combine tools for AI agents, like building blocks. So, my name is Martin Seeler, and I am a senior staff engineer in the GenAI team at BlueYonder. And for those of you who don't know BlueYonder yet, we are the only end-to-end supply chain platform. But don't worry, that's not going to be a sales pitch here. But we cover nearly everything around supply chain, like warehouse management, transportation management, demand forecasting, pricing, and, of course, all those platforms have APIs. And you can interact with all those products via the UI, and they also can share some data. But from an agentic point of view, those systems are pretty siloed. Of course, we can fulfill workflows, like if a truck comes in early, 10 minutes, tell all the workers to stop picking up and instead clear the truck. But the higher the complexity gets, it more breaks simple workflows. Like take, for example, if a ship is stuck in a Suez Canal, right, could I tell an agent to reroute all my trucks and ask it what goods are delayed now, or can I ask it to rebalance all my stocks across all warehouse systems, right? That would be pretty awesome. And this is where agents come into play. They plan and act on your behalf. That's one step above simple workflows. And even yesterday in our keynote, we heard that agents even win Nobel prizes nowadays. So under the hood, an agent is powered by an LLM, right? So LLMs are just token predictors. They just predict one token after the next one. One token at a time. So how are we going from token prediction to rerouting trucks to rebalance warehouses? How can agents actually perform actions on behalf of us? Please hands up if you think you're comfortable knowing how agents actually perform actions. Five, six, I see maybe six hands. Okay. Then let's start by defining what an agent actually is, how it uses tools. And we need this especially to understand what MCP really does for us. So an agent at its core is just a model like GPT-4O with tools like we see down there calling in a loop. And without tools, it would be pretty dumb. Let's say we have a weather agent. You are a helpful assistant for the weather. And this agent doesn't have any tools at all. If I would ask it what's the weather in Hawaii, it would either say, I don't know, that would be the best case, or it would come up with something. It would hallucinate, right? And tools let agents fetch information and trigger systems and get stuff done. And they need those tools to act on behalf of the user, to perform actions. And that's why OpenAI came up with this concept called function calling back in 2023, which gives LLMs the possibility to output structured JSON to call predefined functions. And tools like here in the bottom could be defined as get the weather for a certain city. And we have a function with a name, get weather, with some parameters like the location and the description. And the LLM then was trained to use those tools if it thinks it makes sense in this context. So looking at this in this workflow, it could look like this. I asked the model what's the weather in Hawaii, and the LLM now sees this function, oh, it now turns out that I have this function getWeather with a parameter called location. I would like to call this function, please. And we, as developers, now see that the AI asks us that it wants to execute this function, getWeather, with the location Hawaii. But it doesn't know how to execute it. All it knows is what the name of this tool is, what parameters goes into it, and what's the description of this function is. All those information get injected into the system prompt as well. But it's our job to now actually execute this function either by reflection or looking it up in a dictionary, and, of course, then we can feed this back to the LLM, and the LLM and then transform this response into a more human readable form like Hawaii is 85 degrees Fahrenheit and sunny. That is how LLMs actually perform actions by using function calling. And of course nowadays modern frameworks do all of that for you. So let's have a look at an example. Here we have the framework small agents by HuggingFace and as I described earlier, no, I want not this one. Let's take a highlighter. We have a name of a function, get weather, we have a parameter and we have a doc string for the description and then we have some implementation. And down there, we define our agent, and we define a model, GPT 3.5, and we give it a list of tools. And whenever the LLM decides, oh, I want to call this function, then the process we've just seen gets executed. So it will then see, ah, this function should be executed, and it goes on. Looks pretty fine, right? So what's the problem, and why do we need MCP? Well, first problem here, you're locked into a vendor, so when you want to use the latest and greatest framework, which might not be small agents, then you have to copy-paste code over to a new framework. That is not great, but doable. But there's an even worse problem. Let's say you build a chatbot that talks to Jira and your warehouse labor data, you would have to build a custom API to call this endpoint. You have to handle authentication, and especially you need to learn everything about this domain you're accessing in dev. So back to Blue Yonder world, we would need to talk to so many different products like warehouse management, APIs, transportation management, each with its own quirks and teams that wouldn't scale at all, right? So thankfully some clever folks at Anthropic thought, what if agents didn't need to know everything about everything. And that was when MCP was born. MCP stands for Model Context Protocol, and it's an open standard by Anthropic. It connects AI agents with internal or external tools and data. It's much like a USB-C, right? Maybe you can remember your drawer full of different cables and connectors. Every phone came with its own charger, right? And then suddenly we had USB-C. Now we don't worry anymore. We buy a new device, we plug it into USB-C, and that's it. We don't have to worry about it anymore. And MCP standardizes how tools are exposed via MCP servers to agents. So any MCP-compatible app or agent can plug in any MCP server without any extra work. Maybe you need Jira, you need MongoDB, there is definitely an MCP server for that. And in the beginning, the adoption was good but not great. But after OpenAI and Google announced that they support MCP, the adoption exploded. Like now we have thousands of servers available. It's like an app store for tools for agents. And as a developer, that's great because we don't have to reinvent the wheel over and over again. We can use existing servers and let your agent do everything much faster. So let's have a closer look here. So instead of hard coding APIs, your agent now talks to MCP servers. They wrap external systems. MCP is basically the translation layer between agents and the real world tools. So let's start with this host here. And your host can be anything. It can be your application, your agent, but it can also be an existing application like cloud desktop, cursor, even VS Code has MCP support now. And then you have MCP clients. And mcp clients runs on the host and talks to the mcp servers. They start by calling something like list tools. So the client tells the server, hey, what actually are the tools you are supporting? And it responds back with a list of tools available. So the transportation is either standard I.O. if it's running locally or server sent events if it's running remotely. And even in the next version, there are stateless, more streamable HTTP connections supported. But nowadays we are running them with standard I.O. and servers and events. And now the MCP server exposes the actual tool. Like you could have a local file system, MCP server, which has tools like list all files in this directory, read this file, delete this file. Now you could have an application and talk to the agent, hey, please count the number of files in this directory. Then of course you can call external APIs. Let's say Slack, for example. Then the Slack MCP server might expose tools like list all channels, join channel, kick user from channel, or send message to a channel. And by combining those two MCP servers into one application, you could now have something like, please find out who was the last editor of this file and write them on Slack, for example. So, just by combining two MCP servers, we have now much more capabilities in our agent. And of course, there are MCP servers for data access, like PostgreSQL MCP server with tools like get me the schema, execute SQL, or even Snowflake or Elasticsearch MCP servers. also more human-like MCP servers like browser, click this button, enter this text, move mouse to this position, so you can now have an agent which interacts with your browser. You can mix and match servers to build really powerful and flexible agents. So let's say, for example, you have a marketing team and you give them access to cloud desktop and an MCP server connected to your database. They wouldn't need to write this Jira ticket anymore. Please aggregate the latest quarter results from the database for me. They could now interact directly with the database without knowing anything about SQL just by composing those different tools. Pretty cool, right? So, in code, this would look like this. MCP servers are actually surprisingly easy to develop. It's actually so easy. There's even a target show MCP server yet, so just to give you an idea. So we are using fast MCP, which basically looks like fast API. You define your server, and much like before, you define your function, your description, and your parameters. And then, of course, some implementation, how you want to implement this method. And then you run it. So build one is exposed, two is everywhere. This is where Composable AI starts. And yes, we are at PyCon, but MCP isn't bound to Python. There are SDKs available for every kind of language. But one thing we have to notice here is, as an MCP server developer, you don't have to know anything about AI, actually. It's much like building an API before. And then, of course, we have the agent side. And by using MCP, you can stay framework agnostic. Now back to our small agent framework, we are using the same framework and we also define this agent again, we use the same model, but this time we say we want the tool collection from MCP and point it to the endpoint we just deployed. And what it does is when the agent starts, it spins up this MCP client which talks to the MCP server, hey, what are the tools you're exposing? And all those tools get injected into this agent, which means every time you start your agent and you have updated your MCP server, the agent can get more and more cool tools to use. So the and now it's very easy to switch a framework because this conversion is supported by every major framework, like and even Google's development kits. So you can have your tools, how you interact with your code in MCP servers, but your agents are completely decoupled. So function calling is about what and when to use the tool, and MCP is about how tools are served. So what we get with this is without MCP, agent builders must understand every API, authentication flows, data formats, the integration is painful and tightly coupled, but with MCP there's now this great separation of concerns we got. Product teams own their data again, ingestion logic and authentication flows. The MCP server developers can handle domain specific details and the agent builders on the other side can focus on all the AI stuff like model selection, prompt design, evaluations, user experiences and so on. This means real ownership and real speed during development. But just as a heads-up, agent builder is quite a buzzword, so expect people to want to work on this side here because they want to see it in their resumes. And it's not about AI, actually. It's about creating value. Most companies don't care about AI. They care about what AI delivers. For example, they care about saving time, cutting costs down, increasing value. Ultimately, return on investment is what they care about. They want to save money. So chances are high that you are coming from a company which already solves such a real business problem, right? You may have already invested time and resources in building APIs for your customers or for internal services and systems. So it would be great if you won't have to reinvest everything again just to build agents. It's better to give agents access to what you already have. So, for example, you could have an MCP open API server. This is a bridge between an open API spec file and an agent. For example, if you have an API which runs on Swagger, you have an open API spec. And you might have an endpoint health with a get function. without any parameters. This MCP OpenAPI server would now read the OpenAPI spec file and see this endpoint and create a tool called get health without any parameters and the description out of your OpenAPI spec file. And propose this, expose this as a tool to your agent. And you could now talk with Claude, for example, or even VS Code, hey, check if my API is running. And it would call this tool. And you would get the human readable response, yeah, your API is running. So this is where MCP can help you build composable AI. And just to give you another example, this one here is Manus. Manus is an MCP-based tool. And it's an agent. And people pay $200 per month for it. And this is Manus in action managing 50 social media accounts at once. Pretty amazing, right? And as I already spoiled you, turns out Manus in the end is only a Claude Sonnet model running with 29 different MCP servers under the hood. So, actually, you don't have to create something completely new To create real value. It's about composing what you Already have in a clever way to create new value for your Customers, right, because people are willing to pay $200 per Month just by combining the right tools for your customer. So, hands up, who remembers this guy? Yeah, right. This is Inspector Gadget and he had tools for everything. So now that we know how to attach MCP tools to an agent and give it more tools, why not attach like all the tools we have to one agent and make the ultimate agent, right? Well, turns out 50 to 100 tools is a sweet spot. If you have more than 100 tools, you should lift this up into a more meta level. Remember, all the tool definitions get injected into the system prompt together with the description, the parameter list, typings, and everything. So the more tools you have, the longer your context already gets and the more confusing it also gets. So if you have more than 100 tools at hand, I would suggest lifting it up into this meta level and you could have a tool searching tool, a tool which helps you search other tools or since we already have seen how easy it is to create an MCP server, why not have a tool which creates MCP tools on the fly for us? And turns out there is actually such a tool from Hanweg Alzheimer, which is called MCP Tool Builder. a tool which creates tools on the fly for you. So, as I said, watch out when you define your tools. Just like we care for accessibility for our users, we should care for LLM accessibility as well. So let's say you have a tool called foo with a parameter x. Don't expect the LLM to understand what this function does, right? I mean, in the end, it just understands what this function Does by reading the description and the name of the function. And on the other side, if you have a function maybe based From an api and a very complex post method, which has a name And a description, okay, but then it expects 50 different Parameters, that might also be very confusing for the llm. So you might train your llm to use those tools better, but i But I would suggest then turning it into a real MCP server, making it more accessible for an agent. Then, of course, your tools can be malicious. Let's take our GetWeather example again. As we have seen, we deployed an MCP server externally, and we now inject all the tools. But what we inject is only the name, the parameters, and the description. What if the actual implementation caused some malicious code? Because we don't see the code of this mcp server anymore. And also, just because there are some data connectors like PostgreSQL mcp servers, yes, they have tools like read the Database schema, for example. But in the end, they don't Really know your domain, your database. They simply can execute some tools. Don't expect them to understand your whole database, okay? So, now that we know how to give a single agent access to data and tools, turns out Google just released agents to agents and even newer protocol for AI agents. But don't be scared, it's not a replacement for MCP, thank God. They actually go pretty good hand in hand. For example, we can now have multiple MCP-powered agents because the real power comes by composing multiple highly specialized agents together which can communicate with each other and hand over tasks and information and plan how to execute a certain action. And it's even better when they can communicate across organizational or technological boundaries. Like you could have your local AI agent talking to an externally running restaurant agent to book you a table for tonight, that would be pretty easy with A2A. And in the end, it makes sense for you to build composable systems by adopting standards like MCP and A2A. As we have seen earlier, especially for large companies, it makes sense to keep this ownership. So you have teams building MCP servers and AI teams building the AI stuff around and let them work together. But there's also an external factor which makes sense for you to build MCP servers or agents. Because let's take the Tagesschau, for example. Tagesschau MCP server. Tagesschau is not really known for a crazy AI company, right? But they released, actually it's someone else, but they have an MCP server. And this makes other agents and companies use their product via an agent. So this is a whole new industry. And another example is Spotify, for example. Spotify also has an MCP server with tools like play, start, look up songs. This is also only the case because Spotify wants to give the possibility to agent builders and other people to interact with their products via this new standard like MCP. So it does make sense for you to adopt the standard. But in the end, let's see how things evolve and which standards get adopted. We don't know yet. Only time will tell. And with that said, I thank you very much. I'm happy to answer any questions. We have a booth down there outside, and you should check it out. We also have some pretty cool Vibe coding stickers there. So thank you very much. Happy to connect.

Speaker 2 [23:48]

Thanks for that great talk. We have a lot of questions here.

Speaker 1 [23:53]

That's not good.

Speaker 2 [23:55]

Let's start from the top. Agents are powerful if the LLM can detect the right tool. Is there any metric to reliability of correct function being chosen?

Speaker 1 [24:10]

There is none that I know of. What you can do, as I said earlier, models are trained to use the tools they are given to. So during training, models were trained on using the right tools. And if you have very complex tools, for example, it is also possible to fine-tune models on function calling. So if you define your own functions, you can fine-tune your models to better call the tools you give it at hand. So this is one possibility. And that, of course, enhances the accuracy of the correct tool calling. That is one way to achieve this.

Speaker 2 [24:54]

How does one usually deal with security in the context of MCP? How is it usually validated that the user can ask and have information about a given topic?

Speaker 1 [25:06]

Good question. Well, when you start an MCP server, most of the time you start it with some environment variables like a connection string to your database and a password and so on. But the protocol actually supports authentication as well. So you can forward authentication headers like bureau tokens on behalf of the user, which makes sense because in the end you want to have some traceability, right? You have an entry point and you receive a request from the user and the agent might even get this bureau token from the user from an initial system. And you pass this authentication bureau token forward to external services. This is already supported by the protocol. But, yeah, those are just two methods to handle this.

Speaker 2 [26:03]

Do I need specific MCP servers or can I just easily reuse my existing APIs? You mentioned an OpenAPI MCP server. Yes.

Speaker 1 [26:14]

Yes. That's actually the best case to start with. There is actually multiple, not only one. Just look up at awesome mcp servers. There's a huge list of available mcp servers. Or look it up. I don't pick one specifically. But there are open api mcp servers available. And they get as input an end point to your open api spec file. Read all your specs. So every end point. Endpoint, like get posts, update posts, and transform every Endpoint into a tool, but remember, only the name of the Function, the method, and the description, and expose them as Tools to your agent. That is actually possible, and That's a great way to start using mcp for your agents, Because you then reuse your existing infrastructure.

Speaker 2 [27:09]

It feels overkill to split every bit of logic into a separate server are there any concerns? performance and resource wise

Speaker 1 [27:18]

Well, I mean, one server can expose lots of different tools. That's not the case that you have only one tool. I just implemented one tool, but you can have multiple tools in one server. And in the end, it's the same discussion with APIs. How are you managing APIs and how are you shaping them? How many endpoints have one API? So that's the same discussion. That depends completely on the use case, on your environment, on your products. That's not something MCP can solve for you. Thank you.

Speaker 2 [27:49]

Does the MCP server developer have to know which function return values are LLM friendly? For example choose between plaintext, json and so on?

Speaker 1 [28:01]

That's completely up to the MCP server developer. So I would suggest you start with whatever you prefer. Let's say JSON, for example. And especially if you are the developer and the consumer as well, so you're building internal agents powered by MCP, it does make sense that you have evaluation pipelines and over and over again evaluate those metrics and compare what performs better. So you could start with JSON or plain text and change things up Because you can measure it and improve it If you can't measure it, you can't improve it, right?

Speaker 2 [28:41]

How can we make sure our data is safe when using such MCP model?

Speaker 1 [28:51]

It's a too broad question, I mean, you can run your MCP servers internally inside of your VPN without access to the outer world, and if you have a laptop running in your company, you can, of course, attach cloud desktop or VS code to your internally running MCP server. And you can even use authentication in the protocol, so forward an MCP, a bureau token, So yeah, that is pretty secure for me. But in the end, it completely depends on your infrastructure and what use case you're building. If you have an MCP server running somewhere publicly, because that's your infrastructure, and agents cannot run in the same VPN, for example, then of course you have to think about this again to make it more secure.

Speaker 2 [29:44]

And now we are at the end of time. So let's thank the speaker again