Building Agentic Systems with Python, LangGraph, MCP, and A2A

Speaker 1 [00:18]

Hi everyone, dear attendees, please take your seats. And let's give a warm welcome to Håge Nosekabel with his talk on building agentic systems with Python, Lungraph, MCP, and A2A agent.

Speaker 2 [00:47]

So yesterday evening, I actually thought I should create a bingo card with all the AI keywords and buzzwords and upload that to my talk. I felt it was a little bit pushing the boundary, but you could probably have filled half of that card just with my title alone. going back to the keynote yesterday at the end when you saw all those little llms being little motors fancy motors then stripped to a go-kart we can definitely say that ai is the engine that drives a lot of what is happening in the history today but what we should not forget That is that data is still the fuel, is still the oil that we need to get proper results from whatever you want to get out of AI. And obviously, it's easy. We have AI, we have LLMs, we have the data. But sadly, no. You don't usually want to have static data because then everything is reproducible. it doesn't really give you any insights nothing is new so static data nice but not enough you want dynamic data fresh data if you have been working in the area of ai and since you're here python very familiar there is a lot of new ai tools pretty much every month there is a new toolset, a new version. If you have been at this game for a long time, we were doing something with length chain 0.1, then came 0.2, we had to rewrite the entire codebase. Yay. And whatever your needs are today may not be the needs of tomorrow. So you want to have some flexibility. You want to be able to introduce some changes. So you go back to something that is called loose coupling, separation of concerns, make things a little bit easier on the architecture side, a little bit harder on the implementation side. So this is what I want to talk to you a little bit about because we encountered all this in a practical project. What have we learned from that one? What can I impart from these learnings to you? As we have heard, my name is Holger Nussekabel. Two things about me. First one, I'm almost a decade CTO at TD Reply, which is a boutique consultancy in Berlin. The second one, if you meet anyone with that last name on the world, there's a 90% probability this person is related to me. This slide may look like a plug. It's not. Yes, it is. But we as TD, we are part of the reply network coming out of Italy, and this is a collection of over 200 companies working together as a network. It's not a bad network. They're really nice people. If you have worked in an agency like WPP or something, that's different. So reply is really nice. But this is also the problem statement. They are constantly on the lookout for companies that they try to acquire through mergers and acquisitions, M&A. So they are dedicated teams and now the idea is how can we make AI do the heavy lifting of identifying companies that would fit well into this entire ecosystem. So the task is, find the company, give it a scoring, and do some additional stuff like prepare an outreach message to the people and say, hey, we would be interested in buying you. So this is what we have built. It's a web-based tool. You have what we call the company scorecard, their scores, descriptions, when it was founded, Approximate revenue, number of employees, number of the CEOs, which areas are they working in, are they doing AI, what are their clients, financial, automotive, maybe even defense industry, because that's a whole different ballgame. So everything we can learn about it. And it's not like that this data is lying around on the street and freely available. So how do you get to this? We have to think about that. How do you build the entire system? Let's do something that is essentially my day-to-day work. So if my mom asks me, hey, what are you doing as a CTO the entire day? I say, okay, I create presentation with boxes and lines. So the first one, you need to interact with the system. We have a conversational interface. You can talk to the agent. a web UI, simply enough. What is it connected to? Fast API with an Nginx proxy in front of it. There's an API. Nothing fancy about this one. Who do they talk to? Several agents. So there is one coordinator agent that gets all the requests and then sorts it out to Who is the specialist that can give an answer? And we go for the hub-and-spoke model here because we don't think those agents really need to talk to each other all that much. So we have different agents doing different things. Giving a score, making some basic checks, getting the company information, writing an email, running updates in the background every day, every week. obviously they all use llms that's the foundation of everything they need to have some knowledge base and this is where the dynamic data comes in naturally llms have already a lot of information a lot of knowledge baked in through the training data but that's static that is old we are looking for new companies that pop up in the market in the past three months six nine months a year, that might already be too young to be included in training data. So, knowledge base and it's an SQLite. Why not? We don't need to shoot something like this with an Oracle database or something that scales in read time. We don't need Kafka or any of this nonsense, which is great in other applications, but for us, a simple SQLite, more than sufficient. And for good measure, a graph database, because we could, because we wanted to, we're not actually using it all that much. But it's there. It's nice. It's paid for. So let's fill it with data. We have a Redis. And this is actually something, the first learning, or the first approach that you might take away from this talk. This is a little database that is also connected through fast API, server-side events. And this is used by all the agents to push little status messages in the Redis, which is then propagated to the front end. So we're not exposing the thinking of any of the scoring or the check agent to the user, which would say, oh, this is bad data, you have to fetch data again. No. It's just checking the data, fixing the data. This is propagated to the user. So they don't really see what all the agents are thinking. what we want them to see. And then, where's the data coming from? MCP servers, of course. And here we have different data sources that could be partner websites, technical partner sites, so maybe AWS gold partners, Adobe partners, Oracle partners, you name it. They have a a list of companies. You find a lot of information on LinkedIn that is more or less available. So we only go to the public information, nothing behind the paywall. You will find a lot on the internet. If you scrape a company website, you can get that data. That works quite nicely. And we leaned heavily on perplexity. So the answers there are pretty decent. They have some other problems, we will get to that later, but in the end we used that quite a bit. The benefit was for us that our stakeholders who wanted to have the project said 80% data accuracy is fine, they didn't go for 100%, they would be happy if we just asked ChatGPT how many people work in this company and if ChatGPT hallucinated 150, they would be fine with it. We as engineers had a little bit higher standards, so we tapped into different databases and data sources and thought, okay, let's figure something out that makes sense that is actually a little bit more proven. I have prepared a little repository that you can tap into. We will not open it. We will not look at any code. We will not run any code that is pretty much downsized. You're not getting the keys to the kingdom, but a small subset that will demonstrate pretty much all of what is following on the next slides. In the end, when we look at the different data sources, like I already mentioned, we are tapping into different things, fetching information from the different sources. And And this is a two-step process. We get the basic information first and then extend it with additional information. So there's always the core information like the founding year. We try to get the revenue. We try to get an estimate on the number of employees in which areas they work on. And then we just see, okay, can we get a history of the employees to determine if a company is growing or not? And we get a history of revenue. And this is usually where the chain of data fails. Getting that information is pretty hard. This is also one point where we looked at available paid data before, like Dun & Bradstreet. You can buy those catalogs. That's curated data, which is outdated or, let's say, mature, two years. So every company that you find in there that you would say, I might want to buy this, somebody has already bought one and a half years ago. So that is why we chose to go fully agentic, really pull the data from the internet and try to be as up-to-date as possible. And let me tell you, Dun & Bradstreet is expensive, really expensive. So we hurt their revenue a little bit. A quick overview of what we are storing. Like I already mentioned, we have a few key elements. And then we also look at what services are they providing and at which level. Because obviously, I mean, if you have a company, if you work for a company, you look at the website of your company and all the buzzwords. Everybody will say they will do AI. Obviously, it's on our website. But how far does it go? Do we have like one showcase for one client? Do we even have a client? Spoiler, we have several. But in the end, everybody claims it. How far does it really go? Same with the industries. When you say, okay, you work for automotive, do we have like four, five, six clients to back it up? Or is this one project that you did for one client six years ago that is still on your website? So we use to determine that, give it a score, and feed that also into the scoring algorithm. system. A quick recap, very basic, because now we're talking a little bit more about the technology. Agents. What are agents? Yesterday I saw a very nice presentation, a picture of Inspector Gadget. I was feeling old because I knew who Inspector Gadget was. I was watching it as a kid, several decades ago, not that many decades, but essentially an agent is something controlled by an LLM, so there's a brain, it looks at things, makes a decision whether a goal is achieved or not, and then performs certain actions. So that's a very typical react pattern, this is a goal, I need to do something, I do this, have I reached my goal, no, I will do something else or I will do it again. So it looks at an environment, thinks about what it needs to do, takes actions. Very simply. If we put that into connection, now let's mix it, MCP, model context protocol, and agent to agent communication. So as an agent observing the environment, taking actions, it needs tools, it needs data, it needs a way to interact with the environment, create a file, delete a file, make a reservation, you name it. So the MCP protocol really offers a way to expose tools to agents. It says, hey, here is a tool, here is a screwdriver, here is a way to make a Google search, here Here's a way to make a reservation, here is a way to pull weather data. You name it, it's a tool, the agent understands what the tool does, what it will get from the tool, and when it decides to use it, calls the tool one time, multiple times, and gets the information back. If you have agent-to-agent, that's relatively new-ish protocol, it's like a business card from a client that says, okay, I as an agent, this is my speciality, this is what I do, this is what you can typically ask me, and then the other agent can say, hey, I have a request for you, the other agent works on it, and then responds to that agent. I guess you can already see where this is going when we have the controller agent that says, okay, I need information on a certain company, hey, company information agent, give me this information, give me this data. Then the company information agent says, yes, sir, looks through the internet with the help of MCP, all the different tools, compiles the data, pushes it back to the controlling agent, that one formulates the response and says, hey, dear user, I was so busy, I'm sorry, is the response. Problem solved. Now, getting an MCP server is really easy, because all you need is essentially a fast MCP library. You need a function, put a decorator in front of it, and depending on where you stand on documentation, provide the documentation. What does this tool do? What does this function do? Something you should do anyway. Now you need it because this is what then the agent uses to call that tool. And obviously use AI to build AI. What I would typically do or what we would typically do is write this documentation by hand with our brain and then feed it to Claude, ChatGPT, whoever. Now make it usable for your own kind, because we don't think like LLMs, thank God. Let someone else who thinks like an LLM write for an LLM. So it would just restructure it a little bit, make it more complex, really hard to read, but hey, if it's good for an agent, who am I to argue? So this is essentially all you need, an input, a decorator, MCP server running. that is here is your usual business logic anyway. You call an API, you make a calculation, all that is good. This is then the other part. Once you have this running, either as a network service or via HTTPS or standard IO, doesn't matter, you just give it a URI, pass that to the MCP client in this case that fetches the tools and link that to your agent. Job done. That's all you need to do. Once you have this, you have a fully functioned agent, able to call all the tools, work with them, get the results, easy peasy. With all things that are easy, there's always a caveat. So you might be tempted, and there are a lot of MCP servers on the internet, let's connect as many MCP servers as possible because then my agent will be super powerful. Yeah, he will also be super overwhelmed because you're stuffing all the information about all the tools that he does not need at all into the context. So it's like when you tell a developer to do something, you give that poor person the whole background of the company and the project, you talk to them for 30 minutes, and then, by the way, can you just commit and push the latest changes? So there's a a lot of preamble for nothing. So don't put too many tools in the lap of the poor client. First learning, essentially, whenever you have, I don't know, API code, data source, put MCP on top of it. Doesn't cost you anything. Easy peasy. You can reuse it. You will build other agents later. Trust me. We thought, hey, we did our first agent. We are done. more agents are coming then you can reuse it you already did the hard work so that's great. I already mentioned perplexity trust but verify. When you talk to perplexity the great thing is it gives you information what are my sources? Gives you like little brackets and numbers and then puts at the end a list of the links. Sometimes you don't want that I just want to know is this an outsourcing company yes or no Yes. No. Here are my sources. No, don't give me any sources. I don't want any brackets. I don't want any numbers. Okay. No. Next time. Okay. Yes. Here are my sources. I just told you. Don't give me any sources. It doesn't matter. So you will need to, at least if you work with perplexity, do some response clean up. Remove all the fluff that you don't want Because in the end, you just care for yes or no. Which means, yeah, you will run it through regular expression, look for yes or no, and feed that as a Boolean into your database. Because perplexity just can't be bothered. Which is also true for your agent. Because we have a certain workflow. We say we want company information, which means go get the revenue, Go get the employee numbers. Go get the services that they offer. Go get the clients they work for. So this is true for all the companies all the time. This is what we need to do. Now, the first approach is you put it in a prompt. These are the tools that give you the services, the clients, the revenue, the employees. Call every tool at least once. Get the information. Make sure it's there. You have 150 lines of prompt just to make sure that all the data is there. You have a helper function that counts which tools were used and doesn't let the agent leave the loop until every tool has been called. Boy, that didn't work out. So fail number one, giving the agent too much agency. it just decides I'm done here I have everything I want whoops half of it is missing I don't care so the better approach is to use length graph workflows where you can just specify call every of these tools before the agent is even called so you just stuff everything in the agent context run everything in parallel makes it faster, too, instead of looping, looping, looping through all of that, and give that en bloc to the agent to proceed further. So that was, I think, a rather painful lesson when we said, OK, agents, great. Too much autonomy for them doesn't work out too well. You need some controlling structure. A few examples that we used. So the first one is the loop. So this is the traditional one. The agent makes a decision. I need to call a tool and just keeps on looping until it thinks I have everything that the user needs. So, fine. Time-consuming, takes a while, but, hey, you will get there. If you already know that certain tools need to be called, you can put that in the flow, have them all be called in parallel, merge the data together later. That's the fun part, because you need a function to merge everything together whenever that data comes. Keep it consistent. Or you just create a complete different branch that at some point ends. so those are let's say the three main things and if you look at the demo implementation you will find all of them at some point so learning workflows they make sense they are maybe a bit counterintuitive because you think let the agent go let it happily jump over the fields get everything you want no instead of over-prompting instead of really writing a thousand lines of prompting, just control it. If you know something needs to be done, do it yourself. That doesn't change. Now that we have the agent working with MCP, let's talk about agents talking to agents. One way is to build a repository of agents. So at the beginning, when the agent runs, it collects the agents from all the different URLs, gives me your agent card, builds a nice repository, and then in theory, you would say, oh, I need a specific function, go through the repository, find the card of the agent that does the thing that I need now and call that agent. We shortened that a little bit because, again, through the workflow, we know which agent we need to call. We have the URL of the agent that we need to call. So we don't really need the card all that much. We just pull it because it's needed to initiate the conversation, the protocol. But yeah, we just get the card and then start talking to the agent, just based on the URL. So much simpler, because it's much more controlled. But hey, if you still want to build a repository of agent cards, you can do so. You can look through the agent cards, look for a specific keyword. Works as well, just uses a couple of more lines of code. So yeah, that's pretty much what we did, much, much easier. Fun fact to watch out for, the response of your agent, especially if you collect multiple agents over time, might be different. It might be a structured JSON. It might be a plain text answer. It might be a Boolean yes or no. And if you, at some point, try to work with that response in some shape or form, you just need to be aware that sometimes you're dealing with a plain text. Sometimes you need to pass it through JSON. You need to detect what is the kind of response. It's not one size fit all. You can either do it based immediately on the response, or the agent state is just a place where stuff goes. So put it in there. Let someone else deal with it later. Not your problem. When we were talking about the web server who is receiving all the input from the users, eventually it will talk to the agent. So why not make the web server an agent client and just call up the coordinator agent with a query that is coming from the user. Much easier. Actually, what a colleague pointed out, because this is what I did, I was feeding all the endpoints with fast API and had my definitions in a simple list with this is the endpoint, this is the protocol, this is the function to call. and just iterate over that list and create the endpoints programmatically. I found that much easier than, I don't know, passing the fast API object to all the different files that might be in subfolders and use the decorator. So for us, this was one clean way of defining everything in one place, easily extendable, easily understandable, and nothing gets lost somewhere because it's in a folder. One small tidbit that maybe you can apply. Like I said, if all your agents already use A2A protocol and your backend talks to an agent, make your backend an A2A client. The protocol itself is currently maybe a little bit over-engineered, a bit too complex because it offers a lot of options and for the people that need those options, hey, I'm happy for you. For someone like me who says, I just want to call it, I just want a response, maybe make it streaming, that would be nice. That's enough. I don't need push updates or anything like that. It's sometimes a little bit too much. But there's another learning coming up. In the end, what you also want to make sure is that your agent, your system is not bullshitting you. So you want to run some QA and that QA can have different levels. The first learning is actually, nobody likes to write tests. Let AI do it. AI knows your source code anyway. So, hey, write me some tests, cover the edge basis, be creative, try to break the system, get me a nice code coverage, Claude will do it. What you still need humans to do is evaluate if the answer makes sense, if the response is actually what you were asking for, if this is what you need as a user. There's no way around this. So that is still an evaluation that you need to do as a human. You also want to automate it. We heard about that yesterday quite a lot. Testing is very, very important. And we also introduced the check agent, which occupies the next slide, to verify that whatever is coming in, whatever is going out, makes sense. And then you have the evaluation metrics. We had to go through the entire AI ethical certification for that system. That is a 12-page document that we created with the help of AI, 10 point. So that was quite a lot of information they wanted to have. We said, OK, some of this doesn't even apply, but hey, you want metrics? We will give you metrics. And with AI, that really worked quite well. As I mentioned, the check agent, that is something sitting in the middle that makes sure that the user is not sending any malicious prompts, that whatever we return to the user makes sense and actually answers the question, and also validates the data that we get. So does a company report 1 billion revenue with 10 people? That is a little bit of a stretch. If not an outlier, somebody needs to redo the numbers. Obviously, since we serve ourselves from the Internet, if there is a company who successfully manages to promote itself of making one billion revenue across all the channels, that's what we will get. Eventually, you will end up what the Internet tells you. Again, we got the guideline, 80% good enough. One point where we said AI can help with the set-up of tests, but AI can also help you with debugging. If you recall, at the beginning, I was saying there's a lot of change in the tooling landscape, the protocols, and we, I found that out the hard way, because when I was connecting the back end to the controller agent, things were not breaking, there was no response coming, and either I could spend a day reading documentation, or I just put AI, say, here, this is my messages, this is what I see on the console, this is the log output, fix it. And it fixed it in five minutes. Instead of me reading one-day documentation. I prefer to let that happen. So AI, helping building AI, actually a thing that works quite well. All right, moving to the summary already, recapping what you have heard. So good data means good answers from your agent. Stale data means boring answers from your agent. So feed them good data. if you run and build an agent because someone told you that is already kind of setting you up to fail there should always be a good valid business question a business reason to build something to spend money on this one and one of the mantras that i really grew up with professionally is really technology is an enabler, not a driver. So we don't build AI agents or AI systems because AI is a hot topic, but because there's a benefit for the clients. Otherwise, it will just contribute to the hype, and then people will say, oh, this doesn't really make sense because what is the purpose of this AI system? MCP, A2A, really helpful to keep things scalable, flexible, you can swap in new agents quite easily, you can add new data sources quite easily, just put on an MCP server, connect it to the clients, change the configuration file, link it up with A2A and you're done. So your ecosystem can really grow. I already mentioned it several times, AI came to my rescue many times, made the development process much, much easier and much, much more enjoyable for me. And, yeah, don't forget to also cover test cases, testing with AI, making sure the agent is not only working, but also responding properly. Because a working system, that's easy, but detecting when it starts to sprout, nonsense, that's pretty hard. And monitoring, tracing helps a lot, because many times we were called by the client, the answer is not good. We were looking at the traces, and we saw that someone was just entering one word. I can say, okay, they were entering like Air France. What is the poor agent supposed to do with that? You want the stock price, you want what people think about it, you want a competitor comparison, so what do you want with that? That doesn't help. So tracing, really tracking what the people are then asking the agent in real life and tweaking your implementation accordingly, absolutely valuable. All right, for that I leave the space, I wish we could open a window, but we have to do with questions instead.

Speaker 1 [36:27]

thank you Olga once again for a great presentation and we have a lot of questions so let's first start with the first one how do you judge how many MCP tools per agent are too many

Speaker 2 [36:48]

That is a good question. There is no hard and fast rule, obviously. But I would think gut feeling more than 10 is already suspicious because your agent should also be very specialized. You don't want the jack-of-all-trades agents with 20 tools, 50 tools. You would rather have like two, three agents with, I would say, 10 tools or less. It depends whether those two are warranted, but I would say 10 above pushing it.

Speaker 1 [37:27]

So keeping the agents focused on sound.

Speaker 2 [37:30]

The more focus you can get, the better it is, in my opinion, yes.

Speaker 1 [37:37]

The next one is, are you worried that companies will soon figure out how to poison the information into your agents so they think that the company is better than it is actually?

Speaker 2 [37:52]

actually is counter question because that is a really good one what stops them from doing it already now because what are the people researching those companies looking at spoiler it's a google search it's maybe a catalog or some i don't know website that they find and if a company is claiming to have a revenue that is widely exaggerated it will appear somewhere and it will be found by the humans. Now it will just be found faster by the agents and it can be cross-checked by another agent which may or may not also happen with a smart human who has been doing this for a long time may not happen with a human who has been doing this for 8 hours a day and at the end of the day when he or she wants to go home they may not look so closely or they may just see the comma is slightly shifted so yeah in the end if your end goal is to be the unicorn to be bored you will make yourself larger than you actually are so that is a real problem but it's not a specific AI problem hopefully AI can help to solve it

Speaker 1 [39:13]

That's the good answer. Okay, the next one is about the skills. So what about the skills instead of MCPs at least as a replacement for some of the MCPs that you named?

Speaker 2 [39:31]

Yes and no. So skills were relatively new and introduced when we were pretty much finished with the project, so we couldn't really build them in. Looking at the skills at the moment, they look more like cooking recipes, kind of, instructions how to do something. The MCP servers are really about fetching data. And that is something, yes, you can have a little bit more lightweight as a skill, which means like you go down a tree, you say, okay, I need certain information. I need to find the CEO of a company. There will be a skill description. Use this MCP tool to get this. And that is a, let's say, lightweight option to do this. Yes, definitely worth looking into. At the moment, again, we offloaded a lot of the interaction with the tools anyway to the workflow. and we just run the Google search, the LinkedIn or the PackCity searches and merge it together. But yes, moving forward, that is definitely something to consider as a more efficient way and not poisoning the entire context with MCP instructions.

Speaker 1 [40:47]

Good the next one. What are you doing to keep that token costs low? Are you using some local hosted LLMs or some and this is good one because you mentioned perplexity And I know that they are Sometimes very expensive

Speaker 2 [41:05]

They were actually the cheaper variant of this one, so perplexity was surprisingly cost-efficient. Tokens, not that much of a problem, because we were requested to use the corporate LLM access anyway, so we didn't have to pay for this. But cost is indeed a factor because we would collect information on thousands of companies. And we calculated that approximately we would have to pay 50 cents per company. So that's pretty expensive. It was higher a couple of years ago. We calculated it would be like $2. the benefit is that 30, 40 cents of that price goes to the provider that we use for LinkedIn data which is quite useful but there are other ways to get it so if we just cut LinkedIn data cut a little bit of the fluff we still get 70% of the data we want and then it's only 10, 20 cents per company which is a lot but it's just building the catalog and then you search through the catalog and it saves time and money because another person doesn't have to do the research and their hourly rates are much higher

Speaker 1 [42:36]

It is absolutely incomparable with just a regular employee doing this work.

Speaker 2 [42:43]

In a way, yes. It's the same calculation that we would do with, I don't know, cloud code licenses. So how much time do we save with this versus actual time? How much does that translate into money? We can do that. What we also see is that, of course, people are doing extra work that they wouldn't normally do because now they have a little bit of capacity and they just let cloud run. So, the same here, you might be very focused on your search for a specific company, but if an agent can do it, at little or no additional cost to you, you just might run five different searches, slightly varied, because it doesn't cost you anything. You just put it in once, wait for the results, go through it, and have your options.

Speaker 1 [43:35]

Great. The next one.

Speaker 2 [43:38]

Then why don't you just use structure?

Speaker 1 [43:39]

use structured outputs instead of regex for perplexity, question answering.

Speaker 2 [43:47]

if it were that simple. Yes, you can tell perplexity to use structured output. Yes, sometimes perplexity adheres to those instructions. Not 100% of the time. So, yes, we tried it. It didn't work out in all of the cases. So, in the end, using a simple regex was sadly the simplest fastest solution with another llm yes absolutely just give the structured output we use that for the company information agent you will get your json nicely that's that's fine but in some cases just not possible or the llm or the service just doesn't want to adhere to it for some for some reason.

Speaker 1 [44:35]

Okay, good. The next one is did you consider using tool calling instead of It is yeah agent to agent protocol where the agent is simple a tool and you connect it via the HTML if your agents are on the same network, of course

Speaker 2 [44:54]

We opted for a bit more flexibility. So usually...