Schema-Driven Lambdaliths in Python with AWS Lambda Powertools and Pydantic
Large-scale serverless applications often suffer from operational overhead when split into numerous small functions. In high-frequency data analysis environments, such as air conditioning operational monitoring, this fragmentation leads to excessive cold starts, duplicated dependencies, and increased bundle sizes. These issues can result in API Gateway timeouts and slower overall system performance when multiple functions must be invoked sequentially.
The Lambdalith approach addresses these problems by consolidating multiple endpoints and business logic into a single AWS Lambda function. Initial implementation utilized FastAPI combined with the Lambda Web Adapter to handle routing and validation. Performance benchmarks showed that while a single request may be slower due to a larger initial bundle, sequential requests are significantly faster. For a sequence of six endpoints, the Lambdalith approach reduced latency from six seconds to three seconds by eliminating repeated cold starts. Additionally, deployment times were reduced from 115 seconds to 62 milliseconds for initial deployments using AWS CDK.
To ensure operational sustainability, AWS Lambda Powertools for Python was integrated to provide unified logging, tracing, and metrics via decorators. The library also simplified parameter retrieval from AWS SSM and Secrets Manager using TTL-based caching and enabled the function to handle non-HTTP events, such as those from EventBridge or SQS, natively.
The architecture achieves end-to-end type safety by using Pydantic models to define schemas, which are then exported as OpenAPI specifications. Tools like OpenApi-TypeScript (OBL) generate TypeScript client code and type definitions for the frontend. Furthermore, SchemaSynthesis is used for property-based integration testing, automatically generating test cases from the schema to verify API correctness and edge-case compliance within the CI/CD pipeline.
This description was generated by Open-Source AI using the transcript of the session and the original submission contents.
This session took place in track Django & Web and was classified suitable for intermediate domain / intermediate python by the speaker.
Submission
The proposal as submitted by the speaker before the conference.
The rise of modern web frameworks such as Hono has brought increased attention to schema-driven development and the “Lambdalith” architecture, where an application is delivered through a single Lambda function. These approaches offer a highly streamlined developer experience, but many existing Python-based systems struggle to achieve the same level of consistency, validation, and maintainability.
In Python, closing this gap often means introducing additional execution layers outside the language itself. When frameworks designed around web servers and request lifecycles are deployed on AWS Lambda, they typically require ASGI adapters, web adapters, or container-based runtimes. While powerful, these layers can make it harder to focus on what many teams actually want: writing clear, minimal Python handlers with explicit data boundaries.
This talk explores how combining AWS Lambda Powertools and Pydantic can close that gap and enable a modern, predictable development workflow—even in established Python ecosystems. Drawing from real-world product use cases, we will examine how these tools can simplify handler-level logic, standardize request and response validation, and improve observability and error handling.
Lambda Powertools provides far more than logging and metrics: it includes utilities for structured tracing, data parsing, idempotency, typed configuration, and other features that bring Python serverless development closer to the ergonomics of newer frameworks. When paired with Pydantic, developers can enforce clear data contracts, reduce boilerplate, and achieve stronger guarantees around application behavior.
Attendees will learn practical patterns for improving quality and productivity in Lambda-based applications, including how to:
- Validate event payloads and responses using Pydantic models
- Implement consistent error handling strategies
- Structure a Lambdalith-style architecture in Python
- Leverage Powertools utilities to enhance reliability and developer experience
This session will be valuable for Python developers who want to apply schema-driven design principles, modernize existing serverless codebases, or build more maintainable Lambda applications with confidence.
Transcript (auto)
Auto-generated from the recording utilizing Open-Source AI. Speaker labels (Speaker 1, Speaker 2) reflect diarization, not identity. Timestamps refer to the recording.
Speaker 1 [00:02]
Right. Yeah. So good morning, everyone. The last day of the conference. So enjoy the last talks as much as you can. And welcome to this session. This session will be 30 minutes long with five minutes of questions and answers session. At the end, you can ask your questions talks.pycon.de Yes, and the session is Schema-Driven Lambda Lease in Python with AWS Lambda Power Tools in Pydantic and presented by Tanya Otoranasuke and Haruto Mori. Yes, they did all the way from Japan, Osaka, Daikin Industries. So, show all your support and give a round of applause.
Speaker 2 [00:58]
This wear is a dress code at a manufacturing factory in Japan. But we are not device teams, so this is the first time to wear this. Did you hear today's keynote? I'm worried I have to talk about AWS. Okay, let's start. Today, Mori and I will present on schema-driven Lambda in Python with AWS Lambda Power Tools and PyIdentic. First, I'd like to briefly introduce myself. My name is Toranosuke Tanyo. I came here from Japan, and it took more than 10 hours to get here for this talk. I have a master's degree in information science, and this is my sixth year at Daikin. From my first year, I joined our in-house web application team as a developer, and from my second year, I also took on the Scrum Master role. Since last year, I have also been working with other teams as an Agile coach, so my career has expanded mainly in the Agile space. I also have organized an internal engineering community of around 500 people, so I spend a lot of time enabling activity as well. At this point, I would like to ask a question. I know this is PyCon, so it may be a risky question, but do you like TypeScript? I don't answer. The reason I ask is that full stack development with TypeScript is getting a lot of attention right now. On the frontend side, framework-like Reactor and Vue are still very popular. On the backend side, a framework called Hono has recently become a hot topic. Hono is a lightweight and fast web framework born in Japan. It supports multiple runtime, follows web standards, and is written entirely in TypeScript. With Hono RPC, you can also share API contracts directly between the server and the client. And for infrastructure, there are many tools like AWS CDK that can also be written in TypeScript. So today, almost everything needed for development can be covered by TypeScript. With the rise of HONOR, the idea of the Lambda list has also started getting attention again. One appearing point of HONOR on Lambda is that you can run the app on Lambda by adding just two lines. Once you write in this way, a single Lambda function can handle multiple events. For example, not only get books, but also post requests or other endpoints can all be rooted inside the same Lambda. In other words, one Lambda can handle many kind of requests through routing. Because of this simplicity, the LambdaList style has become interesting again. But today, this is PyCon, so let's stop the TypeScript discussion here for now. Do you love Python? Oh, of course I love. The goal of this session is mainly twofold. First, can we get the same benefit of a LambdaList in Python that people are now talking about in the TypeScript world. Second, schema-based web application can connect the server and the client end-to-end in TypeScript. But what happens when the backend in Python? Through this talk, we would like to revisit the value of the kind of web application architecture. Before moving into the main topic, let me also briefly introduce our company. Since we came all the way from Japan, I thought it would be helpful to first share a little background about who we are. We belong to Daikin Industries, a company based in Osaka, Japan. Some of you may already know the company. One major feature of Daikin is that we handle everything in-house from refrigerant development to manufacturing, sales, and after-sales service. In that sense, we are a very unique air conditioning company. Our business includes air conditioning as a core business for our chemical products that support refrigerants and filter-related business that improve air conditioning efficiency. Basically, almost everything in the company is centered around air conditioning. Let me also say a few words about the scale of the company. In Europe, Daikin may not feel as familiar as it does in some other regions, but the business is very global. About 80% of our sales come from overseas markets. We operate in more than 170 countries, and we have more than 130 manufacturing sites. With strengths in areas like humidity control, humidification, and ventilation, we continue to lead the air conditioning business globally. Within that kind of company, we are also working on agile development and in-house product development. In Japanese companies, people often still have the image that waterhole and outsourcing are the norm. And in reality, enhanced development and agile practice have only started receiving serious attention relatively recently. In that environment, our team was launched about seven or eight years ago, and we prompt agile and enhanced development from the bottom up in order to speed up hypothesis testing. A group of relatively inexperienced members came together, and we kept growing through continuous improvement. As a result, we have also spoken several times at major conferences in Japan, for example including JustTokyo and Region Scrum Gathering Tokyo. To explain the context of this talk, I'd also like to introduce the kind of service for which we develop Python applications. In Japan, we provide a service called DK Connect. It is a cloud-based air conditioning control service that enables centralized management of building equipment, such as air conditioning and lightning. It helps buildings owner, tenants, and facility manager operate and manage equipment more easily. On top of that, we create new value from the operational data stored in the clouds. For example, we visualize power consumption, reduce peak power to improve energy savings and cost savings, and apply AI-based remote control. In this way, we aim to keep buildings comfortable, let customers use air conditioning longer and more efficiently, and reduce environment impact at the same time. Within the large service, our business domain is an optional service in DK Connect called NFocusAlpha. As I mentioned, operating data is accumulated in the cloud every minute. By analyzing that data, we provide proposal for better air conditioning operation and ongoing support for customers. For example, we help suppress excessive operation, prevent air conditioning from being left on, provide on-call support from service staff, and create reports that show the effect of energy savings. Our team develops and operates in-house the back office application that supports this work, as well as the parts used to make proposals and reports for customers. Now, let me gradually move into the main topic. This initiative started around 2019. At first, it was a proof of concept within R&D. But once we saw that it could work, it become a real system, and the service gradually grew. As that happened, the software also kept on growing. We increased the number of Lambda functions and split them by responsibility, such as data fetching, aggregation, analysis, and thread generation. In the early stage, responsibilities were clear, and everything was easy to control. But as the overall application became larger, operational pain started to appear. One of the biggest issues that became visible was cold start and bundle size. Because each Lambda 100 shared library separately, Duplicated dependencies made the development architecture larger and larger. And as the number of functions increases, the cost of initial startup also becomes impossible to ignore. In our use case, we analyze air conditioning operational data every minute, and we do that for each indoor unit. In large properties, there can be around 900 indoor units, And that data has to be 124 hours a day, 365 days a year. Because of that, the startup time for each function became a serious burden. In some cases, we even hit API gateway timeouts. So in our use case, splitting lambda functions to finally actually starting to hold us back. Every time processing closed a Lambda boundary, we paid both start-up cost and hand-off cost. In air conditioning big data analysis, the workload is highly sequential, and there are many shared dependencies.
Speaker 3 [11:33]
Hi, I'm Haruto Mori, and I'll take over from here. My background is Master's in Applied Physics, and I'm in fourth year as a software engineer in Daikin. This is my first conference talkable, including in Japan, so I'm very nervous, but I will try my best. So, let's get back to the main topic. The approach we talked about is called Lambda Leaps. It is a portmanteau of Lambda and Monolith. It is the architecture that 1Lambda handles all loading and processing logic, API gateway and application logic, leaving a single Lambda. You can load dependencies once and share them across all endpoints, so you can minimize the cold start. And, you can separate domains per bounded context. Domain isolation as an infra-label. It is a natural fit for a domain-driven design and reduces cognitive loads for developers. Our first approach was FastAPI and Lambda-Wave Adapter. FastAPI unifies routing, validation, and response. Many of you probably already know. Lambda Web Adapter runs a web app on Lambda. It converts FastAPI's HTTP requests and AWS Lambda events and context to each other. The big benefit of this approach is a familiar web framework developer experience. It is easy to prototype and validate for us. For the initial implementation, just add this line to your Dockerfile. Lambda web adapter handles execution on Lambda, while routing and validation are delegated to the first API. That means you can develop and debug locally, like any other regular web backend, without changing a single line in your backend Python file. Okay. That is the basics of LambdaList. Now let's look at what LambdaList delivered performance. Let's compare a split Lambda approach with LambdaRis. Here's the setup, 45 endpoints and the shared business logic and same container image. Split Lambda uses 45 functions with API Gateway loading all the requests. LambdaRis uses the setup I have just described. results are shown below are the average of over 10 lands. Now, I know this is a pygon and a pyretor. So, we have any data scientists here? Please raise your hand. Oh. In this cite these words, n equals 10, and no confidence intervals, and non p-values. So please be gentle with me. So let's look. First, looking at the single request. Split lambda wins one enough code cert. Split Lambda has taken about 780 milliseconds, and LambdaList has taken 2,500 milliseconds, about three times slower. This is because LambdaList has to load the interior code base and shared or dApps at first, first API, adapter, and others shared dApps in one shot. So, for single invocations, the larger bundle does take longer to initialize. That's expected. However, production is one request. In real systems, request doesn't come one at a time. So we hit six endpoints in sequence, and the result flipped. Split lambda has taken about six milliseconds, and lambda least has taken about three seconds, about two times faster. Split calls out all six functions one by one, and lambda least cold start once, and the other five are already warm. Even with small functions, when you trigger a lot of cold start in production, the overhead adds up fast. In our use case, first visit experience imports about two times. So what about warm latency? Nearly identical. Split lambda, 88 milliseconds, and lambda is 96 milliseconds. The loading overhead was lighter than expected. The cost of better DX is just 8 milliseconds. Deploy time tells the same story. Measured with AWS CDK for TypeScript, the initial deploy took about about 115 seconds, and the LambdaRisk took about 62 milliseconds. On incremental deploys, the gap is not allowed, but LambdaRisk is consistently faster. This is because LambdaRisk is a single resource, so that IAC tool spends less time to detect the difference and make Take REST API calls to create a resource. The initial deployment matters most, and the bottom line is that LambdaList also speeds up your CI, CD, and validation cycle. So let me summarize why LambdaList worked for us. In our use case, optimizing globally beat optimizing locally. Lambda won one of cold start, but eliminating repeated cold start and duplicate loads won overall. And shared library management is simplified, and we got less cognitive load for developers and more readable code base. We built our system with this architecture and learn it in production. And what we found was there were still other challenges to deal with. We weren't done yet. Beyond performance, daily operations demanded more than speed. First, we needed unified logging, tracing, and metrics. Observability logic was scattered across middleware and handwares, so it was hard to keep consistent. Second, shared parameter fetching and caching logic. Parameter fetching from SSM or Secret Manager or DynamoDB lookup logic, duplicated everywhere, and it is also hard to maintain. So, we wanted to keep the Lambda-less benefit and evolve it from this architecture into an operationally sustainable architecture. That's where Lambda Power Tools for Python came in. This is an AWS-native utility library. Here's what we solved as a need. First, loading syntax is similar to Flask or FastAPI. Second, a consistent API for logger, tracer, and metrics. Third, built-in parameter retriever and casting utility. And fourth, it can handle non-HTTP event sources in the same function. I will walk through each point in the next few slides. First, loading. As you can see in this code, it's almost identical to Flask and FastAPI syntax that made migrating our code base easily with low cognitive load. Second, all that middleware logic now is green. As you can see, just add a decorator for logger and tracer. That's it. It can be added on the root label or each handler label. No more NC middleware logic in your code. Overall readability improved well. Next, building parameter retrieval and casting logic utility. Fetching logic from parameter store or secret manager became simple. You can fetch in one line with TTL-based casting. All the custom retrieval and casting logic scattered across the code goes into one clean line. And finally, event handling beyond HTTP. FastAPI and Lambda Web Adapters can only process the HTTP request. So if you need to process the non-HTTP request from, for example, EventBridge or SQS, you need to build a separate Lambda just to convert them into HTTP requests. PowerTools, on the other hand, works directly with event and context. One function handles both HTTP and AWS events natively. For example, we could easily add scheduled events in production. And what about performance? Did PowerTools throw us down? Not at all. In fact, it was sometimes faster. Without the LSG layer in the middle, there's one less abstraction, which means less overhead. So, we kept all the Lambda-less benefits while evolving into more AWS-native, production-ready architecture. Alright, we have covered the Lambda-less story. Now, back to the question Tanio started with. Remember what Hanoi RPC archived? Server types propagate directly to client. How do we get this in Python? Pylantic and OpenAPI and CodeChain get us there. Let me walk through the schema-driven big picture. Pylantic model to OpenAPI spec to the client. First, you define the schema with pylantic-based model. And next, you use power tools for fast API to auto-generate the open API spec. For example, in Lambda power tools, it can be, it takes just one line. Then, you use a code generation tool like Ober to produce front-end type definitions and API client code from that open API spec. And the result is datatypes and fetch functions, or React query folks, all are generated. Your frontend can call APIs with full type safety with no manual coding. Beyond the frontend, you can also run schema-based integration tests using SchemaSynthesis. It is pure Python OSS, a property-based testing tool for web APIs. With a single command, you can verify API correctness without writing a single test. Drop it in your CI-CD cycles, and you can verify schema compliance on every push. Schema Stasis also auto-generates test case from schema definitions. Schema example test, boundary value test, schema-based random tests, and negative tests. It exclusively tests edge cases no human would think of. Let me wrap up this session. First, Lambda portals and Lambda lists. Cold start reduction brought measurable performance gains. And all-in-one integration with AWS services beyond Lambda brought simpler code and less cognitive loads. Those were the major wins. Second, schema-based front-end integration and testing. Using tools like OBL, you can all generate TypeScript schemas and API client code directly from your back-end code base. And with schema-services, you can all generate test cases from the same schema, giving you integration level API quality assurance. Together, Power Tools and Pylantic gave us, with full coverage, a comprehensive end-to-end development workflow. Every product has different contexts, traffic patterns, team sites, and deployment needs. What we share today is one approach that worked well for our use case. We hope it gives you ideas for your project. That's everything for us. Thank you for listening.
Speaker 1 [26:46]
Thank you very much, Tanya and Haruto. So we have several questions. I will start with the most voted ones. So how do you handle the personal authentication, for example, AWS Cognito to special endpoints behind fast API if you use one Lambda organization? Because sometimes it is necessary that not all users have access to all available endpoints.
Speaker 3 [27:20]
That's a good question. So the best practice is split lambda races into the bounded context the user needs. You can split one lambda race of group A people in and the The other lambda list, the group 2 people are in, and so on. Spreading for bounded contexts is a great practice, I think.
Speaker 1 [28:12]
Thank you. So the next question is, in my experience working with Lambda functions, you easily tend to split your code base into too many different functions because it's so easy to create one. Your approach, on the other hand, combines a huge part into one service. Do you see risks in terms of separation of concerns principle? And what's the optimal size for one Lambda in your opinion?
Speaker 3 [28:42]
As I mentioned in the last question, it is a good sign for splitting lambda into bounded context. And the proper size for lambda depends on the context. It is not important of size, But the bounded context is more important, I think.
Speaker 1 [29:34]
Right, thank you. Yeah, how long did it take you to convert the existing lambda? Okay, it disappeared.
Speaker 3 [29:47]
Is there
Speaker 1 [29:48]
Is there any plans to try this over other cloud providers, for example, OpenStack, Azure?
Speaker 3 [29:55]
Mm-hmm.
Speaker 1 [29:56]
Is there any plans for that?
Speaker 3 [29:58]
As we can see.
Speaker 1 [29:59]
Is there any plants?
Speaker 3 [30:02]
Oh
Speaker 1 [30:03]
No.
Speaker 3 [30:06]
I have never planned, but I don't know about utility tools like Lambda power tools for Python or in service in other cloud service, in Azure functions or GCP functions and so So, I never think to do this.
Speaker 1 [30:40]
How long did it take you to convert the existing Lambda to the Lambda-leaf methods? How big was the team doing this?
Speaker 3 [30:50]
So, just about within one week, and so, the thing, it is a great point of identical, the big point is the identical syntax, and we could, less changes to our code base. And it was easier than expected.
Speaker 1 [31:21]
Right. There are more five questions, but we are already one minute late. So, yeah, please, again, a round of applause to our speakers.
