Building Hexagonal Python Services

Speaker 1 [00:03]

Thank you very much for coming and today we are going to talk about about and it's share slides so the um the strategy i'm going to follow is to explain a bit of theory and just the talking about why we need architecture and things beforehand and then we are going to have some kind of real world application you can follow alongside with me and type a code but it's not a requirement i'm going to make it a smoother for you you can just watch and then ask the questions in the middle as well um yeah that's the case we are so this is the first time of mine to be in a point conference i'm quite excited and again thank you for coming uh hexagonal python services and how to build them. Today's agenda is about me and our Python user group, and then it's going to a bit of historical reasoning, how it started about me, and a better way to teach complex topics. So we are constantly searching to teach people with complex topics, and there's a kind of still have a better way, a better room for that kind of thing. And the few words about maintainable software, a few words about the architectures, and then we get started with the coding. Azobej and Python user community, it's founded in 2015. We are active in Telegram and Discord. The predominant communication language is Azobejani, and we do live streams, helping each other, answering the questions, and doing local and online meetups, obviously. This is my career waves. It's okay, it cannot be interesting enough, but why I call this the waves, because there are a lot of waving in our life, and my career just mirrored my life decisions, this kind of waves and curves. I started in Linux system admins and a bunch of things, dropping master degrees and going through the software engineering. And four months ago I relocated to Berlin as a senior software engineer and I work for the North Security as a senior software engineer in the threat intelligence team. This is my ways. And it all started from this wonderful book, Standard C, in 2009. And rest in peace, Dennis Ritchie, and thanks, Brian Kent, again, to give me a chance to change my life, actually. And first thing I built was a simplest of a calculator. And I cannot share this kind of exciting that, oh, man, there's a code. It's compiled successfully. This is a way that it gets an input. It adds two numbers. And wow, that was magic for me the first time. And keeping in mind that I have graduated in finance, this was a totally new thing for me. So I didn't see any kind of code before. And then after a decade, I found this book, it's Architecture Patterns with Python. And thanks again, Harry and Bob, for this wonderful book. And the samples, the simple code and applications I built in the book, it's somehow complicated for newbies. And I just remember how I started the first time, and it was a building and a calculator. And the question is that why not to build the same thing, but using enterprise architecture patterns? So just start with the simple, then go to the complex topics. And today we are going to build this kind of application, but we are going to over-engineer things. Okay, the question can be, what to do with an architecture, patterns and other stuff? So the main topic is, what is a software maintenance? So you can develop the application or the product for two years, and if obviously somebody is going to use your product, you are going to still wait for, in a maintenance state, 15, 20 years until you are going, I don't know, bankruptcy or somebody acquires you and pays more money. This is the most crucial thing to remember about the software. And there's a distribution of maintenance effort distribution for the software. And as you see, the 67% of the effort is going to maintain the software. And the people with a brilliant mind came to the solutions, or what kind of maintenance types we have, and basically it is an endless loop. So we monitor, we enhance the software, we support the software, and then we bugger fix this, bugger fix this, fix the bugs, and then it's going to continue with the loop. So what does it mean for enhancements? The people in the business can add a few features. They can add a support effort for your customers, or you have a bunch of bugs and you need to fix it. The next question is, what is a maintainable software and how to ease this process? The maintainable software is a combination of testable, extensible, and understandable software. But you can add other elitists, like observable software and other stuff also. So what is a testable? If the software is easy to test, this is a testable, easy to change, modify, extend, that means it is extensible. And low cognitive load, when you're onboard somebody new in your team, in your project, if it's a higher load of the cognitive dissonance, That means that you have a higher cognitive complexity, and it's not understandable at all. And if you combine all three, you got a maintainable software. And of course, this is not easy in general. And the people came around with the ideas how to simplify things. And they invented this onion-clean and hexagonal architectures. The hexagonal architecture is a different name as ports and adapters. You can search for it as well. And the question is, what is common for all these architectures? Yeah, I'm asking you. Yeah? The direction of the dependencies. Correct. The first commonality is direction of coupling or direction of dependencies. It goes from outside to inwards versus from inwards to outside. And the second commonality, what you see here, what's a common. The old architecture centered the domain model. So this is the heart of the complexity of the software. This is where the business logic or the problem you are trying to fix sits there. And that's why the center admits that this is the most crucial part of your application. But as you see, there's other commonalities. The user interface, tests, frameworks, UI, database, external systems, I don't know, web frameworks sit outside of the architecture. That means there are details. We can see the database and the frameworks as a detail, and it will be addressed when we really need it. So this is a crucial part about architectures. Just focus on the domain model, or just focus on the business domain. what your software is going to fix, what kind of solution you are going to invent. Yeah, the next is we have this app requirement, the wonderful app requirement. So our calculator should do. It should have fast API support to add, subtract, multiply, and divide numbers. It should store the result of actions in the database. The calculator operates only with two numbers because it was easy to write. Add, subtract, multiply, and divide are supported, only supported actions. You have this kind of draft of the requirements. And the first thing that most of us is going to have, how do you want to start to build this application? And usually, when you follow all the tutorials, they are directly starting from the database modeling, or they are directly starting from installing a framework, building an API and going through all this hell at the first hand. But this is where it's starting to go wrong. Let's just revert the ideas how we should act. What we do we usually forget about, we forget about the business domain logic or this centralised section of the architecture, the core of the software. And how it should look like instead, it should look like this one. So we first start from the domain models, then go upwards to the layers. No framework yet, no database yet, and this is a crucial part of architecture and the things. Why I have two app requirements? OK, this is, sorry. We have an application called this hexagonal architecture. We can right now install it and use it. And also, there's a guide for the running test, for running actual fast API and flash things. But instead, I would like to go directly with the code and to start the code things. To get started, what do we need? We need to create a fault jar. And I'm going to just copy it from my slides. I have actual .. The size is small, right? I need to increase. OK, is it OK? Yeah. Yeah, good. And yeah, just create some kind of hexagonal calculator and go through the folder. Then what we need, we need to get in it as an empty repo. The second is we need to register as a remote origin. Here, the way that we can do it, and we can verify remote, not remove, remote v. And we have added this remote repository as well. The second, the next step is fetching the remote. JIT fetch the remote. It's going to fetch all the commit history as well. And for understanding the things that you can grab all commit hashes from remote region And the reverse order, that means the first commit is showing that it's the initial commit. And here is the output from the Git log. And this is an initial commit. This is an initial commit of full just structures and other stuff. And we are going to use a different thing here. I'm going to cherry pick through the commit hashes. This is a new way that I'm trying to show you. And then we can just fire up our PyCharm. I use PyCharm. You can use the fencing as well. But PyCharm has this neat git extension. That's pretty useful. No, I don't want. And I need to remove this. Excuse me? Yeah? where we can find the slides for that i have shared in the discord room channel yes i have shared the slides and report like there as well font font bigger okay sure sure sure uh i need to find how it was yeah it's control alt s i believe and editor formed yeah like in the first register zoom where just the first one you open okay there's a ctrl s uh appearance and appearance okay oh yeah oh nice thank you very much is it okay you want but okay you went you want more Sure, sure, sure, that's better, much better, thank you very much for helping me. And in order to get works, just we need the dummy first commit, so I'm going to create some kind of PyCon and PyCon TXT and if we git checkout be main and then git status we have this PyCon TXT and ID I'm going to git add PyCon TXT and then git commit this is a initial commit and then we open we can open this kit and if we go go to the main of the remote branch we can see all this commit history and what we can have here the initial commit and we can cherry-pick this change if If you hit inside, and this is a nice cherry icon, you can cherry-pick, and that's it. So we have this first initial commit from our remote branch. This is just license, git ignore, and readme. Nothing fancy. The second thing that we can cherry-pick is the Folger structure. And let's go through the Folger structure. Our hexagonal calculator has source and tests. inside the source we have domain inside the domain model ports and adapters and that's why it's called the hexagonal architectures other name of this architecture supports and inductors and this is a way that I'm going to follow the tutorial cherry-pick the commits and explain is it okay for everybody I guess system home yeah amazing lighter is it lighter no hi yeah it's the same for me So I don't know. Maybe material sandy. How about now? No? Material sky blue. It's the first time I'm checking those kind of things. Is it OK now? Good. Nice feedback from the people. So we have constructed our Folger structure. The next is, as we have talked about domain modeling, decentralizing our domains. And if we go back to the app requirement, the app requirement states that calculator operates only with two numbers. And that means it's a core problem. We have eliminated the first API side. We have eliminated the database side. And we're directly going to address the domain modeling. And this is like creating a class for operands. And let me show you how, just simple enough. So add operands, it's just adding the model.py file. And if you cherry-pick it, now we have this model.py. And the model.py is just a data class. In a terminology, this is a value object. But just don't stack with the terminology, it's a class. And the operands, it has left and right. And it has returning the hash, because to make it unhashable. And it has an equality check. We can see the operands to be equal only if the left and right values are equal together. The next is to add, let's see what I'm going to add. Yeah, it's this strategy called to create and get domain factories. And it's just simply returning back the operands domain model. Why it's powerful? You can add validation, domain validation, and extra things inside the operands factory instead of making this validation inside the class. Domain model should stay pure. An operands factory can handle all this validation stuff. And as we have the all kind of, why it's disabled? Yeah, the next thing, of course, we have added the first domain model. We have added the factory. That means we can add the unit tests. Nowhere without tests. The testing domain models. And testing domain models, test if operands is created. Test if operands is created with factory. And you can check with the names that it's intuitive. Test if operands is created with wrong types. And this is a bunch of test suites that we have here. How to install and run a test? If you go to the project TOML, we are going to build system as a fleet. we are going to use a fleet for our build system. And then you have a calculator after you, authors, I don't know, read me, dynamic requires a point 10, and dependencies, this is a production dependencies. And we are going to use dependency injector containers here. And that's why I just installed it beforehand. Optional dependencies, the black, I thought old flag, and all the stuff that you need to have to have in your project for common style, linting, checks, type checks, security check, and other stuff. And for the test, we are going to use PyTest and PyTest coverage. And for the make file, the PyChannel has also this neat thing. So we have make lint, make test coverage for all of things. And also, we have this install dev. You can hit it. It fails for we have not virtual environment. Then we need to create it Again, running this install diff And now it says a normal model named fleet because we need to install the fleet for this And I'm going to source when bin activate pip install fleet 371 and then again once again I am trying to install the diff it picks up all the dependencies it install everything for me it's trying to install and we have finished and now we can what happened here. OK, anyway. And now we can go to the terminal and run make test. And all our tests have been passed. This is a where. So I stick with make files because it's easy to develop with make files. But you can always run the commands directly, so no doubts there. We have added first bunch of tests. We have added our domain model, the next thing. The next thing, if you spot the domain model, it has no validation on it. This is quite like flow for us, and we need to address this. I'm going to use Marshmallow for the domain validation. And so nowadays, everybody is keen to use it like Pydantic things, but it's the Marshmallow also for serialization, deserialization. but it's also powerful for creating the DTOs. The DTO is a data transfer object. And we add an abstraction over the domain model to validate. We do not insert the validation inside the domain model. I just added not this one. And here, we create the schemas. And we import the exclude schema fields from the Marshmallow. And it's just a schema with left and right. You specify they are required true. And also, it has handy class meta, which is unknown exclude. If somebody tries to inject unknown field to our schema, it's going to just exclude, ignore it. This is a case that we can use. And of course, we have a project TOML. we have updated the dependencies. And that means we can run install dev to grab this dependency. And if we go back to schema, this is not a really thing here. How about model? We don't even use it, but I believe I have added some tests here. Yes. We have added the unit test for creating a DTO. And we have this test if operands create DTO created. And we create it. The Marshmallow works in this way. You first declare schema. You then load the dictionary to the schema, and it's going through the validation. And if it's validated, it returns back the specified type of the dictionary. And this is quite neat for API development as well. It just operates on a dictionary. It's quite intuitive. And test if operands can be created with wrong types. Now we are going to try to inject a wrong type, which is a left, is a list, and it fails with a Marshmallow exceptions validation error. We have decoupled the validation from the domain models because it's not responsibility of the domain models itself. Again, that right side is if you try to insert a string, it's going to fail with the validation error. And we can run this test as, again, make test. We have added five tests which all they passed. Green, green, we like green. So next. So next is adding more tests for where? Why is that there? OK, so add operands, create a marshmallow schema. We have added. Add more tests for operands, create DTO. We are adding a bunch of tests as well for the operands, create DTO, which is quite. So this one is new. Test if operands create DTO can be created with extra fields. As you see, we are trying to inject an extra field to our schema, but it's going to exclude it because of this meta exclude. Quite wise. How about the models? Is it changed? No, I believe. Yes, now we are going to change our factory, domain factory. And let's cherry pick it. With the domain factory, we have the schema operands create DTO inside. We create a model of the operand and then validate inside the factory if it's valid, validly created, the type of at least. And we return back the model. So this is just wrapping up. We have created the domain model operands. We have added a validation with Marshmallow schema. We have decoupled the validation logic outside of the domain model. Then we use operands factory to create the operands model with a validation. This is a, yeah. This is the first thing that we have addressed. The question so far, yeah. Yeah. where your validation should be outside of the model. Could you explain that a little bit more? Yeah, sure. Would you mind to repeat quickly the question? The question was that FastAPI has built-in pyridentic validation, and it's validated at the API layer, right? But here, we are validating the domain model. So the question is, could we exploit it a bit more, right? The thing is, what I'm doing, I'm validating each layer on its own layer. So you should not delegate the validation to the other side. In terms of that, the service layer's validation should go alongside with the service layer. The domain model validation should go alongside the domain models. The use case validation should go with the use case, and the API validation should go with the API layer. And this is quite powerful. When you have proper architecture things, when frameworks just forces you to use some kind of tool, you just can't ignore them. And as you see at the end that we are using Marshmallow, and we don't use PyDanting, our first API will work. So it's kind of validation logic. It just goes alongside each layer. Does it make sense? Or you need? OK, good. We have a few more short questions. Yeah, sure.

Speaker 2 [28:42]

thank you one question is do we need to know domain driven development if we want to develop in hexagonal manner

Speaker 1 [28:51]

Not at all. You can build an application without DDD. And as you see, we are using just small portions of DDD. So with the DDD, there's a kind of aggregate, aggregate root, and other stuff that can be developed. But here, mostly, I don't need such complexities. But you can always use the DDD. Hexagonal architecture, onion, and the clean, they are agnostic on what kind of internal architecture you are using. So the question is no, no need for DDD.

Speaker 2 [29:29]

Thank you. Next question. Why create a model in parentheses data class and a schema in parentheses Marshmallow when you can create both in one go with Marshmallow data class?

Speaker 1 [29:44]

Good question. So this is the answer. The same thing goes with the pydantic schemas, right? You can create a model with a pydantic schema. You can create a match model schema. But the thing is that your domain model should not depend on anything. As you see, our operands, it's pure class. It just uses built-in Python, CPython data class. There's no third-party dependence on it. If you put in or use third-party dependencies in your domain models, you are in trouble on testing, and also it's not flexible enough. The requirement for the domain models, it should stay as pure as it can be. And then you can add that extra layer with a marshmallow or Pydantic. You can replace the schema here with a Pydantic. But the domain model doesn't care about what kind of technology you use. It's just a class. It's just the information.

Speaker 2 [30:45]

Thank you. One more question. Why do you keep the validation separate from the model?

Speaker 1 [30:55]

The obvious answer is that what I want to have a dependency flow with the domain models, if I have created the object and in the constructor of object I am going to validate the things, this is a reason to change my domain model if my validation changes in the future. This is a violation of single responsibility principle from the solid. The class should have only one reason to change and only one reason to change. The domain model change dictated by the business change, not the validation change or validation logic change. And that's why we have shifted our validation outside of the domain model and stayed pure, validated, that in the future, their business, there's a kind of validation check. Now we want, for example, password length to be 10. I don't want to touch the domain model because it's a crucial part. It's the center, the heart of the complexity of our application for the simple reason that the password length has been changed. But instead, the validator should be responsible for the validation logic. This is a.

Speaker 2 [32:12]

One more thing This is a Django related question. I'm not sure if you are so much into Django Would you use a Django model to define the domain model or would you use Django only on the outside layer?

Speaker 1 [32:28]

With Django, it's pretty complicated because Django Aureum is just the heart of the Django. It just directly starts from the Django models. And that's why it's quite complicated to apply some kind of architectures. But in this book, Architectural Patterns with Pattern, Bob and Harry defined a way of abstracting things in a Django. So you can read the book. There's a kind of appendix about this topic. And for the Django, it's not opinioned, the framework in terms. It dictates the rules, and it's NVT. So the Django itself is an NVC or NVT framework. But instead, like Sanic, Flask, FastDP, and other stuff, it's just quite flexible on this. You can react to things with the different frameworks.

Speaker 2 [33:26]

Last tiny thing. Maybe you will answer that later. I'm not sure. What are ports and adapters anyway?

Speaker 1 [33:32]

Um, yeah.

Speaker 2 [33:34]

Maybe we defer that to later as we go on with the presentation. One more question here from the audience.

Speaker 1 [33:42]

Thank you. So my question is, when I have a domain model, I really want to make sure that it is impossible to create instances of the model with inconsistent state. So I'm wondering, if you have certain rules in your domain model of potential states that would be inconsistent or incorrect, how would you prevent them from being created? The question is that to prevent direct creation of the domain model? There's something like this? I think maybe this would be a solution, yeah. Is there a way to do this? Or would you have then a different validation in your domain model? How would you solve it? You can have double check in your domain model, if you will. But I do not do the validation inside the domain models, at least inside the value objects. But what you can do, so you define a contract in your team, in your project, that if you want to create a domain model, you should operate with the factories. That is the only way to create it. And this is the only way of creating. And if you see some like merge request or pull request that the people trying to access the domain model directly, just to not merge it, refuse it. And this is kind of cultural change inside the team. So they should accept that there's no way to directly access domain models. There's some kind of, with DDD, you have aggregates, and you have aggregate root. It's hard to hit. But with this kind of architecture, you should just, I don't know, give a workshop for your team that, hey, please do this in like this way, create a proper documentation on it. I don't know, create an RFC, design, record, all that stuff. We can go. Thank you.

Speaker 2 [35:44]

Thank you, then let's go on

Speaker 1 [35:46]

That's a good one. Yeah, yeah, sure. Thanks. And now we are done with what? We are done with the calculator operates only with two numbers. We have created a value object or first domain model. The second thing is add, subtract, multiply, and divide actions. What we can do with that? I'm going to create a service for this. You are going to have, you can call it domain service, or you can call it just the services layer. And if we cherry-pick implement operands add action, inside the ports, we have created the services. And inside the services, there's an operands.py. And operands is an interface. In Python, we have no Golang-like interfaces. But obviously, we can force it with abstract base classes. And operands service interface has this left and right add functionalities, just adding, summing up the numbers, obviously. But if you see that we have public method, add, and inside the public method, we are returning as a protected method. And the protected method, or private method, underscore add, is an abstract method. What kind of question can arise here? We define open service interface, and what is an interface? Interface is a contract. We agree that somebody is going to implement our interface, is going to implement this underscore at. Why we need it? There's a kind of open-cause principle from the solid, or from the solid, it dictates that the interfaces should open for the extension, but should be closed for the modification, for the changing. And we have already shaken our hands, so we have agreed that we have a public ad. But in the future, if you want to change the behavior of this public ad, what to do? Just reimplement the underscore ad. Do not touch the public method, public contract, because we have already agreed. But if you want to change it, by a minute you want to extend it, then you are going to change this underscore at and implement it in a different way so imagine that you have a fancy thing that your calculator not adding but subtracting with the ad your fools of people you can change down the ad without touching the public ad this is a this is a naive way of achieving this up and close yeah you know a question why do you prefer this like templating pattern i would call instead of just having abstract class where the public add function is abstract but has to be implemented by some realization? Yeah, it's a good question. But the thing is that the public method, you give a public method to the contract implementer, the interface who implements. And you agree that, OK, this is not going to change in the future, right? And at some point, if you need some kind of change, you are going back to change your interface. It prevents the change of the interface. You do not touch this public ad inside the interface itself. It's one way of achieving it, but it's the truth that you can also use force to re-implement the publics. But I can't force to re-implement the privates, because the public, we have already agreed it. We don't want to re-implement the sync. There's a design decision. Obviously, you can change it. But this is what, at least I think that it's the proper way. But it's always, there's better ways. This is a port side. So in the, yeah, the question, OK. I'm new to Python, and I was wondering why you don't use the port of course in this case? Oh, sorry. Yeah, sorry. I'm new to Python. So I did a bit of hexagonal architecture with Go. And you can use interface. But you don't have this concept of abstract classes. And I'm not super fan of inheritance. And I heard of protocols. But I wonder why not use protocols here? And why do you use abstract class? Yeah, nice question. With the protocols, it's hard to ensure the integrity. So it's just duck typing for the patterns. There is no kind of runtime check if it's truly implemented all kind of methods inside the interface. But with abstract-based classes, I can force it. I can rise a runtime error, it's not implemented error, if somebody missed this. So that's why I'm forcing the people that, hey, if you want to conform with my interface, with my contract, you have no other choice of implementing this. But with protocols, you can omit it. It's just close your eyes, because there's no runtime check. So with implicit declaration, implicit? Yes, implicit. So it's kind of I'm defining that explicitly that you should implement this. You must implement, not you must implement. This is a correct vote. Can I continue? Yeah, great. This is a port side. As you see, in a port side, we have services and interfaces. In contrast, in an adapter side, we have services, But we have a concrete implementation of this interface. And in operand service, we inherit from the interface and create this and implement this on the scope at functionality. It's just creating a data dictionary with left and right. Then it validates with a marshmallow because we want to use this operands create DTO while creating our operands. And then we create the operands domain model. And afterwards, we add these two numbers. So did you see the layers? The service hits a schema validation. The schema validation calls, after schema validation, we call this domain factory. Domain factory creates a domain model and returns back. This is an idea, so we have layered. And that means that we can have a test right now. is have a test for our service add functionality. It's cherry pick. And it go to the test. We have this test services. For the test services, we are going to check the test operand service add result. We have this fixture called get operand service. If you go to the fixture, it just creates an operand service instance and returns it. You can use this later on with your tests. And we are asserting that, OK, actually the addition was correct. Then the next test, the test operand service added with the wrong data types. We're trying to inject the wrong data types to our service layer, but it will catch by the marshmallow that hits a validation error. We covered our safety net with that. More unit tests. Yeah, sure. We love tests. And the test operand service at result, we are just adding a bunch of tests here also and making sure that the left side and the right side, if you insert the wrong type, it's going to fail with the validation error at the end. Then it's kind of subtraction. It's pretty simple. It's the same thing. We can go to operands and see that we have at the end, this is a total same thing. And at the end, we are going to subtract. The same thing for the multiply, I'm just going to quickly through that. The multiply is a multiply and the divide is a divide. And this is a kind of encapsulated the actions inside the service. But if you see that we have duplicated the quote portions, and that's why I'm going to introduce a context manager, which is quite neat. So from ContextLib, import ContextManager. And here, we are yielding the actual operands domain model. And inside the service, we can use with operands factory validation. And it is indeed a factory validation. And then get the operands and return the actual result. So we have implemented the ContextManager in order to eliminate this duplication. And also, we have extracted the validation logic out the service, as you see. Now it's the responsibility of the context manager, not the service, because it's the context manager validates the schema. And let's see. So what we have, OK, obviously, this is the most well-known prominent exception, zero division. And now we can hit it here, but I'm going to show you another pattern called design by contract, or DBC, where you can cover your code portions with the contracts. So what we mean, if we go and cherry-pick this defend, let's just go defend, divide, action from zero division using contract. And for these purposes, let me just go to the PyProject.toml. Inside the PyProject.toml, we have a new dependency called iContract. iContract is a library which implements a design by contract pattern. It's quite powerful for your domain models. It has prerequisite, post-requisite, and then it was an invariant check. If you have a question, what is a class invariant? It's a state that should be always true through the lifetime of your application or through the lifetime of the class. You can define your invariant with the I-contract. And if you go back to the operands adapters, here we simply define that I-contract require right side to be not equal for specific divide action. This is a different thing, what we usually see in the tutorials, the most of the things that we are going to try accept, then catch the zero division, and then act accordingly. And now you see that there's a contract I require before hitting my divide, the right side should not be unequal. It's a different kind of thinking, and there's no need for creating a validation to catch exception inside the divide action we have excluded again this responsibility outside of the service and delegated to the I contract what you can use for the art contract it has you can stack this obviously it's a decorator you can stack with all decorators and create a different rules for your critical section this is our critical section because I don't want to see the zero on the right side that's it and if we check the test let's go back to the tests let's check if we now we have concluded with the trade picking let's go to the test and inside the test services this is quite interesting just maybe you can um you can see this so we expect the not zero division error we expect a violation error this is a violation of the contract because we have defined that we don't want to see the zero on the right side and if you want to violate you hit by the eye contract and it's a violation error So you can check the difference. This is a validation error, because it's a validates. It hits already there and validates. The violation is just you violate a rule. It's a different mental thinking state, I'd say. And this is a quite powerful tool you can use for your applications. Just check it, design by contract, DPC, iContract library. I want to install it, and then for sure run the test, just to be sure that we don't make anything. 15 tests passed, green, we are done. We are pretty happy. The next. Okay, the requirement has been met, I'd say. We have operands, we have the service, and now this section, it should store the result of actions in the database, but we are interested in this first part. The question that we need to ask to stakeholders interested part, what is the action for you? And what is the result for you? And the obvious answer for our calculator, action is some kind of add, multiply, divide, and subtract. And the result is a result of these actions. And when you write down the words, okay, you should go back to the domain modeling because you should model your ideas. And if you, yeah, so here, add calculation model and factory. I'm going to cherry pick. And again, once again, we are going to model. And now we have this data class called the calculation. The calculation, something that has UID, unique UID, something that has left-right action result and created at an update at fields. The calculation factor is responsible for instantiating the UID. I'm using ULID. It is from Python ULID. You can check why it's better than an ordinary UID4, but it's a different topic. And you can check it on your own. Let's just install it quickly. And it's installed. Go back to the model. And here, if you think about what we can have differently, we can have a calculation class within it, the constructor, And we can create this UID inside the constructor. And once again, we don't want to do any initialization for the domain models. We delegate it to outside the calculation factor. And it is now responsible for creating UIDs. It's also responsible for extracting the action type. An action type is an enum, which just predefines the actions. It's quite simple. Add, subtract, divide, multiply, which has its own values. And the calculation factor is responsible for creating UIDs, for extracting the action type, for instantiating the date time, this current date time. So all those responsibilities not burdened inside the domain model, but it goes for the factory. So next, we need to add a calculation create the question. So the unit thing, you're kind of adding dependencies inside your model .. No. Sorry. The question is that with ULIT, we have added dependencies for our domain model. But if you see our domain model, we are storing string. Yes. That's why I'm a bit confused here, because you're having your actual model calculation, and then you have the factoring for the calculation. Yes. In your case, both in the same module inside your domain, which is confusing me. Then you have the definition to deal with. I don't know to which point you should be with this, but for me it feels wrong to ask, there's no library inside of this domain layer. You mean that why is the calculation factor used? OK, this is a good question. The idea was that if your calculation factor depends for the third party library, you can extract out from the domain model layer. That's correct. I totally agree. But the case of the simplest, just imagine that it's a kind of extra layer. And before you hit your domain model, it goes to the factory. So yeah, this is a good point. Thanks. And yeah, the next. We need to create a DTO for our create model. Let's go to the schema. And inside that, so now we are pretty done with your logic. So you say that you should extract out this ULID and, I don't know, validation steps. Now we are extracting this to the outside schema. And inside the UID, we have this default, which is a callable. We create a ULID with our Marshmallow schema. Then we validate it as one of this is going to, so that the action should be one of the keys from the enum. And also, we are creating the datetimes inside the schema. And the class meta just ignores the unknown explode. And let's go back. What's next? Change the calculation factor. Amazing. So we need to change the calculation factor. The calculation factory now just accepts the UID. We have eliminated your concern. And here, we just add an extra layer of the schema validation. So the UID creation and also the action, UID creation is delegated to the schema. Now, our calculation factory simply accepts any kind of information from the outside. side. Obviously, I hit a circular import issue here. Sorry for that. And I'm going to fix it. The idea was that we have now this kind of requirement inside the schema as well. This is a double check. The first, so what we have, the first things that we have is the schema calculation create DTO It just validates against the strings. And inside the model, we have extra validation with the enum type. It's a double check of that. Because the action part is a crucial part for us. So next, obviously, I had a failing test. And I need to try to pick the fixes. Let's go to the test then. With our services, what kind of, I need, I need, I need, what it was, okay, so I need to test it, yes, all tests have been passed, I need to find this calculation model, because I'm in the wrong place, I need to check the test domain models, this is it, this is a place there where we are creating the calculation. So test if calculation model created with a factory. Calculation created here. Now calculation factory accepts just the keyword arguments. And if you go to the calculation factory, we have just changed it. We have added an extra check for the action as well. So there's a double check. Why? I'm going to explain it. So if you are going to test if calculation model created with factory with wrong action, if we are going to use it with the wrong fake action, we hit the marshmallow. It says validation error. But we can always bypassing. We can create a proper dictionary. Then we can check the action. And then we want to inject this to our calculation factory. And because the enum checks this action, it fails at the enum level. So it's a double check to ensure that our domain model is treated properly, I would say. And we have all tests passed. Good. Can we take a few more questions? Yeah, sure. OK.

Speaker 2 [57:24]

I'm going in some randomized order. What if we want to add additional action type or types? Will we have to change the validation as well?

Speaker 1 [57:38]

Yes, sure. If you need to add an extra validation for a new action type, obviously, you can centralize it. You can create some kind of common settings and you can grab all kind of actions from the common place, or you can store it some kind of in a database with the supported actions, and you can insert a new action to the database and fetch there. There's a kind of different styles you can follow.

Speaker 2 [58:07]

Next one. How do you not confuse the database implementation storing details and the domain model, meaning that it's easy to pollute the model with too many details?

Speaker 1 [58:21]

again

Speaker 2 [58:23]

How do you not confuse the database implementation storing details and the domain model?

Speaker 1 [58:29]

All right. So the question, I guess, is about decoupling the ORIM side or the database layer with a domain model. We will see it because we have this commit regarding to this repository patterns and unit of work pattern and then abstracting the ORIM side from the domain model. The obvious solution is that when you start your application, you map your domain model with your ORIM model. but it's only with the start, you don't need it anymore. It's just started, mapped and used. It will be clear in a moment.

Speaker 2 [59:06]

Okay, thanks. Why do we specify the I contract in the implementation and not the interface?

Speaker 1 [59:16]

In the interface, we cannot force what kind of implementation that people should choose. We just give a contract that A, you want to create this add or divide action, right? And the implementer is responsible for further actions, the validations. The interface is just a public facing. So I have this one you need to implement. I have this one with two arguments you need to implement it. You cannot force, or you should not force the people to stick with iContract. It's a detail, it's like one possible solution. Maybe the people want to use a different solution, that's why it should delegate to the implementer.

Speaker 2 [60:02]

Thank you. Why do you define double underscore hash, double underscore or dun-dun-dun-dun hash, on a mutable data class?

Speaker 1 [60:14]

What? Immutable?

Speaker 2 [60:16]

on a mutable data.

Speaker 1 [60:17]

data yeah obviously the value objects you can use this is frozen true to make it immutable but yeah I have omitted this process here it should not be checked it should not be changed in the runtime but we creating a hash first of all that why I've created because I want to make this object hashable and want at at some point to store it as a, for example, a key in a dictionary, my domain model, right? And if you implement hash and equal, it's possible. That was the reason for me, at least. But if you know some kind of other, I don't know, anti-pattern, you can share it, obviously, so we can learn.

Speaker 2 [61:07]

Thank you. Let's move to the next one. What is the effect of the meta class you added in the schema validation?

Speaker 1 [61:16]

That's a good question. Let's just clarify it. So here, right? In a schema validation, we declare that if the field is unknown, we need to exclude it. The reason is that somebody can try to inject some kind of fake field to the schema. And our system, instead of checking all the fields, checking the field count, I don't know all the extra check. I'm going to exclude it. I'm going not to take it Take it into account. This is a the unknown exclude. So it's it's a feature of the the marshmallow. I just use it

Speaker 2 [61:57]

Thank you. One more question live here, the audience? No? Let's move on.

Speaker 1 [62:05]

Yes, and now we are done with these requirements. It should store the result of the actions. We have defined what is the result, what is the action. It's a calculation domain model. And this is a tiny part in the database. We love it. And we are going to implement it right now. But we are going to abstract away the database implementation using the repository pattern. Let's just go through. Implement calculation repository pattern. the cherry pick. Inside the ports, now we have these repositories. This is a kind of interface. And inside the adapters, we have this calculation, the calculation SQLAlchemyRepository. From the name, it's clear that we are going to use SQLAlchemy Orion for our purposes. It's related to the calculation. And it is a repository. It implements calculation repository interface, what it has. It has this underscore add, which is adding. Self session is a connection to the database from the SQL Alchemy, and we are adding directly our domain models. Who sees the database models here? There's no database models. We map at runtime the database models to our domain. We operate solely with the domain models. We are adding to the database our model calculation. We get, using model calculation, filter by ID, get by UID, get by action, and get all. It returns a list of model calculation. This is just SQL Alchemist sync. So it's kind of ORM style to executing SQL queries. We have abstracted all of those things away from our domain model, schema validation, services, and other stuff. And now we have that layer of the repositories. That means that we can obviously test it without no database. How to test it? We can fake our repository, add fake repository and the tests. What is a fake repository? We can go to the test. We can, hmm, amazing, fake repository. The fake calculation repository still implements the calculation repository interface. That means it does not violate any kind of interface. It still conforms the contract. And for the sake of simplicity or testing, we are faking the database with a dictionary. Then while adding the database, this is the actual insert. We just store this random ID in a dictionary. When getting back from the ID, we just use this ID as a dictionary key. Get by UID, we traverse from the dictionary. if the UID is equal to the UID that we are searching for, we return this model calculation. And for the getAll, we just return back the self database itself. Obviously, so expect the type list calculation, but gotDict. This is a fake repository, so no need to to correct everything. Just use it. And then that means that we can test the repository. We can add a test calculation. So test calculation, test add calculation. We can get fake repository, returning back a fake repository instance. You can check it from the conf test and where it was. And it's get calculation model object, which is another fixture in the conf test. We just create a dummy model and return it as a fixture to the test calculation. And then we somehow add it. But obviously, it's an addition to the dictionary. But imagine that it's a true database connection and it's adding. There's nothing different with that. And you can get the dictionary values. You can test get calculation by UID. You can test get calculation by action, tests get all calculations. And for sake of the bravery, I just install it and test it. Our repository test also passed. That means we have abstracted away our database actions with a repository pattern. And even we write a unit test without a database. Because a database is here, is outside. It's a detail we don't need right now. We will need it in the future. The next. OK, obviously, with the databases, we need some kind of database migrations and also this ORIM. Now we are ready to work with a real database. This is the add Alembic things. Alembic config, Alembic has its own configs. And we store, I store, actually, in an adapters, In a DB, this kind of already configuration, config change come from Alembic. There's nothing fancy here. The rest. Implement Alembic database migrations. I'm going to implement this. And now we have this Orion Pi. And this is the answer to the question of how you decouple the database layer from the domain model. Just install it in order not to see this red. Great. So with SQLAlchemy OEM, there are two types of mapping. One is declarative mapping. The declarative mapping is what you see with Django models. You create a Django model, for example, class user. It inherits from the Django model. And this is the declarative way of the actions. But there is another kind of mapper with ORION. It's called classical mapper or imperative mapper. And we are using the imperative mapper here. We are creating a metadata. We are creating a mapper registry. And we declare the table imperatively, like manually, what kind of steps it should have. It should have ID, UID, left, right, action, result, created, add, updated, add, and a unique constraint. As you see, this thing is new because it's from the database, but the rest of the column is a copy, like the mapped copy, of our calculation domain model. And SQL Alchemy understands it when you map it. So we define a start mappers. You have a bunch of 10 tables, right? You stack the mappers inside the start mappers function. And you use mapper register with map imperatively. We implicitly, not implicitly, imperatively declares that our model calculation mapped to the OREM calculation. And that way, we keep clean our domain models. We do not import any kind of OREM-like thing inside the domain models. Domain model still stays clean. But we have decoupled the OREM to the outside And now if you see the arrow, OREM itself depends on the model Model not depend So you see the arrows, the dependencies on a slide And this is a true way of treating the OREM as well OREM depends on the domain model, but not domain model Depends on...

Speaker 2 [70:05]

two more short things the domain model data data class used for sequel alchemy as a data sorry one more time the domain model data class is used for sequel alchemy as a database table representation what if the model needed a relation to another database table

Speaker 1 [70:26]

That's a good question. This is actually the most naive way of implementation of domain modeling. Obviously, you need to have aggregates, aggregate root, and other stuff that acts on top of the tables. But I usually would do, if you want to create a relation without domain models, you can create here with the OREM and define the relations in the map imperatively. So here you can define the relations. I believe it's kind of, I need to find how it's doing. But this is kind of things that properties, yes, properties, you can define that, okay, model calculation and the calculation. Then imagine that, so you have this calc mapper, then a different mapper, stack of the mappers, and then you define the relation between the mappers this is a way of how you can do it the relation is imperatively

Speaker 2 [71:31]

Thank you. Can one think of the domain model as the interface for the database or would this be a reversing of the direction of dependency?

Speaker 1 [71:43]

Thank you

Speaker 2 [71:45]

Can one think of the domain model as the interface for the database or would this be a reversing of the direction of dependency?

Speaker 1 [71:56]

So, I guess the question is about if we can create database models first, and then create the domain models on top of that, or revert, ah, no, it's not the way that we should do, because the domain model, our domain models is a simple, they has no business logic, they has no aggregate, aggregate rules, some kind of other stuff in it, it's just a value object, But imagine that you have this big domain model, which has a hard business logic, solving some kind of problem. It's nothing related to the database itself. Domain model should solve the problem of the domain, and that's why you should first tackle with a domain model, no databases.

Speaker 2 [72:47]

Thanks, yes. But now we have two places where the fields of the calculation model are defined. If we change the model, we will also need to change the table.

Speaker 1 [72:59]

Yes, this is the idea behind this, if your domain model changed, then the rest of the things should be changed because this is the core of your application and now as you see it, now ORM depends on the domain models but imagine the vice versa, you are going to change the database tables, fields and everything is breaking down but at least with this strategy your test will pass, except some kind of database test

Speaker 2 [73:27]

I think if you use Alembic here, there will also be some sort of a migration process helping you.

Speaker 1 [73:31]

Sure, and then you come back to the SQLAlchemyOrient to change the fields, then create the migrations

Speaker 2 [73:32]

Yes, sure.

Speaker 1 [73:39]

and then run the migrate.

Speaker 2 [73:41]

One last thing, maybe. Why do you inherit the repository in fake repository instead of using the real one and mock the methods?

Speaker 1 [73:52]

This is a battle between testing strategies that mock stops, I don't know, if they're powerful. What I see, my idea related to the mocks, if it's getting more, I lose attraction with the mocks. Where I did the mock, when I did the mock, why I did the mock. And this overcomplicates the testing strategy. And I am using the fakes. This is a getting a same result more clean way and even your fake repository like fake Classes should conform with your with your interfaces by that way you ensure that you do not violate the interface there as well, so using real We are going to use the real actually database connection, but with fake dependency injector container It's nothing related to the actual database. We just we just make a connection to the SQLite if it's test and then the production database but inside the dependency injection container we will see it

Speaker 2 [75:01]

Okay, cool. We have 12 more minutes to go.

Speaker 1 [75:03]

Oh, yeah. Moment, Will. OK, this just goes through quickly. And at the end, we have the questions. So we have implemented repository, but we are missing the repository pattern, the atomic actions with the database. So if you see that we have no commit and rollback things. For creating this kind of stuff, we are going to use a pattern called unit of work. And let's just implement it quickly. We cherry-picked, yeah. So again, the port size is an interface. They just go with adapters because it's real implementation. So CalculationSqlAlchemyUnitOfWork. The name indicates that this is using the SQL alchemy. This is a unit of work pattern. And for the domain model of the calculation, it implements CalculationUnitOfWorkInterface. And we have defined this as a context manager with a class. What it has? It has a session factory. Session factory came from the SQL Alchemy connection. This is the actual connection. When we use with unit of work, we enter to this portion as a context manager, and we open a connection. We open a connection to the database when we need it, with. With unit of work, we have a connection. Then we have a calculation, which calculation sql alchemy repository so we operate on top of the repository pattern using this unit of work and obviously when we the context manager exits it's we are going to close the database connection we have commit which is calling this sql alchemy commit we have this rollback which is using this session rollback just as sql alchemy things and obviously we need we can test it right Let's just first restructure our test forges. Now we have this domain, repostered services as a restructured test forges. And the next is add fake unit of FERC. We have faked repository. We can use it inside the fake unit of FERC as . Let's go back to the test unit of FERCs and before that, fake UAV. The fake UAV, we're faking the calculation unit of work. We say that, OK, self-committed, it falls. And the self-calculation, we do not need the session inside the enter, because we have no actual database connection. That's why we have omitted it. But we inject the fake calculation repository inside the context manager. And the self-committer is committed to roll back, just passed because we don't care about the rollback, nothing to rollback with the fake. And if you go to the test calculation UAEV, now the magic happens here. With gate fake UAEV, we have a great connection to the database. And then we use gate fake UAEV calculation at. This calculation is a repository. This is an exact copy of the test, what we did in the repository, as you see. But the thing is that now we operate with a repository pattern using unit of work. We have added extra abstraction over the repository. And I believe that we can install it, and then even hopefully test it. Yes, all tests are passed. So without any kind of database, we have operated to create an abstraction using repository and unit of work. We have fully tested it. Yeah, magic. Good. The next, just quickly, so we don't need it. And now is the most interesting part that we have unit of work, repository, domain model, and all the stuff, even OEM. But how are the things wired up together? Now the dependency injection container comes to the land of the workshop. And feature, add production DI container. Let's just cherry pick it. The iContainer is a thing that came from the Dependency Injector library. It's quite a really neat and powerful library. You can use it. And if we go to the layers, we have added this configurator, which has settings. The settings is just the DB URI, database connection string. And I use the extracting methods for each settings as well. This is a pretty neat and clean way. If you want the settings URI, just call this gate database URI, and it will extract the proper settings for you. And inside the containers, so .. Inside the container, we have this containers declarative container, and we get the database URI. We create an engine. This is an SQL alchemist thing, as you see. We open, we create an engine, and we define default session factory, which is a laptop function with a session maker. This is an actual database connection, so it's going to open the connection. But we define as a lambda. It will be used later in a unit of work. We don't need to open a connection directly inside the dependency injection container. We delegate it. And calculation UAV will provide a factory. It's a provider. That means it depends as a provider with actual calculation SQL AlchemUnit of work. And we inject the session factory as a default session factory. Obviously, the thing is that, yeah, if you go to the SQL AlchemUnit of work, it expects a session factory. And it's a callable. And indeed, we just injected the callable. with a lambda so it it gets what it expects and let's just install i see some kind of weird reddish thing quickly quickly quickly fake container amazing we need the fake container for faking our dependencies and the fake container is an is a copy of so hardcodes adb connection if you want here like a fake container should open a connection to the fake database or or SQLite, at least, or your development. And you can create a dev container, staging container, to manage your dependencies properly. And from the settings, I believe we have changed the settings. Now if the test runs there, we are going to return SQLite. If there is no test run, that means we are going to use a production connection. That's pretty good. OK. We are going to implement the integration test. Integration test I'm going to show, show, show immediately. Yeah, if we mark it as an integration, we get the calculation unit of work from the fake container. We actually add the calculation model, then this is a true commit. This is a way that actual database used. And then we check that if the database has these values added. So this is actually the integration test for our working. We have fake test with the calculation unit of work. Now we have integration test with the calculation unit of work. And then it's no, no, no. Oh, yeah. Then the next layer. But I feel that we are running out of time. But just let's quickly freeze. We add the next layer with a use case. Let's just go through the use case. And I'm going to try to pick all of them, just not to waste the time. Getting all UIDs and getting all. And then immediately go to the adapters. In adapters, we have these use cases, which is actual things that wear everything wired up together. The calculate use case implements a calculate use case interface. It has unit of work dependency, and it has a service dependency. The calculation unit of work interface and operands service interface. The question can arise, OK, dependency is an interface. Where is the actual instances of this class? And this is the magic of the dependency injection container. It's injected dynamically by provider, dependency injection provider. And if you go to the container, we have the operand service. We have a calculate use case. And it injects it to the use case in the runtime. As you see, we are getting the actual object from the dependency injection container using this dict key notation. And then inside the add, we calculate the result. We save the information with the result. Then we subtract, we multiply, and other stuff. We have this get all, get by UID, which is pretty similar thing. So we open a database connection, and we get by UID. And if we have the result, we return back as a dictionary. This is a use case, do we have time in a minute?

Speaker 2 [85:08]

We have a few more minutes, like two or three or so. We have a number of questions. I bet there is more content on your slides, but I don't think we will have the time for all of them.

Speaker 1 [85:19]

It's just the last part, the first API.

Speaker 2 [85:22]

Then rush through it, maybe if you can.

Speaker 1 [85:26]

So just we have this, that everything is placed. And then we have this adding fast API calculation. And let's just quickly go through the entry points. Entry point is a layer that we hit Enter to our application. We have API. We have V1 and root calculate. Let's just try to pick everything. And here, you're, yeah? Could you just address the questions? Because I can go through the commits at home. OK, sure, sure, sure, thank you. So obviously, people are asking for the questions.

Speaker 2 [86:00]

Okay, then we switch over to the remaining questions.

Speaker 1 [86:01]

OK.

Speaker 2 [86:05]

Finally, the one waiting here for the longest time. What are ports and adapters, by the way?

Speaker 1 [86:11]

It's another name of the hexagonal architecture from the original author and the name is ports and adapters architecture, but the people name it hexagonal as well. So as you see the ports is a contract, putting it simple, the adapters is the actual implementations. So ports and abstraction, an interface, and adapters is the actual implementation of these interfaces, putting it simple.

Speaker 2 [86:39]

How easy is it to change the database adapter and use a non-database storage, for example, JSON or some text file?

Speaker 1 [86:50]

As we have decoupled our domain model from the ORIM side, you can shift with ORIMs anytime.

Speaker 2 [86:59]

At which point should I use a more enterprise-suitable programming language? AKA, should a service like this even be implemented in Python?

Speaker 1 [87:12]

This is a question that what we struggle with. So Python is considered for a long period of the years as a scripting language, but we love it and know that it can be applied to the enterprise level as well. But the thing is that we are missing the teaching materials on enterprise architecture, specifically implementation in the Python versus Java or C Sharp, because they have been using this for 15 years or 20 years. As this is a relatively new thing for the Python ecosystem, I bet that the Python can be used in any kind of application complexity.

Speaker 2 [87:50]

And there's one last one here on the list, which makes a last overall question. What advice would you give a developer to not overthink the solution, think technical implementation, and focus on the domain and not mix the different layers?

Speaker 1 [88:12]

That's the hard question, but the thing is that we should change the mindset on how we are going to develop the application. The first thing that we should adopt in ourselves, asking a lot of questions, understanding the domain, what kind of problem we are going to fix, to asking a lot of questions to the stakeholders or your manager, I don't know, team lead, try to understand the domain and then fix this with fake repository, with, I don't know, you don't need the database, you can store it in a txt file as well, right? But you can show demo something. And the thing is that if your application, your code, just fix the core domain problem, the rest of the things are details. So just focus on the domain, what kind of problem you are working with. And fix it first, then think about, oh, we need to choose a framework, and we need to choose the type of the database we need to choose I don't know where to store the logs we had to store the monitoring metrics and other stuff this is all the details

Speaker 2 [89:18]

Thank you very much. I think that was a very nice exercise in adding a lot of overhead to a simple calculation, as we have seen. Nevertheless, I think you will also agree that on larger systems this will pay off.

Speaker 1 [89:35]

Yes, definitely. This is the idea. Thank you. Thank you very much.

Speaker 2 [89:37]

Thank you very much.