How to connect your application to the world (and avoid sleepless nights)

Speaker 1 [00:02]

I'm super happy to talk in PyCon. It's my first time approaching this particular community. I've done talks with other communities, but I've been a great group of individuals, and I see a lot of innovation going on, both in data science and in development front. So just as a quick introduction, my name is Luis Fernando Alvarez. I'm an engineering manager at Stack Builders. Our company focuses mostly on functional programming, I'm happy to talk about that after the talk because it's a big topic, I have a few years of experience in the industry, I come from Ecuador, this is 10,000 kilometres away, and I'm going back this next weekend, so it's been a long trip. And yeah, I do all sorts of languages and a bunch of stuff, I am a musician as well, So yeah, follow me on my social media, I think you'll see some interesting things. And as a disclaimer, I don't consider myself to be an expert in Python. I have been focused most of my life in JavaScript. I have been able to participate in a couple of complex Python projects, but the aim of this talk is just to present a few tools that I find a bit useful in the path to connect an application to external services, right? So let's start. So this is your application. This little light line is your application. It's super small in just comparatively, right, to whatever you have on the rest of the world. And in time, you want to make it grow, and it lacks right now a lot of natural resources, you just have a little house and a couple But you have two ways to go in this sense, right? You have two choices, like in the matrix, right? Red pill, blue pill. So you have the first one, which is investing a lot of money and investing a lot of resources into exploring your island and just sourcing those resources from inside. Or you could just start importing foreign resources, right? Bringing them from other services or pushing them to other services and just offloading that kind of work. So here's where external services come in, right? This is a big external service and a lot of consumers going into that service. And as developers, as you know, we build things with one thing in mind, which is challenging ourselves to build the next big product, right? The most complex stuff that probably I won't need. But businesses don't work like that, right? Time and money are super limited. The business requirements change every day. And most of the time, it's a matter of building faster than your competitors. And as a side note, I think this is what makes, what differences a good engineer from a great engineer, right? It's having a clear picture where the business is going and taking pragmatic decisions on top of that instead of just using what's hip or using what everyone's using, right? And building fast is easy most of the time, But building fast while maintaining the good standards and having a resilient system is not as easy as it looks, right? So talking about these external services, we have a few considerations. We have uncertainty on these systems. Will I be able to connect to the other system? Has something changed in the other system lately? Because external APIs and distributed programming on concept is unreliable, right? Even if they guarantee you 99.95% of uptime, sometimes it's going to fail, right? So a good rule of thumb would be to distrust everything, right? Even things that you've built, even the APIs that you've built, you should distrust those as well. I distrust mine more than the external ones sometimes. And you have also a very varied ecosystem, and you have non-standardization, which is an issue, right? Some APIs return JSON responses, some are SOAP APIs, some are REST APIs, some are GraphQL, and not everyone uses the same standards, right? Some use correct codes, some don't. So these operations might be exposed through nonconventional means, right? And the documentation, that's hit or miss. Sometimes you have good documentation, sometimes you don't. APIs are also hard to test. And since we have that 99.95% of time on our minds, then under the hood we think that they shouldn't be tested, right? And testing is expensive and not really straightforward. And security is a big point. I've been going through an ISO certification process, which is a time-consuming process, and has gotten me thinking, are we running risk assessment on every integration that want to build into our systems, and that is something to think about, right? How do we track the changes? How is my data going to be handled in these services? So we're going to try and explore some of the concepts that help us mitigate some of these issues, right? And hopefully survive the process of integration. Who has integrated previously with external systems? A lot of people? A lot of traumatised people? Sorry. Awesome. So, first tool that we have on our hands, right, it's caching, and you might have seen this phrase before, but 90 per cent of the time it's been aimed at the naming things is hard, right? So now we're going to focus on caching. And what is caching? It's just storing a subset of the data in my own system, right? So I won't do a lot of calls to the backend, I won't do a lot of calls to the external system, and the end goal is to reduce them, right, in some way. And then I can later efficiently reuse this data, right, considering it might not be the latest data because it's something that I consumed in the past, but, yeah, that's a risk we have to live with sometimes. And there's a trade-off here because we have high speed when consuming this data, but we have low capacity. We are not going to pull everything from the API to our local environments, right, or our servers. And thankfully, there is just a lot of different backends that I can use to do this, like memcache, Redis, database, system, every system is different, right, and every requirement is different. So for this, there are a couple of strategies that we could follow. So one of them would be the cache aside strategy, which is I think the most common strategy. It works in three steps. You try to read from cache, if it's not found, read from the data source, and when you've read from the data source, you write back that data to the cache, right, for next consumption. Some key parts here, I've used Django in this example, I've used the caching system in Django, but it's super similar using other libraries, right? I want you to notice the third argument to cache.set, which is almost at the end, which is the timeout, right? And here's where it's key for you to know the business, for you to know how often should you pull this data, how urgently do you need this data, right? is set to every 10 minutes, which means the data will go stale every 10 minutes, and this would work for applications that don't need real-time data, right? And it's going to give you this 10-minute safety net. If the requirements I've had to implement systems where I've had to set a longer-term cache, let's say a cache on top of another, right, and one cache that keeps the data for 24 hours, and that would give me a few more hours to solve any issues that I had with the external service. And that's kind of your final safety net, right? So this is the strategy when you're pulling data, and then when you're trying to write back data, it's useful sometimes to batch update, right? And this obviously assumes that you have a way to process that batch information in the end point. So this is also simple enough, what I'm doing here is just getting what is in cache, appending more customers to that cache, and then setting that cache back, and at some point, maybe with a cron job, maybe with a scheduled job, I will send that cache to the remote, so look at that little boat go with a lot of stones. So with everything in technology, this has its issues. The first risk that I've already mentioned is having stale data, right? In some cases it might be fine, but it won't work very well for real-time data in applications. There's a lot of complexity that you add when you add these things to your system, and these are not things that junior developers think about too much. Maybe we are all or most of us are experienced developers and have done this in the past, but in the end, you write code for other developers to read, and sometimes you need that clarity for more junior developers. Security is what I mentioned, right? How am I taking care of my information? Most databases are encrypted, and my data that goes to the database goes encrypted, But most cache backends are not encrypted, and I could suffer an attack from there, right? So a general recommendation here would be to encrypt things before putting it into cache, and maybe when you try to process, you decrypt it. Some common libraries in Python, just to deal with these things, right, in caching. So anyone curious why there's a long line of boats over there? It's kind of like traffic in my hometown, but no, I'm going to talk about task queuing, right? Asynchronous task queuing. Which is interesting because as a JavaScript developer in the past, this was one of the easier concepts for me in Python because I was trained in the dark arts of the asynchronous. So the main idea is that you have to alleviate the load in your main thread, JavaScript being a single thread language, you don't want to block that main thread, so for that you need to deal a lot with synchronicity, right, and just do deferred operations. Network calls are very expensive, computationally speaking, and they take some time, so in some applications this is OK, you can make your user wait a bit, in other cases not so much. So instead of trying to process everything on the main thread, we are going to put some some of those expensive operations in a queue and process it later, because we need to keep that cycle unblocked. Let's think about this for a bit. If we're going to make calls from our main application, network calls, and something fails, what are we going to do? We're going to stop the system, we're going to absorb the error, we're going to make the user wait and then just pop a big error screen there. These asynchronous task processors are going to help us to retry some of the failed calls, and thankfully there's a lot of things that we can do for that, right? There's just a bunch of libraries that do this in Python, we'll go over the list later, but the main operation here is you just enqueue an operation, you process it later, and then most of these systems can do auto retry if the operation examples. You have a little code example here, you are creating a connection, you are creating a queue which is a line of tasks that will get processed in the future. One thing to consider is, as you can see, the third argument is the retry mechanism here. You have three retries with three intervals which would give you, if the remote system fails, you can just send retries over there. Another thing, another important thing is that on the first argument to enqueue there, it's update customer, right? And in Python you have functions that work as citizens, first-class citizens, right? And what does this mean? That functions in Python can be passed to other functions or can be returned from functions. So it's not a reference of the function, it's not like the string name of the function, it is the actual function, right? And this is important, bear with me on this one, because next slide we can see some issues that we can have in task queueing, right? Again, the complexity increase, and again the security, because you are having that information put into another place, right? You're not putting into a database, you're putting that into somewhere that someone can tap in and consume. But one of the key things here is I've had in the past with some of the libraries, maybe not, I haven't tried all of these, in other languages I've had some of these issues as well, but it can be a source of errors. What happens if, for example, you enqueue a function, you enqueue an operation that should be done later, and in this case you are sending the actual function that you want to execute, but in some other cases, in other libraries, you would send the text name of the function, and this would be problematic because if you run a deployment and you deploy to production and you have elements in the queue that need to be processed, then it's going to look for that function in your code database, and that function might have changed over time, so it could get you to have problems over there. And regarding security issues, here's a super simple way of encrypting things and just put that in the queue, and, yeah, this is using just Fairnet. Thank you. Awesome. So another technique is just using an API abstraction layer, right? This might be super They're obvious to some a bit, but as well there's just a lot of people with different levels of expertise. So what's the API abstraction layer? What is an abstraction, again, it's I usually run a talk inside my company which is called tech concepts for non-technical people, and the best way to define this and the best way to make them understand what is an abstraction is hiding the weirdness, right? We are trying to hide the weirdness from the external systems or the APIs or the SDKs that we are doing. In this case, we're going to hide things like authentication, throttling, encoding, encryption, things like that. We want to hide that, and we want the implementation to be as straightforward as possible for the one that is consuming that. This means wrapping our API calls or external SDKs with classes of our own, which make them more maintainable. It helps in many ways. One of the biggest ways in which it helps is having our system not tightly coupled with a particular implementation. If someday we want to switch an implementation or switch to another service, it's rather easy to do. And dependency injection is a key concept to achieve this. This is just a little example. You can see over there on the left a client, a not very well-implemented client, which just sources everything on its own. But on the right, you are injecting those properties into those clients. And you are even injecting that client into the consumer, so it makes it super flexible. What you would have to do is inherit from that client, if you want to use that in the future and heard from that client and add the specific behaviours for each of the functions that you want to implement. So it's easier to swap and test, we're going to talk about it later, and easier to maintain the code in the future. And this is what I was talking about, the left example, it's going to require a lot of changes if any end point or the base URL change, on the right you just feed that information to the clients and you just abstract those into inner functions, right? So that's about maintenance. So as everything that has issues, it's not a silver bullet, depends on the business, use cases and how often do we plan to change integrations, we might never need to change and integration on our system, and we might be preemptively optimising things which we are not going to need, right? So it's always about knowing the business and how it works. And finally, I'm going to talk about testing and observability, right? Which is very important. If I can't control external systems and they're particularly unreliable, I need to take a decision if I'm going to use them and how I'm going to use them. So external systems are not reliable. Some of them have SLA, some of them have better ways of guaranteeing this than others, but some don't. Besides coding defensively, which we've done, and helping our system to deal with the unexpected, I think we need to have better observability on these services. This happens through continuous integration processes, for example. We can cover some of those external APIs with integration tests, this we can add with monitoring using tools like air break, data talk, roll bar, and also this is another case, sometimes services are very sneaky, APIs change, and our external services might push changes onto us that might not be documented sometimes, that's a big problem, some of them have a very orderly process to change them. They provide the change logs, the version history, and backwards compatibility and a bunch of stuff, right? But some may not provide that, so this is also a driving factor when choosing an external service. A general rule of thumb is if you have an SDK, use it, because those are usually very well maintained and they are very well documented, And you can pull just the new version easily through your CI process. And as a good local practice, every integration you make should have models or type hinting. You can use PyDantic, you can use MyPy, and this way you know what to expect and just be better at dealing with errors. And one thing that I should say is that you should test on sandboxes when possible. If your service offers a sandbox, it's way better to just connect to that sandbox instead of testing. Look at that awesome sandbox on fire. Even the water is on fire in that sandbox. But what you would do is just create a specific client for the sandbox, as you can see the class sandbox client, which inherits from client, right? And then you have that ignite water function and then you can abstract that behaviour. Right? When you're testing, which is the part below, you can inject that client into your functions, right? Or you can inject that into another object and just test the behaviour of that. And what happens if there's no sandbox, right? I think mocking it is the best way, right? One particular thing with mocks that I might not agree completely with, and that's why I'm suggesting sandboxes is that they have to be maintained over time, and your continuous integration tool might be saying that everything is perfect, everything is working because your mock is working as you coded it, because tests need to be maintained as well, but the live API could be failing, and the only way to know that is to connect to a live service. So then you can see some useful tools over there. And yeah, as you can see there, we have created a local client which what it does is just exposes these fake customers variable. This is similar to a fixture. So disadvantages of testing, because everything has disadvantages. No, testing has no disadvantages. Maybe that they take some upfront time to set up, right? But would you rather spend that time debugging that later? And that's where the sleepless nights come, right? Would you want to do it in a control way or would you like to debug that at 2 a.m.? I'm not sure which I prefer. And also, adding systems, adding tests into a system that doesn't have them, it's going to be a bit hard and it's going to take a while, right? If you have a system, a legacy system in which you want to put some tests in, my approach usually is to code a lot of integration tests, right, and just cover the basic cases, and then start adding the lower level tests. But it's key to know that not every test is the same, and you can use several types of tests for your system. There's going to be awesome talks on tests, and there have been these days, awesome talks about testing through PyCon. So as final considerations, the first part is trust everything, right? Even your own code, because we are human. When you code your own APIs, usually you have to code all of the cases, all of the error cases, all of the different combinations of those error cases and the messages, so it's It's going to be hard to test. For that, property testing is a good fit there. Not everything we discuss today will apply to all systems. It's about knowing the business, it's about knowing where your business is going, where the direction is going, right? So you don't just implement things that you are not going to use. A good thing is make an inventory of your data, how often do you need to update it, and how critical it is to your system if that information would be outdated, right? And always make sure that your fallback mechanisms and ways to retry are there in the code. Also we haven't talked about throttling too much, but throttling is in essence just spacing out the connections you make to an external service. And this is helpful just because you don't want to cover all of your rates or your quotas the services. In general, I think this is more of a life advice. Follow the test pyramid and implement different kinds of tests, right? Based on your use cases. It doesn't make sense to just implement a test suite and bump up that test coverage number. Test coverage means nothing if you're testing everything that you shouldn't test. So that is kind of my suggested. So I think this is the questions part.

Speaker 2 [24:43]

That's an amazing presentation especially the slides so we have a couple of questions

Speaker 1 [24:47]

of questions, please.

Speaker 2 [24:48]

precisely two. So the first one is how did you create those nice graphics in the beginning?

Speaker 1 [24:54]

That's not me. I always get that. Just download a sprite sheet from the internet and spend sleepless nights working on the presentation.

Speaker 2 [25:04]

That's just amazing. And the second question is, why should we use Python task queue instead of async IOs, async await event loop for asynchronicities? I mean, there's so many asynchs that I got asynchs, so yeah.

Speaker 1 [25:20]

think so. So there's just a bunch of different methods that you can use, right? And again, I think the question is what do you want to achieve? If one is easier than the other to implement and to maintain over time, choose that one. If your team is small, you are not going to choose a big tool for that and a big complex thing because it's just a liability for you. So I think it's more about following common sense and being pragmatic about things, right? As a developer, I've been in that situation that I wanted to... We say in Spanish, reinventar el agua tibia, or inventar el agua tibia, which is invent warm water, right? It's there. You have to just use whatever tools you have for the job.

Speaker 2 [26:11]

There is a new question. In your stack, where do you see the central truth? Example, API specification, use cases.

Speaker 1 [26:19]

Sorry, I didn't get the first part.

Speaker 2 [26:21]

In your stack, where do you see the central truth?

Speaker 1 [26:25]

In my stack sorry

Speaker 2 [26:27]

Where do you see the central truth?

Speaker 1 [26:29]

The central truth. So I think the source of truth would be the remote system. If you're connecting and pulling data from there, that will be your source of truth. So caching, what it does is just bring that part of the truth. It might have changed in the remote, but you have to kind of give or take, right? What are you expecting? Are you expecting that data to be 100% there always? Or can it wait, right? or is it updated once every day with a batch job, then I think that's your source of truth, the remote one.

Speaker 2 [27:06]

think there are no more questions so let's so let's thank the speaker again