Tidy Finance in Practice: How Explicit Assumptions Avoid Bad Investment Strategies

Speaker 1 [00:18]

Hello and welcome back. We continue with the next talk.

Speaker 2 [00:26]

which is

Speaker 1 [00:27]

Tidy Finance in Practice, How Explicit Assumptions Avoid Bad Investment Strategies. Please give a warm welcome to Christoph.

Speaker 2 [00:43]

Hi everybody from my side. I'm very happy to be here. I'm not really sure how my talk fits into the general agenda. It's my first PyData conference. I'm a research fellow at Lancaster University and I also work for a private investment firm in Hamburg. and we look at all kinds of strategic investment opportunities and investment strategies. And some of the tools I'd like to present to you are part of my day-to-day life and I do a lot of my analysis in Python so I just thought to share a little bit of the pitfalls you have when you talk to investors that are not so mathematically trained as you are. so I thought to start with an investment proposal I have strategy one for you you can decide if you like it and I have strategy two for you maybe you like that strategy even more I give you a hundred dollars or a hundred euros and you can decide which strategy you want to put your money in and I'm not sure but maybe 99% of us would choose strategy two of course if you be invested from the early 2000s to the end you would have almost double your money in comparison to strategy one so you might have some more bumps on the way there but of course it looks so much better that's the first impression however, when I tell you that strategy two and strategy one are very highly correlated and the only difference between the two is that from strategy one, I deleted the three worst observations and set them to zero and then let the plot go ahead and see that these strategies are very, very similar. Out of those 26 years of returns, only three of those are different and all the other returns, monthly performance numbers are the same. And it's just the graph that makes it hard to see the difference. So what you can do is you can change the Y scaling and set up a linear scale. You can use a log scale and you would say, okay, well, they are both up trending, that's nice, but you can also see that they are now shifting in parallel. So if you would have invested in strategy two, and I don't know, after 2016, you would have get the same return if you would have invested in strategy one. So both of the strategies are not very different in most of the times, just of course in the extreme times and that makes such a big difference and for our perspective and perception of those two strategies it makes everything different. So it's just three rows of data that changes our whole impression of what we are seeing, actually. And that's a very common trend in finance, and I'm not sure how often you talk to your representative at your bank or how often you look at your Scalable account or Trade Republic account. Numbers can be misleading, and especially when they are compounding, And that's a problem in finance in very general terms. And me and some of my colleagues, we not only saw this effect in real life, we also saw that in research. We were trying to replicate some papers published in very good journals, Journal of Finance, very highly regarded journals, A-journals, and we couldn't replicate the code. So we thought, okay, Let's try to write down our experience. And that's how we started a project called Tidy Finance. It is a website and also a book that you can read. And what we are trying to do there is show the reader, professionals, students, or also professors, how you can reproduce a certain example, data set, or a certain strategy. And it is tidy in two ways. Of course, first of all, when you use finance data, it's usually structured data, but still we want to have tidy data, but we also want to have tidy code. So those two principles we try to apply on various topics in finance. And I'm going to show you today some of the examples that you see when you see backtests. What we call, you have a trading strategy idea and you want to test it in the past to see, oh, is that strategy actually a good idea? So we have a website that you can go to with many different chapters. And we have two books. One is an R. And then we realized, ah, many people are doing Python. Let's do all the analysis that we did in R also in Python. So now we have two books. And how are our chapters structured? Well, we combine code, results, and theory behind. So you see, mostly we start with some text, some real textbook introduction to, in this case, classical portfolio optimization. problem, then we give you some code how you can write up this formula and then we also give you some results. So all together all available to use at home so you don't have to figure it out yourself. And two of my colleagues are at university so they are using the book to teach master students, bachelor students about finance and in this connection with explaining the models the feedback is very nice. So what I want to talk about today is backtesting. Backtesting is, okay, you have an idea for I don't know, a model or trading and you want to apply it to real data and then you Google how can I like build this in Python for example and you find a lot of great libraries that give you very nice outputs and that do the aggregation of the information very well. So you have PyPortfolioOpt that gives you a lot of different portfolio strategies already implemented that you can apply and see, okay, what kind of risk do I want to use or can I withstand how much drawdown is okay for me? Then you have backtesting libraries that do all the calculations in the background, how you get your portfolio weights and how these portfolio weights are translated into actually performance time series. And then you have PyFolio and QuantStat that gives you these nice outputs that generate PDF files for you that you can just summarize the information for you. And we really like those tools, and I use them often. but I want to point out that these tools are just one side they take whatever input you gave it and then they give you a very nice output if the performance looks good or bad that is up to you to decide but those tools are not really talking about inputs and that's what I want to talk about what inputs do I use for my backtest for my model validation estimation procedure to figure out, oh, this is actually realistic or this is not very realistic. And then I looked into the PEP 20 and it says explicit is better than implicit. So what we try or what we are doing, and hopefully you cannot find any spot in the book where it's not like that, that we put every assumption, every parameter set that we use, we put it explicitly in there. And usually we also comment on it. So we try not to hide any parameter choice that might impact the total performance at the end. We really try to be explicit about that. So an example where it's not very explicit is this, for example. So you download some kind of library and it tells you, okay, there's a portfolio optimizer with two inputs, mu and s. Mu is the mean return. So just say, I don't know, maybe the arithmetic mean over the past, or it is some kind of factor model. We call it the capital asset pricing model. It's a very famous one. But what mean do you use? Maybe you can also use the logarithmic mean, or you can decide, do I use all the data from 2000 until now, or do I only use the data from 2010, because the 10 years before, they weren't really important to us. And all these kinds of decisions that you have to do beforehand are left to you, basically. The same goes with the matrix S. It's an N by N matrix for, let's say, you have N stocks you are looking at. When you have monthly data for over, I don't know, five years, it's, I don't know, 50 observation points, a little bit more. But when you look at, I don't know, 500 stocks, you can still calculate S, but in order to calculate the portfolio weights, you usually use the inverse of it. So you have this invertibility problem, this course of dimensionality, what the finance people call it. So you do some kind of shrinkage. But the backtesting tools I showed you, they are not really talking about shrinkage. So you have to think about that yourself beforehand. And with Tidy Finance, we're trying to give you all the good inputs and some good outputs. The next question is, okay, do I want weights that are just positive? That's the usual case that you do. But when you work for a big investor that can trade in, I don't know, derivatives or futures, for example, where you can also take short positions, is that in your optimization? Where do I put this in? What is the standard? was the default setting. I have to look at that. And also, what are my transaction costs? When I do the backtest, every trade you do has a market impact. I mean, the assumption of financial markets is that there is no impact of your single trade on the whole market prices. But of course, there are trading algorithms that are trying to optimize how much trading costs you use or you produce. So you really have to be thinking about that in your backtest how you model that. You cannot just say, okay, 1% is all the time, it's just I put into trading. When you want to buy or sell an NVIDIA stock, for example, that's not very costly, but if you want to sell or buy a very small stock that is not traded very often, the transaction costs are much higher. Another problem is overfitting. So, you can think about it as training and testing, and in the test set you put all the parameter choices in there and you do this big, big grid and then you don't really control for that in your test set. So you just go through all kinds of windows, for example, or threshold parameters that might use when you estimate your mean or your variance matrix or yeah and then you just append, append, append all your results or each one is a new trading strategy and then you just use the best one based on a certain criteria I don't know out of sample performance or what we use very often is the Sharpe ratio that's the return divided by the variance or so to say the risk-adjusted adjusted return. And then you plot the best performance. But is that really a good choice? Is this in-sample optimisation good when you do it out of simp, when you do your live trades? And maybe you have seen examples. I have seen tons of examples. What they are saying is I have never seen a bad backtest. So we really want to or I would like you to be aware of that, that this is the case for all of us. Right, so what is tidy finance about? It's about the input side. We want to be explicit how you get to a certain mean, how you, when you get a data set that is well known, for example, Pharma French factory data, how you use that, how you transform that into actual trading strategies. And we want to be explicit about how you do the sorting, for example, of your assets that you use in your portfolio, how you determine which one is a good fit for your risk aversion. There are so many choices, and all these choices are different for us, but we just want to give you one, I don't know, let's say a template maybe, so you can see that you have to think about this a lot. Okay, so one example is this chapter, and I can show you, this is from the website, it's called Constraint Optimization and Backtesting. And what we do is, well, we first load all the packages that we use, and you will see that most of them are very standard packages, so we try to program all the optimization functions by hand and of course that might be not very efficient in terms of computation time and that but for learning purpose that's perfect because you see what is happening and what are the assumptions, what is the mathematical model and this combination that I think is unique in the space where we operate with this book. So you get the returns you see, we use farmer French data here, those are standard industry return, and then we give you some more ideas about what do we try to optimise and you can see what we want to do is, well we want to minimise the portfolio variance, so the omegas are your portfolio weights and sigma is your variance-covariance matrix, what I called as before, and we have one assumption here, the weights, they sum up to one, yeah, cannot invest more or less than 100, and yeah, and we calculate those weights here, and you see you get some weights, and you see, well, when you add those up, they might be all one, but you see one, for example, here, down here, this is less than zero, so you might be, I only go long, so we have to put in some constraints at the later end. The first optimisation I showed you was the minimum variance portfolio, so I just want to minimise my risk. I don't really care about the returns, but you can also make this very famous formula from, I don't know, 1952, and people are still talking about it, or at least we are, and All the finance people I talk to talk about it is, okay, but yeah, I want to minimize my variance, but I also have some kind of assumption of how much return I want to have. I want at least 5% per annum or maybe 15% if I only go on equity and do that. And then you can write down a solution, how this portfolio would look like. And you see what I need here is the sigma inverse. I need those inverse of the variance-covariance matrix. And then we go into explaining this formula and explaining to you, okay, what are these inputs and how do we get good estimates for these inputs? Because as you can see here, the sigma has no head on it. So it's in theoretical optimal, but you have to estimate it from real data. And there the tricky part starts. So we give you an overview of the literature of all these estimations, and then we show you one example of how to do it, and we do it in two ways. We, on one side, try to tackle the estimation problem in the sigma inverse, so how to get a robust estimate that is also valid when the market dynamics change. So you have dynamic market participants, and maybe the participants today are AI-driven and much more volume than you had 20 years ago. So the dynamics and how asset prices are built are much different than they were in the past. So we want to get this estimation uncertainty, that is dynamic. And we also want to look at, okay, how do we model transaction costs? And how do we do that? We just use a penalty term on that. So the more you trade, the more the trade is penalized. and we do this through in this case a quadratic function of the weights in t plus 1 and t plus so that's the performance or the weight just after you have seen the return of t plus 1 but you are still in the position of the period before so you have to be very specific about that too and then you can rewrite the optimization problem in a transformed way where you have all these stars and there you can see this quadratic term it enters in here and then you can write it down again and you can see actually that you get now a penalty term here that is somewhat connected to a naive portfolio. When I have no information, I give you $100, you just put in each asset, you put the same fraction, for example. That would be one very naive, that's why we call it a naive solution. And there are papers out there that show that this is a very strong benchmark, hard to beat out of sample. And so we derive at those optimal weights and then we estimate those and calculate an efficient frontier. And this goes on and on and on and I don't want to dive in too deep. But going back to my slides, you see we put some parameters here. And what those parameters are we explain in the text. So the window length, for example, gives you, like, I do rolling window estimation, like, in this case, for 120 months, so for 10 years, I estimate by parameters, then I do a one-step-ahead forecast of the optimal portfolio, and then I shift my estimation period again, one period, and do the same thing again, and so I get what we say a pseudo-out-of-sample test for a certain trading strategy. The beta you have just seen, that's connected to the trading cost parameter. The penalty term and gamma is my risk aversion. So how much risk I'm willing to take. And then we give you this evaluate performance function where you just get out, okay, we get out the raw returns of my portfolio. We calculate the turnover. I also get the net return. So the return after cost. And then I put it all in an area. and here's the function that you can also see on the website how to compute my portfolio weights and you see we here define this objective function i've just shown you with some initial weights and then we give you some of of the constraints that we use yeah equality constraints that you always sum up to one, and then we use, in this case, the Skype minimize function with all the parameters you can see. So not just putting in one line of code says calculate efficient portfolio, we give you the whole thing where you can see all inputs and all parameter choices. And then we calculate the backtest, and we do this in a very old-fashioned sense, maybe you would say. It's not paralysable because we go from PEEF every period, so we do this out-of-sample backtest one period at a time, so when you read the book, you understand at each point in time what is going to happen. And we always define the returns I use, and then the next return is the return I use to evaluate my trading strategy. And that's why this minus one I have marked it, just to show you that it is important to have a difference between the evaluation data and the estimation data. And then you use this returns window to calculate your sigma and your mu, and then you calculate your w1, so these are your portfolio weights, and then you put it into a performance characteristics to get to that. So to go back to the beginning, I showed you some performance charts and now I show you some performance charts again. And it's about this minus one, this very crucial parameter. If you forget about that, you might end up with your performance chart that is the red line, but when you do it correctly, and you have the minus one in there, your actual performance is this blue line. And that's the look-ahead bias. You use past data to come up with your portfolio weights, and then you use returns that you have not used to calculate the weights. And the difficulty here is a thing in your mind that the return at a certain day t, you only observe it at the end of the day. So you don't see it at the start of the day, but at the end of the day. So that's why the slack is very important. You cannot use that day to calculate your performance. And when I do the same for the log scale again, because I like to look at backtests on the log scale, you see that the performance difference now is much more wider. So you really see there's a difference in the actual strategies. So it's not as parallel as in the beginning where I just changed three rows of observation. But now you can see it here much more. So we have this principle of very clear code in the whole book for different chapters, beta estimation, PharmaFrench replication, option pricing. So just look at it, and maybe you can find some nice things. So three things to remember. When you have applications, be explicit about your assumptions. When the assumptions are not in your code, they might be in your head, and your head might be here or there, or you never know. And when somebody shows you a backlist, don't ask about the Sharpe ratio, ask about the assumptions. and the look-ahead bias. Thank you.

Speaker 1 [25:50]

A real quick reminder, you can ask questions on talks.pycon.de as well as upvote existing questions, so they are more likely to be asked. We have already the first question.

Speaker 2 [26:05]

Oh

Speaker 1 [26:06]

Oliver Aguri, do you think it is scientifically surely possible to outperform a simple world ETF portfolio with optimization?

Speaker 2 [26:18]

This question is a theoretical question and of course that's not possible but when you look at your ETF positions it might be possible to outperform it but markets change dynamically so I think it's good to think about the Olympic principle sometimes to be there is the most important thing and what kind of strategy you use is not as important. So as a disclaimer, there are trading firms out there that use a lot of AI tools and a lot of market data, priority market data to get the edge, to get a trading position 51% of the times right. And that's enough for a certain volume to produce alpha performance. I work at an investment firm. My private portfolio is still just the ETF index, very simple. But of course, I would not put all eggs in one basket. I would, of course, try to use different asset classes. And the examples in our book are just equity data because that's the most available.

Speaker 1 [27:37]

Awesome. The next question is, who is the target audience of Tidy Finance?

Speaker 2 [27:44]

So we started with there are many papers a couple of years ago about the replication crisis and finance so the first audience was Academic audience you have for students who might not have a big background in programming to make it very easy and accessible for them to start coding and to understand the models because they get very technical very quickly and finance people have a very unique jargon, how they name parameters and things, so we wanted to make it easier for students to see the cool stuff that we care about. But in my daily workings I see that most of the tools I use for the students I can also use for some of our investors to explain to them what we are actually doing.

Speaker 1 [28:38]

The next question is, can the code, your library, account for current market conditions, world events in the portfolio?

Speaker 2 [28:48]

up

Speaker 1 [28:49]

of political or other nature.

Speaker 2 [28:51]

Yeah, so we are not coming up with a new trading strategy that is better than everything that's out there We're just showing to you. Okay. Those are the standard models that are used by mostly everybody in finance and you can check that with new data yourself if these models are stable robust and if the claims of your Banking advisor are really valid in very dynamic markets

Speaker 1 [29:20]

Is there a big difference between the R and Python implementation?

Speaker 2 [29:32]

So we did that three years ago, so we didn't have good LLMs that helped with that, so it was actually done by hand, I would say. We have to maintain both repos, so to say, but we really try to use as little packages that are available as possible just for the stuff that...