The secret sauce of data science management

Speaker 1 [00:03]

Thank you very much, super glad to see all of you here, very nice. When I first joined Intuit four and a half years ago as a data scientist, like probably some of you here are, I told my manager back then that someday I would like to become a data science manager. I was already a data scientist in the industry for five years, but I thought that someday it would be nice to try to use other parts of my brain as well. So fast forward four weeks, my manager come to me with three management books. Okay, Shir, you wanted to be a manager? I need you to become a manager now. You see the Intuit Israel site was growing, the 8i team. Intuit, for those of you who are not familiar, is a very big financial software company. In America, it has TurboTax and QuickBooks. It helps people with their taxes, with managing their small businesses, with marketing their small businesses, with their MailChimp as well. So it was growing the AI team in Israel, and he was the only leader, and he needed help with leadership, and that was my opportunity. So although I was four months pregnant at the time, I was the last person to say no to an opportunity and I took on the role and I won't lie to you my needs shook a little but I made a lot of effort I learned a lot I did courses I read books and I'm happy to say that we learned a lot and we were successful we were able to grow the team from two data scientists to 11 data scientists and really move the needle on into its fraud prevention as well as customer experiences with the AI models we developed in our team. And today I came to share with you some of the lessons we learned because I think that in this journey we were really able to crack some of the code of the secret sauce of the data science management. So whether you are a manager of data science or an aspiring manager or a data scientist and And whether you work in a startup or in a corporate like Induit, I'm sure there are some useful tips you can take from this talk to elevate your team to the next level. So we're going to talk today, first of all, about what does it mean to be a successful data science manager? And afterwards, we're going to talk about the three pillars of management and how we apply it in data science management. Managing down, managing across, and managing up. What does it mean when you're a data science manager and how to do it in the best way possible? This is a little bit about me. I started my journey in the Israeli Air Force as a pilot instructor for helicopters, Blackhawk and Super Stallion, where I trained pilots for emergencies, to operate emergencies on air. and later on I went to work in two startups a takadoo and blue vine takadoo helps save water in with to the world using AI and blue vine helps small businesses with their cash flow from there I went to lead a I went to be a data scientist in intuit and as I told you four weeks later I became a data science manager and then grew my team from two to eleven in these four years. In parallel to that, I also had three kids, Danny, Erden, and Guy. And I'm also very active in the data science and Python community. I founded the Israeli branch of PyData. Yeah. And I'm very, thank you, I'm very enthusiastic public speaker. I also founded the Israeli branch of Women in Data Science, and I'm the WIDs ambassador, and I have a podcast in Hebrew, it's called Unsupervised, it's about data science in Israel, and I participate in many conferences, and I'm very, very happy to be here today. Okay, so let's talk about data science management. Sorry, it's a reminder to take my girl from kindergarten. So what does a successful data science manager look like? I'm hopeful we can engage the crowd in this talk, so do you have any ideas? What does a successful data science manager look like? You can just say. Anyone? Come on, don't be shy. Anyone? Attentive. Okay. What else? Yeah. Great. Okay, keeping the team motivated. Yeah? Oh, very, very good. Very good, important. Able to communicate the technical things to the higher management, to other levers. Great. So I loved all of the things you said. And to me, to say when something is successful is to be able to measure it. So how do we measure a data science manager? So based on my experience, there are two things you can measure pretty well. one is how much impact your team brings to the business how much value you brings with your AI that you develop and the second is how successful and engaged is the team and the question is how do we do with this where do I spend my time as a manager we mentioned there are three pillars we have managing up down and across where do I need to spend my time this is a question that I used to ask a lot in the beginning as a manager and well the answer is it depends it changes you have to be adapted you have to shift efforts according to demand and you have to step in when where you see there's a gap and put your efforts there and the most important thing and it was also mentioned in the previous talk you always have to be with the growth mindset. And if you didn't read the book by Carol Dweck named Mindset, I really recommend you to read. It talks about how anything that you go through, any feedback you receive, you can use for your own good. You can use it to grow. You can use it to improve. And besides yourself, as a manager, you have the perspective. You see the entire team and how it behaves. Always be with the growth mindset, and think, what can we do to get better? So let's start with managing down. Here are a few of the basics we need to start with management. Let's start with the challenge. I want you all to think with me, what should we do in this situation? Dana received a request to build a machine learning model that would help more customers make their transaction through our product. Dana is not making progress as you would expect, as their manager. What would you do? Anyone has any idea what would you do in this kind of situation, your employees not making enough progress? Yeah? Ask her what's the problem. OK, great. Start with asking her what's her problem. What's the problem? Yeah? Ask her how we can help her. Yeah. Ask her how we can help. Great. . Go through her current progress. Great. Anything else? Yeah? Do we really know what is progress or what ? That's a really, really great question. And that kind of brings me to my answer. The question was, do we really know what is progress? What do we expect? OK, so this is one of the first things my manager told me about being a manager. He told me, Shil, when you're a manager, you have to give your team members three things. expectations, training, resources. So this really connects to what? What's your name? Rafael. Rafael. So it really connects to me to what Rafael said because the first thing is pointing a finger to myself and asking, is the expectation clear? Do I set correctly? What do I want? What do I expect from Dana to achieve? So these are the three questions I ask myself in this kind of situation. Are the expectations clear? does Dana have the training to achieve this expectation and that you have the resources required to achieve this expectation and once I figure out all of these things with myself and maybe with a conversation with Dana I need to adapt my own management style this is very important every person at every situation needs to be handled differently by their own manager so for each situation I ask myself where is that person now if a person is just starting to do a task or a goal and it's doing it for the first time I need to adapt myself I need to be more directive maybe give more a specific mentorship to that person but if this person has already some experience maybe I need to provide a little bit less directions but more support if the person is more of a mastering a task or a goal, then I need to be more of a coach and asking questions to unlock insights and provide stretch opportunities to that person. So it's very important to adapt yourself as a manager. But what if the person, I did all that, I gave the expectation, they figure out the training, the resources, I adapted my management style. But what if all that doesn't work and still the person is not performing as expected. So the next stage would be to give feedback. And I have a mentee not in my company, she's also a manager in another company. And we talked about giving feedback to employees. And she told me, Shir, I don't like giving feedback to employees. I'm really afraid to do so. I'm afraid that it will make them not engaged, that they will feel unmotivated, that they wouldn't like me as a manager and I told her what are you crazy feedback is the best gift you can give your employee and I remind you again what we talked about in the beginning the growth mindset right feedback is an opportunity to improve to be better so we as manager we have to use it to make our team be better but we cannot just come to people and give them constructive feedback on air we need to build a relationship with the person. We need to get to know them really well and show them that we really care about them and that we want them to develop. Then we can make feedback really count and really bring the people to the next level. So now that we're able to give someone a feedback, how do we do it? So we have a very good model called SBI feedback, situation behavior impact so in our situation I would come to Dana and I tell her Dana in this kind of situation when you are on a project and you're not making enough progress and your behavior is that you don't share with me your challenges or not tell me that you're not making enough progress. The impact is that we do not deliver on time. Our management, our partners are not happy with us and they lose the trust in us. So let's work on what can we do in the future so it won't happen. Maybe we will have a certain milestone in which you share your progress. Let's work together on a plan to avoid these kind of situations in the future. So this is an example to how you would give SPI feedback. And another very important point about feedback is that you don't only use it to give constructive feedback, but you also use it to give appreciation and to amplify good behaviors in the team. I use it very, very often with my team when someone does something very good for the team. Like, for example, someone builds a tool that everyone can use for better monitoring machine learning models in production. I tell everyone in the team publicly I give the feedback that, wow, this person built this tool for us. This is a great example of winning together, of bringing contribution to the team that takes our engineering excellence to the next level. So by sharing this good behavior publicly, I show everyone in the team the example of the good behaviors we would like to amplify. So don't forget, use feedback also to amplify good behaviors. Also, when someone is struggling and doing small steps in the right direction, it's also good to give them positive feedback in that situation in order to get them going in this good direction they started. So that was the basics of the basics. Now let's go up one level. And now that we have the basics covered, how do we help data scientists grow so of course feedback is an important tool for that but also other tools we need to remember that data scientists are top talent right these people have probably have masters or phd or learned a lot by their own they're regular to be independent and perform research on their own and they want to be creative so we need to empower them we need to make sure that as managers we give them end-to-end ownership on the tasks they deliver on their models we need to hold them accountable for the results and and we also need to provide them with choice if it's possible you know give a few ideas of projects and let them choose or even let them come up with their own projects also with the directions of research we need to let them choose on their own which will keep them more motivated and something also very important to help people grow is to get a to define goals together short-term and long long-term goals and I would elaborate in a minute and also we need to make sure we provide data scientists with space for research in order for them to be to be able to be creative and innovative and and we need to find the right balance between research and delivery. So, how do we define goals with people? What we do is that each person define their own goals together with the manager, with the help of the manager, and we meet every month. We have two types of goals. Input goals, which are about deliverable for the business, and development goals, which is about personal development and growth and how do we set good goals we need to have goals that define what do we want to accomplish and we need to align it with the business goals when it's an input goal we need to have a measurable metric for that goal so we would know when when you achieve that goal how you were successful and we need to have some kind of a deadline or a milestone for that goal so we will know when to check that we achieved it and beside the monthly check-in also in the end of the year when we want to evaluate the performance of workers we use that goals that they set for themselves and the progress according to the goal to measure the performance and something more that we do is to we have a data science rubrics for the team data science rubrics is by criteria that we defined in our group. For each level, we define what do we expect for this level to operate. For example, if you're a junior data scientist, we expect you for X, Y, and Z. But if you're a senior, we expect you for W and V. Okay, so it's very important also in relation to the previous talk that was here about diversity. If you have a defined criteria of what you expect from the different level of data scientists in a team, it allows you to be more exclusive and to remove the bias from how you evaluate your team members. It also allows you to remove the bias when you hire people if you define the criteria by which you hire the people and not just say, okay, I like this person, he looks nice, I will take him, okay? So I really encourage you to work with your team or with your company to understand what do you expect from the data scientist in different level in respect to what is important in your company. Something more we do is career conversation with long-term goal. Every six months, I talk with my team members about what do they want to do in five years from now. I send them these five questions in advance so they get to think about it, about what they want to do. And then we have this conversation, and then I talk with them about what are the gaps and what are the things they need to work on in order to achieve their long-term career goal. We can come up with a plan together of the certain steps they need to follow in order to get there. And the last thing about growing data scientist is about balancing between research and delivery. So some ways we found are good to allow for more research and innovation to grow in our team is doing periodical hackathons in which we stop all other works and all meetings and just code in combined groups between the team. And this gives room for a lot of innovation and a lot of our great projects started from these kind of hackathons of a few days just coding and doing ideas from the team. And we also have an ongoing long-term research track in which x percent of the time we invest in developing a new technology or addressing new problems we don't usually address. And to help data scientists not to get lost inside all this research and all of these projects, I encourage you to work in small teams, have plans to what are you going to do, define the research period, when does it start, when does it end, and what is the plan, and also define standard sharing milestones and the review milestones for each product. When do we do brainstorming, when do we do model review, design review, peer code review, all of these time points when data scientists are obliged to come with something to the table and present something, help them not to get lost and make progress. OK, so this was about the individual level. What about the team level? How do we build a high-performance culture in our team? So first of all, we have to have diverse skill sets and experience level and encourage sharing and learning. Something more that encourages belonging in the team is to define what is the identity of the team and even come up with a name for the team. For example, my team's name is Oz, the data wizards of Oz. And you can see here the small logo we have here. We even have a sticker for the laptop. I don't know if you can see it here, but it's on my laptop. And we define together the vision and the goals for the team. So this is very helpful for the team to be motivated to know what team do they belong. And they are also well known in the organization. This is the Oz team and this is their delivery. so it must be very high quality. And also, we have all kinds of ceremonies and standards we have in the team, like the ones written here. So I encourage you to also think what should be your team standards, what are the ceremonies you do with your team in order to make sure you deliver things in the highest standards, like what are your monitoring requirements, what is your project's playbook, what are the metrics you want to measure about engineering excellence and so a little bit about our weekly team meetings something i always try to do as a manager is to be very transparent with the team and from my own share my own lessons and even failures with the team and what i've learned this encouraged the team to have psychological safety in those team meetings and really bring them to the table and let them be able to share from their own learnings and perspective. We have an open agenda in the team meeting. We have a Google Doc, and each team member can add their own items to the agenda. So of course, they are shy to do so in the beginning. So what I do is in the one-on-ones I have with them on a weekly basis, if I see something interesting they have learned and they have done or they have some challenge they need, some ideas about, I encourage them to add it to the agenda, or I even add it for them to the agenda, and then we use the team meeting to consult to each other, to learn from each other, and to share the best practices and learnings. And also something very cool and interesting we do in the team meetings, it's called Rainbows and Butterfly. And what it means is a ceremony that in the beginning of the team meetings, each person can say thank you to another person from the team about something nice or good that this person did for him or for the team. So this starts the team meeting with a really great atmosphere and it's also a great opportunity to amplify good behaviors within the team. Something more that we do within the team is the quarterly retrospective planning. So this is a ceremony in which each of the projects in the team define for the next quarter the from-to goals from this project. For example, we want to move this business metrics from X to Y in this quarter. This is what we evaluate we would achieve. And the different team members present the goals to the project to each other. It is an opportunity for them to learn from one another and also ask questions and advise and contribute. And it is an opportunity for us as a team to do prioritisation by impact, because we know for each project what do we expect from that project to happen, what do we expect to achieve, so we can put more resources in projects that we know that would be more impactful by the from-to goals we have. This is data-driven decision-making, which is very important to be inclusive and to have the right decisions. And after we have this prioritised plan we build as a team, we share it with the partners in the company and also with the leadership. So we create an aligned prioritised roadmap, which is very important for transparency and to achieve the support and resources we need from our partners and leadership. So this all was about managing down. And what about managing across? So I want to ask you, once again, when you are working with people in your company which are not data scientists, like engineers or product managers or analysts or different other managers in the company, what are the challenges that you have? So can someone say? Yeah. Oh, that's a good one. Yeah, expectation management. Yeah, okay, what else? Yeah? What? The X and Y problem. X and... Oh, that you don't understand what problem you're solving. Right, okay, yeah, that's a very good problem, yeah. Yes? Oh, yeah, that's very good. Yeah. Figuring out the responsibilities. Perfect. What else? Yeah. Oh, very good. Understanding that their priorities are not necessarily our priorities. What else? Yeah. Oh, we're taken as the technical guys and can give our opinions on other things because we know only about the technical stuff. Okay, these are all great things. And I felt all of these too, so I understand you guys. And I made some cartoons to show a little bit about some of the regular conversations we have with people around us. This, for example, is the data scientist and this is product manager. So the product manager would usually ask, Why does it take so long, right? And they ask, why do you need my help? Just do it. And why don't you just retrain your model? Or I want 99% accuracy. And the data scientist might feel, but well, what the data is so noisy and it's not relevant to solve this problem at all. Or what is the relevant business metric we want to measure? Or what's the problem we're trying to solve, right? Someone said this before. So all of this can happen, and all of this probably happened to some of you. So here, let's dive into one of those challenges. For example, we have a PM that we are working with, and he wants us to deliver a solution with 75% precision. But due to the limitation of our data, we can actually only achieve 50% precision. So what can we do in this situation? Any ideas from the crowd? yeah okay challenge but what else but challenge it yeah oh great idea change the scope somehow yeah okay suggest to iterate okay okay yeah Yeah, yeah, try to start with discovering what is possible with this problem, right? Yeah? Very good, try to understand whether precision is even the right metric. Okay, so let's continue. In this situation, well, this is kind of a crucial conversation, right? Because we are only able to give something to the product manager and he's not happy because he wants something else from us. So in this kind of crucial conversation, I really recommend you to read the book, Crucial Conversation. It's really good. It talks, it describes how to behave in this kind of conversation when your palms get sweaty. Conversation when you know it's going to be hard and how to approach them. So this is just a side recommendation. And besides that, I encourage you to, first of all, listen. Okay, what does the PM really want, right? Is this even the right problem we are solving? Or is this the right metric we are aiming for, precision? Maybe something else can be achieved in a different kind of solution. And also, try to understand what's really possible and maybe demonstrate to the product manager Well, maybe you have some kind of a benchmark you can show from a similar problem. Well, look, this is only what can be achieved in this type of problems, okay? So you can't ask me to go that high or provide some kind of upper bounds, which is some kind of a mathematical proof. And also, maybe we can try to ask for more data. Maybe it can help us achieve better results. So this is what we can do at the point of time of the conflict. But what can we do to proactively avoid a conflict like this with partners? So what we did is we tried to talk with our partners about how do we work together. We tried to, as you can see, we tried to achieve for world peace between the AI team and all the partners teams. and by talking we are investing in creating trust and alignment and this is a link to a blog post I wrote about it that I can share with you later and what does it mean so basically what we did is we sat down me and my partners the product manager the leading analyst the leading engineer all of the management team and talked about how do we work together and then we wrote together a document called the project playbook and the project playbook basically defines what do we do when we deliver AI products to production how do we start how do we finish what goes on in the middle what are the different stages who contribute and who needs to approve each stage and how how long do you do we expect each stage each stage stage to take and so we wrote it together, and we defined that the success metric for these kind of documents is to achieve zero conflicts with our partners, to solve all of those problems we talked about that we have. So I encourage you to try to sit down with your partners and talk about, not about the specific project, but in general, how do we want to work together and how do we see our joint work, clarify the expectations, as someone said before. So here is some of the things we discussed and that are included in this playbook. First, we defined project prerequisites. What do we need to even start working or start exploring an AI project? We need to define a customer problem and we need to make it clear for everyone to make sure we're solving the right problem and not something else. And we need to understand what is the ideal state. and we need for this problem to be significant enough to even invest data scientists because as you know data scientists machine learning engineers all of these people are very expensive resources we need to make sure we're investing these resources in something that really counts for the company we need to make sure that we have enough available relevant data to solve the problem and that we have committed partners people that are working with us on the integration and the development so we won't develop something that won't even get to be called by anyone and we need to make sure that a simple solution was already tested because we don't want to go and solve a problem that can be solved with just a basic business rule right so what do we need all the efforts for so these are our project prerequisites and for the rest of the playbook We wrote it, guided by D4D. It's a principle that we use at Intuit that represents how we innovate at our company. So first of all, we have deep customer empathy. We start with really deep understanding of the customer problem by doing follow me homes, which is a ceremony in which we talk with the customer and really understand their problem deeply to be able to define it correctly. We go broad and then go narrow. We try to look for many, many ideas to how to solve the problem and then narrow it down. And we do rapid experimentation. We don't go and develop something many, many months and then go out to the sun. But every few weeks, we iterate what we already have to get some kind of feedback and see that we are in the right direction. So in the playbook, we have different stages. The very first stage is to identify the mission team to deliver the project because data scientists are not enough. We need to have engineers and product developers and product managers and analysts. And we need to start meeting. We need to put in the calendar a weekly meeting for this project in which we discuss the problem, what we are solving, the progress, the plans. And we keep on aligning on these plans. In this mission team, we write together the customer problem and the ideal state. And this is the format of the customer problem we use at Intuit, which basically says, okay, what is the customer problem we're solving? What is the benefit we're trying to achieve? And what kind of improvement would it bring to the business? What business metric we will move and how much will we move it? So this is all very, very important also to align the expectations, to understand these things from the beginning. We need to understand the business metrics and the suitable business metrics that we're aiming to move. And then we need to define some data exploration period that would help us, that would give us time to explore the data, to actually understand, do we have our prerequisites? is it enough time there is there enough relevant and sufficient data to solve the problem and what is the potential that we can achieve so if we go back to our challenge from before with the demanding PM we need to from the beginning say our potential where can we even get to so you know in order for later not to have the PM say okay but I want 75% precision but this is not even possible so from the beginning do an exploration and say okay this is the highest we think we can get. And after we have some initial exploration and research, we align on a roadmap for the solution to be delivered and we keep reiterating the solution as we said with the product manager to get a feedback. So here are some more tips on working with partners, very important for any data scientist or for any manager first of all we want to listen we want to invest time in building relationship as a manager I meet with all the manager of my partner teams to have a direct channel to to them to get their feedback on the on the progress of their team on the behavior of the team if they are happy on working with us and if they see some challenges that they are not happy about. We also want to make time to learn the business to really help us not being just the technical guys and someone said here but people that actually understand the business and be able to contribute to it with our data and insights. We want to understand the priorities and challenges and that will help us help the business and we want to define the projects from our partners eyes and not just from the data science side and by their metrics by the business metrics that they look at. Next thing is to have transparency at all stages so as we said we have recurring project meetings and for these meetings we also send meeting summaries if there are important action items for everyone to know about it's very easy that everything is transparent in the meeting summaries and And we use those meetings to have constant alignment on the roadmap. We always look at the roadmap so everyone will know when do we expect things to come out. And the last thing is that we keep on working to improve our collaboration by doing retrospective sessions with our partners to learn. And we update our project playbook as we go because there's always things we can add. give credit to other people on their work when the credit is due. We have this mechanism in Intuit called Spotlight of Recognition in which you can recognize the contribution of someone to a project. So I encourage you to think about ways to give credit to your partners on their contribution. This will help you speed up the process of delivering your AI projects by giving credit to people. I encourage you also to work on the business long-term vision and see how the insights you have from the perspective or data science to give to the business a long-term strategy. And I also encourage you to invest in educating your partners about what is AI and how does our work look like. Bring them more into the details So they can understand your side of the problem as well and also do some fun stuff together That's always helpful So two more important things about the playbook and there's it's a big document I can share a later also a the blogger wrote about it and Before we go to production We have a model sign-off process in which we share with all the stakeholders that the model relates to them and the results from the silent release of the model or from the A-B test so everyone would know exactly what the model is going to do before it's turned on. We share with everyone the dashboards, the business dashboards as well as the performance dashboards with all the metrics and we create all the relevant post-production communication channels. For example, we have a channel in Slack to ask questions about the model if people are not sure about the model results. And we define some kind of incident management process for the model. And after going to production, of course, we need to have monitoring with all the dashboards and alerts. We have to look at business as well as engineering metrics in this monitoring. we have to define what are the conditions in which we retrain the model because otherwise the PM can come after a few months and tell us well why don't you just retrain the model now and I will just say no but it wouldn't give anything so we need to think together in which condition we do a retrain maybe below some threshold or maybe in some condition where the data has changed drastically so I really encourage you to think about all of these things and define them in advance once you deliver the models. You might also consider to build an auto-retrain pipeline if this is a very important model and the data changes a lot. I also encourage you to do a retrospective after delivery to understand what can we do better for the next time and continue meeting about this project even if the model is already in production. It's a very important model now, right? It's working, it's answering some questions, is solving some use case we need to see it's still on track even months later so continue to meet and monitor the model together and see that everyone is happy with it so the last part and don't worry it will be very short just one slide is about managing up to our leadership we talked about managing down managing across and now managing up what does it even mean So, first of all, managing up, the first thing about it is to understand how can you help your manager succeed. Because as a manager, this should be your primary goal. You need to make sure your manager is successful, your manager is happy and well appreciated. Because it all comes down back to you. If your manager is successful, then you are successful. Your manager gets resources and interesting projects, you get these things. And very important to define and share the vision and scope with your team. It will help your manager represent you with the higher leadership and give you everything that you need. And to align the work your team does to the company goals. So everyone would know what does this AI really contribute to our company. It's very helpful to communicate the team plans and also communicate the achievements we did in the past. We use the quarterly planning sessions to do that. We invite all the partners but also the leadership so they would see our plans and they would also see all of our achievements. And we as managers also need to find opportunities to share the team works and let the team members share their work on their own. It's very important so everyone would know all the great stuff that we are doing. and something more that I do and many people are very surprised by this I collect feedback actively I have a Google Doc in which I just copy paste or write down when people give me or my team feedback and I share this Google Doc with my manager once a month so my manager would know how with people in what does people in my company think about my team and about me and a how How well are we doing? This is very helpful for the manager to see that, think about it. And all of this basically is an investment in the future of your team. Making your manager happy, making your manager see that you're on track and that you're delivering good things for the company will help you get more resources, get recs, and get new projects which are impactful and interesting and aligned with the vision of your team. So to summarize the data science managers role is basically to create the platform for the team to succeed and grow and we do that by three things managing down and encouraging growth innovation and building a a high-performance culture, by managing across, creating trust and alignment with our partners, creating the right processes for the project to succeed, and managing up, sharing and communicating with our manager the vision and achievements of the team, and by that, investing in the future of the team. So thank you very much, and I will take questions now.

Speaker 2 [44:32]

Thanks a lot, Shir. So the most upvoted question is actually a lot of people took pictures of your slides. Would you mind sharing them somewhere?

Speaker 1 [44:43]

Sure, yeah, I can share the slides somewhere. Maybe PyCon has some kind of platform for that.

Speaker 2 [44:50]

I'll have a it as a question. I guess you can update it in the description So I think you can find it then in the description just off this talk. Yeah

Speaker 1 [44:58]

Yeah, and and I guess there will be some kind of video right shared in YouTube So you can also use it from there, but I'm happy to share the slides Also, you can you can email me if you want and I can share the slides with you

Speaker 2 [45:12]

So do you think technical knowledge is inevitable for a good data science manager?

Speaker 1 [45:17]

Oh, that's a great question. I have so many discussions about this with people. And I think it kind of depends on which level of management you're talking about. If you're a first line manager, and you manage a team of data scientists directly and not manager of data scientists, I think that technical knowledge is crucial. but with that I think you don't have to be the most experienced data scientist in your team to be a good data science manager. You need to be a good enough data scientist to understand the work, to understand the challenges and of course you need to be a good manager and invest in all the things we talked about. So it's a kind of combination and as you go up the leadership ladder So the details, of course, of the technical parts matter less.

Speaker 2 [46:10]

So the next question is, how do you make sure research track time is not used as buffer that is always used for origin tasks?

Speaker 1 [46:20]

that's a great question and actually how we implement the research track is not always by five percent or ten percent of the ongoing tasks it is sometimes like a quarter that someone takes to research something after for example after a big delivery someone would take a quarter now to just research a new direction so this is also you know a more effective way to deliver it And something else, if you do use it as a 10% or 50% track, then you can make sure you just block someday in your calendar to do some concentrated research work. So that's a tip I always give my people. But it's basically a question of discipline, how much you enforce yourself to really apply it.

Speaker 2 [47:12]

What do you think should be the policy for firing people for example Netflix has the mentality that good performance gives you a nice leaf bonus

Speaker 1 [47:21]

I didn't understand the question

Speaker 2 [47:24]

To be honest, I'm sure I can repeat it

Speaker 1 [47:28]

Should the mentality for holding someone in the team, let's think strictly on a performance base. Be of, if you are, we keep going on the team, only if you are very, very good performance, so the example from Netflix, and then low to medium performance should leave the team, and then you would enforcely make them leave the team? or should you just keep low-performing the team until they leave? Well, I think that in general, I didn't have the pleasure yet to fire people, fortunately. But I think that you don't want low-performance performers in your team because they affect badly other team members because if someone in your team sees that someone else can not do much and stay in your team, then it reflects on them they and and also they might be dependent of this person works so in general you don't want low performance people in your team and if someone gets to the point when he's low performing then or he she's low performing then it's the obligation of the manager to communicate the expectations to this person and reflect to them you know their current situation give them feedback and give them consistent feedback a few times and document the process. And if after, you know, X times of consistent feedback, there is no improvement, then, yeah, this person should be fired because, you know, company should, in the end, should produce some results. You can't just pay people for being low performance. Did that answer your question? How much should that be? So, sorry, how much should this define low performance for you? For how long and, like, for how much, for six months? Usually, most of the companies that I know of have something called PIP, that is Performance Improvement Plan, which I think is about three months or four months that someone has to be communicated, then it's in the situation, and that gets an opportunity to get improved. but even before you get into the peep situation your manager should be transparent enough to give you the feedback and to help you improve and I think that if a person really wants to improve and really suitable for a certain role then he or she will approve but maybe this person is just not suitable and could find himself better in a another position or another company so maybe it would be better for for him or her to to to leave if if they're not able to perform Thank you.

Speaker 2 [50:19]

Thanks. I think we have time for three more questions maybe. So what book influenced your thinking the most on this topic?

Speaker 1 [50:26]

Oh, that's a great question, because as I mentioned, I really love books. I read a lot of them. I think the most basic one is the effective manager. So if you're starting to be a manager, I really recommend you to read that one. And another one that was also, you know, mind blowing for me is multipliers, which talks about how we as manager and we in general as leaders of anything in companies need to not always not be the smartest person in the room, but help others be smarter. Because together, when we are all working together, we can reach much further and not just make ourselves be the best on account, on the expense of other people.

Speaker 2 [51:15]

What's the biggest mistake you made when you started out as a manager?

Speaker 1 [51:22]

Well, I did many mistakes. One mistake we did was actually to take on a project. I was not a manager then. I was just starting leading, being a data science lead, but afterwards I transitioned to a manager with this project. And we started working on the project with a company. It wasn't clear. It was an anomaly detection work in security, and it wasn't clear who would actually use the result of the anomaly detection system. And I think maybe some of you are also familiar with this kind of system. You know, someone comes to you and says, oh, we have a lot of data. Please do some anomaly detection from it because maybe there would be some security breach or something like that. But no one really has the tools or maybe the times to actually consume the alerts this system brings out. So my lesson from that is to make sure that if you write some system someone uses the output it gets out

Speaker 2 [52:28]

Thanks. One last question. So I think it's easy and hard to answer. Is there a tension between time spent developing goals, making plans and reviews, et cetera, and time spent doing applied data science work?

Speaker 1 [52:42]

I probably mean as a manager, right?

Speaker 2 [52:44]

right? I guess so, yeah.

Speaker 1 [52:46]

Yeah, well, of course. And I remember myself in the first two years being a manager always, you know, with the tension, with the boundaries of how much hands-on work should I do and how much would I invest in management. And I think it really depends on where you work and how big is your team, because when you work in a corporate and you have a lot of, you know, bureaucracy and a lot of management stuff that you need to do, then you have less and less time for coding. So as the team grew, I think in about seven people, I understood that I can't be much hands-on and the best use of my time would be to invest in more of long-term planning, in developing my team members, because this would be the most effective investment for me as a manager. And I found myself doing hands-on work mainly in hackathons, which I mentioned are very, very important for me.

Speaker 2 [53:40]

Great. Thank you very much. Let's give her one round of applause. Thank you.