Trojan Source Malware - Can we trust open-source anymore?

Speaker 1 [00:04]

And our next speaker is actually me, so hello. Yeah, I feel very shy when someone introduces me, so that's why I will introduce myself in a bit. So this talk is about the children's software malware. So I don't know how many of you have heard about this children's source, or maybe you only know about the children's horse. But anyway, so yeah, here are my contact details as well, So feel free to chat with me. I really love chatting, as you can see. When I'm all in about, I love talking to people. So let's get started. So before I introduce myself, I want to, I know that some of you, or maybe many of you, are very good Python programmers. So can you please look at this and see what's wrong? Anybody found out what's wrong and want to tell me? Or anybody think there's nothing wrong? Hey, be confident, be confident. Yes, actually, I can show you that actually there's nothing wrong here. It was a trick question, and sorry about that. I tricked all of you. Yeah, I'm not a very nice person, but no, I'm joking. So, let me zoom in a little bit so you can see this wonderful interpreter. Oh, maybe it's too big. Sorry about that. Okay. So, this will do. So, I would paste all the code here and run it, right? So, what should print out? No. Hello, world! Yes! Nothing's wrong. Good. All of you are correct. Who put their hands up? But I have another question for all of you. Anything wrong with this one? Anything wrong? Who think nothing's wrong? You're tricking me again fool me once shame on you. We try Yeah, you you well you probably have read the paper But anyway, I would just show you do it to make a point So I would just copy and paste the code there Um, so, you know and run it and it said that all invalid character in identifier So you probably think that yeah, I'm doing something fishy here, right? So So exactly something to do with Unicode, but what? I will give you the details. But anyway, before that, I promised that I would introduce myself. So this is me, the cartoon version of me, actually. This is my full name. I also have another licensing talk about names before, but if you want to chat about Asian names and why they're funny, I'm here. But anyway, I love open source projects. I have created some in the past. It's not very popular. Nobody knows about them. But I also work for an open source software right now. So it's terminusdb. It's an open source graph database. I won't talk too much here, because this talk is not about graph database. But if you're interested, talk to me afterwards. So it's open source. I love open source. And I also love the community. That's why I'm here. That's why I organized some event as well. I also organized EuroPython this year. I will tell you more at the end of the talk. Stay tuned. But anyway, there's a lot of events that I try to organise. Last year more online, this year we go back in person, which is great. Also, I stream a little bit in the past, during the pandemic, but not so much right now, so don't be disappointed. So, we all love Unicode! Yay! Especially this mug! Because it's a duality of reality, because it could be a heart and it could be a poop, you never know, right? So who knows? Yeah, and also my friend loves Unicode as well. So this is my friend Mark. Some of you may know him. He's also a very speaker that before the pandemic, you would find him everywhere. He makes this very interesting lightning talk about importing pandas as panda emoji, because he also maintains pandas. So that's why he loved making fun of it. But actually, Unicode is more than that. it's not just like for fun, it's not just for lightning talk, you know, it's also for handling some other languages, so if you don't know, I am actually, I grew up in Hong Kong, my first language is Cantonese, which is, you know, written is the same as Chinese, so you can say that my mother's tongue is Chinese, so it's very funny, because you can't have, you know, the Chinese character, I'm sure that you know that it's not something like the alphabets, right, so you can't use ASCII code for it, it won't work. So we need Unicode. And also maybe some functional characters to assist it. For example, telling that actually, oh, this character takes a little bit more than just, you know, a byte for it or something like that. So in early days of the internet, in the 90s, when I was still very young and starting to know what is an internet, so some Some of the pre-, I remember still using something called Netscape, it's like a very old browser, so it's, you know, sometimes the coding is very weird. The web page showed to me is funny characters, and because the coding was not right, it's actually some Chinese website, so I have to change into Unicode so I can, oh, I can read, so that's what I knew about Unicode before, you know, when I was not a developer, I was just a kid. So the most popular version of Unicode, or most popular, you know, setting is UTF-8. There's other UTF-16, those, but there's also other that's not UTF as well, so I think the most popular one for Chinese right now is, you know, there's two types of Chinese, right? There's traditional and simplified, and they're both different, so funny. Anyway, emoji. We all love emoji, right? Emoji is, well, actually it's older than in 2014, but the most common one that we saw on our phone, on our chat, on the internet is actually added in 2014, so you can have a thumbs-up, you can give your friend a thumbs-up, or now there's a lot of different emojis, they said there's an Italian hands emoji, but I don't know, I never used that one. So one thing that Unicode handled very well is language that goes in the other direction. So what do I mean is that because language around the world is not always read from left to right. Some languages actually read from right to left. For example, Arabic or Hebrew, they are in the other direction. So how can we deal with it? Because when a computer sees a text, they will just print out the character by character from left to right. So that's why they have this special thing for Unicode to deal with these different languages, is to have a special character that is hidden, it's not visible for a human eye, it's just not visible when you display it on the screen, right? And when the computer kind of reads the text and sees that, it's like, oh, actually it's supposed to be right to left, so it will flip the direction of how they display the characters. So yeah, the flow of the text will be reversed. So it will just make the whole thing very interesting if you use it in a code. So yeah, well, what happens if you use it in a code? Because coding, I mean, most of the time, we are still coding using English, at least the English alphabet to code. So yeah, it's left to right is the standard. and what we understand. But if we put this bi-directional character in the code, also because Python also accepts Unicode, that's why we can import pandas as panda emoji, because we take Unicode in Python. So when you have it in the code, so when the interpreter see it, there are things that would be unexpected when you read things with your eyes from left to right. because the interpreter will read it in the other direction if you have this character. So, that's why there's Trojan Horse malware. So, it's named after, obviously, the Trojan Horse, which is a, I think it's a mythology, I don't think it's real history, but correct me, I'm not a historian, that there used to be a city, I think it's a Roman city, And then there are some other, you know, army invading it. And then they assume victory, defending this city. And then they try to send in some army inside a big horse as a gift to celebrate their victory. But then they turn the table around because all the army is hiding in the horse. Later, when everybody gets drunk, they come out and kind of capture the city. So that's why it's named Trojan sauce, because something is hidden there, and it could be dangerous. So in the paper that I read, so there's this paper. Some of you may have already read it, but I just explained it in a way that, like, oh, everybody can understand, right? So you don't have to read all the things. So in the paper, they describe there's, like, three categories of Trojan sauce. There is the early returns, so it forced the code to stop prematurely. There's commenting out, so there are codes that you assume that they're there, but they're not. There is also the stretch string, so something that you think is not in the string, but it's in the string. Okay. Example. So this one is a Python example, so I'm sure that we all love it. Because it's Python. Just because it's Python. When you see this code, nothing looks funny, right? It's just example where, of course, the bank won't have something so simple, but there's a docstring. Good practice, writing docstring, very good, but why there's a colon after the return is so strange. So they revealed that in a paper, oh, actually, someone put a reverse or something character there. When the computer sees it, it will flip, and then it will make the doc string close earlier, and then the colon as well, and then, you know, in Python, you can actually make a new line by just, like, making colon, right, like, of course, you know, formatting your code, and then it should be on the other line, but you can actually use the colon to force it to, you know, the thing is another line, so it will return after the doc string. So when the function returns there in the second line, or the first line, actually, if you don't count the doc string, then it won't carry out the one that minus the amount. So someone with the bank account will spend the money and keep the money in this example. It's just an example, but that's how early return works. Another example is commenting out. So this one, unfortunately, is in C. But you know Python actually be on C. So if something happened to C, something happened to Python. So yeah, I will explain what doesn't work later, because they also test it out afterward. But this is just an example of how this can be exploited to make your code not behave as what you expect. So here is very legit as well. Again, there are some comment there, like all the good developers, they would put comment in their code, so there are comment there. But the problem is that all of this looks a bit funny, but if you're not paying attention, then it could be something there like this. So this time is more complicated. They put more hidden character there. So what happened is I would just flip the order of how the first comment was written. So the if that is following on that line got buried inside a comment. So it won't check the if case. So it will just execute the second line. And another comment also buried the close bracket as well. Of course, in this case, it's flipped, so the brackets also got flipped. But it doesn't matter. It's still just take away the if course, so something that we don't want that to happen. The last example of the stretched string. So this one, again, looks legit. You just check whether something is a user. But this is very bad, actually. You shouldn't just check the string, or whether it's a user or not. But anyway, this is just an example of how things can be changed. So instead of just user, four characters, and then close quotation, it put all the comment and all these text that is not supposed be in the string, inside the string, so now it's the whole thing is of user, it's user slash slash check if a mean, you know, this, so if you rely on it to be exactly user, then it won't match and things went wrong. So this is in JavaScript, I believe, yeah, or Node.js or something like that, yeah. So it's not what it looked like. So, those three are the three types of Trojan sauce. But there are also things that they talk about in the paper that is a similar exploitation of a Unicode. So, for example, actually, sometime before COVID happened, we had a drink after the conference. I think it's PyCon UK, I believe. I can't remember. So, we had a chat with a bunch of friends, just like we will have a chat later today. that, like, actually, oh, we can actually mess someone up if we have some Stack Overflow answer but hide an invisible character there and people run that code and it doesn't work. So this is exactly what it is, but it's where someone can have a worse intention and not just trolling with people on Stack Overflow, like Stack Overflow, but, like, you know, can make a malware attack, whatever you call it, Or they can use, because inside Unicode, there are different characters, for example, like this one, that they look exactly the same, but actually they are a different character. So you can do something like that, that flip things around. You know Python? So this is not Python. This is C, I think. But in Python, if you declare multiple functions that have the same name, the last one got executed or something. But you can actually try to flip it. So the second one you use a different character. So it and then you and then it would just keep on the first the first one so I think in this case you see so it's the other way around but The ideas are the same just like things looks different same, but the same base looks different So for this one, it will actually print out goodbye world instead of hello world Yeah so Ha what works in Python because the example above use different code. So what exactly work in Python? So Early return as before shown that it kind of works, you know, so be careful of that also commenting out in the paper They actually have a github Page to show what what works and what doesn't so if you really into it then you could go inside So, this is just a source, so you can find it on GitHub, and you can see what works, what doesn't work. So, they try a little bit of, like, different things, so it's easier to see, I'll just show you this. Commenting out, so, it works on 3.9, so, I know that earlier, Liza talked about 3.10, so maybe using 3.10 is the ultimate answer, maybe this all won't work on 3.10, but I don't know, please test it out and let me know, or let the researcher of this paper know. So there's also an invisible function, like the example just before, the one using the similar characters. And also, yeah, all this homograph. I think it's called homograph. Yeah, the homograph function. I'll show you a little bit of the homograph function. So here, you can see that this is on GitHub. And then on GitHub, you can actually tell which one is which. So GitHub has a really good syntax highlighting. So it will just show you. It will color differentially if you use a holograph of it. So you see this one. You will see it's, oh, actually, it's good. Bye, world. But without the coloring, then you think it's hello, world. So yeah, it will work in Python just because how Python works. There's also other things like I will show you another example that they have here and see how GitHub is doing something to help you. So for example, commenting out here. You see that there's a warning here that's saying, oh, there is actually a bi-directional Unicode text. So it will interpret difference from how you see it. So actually, GitHub already did something to help developers, especially any of us that work in open source, to see maybe it's happening in a pull request. Then you kind of, ah, you have alert. and then you kind of really check what's going on there. It could be legit, it could be something within the comment or something within a text, but most of the time, it may be something that you don't want. So yeah, I think at the end, I want to talk about what should we do as a developer, especially those who work in open source or someone like me that love working in open source or contributing to open source or even maintaining some libraries yourself, or maintaining something on GitHub yourself. So shall we be worried? Shall we be very worried that we should actually ban text, this bi-directional character on our interpreter? Shall we do that? I think, I don't know. Actually, it's an open question. I'm not a core developer of Python, so I can't really say for Python. Of course, everybody can have an opinion, but my opinion will be that maybe we should not be so scared because, like I said, we want to be inclusive and there are languages that work in the other direction, so maybe that will limit the use case if you're working with other languages. So I think what I would recommend everybody to do is to use a really good text editor on good syntax highlighting. So actually, I showed you earlier this coding kind of platform that I could demonstrate the funny thing. So this one, actually, I kind of go through quite a lot of different sites to came across that one. Because most of the online interpreter or things like that now, or online editor, they have quite good syntax highlighting. It will show you what is invisible. So also, use one that has good syntax highlighting in your computer. I think the most common one that you use nowadays, you know, PyCharm or VS Code or myself use Atom. And it also shows invisible things as well, which is quite good. Also, scan the invisible characters and the bi-directional control characters. There are tools online to do that. And also, PyLint do that as well, I've been told. So I have a good linter, it's very important. I think it's a good practice for all the developers to use a good linter. So that would really help as well. So last thing, I think I have been that person that when I was a new Python programmer, I have looked for some code. Because I don't know how to do this, like how to sort out all these things, right? OK, look online and find an answer, just copy and paste. Don't. If you copy code online, the best case would be, oh, just someone trolling you, maybe. Or someone really want to help you, that's good. That's really good. Thank you, very helpful. But you may come across someone who just want to troll you. Or even worse, they may actually want to make your code, put some malware in your code. So really look before you run the code. One thing that I want to make an example is that, for example, pip install something. I think some inexperienced developer would be like, oh, I just pip installed the thing if it's not there. But sometimes they misspell, and then sometimes they install something that they should not be installing and could be dangerous. So yeah, always look, double check before you run the code. So we all love Unicode and emojis, so don't be too worried about it. We just have to be more responsible and more careful about the code that we run, that we write, or maintain. If you found this interesting, of course, check out the Children's Source website. Check out their work. Because these are all not my original work. These are the researchers' work. So you can go there and see the work that they published. So at the end, I promised to talk about EuroPython. So this summer, 11th to 17th of July, finally go back in person, fingers crossed, in Dublin. We will also have some remote thing, but details will be published in the future. Right now, if you buy the ticket this week, we are doing the voting, because the CFP have been closed. So if you have any talk you want to see in your Python, if you haven't got a ticket yet, get a ticket. If you already have the ticket, you can already do it and go to vote for the talk, anything that you want to see. So, I think that's it for me. Let me check how much time do I have. Oh, I haven't set the timer. Anyway, so I think I do have time for Q&A, so if you have any Any questions? I will have a look right now. Thank you. Yeah. I didn't see any questions, though. Or is it just me using things that is... Oh. Oh. Why is it on the other tab? Anyway, I saw some questions. examples know where true harm was done. Is that about this talk? I hope it's about this talk, because it's not in the Q&A, it's in the review. So I'm not using, I don't know how to use slide over. Anyway, I saw some questions. So an example, know where true harm was done. I don't know, really. I really don't know. But that was from the paper, this example. But I think it's a good thing that they raised this alarm and published it, so then we know that actually, oh, GitHub is protecting us. It will provide some warning and stuff. So it's just something that we have to be very aware. I think that maybe less so in Python, because Python developers have really good practice, but maybe in some other code, also some of them actually work on other programming language as well. Really can't I don't know is the answer So are there any to to reject suspect dependencies within my CI pipeline? Who or do I need to link all the dependencies? So I think that now So dependencies, okay dependencies is a topic that I came across quite a lot in my work I think that for me dependencies. I will try to keep them update all the time because you know sometimes you just forget and then like you know it would just there will be some old version that may have security issues but if you try to keep all your dependencies update all the time I think it's relatively safe especially when it's in Python everything is open source so if someone found a vulnerability then they would hopefully get resolved quite soon so So yeah, try to update your dependencies. Yeah, reject any, like, if it's suspicious, like if you can't update it or if it's suspicious, so maybe you should not use it. Besides PyLynch, do any other static code analyzer already check for those kind of attacks? I'm sure they are. So I just know that, like, PyLynch, it would tell you, oh, there's a Unicode there, or you can have some setting of, or only allow the Unicode inside the strings or something like that. So I only know that PyLint do it, but I'm quite sure that there's also others that also can do things like that. So do you know of any tools to check that source code file don't contain invisible characters? Yes, it's on the site. Yes, PyLint is a very good one, try it. Also, yeah, I mean, a lot of text editors already do that, so yeah. I think that's it. I can't find it. Oh, there's now the, oh, no. OK, I can't see any live questions now. So, ah, so do you mind repeating it again, or do you want the mic? Ah, OK, the user, oh, so the user data may embed something. So for injection attack is another thing. I think is, yeah, that's another thing that there may be tools to check the input that's protected from injection attack. But that's another thing. That's more for injection, not the source. I think it's more for if you put things in the source code. And because interpreter would also interpret the unique code. Yeah. I don't know any tools to check the input, though. I'm sure there will be something out there. Is there any other questions? Or I can just wrap it up and let you all go to get ready for the lightning talk on this really good day, for the first day today. Thank you so much for coming for PyCon DE.