What Breaks When Automatic Speech Recognition Systems Go Multilingual

Speaker 1 [00:14]

Shree Nagpal, just a short introduction to her that she's an AI research scientist at Posadon and a researcher at MIT working the intersection of cyber security and artificial intelligence.

Speaker 2 [00:29]

I'll be on

Speaker 1 [00:30]

Beyond coding, she's an astrophile, and she likes to engage in board games with friends. So yeah, again, sorry, just a shout out that, quick reminder, if you want to provide any lightning talks, so please go to the booth, and you can submit your ideas there for the lightning talks. With that being said, I would like to give the stage to Rashmi. So yeah, let's welcome our speaker with a big

Speaker 2 [01:08]

so I just want to see people in the back can they hear me clearly or should I yeah so today let's build automatic speech recognition systems, multilingual recognition systems I just want to get like the hand like how many people in the audience know about you know like multiple languages for example how many people in the audience know mandolin please raise your hand. Great. I see three people up there. And German? Majority? Awesome. Great. Yes. And English? Okay. Awesome. Now I have a lot of guinea pigs for me to test out different systems, which I'll demo in my slides. Okay. So, yeah, I'm just a research engineer and, you know, doing day-to-day all these models, playing around with audio voice systems and also doing part-time you know it's an internship slash whatever the researcher at MIT and yeah I just like play around with a lot of systems because that gives me a lot of boost okay so first of all just to you know play though I'm pretty sure everyone is aware about what is an AI what is machine learning you know and all these systems around just to give a brief idea what we are going to talk in this entire session is mainly heavily actually on machine learning systems. So AI is kind of a broader umbrella. It's a broad term altogether. Machine learning systems is kind of subsets. Within the AI altogether, if you have given a data, you want to understand what this data comprised of, building some downstream machine learning task. That's what machine learning comprises. And deep learning is like one step beyond what machine learning offers. You don't need to have like a supervised data, but you want to extract certain patterns from it. So that's what deep learning gives you. Example is handwriting recognition. I have a very bad handwriting, so, you know, I have this model which helps in figuring out what I've actually written in my sub-notes. So, yeah, this is keep learning. Okay. Now I've explained basics. I just want to ask a question from everyone in this audience. Which of the following faces or you know, images are generated by LLMs? I'll just give you everyone a couple of seconds to think about it and then let's see how many people are able to identify it. I see the response from someone on the back. That's great. Okay. Please raise the hand if you think phase A, it's a real phase, not generated by LLM. Beautiful. What's that? Oh, so the question is, which of the following phases is generated by LLM? Yeah. So, do you think the face B is generated by NLM? Beautiful. What about face C? Nice. So, are there any people who think any of the faces are real faces? Come on, there should be some guinea pigs around. Okay, great. So the answer is yes. None of these people exist on the planet Earth. I'm not sure about the Mars. They are all generated by LLMs. So that's a good one. You know, basic building block behind all these LLMs is a transformer model, which is like you have a stack of encoders and decoders, and then it gives you the response based upon the input data that we provide to it. And different use cases around it is code generation that we all use for a cloud and so on and so forth. Now, what this talk is actually about, so given an audio sample, right, like the audio, whatever form of the audio sample it is comprised of to a specific language, then we build an ASR pipeline which basically converts that audio samples to the features for processing. And that feature is passed to this deep learning pipeline or this entire audio pre-processing pipeline by analyzing certain features. For example, some people say that I speak very softly while my brother who speaks very loudly at home. So, you know, every individual has a different linguistic features or different acoustic features, like the pitch variance, loudness, you know, all these features. So we need these features to build a specific ASR model. And what is an ASR? It's automatic speech recognition based on, you know, whichever downstream task you want to build on top of it. So this sounds simple, right? If I'm speaking in English, people can understand me in English. If I'm speaking in Hindi, but I'm using another language with Hindi, Then it becomes, hey, I'm not able to understand you. Hold on for a second. So that is called as a code mixed. When you have combining like two languages together, that becomes a code mixed audio samples. So we'll talk about it, you know, how we can build all these code mixed audio samples and make sure whatever the audio or downstream task we are working on, it's intuitive. Okay, so this is, I was actually talking with another colleague of mine, and we were thinking about, you know, do you understand, like, hey, Rashmi, once you have built this ML model, do you understand what does data leakage imply? Which means the data which is used for the training, it's a similar data which is used for the testing or for validation. situation? Has anyone ever faced this situation? Beautiful. So, well, we are on the same page then. Okay. So, you know, we have experienced when there's a lot of data leakage. Now, I just want to play a little game. These audio samples are generated using German, English, audio samples from different languages and I would like people to raise the hand and tell me which of these samples are real samples, which means a proper human, you know, it's a human audio, not generated by LLM. So let's play this game.

Speaker 1 [08:26]

Hiking up the mountain trails brings adventures and fresh air.

Speaker 2 [08:35]

Ich bin Leonie. Ich bin eine hervorragende deutsche Stimme für ihre Projekte. And I am fluent in English and other languages with my German accent.

Speaker 1 [08:46]

Wenn es dich nach Gespräch anhören soll, nicht nach Werbung, dann bin ich deine Stimme. Locker, direkt, professionell, für Projekte, die echt sind.

Speaker 2 [08:58]

want me to play again any of the audio samples? Which one would you like me to play before I ask the questions to the audience? Let me play this one.

Speaker 1 [09:14]

Picking up the mountain trails brings adventures and fresh air

Speaker 2 [09:23]

Out of all these speakers, which one of the speakers, and I'll ask people to raise their hand, do you think the speaker A is a real person? Yeah. Good. Okay. What about the speaker B? It's a real person, right? Okay. B? It's a real person, right? Okay. What about the speaker C? Come on. Very cool. Okay. Interesting. So none of the speakers exist on the planet Earth. Okay. So these audio samples are generated using AI models, voice AI models. And, you know, so, again, coming back to the question is can you even detect whether those are deep fake audio samples versus not? That's, like, the entire question that, you know, we'll be figuring out in this entire talk. Okay. So, what the problem statement is. First, we want to build multilingual ASR systems and where do they generally fail? Once we have seen, okay, we have audio samples from different languages, people speaking in English, people speaking in German, people speaking in, you know, Hindi, Korean, Mandarin, what not, different languages. If we are training the model using all the data itself, model will hallucinate, which means the model will not be able to distinguish based upon different languages and the patterns. I speak English but I used Indian English which is more daisy nuanced as compared to American English versus a British English right the way that I give the stress of the vowels or the consonants so those are called as linguistics and acoustic features and where do all these ASR models fail, they are not able to comprehend different languages. So, A, we need to provide a lot of data, diverse data, well-representative data, and we also need to ensure that our test samples or training samples, they are, you know, there is a distinguish between them, that we are not overlapping, we're not building a model which is screened on the same set of data, and then we are testing on the same set of the data, right? The reason is we don't want the model to overfit on a specific data sample. And here, misleading validation implies that you have a data sample which is well bifurcated into various sets, and then you are, once you have trained the model on the various languages, you are testing on different validation samples, unseen data, which is termed as. But a lot of people, sometimes when they build, they claim, oh, we built this ASR model, but there is a data leakage, which is not being, you know, well understood or handled in your systems. So that's an important part that one needs to understand before building all these ASR systems. Yeah. So let's say, yes. Yes, so what I'm trying to say is, as a speaker, I speak multiple languages. I speak Hindi, I speak Punjabi, different, different languages altogether, right? So if I'm using my data for training the model, and I'm testing the model on my data, So then the trained test, like the same speaker can't be in the two data sets. Okay, so now given this format, like now I give you different samples of the data. I give you one million rows of audio in English, in German, in different, different languages. And you know that in all these languages, there is a key challenge to it. Why? Because the way that we pronounce the words in English, it's not the way that how we pronounce in German. So, and also there is a morphological linguistic variance besides when we are speaking the words or the way we are stressing on certain specific words, right? So these are the key challenges which I've highlighted through my research, you know, like doing a lot of, while we are building this voice AI models. Now if I give you this data set, which is different languages, and we know that all these languages have issues, like the way people speak. Now I want you to build a model on top of it, like ASM model. So this is like just a generic overview, So, here, what I'm trying to say is, given an audio sample, you pass that sample through a config. When I say the config, which means every language will have a different pre-processor, a normalizer, you know, so on and so forth, which is kind of a data pre-processing technique. And at the same time, every model will not be trained on a single, like, we won't be having a global model. The model needs to converge on one specific language first and then it should be trained on a universal different languages so that it learns the patterns from one specific data and can we do the transfer learning on a different language. So another part is the loanword cache. So the example for this is normalization. For example, if I remove like a stem word from a language, like let's see if I'm saying, hey, this place is beautiful, but the weather is pretty hot. I don't like the weather. Maybe it's too clumsy, you know, whatever. So that's now I'm talking first in English, and now I started speaking in my own language. So, the model needs to understand when the language switch happens. So that the model does the context switching. And therefore, we need register language, a known word cache. So that it does the pre-processing based on a specific language. Once we have done that, there is, of course, you know, word error rate, character error rate. These are different evaluation metrics one can use and build a model on top of it to how the model is performing. These are a couple of, again, this is not an exhaustive list, some of the challenges which we encountered when we were building all these ASR pipelines. The first one is actually the memory efficient data loading. Once you have trained a model, you want to ensure that the model can be retrained from a certain checkpoint so that you don't have to, you know, train the model again and again. So that's like the data efficient loading. And the second one is pre-processing workflows, making sure that every language has a specific data pre-processor, normalizer, and so on and so forth. That's kind of a downstream, you know, task that you want to achieve. And then long-running job management is basically about, once you have created the model, if there is a failure. So just locking everything properly, making sure the checkpoint exactly covers all the tokenizers, config, and so on and so forth. The last one is pipeline robustness. It's, again, it depends upon, our language. For example, so this is like, just to give an example, like if two people are speaking and someone becomes quiet, like right now I was talking and explaining and then I became quiet because, you know, the throat or whatever, it's just lack of not having the water in between. So then the model should be able to understand, hey, where is a silence coming in, when the other person is not saying, so that is called as, you know, the silence trimming, VAD is one of the algorithms that we use for that, and also making sure that every language is processed through their own configs. Yeah, so this is like the evaluation pipeline again for cross-lingual, to avoid that kind of a cross-lingual leakage and the data leakage altogether. What the flawed approach is, so this flawed approach is coming after a lot of iterations once we have built our Moiseval models. And this correct approach is what we have identified after running multiple iterations on the chain points of the audio samples and building the ASR pipeline. So the correct approach, so first of all, why some approaches called as a flawed approach? Like what was the reason behind it? If you see in the first box, it says it's a mixed pool of data, which means people speaking in different languages, you have training and test. There is no validation also in here, which is wrong. You need to have a certain set of samples, which are completely unseen samples. And then there could be a possibility that a speaker, which is in the training sample, it could possibly be also in the test sample. So data leakage or the speaker similarity is there. So the model is going to be biased towards one specific speaker. And the correct approach is, if you are having multiple speakers speaking in different languages, make sure there is a speaker disjoint. There is also, you know, after a speaker disjoint for every specific language, and at the same time, every language has different varied speakers. Like a speaker A, if that speaker is speaking in Hindi language, he or she should not be saying in English because the model is going to converge and say, hey, this speaker has spoken previously. So it's more likely to say a deep fake audio sample. Okay, so these are certain design patterns that we have used in our pipeline when we were building ASR models. The first one is, you know, the data leakage, making sure every speaker ID is disjoined so that there is no commonality between two speakers and different samples for the splits of the data. The other one is the language config, like every language should have a different pre-processor, normalizer, so on and so forth. Third one is on the feature extraction. This is more on how we want to use embedding models based upon the audio samples data that we have. The last one, I think this is, yeah, this is interesting. Actually, I saw this very recently because I used to pronounce this German word like strass. And the other day, someone said, hey, Rashmi, you're doing the wrong pronunciation. Come on, you're in Germany. So, again, giving an example here, if you're passing these two words, if you do the cosine similarity or a semantic similarity, they have the similar meaning, right? But if the model doesn't have the right hypothesis to understand, it's going to mix it. So, one needs to have understanding about what kind of ASCII, you know, all these patterns needs to be involved before building a hypothesis testing for transcript generation of the ASR model. This one is, again, for every different language, if you want to build an ASR model, once you have trained on a specific language, it should be easy flow. We want to just add a new language in the form of a function, and the rest of the pieces of the code will remain intact. So here I've added a German process. It's a decorator design pattern. you all see that German processor, then it's normalized and different pipeline will go. And if you want to include different language, so you have to write the same code, but for a different language. Like English processor, the normalized for English processor will be different, and so on and so forth. So we are not, you know, so basically this is like the separation of concern principle and also at the same time we are not breaking the existing pipeline which is well-validated for one specific language. Okay, so, yes. Once I have built ASR model, well, one of the customer was like, hey guys, I'm using your API, but it's not working, you know, he was Italian, so it's not working for our language. what's going on, and we pay like a lot of bucks. So we figured there is an issue, and that issue, which was I deployed in production late in the Friday night, which is also good learning. You should never deploy anything on a Friday night when everyone is drunk and having the pens and whatnot. Okay, so has anyone ever faced a situation, right? Like, you know, when the bug is reported when some system or model has been deployed in production. Just want to see how many people are in the same phase as me. Good. So when this happens, what are the different techniques that we can follow? So I'll give an example of defect detection. So I built this defect detection for one of the clients, and we got to know that our model was not able to detect for their system altogether. So I'll give a brief glimpse of how we achieved it. Yeah, so this is already, I mean, I'll share the codes and slides also with everyone on our, you know, the platform. But here, what I wanted to say is, given a language model, you already have a config, you know, for a specific language, what are the pipelines? Extract from the audio samples, some of the acoustic features, like pitch, loudness, you know, so on and so forth. And then you build a model, like train a model based upon those linguistic features and get awarded with a certain confidence. Here you see that this ASR model, it comprised for like two languages. First one right now here, which is a German and an English. And then I've extracted all these different features. Jitter, shimmer, F0, SNR. SNR is like signal to noise ratio. When an audio has a lot of noise, then it's more likely to have a deep fake. But again, it's more likely to have a deep fake in a sense that if people are using studio audio samples and it's very clean, so that is not, I mean, so that is definitely a deep fake, but when you are speaking in a different tone and mixing different languages. So there are some hypothesis one needs to test to see. Okay, so here once we have calculated, you know, different features for different languages, and we know that we have the sample label data set, which is a deepfake and a real. We calculated these features. We found that the deepfake voices have lower shimmer and lower jitter variance, which means if you listen to deepfake audio samples, they sound very smooth, like, as if it's a Robotech a voice which is coming across, right? This is what the key observation entails. And once you, like for the human speech, we have this natural variance. I'm sometimes speaking in high variance, sometimes I'm speaking in lower variance, right? So it depends upon the pitch, loudness, you know, all these different, different acoustic features. So this is what our wording, which is coming from this model and then you get like the word error rate and character error rate. So as a human, I'm going to mispronounce some words. Mistakes are bound to happen within us, right? So but for an audio defects, they are very smooth in a sense that they don't create like mistakes very often. So therefore, if they have the lower word error rate or a character error read, it means the audio is super smooth. It's more de-fake. Those are different eval metrics. Once you have created a model for different epochs, you see what is the loss curve which is coming across, and then you assign different weights. It's weighted in sample kind of a model. And then once you have evaluated on a small test set right now, you will get an accuracy, and at the same time you get like a word in here um yeah so here you see like the german like the word that i was saying stress but correct me if i'm still pronouncing wrong so you know if you have stress like different words have the same literal meaning like semantic meaning but the way they are presented that's wrong so therefore it will inflate error deviations and therefore this model will understand and give you the wording hey this audio sample it's a defake versus a real yeah so um this is um you know pretty much an example of how we build all these um defake systems and what are the biometrics one can use um i want to give one more Yeah, so once we have built this detection system, defect detection system, we want to really provide a verdict whether the model is not hallucinating and it's giving response with a certain confidence with the human in the loop also. So, here I am evaluating every stage of this ASR model, which was given for, you know, which was trained for this defray detection, and this ASR pipeline is going to do different stages of validation. First is the data, whether the model which has been trained and the verdict has given, whether there is any data leakage there versus not. And then it uses the eval metrics, normalize, you know, that's like a pre-processing. And then it's going to compare with the baseline or the benchmark results. So here we see that for, you know, different samples, again, this is a very small test samples, which I use because model has already been trained. Here you see that, hey, six of seven test speakers are also in the train. So the model will memorize the speaker identically, not the test signal. So here it found an issue, and now it is going to run the leakage check. Basically it wants to make sure that the speaker disjoint split habits, that none of the speakers are in the test and the trained samples. So when it has done it, then it's... So you know, you create the heuristics, p-value, all these mathematical heuristics. you want to ensure that there is zero data leakage across different languages. Yeah, so here it's a very interesting example. I think, actually I can give for this one, English one is, she said, let's go immediately. And the hypothesis was, she said, let's go immediately. Now, if you see, both the sentences are similar, right? Semantically similar and the response should be similar. But no, here you see that the model exactly says that, you know, after we have done all this punctuation, after all this pre-processing, we see that that cart, the way that the cart is written is similar, the meaning is going to remain the same, but the character error rate is still bad as compared to the word error rate, right? So one needs to make sure which evaluation metric one is building for this ASR downstream task. So once it has identified this issue, it will go back, iterate, and build again a model. So here you see that these two words mean the same, but after normalization, it's a loanword normalization once you have done, then now the character iterate comes across like 0, which was initially 0.14. When I say it was 0.14, it was before normalization or before any pre-processing technique. Therefore, it's important, while we are building ASR models for different languages, picking the right technique, picking the right metrics to evaluate and quantify how our model is varying. Yeah, so this is, yeah, so basically this will give, you know, this evaluation pipeline will give a downstream task of how we want to evaluate our ASR model. Okay, so coming back here, I think this is pretty much I had for all of you today. Yeah, again, just like speaker bugs, you know, this is important that we need to ensure that there is no data leakage versus the overfitting of the model doesn't happen from one samples of the data versus another. And also making sure that we are building whatever the pipeline we are building for one specific language, it should be the reusable component for another language. And the last part is leakage controls is also equally important, I think, which ties back to the first one, making sure that the language, every language is pre-processed to a different normalizer, you know, a different tokenizer for every language that we have. Yeah. And, yeah, thank you. Happy to take any questions.

Speaker 1 [33:19]

for me to talk we have a couple of questions for you so i'll just take couple of them first one is there are llm based asr multi-language model that is whisper from open ai do they follow the same technique meaning do they have a model per language under the hood

Speaker 2 [33:49]

For example, the other day, I was using Whisper language model for Bengali. The Whisper is definitely one of the state-of-the-art models. Eleven Labs, Assembly AI, you know, all these commercial APIs, they also have all these voice models. But Whisper is like an open source. One can actually find you based upon their own proprietary data. But not all the languages are supported by Whisper. In fact, there is another model, ASR model, released by Meta, which is omni-lingual ASR. I think it's trained on 1,600 plus languages. So one can fine-tune based upon their own data set and build on top of it. Having said that, all these languages, you know, all these ASR models have different evaluation metrics. WER, which is word error rate, it's industry standard. and that's also reported by different models, open source as well as closed source. So yes, the techniques that I showed in here are also being used by open source.

Speaker 1 [34:58]

Next question. How do you decide which language data set a speaker goes into if they have multiple language data points in the data set? Their mother tongue as it will probably be clearest, the longest chunk, specifically not their mother tongue so you don't overfit a language model on excellent speakers.

Speaker 2 [35:24]

Got it. It's a very lengthy question. I'll divide this question into two parts. First one is, how can we segregate the same speaker, not splitting in different data sets? And at the same time, if the same speaker is saying different languages, one is a native language, another is, let's say, English, then how can we ensure, what was that? The same speaker is using a native language.

Speaker 1 [35:51]

or fit a language model on excellent speakers.

Speaker 2 [35:57]

There are techniques like e-kappa, released by Stanford, that's one of the packages, which gives you the speaker's similarity. If I speak in Hindi and I speak in English, I have the same pronunciation of the words, like my pitch is going to be the same. Just to give an example. Now I'm going to give an example of Indian English. It's gonna be funny though. I love the silence in the audience too, they are actually curious what I am going to say. I am actually very blessed to get this opportunity to speak in front of you and also to see lovely faces trying to comprehend what I really want to say, even though I flumber and mumble a lot, I am very proud of it. See the way that I was saying it was in English, but I also switched to my own native language which is Hindi, right? So just to ensure that the model doesn't over-fizz on a specific speaker for two different languages, we need to do a cut, cut in a sense that let's English spoken word by this speaker should go in the English model and the Hindi part of the entire transcript that I said goes in that particular ASR model, but at the same time ensuring that the same speaker is not over fitted. We therefore exclude one part of my audio on the test sample, right? Why do you want to do it? Does anyone want to give a guess? Data leakage? Data leakage? Yeah, data leakage is for sure. Any other insights? Any Indian here, I'm pretty sure, well not every Indian knows Hindi, but, you know, majority of them. Any, or, you know, anyone who want to give, you know, just a random guess? Anyone from the audience? I don't want to pick you all. Because people in the front row, they are like, ah, no, not me. Yeah, a random guess in a sense that if I, as a speaker, if I'm speaking in English, and if I'm speaking in German in the same sentence, why do you want to discard my German spoken text or an audio as compared to English so that the model doesn't overfit? he gave an example like he gave a response as data leakage that's true, anything else? probably because of other features right? yes, correct awesome, yes because I speak in the way that I speak I have low pitch and I also use a lot of filler words I use um, the, they you know all of these filler words so the model will know hey this speaker a rashmi which is again in a form of encoding which will be passed to the model she uses a lot of filler words so the model doesn't overfit therefore we'll discard her in other language if we want to build a you know a multilingual asr model

Speaker 1 [39:29]

for the elaborative answer so final question one way to handle multiple languages spoken in one sentence is by detecting silence segments indicating a change in language how we enhance that if the speaker is fast when speaking multiple languages in the same sentence without breaking the context if using an LLMB is.

Speaker 2 [39:59]

It's a heavy-loaded question. Okay. I think I want to take an audience poll or audience help to answer me the question just to make sure everyone has understood what I've explained. Does anyone want to give a supportive answer before I give the response? Do you want to? Yeah.

Speaker 1 [40:25]

This kind of sentence

Speaker 2 [40:28]

If it's too...

Speaker 1 [40:30]

It's switching.

Speaker 2 [40:36]

That's an easy way out. Anyone else? Anyone from the back or front? Yes. That's right, good. It's not a test. Honestly, just give a random guess. Just want to brainstorm with all of you. Yes. You can add some sort of vocab. You can just, all the transcription they say, okay, you switch to English now. You don't have to do it on the old yeah that's right yeah anyone else so let's say um i'm talking to someone else um i'm talking to you and i speak i'm not going to ask the question don't worry i see the oh no Okay, question. So the way that I'm speaking, I'm using a lot of, you know, I speak very fast when I'm absolutely nervous. When I'm confident, I speak slowly. So the question was saying, how can we ensure, like, silence from the speakers, right? If you are saying something in German and I'm speaking in English, how can we ensure that every speaker is joined? So every person has a different set of features, which someone from the back, she gave the beautiful answer. And at the same time, silence is for different ASR model. Have you heard, I think Claude and Chad also has a voice mode. I think Chad has a voice mode now. They try to use a lot of filler words. We humans take pauses, sometimes take bigger pauses, sometimes use a lot of filler words, hey, um, the, you know, all these things, to make sure how a different vocab that this gentleman give the answer, you know, every person has a different vocab, ensuring that we have the ground truth, data which is well annotated, but at the same time, different ASR models need to converge on certain specific examples. So the answer to that question is a little nuanced. I don't think there's still a lot of work needs to be done, but this is pretty much what I've understood.

Speaker 1 [43:01]

We have two more minutes for one final question. Which deepfake audio generation did you use in your evaluation? Today, native audio, LLMs like Gemini native audio, Noasonic, etc. can produce or simulate perfectly human voices even with pauses, breathing, and mistakes. It seems that this will be an impossible task, same as trying to detect text generated by ear.

Speaker 2 [43:34]

possible task achieved, right? And this is technology. I agree to an extent that, you know, it's hard to distinguish between the deepfakes versus not. And that's why all of us in the third slide, we messed up. We don't know which one is the audio samples, but they were generated using, you know, LLMs. To give an example, we also have like internally ASR models, a lot of open source models that one can use as only lingual model released by META, and there is also MMS models also released by META. All these models, they have a nuanced understanding in a sense that we humans are very creative. If I don't know the answer, I will be either surprised or I will try to engage the other person in asking about, hey, you know, I don't know, I use like filler words, interruptions, so on and so forth, but can LLM talk in that nuanced, natural way? Answer is definitely to an extent, but there are certain guardrails in place, right? The way that we code switch, the way that we do logical, you know, questioning and answering creatively, we are pronouncing the words. so having that human in the loop while you are