Language, technology and society are the three worlds that Flor Plaza del Arco (29 years old, Villacarillo, Jaén) intertwines in her statistical research. She analyses the emotions that are present in the texts on social networks and detects whether they contain hate crimes, that is, comments that discriminate against people based on their characteristics such as gender, race, origin or disability. She also looks for methods that help her identify and mitigate biases or stereotypes that are present in language models, such as ChatGPT. Thanks to her work, she has been awarded the Awarded by the Scientific Informatics Society of Spain (SCIE) and the BBVA Foundationwhich encourages young computer science researchers.
The researcher compares her work in artificial intelligence (AI) to the education that a parent gives to a child: “Imagine that the child is the model. At first we teach him to speak in Spanish, and the child has a general knowledge of Spanish. As he grows up, you can tell him not to say certain things because, for example, they are an insult or to be careful if he is going to hurt someone. You adjust the model so that it learns to detect hate speech.” Plaza del Arco is a postdoctoral researcher in computer science at Bocconi University in Milan.
Ask. What are the most common hate crimes on social media?
Answer. Hate crimes regarding sexism and migration are the two we hear most about. We have also focused a lot on crimes regarding misogyny, as we see that there is discrimination in this regard. We have developed different models and resources so that the models learn to detect hate crimes. One of the risks of these models is the biases and stereotypes they produce; they have been trained with data present on the Internet, on Wikipedia, and they are a reflection of society.
P. Can you give some examples of stereotypes?
R. We developed a method to see if the models represented these stereotypes and we saw that women were more associated with emotions such as sadness or emotions related to care, while men were more associated with pride, ambition or aggression. An example is the Google algorithm that labeled people of color as gorillas, and they had to remove it. It was a bias. racist of the machine, and Google has committed to finding a solution to the error. They have material consequences and that is why it is important to detect this type of bias and mitigate it. For this, the quality of the data with which the systems are trained is essential, and that the human is there reviewing it, too.
P. What do you mean by quality data?
R. That the models do not contain personal information. Before, for example, if you asked the model how to commit suicide, the model responded by giving advice. Now the model says that it cannot answer that. A lot of work is being done to detect these types of security issues, biases and stereotypes.
P. How can artificial intelligence detect hate crimes?
R. We saw that there were no Spanish resources for detecting hate speech, both in tagged texts and for teaching the machine to detect this hate speech. It is not valid as a mere translation from English to Spanish. In Spanish we have our own expression and we have to teach it to the models. I focused a lot on the development of this resource and I used it to train artificial intelligence systems to detect hate speech in Spanish. At first they were very simple models and now they are more complex, capable of understanding and generating human language. It adjusts with tagged texts, I give it a text and I say: this text is hate speech, and this is not.
P. How much Spanish does ChatGPT know?
R. Artificial intelligence or artificial intelligence model has been developed primarily for English, the texts that have been taught to it are in English. Many of them say that they are multilingual, but perhaps the percentage that has been taught in another language such as Spanish is 20%. The model is much better at recognizing, generating and understanding English than Spanish, because it has not been taught as much text. That is why it is so important that the government now, with the National Artificial Intelligence Strategy, wants to create a model that understands Spanish, and not only Spanish, but also co-official languages such as Catalan, Basque, etc. It is very important to have models that understand different languages, because all people are using it, not just one from a certain country. Content in Spanish is less represented.
P. How can model biases be mitigated?
R. The key is to assess the quality of the data when training the model. You can also adjust the model once it has learned the data. You can adjust it or not so that it tries to unlearn it. We need different disciplines; computer scientists alone cannot work on this. We need philosophers, sociologists, psychologists to help us develop these types of models so that they are more inclusive, more ethical, more fair and responsible.
P. How do you promote a safer environment on social media?
R. Especially with research to combat hate speech and misinformation. If 1 million tweets are published on Twitter every second, it is impossible for a single person to deal with it. This type of machine helps us detect them. Alerts can be raised saying that the tweet is offensive or contains misinformation. The responsibility of generating the policies to see when a tweet is deleted is the responsibility of the content moderators.
P. Can AI detect whether there is a real person behind that content or not?
R. Exactly. With AI you can do a profile study of the person, if they publish many offensive tweets or many postsand can be transferred to any social network. It can also detect bots or detect spam, since these are linguistic patterns that these models learn. For example, bots They usually, almost always, follow certain linguistic patterns.
P. Are they similar sentence structures?
R. Yes. The models learn from hate speech. We are teaching it a text that contains Spanish insults, offensive expressions, etc. These are linguistic patterns that the model learns. In my thesis, I focused on developing a model that not only takes hate speech into account, but also takes into account that if the emotion is negative, such as anger, it is more likely that hate speech will occur. Irony and sarcasm are one of the most difficult challenges to detect for these language models. Why? Because even a human has a hard time detecting when there is irony or sarcasm in a text.
P. What can be done in other environments?
R. Especially in education, it is very important that children are taught about the risks of social networks, everything that can be found on them, and not just hate crimes. One example is cyberbullying, how social networks have promoted it. Technology education is very important from a young age because we are going to be in constant interaction with this type of technology in our daily lives. To know how to use it: when I see that a person is being attacked, to know how I can notify the social network that it is happening too far or if it is happening to me, and to know what the solutions are and the support I have. Will the social network delete this message that they are attacking me? Or will it delete the person’s profile? I think it is very important that this is transmitted to society in general.
P. If someone is the victim of a hate crime online, how can AI help?
R. It can help you by detecting those messages that are attacking you. Content moderators on social networks will be alerted that this content is being generated and will have their policies to see how to combat it, how to eliminate it or how to get in touch. My team at the University of Jaén, with whom I worked during my PhD, is now developing a type of research in which if someone has commented on an offensive message, it generates a counternarrative to make the person generating that hate think. For example, if it is a sexist or xenophobic message, a counternarrative would be, for example: “You have to think that all humans have the same rights. You cannot discriminate on the basis of gender or race.”
You can follow THE COUNTRY Technology in Facebook and X or sign up here to receive our weekly newsletter.
