2026 started with a horrifying example of generative AI’s potential for abuse. Grok, the AI tool from Elon Musk’s xAI, was used to undress or nudify pictures of people shared on X (formerly Twitter) at an alarming rate. Grok made 3 million sexualized images over a span of 11 days in January, with approximately 23,000 of those containing images of children, according to a study from the Center for Countering Digital Hate.
Now, competitors like OpenAI and Google are stepping up their security to avoid being the next Grok.
Advocates and safety researchers have long been concerned about AI’s ability to create abusive and illegal content. The creation and sharing of nonconsensual intimate imagery, sometimes referred to as revenge porn, was a big problem before AI. Generative AI only makes it quicker, easier, and cheaper for anyone to target and victimize people.
2026 started with a horrifying example of generative AI’s potential for abuse. Grok, the AI tool from Elon Musk’s xAI, was used to undress or nudify pictures of people shared on X (formerly Twitter) at an alarming rate. Grok made 3 million sexualized images over a span of 11 days in January, with approximately 23,000 of those containing images of children, according to a study from the Center for Countering Digital Hate.
Now, competitors like OpenAI and Google are stepping up their security to avoid being the next Grok.
Advocates and safety researchers have long been concerned about AI’s ability to create abusive and illegal content. The creation and sharing of nonconsensual intimate imagery, sometimes referred to as revenge porn, was a big problem before AI. Generative AI only makes it quicker, easier, and cheaper for anyone to target and victimize people.
On Jan. 14, two weeks into the scandal, X’s Safety account confirmed in a post that it would pause Grok’s ability to edit images on the social media app. Grok’s image-generation abilities are still available to paying subscribers in its standalone app and website. X did not respond to multiple requests for comment.
Most major companies have safeguards in place to prevent the kind of wide-scale abuse that we saw was possible with Grok. But cybersecurity is never a solid metal wall of protection; it’s a brick wall that’s constantly undergoing repairs. Here’s how OpenAI and Google have tried to beef up their safety protections to circumvent Grok-like failures.
Read More: AI Slop Is Destroying the Internet. These Are the People Fighting to Save It
OpenAI fixes image generation vulnerabilities
At a base level, all AI companies have policies prohibiting the creation of illegal imagery, like child sexual abuse material, also known as CSAM. Many tech companies have guardrails to prevent the creation of intimate imagery altogether. Grok is the exception, with “spicy” modes for image and video.
Still, anyone intent on creating nonconsensual intimate imagery can try to trick AI models into doing so.
Researchers from Mindgard, a cybersecurity company focused on AI, found a vulnerability in ChatGPT that allowed people to circumvent its guardrails and make intimate images. They used a tactic called “adversarial prompting,” where testers try to poke holes in an AI with specifically crafted instructions. In this case, it was tricking the chatbot’s memory with custom prompts, then copying the nudified style onto images of well-known people.
Mindgard alerted OpenAI of its findings in early February, and the ChatGPT developer confirmed on Feb. 10 — before Mindgard went public with its report — that it had fixed the problem.
“We’re grateful to the researchers who shared their findings,” an OpenAI spokesperson said to CNET and Mindgard. “We moved quickly to fix a bug that allowed the model to generate these images. We value this kind of collaboration and remain focused on strengthening safeguards to keep users safe.”
This process is how cybersecurity often works. Outside red-team researchers like Mindgard test software for weaknesses or workarounds, mimicking strategies that bad actors might use. When they identify security gaps, they alert the software provider so fixes can be deployed.
“Assuming motivated users will not attempt to bypass safeguards is a strategic miscalculation. Attackers iterate. Guardrails must assume persistence,” Mindgard wrote in a blog post.
While tech companies boast about how you can use their AI for any purpose, they also need to make a strong promise that they can prevent AI from being used to enact abuse. For AI image generation, that means having a strong repertoire of prompts that will be refused and kicked back to users.
When OpenAI launched its Sora 2 video model, it promised to be more conservative with its content moderation for this very reason. But it’s important to ensure its moderation practices are consistently effective, not just at a product’s launch. It makes AI safety testing an ongoing process for cybersecurity researchers and AI developers alike.
Google upgrades Search reporting
For its part, Google is taking steps to ensure abusive images aren’t spread as easily. The tech giant simplified its process for requesting the removal of explicit images from Google Search. You can click the three dots in the upper right corner of an image, click report and then tell Google you want the photo removed because it “shows a sexual image of me.” The new changes also let you select multiple images at once and track your reports more easily.
“We hope that this new removal process reduces the burden that victims of nonconsensual explicit imagery face,” the company said in a blog post.
When asked about any further steps the company is taking to prevent AI-enabled abuse, Google pointed CNET to its generative AI prohibited use policy. Google’s policy, like many other tech companies’ fine print, outlaws using AI for illegal or potentially abusive activities, such as creating intimate imagery.
.
AI safety is an ongoing challenge. ismagilov/iStock/Getty Images
In the world, human beings are facing various problems. People who are full of greed, anger, and ignorance are seen to be fighting with each other in words and deeds. As the fighting continues, various problems such as civil wars, wars of aggression, and wars of aggression are occurring. As world peace is being destroyed, humans are facing unbearable problems. Innocent children and women are not getting the rights they deserve and deserve. I can't think of one thing every time. What is it? The United Nations. If we look at the responsibilities and duties of the United Nations, I think it is not suitable for solving the problems that are happening in the world today. In fact, the United Nations and the World Security Council, such an organization must rule the world, must be able to give orders, and must have various ways to obey orders. There must be no organization that can influence more than that organization. There should be experts from each country. They can even dominate the superpowers like the US, China, and Britain. Only then will global peace be achieved quickly. Now, if a country with power develops more militarily, economically, or socially than its own country, then powerful countries will jealously seek out enemies and wage war. Or they will attack with fighter jets. In this, innocent people are losing their lives. No one understands the value of a life. You should think about how to achieve global peace by controlling these. If the US continues to be the permanent president of the UN, world peace will never be achieved. If you think carefully, no one sees that they are working for the common good, but for their own benefit, for the benefit of their own country. The permanent president of the UN and the Security Council has rejected this, As a young Rohingya, I have come to realize that only by electing the president and secretary from among the countries in the world through a secret ballot system and governing the world fairly can we achieve global peace. The United Nations and the Security Council are organizations in which any country must govern, give orders, and make them obey their orders. For example, only when a defendant is tried in court can we hold the leaders of the countries in check and control them, can we achieve global peace smoothly. Otherwise, it will not be easy for powerful countries to use various excuses to end this war every year, every year, for the benefit of themselves and their countries. No country's leaders pay any attention to the words of the United Nations and the Security Council. If we look at it in practice, look at the Rohingya issue. Myanmar is a small country. Even that country has been under siege for 8 years. What action can be taken? Why is the UN and the Security Council Powerless? Without such Power, how can the organization bring about global peace? Everyone should think about it. Whether there is a veto or a strong one, all countries in the world can only achieve global peace if they are subject to the UN and the Security Council. Seek peace in the quiet embrace of nature. Peace is an indispensable need of the human race. It is not as if one person can kill another person, and it is as if one country can attack another country, but instead of trying to build peace, we must try to destroy the peace now, and the world is getting hotter and is on the verge of collapse. I especially appeal to all the world experts to urgently try to bring about peace in all parts of the world. Mr, Noor Muhamed Arkhan Rohingya Youth Myanmar
Film and Writing Festival for Comedy. Showcasing best of comedy short films at the FEEDBACK Film Festival. Plus, showcasing best of comedy novels, short stories, poems, screenplays (TV, short, feature) at the festival performed by professional actors.
Leave a Reply