What they have accomplished is to actually separate the wheat from the chaff and this is important as it all naturally reverts back to a complaint driven system.
We want materiel to be challenged easily enough in order to trigger a review. That leads to a human review and the source is flagged easily enough as trusted or not. After which the source can cease been heavily monitored.
It is a form of soft censorship that especially stops abusive suppliers. Those suppliers still have the option of developing a closed community prepared for them or should.
I do notice that much of the more overt abusive material is slowly falling away.
Facebook's AI Can Analyze Memes, but Can It Understand Them?
Author: Louise Matsakis
https://www.wired.com/story/facebook-rosetta-ai-memes/
Billions of text posts, photos, and videos are uploaded to social media every day, a firehose of information that’s impossible for human moderators to sift through comprehensively. And so companies like Facebook and YouTube have long relied on artificial intelligence to help surface things like spam and pornography.
Something like a white supremacist meme, though, can be more challenging for machines to flag, since the task requires processing several different visual elements at once. Automated systems need to detect and “read” the words that are overlaid on top of the photo, as well as analyze the image itself. Memes are also complicated cultural artifacts, which can be difficult to understand out of context. Despite the challenges they bring, some social platforms are already using AI to analyze memes, including Facebook, which this week shared details about how it uses a tool called Rosetta to analyze photos and videos that contain text.
Facebook says it already uses Rosetta to help automatically detect content that violates things like its hate speech policy. With help from the tool, Facebook also announced this week that it’s expanding its third-party fact checking effort to include photos and videos, not just text-based articles. Rosetta will aid in the process by automatically checking whether images and videos that contain text were previously flagged as false.
Rosetta works by combining optical character recognition (OCR) technology with other machine learning techniques to process text found in photos and videos. First, it uses OCR to identify where the text is located in a meme or video. You’ve probably used something like OCR before; it’s what allows you to quickly scan a paper form and turn it into an editable document. The automated program knows where blocks of text are located and can tell them apart from the place where you’re supposed to sign your name.
Once Rosetta knows where the words are, Facebook uses a neural network that can transcribe the text and understand its meaning. It then can feed that text through other systems, like one that checks whether the meme is about an already-debunked viral hoax.
The researchers behind Rosetta say the tool now now extracts text from every image uploaded publicly to Facebook in real time, and it can “read” text in multiple languages, including English, Spanish, German, and Arabic. (Facebook says Rosetta is not used to scan images that users share privately on their timelines or in direct messages.)
Rosetta can analyze images that include text in many forms, such as photos of protest signs, restaurant menus, storefronts, and more. Viswanath Sivakumar, a software engineer at Facebook who works on Rosetta, said in an email that the tool works well both for identifying text in a landscape, like on a street sign, and also for memes—but that the latter is more challenging. “In the context of proactively detecting hate speech and other policy-violating content, meme-style images are the more complex AI challenge,” he wrote.
Unlike humans, an AI also typically needs to see tens of thousands of examples before it can learn to complete a complicated task, says Sivakumar. But memes, even for Facebook, are not endlessly available, and gathering enough examples in different languages can also prove difficult. Finding high-quality training data is an ongoing challenge for artificial intelligence research more broadly. Data often needs to be painstakingly hand-labeled, and many databases are protected by copyright laws.
'In the context of proactively detecting hate speech and other policy-violating content, meme-style images are the more complex AI challenge.'
Viswanath Sivakumar, Facebook
To train Rosetta, Facebook researchers used images posted publicly on the site that contained some form of text, along with their captions and the location from which they were posted. They also created a program to generate additional examples, inspired by a method devised by a team of Oxford University researchers in 2016. That means the entire process is automated to some extent: One program automatically spits out the memes, and then another tries to analyze them.
Different languages are challenging for Facebook’s AI team in other ways. For example, the researchers had to find a workaround to make Rosetta work with languages like Arabic, which are read from right to left, the opposite of other languages like English. Rosetta “reads” Arabic backwards, then after processing, Facebook reverses the characters. “This trick works surprisingly well, allowing us to have a unified model that works for both left to right and right to left languages,” the researchers wrote in their blog post.
While automated systems can be extremely useful for content moderation purposes, they’re not always foolproof. For example, WeChat—the most popular social network in China—uses two different algorithms to filter images, which a team of researchers at the Univeristy of Toronto’s Citizen Lab were able to successfully trick. The first, an OCR-based program, filters photos that contain text about prohibited topics, while the other censors images that appear similar to those on a blacklist likely created by the Chinese government.
The researchers were able to easily evade WeChat’s filters by changing an image’s properties, like the coloring or the way it was oriented. While Facebook’s Rosetta is more sophisticated, it likely isn’t perfect either; the system may be tripped up by hard-to-read text, or warped fonts. All image recognition algorithms are also still potentially susceptible to adversarial examples, slightly altered images that look the same to humans but cause an AI to go haywire.
Facebook and other platforms like Twitter, YouTube, and Reddit are under tremendous pressure in multiple countries to police certain kinds of content. On Wednesday, the European Union proposed new legislation that require social media companies to remove terrorist posts within one hour of notification, or else face fines. Rosetta, and other similarly automated tools, are what already help Facebook and other platforms abide by similar laws in places like Germany.
And they’re getting better at their jobs: Two years ago CEO Mark Zuckerberg said that Facebook’s AI systems only proactively caught around half of the content the company took down; people had to flag the rest first. Now, Facebook says that its AI tools detect nearly 100 percent of the spam it takes down, as well as 99.5 percent of terrorist content and 86 percent of graphic violence. Other platforms, like YouTube, have seen similar success using automated content detection systems.
But those promising numbers don’t mean AI systems like Rosetta are a perfect solution, especially when it comes to more nuanced forms of expression. Unlike a restaurant menu, it can be hard to parse the meaning of a meme without knowing the context of where it was posted. That's why there are whole websites dedicated to explaining them. Memes often depict inside jokes, or are highly specific to a certain online subculture. And AI still isn’t capable of understanding a meme or video in the same way that a person would. For now, Facebook will still need to to rely on human moderators to make decisions about whether a meme should be taken down.
Author: Louise Matsakis
https://www.wired.com/story/facebook-rosetta-ai-memes/
Billions of text posts, photos, and videos are uploaded to social media every day, a firehose of information that’s impossible for human moderators to sift through comprehensively. And so companies like Facebook and YouTube have long relied on artificial intelligence to help surface things like spam and pornography.
Something like a white supremacist meme, though, can be more challenging for machines to flag, since the task requires processing several different visual elements at once. Automated systems need to detect and “read” the words that are overlaid on top of the photo, as well as analyze the image itself. Memes are also complicated cultural artifacts, which can be difficult to understand out of context. Despite the challenges they bring, some social platforms are already using AI to analyze memes, including Facebook, which this week shared details about how it uses a tool called Rosetta to analyze photos and videos that contain text.
Facebook says it already uses Rosetta to help automatically detect content that violates things like its hate speech policy. With help from the tool, Facebook also announced this week that it’s expanding its third-party fact checking effort to include photos and videos, not just text-based articles. Rosetta will aid in the process by automatically checking whether images and videos that contain text were previously flagged as false.
Rosetta works by combining optical character recognition (OCR) technology with other machine learning techniques to process text found in photos and videos. First, it uses OCR to identify where the text is located in a meme or video. You’ve probably used something like OCR before; it’s what allows you to quickly scan a paper form and turn it into an editable document. The automated program knows where blocks of text are located and can tell them apart from the place where you’re supposed to sign your name.
Once Rosetta knows where the words are, Facebook uses a neural network that can transcribe the text and understand its meaning. It then can feed that text through other systems, like one that checks whether the meme is about an already-debunked viral hoax.
The researchers behind Rosetta say the tool now now extracts text from every image uploaded publicly to Facebook in real time, and it can “read” text in multiple languages, including English, Spanish, German, and Arabic. (Facebook says Rosetta is not used to scan images that users share privately on their timelines or in direct messages.)
Rosetta can analyze images that include text in many forms, such as photos of protest signs, restaurant menus, storefronts, and more. Viswanath Sivakumar, a software engineer at Facebook who works on Rosetta, said in an email that the tool works well both for identifying text in a landscape, like on a street sign, and also for memes—but that the latter is more challenging. “In the context of proactively detecting hate speech and other policy-violating content, meme-style images are the more complex AI challenge,” he wrote.
Unlike humans, an AI also typically needs to see tens of thousands of examples before it can learn to complete a complicated task, says Sivakumar. But memes, even for Facebook, are not endlessly available, and gathering enough examples in different languages can also prove difficult. Finding high-quality training data is an ongoing challenge for artificial intelligence research more broadly. Data often needs to be painstakingly hand-labeled, and many databases are protected by copyright laws.
'In the context of proactively detecting hate speech and other policy-violating content, meme-style images are the more complex AI challenge.'
Viswanath Sivakumar, Facebook
To train Rosetta, Facebook researchers used images posted publicly on the site that contained some form of text, along with their captions and the location from which they were posted. They also created a program to generate additional examples, inspired by a method devised by a team of Oxford University researchers in 2016. That means the entire process is automated to some extent: One program automatically spits out the memes, and then another tries to analyze them.
Different languages are challenging for Facebook’s AI team in other ways. For example, the researchers had to find a workaround to make Rosetta work with languages like Arabic, which are read from right to left, the opposite of other languages like English. Rosetta “reads” Arabic backwards, then after processing, Facebook reverses the characters. “This trick works surprisingly well, allowing us to have a unified model that works for both left to right and right to left languages,” the researchers wrote in their blog post.
While automated systems can be extremely useful for content moderation purposes, they’re not always foolproof. For example, WeChat—the most popular social network in China—uses two different algorithms to filter images, which a team of researchers at the Univeristy of Toronto’s Citizen Lab were able to successfully trick. The first, an OCR-based program, filters photos that contain text about prohibited topics, while the other censors images that appear similar to those on a blacklist likely created by the Chinese government.
The researchers were able to easily evade WeChat’s filters by changing an image’s properties, like the coloring or the way it was oriented. While Facebook’s Rosetta is more sophisticated, it likely isn’t perfect either; the system may be tripped up by hard-to-read text, or warped fonts. All image recognition algorithms are also still potentially susceptible to adversarial examples, slightly altered images that look the same to humans but cause an AI to go haywire.
Facebook and other platforms like Twitter, YouTube, and Reddit are under tremendous pressure in multiple countries to police certain kinds of content. On Wednesday, the European Union proposed new legislation that require social media companies to remove terrorist posts within one hour of notification, or else face fines. Rosetta, and other similarly automated tools, are what already help Facebook and other platforms abide by similar laws in places like Germany.
And they’re getting better at their jobs: Two years ago CEO Mark Zuckerberg said that Facebook’s AI systems only proactively caught around half of the content the company took down; people had to flag the rest first. Now, Facebook says that its AI tools detect nearly 100 percent of the spam it takes down, as well as 99.5 percent of terrorist content and 86 percent of graphic violence. Other platforms, like YouTube, have seen similar success using automated content detection systems.
But those promising numbers don’t mean AI systems like Rosetta are a perfect solution, especially when it comes to more nuanced forms of expression. Unlike a restaurant menu, it can be hard to parse the meaning of a meme without knowing the context of where it was posted. That's why there are whole websites dedicated to explaining them. Memes often depict inside jokes, or are highly specific to a certain online subculture. And AI still isn’t capable of understanding a meme or video in the same way that a person would. For now, Facebook will still need to to rely on human moderators to make decisions about whether a meme should be taken down.
No comments:
Post a Comment