Did you know that since the onset of the Covid-19 pandemic Facebook’s daily active users have gone well over 1.73 billion source: engadget.com, Zoom’s daily meetings are over 200 million and Google’s video conferencing is 25 times higher? Source: theatlantic.com. From positive work related interactions, communities coming together to help the most vulnerable, support and encouragement for the health care workers, humorous posts about the lockdowns to negative and atrocious content such as, fake news, racists posts, child abuse and pornography, the shared content varies from one user to another. With this immense increase in content creation, uploading and consumption, the internet can become a dark place.

And because people are free to anonymously publish posts or stream live, the anonymity shield makes it easier for them to go out of line. Content moderators who manage these online communities indeed have their work cut out for them. Reviewing User Generated Content (UGC) is not only a challenging but also a very demanding task.

Tech giants like Facebook, YouTube and other social media platforms were forced to close their moderation centers in March sending most of their content moderators’ home for safety reasons making a risky bet on AI to entirely moderate content. This caused a serious void in the content reviewing of ads and posts. Despite their efforts to curb the myths and false information related to the pandemic, these social media platforms faced the harsh reality of the gruesome content like child pornography that had leaked into the internet!

As a result of the lockdowns and also the security of office-attending employees, the content reviewing system was hugely incapacitated. Consequently, Facebook had to make the hard decision of calling back its content moderators whilst ensuring safety protocols were met i.e. checking temperatures, reducing building capacity, and providing protective equipment to enable the reviewing system resume blocking away and filtering out of child exploitation, terrorism and misinforming content.

Is AI reliable enough?

Situation awareness
Last year in March, a terrorist in New Zealand live-streamed from two different mosques, the brutal killing of 51 people. Unfortunately, Facebook’s algorithms failed to timely detect and block the gruesome video. It took them 29 minutes to detect the brutality in the video which was watched live with nearly 4000 people. In the aftermath, they struggled with taking down the posts from users who reposted the video. Although the company uses the most advanced innovation and technology, its AI algorithms still failed to correctly interpret the odeal.

Content & intent discernment
One of the drawbacks facing neural networks is their inability to correctly understand content and intent. In a call with analysts, Mark Zuckerberg Facebook’s CEO stated that it is much easier to train an AI system to detect nudity than it is to distinguish what is linguistically considered hate speech. According to Facebook’s statistics, its AI system is able to correctly detect nude content 96% of the time but struggles to discern safe nudity e.g. breastfeeding and prohibited content of sexual activities.

A good example of misinterpretations of AI algorithms is when the Facebook post of Norway’s Prime Minister was flagged as child pornography because it showed the image of the famous “Napalm girl”, a naked girl fleeing from an attack in Vietnam. Later on, Facebook apologized and restored the post.

And as the Corona virus continued to surge, Facebook experienced a massive bug in its spam filter for News Feeds that flagged URLs from genuine websites like USA Today and Buzzfeed source: techcrunch.com that were sharing Corona-virus related content most likely because of content misinterpretation with the AI systems.

Societal subjectivity
Because we are intrinsically diverse, our beliefs, values, cultures and religion differ from one region to another. What is considered okay in one country might be taboo in another for example wearing a Bikini is appropriate in most cultures but considered nudity in other cultures. Since most of the Application Programming Interface (API) providers are from the U.S. and Europe, they are often not in-tune with the cultures in the conservative parts of the world. So apart from the obvious explicit content, tackling the question of what is accepted is very country and region-specific and can only be effectively approved with human moderators from the different regions to avoid false positives or negatives flagged with the AI systems.

Racial disparity
In a content moderation study conducted with Nanonets, they assessed the accuracy of two API systems in detecting a Not Safe For Work (NSFW) image. The picture was contained a nude Japanese Woman dressed in a kimono. So because the neural networks were trained with pictures of European individuals, they failed to flag the image as NSFW. Users who are not based in the EU or U.S. were able to upload offensive content without the AI systems blocking them.

Stay tuned for part two….