Impact of AI on online content moderation

We have all heard about Artificial Intelligence (AI) and the numerous potentials impacts it will or already has on our daily lives.

Machine Learning through Data Annotation is teaching computers to recognize what we show them, what we say, and how to react accordingly.

When trained well, the impacts it could have on online Content Moderation seem quite straightforward at first. Nonetheless, we will see that AI brings opportunities in the field as well as new challenges, not forgetting that we are only witnessing its genesis – there is still great room for improvement.

Implementing the process, but not totally developed yet

Virtually, AI seems to be a no-brainer as it will take the hit on the most sensitive contents. It will work as a fully impartial chooser instead of moderators having to approve or deny harmful posts.

This is currently put into practice within Webhelp – thanks to our in-house technology handling a growing part of the incoming User-Generated Contents, and attributing priority levels for moderators to take care of the most urgent ones first.

We have established that if AI obtains total control over what can appear on the internet, it will start to get messy very quickly. 2020 pushed tech giants to send workers home and to rely on algorithms to moderate their platforms. As soon as this happened, issues were observed across the two extremes. In fact, on Twitter, there was a steep increase of 40% of hate speech in France, while Facebook and Google both doubled the number of pieces of content flagged as potentially harmful material from Q1 to Q2.

Several examples of artificially intelligent moderators failing their tasks have been observed as not being able to understand human expressions in the first instance, such as irony, sarcasm, or more striking and unambiguously harmful words, however when they are put into context they reveal to be harmless.

This happened over a live chess game on YouTube which has been taken down due to hate speech, but only chess strategy was talked about. The limitations Artificial Intelligence encounters start to fade away as researchers from the University of Sheffield are starting to successfully integrate context in Natural Language Processing algorithms. This technology will be able to detect the differences of languages across communities, races, ethnicities, genders and sexualities, but as Ofcom says: “Developing and implementing an effective content moderation system takes time, effort and finance, each of which may be a constraint on a rapidly growing platform in a competitive marketplace”.

Beneficial in fighting discrimination and derogatory speech online

Following an objective of moderating online content solely through Artificial Intelligence, several start-ups are arising in the market with ever-improving AI-driven solutions.

Platforms have started developing their own tools: Pinterest unveiled AI that powers its Content Moderation and highlighted its benefits since its implementation: over a year, non-compliant reports have declined by 52% and self-harm content by 80% in the past two years. As we already mentioned, the quality and the quantity of labelled data is key -Facebook, thanks to 1 billion Instagram photos, has also succeeded in developing an innovative image-recognition AI system aiming at moderating the platform almost instantly. As it has just been launched, we are not able to appreciate SEER’s (SElf-supERvised) direct effects on the platform yet.

Watching out for the deepfakes

While these new technologies have potential for positive impact on Content Moderation, they have also created new challenges which plenty of us have already come across, growingly without even noticing it: deepfakes. When analyzing the credibility of content sources, AI can more easily recognize a bot that would be used by malicious users to amplify disinformation, and we can reasonably assume that it would do so for AI-created deepfakes. This issue is way more difficult to detect for the human eye, but appropriately trained moderators, supported by the right AI-driven tools is the perfect combination to complement purely automated or purely human moderation, quickly and effectively.

The first big reveal when it comes to this technology is Microsoft’s deepfake detection tool which has been trained on over 1,000 deepfake video sequences from a public dataset, in a similar manner Facebook has trained its moderation AI. Disruptors also enter the market: platforms like Sensitivity.ai are specialized in detecting face-swaps and other deepfakes which can have deep impacts on the political scene for instance. In fact, the most famous and recent example of deepfake was the face swap of Tom Cruise on Chris Ume’s body and which effect was that it impressed a consequent part of the internet and went viral. When applied to politic speeches, debates or else from official, the impacts could be way more considerable.

AI is not the silver bullet – there’s still room for improvement

Artificial Intelligence is a solution for greater accuracy and efficiency in Content Moderation. Nonetheless, it must not be forgotten that there is still huge room for improvement, as well as growing challenges because of its development for malicious purposes. It is important for any social platform and online community to appreciate how central Artificial Intelligence is becoming in the Moderation field, as both a threat and an opportunity.

Reacting accordingly by getting the right combination of human moderators and technological solutions is in fact needed, as the possibility the impacts on real life and brand image it could generate might rapidly become overwhelming.