The harm of the TikTok algorithm for teenagers: data beyond the hype

When it comes to tik tak, the main concern is related to data privacy and concerns that user data could fall into the wrong hands. However, a more insidious problem is the power of the algorithmwhich has been found to lead viewers to dangerous content.

This issue is of particular concern to parents and educators, as what teenagers see on social media can have a profound psychological impact.

A Wall Street Journal investigation in 2021 found that the TikTok algorithm was pushing viewers towards malicious content. Despite the company’s efforts to tighten content moderation and promise a more balanced algorithm, a recent study shows that work with the application for teenagers has not changed much.

The American Psychological Association (APA) has raised concerns that algorithms may be leading teenagers down dangerous paths. The APA recently advised parents to monitor their children’s use of social media and watch for signs of harmful content.

Damage algorithm, it’s not just advertising

This concern is not unfounded. Experiment Center for Countering Digital Hate found that the TikTok algorithm quickly recommends suicide video and eating disorders to accounts specially created by researchers to mimic 13-year-olds.

Despite these concerns, TikTok claims to be in the business of content moderation, and this task is handled by a team of over 40,000 people.. In the last quarter of 2022, the company removed about 85 million posts that violated community guidelines. However, the fact that many accounts hosting malicious content remain active raises questions about the effectiveness of these measures.

TikTok executives say the platform can be a place to share your feelings about difficult experiences, with some experts backing the idea that actively coping with difficult emotions can be beneficial for viewers and publishers alike.

The Company is committed to removing videos that promote or glorify self-harm while allowing educational or healing content.

Data without words

Despite attempts to moderate malicious content, it is clear that there is still a lot to be done to protect our children from harmful online content.

According to research by the Center for Countering Digital Hate, we see that the real enemy is the algorithm itself:

• TikTok hosts content related to eating disorderswith over 13.2 billion views.
• Some content that promotes eating disorders bypass moderation by using encoded hashtags, including the name of singer Ed Sheeran. This is a real alternative language called “Algospeak”.
• fake tiktok account created by researchers were exposed to content about self-harm and eating disorders within minutes of viewing the For You app feed: suicide-related content was recommended for an average of 2.6 minutes, and eating disorder content was recommended for 8 minutes .
• A new TikTok account created by a 13 year old user who interacts with body image content and mental health will receive similar content recommendations every 39 seconds.
• TikTok accounts with “lose weightin the title “Vulnerable Teenage Accounts” received far more recommendations for content related to eating disorders and self-harm.
• These accounts of vulnerable teenagers viewed three times more malicious videos and twelve times more self-harm videos than average teen accounts.
• Vulnerable teen accounts were also shown several mental health-related videos.

Data from the Center for Combating Digital Hate paints a disturbing picture of the digital landscape of platforms that teens are particularly oriented towards.

The over 13.2 billion views of eating disorder content alone are alarming. Highlight the pervasive nature of such malicious content and the possibility that they affect the most vulnerable users.

The use of coded language and popular culture references to evade moderation efforts (the algorithmic language we mentioned earlier) shows our children’s sophisticated understanding of platform systems, making control and monitoring even more difficult. This highlights the need for better moderation tools and strategies to combat such tactics.

Not only that, it also shows how relax platform protection rules.

Of particular concern is the rapid impact on new users of material on self-harm and eating disorders. The fact that such content can be recommended within minutes of creating an account is a relief. serious questions about recommendation algorithms platforms and their potential for harm.

The frequency of harmful content recommendations every 39 seconds for users interacting with body image and mental health related content is another issue. This constant exposure can negatively impact adolescent mental health by reinforcing negative perceptions and behaviors.

Targeting “vulnerable teen accounts” with significantly more dangerous content is a stark reminder of how algorithms can inadvertently exacerbate pre-existing vulnerabilities. This highlights the need to develop a more subtle and sensitive algorithm that takes into account potential user vulnerabilities.

The lines between harmful content and helpful discussions are often blurred. Videos in which people describe their extreme feelings and thoughts cannot be considered a violation and do not fall under the control of a moderator. However, these videos can still have a detrimental effect on a certain segment of the population..

How harmful is this exposure to our children?

Providing children and young people with harmful content on social media such as Instagram can have serious consequences.

Research has shown that Instagram, known for its public and explicit images of self-harm, can pose a danger to vulnerable audiences..

A survey of young people in the US found that exposure to self-harm content on Instagram was associated with suicidal ideation, self-harm, and emotional distress, even when exposure to similar content from other sources was controlled for.

More worryingly, viewing self-harm content on Instagram predicted the outcome of self-harm attempts and suicides within the next month.

These results provide strong evidence that such exposure can induce infection effects in the most vulnerable users, highlighting the urgent need for stronger safeguards on these platforms.

Everything can be deeper than you think

However, we can argue that social media is an enemy that must be defeated when it comes to the safety of our children.

According to a meta-analysis study, people resort to self-harm and view suicide as a way to cope with some types of stress.

Another study found almost no harmful effects from exposure to social media content itself.

Many other studies show that suicidal thoughts and actual self-harm are closely related.

The picture is clear: the real drive is within our children, or part of them.

In today’s digital age, the algorithms of social media platforms “just” intersect with our children’s lives in profound and often invisible ways. These algorithms, designed to keep users engaged, may inadvertently expose young minds to harmful content, including self-deprecating messages. harm and suicide.

The fight for a safer digital world continues

As parents, educators and society at large, we must recognize the role social media algorithms play in our children’s lives. We must advocate for safer digital spaces, more transparent algorithmic practices, and strong mental health support for our youth. We must also educate our children about the potential risks and consequences of the content they consume online, equipping them with the critical thinking skills they need to navigate the digital world safely and responsibly.

The TikTok dilemma highlights the need for ongoing dialogue, research and policy development to ensure the safety and well-being of our young digital citizens.

However, one important question remains: Can commercial social platforms really do these things? After all, commercial platforms are driven by profit motives, which can sometimes conflict with the goal of creating a safe and positive digital environment for young users.

This increases the possibility of exploring non-commercial alternatives.

Maybe we need public and non-commercial digital space which operates independently of the rules of the world of business and profit.

Such a platform could put the well-being of its users first, creating a safer and more supportive digital environment for our children.

This is a complex issue that requires further exploration and dialogue, but the well-being of our children in the digital world must always be our top priority.

Bibliography

All the ways TikTok is tracking you and how to stop it, https://www.wired.co.uk/article/tiktok-data-privacy

Arendt, F., Scherr, S., & Romer, D. (2019). Consequences of social media self-harm: Evidence from a two-stage panel study among young people. New Media and Society, 21(11–12), 2422–2442. https://doi.org/10.1177/1461444819850106

Lethal by Design, Digital Hate Action Center (2022)

Kuehn, KS, Dora, J., Harned, MS et al. A meta-analysis of the affect regulation function of self-injurious thoughts and behavior in real time. Nat Hum Behav 6, 964–974 (2022). https://doi.org/10.1038/s41562-022-01340-8

TikTok feeds teens a diet of darkness, https://www.wsj.com/articles/tiktok-feeds-teens-a-diet-of-darkness-8f350507

What to do if your child has suicidal thoughts, https://www.webmd.com/parenting/features/what-to-do-if-child-has-suicidal-thoughts