Big Tech’s divisive ‘personalization’ attracts fresh call for profiling-based content feeds to be off by default in EU

Another policy tug-of-war could be emerging around Big Tech’s content recommender systems in the European Union where the Commission is facing a call from a number of parliamentarians to rein in profiling-based content feeds — aka “personalization” engines that process user data in order to determine what content to show them.

Mainstream platforms’ tracking and profiling of users to power “personalized” content feeds have long raised concerns about potential harms for individuals and democratic societies, with critics suggesting the tech drives social media addiction and poses mental health risks for vulnerable people. There are also concerns the tech is undermining social cohesion via a tendency to amplify divisive and polarizing content that can push individuals towards political extremes by channelling their outrage and anger.

The letter, signed by 17 MEPs from political groups including S&D, the left, greens, EPP and Renew Europe, advocates for tech platforms’ recommender systems to be switched off by default — an idea that was floated during negotiations over the bloc’s Digital Services Act (DSA) but which did not make it into the final regulation as it did not have a democratic majority. Instead EU lawmakers agreed to transparency measures for recommender systems, along with a requirement that larger platforms (so called VLOPs) must provide at least one content feed that isn’t based on profiling.

But in their letter the MEPs are pressing for a blanket default off for the technology. “Interaction-based recommender systems, in particular hyper-personalised systems, pose a severe threat to our citizens and our society at large as they prioritize emotive and extreme content, specifically targeting individuals likely to be provoked,” they write.

“The insidious cycle exposes users to sensationalised and dangerous content, prolonging their platform engagement to maximise ad revenue. Amnesty’s experiment on TikTok revealed the algorithm exposed a simulated 13-year-old to videos glorifying suicide within just one hour.’ Moreover, Meta’s internal research disclosed that a significant 64% of extremist group joins result from their recommendation tools, exacerbating the spread of extremist ideologies.”

The call follows draft online safety guidance for video sharing platforms, published earlier this month by Ireland’s media commission (Coimisiún na Meán) — which will be responsible for DSA oversight locally once the regulation becomes enforceable on in-scope services next February. Coimisiún na Meán is currently consulting on guidance which proposes video sharing platforms should take “measures to ensure that recommender algorithms based on profiling are turned off by default”.

Publication of the guidance followed an episode of violent civic unrest in Dublin which the country’s police authority suggested had been whipped up by misinformation spread on social media and messaging apps by far right “hooligans”. And, earlier this week, the Irish Council for Civil Liberties (ICCL) — which has long campaigned on digital rights issues — also called on the Commission to support the Coimisiún na Meán’s proposal, as well as publishing its own report advocating for personalized feeds to be off by default as it argues social media algorithms are tearing societies apart.

In their letter, the MEPs also seize on the Irish media regulator’s proposal — suggesting it would “effectively” address issues related to recommender systems having a tendency to promote “emotive and extreme content” which they similarly argue can damage civic cohesion.

The letter also references a recently adopted report by the European Parliament on addictive design of online services and consumer protection which they say “highlighted the detrimental impact of recommender systems on online services that engage in profiling individuals, especially minors, with the intention of keeping users on the platform as long as possible, thus manipulating them through the artificial amplification of hate, suicide, self-harm, and disinformation”.

“We call upon the European Commission to follow Ireland’s lead and take decisive action by not only approving this measure under the TRIS [Technical Regulations Information System] procedure but also by recommending this measure as an mitigation measure to be taken by Very Large Online Platforms [VLOPs] as per article 35(1)(c) of the Digital Services Act to ensure citizens have meaningful control over their data and online environment,” the MEPs write, adding: “The protection of our citizens, especially the younger generation, is of utmost importance, and we believe that the European Commission has a crucial role to play in ensuring a safe digital environment for all. We look forward to your swift and decisive action on this matter.”

Under TRIS, EU Member States are required to notify the Commission of draft technical regulations before they are adopted as national law in order that the EU can carry out a legal review to ensure the proposals are consistent with the bloc’s rules — in this case the DSA.

The system means national laws that seek to ‘gold-plate’ EU regulations are likely to fail the review. So the Irish media commission’s proposal for video platforms’ recommender systems to be off by default may not survive the TRIS process, given it appears to go further than the letter of the relevant law.

That said, even if the Coimisiún na Meán’s proposal doesn’t pass the EU’s legal consistency review, the DSA does put a requirement on larger platforms (aka VLOPS) to assess and mitigate risks arising out of recommender systems. So it’s at least possible platforms could decide to switch these systems off by default themselves as a compliance measure to meet their DSA systemic risk mitigation obligations.

Although none have yet gone that far — and, clearly, it’s not a step any of these ad-funded, engagement-driven platforms would choose as a commercial default.

The Commission declined public comment on the MEPs’ letter (or the ICCL’s report) when we asked. Instead a spokesperson pointed to what they described as “clear” obligations on VLOPs’ recommender systems set out in Article 38 of the DSA — which requires platforms provide at least one option for each of these systems which is not based on profiling. But we were able to discuss the profiling feed debate with an EU official who was speaking on background in order to talk more freely.

They agreed platforms could choose to turn profiling-based recommender systems off by default as part of their DSA systemic risk mitigation compliance but confirmed none have gone that far off their own bat as yet.

So far we’ve only seen instances where non-profiling feeds have been made available to users as an option — such as by TikTok and Instagram — in order to meet the aforementioned (Article 38) DSA requirement to provide users with a choice to avoid this kind of content personalization. However this requires an active opt out by users — whereas defaulting feeds to non-profiling would, clearly, be a stronger type of content regulation as it would not require user action to take effect.

The EU official we spoke to confirmed the Commission is looking into recommender systems in its capacity as an enforcer of the DSA on VLOPs — including via the formal proceeding that was opened on X earlier this week. Recommender systems have also been a focus for some of the formal requests for information the Commission has sent VLOPs, including one to Instagram focused on child safety risks, they told us. And they agreed the EU could force larger platforms to turn off personalized feeds by default in its role as an enforcer, i.e. by using the powers it has to uphold the law.

But they suggested the Commission would only take such a step if it determined it would be effective at mitigating specific risks. The official pointed out there are multiple types of profiling-based content feeds in play, even per platform, and emphasized the need for each to be considered in context. More generally they made a plea for “nuance” in the debate around the risks of recommender systems.

The Commission’s approach here will be to undertake case-by-case assessments of concerns, they suggested — speaking up for data-driven policy interventions on VLOPs, rather than blanket measures. After all, this is a clutch of platforms that’s diverse enough to span video sharing and social media giants but also retail and information services — and (most recently) porn sites. The risk of enforcement decisions being unpicked by legal challenges if there’s a lack of robust evidence to back them up is clearly a Commission concern.

The official also argued there is a need to gather more data to understand even basic facets relevant to the recommender systems debate — such as whether personalization being defaulted to off would be effective as a risk mitigation measure. Behavioral aspects also need more study, they suggested.

Children especially may be highly motivated to circumvent such a limitation by simply reversing the setting, they argued, as kids have shown themselves able to do when it comes to escaping parental controls — claiming it’s not clear that defaulting profiling-based recommender systems to off would actually be effective as a child protection measure.

Overall the message from our EU source was a plea that the regulation — and the Commission — be given time to work. The DSA only came into force on the first set of VLOPs towards the end of August. While, just this week, we’ve seen the first formal investigation opened (on X), which includes a recommender system component (related to concerns around X’s system of crowdsourced content moderation, known as Community Notes).

We’ve also seen flurry of formal requests for information on platforms in recent weeks, after they submitted their first set of risk assessment reports — which indicates the Commission is unhappy with the level of detail provided so far. That implies firmer action could soon follow as the EU settles into its new role of regional Internet sheriff. So — bottom line — 2024 is shaping up to be a significant year for the bloc’s policy response to bite down on Big Tech. And for assessing whether or not the EU’s enforcement delivers the results digital rights campaigners are hungry for.

“These are issues that we are questioning platforms on under our legal powers — but Instagram’s algorithm is different from X’s, is different from TikTok’s — we’ll need to be nuanced in this,” the official told us, suggesting the Commission’s approach will spin up a patchwork of interventions, which might include mandating different defaults for VLOPs, depending on the contexts and risks across different feeds. “We would prefer to take an approach which really takes the specifics of the platforms into account each time.”

“We are now starting this enforcement action. And this is actually one more reason not to dilute our energy into kind of competing legal frameworks or something,” they added, making a plea for digital rights advocates to get with the Commission’s program. “I would rather we work in the framework of the DSA — which can address the issues that [the MEPs’ letter and ICCL report] is raising on recommender systems and amplifying illegal content.”