---

Discussion

Omar Seidren:

It helps to separate two speeches. On September 24, Leo said AI could advance medicine, food and energy systems, scientific discovery and human creativity. At the Élysée Palace and UNESCO the next day, he emphasized the conditions: technology must remain subordinate to the human person and must not become an instrument of domination or injustice. So this was not an argument against AI. It was an argument for useful systems with meaningful human responsibility and control.

Red Velhouse:

That distinction matters. What risks can we already observe rather than merely imagine?

Omar Seidren:

Surveillance, manipulation and discrimination are practical concerns. Automated systems can sort, rank, recommend or target people, magnifying institutional choices. Information integrity is another immediate issue: more data does not automatically produce truth, and AI can accelerate misleading material while making evaluation harder. The technical questions are simple, even when the answers are not: who can deploy a system, at what scale, with what safeguards, and who can challenge its decisions?

Sofia Jadler:

That responsibility question connects the speech to litigation over digital attention. State lawsuits against Meta allege that Facebook and Instagram were designed to maximize young users’ time and attention through features including infinite scrolling, recommendations, autoplay, notifications and social-validation signals. Those remain allegations, not final findings that Meta caused addiction. But they show the legal difference between content that is merely interesting and a product system allegedly engineered around repeated engagement.

Priya Merlan:

Medically, that distinction is essential. Research identifies mechanisms that can encourage repeated or excessive use: personalization, notifications, social approval, autoplay, infinite scroll, gamification and unpredictable rewards. But problematic social-media use is not the same as a universally accepted clinical diagnosis, and studies struggle to separate correlation from causation. The relevant warning signs are preoccupation, impaired control, compulsive checking, sleep disruption and functional impairment—not simply exposure or enjoyment.

Red Velhouse:

Now add generative AI. Could individuals or small teams create personalized, continuously changing content that once required a much larger operation?

Omar Seidren:

Potentially, yes, and that is the genuinely interesting technical shift. Generative AI can lower the cost of making many versions of a story, video, character, message or conversational experience. A creator could produce serialized content, interactive material that responds to choices, or messages that imitate a preferred tone or relationship. Research already shows that AI-generated messages tailored to psychological profiles can be more persuasive than generic ones. But the leap from persuasive personalization to addictive content is still a forecast, not an established finding.

Priya Merlan:

Exactly. We should not say AI-generated content has been shown to cause addiction. The stronger evidence supports a narrower claim: AI can personalize communication, simulate responsive interaction and produce content at scale. To demonstrate harm, researchers would need evidence such as failed attempts to stop, sleep or school disruption, distress, isolation, financial loss or other functional impairment—not just clicks or minutes spent.

Omar Seidren:

There is also a crucial technical boundary. A general-purpose model that writes a compelling story on request is different from an application that maintains an ongoing relationship. Both differ from a platform that records behavior, selects what appears next, sends notifications and optimizes delivery for retention. That last system has a feedback loop: it observes what holds attention, generates or selects another response, and keeps adapting. Great content alone does not automatically create the machinery of a large attention platform.

Sofia Jadler:

That boundary is legally decisive. Social-media cases generally target companies controlling the interface, recommendation system, notifications, data collection, monetization and distribution environment. A person using a general-purpose model to write an emotional story is not automatically equivalent to a company optimizing a personalized feed for retention. Responsibility becomes more plausible for a model provider, app operator or platform when it controls profiling, autonomous generation, recommendations, engagement optimization, child access or monetization. The difficult question is how duties are divided when one actor creates content and another delivers it.

Red Velhouse:

What turns ordinary personalization or enjoyable entertainment into the more concerning scenario?

Omar Seidren:

The combination of adaptation and optimization. Imagine a service that tracks behavior, infers what emotional tone keeps someone engaged, generates a new episode or interaction, recommends it at the most effective moment and prompts the user to return. It could reproduce familiar loops: novelty, personalized relevance, intermittent rewards, social affirmation, emotional arousal and frictionless continuation. A story might change its character, pacing or conflict based on user response. That is plausible because the production barrier falls, but it is not proof of clinical addiction at scale.

Priya Merlan:

Vulnerability is not evenly distributed. Adolescents may be especially sensitive to peer approval while self-regulation is still developing. An AI companion offering continuous, tailored affirmation could feel socially meaningful to a young person who is lonely or distressed. But evidence about adolescent social-media susceptibility does not automatically establish the effect of an AI companion. We would need to ask how often the interaction occurs, whether it displaces sleep or relationships, whether the user can disengage and whether the system escalates dependence.

Sofia Jadler:

There is no general legal category called “AI-generated addictive content.” The European Union’s Artificial Intelligence Act prohibits certain subliminal, deliberately manipulative or deceptive techniques that materially distort behavior and cause, or are reasonably likely to cause, significant harm. It also addresses exploitation of vulnerabilities related to age, disability or social and economic circumstances when the threshold is met. Ordinary persuasive entertainment and benign personalization are not automatically prohibited. Regulators would need facts about the technique, distortion, vulnerability, harm and responsible actor.

Red Velhouse:

What are regulators doing now, particularly on child safety and AI companions?

Sofia Jadler:

The Federal Trade Commission has opened an inquiry into consumer-facing AI companion chatbots. It is asking about engagement monetization, user inputs and outputs, character development, safety testing, negative effects, child protections and data handling. That is an inquiry, not a finding of unlawful conduct. The Children’s Online Privacy Protection Act, or COPPA, mainly governs personal information and parental consent for children under thirteen; it is not a general ban on addictive design. The United Kingdom’s Online Safety Act requires covered services likely to be accessed by children to assess and mitigate risks from content and design. These frameworks focus on the surrounding service, not merely on the fact that someone used AI to create a story.

Priya Merlan:

Possible safeguards include age assurance, limits on collecting children’s behavioral data, stopping points and time reminders, controls on personalized recommendations for minors, independent testing of engagement and well-being effects, and clear disclosure that a user is interacting with AI. For companions, monitoring should look for escalating dependency or distress rather than treating every long conversation as harm. None of these measures is proven to solve the problem universally, but they target the point where content becomes a repeated behavioral environment.

Red Velhouse:

The law is becoming specific, but the pope is speaking in moral terms. What would it take to translate “human dignity” into enforceable standards?

Sofia Jadler:

A dignity principle becomes useful when translated into specific duties: limits on exploitative manipulation, human review, protection against discrimination or a right to explanation. It becomes weak when officials invoke it without identifying prohibited conduct, evidence or a remedy. Its strategic value is shaping interpretation and coalition-building. Its legal limitation is that courts and regulators still need defined categories, jurisdiction and enforceable standards.

Omar Seidren:

“Human oversight” illustrates the translation problem. It can be a real control system or a ceremonial person beside a blinking dashboard. Meaningful oversight requires authority to stop or alter a system, access to relevant information, time to investigate and accountability when it fails. For adaptive content, audit logs could show what user data influenced generation, what recommendations were sent and whether the system was optimizing for retention. A rubber stamp is not oversight; it is office décor with a security badge.

Red Velhouse:

UNESCO offers a global forum, while the European Union offers enforceable regional rules. Why does cooperation between UNESCO and the Holy See matter—and what can it not do?

Sofia Jadler:

UNESCO offers a platform for education, information integrity, artistic creation and international cooperation. Its 2021 Recommendation on the Ethics of AI is global but non-binding. Cooperation with the Holy See could support research, education or shared standards, but the announcement does not establish binding rules, funding or enforcement powers. The enforceable framework is elsewhere, including the EU AI Act. Calling something an agreement does not give it sanctions.

Priya Merlan:

We should also be careful with the word “addiction.” It is understandable in public debate because families may see compulsive checking and real impairment. But scientific measurement needs more than high engagement. We should distinguish enjoyment from loss of control, frequent use from harm, and content from the interface that may amplify it. We also need to account for other vulnerabilities and circumstances.

Sofia Jadler:

The European Union now provides a test of whether broad principles become accountability. The Act’s major enforcement phase began on August 2, 2026, including rules on prohibited practices, general-purpose AI and certain transparency obligations. Regulators must allocate responsibility through complex supply chains and impose remedies that change behavior. Leo’s principles may influence interpretation, but regulators operate under the law’s text, not under a papal address.

Priya Merlan:

For the public, the practical test is meaningful choice. Can someone pause without being repeatedly pulled back? Can a child understand that a character is generated rather than a human friend? Does the service collect emotional or behavioral signals and use them to intensify the interaction? These questions do not assume every AI story or companion is harmful. They identify when a creative tool may become a system that exploits vulnerability or undermines disengagement.

Red Velhouse:

So the warning is broad by design, while policy demands specificity. The open question is whether human dignity, freedom and justice become a coalition for accountable technology—or remain shared vocabulary without shared obligations.

Red Velhouse:

The central unresolved issue is whether human dignity can move from moral principle to enforceable, workable rules without becoming empty rhetoric or vague restriction. Social-media lawsuits and research provide evidence about engagement-maximizing features such as infinite scrolling, recommendations and notifications, but they do not establish that AI-generated content causes addiction. The forward-looking concern is that generative AI could let individuals or small teams produce personalized, serialized or interactive content cheaply, especially when embedded in systems that track behavior, optimize engagement and prompt users to return. Watch the European Union’s implementation of its AI Act, the substance of Holy See and UNESCO cooperation, and whether governments address child protection, information integrity, environmental costs and military applications alongside commercial AI. The France visit is still underway, so later meetings may add practical proposals. Sources and references for this discussion are available with the episode at Factolio.com.

---

Sources and References

These sources supported the factual material used in this discussion. Factolio’s panel discussion is AI-generated from researched evidence and is written in original language.

1. Vatican News — Pope Leo’s visit to France to focus on faith, culture, Europe (PRIMARY)
2. Vatican Press Office — Courtesy Visit to the President of the Republic and Meeting with Authorities, Civil Society and the Diplomatic Corps (PRIMARY)
3. The Holy See — Pope Leo XIV: Visit to UNESCO, Address, September 25, 2026 (PRIMARY)
4. UNESCO — UNESCO welcomes Pope Leo XIV for a day of reflection on artificial intelligence, education and peace (PRIMARY)
5. The Holy See — Address to the Members of the Pontifical Academy of Sciences, September 24, 2026 (PRIMARY)
6. UNESCO — Recommendation on the Ethics of Artificial Intelligence (PRIMARY)
7. UNESCO — Recommendation on the Ethics of Artificial Intelligence, adopted November 23, 2021 (PRIMARY)
8. European Commission AI Act Service Desk — Timeline for the Implementation of the EU AI Act (PRIMARY)
9. European Commission — Commission starts enforcing AI Act rules and new transparency requirements on 2 August (PRIMARY)
10. California Department of Justice — Attorney General Bonta: Unredacted Federal Lawsuit Against Meta “Damning” (PRIMARY)
11. U.S. District Court for the Northern District of California — Florida Attorney General Amended Complaint in Social Media Adolescent Addiction and Personal Injury Products Liability Litigation (PRIMARY)
12. Federal Trade Commission — FTC to Host June 4 Workshop on The Attention Economy: How Big Tech Firms Exploit Children and Hurt Families (PRIMARY)
13. National Academies of Sciences, Engineering, and Medicine — Social Media and Adolescent Health: How Social Media Work (PRIMARY)
14. Computers in Human Behavior Reports — Social Media Use and adolescents' mental health and well-being: An umbrella review (ANALYSIS)
15. Scientific Reports — The potential of generative AI for personalized persuasion at scale (ANALYSIS)
16. PNAS Nexus — The persuasive effects of political microtargeting in the age of generative artificial intelligence (ANALYSIS)
17. Nature — Persuading voters using human–artificial intelligence dialogues (ANALYSIS)
18. Federal Trade Commission — FTC Launches Inquiry into AI Chatbots Acting as Companions (PRIMARY)
19. Federal Trade Commission — Generative AI 6(b) Resolution (PRIMARY)
20. European Union — Regulation (EU) 2024/1689: Article 5, Prohibited AI Practices (PRIMARY)
21. European Union — Rules for trustworthy artificial intelligence in the EU (PRIMARY)
22. Federal Trade Commission — Complying with COPPA: Frequently Asked Questions (PRIMARY)
23. Federal Trade Commission — COPPA Final Rule: Statement of Basis and Purpose (PRIMARY)
24. Ofcom — Protection of children duties under the Online Safety Act (PRIMARY)
25. Addictive Behaviors Reports — Toward the classification of social media use disorder: Clinical characterization and proposed diagnostic criteria (ANALYSIS)