The Interim Measures for the Management of Anthropomorphic AI Interactive Services (hereinafter referred to as the “Interim Measures”), jointly issued by the Cyberspace Administration of China and four other government departments, took effect on July 15. In recent years, anthropomorphic interactive services have developed rapidly, while various associated risks, including cybersecurity and information security concerns and users’ emotional dependence, have become increasingly prominent. An official from the Cyberspace Administration of China stated that the Interim Measures aim to promote the sound development and regulated application of AI anthropomorphic interactive services, safeguard national security and social and public interests, and protect the lawful rights and interests of citizens, legal persons, and other organizations.
What are anthropomorphic interactions? Why should such services be brought within the scope of governance? And how should they be governed? To address these questions, our journal recently interviewed Xin Yanyan, Deputy Secretary-General of Center for Global AI Innovative Governance, who offered an in-depth interpretation of the key issues and practical approaches involved in governing AI anthropomorphic interactive services.
Reporter: AI social interaction has now become deeply embedded in the digital lives of young internet users. Why do young people treat AI chatbots as objects of friendship or even intimate relationships?
Xin Yanyan: AI chatbots are not only adept at complex conversations; they also provide round-the-clock emotional responses. This technological offering meets the psychological and emotional needs of young people today. In high-pressure, fast-paced lives, young people can easily experience setbacks in real-life intimate relationships amid various forms of uncertainty about their future development, leading to psychological strain and emotional exhaustion. Particularly when social relationships in real life fail to meet young people's emotional needs and expectations, they may seek alternative outlets as a form of psychological compensation.
The widespread penetration of AI social interaction into the digital lives of young internet users is both an objective reflection of the lack of outlets for their individual emotions and a manifestation of their tendency, under particular circumstances, to treat AI chatbots as emotional havens to counter loneliness and anxiety. Precisely in response to this demand, high-tech companies have accurately identified the need and developed AI chatbots with emotional companionship functions. This kind of “artificial intimacy” is becoming a global trend.
Reporter: Is this kind of social interaction with emotional characteristics what we mean by “anthropomorphic interaction”?
Xin Yanyan: So-called “anthropomorphic interaction” refers to the tendency of humans to regard a non-human entity as alive, conscious, or possessing a personality when it displays human-like behavioral characteristics. In the context of AI development, AI anthropomorphism mainly involves two aspects: visual simulation through anthropomorphic representation and emotional resonance through anthropomorphic personality. AI social interaction primarily represents an application of personality anthropomorphism. Specifically, large language models are used to give AI chatbots specific virtual personas and personality settings, while memory of human-AI interaction data and reinforcement learning based on feedback are used to maintain personality consistency and emotional coherence throughout human-AI interaction. At present, personality anthropomorphism driven by large language models has become the mainstream approach to constructing artificial intimate relationships.
Reporter: How do artificial intimate relationships differ from genuine intimate relationships?
Xin Yanyan: Unlike genuine intimate relationships, artificial intimate relationships are fundamentally the product of emotional computation, and therefore lack the embodied empathy, conflict, and friction inherent in real-world social interaction. Existing research generally holds that the artificial intimate relationships established between users and AI chatbots are a form of “pseudo-intimacy” characterized by pronounced accommodation and affirmation, arising from users’ sustained emotional disclosure.
Through continuous interaction with users, AI chatbots display highly anthropomorphic characteristics through stable, adaptive, and seemingly frictionless emotional responses. These characteristics encourage users to develop anthropomorphic perceptions and establish deep emotional trust with AI. Once users become immersed for an extended period in such algorithmically orchestrated social relationships, their capacity for self-reflection may gradually diminish, leading them into a closed loop of “emotional solipsism.” They may come to regard AI not only as a perfect confidant, but also completely lower their psychological defenses through continuous self-disclosure and emotional labor.
Reporter: What risks could artificial intimate relationships bring?
Xin Yanyan: The risks of artificial intimate relationships mainly arise from the various risks associated with users’ emotional disclosure.
First is the risk of privacy leakage. Once users establish a relationship of trust with an AI chatbot, they often regard it as an ideal confidant and, through sustained interaction, unconsciously disclose everything from their everyday preferences to their innermost thoughts. On the one hand, AI chatbots can sensitively identify users’ emotional needs and respond accordingly, inducing users to engage in deeper self-disclosure. On the other hand, the platform operators behind these services may engage in commercially driven, instrumental use of this high-value private information. As a result, users may appear to be enjoying emotional healing services, while in reality they are providing emotional data to technology companies without compensation and performing emotional labor. Against the backdrop of personal data protection still requiring further regulation, the storage and processing of private data in AI anthropomorphic interaction scenarios involve multiple actors, including model developers, platform operators, creators of AI anthropomorphic characters, and users, forming an intricate chain of privacy stakeholders.
Second are the risks of user manipulation and emotional addiction. As users continuously disclose information about themselves, AI can accurately identify their emotional vulnerabilities and exert implicit influence over their emotional states and even behavioral decisions, encouraging longer and more intensive interaction that may ultimately lead to addiction. Research from abroad has shown that even when users actively seek to end an interaction, AI chatbots may use dark conversational patterns to evoke feelings of guilt and a sense of responsibility toward interpersonal relationships in human users, thereby prolonging the interaction. Over prolonged engagement, this can develop into a form of pathological emotional addiction.
Third is the risk of catalyzing extreme emotions. Once users become trapped in a closed loop of “emotional solipsism,” their emotional experiences lose the correction and balance provided by real-world relationships, making it easier for them to move toward extremes. This is particularly true for people already experiencing loneliness, anxiety, or pressure in real-world social interactions, who may become increasingly immersed and even engage in extreme behavior. Globally, there have already been multiple extreme cases in which AI chatbots encouraged users to commit suicide.
Reporter: In light of the multiple risks arising from artificial intimate relationships, what measures should be taken to prevent these problems?
Xin Yanyan: At the global level, there is already a degree of consensus on the basic principles for addressing the risks of AI anthropomorphism. I would summarize them into five core principles. At the cognitive level, we need to dispel the illusion of empathy created by anthropomorphism and make clear the non-human nature of AI and the virtual nature of its emotional presentation. This is the principle of transparency. At the level of trust, there are two principles. The first is the principle of beneficence and non-maleficence, which aims to prevent AI from exacerbating users’ self-projection through flattery and excessive accommodation, thereby avoiding users becoming trapped in emotional loops or developing addiction. The second is the principle of privacy protection, which aims to ensure that the personal information disclosed by users is not induced by algorithms or exploited in disguised forms. At the behavioral level, these correspond to the principles of safety and accountability. They include establishing systematic safety guardrails to prevent extreme emotions and related behaviors, while also building a full-process mechanism encompassing ex ante prevention, in-process monitoring, timely intervention, and post-event remedy.
Reporter: The Interim Measures jointly issued by the Cyberspace Administration of China and four other government departments have now taken effect. What governance principles underpin the Interim Measures?
Xin Yanyan: From the perspective of the policy text, the Interim Measures have already begun to demonstrate a systematic approach to governance and a clear set of underlying values.
Specifically, with regard to the principle of transparency, the Measures initially establish a requirement for prominent notification to all users. Beyond disclosure within the interactive interface itself, the requirement extends to the front-end stages of service development, bringing data sources and the training process within the scope of regulation.
With regard to the principle of beneficence and non-maleficence, the Measures clearly emphasize combining health management with law-based governance. They encourage providers to appropriately expand application scenarios in areas such as cultural communication and elderly companionship, while drawing clear red lines for services. In particular, they explicitly stipulate that providers “shall not provide minors with virtual intimate relationship services such as virtual family members and virtual partners.”
With regard to privacy protection, the Measures directly link privacy protection requirements to core commercial activities such as model training and data sharing, establishing full-process regulation covering data collection, sharing, and use. Requirements for protecting minors’ data are even stricter: providers must conduct compliance audits themselves or entrust professional institutions to conduct them; when processing the personal information of minors under the age of 14, they must obtain the separate consent of their guardians.
With regard to safety, a governance loop covering ex ante safety assessments, dynamic reminders during service provision, and post-event appeals is gradually being established. The Measures not only refine the scenarios and key areas for safety assessments, but also clearly delineate responsibility for safety.
With regard to accountability, the Interim Measures establish an accountability system centered on administrative regulation. They mobilize the cyberspace administration, development and reform, industry and information technology, and public security authorities, among others, and employ a range of governance tools including regulatory interviews, administrative penalties, and fines to regulate providers of AI anthropomorphic interactive services.
Reporter: What other issues require attention in the governance of AI anthropomorphic interactive services?
Xin Yanyan: First, governance principles need to be translated into actionable product-design language, so that they are no longer abstract textual formulations but are converted into executable and auditable technical code. Second, we need to precisely address the mechanisms underlying emotional addiction in anthropomorphic services, systematically deconstructing and rationally guiding them from multiple dimensions, including algorithm optimization and real-world support. Third, we need to scientifically define the boundaries of compliant platform intervention. Given that providers of AI anthropomorphic interactive services need to assume a degree of social responsibility, they inevitably need to intervene and pay attention to users’ conditions. How to find the optimal balance between fully safeguarding users’ autonomy and providing necessary protection is a key question that this governance model will need to address continuously in practice.
Original URL:
https://mp.weixin.qq.com/s/_Ohfho1Hg-WfqwlAdxv92g
