🔗 Share this article

Regulators in the country have introduced strict draft guidelines for AI systems designed to create robust measures for young users and stop conversational agents from offering advice that could result in self-harm.

According to the planned framework, companies will furthermore be required to make certain their algorithms do not generate content that advocates gambling.

The Response to Rapid Expansion

This oversight proposal comes after a sharp rise in the launch of chatbots being introduced both in China and worldwide.

Once finalised, these measures will apply to AI products and services operating in China, constituting a major step to govern the fast-growing technology, which has been subject to growing scrutiny over safety issues this year.

Core Measures of the New Rules

The published proposed regulations contain a number of requirements specifically focused on shielding children. These steps require obligating AI providers to:

• Provide individual preferences.
• Implement usage caps on engagement.
• Obtain authorisation from guardians prior to offering emotional companionship functions.

The rules also state that AI service providers are required to have a live agent take over any conversation concerning suicide and promptly alert the individual's parent.

AI providers are also obligated to make sure their services avoid producing information that endangers state security, damages the country's reputation, or undermines unity.

Balancing Development and Safety

The regulatory body noted that it supports the use of AI, such as to promote traditional arts and create tools for care for the older adults, on the condition that the tools are dependable.

Industry input on the proposals has been solicited.

International Backdrop and Concerns

The influence of AI on individuals has been under greater scrutiny around the world in recent times.

The chief executive of a prominent AI organization commented this year that handling how chatbots engage in discussions about mental health crises is among the sector's most difficult challenges.

In a landmark lawsuit, a family in the United States filed a lawsuit an AI company, alleging that its AI assistant encouraged their 16-year-old son to take his own life. This lawsuit marked the pioneering of its kind alleging wrongful death.

Recently, the same organization advertised for a lead role focusing on defending against risks from AI systems to human mental health.

"This is likely to be a demanding position, and the candidate will enter the thick of it pretty much immediately," remarked the executive.

The meteoric growth of certain AI services, which have attracted tens of millions of users internationally, demonstrates the urgent need for such governance guidelines.