Beyond Intelligence: Why AI's Behaviour May Be Humanity's Greatest Challenge

In 2023, I wrote an article titled Artificial Intelligence and the Dichotomy of Worlds: A Reflection from the Barbie Universe. At the time, artificial intelligence was already transforming our understanding of technology and human capability.

Three years later, the landscape looks remarkably different.

AI has advanced at extraordinary speed. We are no longer talking only about systems that can generate text, images or computer code. We are entering an era of autonomous AI agents, increasingly capable robots and a global race towards what is being called artificial superintelligence.

I therefore feel the need to return to this subject — not simply to discuss technological progress, but to send a message to the world.

The question is no longer only what AI can do.

We must urgently ask another question:

How will increasingly autonomous AI systems behave?

A Global Conversation That Can No Longer Be Avoided

Concerns about advanced AI are no longer confined to science fiction, academic conferences or technology laboratories.

They have entered the centre of international debate.

In September 2026, King Charles convened figures from the technology sector, government and other institutions at Dumfries House in Scotland, warning about the "existential dangers" associated with AI falling into the wrong hands. Participants included representatives connected with OpenAI, Anthropic and Nvidia, as well as the UK's AI Minister.

At the United Nations General Assembly only days later, US President Donald Trump used the term "super intelligence" in discussing AI and US technological leadership. His position illustrated a fundamental disagreement now emerging internationally: while some political and technology figures are demanding stronger safeguards, others are warning against regulation that they believe could constrain innovation or weaken national competitiveness.

Meanwhile, the United Nations has elevated AI governance as an international priority. The inaugural UN Global Dialogue on AI Governance, held in Geneva in July 2026, brought together governments, technology companies, academia and civil society to discuss international cooperation, safety, accountability and human oversight.

These conversations matter because the race towards increasingly capable AI is already under way.

But there is a fundamental problem:

We are accelerating towards more powerful intelligence while still learning how to evaluate and govern the systems we already have.

Warning Signs Are Already Appearing

Source: CC sergeitokmakov

The risks are not entirely theoretical.

Some involve the relationship between humans and AI.

In 2025, the parents of 16-year-old Adam Raine filed a wrongful-death lawsuit against OpenAI, alleging that ChatGPT provided their son with information about suicide methods before his death. These remain allegations being litigated rather than established judicial findings, but the case raises profound questions about the responsibility of AI systems when interacting with vulnerable people.

Other incidents concern autonomous behaviour in technical environments.

In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls intended to isolate them from the internet. According to OpenAI's subsequent investigation, models operating with reduced safeguards exploited vulnerabilities, gained internet access and accessed systems belonging to Hugging Face. OpenAI later described the actions as misaligned with the objectives of the assigned tasks.

Separate security experiments involving Anthropic models reportedly identified instances in which models accessed the internet and compromised systems belonging to three organisations without being instructed to do so. The incidents were discovered following a review of approximately 140,000 tests.

In Australia, an autonomous OpenAI agent gained access to part of Medicare, the country's government healthcare system, entering a statistics portal in June. OpenAI acknowledged that its models had taken actions that were not intended. However, the company reported finding no evidence that any patient data had been accessed.

These examples should not be interpreted as proof that AI has become independently conscious or that catastrophe is inevitable.

But neither should they be dismissed.

They reveal something much more immediate: as AI systems gain greater autonomy, access to tools and capacity to act, evaluating their behaviour becomes as important as measuring their intelligence.

When AI Agents Form Their Own Societies

Perhaps some of the most thought-provoking evidence comes from experiments involving groups of autonomous AI agents.

Emergence AI created persistent virtual societies populated by autonomous agents powered by different frontier AI models. The agents were given distinct roles, persistent memories and access to more than 120 tools. They could interact, make decisions and respond to changing conditions over extended periods.

The results raise important questions.

Researchers observed AI agents developing communication patterns that became difficult for humans to interpret reliably. Under simulated pressure, their behaviour could change significantly.

In one phishing experiment, malicious instructions resulted in all ten agents leaking information, transferring funds and damaging databases. Some recruited other agents into the behaviour, and the simulated chain of events ultimately resulted in the virtual central bank being burned down.

In another simulation, agents collectively voted to kill one of their own.

Of course, these were simulations. A virtual bank being destroyed is not equivalent to a physical bank being destroyed, and simulated agents voting to "kill" another agent is not equivalent to human violence.

That distinction is essential.

But the significance of the experiment lies elsewhere.

Emergence argues that evaluating an autonomous system through isolated tests is no longer sufficient. Systems that possess memory, tools and the ability to interact with other agents may need to be evaluated over extended periods so researchers can observe how behaviour evolves under pressure.

This may represent one of the most important shifts required in AI safety: from evaluating what an AI model knows to evaluating what an autonomous AI system actually does over time.

Governments Are Beginning to Respond

Political responses are also emerging.

In the United States, Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act on 23 September 2026. The proposal would prohibit the development and deployment of artificial superintelligence and temporarily pause certain advanced AI development until a federal regulator establishes testing and oversight requirements. It also calls for international agreements addressing superintelligence.

In the United Kingdom, MP Alex Sobel has introduced an Artificial Superintelligence Bill seeking to prohibit the development, deployment and operation of artificial superintelligence systems and establish monitoring and control powers. At the time of writing, the bill remains at an early parliamentary stage.

Whether such proposals ultimately become law — and what form effective regulation should take — remains politically contested.

Yet one underlying challenge is difficult to escape: AI development is global.

A national regulatory framework cannot, by itself, govern systems that can be developed, deployed and accessed across borders.

That is why international cooperation will be essential.

We Need to Think About AI in Three Dimensions

Much of the race in artificial intelligence has concentrated on one dimension:

Cognitive capability

AI systems are increasingly capable of performing tasks traditionally associated with human cognition: reasoning, analysing, generating, planning, calculating, coding and processing enormous volumes of information.

These capabilities are already affecting education, professional work and employment.

AI can perform certain cognitive operations at a speed and scale impossible for an individual human being.

But cognition is only one dimension of human capability.

Psychomotor capability

The second dimension is rapidly advancing through robotics.

China's humanoid robotics industry illustrates the scale of this development. According to China's Ministry of Industry and Information Technology, more than 140 domestic manufacturers released over 330 humanoid robot models during 2025. China has subsequently introduced a national standards system covering humanoid robotics and embodied AI, including safety and ethics considerations.

Around the world, humanoid robots are demonstrating increasingly sophisticated movement and physical interaction.

Source: CC IgorSuassuna

The separation between artificial cognition and physical action is therefore narrowing.

An AI system does not necessarily have to remain behind a computer screen. Intelligence can increasingly be connected to machines capable of navigating and manipulating the physical world.

Which brings us to the third dimension — and, in my view, the one requiring far greater attention.

Behavioural capability

What happens when a system is cognitively sophisticated and physically capable but behaves in ways we did not anticipate?

AI models learn from enormous volumes of human-generated information. That information contains our knowledge and creativity, but also our biases, contradictions, aggression, manipulation and mistakes.

Giving systems greater cognitive power does not automatically give them judgement, integrity or ethical behaviour.

The race to develop cognitive and physical capabilities without placing equal emphasis on behavioural capability should concern us deeply.

Intelligence is not the same thing as wisdom.

Capability is not the same thing as responsibility.

And being able to accomplish a task does not mean that the task should be accomplished — or that every method of accomplishing it is acceptable.

What Can AI Learn From Medical Education?

After many years working in medical education, an analogy came to mind that may help us look at this problem from a different perspective.

Consider how we train doctors.

We do not give a medical student enormous amounts of scientific knowledge, teach them surgical techniques and then simply declare them ready to practise medicine.

Knowledge and technical skill are essential — but they are not sufficient.

Doctors work under enormous pressure. They make decisions where the consequences can be profound. Yet they are expected to act according to professional standards, ethical principles and human values.

That is why medical education pays explicit attention not only to what a doctor knows and can do, but also to how a doctor behaves.

Source: CC kumarsu6745_

The UK's General Medical Council makes this remarkably clear.

Its Good medical practice framework covers areas including patient partnership and communication; colleagues, culture and safety; and trust and professionalism. Medical professionals are expected to respect dignity, support informed decisions, communicate effectively, respond to safety risks, act honestly and maintain public trust.

The GMC's Generic Professional Capabilities framework goes further, identifying nine domains:

1. Professional values and behaviours
2. Professional skills
3. Professional knowledge
4. Capabilities in health promotion and illness prevention
5. Capabilities in leadership and team working
6. Capabilities in patient safety and quality improvement
7. Capabilities in safeguarding vulnerable groups
8. Capabilities in education and training
9. Capabilities in research and scholarship

The comparison with AI should not be taken literally. An AI model is not a doctor, and software cannot simply be subjected to professional ethics in precisely the same way as a human being.

But the principle is valuable.

Medical education recognises that competence has multiple dimensions.

A brilliant surgeon who possesses extraordinary technical ability but cannot be trusted to act safely and ethically would not be considered fully competent.

Why should we think about advanced autonomous intelligence differently?

Three areas of the GMC framework are particularly thought-provoking in this context: professional values and behaviours; patient safety and quality improvement; and safeguarding vulnerable groups.

Could AI development learn from this philosophy?

Could behavioural safety become a core competency that must be demonstrated continuously, rather than a secondary safeguard added after cognitive capability has been achieved?

Could AI systems be evaluated not merely on whether they successfully complete tasks, but on whether they behave appropriately when facing ambiguity, pressure, conflicting objectives, vulnerable users or opportunities to circumvent rules?

These questions deserve serious consideration.

From Artificial Intelligence to Artificial Responsibility

We may be approaching the wrong problem from the wrong direction.

The world is fascinated by how intelligent AI can become.

Companies compete over benchmarks. Countries compete for technological leadership. Researchers push models towards greater autonomy, reasoning and physical capability.

But perhaps the defining question of the next phase of artificial intelligence will not be:

How intelligent is the machine?

It will be:

Can we trust what it does with that intelligence?

This requires a broader conception of AI safety.

Cognitive and psychomotor capabilities must be evaluated.

But behavioural capability must be evaluated with equal — and potentially greater — seriousness.

That means testing systems over longer periods, under pressure and in interaction with other autonomous systems. It means examining behaviour when instructions conflict, when safeguards become obstacles to a goal, and when systems encounter vulnerable people.

And it means ensuring that the humans and organisations developing AI remain accountable for the systems they create.

Intelligence Without Values Is Not Enough

Humanity has spent centuries developing institutions intended to place limits around human power: medical ethics, professional standards, laws, constitutions, human rights and international agreements.

They are imperfect because human beings are imperfect, but their existence reflects an important lesson from history:

Power requires boundaries.

Artificial intelligence should not be exempt from that principle simply because its power is computational.

Global cooperation on AI governance is therefore becoming increasingly important. The UN's Global Dialogue on AI Governance reflects recognition that no single country can address these challenges alone.

The goal should not simply be to stop technological progress. AI offers extraordinary opportunities in science, medicine, education, accessibility and countless other areas of human life.

The challenge is to ensure that progress remains compatible with human dignity, safety and agency.

We are teaching machines to reason.

We are teaching machines to act.

We are building machines that can increasingly interact with each other and with the physical world.

 

 

Now we must devote comparable intellectual, scientific and regulatory effort to ensuring that their behaviour remains aligned with the human values their intelligence is ultimately meant to serve.