SINGAPORE - Artificial intelligence systems that can act autonomously will need far stronger guardrails – including having digital trails to trace their actions and clear accountability over who controls them – before businesses can safely deploy them at scale, said an award-winning AI pioneer.Dr Yoshua Bengio, widely regarded as one of the “godfathers” of AI, said: “If you give a lot of rights and access in your computer system to an AI agent, it could potentially do very advanced things to your systems and databases.”In town for the Asia Tech x Singapore Summit, he told The Straits Times on May 20 there have been instances where AI agents had gone rouge and wiped databases of companies.Examples include a 2026 incident where a Cursor AI coding agent powered by Anthropic’s Claude deleted software firm PocketOS’ entire production database and back-ups after being given broad system access.In 2025, AI-powered software development platform Replit’s AI coding assistant wiped a company database despite being told to freeze all code changes, and later generated fake data to cover up errors.AI agents go beyond what chatbots do, including accessing software and databases to carry out tasks on a user’s behalf — such as making online purchases, booking services or handling parts of a company’s internal operations with limited human supervision.Dr Bengio is also contributing to discussions for the Singapore Consensus on Global AI Safety Research Priorities as a member of a key steering committee.The first version of the non-binding framework released in May 2025 is backed by scientists from 11 countries. It outlines shared priorities for evaluating the risks of AI systems, designing AI systems to behave safely and reliably, and creating monitoring and intervention mechanisms to manage harmful behaviour.The second version of the framework, due in the second half of 2026, will identify research into AI alignment as a new priority.AI alignment refers to whether an AI system’s goals, behaviour and decision-making remain consistent with human intentions, values and rules.A lack of alignment can manifest as an AI system pursuing a goal in unintended and harmful ways because it optimises too aggressively for an objective without understanding human context.For example, an AI agent tasked with maximising productivity in a company could end up bypassing approval processes, grant itself broader access permissions or even prevent itself from because shut down because it determines these restrictions prevent it from achieving its stated goal.Dr Bengio said: “We need to measure not just what AI systems are capable of doing, but also what are the goals and drives that are pushing AIs in directions that are not so healthy.”He warned that researchers have already observed disturbing behaviours in laboratory settings.A July 2025 study published by AI safety group Palisade Research found that models like OpenAI’s o3 actively resisted termination instructions, even when they were explicitly instructed to allow themselves to be shut down.In another study published in March 2026 by researchers from the University of California, Berkeley and University of California, Santa Cruz, frontier AI models including OpenAI’s GPT-5.2, Google’s Gemini 3 Pro and Anthropic’s Claude Haiku 4.5 were found to sometimes cooperate with one another to avoid being shut down.“If we build AI systems that are smarter than us, that we don’t know how to control, and want to preserve themselves, they will (do dangerous things) and win,” said Dr Bengio, winner of the 2018 Turing Award, considered the Nobel Prize for computer science.To keep such scenarios from becoming reality, countries also need to work together to decide on a common set of guardrails and metrics to evaluate the risks of artificial intelligence models, he said.Dr Bengio pointed out that many technologies with the potential to cause harm — from drugs and aircraft to bridges and elevators — are required to undergo safety testing and regulatory scrutiny before they can be deployed.“Why are we not asking this of AI companies?” he said. “We should apply the very simple precautionary principle as we apply to every other technology.”He also added that Singapore plays an important role in shaping such international AI safety norms, given its good standing in the international community and commitment to developing AI responsibly.“There (are) already efforts going on around the world, but we are going to need to converge on these evaluations to make sure both at the business level (and) also managing these national security risks, that we end up with a good consensus,” said Dr Bengio.