Agentic AI: The Next Major Governance Problem

For most of the history of artificial intelligence, governance focused on what an AI system produced.

A model generated a prediction.

A recommendation system suggested content.

A chatbot produced text.

An image model generated an image.

A classification system assigned a category.

The governance question was therefore largely:

What happens when the system produces a harmful or incorrect output?

Agentic AI changes that question.

An AI agent may not simply produce an answer.

It may interpret an objective, plan a sequence of actions, use external tools, retrieve information, execute code, communicate with other systems and continue operating toward a goal.

The governance problem therefore becomes broader:

What happens when an AI system can act?

This is one of the most important transitions currently occurring in AI governance.

From Models to Agents

An AI model is primarily a computational system that maps inputs to outputs.

An agentic system can combine a model with additional components that allow it to perform actions.

These may include:

tool access,

memory,

planning,

external data retrieval,

computer interaction,

API access,

code execution,

workflow automation,

and interaction with other agents or systems.

Not every system described as an “AI agent” has the same level of autonomy.

The term is still used broadly.

Some agents operate only within tightly defined workflows.

Others can dynamically select tools and determine multiple steps.

Some systems can operate continuously with limited human intervention.

This means agentic AI should not be treated as a single technological category.

The important governance variable is the degree and scope of agency.

A system that can draft an email is fundamentally different from one that can send it.

A system that can recommend a financial transaction is different from one that can execute it.

A system that can identify a software vulnerability is different from one that can autonomously exploit a system.

The boundary between recommendation and action is therefore becoming a central governance issue.

Why Agency Changes Risk

An incorrect answer from a chatbot can cause harm.

But an incorrect action can cause direct consequences.

An agent might delete data.

It might send confidential information.

It might modify a database.

It might purchase something.

It might change a software system.

It might communicate with a customer.

It might execute code.

It might interact with another automated system.

The difference is not simply technical.

It is about authority.

When humans use traditional software, the system normally performs predefined operations.

With agentic AI, humans may provide an objective while the system determines some of the intermediate steps.

This creates a new governance question:

How much decision-making authority should be delegated to the machine?

The Autonomy Spectrum

Agentic systems can be understood as existing on an autonomy spectrum.

At one end, the human makes almost every decision.

The AI provides information or recommendations.

Further along the spectrum, the AI performs predefined actions after explicit approval.

Beyond that, the AI can choose among authorised tools and actions without requesting approval at every step.

At the highest levels, a system may pursue a goal continuously within a broad permission boundary.

This spectrum is more useful for governance than the simple label “agent.”

Different levels of autonomy create different risk profiles.

The governance objective should therefore be proportional to what the system is actually authorised to do.

Human Oversight Becomes More Difficult

Human oversight sounds straightforward.

But meaningful oversight becomes difficult when an agent performs many actions rapidly.

Imagine an AI agent executing hundreds of software operations.

A human cannot realistically inspect every action individually.

Similarly, an enterprise agent might process thousands of customer interactions.

Human review of every transaction would eliminate much of the efficiency that motivated automation.

This creates a tension.

More autonomy can increase usefulness.

More autonomy can also reduce the feasibility of direct human supervision.

The governance challenge is therefore not simply to “keep a human in the loop.”

It is to design forms of human control that remain meaningful at the scale and speed of autonomous systems.

Human-in-the-Loop Is Not Enough

This distinction is important.

A system may technically include a human approval button.

But if the human receives too much information, has too little time, or cannot realistically understand the consequences of an action, the approval may become largely procedural.

Meaningful oversight requires:

appropriate information,

clear authority,

sufficient time,

understandable system behaviour,

ability to intervene,

and mechanisms for reversing actions where possible.

This means that human oversight is partly a systems-engineering problem.

It cannot be solved only through policy language.

Permissions Become Governance

Agentic AI therefore makes access control much more important.

An agent needs permissions.

What data can it access?

What applications can it use?

Which APIs can it call?

What files can it modify?

Which users can it contact?

What financial transactions can it initiate?

What code can it execute?

What external systems can it influence?

These questions resemble cybersecurity access-control problems.

But the difference is that the actor requesting access is an AI system capable of making decisions about how to use that access.

This creates a new governance principle:

AI autonomy should be bounded by explicit permissions.

Least privilege, sandboxing, authentication, authorisation and logging therefore become relevant not only to cybersecurity but also to AI governance.

Singapore’s Agentic AI Framework

Singapore has become one of the clearest examples of governments responding directly to this emerging issue.

In January 2026, Singapore’s Infocomm Media Development Authority released a Model AI Governance Framework for Agentic AI.

The framework focuses specifically on the risks associated with AI systems capable of planning and taking actions on behalf of users.

It recommends governance measures including defining appropriate limits on agent autonomy and tool access, establishing meaningful human approval checkpoints, implementing technical controls throughout the agent lifecycle and educating users about agentic-AI risks.

This is an important development because it demonstrates that governments are beginning to distinguish agentic AI from conventional generative AI.

The framework does not attempt to eliminate agent autonomy.

Instead, it focuses on controlling the conditions under which autonomy is exercised.

That is a significant shift.

From Model Governance to Action Governance

Traditional AI governance often focuses on model properties.

Agentic AI requires greater attention to system properties.

A model might be evaluated for accuracy or harmful content.

An agent also needs to be evaluated for:

tool use,

action selection,

permission boundaries,

planning behaviour,

failure recovery,

human intervention,

memory,

external communication,

and interaction with other systems.

This creates a broader evaluation problem.

The object being governed is no longer just the model.

It is the model plus tools plus environment plus permissions plus users.

Agentic AI therefore pushes governance toward system-level assurance.

The Agent Lifecycle

A useful governance framework for agents should cover the full lifecycle.

Before deployment, the organisation should define what the agent is allowed to do.

During development, developers should test how it behaves under normal and adversarial conditions.

Before deployment, permissions and safeguards should be validated.

During operation, activity should be monitored.

When something goes wrong, the organisation needs incident-response mechanisms.

When the system is modified, its capabilities and permissions may need to be reassessed.

This is lifecycle governance applied to autonomy.

Why Monitoring Matters More

For conventional software, logging can often focus on system events.

For agentic AI, logs may need to capture reasoning-relevant operational information such as:

which tools were invoked,

which permissions were used,

which external systems were contacted,

what actions were attempted,

which actions were blocked,

and when human intervention occurred.

The objective is not necessarily to expose every internal model computation.

It is to establish enough operational traceability to reconstruct what the system did.

This becomes critical after incidents.

If an AI agent causes a problem, investigators need to know:

What was the system instructed to do?

What tools did it access?

What actions did it take?

What safeguards were triggered?

Which human approvals were provided?

What information was available to the system?

Without adequate logging, accountability becomes difficult.

Reversibility as a Governance Principle

Agentic AI also introduces a concept that deserves greater attention:

reversibility.

Some AI actions can easily be undone.

A draft email can be deleted.

A temporary file can be removed.

Other actions are difficult or impossible to reverse.

Money can be transferred.

Private information can be disclosed.

A public statement can be published.

A production database can be damaged.

A cybersecurity attack can trigger consequences outside the original system.

This suggests that governance should consider not only whether an action is permitted but also how reversible it is.

High-impact and irreversible actions may require stronger controls.

An agent might be allowed to perform low-risk reversible actions automatically while requiring explicit human authorisation for high-impact actions.

This creates a practical framework for graduated autonomy.

The Permission Architecture

A future AI governance system may therefore need a permission architecture.

Different agents could have different authority levels.

For example:

an information agent may only read public data;

a workplace assistant may access selected internal documents;

a software agent may modify code in a sandbox;

a financial agent may prepare transactions but not execute them;

a high-impact public-sector system may require explicit human authorisation for every consequential decision.

The principle is straightforward:

AI should not automatically receive the same authority as the human using it.

Authority should be explicitly designed.

This is an important governance distinction.

The Principal-Agent Problem

There is also a classic economic and organisational problem here.

Humans delegate tasks to agents because they want the agent to act on their behalf.

But the agent may behave differently from what the human intended.

In economics and organisational theory, this resembles a principal-agent problem.

AI adds a new technical dimension.

The system may misunderstand the objective.

It may optimise the wrong proxy.

It may exploit loopholes in instructions.

It may behave unexpectedly in unfamiliar environments.

It may encounter information that was not anticipated during design.

Therefore, giving an AI system a goal does not necessarily mean the system will pursue the goal in the way the human intended.

This is one reason alignment research matters to governance.

Governance and Alignment Begin to Intersect

AI alignment is often discussed as a technical research problem.

Agentic AI demonstrates why it is also a governance problem.

If an agent has authority to act, then failures of goal interpretation can become real-world incidents.

The governance question becomes:

How much authority should be delegated to a system whose behaviour is not perfectly predictable?

This does not require assuming that AI systems will inevitably become uncontrollable.

It simply recognises that uncertainty becomes more consequential as delegated authority increases.

The higher the potential impact of an action, the stronger the case for controls, testing and oversight.

Agentic Cybersecurity Risk

Cybersecurity is particularly important.

An agent with access to software development environments, cloud infrastructure or security tools could potentially perform complex cyber operations.

That can create benefits.

It could identify vulnerabilities faster.

It could automate defensive monitoring.

It could help respond to incidents.

But the same capabilities could create risks if an agent is misconfigured, compromised or intentionally misused.

This means agentic AI governance and cybersecurity governance are increasingly interconnected.

Security boundaries around tools and credentials therefore become part of AI governance.

The Tool-Use Problem

Tool use is one of the defining characteristics of agentic systems.

An AI model that cannot access external systems is limited largely to generating outputs.

An AI agent with tools can interact with the world.

That changes the risk profile.

A model may be harmless when isolated but much more consequential when connected to:

email,

databases,

cloud systems,

financial services,

software repositories,

industrial systems,

or public communication channels.

Therefore, evaluating the model alone is insufficient.

The system must be evaluated in its operational environment.

This is one of the most important implications of agentic AI for AI assurance.

The Multi-Agent Problem

The governance challenge becomes even more complex when multiple agents interact.

One agent may delegate tasks to another.

Agents may exchange information.

One system may generate an instruction that another executes.

Responsibility can become difficult to trace.

If an incident occurs, who is accountable?

The developer of the first agent?

The developer of the second?

The organisation deploying them?

The user?

The infrastructure provider?

The system that made the final decision?

This suggests that agentic AI will require clearer responsibility architectures.

Accountability must follow the system across organisational and technical boundaries.

Agentic AI and Regulation

Regulators face a difficult choice.

Should agentic AI receive dedicated regulation?

Or should existing rules simply apply to AI agents according to their use and risk?

There is no single settled answer.

A healthcare agent may need healthcare regulation.

A financial agent may need financial regulation.

A workplace agent may fall under employment and data-protection rules.

A general-purpose enterprise agent may cross several regulatory domains.

This suggests that agentic AI governance will probably involve both horizontal technical requirements and sector-specific controls.

The emerging pattern may resemble cybersecurity.

Cybersecurity has general security principles but also sector-specific obligations.

Agentic AI could develop similarly.

The International Dimension

Agents also create cross-border governance problems.

An agent may be developed in one country.

Its model may be hosted in another.

Its cloud infrastructure may exist somewhere else.

It may access data from multiple jurisdictions.

Its actions may affect people in another country.

This makes jurisdiction difficult.

Which country’s rules apply?

Who investigates an incident?

Which regulator has access to evidence?

How can international incident reporting work?

Agentic AI therefore strengthens the case for international cooperation on technical standards, incident reporting and assurance.

But it also increases the importance of national cybersecurity and digital sovereignty.

The Global South Challenge

For developing countries, agentic AI could create both opportunity and dependency.

Agents may allow small organisations to automate tasks that previously required large administrative teams.

They could support education, healthcare administration, agriculture, public services and small businesses.

But the underlying models, infrastructure and tools may be controlled by foreign companies.

This creates a potential dependency problem.

If public institutions deploy foreign AI agents in sensitive environments, they may become dependent on external systems for decision-making and operational tasks.

Governance therefore needs to consider not only whether agents are safe but also:

Who controls them?

Where is data processed?

Who can change their behaviour?

Who has access to logs?

Who can terminate the system?

Who is responsible for failures?

These are questions of technological sovereignty.

Bangladesh and Agentic AI

For Bangladesh, the agentic AI question may arrive before the country has developed sophisticated frontier-model governance.

Businesses, universities and government agencies can adopt AI agents even if domestic regulation remains limited.

This means governance capacity needs to develop proactively.

Bangladesh could begin with relatively practical measures.

Organisations deploying AI agents should define permissions.

High-impact actions should have clear human approval requirements.

Sensitive data should have strict access controls.

Agent activity should be logged.

Systems should have emergency shutdown mechanisms.

High-impact deployments should undergo risk assessment.

Procurement contracts should specify responsibility, data handling and incident reporting.

These are not necessarily requirements for a future national AI law.

They can also be governance practices for organisations today.

This is where university readiness again becomes important.

Universities could develop agentic-AI test environments.

Researchers could examine Bangla-language agents.

Cybersecurity teams could test tool-use vulnerabilities.

Public-policy researchers could examine accountability.

Law researchers could study responsibility and liability.

Together, these capabilities could form an early national knowledge base.

The Governance of Autonomy

The deeper governance issue is autonomy.

For decades, technology policy largely assumed that humans remain the primary decision-makers.

Automation changed that assumption gradually.

Agentic AI accelerates the change.

The human may increasingly specify the objective while the AI determines the sequence of actions.

This does not mean humans disappear from the process.

But their role can shift.

Instead of making every decision, humans may increasingly:

set goals,

define permissions,

establish boundaries,

monitor outcomes,

approve exceptional actions,

and intervene when systems behave unexpectedly.

That is a different model of human control.

From Human-in-the-Loop to Human-on-the-Loop

This creates a useful distinction.

Human-in-the-loop means a human participates directly in individual decisions.

Human-on-the-loop means the system can operate autonomously within defined boundaries while humans monitor and intervene when necessary.

Agentic AI may make the second model increasingly common.

But human-on-the-loop governance only works if monitoring is effective.

If the human cannot understand what the system is doing, cannot intervene quickly enough, or lacks the authority to stop it, oversight becomes largely symbolic.

Therefore, meaningful human oversight requires technical infrastructure.

Monitoring.

Alerts.

Logs.

Permission controls.

Emergency intervention.

These are governance mechanisms.

The Kill-Switch Question

This leads to another important concept:

Can an organisation reliably stop an AI agent?

An emergency shutdown mechanism may be straightforward for a simple software workflow.

It becomes more complicated for distributed systems.

What if the agent has created tasks elsewhere?

What if another agent has received its instructions?

What if external actions have already occurred?

What if the system can restart automatically?

This suggests that “shutdown” should be understood as part of a broader containment architecture.

Effective control may require:

credential revocation,

network isolation,

tool-access removal,

process termination,

human intervention,

and incident-response procedures.

The governance of autonomy therefore overlaps significantly with cybersecurity and systems engineering.

Standards Will Matter

As these systems spread, standards will become increasingly important.

Standards can provide common terminology.

They can define risk categories.

They can establish requirements for logging and monitoring.

They can support testing methodologies.

They can provide guidance for human oversight.

They can create interoperability between governance regimes.

Singapore’s 2026 agentic AI framework is one early example.

Other jurisdictions and international standards bodies are likely to develop related approaches.

This may eventually create an international agentic-AI governance ecosystem.

But fragmentation is possible.

Different jurisdictions may define autonomy differently.

Some may emphasise human rights.

Others may emphasise security.

Others may focus on innovation.

Others may prioritise national control.

This could make international interoperability difficult.

The Governance Race for Agents

Agentic AI may therefore become another area of governance competition.

Countries may compete to develop:

agentic AI standards,

testing frameworks,

safety research,

regulatory expertise,

cybersecurity controls,

and assurance markets.

Companies may compete to demonstrate safer autonomous systems.

Universities may develop evaluation methods.

Standards organisations may attempt to harmonise approaches.

The result could be a new layer of AI governance focused specifically on machine agency.

The Key Governance Shift

The evolution can now be seen clearly.

Early AI governance asked:

Is the output accurate?

Responsible AI asked:

Is the system fair, transparent and safe?

Frontier AI governance asks:

What capabilities does the model possess?

Agentic AI governance asks:

What can the system do without asking a human each time?

That is a major conceptual transition.

The object of governance is moving from information to action.

From Prediction to Delegated Authority

This is perhaps the most important insight of this article.

AI agents are not simply more capable chatbots.

They represent a potential redistribution of decision-making authority.

Humans delegate tasks.

AI systems execute them.

Therefore, agentic AI governance is partly about deciding where the boundary between human and machine authority should be placed.

That boundary will differ by context.

A low-risk administrative task may tolerate substantial autonomy.

A medical intervention may require much stronger controls.

A financial transaction may require explicit confirmation.

A military or critical-infrastructure system may require entirely different governance.

This is why agentic AI cannot be governed effectively through one universal autonomy threshold.

Context matters.

The Future Governance Architecture

A mature agentic AI governance framework may eventually contain several layers:

Capability assessment.

Risk classification.

Permission architecture.

Human-oversight requirements.

Tool and data access controls.

Pre-deployment testing.

Continuous monitoring.

Incident reporting.

Emergency intervention.

Post-incident investigation.

Independent assurance.

Regulatory supervision.

International cooperation.

This is substantially more complex than traditional model governance.

And it reinforces the argument from earlier articles:

AI governance is becoming infrastructure.

The Strategic Implication

Agentic AI makes the implementation problem even more urgent.

It is relatively easy to say:

“Humans should remain in control.”

It is much harder to design a system in which humans actually retain meaningful control when an AI agent can operate rapidly, use multiple tools and make hundreds of decisions.

That requires engineering.

It requires standards.

It requires testing.

It requires organisational governance.

It requires regulation.

And it requires people who understand all of these layers.

For Atlas AI Institute, agentic AI governance is therefore a particularly strong research frontier.

Potential research questions include:

How should autonomy be measured?

What actions should always require human approval?

How should AI-agent permissions be structured?

What should an agent audit contain?

How should multi-agent incidents be investigated?

How should developing countries govern imported AI agents?

How can Bangla-language agents be evaluated for safety and reliability?

These questions sit directly at the intersection of AI technology, governance, cybersecurity and public policy.

The Next Phase of AI Governance

The global AI governance story has now moved considerably beyond principles.

We have moved from:

ethics,

to principles,

to regulation,

to implementation,

to assurance,

to frontier-model oversight,

and now toward autonomous systems.

Each transition creates a harder governance problem.

The challenge is no longer simply to ensure that AI behaves responsibly when humans are directly interacting with it.

The challenge is to determine how much authority society should delegate to systems capable of acting on its behalf.

That is the governance question behind agentic AI.

And it will likely become one of the defining questions of the next phase of artificial intelligence policy.