A Company Is Not One Agent

An AI agent completes a task. Somebody still needs to decide whether the result is useful, where it belongs and what should happen next.

Sometimes that is straightforward. Sometimes it creates a chain of checking, clarification and handoffs that is harder to see than the original task.

Multiply that across a function and the management problem changes. It is no longer enough to understand the performance of each agent or the workload of each team independently. You need to understand the arrangement they form together.

Who has the information? Who can act? Where does judgement enter? Who takes responsibility for an exception? What happens when two locally sensible decisions conflict?

The self-improving company has to operate at that level.

The work lives between the participants

A company brings together people with different responsibilities, incentives and information. Software determines what they can see and do. Processes establish expected sequences. Informal routines fill gaps.

Agents join that arrangement. Their instructions, tools and permissions give them a particular view of the work and a bounded ability to act.

A task that looks complete from one participant’s perspective may be incomplete from another’s. A prepared customer response may lack the context a reviewer needs. A correctly updated record may create a conflict elsewhere. A technically valid action may fall outside somebody’s authority.

This is why the unit of improvement is larger than the task. A useful intervention changes how the participants produce an outcome together.

An agent that can do more is valuable when the surrounding work is ready to use that capability. The information, handoff, review and authority around it may matter as much as the model.

Improvement for whom?

An executive might seek lower cost. A service team may be trying to protect quality. A risk function may require stronger verification. Employees need a manageable workload and room to exercise judgement.

Those interests can support one another. They can also pull in different directions.

Imagine an automation that reduces the time spent by a frontline team but increases exceptions for specialists. The frontline result improves. Whether the business improved depends on the size and importance of the additional work, its effect on customers and whether the new arrangement is sustainable.

A team average may hide the same problem internally. Routine cases improve while difficult cases become harder. One role gains capacity while another carries most of the correction work.

A useful company-level picture has to make these consequences visible. It should help leaders decide whether a trade-off is worthwhile, rather than allowing an attractive average to make the decision for them.

The outcome therefore needs more than one condition. Increased capacity might have to sit alongside acceptable quality, effective control and sustainable workloads. There may be several arrangements that meet those conditions.

Technical and social design belong together

The sociotechnical tradition has long examined the relationship between technology and the organisation of work. Trist and Bamforth’s study of technological change in coal mining considered the social and psychological consequences of changes to the work system. [1]

Its setting is far from a modern AI-enabled business. The enduring relevance is the question it asks: what happens when a technical arrangement changes the relationships through which work gets done?

Introducing an agent can redistribute execution, information, judgement and review. It can alter what people need to know and where they develop expertise. It can move an exception from an experienced employee to a less prepared reviewer.

Those consequences are part of the design, even if they do not appear in the technical specification.

A team may need a better handoff rather than a faster model. A reviewer may need the evidence behind an output rather than a more confident answer. An agent may need narrower authority and a clearer escalation route before broader autonomy becomes useful.

This is detailed operating work. It is also where much of the opportunity lies.

Choose a useful boundary

“The company” can be too large a unit for a consequential decision.

A customer function, onboarding workflow or shared-service operation gives the analysis a more practical boundary. The team can identify the outcome, the participants, the relevant evidence and the changes an accountable leader can authorise.

The boundary should follow the work far enough to include material consequences. If an intervention creates downstream review, that review belongs in the assessment even when it sits outside the sponsoring team.

This does not require complete visibility into every person and system. It requires clarity about what is in view, what is missing and which dependencies could change the conclusion.

Privacy also shapes the boundary. The analysis should focus on group-level patterns and avoid exposing individual activity. Aggregation alone does not establish that privacy is protected. Where aggregation hides a potentially important effect, explanation and carefully governed investigation are preferable to pretending the average tells the whole story.

When people do something unexpected

A team may adapt a new workflow or return to an older method. It is tempting to treat that as poor adoption.

Sometimes additional support is the right response. But the behaviour can also reveal a design problem.

Perhaps the new method removes information people use to protect quality. Perhaps incentives still favour the old behaviour. Perhaps a handoff creates uncertainty about who owns the result. Perhaps the task is simply easier to complete without the new tool.

The useful first question is what the behaviour is telling us about the work.

That does not mean every objection is correct or that every preference should determine the design. It means employees’ explanations belong alongside operational evidence when the organisation evaluates a change.

A system that cannot be corrected by the people who understand the work will struggle to improve its own account of the business.

What active inference contributes

Active inference provides one theoretical way to connect incomplete observations, beliefs about underlying conditions, preferred outcomes and action. It also recognises that actions can have informational value: they can help resolve uncertainty as well as achieve an outcome. [2]

These ideas are useful when a company is deciding what to change.

But the company does not have one mind or one naturally given preference. Outcomes and constraints have to be chosen through legitimate human authority. A model cannot settle whose interests count by inferring a single corporate objective from activity.

A formal model can help us reason about a group without establishing that the group has one mind or a naturally given boundary. [3]

The useful ambition here is more practical: help a collection of people and machines understand their shared operation and coordinate better around explicit goals.

Better coordination without surrendering leadership

Human leadership becomes more important as the company gains more ways to act.

Somebody has to decide what the operation is for, which constraints matter and where authority sits. Somebody has to resolve a conflict between faster service and stronger review. Somebody must remain accountable for a change that affects customers or employees.

Evidence can make those decisions much better informed. It can reveal where local improvements fail to add up, where an agent is blocked by the surrounding system and where a control is serving a valuable purpose.

It can also make broader delegation possible. When responsibilities, information and escalation are understood, the organisation has a better basis for deciding what an agent may do and when a person should intervene.

This is the opportunity at the centre of Teho’s thesis: understand how humans, AI and software actually work together, then improve that arrangement deliberately.

A company does not need to become one enormous agent. It needs the many people and agents within it to contribute to a business that can understand its own work and keep getting better at it.

References

[1] Trist, E. L. & Bamforth, K. W. (1951). Some Social and Psychological Consequences of the Longwall Method of Coal-Getting: An Examination of the Psychological Situation and Defences of a Work Group in Relation to the Social Structure and Technological Content of the Work System. Human Relations, 4(1), 3–38. https://doi.org/10.1177/001872675100400101

[2] Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V. & Friston, K. (2020). Active inference on discrete state-spaces: A synthesis. Journal of Mathematical Psychology, 99, 102447. https://discovery.ucl.ac.uk/id/eprint/10115398/

[3] Bruineberg, J., Dołęga, K., Dewhurst, J. & Baltieri, M. (2022). The Emperor’s New Markov Blankets. Behavioral and Brain Sciences, 45, e183 (first published online 2021). https://doi.org/10.1017/S0140525X21002351