An AI agent in manufacturing is software with four parts: perception, live machine states and plant records; reasoning, an LLM bounded by hard constraints; action, a limited toolset like creating work orders; and memory, the plant's own context. It executes multi-step work within guardrails, and a person approves what matters.
That definition is deliberately mechanical, because the fog around this topic comes from treating agents as magic. They are not. They are a specific architecture with specific requirements, and when you know the four parts you can evaluate any vendor's claim in about five minutes. This post walks the anatomy, the use cases that actually run on floors today, and the limits nobody selling agents leads with. For the conceptual grounding, what agentic means and how it differs from dashboards and traditional automation, read the sibling post agentic AI in manufacturing first; this one is about how the machine is built and what Harmony AI builds it on.
What are the four parts of a manufacturing AI agent?
Perception, reasoning, action, and memory. Every real agent has all four, and most disappointing "agents" are missing at least two.
Perception is live data, not exports. An agent can only respond to what it can see, and what it needs to see is the plant's present tense: machine states and fault codes as they change, operator entries as they are made, quality results as they land, schedule status as it slips. This is why the machine connectivity work in the real-time machine data guide is the non-negotiable prerequisite. An agent fed yesterday's CSV is a report generator with better marketing.
Reasoning is an LLM plus constraints it cannot argue with. The language model, typically a frontier LLM from a major model provider, supplies the flexible part: reading records, correlating events, writing the draft. The constraints supply the trustworthy part: which lines this agent covers, what it may never propose, when it must stop and escalate. The LLM reasons inside the box; the box is written by people who run the plant. Reasoning without constraints is a liability, and constraints without an LLM is just the workflow automation plants already have.
Action is bounded tools, not general capability. A well-built agent cannot do things in general. It can do specific things: create a work order, assign a task, reorder a queue, send a notification, file a record. Each tool has limits, and the consequential ones route through approval. Nothing in this architecture touches machine setpoints or safety systems; that remains controls engineering, full stop.
Memory is plant context, not chat history. The agent that drafts a useful work order knows this press, its maintenance history, the SOP for this changeover, what fixed the same fault in March, and who approves overtime. That knowledge lives in the records the plant has been accumulating for years, most of it as unstructured tribal knowledge and paper. Getting it captured and searchable is half the value of the whole project, and it is why an agent bolted onto a plant with no digital records has nothing to remember.
What do AI agents actually do on a plant floor?
Three use case families are running in production plants today, and they share a shape: high-frequency events, clerical work surrounding a human judgment call.
Downtime response. A fault lands in the stream. The agent correlates it against history, notices it is the third occurrence this week on the same product family, drafts the work order with the history attached, pre-codes the downtime entry, and routes both to the supervisor, who approves from a phone. The judgment, is this worth stopping the line for, stays exactly where it was. The twenty minutes of radio calls, walking, and typing around it are gone. The economics of those minutes are covered in machine downtime.
Reporting. The shift report, the daily production summary, the downtime Pareto: all assembled from data the system already holds, drafted by the agent, reviewed and signed by a person. This is the least glamorous use case and the fastest to pay back, because supervisors are expensive and copy-paste is not supervision. Automated daily production reporting from shift data is one of the documented outcomes of Harmony AI's deployment at CLS, a family-owned glass decoration manufacturer; the full account is in the CLS case study.
Scheduling adjustments. A machine goes down mid-shift and the question becomes what to run instead and what slips. The agent knows the orders, the due dates, the alternate routings, and the changeover costs, so it drafts the reshuffle and shows the tradeoffs; the scheduler approves or edits. The full pattern is in AI agent for production scheduling and real-time rescheduling when a machine goes down.
How much autonomy should an agent get?
As much as it has earned on that specific workflow, and no more. The working model is a ladder: suggest, draft, act on approval, act within limits and report after. An agent starts at the bottom rung for every new workflow, and it climbs only when its track record at the current rung is boring, which is the correct word: predictable, auditable, unsurprising.
Two things make the ladder honest. First, autonomy is per workflow, not per system: the same deployment can run reporting at rung four and scheduling at rung two, permanently. Second, escalation is a feature, not a failure. When the situation falls outside the constraints or confidence is low, the right behavior is to stop and route to a human loudly, with the context attached. An agent that never escalates is not confident, it is unconstrained.
How do you evaluate an AI agent deployment?
Ask the anatomy questions in order, and treat a missing answer as a missing part:
- Perception: what does it watch, live? If the answer is uploads and exports, it is not an agent for your floor.
- Reasoning: what are the hard constraints, and who writes them? The answer should be your team, in plain language, per workflow.
- Action: list the tools, exactly. A bounded list you can read in a minute is the right answer. "It can integrate with anything" is the wrong one.
- Memory: what plant context does it hold, and where did it come from? If your SOPs and history are on paper, capturing them is part of the project, not an assumption.
- Approval: which actions require a human, and how visible is the audit trail? Every action an agent takes should be logged with what it saw and why.
- Value: which workflow pays first, and how will you measure it? Run the numbers before the pilot, not after; the AI automation ROI calculator is built for this.
One more evaluation note: agents are the top layer of a stack, and the stack matters more than the agent. The arc from records to visibility to agents, and why no plant skips stages, is traced in from MES to AI agents.
What are the honest limits of AI agents in manufacturing?
Four, stated plainly. Agents cannot fix bad data: perception built on miscoded downtime or unlogged adjustments produces confidently wrong drafts. Agents inherit LLM failure modes: language models can generate fluent, incorrect text, which is why the U.S. National Institute of Standards and Technology treats confabulation as a core generative AI risk in its AI Risk Management Framework, and why grounding in plant records plus human approval are load-bearing, not decorative. Agents do not do physics: predicting bearing failure from vibration is a different discipline with different math, covered in predictive maintenance. And agents do not replace the people whose judgment they route to; they change what those people spend their hours on.
The adoption data keeps the hype in proportion. The U.S. Census Bureau's Business Trends and Outlook Survey found roughly 17 to 20 percent of U.S. businesses using AI between late 2025 and mid-2026, and Federal Reserve analysis shows manufacturing below the national average. Agent deployments that work are still rare enough that getting the guardrails right puts a plant ahead of its industry, not behind it.
How does Harmony AI build agents for a plant?
Harmony AI is a truly AI-native MES, which means the agent anatomy is not assembled from parts: perception comes from the machine connectivity and digitized records the platform already maintains, memory from the plant's own captured knowledge, and action tools ship with approval routing and audit logging built in. The platform is completely agnostic to your existing machines and software, unifying data across systems, equipment, and people into one foundation. No rip-and-replace: agents run on top of the equipment and systems you have, connected as they are.
Deployment is white-glove and in person. Our engineers lay the data foundation on your floor, connect the machines, sit with your supervisors to write the constraints in plain language, and build each agent custom to your factory with AI agentic coding, weeks of timeline rather than quarters, then watch the first drafts with your team before any workflow climbs the autonomy ladder. The platform behind it is on the features section of our homepage, and the operational results at a real plant are in the CLS case study. For the wider evaluation conversation, the plain-English companion piece what are AI agents for factories is written for the team members who will be asked to trust one.