When a Tool Call Succeeds but the AI Action Is Wrong: A Computer Vision Perspective

Why a Successful Detection Isn't a Correct Action

In computer vision, we often celebrate when a model detects an object with high confidence. But a detection is not a decision. A system that flags a pedestrian in a camera feed has only completed the first step. Whether the car should brake, the drone should avoid, or the security system should raise an alert is a separate question—one that involves context, risk, and authorization.

Consider a warehouse robot that uses vision to identify a package. It sees the package, confirms its shape, and then signals a robotic arm to pick it up. The vision model returns a bounding box with 99% confidence. The arm moves, but the package slips and falls. The detection was right, but the action was wrong—because the system didn't verify the package was stable or that the arm's path was clear.

The Gap Between Seeing and Doing

Traditional software has this problem too, but it's more visible in vision systems because the input is noisy and the output is probabilistic. A vision model doesn't just say "yes" or "no"—it says "I see a person with 85% probability." That probability is a signal, not a command. Deciding what to do with that signal requires a separate layer of logic, one that can be audited and controlled.

In the KDC framework, this is called a Reasoning Object. It's a structured record of what the system saw, what it concluded, and what it decided to do. For vision, that means recording not just the bounding box and confidence, but also the context, the knowledge base used (like traffic rules or object catalogs), and the risk assessment.

What a Reasoning Object Looks Like in Vision

Let's say a security camera system detects a person entering a restricted area. The raw output is a bounding box and a label. But the reasoning object would include:

• Goal: Determine if the person is authorized.
• Context: Time of day, location, camera ID, person's trajectory.
• Knowledge reference: Access control list, facility map.
• Evidence: The person's face, badge, or gait.
• Key judgment: Is this a known employee or an intruder?
• Uncertainty: Face partially obscured, low lighting.
• Risk: If wrong, either a false alarm or a security breach.
• Action suggestion: Should the system lock the door or notify a guard?

This object doesn't replace the model's internal reasoning—it captures the externally checkable facts. It's not a chain-of-thought transcript, but a structured summary of why the system acted.

From Detection to Action: The Missing Link

In many vision pipelines, the output of a model goes straight to an actuator or a decision engine. That's like letting a sensor decide policy. The KDC approach inserts a governance layer between perception and action. This layer checks the reasoning object against business rules, permissions, and risk thresholds.

For instance, a self-driving car's vision system might detect a stop sign. The reasoning object says: "Stop sign detected, confidence 98%, distance 30 meters." But the action to stop is not automatic. The car's control system must check speed, road conditions, and whether stopping is safe. That check is part of the governance layer.

Tools, Skills, and Capabilities in Vision

KDC distinguishes between three levels of execution:

• Tool: How to perform a specific operation, like "extract face embedding" or "estimate depth."
• Skill: How to organize multiple operations around a goal, like "identify a person and verify their access."
• Capability: How to execute an action within defined boundaries, like "grant access to a building"—which includes permissions, risk level, and audit requirements.

In a vision system, a face recognition model is a tool. A skill might chain face detection, embedding, and matching against a database. But the capability to unlock a door is a higher-level abstraction that includes the policy: only certain personnel can unlock at certain times.

Why Prompting Isn't Enough for Vision Governance

Some teams try to control vision systems by writing rules in the system prompt, like "only alert if confidence is above 90%." But that's a fragile defense. The model might misinterpret the prompt or the confidence threshold might be context-dependent. A better approach is to enforce such rules in the system itself.

For example, a factory uses vision to inspect products. The model flags a defect with 80% confidence. The prompt says "high confidence needed to reject." But the operator knows that some defects are more critical than others. So the system should have a policy: if the defect is in a critical area, even 70% confidence triggers a manual review. That policy is not in the prompt—it's in the governance layer.

Bringing It All Together: An Audit Chain for Vision

To make vision systems accountable, we need an audit trail that connects the business goal to the final action. For a vision-based access control system, the chain might look like:

1. Business goal: Allow only authorized personnel into the server room.
2. Vision input: Camera captures a person at the door.
3. Reasoning object: Person identified as Jane Doe, confidence 93%, access level 5.
4. Capability check: Does Jane have permission for this room at this time?
5. Policy decision: Yes, but requires a second factor (badge swipe) after hours.
6. Action: Door unlocks, but only after badge scan.
7. Feedback: Door opened, Jane entered.

Each step is logged. If something goes wrong—say, the wrong person enters—you can trace back to see whether the vision model was wrong, the policy was misconfigured, or the user confirmation was skipped.

The Real Challenge: Knowing When Not to Act

In vision, the hardest part isn't getting the model to detect things—it's getting the system to know when a detection should not lead to action. A vision model might see a person near a door and infer they want to enter. But the person might just be walking by. The system must be able to say: "I see a person, but I'm not sure they intend to enter, so I won't unlock the door."

That's a judgment call. And it needs to be recorded. The reasoning object should include the uncertainty and the decision to hold off. This way, if a person gets locked out, you can see why the system hesitated.

In the end, the value of a vision system isn't measured by how many objects it detects, but by how well it balances action and restraint. By applying the KDC principles—reasoning objects, capability governance, and audit trails—we can build vision systems that are not only smart but also trustworthy.