Vision-Guided Robotics for Precision Positioning

This research developed a stereo-vision system for high-precision robot pose measurement and online error correction.

The system independently measured the robot tool and a reference fixture, estimated their relative 6D pose, and used that measurement to correct robot motion.

A robotic drilling setup was used for validation, but the core work focused on stereo vision, 3D localization, pose estimation, calibration, and closed-loop robot control.

---

Vision-based robot pose measurement

The system used two synchronized cameras to track passive optical targets attached to the robot and reference fixture.

The vision pipeline:

• detected circular targets in each image;
• localized their centers with sub-pixel accuracy;
• triangulated their 3D positions;
• estimated rigid-body position and orientation; and
• calculated the relative pose between the robot tool and its reference.

The resulting measurement provided the robot’s full 6D pose:

X, Y, Z + Roll, Pitch, Yaw

Several target-localization methods were evaluated, including centroiding and ellipse-fitting approaches using edge and gradient information.

Gradient-based methods achieved the highest accuracy, while centroiding provided significantly faster computation with competitive performance once practical calibration errors were considered.

---

Calibration and coordinate transformations

The cameras, robot base, robot flange, tool, and reference fixture each operate in different coordinate frames.

The project therefore included calibration and rigid-body transformation estimation between the vision system and robot.

Robot poses and simultaneous vision measurements were used to solve these transformations and convert vision-measured errors into robot motion commands.

The complete pipeline was:

Stereo Vision → 3D Tracking → 6D Pose → Relative Pose Error → Robot Motion Correction

---

Closing the vision-control loop

Vision measurements were used directly as feedback for robot positioning.

The system compared the measured relative pose with the desired pose, calculated the 6D error, transformed it into the robot coordinate system, and generated corrective motion.

Two correction strategies were implemented:

• Iterative correction: measure, correct, remeasure, and repeat until tolerance was reached.
• Closed-loop correction: continuously use vision feedback through a PID controller to drive the robot toward the desired pose.

This turned the vision system from a measurement tool into part of the robot control loop.

---

Experimental accuracy

The vision system was validated against a high-precision laser tracker.

The best target-localization methods achieved mean 3D distance errors of approximately:

• 0.033 mm with one target;
• 0.025 mm with four targets.

For robot correction, initial positional errors of approximately 1 mm were reduced to below 0.1 mm in two iterative correction steps.

The closed-loop system was also tested with simultaneous translational and rotational errors, demonstrating online 6D pose correction using visual feedback.

---

Technical scope

The project included:

• stereo camera calibration;
• lens-distortion correction;
• sub-pixel target localization;
• ellipse fitting and gradient-based detection;
• stereo triangulation;
• 3D and 6D pose estimation;
• rigid-body transformations;
• robot-camera and hand-eye calibration;
• relative pose estimation;
• iterative robot guidance;
• PID feedback control; and
• laser-tracker validation.

The central engineering problem was converting small image-space measurements into sufficiently accurate physical pose estimates and robot motion corrections.

---

Key takeaway

This project demonstrated computer vision as more than object detection.

Here, vision was used for measurement, localization, state estimation, and closed-loop control.

The system independently measured where the robot was relative to its task and used that information to correct its motion in real time.

Vision became part of the robot’s feedback loop.