# Vision-based Estimation and Control

Recent advances in image-processing and nonlinear state estimation methods have matured to the point that these technologies may be used in many robotics and surveillance applications. Position based visual servo (PBVS) control methods use 3D scene information reconstructed from image data using these estimation methods. A single camera can be used as a "Cartesian Sensor" in vision based-estimation techniques such as pose and velocity estimation. These estimates can provide state inputs to a variety of automatic control problems.

Structure and Motion (SaM) estimation using a camera is a very well-known problem in robotics and computer vision research community. SaM estimation is important for robotic applications such as vision-based urban navigation of an autonomous agent, manipulation of unknown and moving targets, or human-machine interaction applications. The objective in SaM estimation is to estimate the Euclidean geometry of the feature points as well as the relative motion between the camera and feature points.

**Structure from Motion (Stationary Object):**

A reduced-order nonlinear observer is developed to estimate the distance from a moving camera to a feature point on a static object (i.e., range identification), where full velocity and linear acceleration feedback of the calibrated camera is provided. The contribution of this work is to develop a global exponential range observer which can be used for a larger set of camera motions than existing observers.

- Developed an online method to
*estimate the range of a stationary object using a single moving camera*where six degree-of-freedom velocity information of the camera is known *Exponential estimation error convergence*is achieved using less stringent conditions than previous literature results- The observer errors remain bounded even if the stationary object assumption is violated where the object motion is considered as an exogenous input
- No knowledge of the object shape or size is required

**Related Publications:**

- A. P. Dani, N. R. Fischer,
**Z. Kan**, and W. E. Dixon*“*Globally Exponentially stable observer for Vision-based Range Estimation,*” Mechatronics, Special Issue on Visual Servoing*, Vol 22, No. 4, pp. 381-389 (2012).**[pdf]** - A. P. Dani, N. R. Fischer,
**Z. Kan**, and W. E. Dixon, “Nonlinear Observer for Structure Estimation using a Paracatadioptric Camera,”*American Controls Conference*, Baltimore, Maryland, 2010, pp. 3487-3492. [pdf]

**Structure and Motion (Moving Object):**

A new observer is developed for a class of nonlinear systems which can be used to solve the structure and motion estimation problem when an object is moving. The observer algorithm uses camera velocities and the feature point data obtained from an image sequence. The design is based on identifying the linear unmeasurable part of the state from the dynamics of the measurable part of the state using a robust identifier called the robust integral of the signum of the error (RISE). The identifier is then used to stabilize the estimation error dynamics of the unmeasurable state. The proposed method has several advantages over the existing methods:

- Developed an online method to
*estimate the structure and motion of a moving object using a single moving camera*with known motion - No requirements of minimum number of point correspondences or number of views
- Perform real-time computation of the structure and motion of a moving object
- Estimation error convergence is proven to be
*asymptotically stable*provided an observability condition based on the persistence of of excitation (PE) of the camera motion is satisfied.

**Related Publication:**

- A. P. Dani,
**Z. Kan**, N. R. Fischer and W. E. Dixon, “Structure and Motion Estimation of a Moving Object Using a Moving Camera,*” American Controls Conference*, Baltimore, Maryland, 2010, pp. 6962-6967. [pdf]

**Unknown Input Observer (UIO):**

A dynamical system which estimates a full or partial state of a given system with no knowledge of the input is called an unknown input observer (UIO). A UIO is developed for a general class of multi-input multi-output (MIMO) nonlinear systems. Based on the existence of a solution to the Riccati equation, necessary and sufficient existence conditions are derived. The conditions provide guidelines for choosing the observer gain matrix K based on the Lipschitz constant of the nonlinearity present in the dynamics. An algorithm for choosing K based on the Eigenvalue placement is obtained by solving an LMI feasibility problem. Contributions of this work include:

- Design of an UIO for a general class of nonlinear systems
- Developed necessary and sufficient conditions for the existence of the UIO
- The developed UIO can be applied to the structure and motion estimation problem with a moving object, and requires less restrictive assumptions on the object’s motion than existing approaches. The object is assumed to be moving on a ground plane with arbitrary velocities observed by a downward looking camera with arbitrary linear motion. No assumptions are made on the minimum number of points or minimum number of views required to estimate the structure. Feature point data and camera velocity data from each image frame is required

**Related Publication:**

- A. Dani,
**Z. Kan**, N. Fischer, and W. E. Dixon,*“*Structure Estimation of a Moving Object Using a Moving Camera: An Unknown Input Observer Approach,”*IEEE Conference on**Decision and Control*, Orlando, FL, 2011, pp. 5005-5010. (invited paper) [pdf]