Using FPGA Design and HIL Algorithm Simulation to Control Visual Servoing

ABSTRACT


INTRODUCTION
In recent years, robotics research has become proficient with a significant alteration in the automation sector, object handling and tacking applications. The research comforts are moving from the development of robots for organizing manufacturing industries to the expansion of independent movable robots functioning in unstructured and natural environments. These independent mobile robots are appropriate in huge number of complex tasks solving such as vacuuming of hazardous material, investigation, liberation and investigation in free surroundings wherever humans are reserved from easy access. Since it is predicted that this novel class of movable robots will have wide applications in accomplishments where human competences are needed, they have concerned the consideration of the investigators [1]. Mobile manipulator robot is currently an extensive shift that denotes the robots poised with a robotic arm straddling on a movable dais. Such system is usually categorized by a huge degree of redundancy and it also associates the manipulability of a static base robotic manipulator with the mobility of a movable dais. These kinds of systems allow the most normal tasks of robotic systems that require both manipulation tasks and movement. Such systems promises to provide numerous applications in diverse manufacturing and industrious areas as construction and mining or for public assistance [2,3].
The declined price of the Field Programmable Gate Arrays (FPGAs), their growing competences, and the likelihood to increase the performance of processing and computation of tasks with unique hardware features, have improved their usage in the applications associated with control and many more. The current abilities of FPGAs provide a specific hardware technology, which can be engaged to implement embedded control systems [4]. FPGAs have previously been used with triumph in diverse systems as the growth of advanced degree polynomial profile generator for CNC and robotics applications [5], control of robotic manipulator for industrial applications [6], neural and fuzzy based controllers for certain critical applications[7], mechatronic and other automation systems [8], etc. FPGAs and their fractional reconfigurability afford supplementary possessions to control systems for processing, interface, testing, configuration capabilities, etc.). In this article, a visual servoing system based FPGA is proposed to accomplish the regulation of robotic manipulator for object tracking applications using Hardware-in-the-loop simulation (HIL) technique. When high precision and performance of position control of a robotic manipulator for object tracking is considered, visual feedback can be extensively used to improve the accuracy of the end effector for positioning the objects. Visual information, disparate the ordinary sensors yield a measurement of the tip position, evading approximation errors due to a coarse familiarity of the physical constraints symbolizing the mechanical assembly of the robotic manipulator. Visual servoing based control of robots has been growing in recent years [9,10].
The robot manipulator controller positioning for object tracking can be hardware or software oriented device that constantly processes the value of the feedback from the joint sensors and then actuates the joints of the robotic manipulator to exact any deviation from the desired value [11]. The joint positions and velocities are measured by joint sensors such as potentiometers, tacho generators or digital encoders. The HIL based simulation modelling aids, to fast prototyping of control algorithms, rather than researchers investing on the actual robotic manipulator [12,13]. In this present work, the control algorithms are implemented on Xilinx Virtex-6 FPGA board. The implementation part is tested with the HIL simulated model of the three link serial robotic manipulator. Figure 1 shows the overall HIL block diagram implementation of the proposed object tracking controller.

DYNAMIC PERCEPTIBLITY BASED OBJECT TRACKING
Tracking of objects based on dynamic perceptibility is supplemented by the usage of camera model tracking the end effector of the HIL model of robotic manipulator. Representing pas the pose of camera model located at the end effector and tracking the end effector of the HIL model of robotic manipulator. Representing pas the pose of camera model located at the end effector and e=[fx,fy,f2x,….,fjxfgy]T ЄR2j as a vector of j number of extracted feature points of the object desired to be tracked [14,15]. The velocity relation in the extracted feature points is represented by Where Jp(q)is the Jacobin matrix of the robotic manipulator and Ij = Ij(q, e) represents the Jacobin matrix mapping from joint space to the extracted feature point space. The acceleration of the extracted feature point is given by the second derivative of equation (2) By neglecting the friction coefficients of the robotic manipulator the dynamic mode the serial rigid robotic manipulator of n link can be represented as, Introducing a scalar value w for determining the ability of positing of object by therobotic manipulator, it can be defined as in equation (5) taking into account the joint velocities in unit form.
Considering the differential kinematics Ir of the robotic manipulator and relating it to the velocity joint velocities qand end effector velocity pwe obtain the unit form expression as The manipulability scalar value can be obtained as By assuming the joint velocities , =0, the dynamic manipulability can be given by: It is derived based on ignoring the gravity terms such that The closeness of the extracted feature points measures the singularity of the features by defining the perceptibility wPas a scalar function defined as It represents the kinematic evidence and the following vectors are defined toe estimate the dynamic perceptibility value.
From the equations (11) and (12) (13) The dynamic perceptibility wdp, can be obtained as A 2j×2j matrix results while multiplying the term IJM−1(IJM−1)T.This matrix holds the dynamic perceptibility information for each extracted visual feature points. It is feasible to represent j number of ellipses for each of the extracted features. Based on the movement capability of the end effector indicated by the ellipses it is often desirable to make the robotic manipulator move forward or backward [16].
The trajectory of the extracted feature is obtained by expressing reference acceleration of the extracted features using the proportional gain matrix Kp and derivative gain matrix KD. It can be expressed as: (15) Where , and are the desired position, velocity and accelerations of the extracted features of the object. The equation (3) and (15) can be related as: The error in the extracted feature in the controller is reformed to force the robotic manipulator in the direction of largest eigen vector of each dynamic perceptibility ellipses.
rs=rs+||rs||GU (17) Where rs+||rs|| represents the norm of the extracted features of the object and G is the gain matrix that controls the effect of the perceptibility on each visual feature. GU is the diagonal matrix where values of tis diagonal are unit vectors pointingthe direction of the largest eigen vector [17,18].
Applied torques for the joint actuators of the robotic manipulator for tracing the objects can be definedas )) ( .( ) ( Where W represents the time dependent weight matrix and the symbol + points out the pseudo inverse of the matrix and the constraints of the proposed task is given by: Where n m By representing the task constraint of the relation in equation (16) in the form in equation (19) The control law concerned on the feature description of the task is set by: It is derived from equation (20) such that replacing the variables concerned on the feature task description [19,20,21,22].

IMPLEMENTATION OF THE OBJECT TRACKING CONTROLLER WITH FPGA
The vital considerations about the FPGA implementation of the object tracking visual servoing are indicated in equation (21). The pipeline of the implemented controller is shown in Figure that depicts the parallel architecture of the object tracking using servoing accurately. Response of the controller constrained to time factors is managed by the proposed architecture.
The five stage module of the visual servoing system comprises of pipelined conceptualization of observing, resizing, thresholding, erosion and detection of the objects. The vision algorithms take up the pipeline execution stages in the FPGA.
The latency of the processing stages are considerably reduced thereby resources for storage and routing of data traffic are minimized. Time chart of the pipelined servoing for object tracking for the HIL model of the robotic manipulator is shown in Figure 2. The scheme of functioning of the implemented controller for object tracking is coded in Hardware Description Language (HDL) by taking advantage of the pipeline stages in FPGA.
By exploiting these capabilities the matrices IJ, M and C are computed parallel making other dependent ones to wait for their inputs. It is also considered by sharing the cones by optimization strategies. The ellipses presented in Figure 3(a) and Figure 3(b) signify the movement competence of every extracted feature points in the pattern. Here a pattern that consists of three points is considered. The representation in Figure 3(a) shows the perceptibility ellipses and it is acquired only with data available in the image space, which makes the pattern to be perfectly circle. The illustration in Figure 3(b) shows the dynamic perceptibility ellipses instead, take into consideration the dynamics of the robotic manipulator, influencing the contour of the ellipses there concluding that it would be better to change the pattern up or down, instead of sidelong.

EXPERIMENTAL RESULTS AND DISCUSSION
This discussion presents the investigations carriedout to exemplify the performance of the visual servoing robotic manipulator architecture for object tracking based on FPGA. Additionally, the value of the dynamic perceptibility idea suggested in this paper is presented here. The trials shows that this idea affords facts about the system capability to track objects using a controller based on direct visual technique implemented in FPGA. This parameter is a useful measure to govern circumstances where the robot is nearby singularity. The futures of dynamic visual controller implemented in the Xilinx Virtex 6 FPGA board are tested with the HIL model of the three link robotic manipulator implemented in the MATLAB Simulink environment. Additionally, the redundancy of the HIL model of the three link robotic manipulator allows us present the results attained for more challenging tasks. In the experiment described the camera model implemented is calibrated with intrinsic parameters in certain optical centre pointing the focal length in X and Y directions. Owing to the restrictions of this three link robotic manipulator and taking appropriate measures to avoid singularities, the work taken up for experimentation is considered to be in a 2D plane matching to the object position for the preliminary tests. The robotic manipulator implemented has 3 dof, but to accomplish real environment visual servoing for object handling the robot needs to have minimum of 6 dof. The results part paper not addresses the challengling parts of processing tasks on the captured objects, but rather dwells in representing that this system is extremely robust and that it provides high degree of reconfigurability with parallel and fast processing with the aid of FPGA. The investigations are carried out by first positioning the robotic manipulator at the desired coordinates in the plane. The object's under test that needs to be tracked is observed by storing its centre of gravity (c.o.g). For implementing the Direct Visual controller as represented in equation (21), the constants KP and KD are used such that its initial value are chosen as The navigation of the end-effector of robotic manipulator implemented as a HIL model, moves the camera model implemented with a fairly straight path, hardly with negligible oscillations, and without omitting the preferred position. Evolution of the norm error of the extracted features pointes is shown in Figure 4a. It can be observed that the error falls quickly after few initial iterations. Figure 4b demonstrations the output of the actual joint torques of the links during the placing task of the object, the torques inputs sent from the controller implemented in the FPGA to the motors of each robot joint. Around, after 1.5s, the system reaches the position there by the required torque reaches close to 0.   As a final part of the implemented controller Figure 6 shows the three dimensional trajectories of the robotic manipulator obtained from the direct visual servoing controller and dynamic perceptibility based visual servoing controller. It represents the modifications made by the end effector of the robotic manipulator using the proposed controller implemented inFPGA targeting to avoid the robot singularity.It is also evident from the resulting trajectory that the singularity in the robotic manipulator is eliminated thereby provides better tracking of the trajectory to track the objects. The resource utilizations ofthe implemented controller in the FPGA are summarized in the TABLE. 1. Xilinx Virtex6 FPGA that houses the device XC6VLX760 is used to the experimentation. The percentage of the resources used isalso indicated. It is evident that the implementation of the dynamic perceptibility based visual servoing controllerin the FPGA acquires more resources than the direct visual servoing controller. Also, the execution time of the proposed algorithms istabulated representingthe incurred in execution of the algorithm in the FPGA device. Based on the excess resources in the direct visual servoing controller, it also consumesaround 3.20µs delay in completing the servoing task of object tracking.

CONCLUSION
A direct visual servoing controller implementation on an FPGA to accomplish the supervision of an object hold on by a HIL model of a three link rigid robotic manipulator using the camera model system is suggested in this work. The implementation of the reconfigurable hardware architecture permits to acquire persistent processing intervals and ideal set of tasks by performing pipeline processing of the sub tasks. Furthermore, the dynamic manipulability based object tracking is implemented to determine the dexterity of the robotic manipulator to track desired objects by utilizing the direct visual controller. The investigations are also performed using dynamic perceptibility based visual servoing and results are compared with the direct method. It is obvious and evident that the proposed dynamic perceptibility based visual servoing utilizes optimum resources in the hardware architecture with ideal performance and better latency. The observed performances are accountable to the service of object tracking and the control of the joints of the robotic manipulator that handles the desired object.