Design of computer vision systems for process automation

Résumé

Computer vision allows a computer to extract information from an image, just as people use their eyesight to obtain information from their surroundings. So we can use it to find out how many people are in a place or to find out whether a traffic light is green or red. Its versatility has made it applicable to all kinds of processes such as medicine, industry or agriculture. Another advantage is its low cost, since most of the time only a camera is needed to acquire the information and no extra equipment is required. The main lines of research of this thesis have been given by the existing projects and collaborations with other entities where computer vision techniques are required. On the one hand, a project of the Ministry of Economy, Industry and Competitiveness has given rise to the line of research focused on the study of quality in manufacturing processes. On the other hand, the collaboration with the University of Zaragoza from several Art. 83 research contracts has generated a line of research in the field of precision livestock farming and animal welfare. The research line on manufacturing processes focuses on the study of the surface of machined parts and tools. The quality processes required in this field make it necessary to have exhaustive control of surface wear. These tasks are usually carried out with specific equipment, such as the roughness meter that calculates the surface profile. Computer vision allows us to acquire information without coming into contact with the elements. Three problems are dealt with during this dissertation: the study of wear on the surface of machined parts, the formation of burrs in the finishing area of the part and the monitoring of the state of the tool to determine their wear and whether breakage has occurred. For the first of the considered problems -the study of the surface quality of machine workpieces-, two different approaches have been considered: traditional vision techniques based on texture descriptors (achieving an accuracy of 96.2 %) and deep learning techniques by applying transfer learning (with an accuracy of 93.22 %). The second application, burr formation systems, has been developed with linear regression to study the slope formed by the edge of the workpiece; contour information has also been used to classify burrs with a 90.34 % of precision; finally, the edge of the part is delineated and binarized with RUSTICO achieving a 93.20 % of precision. Focusing on burr breakage detection, an optimized network with the minimum possible number of training parameters has been designed getting an accuracy of 90.23 %, including visual explanation of the final model with Grad-CAM. In the last of the problems considered in this field, tool condition monitoring, the wear area of the has been segmented to supervise its condition with a 92 % of accuracy. The research line related to livestock farming focuses on precision livestock farming and the search for maximum animal welfare. On the one hand, the problem of estimating the weight of lambs using computer vision systems has been considered since it reduces the interaction of the farmer with the animal and, hence, the stress caused. It also allows weight monitoring to identify the optimal timing that is useful for meat producers. The proposed approach provides a vision-based system with low hardware requirements, just a mobile or electronic device with a camera, instead of expensive infrastructures such as walk-over weighing (WOW). On the other hand, another proposed system deals assists pasture-based farming by threat detection. To this extend, the presence of species like the Iberian wolf in natural environments or locations that are difficult to access are determined. So, attacks on livestock can be reduced by avoiding those places where predators are located. The proposed system can be deployed on fixed cameras near the grazing areas or even in robots. Experiments of weight estimation consider Rasa Aragonesa lambs, obtaining an accuracy of 0.1 kg MAE and 0.98 R2 using features extracted from the animal’s contour. Besides that, using transfer learning a MAE of 0.59 kilos and a R2 of 0.96 are achieved. Finally, a vision module for herding that can be deployed in a Sheepdog Robot (VISORED) has been developed to distinguish between wolves and dogs in images -being possible to adapt it to other species-; this system uses YOLOv5 to detect the considered species with an accuracy of 99.17 %. In conclusion, traditional computer techniques and more recent ones based on Convolutional Neural Networks have been proposed and applied to different fields to solve several existing problems. The obtained results show that they allow tasks to be automated with high accuracy, fulfilling the established requirements. During this thesis, new datasets have been generated and are publicly available. Moreover, some of the proposed methods have been published through international conferences and scientific journals, while other submitted papers are still awaiting a response.