CN109579695A - A kind of parts measurement method based on isomery stereoscopic vision - Google Patents
A kind of parts measurement method based on isomery stereoscopic vision Download PDFInfo
- Publication number
- CN109579695A CN109579695A CN201811194104.5A CN201811194104A CN109579695A CN 109579695 A CN109579695 A CN 109579695A CN 201811194104 A CN201811194104 A CN 201811194104A CN 109579695 A CN109579695 A CN 109579695A
- Authority
- CN
- China
- Prior art keywords
- formula
- image
- industrial camera
- telecentric
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000691 measurement method Methods 0.000 title abstract description 8
- 238000005259 measurement Methods 0.000 claims abstract description 49
- 238000003384 imaging method Methods 0.000 claims abstract description 38
- 238000000034 method Methods 0.000 claims description 42
- 230000008569 process Effects 0.000 claims description 11
- 238000013178 mathematical model Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 230000003287 optical effect Effects 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000002310 reflectometry Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 101100289061 Drosophila melanogaster lili gene Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000011664 nicotinic acid Substances 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 229960000744 vinpocetine Drugs 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
A kind of parts measurement method based on isomery stereoscopic vision proposed by the present invention, the measurement method uses 1 telecentricity industrial camera and 2 general industry cameras, and relevant treatment is carried out to the image of camera shooting, combine binocular stereo vision and monocular telecentricity vision effectively, the original reason error of common optical system is compensated by telecentric imaging, the advantages of merging binocular vision and the high advantage of telecentricity videogrammetry system are in one, solve the problems, such as that production scene 3 D non-contacting type measurement accuracy is low, with data reliability height, the reliable technical effect of measurement result.
Description
Technical Field
The invention relates to the field of part size measurement, in particular to a part measurement method based on heterogeneous stereoscopic vision.
Background
The research of part detection based on machine vision is started from 90 years in the 20 th century, and gradually enters various industrial fields, and measuring means and methods are rapidly developed. Machine vision is to use a machine to replace human eyes for measurement and judgment, convert a shot target into an image signal through an image shooting device CMOS or CCD, transmit the image signal to a special image processing system, and convert the image signal into a digital signal according to information such as pixel distribution, brightness, color and the like; the image system performs various calculations on these signals to extract the features of the target, and then controls the operation of the on-site equipment according to the result of the discrimination.
Prior art 1:
inspired by human eyes for sensing the environmental depth by the parallax principle, the binocular stereo vision measurement system simultaneously observes a measured part (shown in figure 1) from two points in space, obtains two images of the measured part under different visual angles, calculates the position deviation between pixels by the triangulation principle according to the registration relation of the pixels between the two images, obtains the depth information of any point in a three-dimensional space, finally reconstructs the three-dimensional shape of the measured part, and performs multi-element measurement of the part.
Aiming at the problem of stepped shaft measurement, Wuxiang, Changchun university of industry establishes a set of binocular vision measurement system, and can measure a plurality of elements of a measured part. Experimental results show that the minimum and maximum measurement errors of the binocular measurement system are 0.1mm and 0.9mm, respectively (wuxiang. critical technology research of a binocular vision-based part dimension measurement system [ D ]. vinpocetine university, 2017.). Zhang Jun Yong of Wuhan science and technology university proposes a part multi-size measurement method and system based on binocular vision, and realizes multi-element three-dimensional measurement of a measured part through a series of processes such as calibration, polar line correction, polar line constraint matching, three-dimensional fitting, three-dimensional reconstruction and the like (Zhang Yong, Wushiqian, Xuzhang. a part multi-size measurement method and system based on binocular vision: China, CN107588721A [ P ]. 2018.01.16.). A binocular vision measuring and positioning device and a method for narrow space (Zhao 21089, Su Qing, Wu Fang Lin, Yang Kui, Zhang Xiao Cheng) are provided for three-dimensional measurement of targets in narrow space such as Zhao \21089ofBeijing university of aerospace, China, ZL201210191014.7[ P ]. 2012.11.05)
The binocular vision measurement is a method based on the bionic principle, has a plurality of advantages, and is ideally very suitable for online non-contact geometric precision inspection and quality control in a manufacturing field. However, binocular vision is not as applicable to other methods, and the most critical limiting factor is the measurement accuracy, and the accuracy of most binocular systems is only in the sub-millimeter level.
Prior art 2:
the theoretical basis of the structured light vision measurement is the laser triangulation distance measurement principle. By projecting a structural light source known by a spatial mathematical model to a measured part, a machine vision system projects and images the part and the structured light (shown in figure 2), and the spatial mathematical model of the structured light and the machine vision system imaging mathematical model are combined, so that the recovery depth information can be obtained by solving. The light projection mode includes four types of light, plane, curved surface and light beam, so that the common structured light vision measuring system also includes four types.
A structured light vision measuring system is constructed by the great bear university of Harbin industry, and the measuring range of the system is not less than 200mm multiplied by 200mm when the object distance is 250 mm. The system was used to make repeated measurements on standard gauge blocks of 8.845mm and 20.000mm thickness and width, respectively. In the thickness measurement experiment, the maximum and minimum measurement errors were 50 μm and 30 μm, respectively; in the width measurement experiment, the maximum and minimum measurement errors were 50 μm and 18 μm, respectively (bear great man. development of three-dimensional structured light vision measuring apparatus [ D ]. harabine: harabine university of industry, 2017.). The method mainly comprises the processes of calibrating a structural light vision system by a plane model method, collecting and preprocessing characteristic images, extracting characteristics, calculating characteristic parameters and the like, and has the application characteristics of high precision, high adaptability and high efficiency (Zhang from Peng, Hou, Cao Wen, Lu Lei, Yu of any world). The invention relates to a structured light measuring method and a device thereof for high-reflectivity parts, which mainly solve the problem that the high-reflectivity parts are difficult to measure (about ten thousand waves, Guo Yanyan, forward, and the like). A surface structured light three-dimensional measuring device and a method for the high-reflectivity parts are ZL201310717211.2[ P ] 2016.09.07 ]. The method comprises the steps of measuring the radial run-out error of a shaft part by using a structured light vision system in Tanchang, university of Jilin, and the like, calibrating a vision sensor and structured light by a two-step calibration method of a Zhangzhou plane and a template matching method respectively on the basis of establishing a radial run-out error structured light vision model, and solving a space coordinate of an intersection point of the structured light and the surface of the part by using the measurement model (Tanchang, Bahaohan, Zhang Yachao, and the like, an on-line measurement method for the radial run-out error of the shaft part based on structured light vision, China, CN107101582A [ P ] 2017.08.29.).
The structured light vision and the improvement method thereof both utilize a light source with a known pose to carry out active illumination, and the image depth information can be recovered only by extracting features from one image, so that the difficult problem of registration of a left image and a right image in binocular vision is solved, and therefore, the measurement accuracy of the structured light vision is obviously improved compared with that of the binocular vision. Nevertheless, the measurement accuracy of structured light vision still remains at the silk level. Moreover, compared with binocular vision, the point cloud density of structured light vision is obviously reduced, which is not beneficial to the three-dimensional reconstruction of the tiny features of the part. In addition, the calibration problem of the structured light, the high performance laser and the cost problem thereof, and the like, are still to be researched.
Prior art 3:
the multi-view vision and the binocular vision have the same theoretical basis and working principle, and are formed by additionally arranging a plurality of imaging sensors on the basis of the original binocular vision. The multi-vision simultaneously observes the measured part from three or more points in the space to obtain a plurality of images of the measured part under different vision, and according to the registration relation between the images and the triangulation distance measurement principle, a mathematical model representing the depth of any point in the three-dimensional space is established, so that the three-dimensional appearance of the measured part can be reconstructed, and the multi-element measurement of the part can be carried out. Compared with binocular vision, the multi-ocular vision introduces more constraint conditions and theoretically has higher measurement accuracy. The simplest multi-ocular vision is the three-ocular vision, and its application in industry presents an increasing trend.
Ye et al introduces a third camera into the binocular system to form a binocular vision measurement system, which adds more constraint conditions for stereo matching, reduces uncertainty of binocular vision system matching, eliminates interference information, and effectively improves result precision (Yepan, LiLi, JinWei-Qi, Jiang Yu-Tong. research. creating and transforming binocular vision [ C ]// Proc. SPIE9301, IPTA2014: image processing and Pattern recognition, Beijing, China, y13-15, 2014.). Lu and Shao establish a set of three-purpose system, design a spherical calibration piece, and obtain a translation and rotation matrix through singular value decomposition. The results show that the relative accuracy of the trinocular system is improved to 0.105%, the root mean square error is 0.026mm, and the accuracy and robustness of the system are proved (LuRui, ShaoMingwei. sphere-base calibration method for three-ocular vision sensor [ J ]. Opt. Laser Eng., 2017, 90: 119-127.). Aiming at the problem of low precision of binocular vision measurement of large-amplitude swinging objects, a target tracking method and a target tracking device based on trinocular vision are proposed by the technical company of intelligent-made-to-future (Beijing) robot system (Korea Xin. trinocular vision identification and tracking device and method: China, CN107507231A [ P ]. 2017.12.22.).
Inspired by multi-sensor fusion measurement technology, a three-eye or more-eye multi-vision system is constructed by introducing more imaging sensors on the basis of a binocular system, so that measurement uncertainty is reduced, and measurement precision is improved to a certain extent. However, the precision of the trinocular system still cannot meet the requirement of high-precision measurement of mechanical parts. The reason for this is that many cameras in the multi-view vision system all use pinhole imaging models, and the principle errors such as parallax and distortion caused by the common lens still cannot be eliminated.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a part measuring method based on heterogeneous stereoscopic vision, which effectively combines binocular stereoscopic vision and monocular telecentric vision, compensates the principle error of a common optical system through telecentric imaging, integrates the advantages of the binocular vision and the advantage of high telecentric vision measuring precision, and solves the problem of low three-dimensional non-contact measuring precision in a production field.
The technical scheme adopted by the invention is as follows:
a part measuring method based on heterogeneous stereoscopic vision comprises the following steps:
s1, determining the positions of a No. 1 telecentric industrial camera, a No. 2 ordinary industrial camera and a No. 3 ordinary industrial camera according to the size of the part to be measured and the depth of field of the cameras, keeping the position relation among the cameras, calibrating the cameras, and enabling the cameras to image the part to be measured to obtain a left image, a right image and a telecentric image;
s2, registering the left-right image pair and the left-right-telecentric image pair through processes including image preprocessing, feature extraction and feature matching;
s3, constructing a binocular vision system mathematical model according to the registration relation of the left image and the right image and the internal and external parameters calibrated by the common industrial camera to obtain the depth information of the characteristic points of the left image and the right image;
s4, grouping the feature points of the left-right-telecentric image into a group according to each 2 feature points, calculating the absolute difference of the depths of the 2 feature points in each group according to the depth information obtained in S3, and excluding the group if the absolute difference is greater than a preset threshold;
s5, projecting the feature points reserved in the S4 to a focal plane of a telecentric industrial camera, and calculating the distance of 2 projection points in each group along the horizontal and vertical coordinate directions;
s6, correcting the internal and external parameters marked in S1 according to the distance obtained in S5, reconstructing a binocular vision system mathematical model according to the registration relation of the left image and the right image obtained in S2 and the corrected internal and external parameters of the common industrial camera, and obtaining the depth information of the feature points of the updated left image and the updated right image;
and S7, constructing a three-dimensional model of the measured part according to the updated depth information, performing three-dimensional feature recognition, and extracting outline elements of the measured part, thereby realizing the measurement of the geometric features of the measured part.
Preferably, in S1, the telecentric industrial camera is calibrated as follows: and imaging the calibration piece with the known size by the telecentric camera, and acquiring the ratio of the actual size of the calibration piece to the pixel size to obtain the equivalent pixel of the telecentric camera.
Preferably, in S1, the general industrial camera is calibrated as follows: after the common industrial camera is installed, the common industrial camera is calibrated by adopting a Zhangyingyou plane calibration method, and internal and external parameters of the common industrial camera are determined based on a calibration result.
Preferably, in S4, feature points in the left image, the right image and the telecentric image are extracted, homonymous features of the left image and the right image are found, the euclidean distance between the two feature points is measured, and if the absolute difference is greater than a preset threshold, the group of feature points is rejected.
Preferably, the functional procedure of the part measurement method is as follows:
establishing a world coordinate systemCamera coordinate system of No. 1 telecentric industrial cameraCamera coordinate system of No. 2 common industrial cameraCamera coordinate system of No. 3 common industrial camera；
A certain point on the part to be measuredWorld coordinate isThe coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
（1）
in the formulaAndrepresentation from world coordinate system toj（j=1, 2, 3) coordinate transformation relations of the imaging system camera coordinate systems, also called imaging system extrinsic parameters;
by usingTo representIn the first placejAn image in an imaging system having coordinates ofWhereinj1, 2, 3, the relationship between the image point and the space point is:
wherein,j=1 （2）
wherein,j=2，3 （3）
in the formula、、、u j Andv j are parameters within the imaging system that are,is a picture pointThe pixel coordinates of (a).
Extracting the characteristic points of the images collected by the No. 1, No. 2 and No. 3 imaging systems, and establishing corresponding characteristic point combination sets、And；
from the collectionIn any one of the elements，Using search strategies and measure functions inFind out the 'same name' feature，Respectively to be separately provided withAnddeposit to New CollectionAndperforming the following steps;
from the collectionIn any one of the elements，In aIn which there is its "same name" characteristicUsing search strategies and measure functions in setsIs found inCharacteristic of (1)，Verification using a measure functionAndwhether or not it isA pair of "homonymous" features, which if so, will、、Separately deposit into new collections、Andand if not, from the setIn the process of re-choosing an element，And isRepeating the above steps until the collection is finishedAll of the elements in (1);
the equations (1) and (3) are put together and rewritten as follows, and the spatial coordinates to be found are written to the left side of the equations:
（4）
in the formula，，k=1，2，3
Order to，，，i=1，2，…，mCharacteristic pointHas pixel coordinates ofI.e. byj= 2; characteristic pointHas pixel coordinates ofI.e. byj= 3; substituting the pixel coordinates into equation (4),And：
（5）
spatial coordinates in formula (5)Unknown, and the remaining parameters are known, and rewriting formula (5) into formula (6):
（6）
in the formula，Andsee, respectively, formula (7) and formula (8):
（7）
（8）
solving the formula (6) by the least square method to obtainDefining a new set with the solved spatial coordinates as elements(ii) a From the collectionTwo elements of ZhongrenAnd，and is andcalculating the difference of z-coordinateIf, ifIs less than a given non-negative constantGo to CollectionFind and space pointAndthe corresponding pixel points have pixel coordinates ofAndand measuring by using a No. 1 imaging system to obtain:
（9）
in the formula,andequivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel units;
the confidence level is 95%, and the extended uncertainty of the imaging system # 1 is，True value ofIt should satisfy:
（10）
in the formula,(ii) a Change ofpAndqrepeating the above process until the value is obtained（) All are linked, so subscripts are used in formula (9) and formula (10)kThe value range is，Should not be less than；
Equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
（11）
the measurement accuracy of the binocular vision system can reach 1/10mm, and in the field depth range of the telecentric vision system, based on the two points, the whole formula (10) is used as a constraint condition, and the formula (11) is changed into a nonlinear constraint extreme value problem:
（12）
using the solution of least squares as initial values, i.e.Solving equation (12) to obtain an updated solution(ii) a Will be provided withAnd the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be、、、、、And(ii) a Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to assemble the setAndthe element in (3) calculates its three-dimensional coordinates.
Compared with the prior art, the invention has the following implementation effects:
the invention organically combines the prior art I and a telecentric vision system, obtains a measurement result by fusing data of a plurality of sensors, increases the utilization rate of system information, enhances the reliability of data and improves the reliability of the system.
The invention does not need to use a high-performance laser light source, and has no problem of high cost of the laser light source and no problem of point cloud density reduction caused by interval motion of the structured light.
The third camera introduced by the invention is a telecentric industrial camera which has the characteristics of constant magnification factor, no parallax, small image distortion and the like, and has higher measurement precision.
Drawings
Fig. 1 is a schematic diagram of the measurement of a binocular stereo vision system in the prior art 1.
Fig. 2 is a schematic diagram of a line structured light vision measurement in prior art 2.
Fig. 3 is a schematic diagram of the measurement of a multi-vision system in prior art 3.
Fig. 4 is a schematic structural diagram of the present invention.
Fig. 5 is a schematic diagram of the principle of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Of course, the described embodiments are only some embodiments of the invention, and not all embodiments.
Referring to fig. 4 and 5, in the drawings:
1 (a): telecentric industrial camera No. 1;
2 (a): no. 2 general industrial camera;
3 (a): no. 3 general industrial camera;
1 (b): 1 (a);
2 (b): 2 (a);
3 (b): 3 (a);
4: a part to be tested;
5: any measured point on measured partp；
6: any measured point on measured partq；
7: measured pointPThe image point in 1 (a);
8: measured pointqThe image point in 1 (a);
9: measured pointqThe image point in 2 (b);
10: measured pointPImage points in the imaging system 3;
11: and (5) installing a foundation for the heterogeneous stereoscopic vision system.
Before measurement, the positions of a No. 1 telecentric industrial camera, a No. 2 common industrial camera and a No. 3 common industrial camera are determined according to the size of a measured part and the depth of field of the cameras, the position relation among the cameras is maintained, a three-dimensional measurement space shown in figure 4 is established, and then the three cameras are calibrated.
Is a world coordinate system and is characterized by that,is a camera coordinate system of a No. 1 telecentric industrial camera,is the camera coordinate system of the general industrial camera No. 2,is the camera coordinate system of general industrial camera No. 3.
At a certain point on the part to be measured, its world coordinate isThe coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
（1）
in the formulaAndrepresentation from world coordinate system toj（j=1, 2, 3) imaging system camera standsAnd the coordinate transformation relation of the standard system is also called as an external parameter of the imaging system.
A rotation transformation matrix from the world coordinate system to the camera coordinate system is shown, if j =1, the rotation transformation matrix from the world coordinate system to the camera coordinate system of the 1# camera is shown, the matrix is 3 × 3, and the total number of 9 elements are included, so that the elements thereof are subscripted, including row and column marks, namely, the rotation transformation matrix in formula 1、、And the like.
Tj represents the translation transformation relationship from the world coordinate system to the camera coordinate system, i.e. a translation vector, which is a three-dimensional vector, and the elements are、、。
By usingTo representIn the first placejAn image in an imaging system having coordinates ofWhereinj=1，2，3then, the relationship between the image point and the space point is:
wherein j =1 (2)
Wherein j =2, 3 (3)
In the formula、、、u j Andv j are parameters within the imaging system that are,is a picture pointThe pixel coordinates of (a).
Generally, the telecentric imaging system No. 1 does not calibrate its equivalent pixels by calibrating its internal and external parameters, i.e. by imaging a calibration of known dimensions to obtain the ratio of the actual dimensions of the calibration to the pixel dimensions. There are various solutions to the calibration problem of the imaging systems No. 2 and No. 3, such as Zhangzhengyou, AflexibleNonw technique for Camera calibration [ J ]. IEEETransactionson Pattern analysis and machinery analysis, 2000, 22(11):1330-1334 ]. Now, it is assumed that all three imaging systems are calibrated, that is, the equivalent pixels of the imaging system No. 1, and the internal and external parameters of the imaging systems No. 2 and No. 3 are known.
Zeroing 3 camera pairsImaging the object to obtain a left image, a right image and a telecentric image, and assuming that the images acquired by the imaging systems No. 1, No. 2 and No. 3 are image1, image2 and image3 respectively. Now, feature point extraction is performed on image1, image2 and image3, and corresponding feature descriptors are established, such as the classical feature detection and feature descriptor algorithm, sift flow dg]International journal of computer Vision, 2004, 60(2): 91-110.). Collection、Andare the feature points of image1, image2, and image3, respectively.
From the collectionIn any one of the elements（) Using search strategies and measure functions inFind out the 'same name' feature（) I.e. to perform retrieval and registration of features. Commonly used search strategies are KD trees and their modified algorithms, such as BBF (Best-Bin-First, BBF) search strategy (beijs, lowedgboursearchinhigh-dimensionalspaces[C]// Conferenceon computer Vision and Pattern Recognition, puerto Rico, USA, 17-19June1997: 1000-. The most commonly used measure function is the euclidean distance. TheoreticallyAndshould be zero, in practice their euclidean distances are usually not zero. Therefore, a relatively small non-negative number is often takenWhen is coming into contact withAndhas a Euclidean distance of less thanTime, judgeAndis a pair of "homonymous" features. Respectively to be provided withAnddeposit to New CollectionAndin (1).
From the collectionIn any one of the elements（) In aWherein the same name is characterized in that. According to the characteristic retrieval and registration algorithm of the last step, in the setIs found inCharacteristic of (1)（) Verification using a measure functionAndwhether it is a pair of "homonymous" features, and if so, it will、、Separately deposit into new collections、Andand if not, from the setIn the process of re-choosing an element（And is) Repeating the above steps until the collection is finishedAll of the elements in (a).
Equations (1) and (3) are put together and rewritten as follows, while writing the spatial coordinates to be found to the left of the equations:
（4）
in the formula，，k=1，2，3.
Order to，，，i=1，2，…，mFeature pointHas pixel coordinates ofI.e. byj= 2; characteristic pointHas pixel coordinates ofI.e. byjAnd = 3. Substituting the pixel coordinates into equation (4),And：
（5）
formula (5) is a hollow coordinateAre unknown and the remaining parameters are known. Rewriting formula (5) to formula (6):
（6）
in the formula，Andsee, respectively, formula (7) and formula (8):
（7）
（8）
solving the formula (6) by the least square method to obtainDefining a new set with the solved spatial coordinates as elements. From the collectionTwo elements of ZhongrenAnd（and is and) Calculating z seatTarget difference valueIf, ifIs less than a given non-negative constantGo to CollectionFind and space pointAndthe corresponding pixel points have pixel coordinates ofAndand measuring by using a No. 1 imaging system to obtain:
（9）
in the formula,andequivalent pixels in the horizontal and vertical directions, unit mm/pixel,using camera No. 1 to make a click-through to two points in the measuring rangeMeasurement of line distance measurement.
The confidence level is 95%, and the extended uncertainty of the imaging system # 1 is，True value ofIt should satisfy:
（10）
in the formula,。
change ofpAndqrepeating the above process until the value is obtained（) All are linked, so subscripts are used in formula (9) and formula (10)kThe value range is，Should not be less than。
Equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
（11）
the telecentric vision system has good optical performance and the measurement precision is far higher than that of a binocular vision system. In addition, the measurement accuracy of the binocular vision system can reach 1/10mm, and the depth of field of the telecentric vision system is within the range. Therefore, based on the above two points, all equation (10) is used as the constraint condition, and equation (11) becomes the nonlinear constraint extremum problem:
（12）
using the solution of least squares as initial values, i.e.Solving equation (12) to obtain an updated solution. Will be provided withAnd the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be、、、、、And. Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to assemble the setAndthe element in (3) calculates its three-dimensional coordinates.
And finally, reconstructing a three-dimensional model of the part by using the three-dimensional coordinates obtained in the last step, and performing three-dimensional multi-factor measurement.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (5)
1. A part measuring method based on heterogeneous stereoscopic vision is characterized by comprising the following steps:
s1, determining the positions of a No. 1 telecentric industrial camera, a No. 2 ordinary industrial camera and a No. 3 ordinary industrial camera according to the size of the part to be measured and the depth of field of the cameras, keeping the position relation among the cameras, calibrating the cameras, and enabling the cameras to image the part to be measured to obtain a left image, a right image and a telecentric image;
s2, registering the left-right image pair and the left-right-telecentric image pair through processes including image preprocessing, feature extraction and feature matching;
s3, constructing a binocular vision system mathematical model according to the registration relation of the left image and the right image and the internal and external parameters calibrated by the common industrial camera to obtain the depth information of the characteristic points of the left image and the right image;
s4, grouping the feature points of the left-right-telecentric image into a group according to each 2 feature points, calculating the absolute difference of the depths of the 2 feature points in each group according to the depth information obtained in S3, and excluding the group if the absolute difference is greater than a preset threshold;
s5, projecting the feature points reserved in the S4 to a focal plane of a telecentric industrial camera, and calculating the distance of 2 projection points in each group along the horizontal and vertical coordinate directions;
s6, correcting the internal and external parameters marked in S1 according to the distance obtained in S5, reconstructing a binocular vision system mathematical model according to the registration relation of the left image and the right image obtained in S2 and the corrected internal and external parameters of the common industrial camera, and obtaining the depth information of the feature points of the updated left image and the updated right image;
and S7, constructing a three-dimensional model of the measured part according to the updated depth information, performing three-dimensional feature recognition, and extracting outline elements of the measured part, thereby realizing the measurement of the geometric features of the measured part.
2. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, the telecentric industrial camera is calibrated as follows: and imaging the calibration piece with the known size by the telecentric camera, and acquiring the ratio of the actual size of the calibration piece to the pixel size to obtain the equivalent pixel of the telecentric camera.
3. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, a normal industrial camera is calibrated as follows: after the common industrial camera is installed, the common industrial camera is calibrated by adopting a Zhangyingyou plane calibration method, and internal and external parameters of the common industrial camera are determined based on a calibration result.
4. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S4, feature points in the left image, the right image and the telecentric image are extracted, the homonymous features of the left image and the right image are found, the euclidean distance between the two feature points is measured, and if the absolute difference value is greater than a preset threshold value, the set of feature points is rejected.
5. The part measuring method based on the heterogeneous stereovision according to claim 1, wherein the function process of the part measuring method is as follows:
establishing a world coordinate systemCamera coordinate system of No. 1 telecentric industrial cameraCamera coordinate system of No. 2 common industrial cameraCamera coordinate system of No. 3 common industrial camera；
A certain point on the part to be measuredWorld coordinate isThe coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
（1）
in the formulaAndrepresentation from world coordinate system toj（j=1, 2, 3) coordinate transformation relations of the imaging system camera coordinate systems, also called imaging system extrinsic parameters;
by usingTo representIn the first placejAn image in an imaging system having coordinates ofWhereinj1, 2, 3, the relationship between the image point and the space point is:
wherein j =1 (2)
Wherein j =2, 3 (3)
In the formula u j Andv j are parameters within the imaging system that are,is a picture pointThe pixel coordinates of (a);
extracting the characteristic points of the images collected by the No. 1, No. 2 and No. 3 imaging systems, and establishing corresponding characteristic point combination sets、And；
from the collectionIn any one of the elements，Using search strategies and measure functions inFind out the 'same name' feature，Respectively to be separately provided withAnddeposit to New CollectionAndperforming the following steps;
from the collectionIn any one of the elements，In aIn which there is its "same name" characteristicUsing search strategies and measure functions in setsIs found inCharacteristic of (1)，Verification using a measure functionAndwhether it is a pair of "homonymous" features, and if so, it will、、Separately deposit into new collections、Andand if not, from the setIn the process of re-choosing an element，And isRepeating the above steps until the collection is finishedAll of the elements in (1);
the equations (1) and (3) are put together and rewritten as follows, and the spatial coordinates to be found are written to the left side of the equations:
（4）
in the formula，，k=1，2，3
Order to，，I =1, 2, …, m, characteristic pointHas pixel coordinates ofI.e., j = 2; characteristic pointHas pixel coordinates ofI.e., j = 3; substituting the pixel coordinates into equation (4),And：
（5）
spatial coordinates in formula (5)Unknown, and the remaining parameters are known, and rewriting formula (5) into formula (6):
（6）
in the formula，Andsee, respectively, formula (7) and formula (8):
（7）
（8）
solving the formula (6) by the least square method to obtainDefining a new set with the solved spatial coordinates as elements(ii) a From the collectionTwo elements of ZhongrenAnd，and is andcalculating the difference of z-coordinateIf, ifIs less than a given non-negative constantGo to CollectionFind and space pointAndthe corresponding pixel points have pixel coordinates ofAndand measuring by using a No. 1 imaging system to obtain:
（9）
in the formula,andequivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel units;
confidenceThe level is 95%, and the extended uncertainty of the No. 1 imaging system is recorded as，True value ofIt should satisfy:
（10）
in the formula,(ii) a Changing the values of p and q, and repeating the above processes until the values are changed（) All are linked, so that the subscript k is used in the formulae (9) and (10) in the range of，Should not be less than；
Equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
（11）
the measurement accuracy of the binocular vision system can reach 1/10mm, and in the field depth range of the telecentric vision system, based on the two points, the whole formula (10) is used as a constraint condition, and the formula (11) is changed into a nonlinear constraint extreme value problem:
（12）
using the solution of least squares as initial values, i.e.Solving equation (12) to obtain an updated solution(ii) a Will be provided withAnd the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be、、、、、And(ii) a The updated inside and outside of the cameraSubstituting the parameters into equation (6), and using equation (6) to assembleAndthe element in (3) calculates its three-dimensional coordinates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811194104.5A CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811194104.5A CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109579695A true CN109579695A (en) | 2019-04-05 |
| CN109579695B CN109579695B (en) | 2020-12-25 |
Family
ID=65920012
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811194104.5A Active CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109579695B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110470216A (en) * | 2019-07-10 | 2019-11-19 | 湖南交工智能技术有限公司 | A kind of three-lens high-precision vision measurement method and device |
| CN111426268A (en) * | 2020-04-20 | 2020-07-17 | 武汉大学中南医院 | Binocular vision camera matching method |
| CN111721197A (en) * | 2020-05-14 | 2020-09-29 | 南京工程学院 | Body model measuring device and method based on binocular stereo |
| CN111914857A (en) * | 2020-08-11 | 2020-11-10 | 上海柏楚电子科技股份有限公司 | Layout method, device and system for excess sheet material, electronic equipment and storage medium |
| CN112237416A (en) * | 2020-09-10 | 2021-01-19 | 北京信息科技大学 | Fundus multi-mode imaging system calibration method based on retinal surface blood vessel characteristics |
| CN112489109A (en) * | 2020-11-19 | 2021-03-12 | 广州视源电子科技股份有限公司 | Three-dimensional imaging system method and device and three-dimensional imaging system |
| CN112991460A (en) * | 2021-03-10 | 2021-06-18 | 哈尔滨工业大学 | Binocular measurement system, method and device for obtaining size of automobile part |
| CN114535825A (en) * | 2022-04-28 | 2022-05-27 | 安普森智能科技(江苏)有限公司 | Laser marking vehicle identification code system based on manipulator |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE202010004039U1 (en) * | 2010-03-22 | 2010-06-17 | Modus High-Tech Electronics Gmbh | Device for optical quality control of three-dimensional objects |
| CN204479038U (en) * | 2015-02-27 | 2015-07-15 | 福建浩蓝光电有限公司 | Novel high resolving power vertical 3 D scanner |
| CN106483642A (en) * | 2016-12-14 | 2017-03-08 | 舜宇光学(中山)有限公司 | A kind of doubly telecentric camera lens based on machine vision |
| US20180252896A1 (en) * | 2017-03-06 | 2018-09-06 | Spectrum Optix Inc. | Diamond shaped lens system |
-
2018
- 2018-10-15 CN CN201811194104.5A patent/CN109579695B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE202010004039U1 (en) * | 2010-03-22 | 2010-06-17 | Modus High-Tech Electronics Gmbh | Device for optical quality control of three-dimensional objects |
| CN204479038U (en) * | 2015-02-27 | 2015-07-15 | 福建浩蓝光电有限公司 | Novel high resolving power vertical 3 D scanner |
| CN106483642A (en) * | 2016-12-14 | 2017-03-08 | 舜宇光学(中山)有限公司 | A kind of doubly telecentric camera lens based on machine vision |
| US20180252896A1 (en) * | 2017-03-06 | 2018-09-06 | Spectrum Optix Inc. | Diamond shaped lens system |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110470216A (en) * | 2019-07-10 | 2019-11-19 | 湖南交工智能技术有限公司 | A kind of three-lens high-precision vision measurement method and device |
| CN111426268A (en) * | 2020-04-20 | 2020-07-17 | 武汉大学中南医院 | Binocular vision camera matching method |
| CN111721197A (en) * | 2020-05-14 | 2020-09-29 | 南京工程学院 | Body model measuring device and method based on binocular stereo |
| CN111721197B (en) * | 2020-05-14 | 2022-02-01 | 南京工程学院 | Body model measuring device and method based on binocular stereo |
| CN111914857A (en) * | 2020-08-11 | 2020-11-10 | 上海柏楚电子科技股份有限公司 | Layout method, device and system for excess sheet material, electronic equipment and storage medium |
| CN111914857B (en) * | 2020-08-11 | 2023-05-09 | 上海柏楚电子科技股份有限公司 | Layout method, device and system for plate excess material, electronic equipment and storage medium |
| CN112237416A (en) * | 2020-09-10 | 2021-01-19 | 北京信息科技大学 | Fundus multi-mode imaging system calibration method based on retinal surface blood vessel characteristics |
| CN112489109A (en) * | 2020-11-19 | 2021-03-12 | 广州视源电子科技股份有限公司 | Three-dimensional imaging system method and device and three-dimensional imaging system |
| CN112489109B (en) * | 2020-11-19 | 2022-10-21 | 广州视源电子科技股份有限公司 | Three-dimensional imaging system method and device and three-dimensional imaging system |
| CN112991460A (en) * | 2021-03-10 | 2021-06-18 | 哈尔滨工业大学 | Binocular measurement system, method and device for obtaining size of automobile part |
| CN112991460B (en) * | 2021-03-10 | 2021-09-28 | 哈尔滨工业大学 | Binocular measurement system, method and device for obtaining size of automobile part |
| CN114535825A (en) * | 2022-04-28 | 2022-05-27 | 安普森智能科技(江苏)有限公司 | Laser marking vehicle identification code system based on manipulator |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109579695B (en) | 2020-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109579695B (en) | Part measuring method based on heterogeneous stereoscopic vision | |
| CN106595528B (en) | A kind of micro- binocular stereo vision measurement method of telecentricity based on digital speckle | |
| Yang et al. | Robust and accurate surface measurement using structured light | |
| CN108986070B (en) | Rock crack propagation experiment monitoring method based on high-speed video measurement | |
| CN102376089B (en) | Target correction method and system | |
| CN109242908B (en) | Calibration method for underwater binocular vision measurement system | |
| CN105252341B (en) | Five-axle number control machine tool dynamic error vision measuring method | |
| CN110044374B (en) | Image feature-based monocular vision mileage measurement method and odometer | |
| CN110378969B (en) | Convergent binocular camera calibration method based on 3D geometric constraint | |
| CN102221331B (en) | Measuring method based on asymmetric binocular stereovision technology | |
| CN109859272A (en) | A kind of auto-focusing binocular camera scaling method and device | |
| CN110223355B (en) | Feature mark point matching method based on dual epipolar constraint | |
| CN109961485A (en) | A method of target positioning is carried out based on monocular vision | |
| CN112161997B (en) | Online precise visual measurement method and system for three-dimensional geometric dimension of semiconductor chip pin | |
| CN109163657A (en) | A kind of circular target position and posture detection method rebuild based on binocular vision 3 D | |
| CN106996748A (en) | Wheel diameter measuring method based on binocular vision | |
| CN106500625A (en) | A kind of telecentricity stereo vision measuring apparatus and its method for being applied to the measurement of object dimensional pattern micron accuracies | |
| CN109974618A (en) | The overall calibration method of multisensor vision measurement system | |
| CN104167001B (en) | Large-visual-field camera calibration method based on orthogonal compensation | |
| CN116051659A (en) | Linear array camera and 2D laser scanner combined calibration method | |
| CN102881040A (en) | Three-dimensional reconstruction method for mobile photographing of digital camera | |
| CN111583342A (en) | Target rapid positioning method and device based on binocular vision | |
| CN105434046B (en) | Based on the surgical navigator localization method for eliminating infrared fileter refraction effect | |
| CN114170321A (en) | Camera self-calibration method and system based on distance measurement | |
| CN104123726A (en) | Blanking point based large forging measurement system calibration method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing University of Engineering Science Park Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980001093 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240122 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing Jinxi Yunchuang Technology Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980001805 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240202 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing Institute of Engineering Technical Service Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980002478 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240305 |