1、Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China合肥工業(yè)大學(xué) 儀器科學(xué)與光電工程學(xué)院 光機(jī)電工程研究所盧盧 榮榮 勝勝Tel: 2905872 Email: 機(jī)器視覺(jué)三維測(cè)量技術(shù)機(jī)器視覺(jué)三維測(cè)量技術(shù)3D Machine Vision Measurement TechniquesInstitute of Opto-mechatronics Engineering, Hefei University of Tech

2、nology, Hefei, Anhui, 230009, China大 綱1. 機(jī)器視覺(jué)的概念2. 攝像機(jī)模型3. 三維視覺(jué)傳感器的類型4. 多視覺(jué)傳感器測(cè)量系統(tǒng)的組成5. 攝像機(jī)與視覺(jué)傳感器的標(biāo)定6. 測(cè)量系統(tǒng)的全局標(biāo)定7. 應(yīng)用舉例Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China機(jī)器視覺(jué)技術(shù)的概念機(jī)器視覺(jué)技術(shù)的概念測(cè)量測(cè)量檢測(cè)檢測(cè)識(shí)別識(shí)別 引導(dǎo)引導(dǎo)光源傳感器光學(xué)鏡組傳感單元信號(hào)處理單元控制單元 采用光學(xué)成像技術(shù)（通常使用攝像機(jī)）獲取

3、被測(cè)目標(biāo)的圖像，再經(jīng)過(guò)快速圖像處理與圖形識(shí)別算法，從攝取圖像中獲取目標(biāo)的尺寸、位置、方向、光譜特征、結(jié)構(gòu)以及缺陷等信息，從而可以執(zhí)行產(chǎn)品檢驗(yàn)、分類與分組，裝配線上的機(jī)械手運(yùn)動(dòng)引導(dǎo)、零部件的識(shí)別與定位，生產(chǎn)過(guò)程中質(zhì)量監(jiān)控與過(guò)程控制反饋等等任務(wù)。Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China應(yīng)用領(lǐng)域舉例應(yīng)用領(lǐng)域舉例 1. IC及一般電子業(yè)：PCB、BGA、LCD屏幕、被動(dòng)組件形狀腳位及定位、生產(chǎn)插件、晶元(Wafer)鏡面研磨、生產(chǎn)組裝、被

4、動(dòng)組件辨識(shí)2. MEMS 組件：微小器件的尺寸、形貌、表面缺陷的檢測(cè)，組件動(dòng)態(tài)特性的測(cè)量等。3. 機(jī)械工具/自動(dòng)化機(jī)械：零件尺寸、外形、瑕疵檢測(cè)、零件分類比對(duì)、裝配定位、加工定位、熔焊檢測(cè)4. 電機(jī)工業(yè)：控制器紅外線熱像儀檢測(cè)、電線瑕疵、裂縫檢測(cè)、纜線配置檢測(cè)5. 金屬鋼鐵業(yè)：鋼板尺寸檢測(cè)、表面瑕疵檢測(cè)、鑄件瑕疵檢測(cè)、材料金像檢測(cè)6. 橡膠/塑料制品：保特瓶口尺寸檢測(cè)、制品顏色分類檢測(cè)7. 食品加工/包裝業(yè)：瓶?jī)?nèi)液位高度、異物或灰塵檢測(cè)、包裝印刷辨識(shí)、打印字形及零件編號(hào)檢測(cè)與識(shí)別8. 紡織皮革工業(yè)：表面針織紋路檢測(cè)、色差檢測(cè)、皮革表面特性檢測(cè)9. 汽車工業(yè)：陶軸裁切定位、零件涂黃油檢測(cè)、白車身

5、檢驗(yàn)10. 家電/辦公：產(chǎn)品外殼印刷檢測(cè)、1維/2維條形碼辨識(shí)11. 保全/監(jiān)視系統(tǒng)：人像特征辨識(shí)、指紋辨識(shí)等Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China機(jī)器視覺(jué)在精密測(cè)量中的應(yīng)用照明光源 圖像傳感器成像內(nèi)部三維表面表面輪廓表面輪廓粗糙度粗糙度缺陷缺陷紋理紋理內(nèi)部結(jié)構(gòu)內(nèi)部結(jié)構(gòu)應(yīng)力應(yīng)力變形變形缺陷缺陷被測(cè)場(chǎng)景方位方位三維形貌三維形貌尺寸尺寸坐標(biāo)坐標(biāo)Institute of Opto-mechatronics Engineering, H

6、efei University of Technology, Hefei, Anhui, 230009, China二、攝像機(jī)的建模1.影射幾何與坐標(biāo)變換2.攝像機(jī)建模3.鏡頭畸變4.攝像機(jī)內(nèi)部參數(shù)的修正5.攝像機(jī)的精確模型6.攝像機(jī)的簡(jiǎn)化模型Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China歐式幾何與影射幾何的概念 xyzxyz不變量不變量: Distance and angle不變量不變量: Cross ratiobacdbdadbca

7、cd , c;b ,aRInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China齊次矩陣fzYyXx101000000001zyxffYXzyxffzYX00001zyfYzxfXInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China坐標(biāo)變換xyz110001333231232221131211w

8、wwzyxzyxtrrrtrrrtrrrzyxwxwywzzyxwwwtttzyxrrrrrrrrrzyx333231232221131211Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China眼睛的光學(xué)模型眼睛的光學(xué)模型像平面物體透鏡fImage planeobjectlensInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

9、Anhui, 230009, China攝像機(jī)的光學(xué)投影變換攝像機(jī)的光學(xué)投影變換Image planeobjectlensfzYyXxxyXYzImage planeFocus planefz , y, xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)的光學(xué)投影變換攝像機(jī)的光學(xué)投影變換 （續(xù)）（續(xù)）fzYyXxfzYyXxxyXYzImage planefY,Xz,y,xxXYzImage planeFocus planefz ,

10、y, xy101000000001zyxffYXInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China像平面與幀存體的對(duì)應(yīng)關(guān)系110000)(10101YXvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer framexdyd)( )(00uudYuudsXyxxInstitute of Opto-mechatronics Engineering, Hefei University o

11、f Technology, Hefei, Anhui, 230009, China幀存體與攝像機(jī)坐標(biāo)的關(guān)系1010000000010000)(10101zyxffvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer frameInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2.4 攝像機(jī)建模xyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer

12、framewywxwz110001333231232221131211wwwzyxzyxtrrrtrrrtrrrzyx11000010000000010000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrtrrrffvdudsvuInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)建模（續(xù)）110000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrt

13、rrrvfdufdsvuMtRAm| 未知參數(shù)：00,vusddfxyx(1) Intrinsic parameters:(2) Extrinsic parameters333231232221131211r ,r ,r ,r ,r ,r ,r ,r ,rOrientation: Translation:zyxt ,t ,t69, independent 33Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China鏡頭畸變的概念鏡頭畸變的概念xy

14、XYzImage planefddY,Xz,y,xRadial distortionTangential distortionDecentering distortionThin prism distortionInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China徑向畸變5231rkrkr22ddYXrsinrYcosrXdd 22222212222221ddddddyrddddddxrYXYkYXYkYXXkYXXkInstitute of

15、 Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China偏心畸變0012011cossinjpjp偏心畸變既含有徑向畸變又有切向畸變04221042213cosrjrjsinrjrjtdrdtdrdydxdcossinsincos最大切向畸變與 x 軸的夾角, 設(shè)在直角坐標(biāo)系下22212221223ddddydddddxdYXpYXpYXpYXpInstitute of Opto-mechatronics Engineering, Hefei University of

16、 Technology, Hefei, Anhui, 230009, China薄棱鏡畸變1422114221cosririsinriritprp11211cosissinis薄棱鏡畸變是由鏡頭與攝像機(jī)面有小的傾角造成的1最大切向畸變與 x 軸的夾角, 設(shè)221221ddypddxpYXsYXsInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China鏡頭總畸變鏡頭總畸變 ypydyrdxpxdxrdYYXX,s ,p,p,k ,k ,Y ,XYY

17、,s ,p,p,k ,k ,Y ,XXXddydddxd121211212111ydxdYYXX11)()(2)1 ()(2)3 ()1 (2222221422122122214221dddddddyddddddddxdYXsYXpYXprkrkYYXsYXpYXprkrkX若考慮分項(xiàng)的前兩項(xiàng)化簡(jiǎn)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)內(nèi)部參數(shù)的修正101000000001)1 ()1 (zyxffYXydxd1010000

18、)1/(0000)1/(1zyxffYXyxdd1001/001/)(A0101vfdufdsyyxxx,1212100sppkkvusddfxyx Intrinsic parameters7Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)的精確模型計(jì)算機(jī)屏幕uvO00,vuv ,uwywxwzm11001/001/)(13332312322211312110101wwwzyxyyxxxzyxtrrrtrrrtrrrvfdufdsv

19、uxyXYzImage planefddY,Xz,y,xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)的精確模型 （續(xù)）)1 ()1 ()()(98765498732100ydzwwwywwwxdzwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfvvdYuudsX計(jì)算機(jī)屏幕uvO00,vuv ,uwywxwzmdydddxXprkXYXprk12121212/3一般情況下，取徑向與切向畸變的

20、的第一項(xiàng)就可以達(dá)到滿意的效果xyXYzImage planefddY,Xz,y,xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China6 攝像機(jī)的簡(jiǎn)化模型1000A00vfusfvuMtRAm| 1vum1wwwzyxM333231232221131211rrrrrrrrrRzyxttttxyXYz像平面fY,Xz,y,x計(jì)算機(jī)屏幕uvO00,vuv ,uwywxwzMmInstitute of Opto-mechatronics Engin

21、eering, Hefei University of Technology, Hefei, Anhui, 230009, China三、視覺(jué)傳感器的類型1.主動(dòng)與被動(dòng)2.單目與雙目3.點(diǎn)、線、面結(jié)構(gòu)光4.編碼結(jié)構(gòu)光Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China三角法測(cè)量原理激光器鏡頭CCD 鏡頭xCCD 激光器鏡頭鏡頭Institute of Opto-mechatronics Engineering, Hefei Universit

22、y of Technology, Hefei, Anhui, 230009, China視覺(jué)傳感器的坐標(biāo)系統(tǒng)zwxwywTyTxTzxw0yw0 xsyszsosPointOXYOzxyLaserImage planeOrigin Sensor coordinate system (SCS)Global Coordinate system (GCS)Camera coordinate system (CCS)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 2300

23、09, China單一攝像機(jī)二維傳感器單一攝像機(jī)二維傳感器Image planesssz ,y,xPscyyscxxscooY ,XPXYfMeasured plane(CCS)SCSsczzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China點(diǎn)結(jié)構(gòu)光視覺(jué)傳感器syszsxcocxcyXYZpqPQOCCSSCSLaserInstitute of Opto-mechatronics Engineering, Hefei University

24、of Technology, Hefei, Anhui, 230009, China線結(jié)構(gòu)光視覺(jué)傳感器Laser Light planepPXYcxcyImage planesxsyszCCSSCSczInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China條紋結(jié)構(gòu)光視覺(jué)傳感器條紋結(jié)構(gòu)光視覺(jué)傳感器The k-th light stripe planeImage planeProjector patternkxzyxzywFvuCFoknkF3In

25、stitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaXYcxcyImage planesyszCCSSCScz00110010011001sxProjector條紋編碼三維視覺(jué)傳感器Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China彩色編碼視覺(jué)傳感器彩色編碼視覺(jué)傳感器cXcYcxcyspyyp

26、XpYczspxxprojectorcameraCCSSCSobjectspzzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China被動(dòng)雙目視覺(jué)傳感器syy11X2Ysxx1lamp2X2Y2x2y1zcamera 2CCS 2(CCS1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

27、Anhui, 230009, China編碼照明雙目視覺(jué)傳感器 syy11X2Ysxx1projector2X2Y2x2yszz1camera 2CCS 2(CCS 1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China多視覺(jué)傳感器測(cè)量系統(tǒng)的組成Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hef

28、ei, Anhui, 230009, ChinaCAN總線網(wǎng)絡(luò)控制視覺(jué)測(cè)量系統(tǒng)總線網(wǎng)絡(luò)控制視覺(jué)測(cè)量系統(tǒng)圖像采集卡圖像采集卡計(jì)算機(jī)計(jì)算機(jī)多路視頻切換開(kāi)關(guān)多路視頻切換開(kāi)關(guān)CAN 接口卡接口卡CAN 網(wǎng)絡(luò)控制總線網(wǎng)絡(luò)控制總線傳感器控制單元傳感器控制單元傳感器傳感器視頻信號(hào)數(shù)據(jù)總線視頻信號(hào)數(shù)據(jù)總線 Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1394總線并行圖像處理系統(tǒng)總線并行圖像處理系統(tǒng)傳感器控制單元傳感器控制單元傳感器傳感器計(jì)算機(jī)計(jì)算機(jī)接

29、口卡接口卡數(shù)據(jù)與控制總線數(shù)據(jù)與控制總線 Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China基于工控PC機(jī)的多計(jì)算機(jī)分布式并行圖像處理架構(gòu)PC控制器運(yùn)動(dòng)驅(qū)動(dòng)機(jī)構(gòu)可編程光源可編程光源多PC并行處理器HUB生產(chǎn)與管理系統(tǒng)主機(jī)辦公電腦遠(yuǎn)端大型資料庫(kù)視覺(jué)系統(tǒng)主控計(jì)算機(jī)PC三軸控制器攝像機(jī)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

30、Anhui, 230009, China常用總線的比較 1.RS232 小于15m傳輸速率25kb/s2.RS485 最大傳輸速度10Mb/s3.CAN 小于40m傳輸速率1Mb/s4.USB 最高速率480Mb/s，可掛接127個(gè)外設(shè)5.CameraLink 最高4.8Gbps，最遠(yuǎn)可傳輸10米。6.Gigabit ethernet 帶寬可達(dá)到1000 Mbps， 最遠(yuǎn)可傳輸100米。7. Firewire，最遠(yuǎn)可傳輸100米，帶寬3.2GInstitute of Opto-mechatronics Engineering, Hefei University of Technology, H

31、efei, Anhui, 230009, China雙總線架構(gòu)總線接口PC多路視頻切換器視覺(jué)傳感器1視覺(jué)傳感器N視覺(jué)傳感器2控制總線視頻圖像圖像采集處理平臺(tái)數(shù)據(jù)總線Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China單總線架構(gòu)單總線架構(gòu)計(jì)算機(jī)總線接口視覺(jué)傳感器 1總線總線視覺(jué)傳感器 2視覺(jué)傳感器 N 傳感器控制、圖像采集與并行處理模塊 傳感器控制、圖像采集與并行處理模塊 傳感器控制、圖像采集與并行處理模塊Institute of Opto-m

32、echatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機(jī)內(nèi)部參數(shù)的標(biāo)定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1 攝像機(jī)攝標(biāo)定靶標(biāo)攝像機(jī)攝標(biāo)定靶標(biāo)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, An

33、hui, 230009, China標(biāo)定靶標(biāo)的控制點(diǎn)類型標(biāo)定靶標(biāo)的控制點(diǎn)類型幾何中心與投影中心的偏移Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2 直接線性變換法(DLT)1 用一個(gè)齊次矩陣建立場(chǎng)景與控制點(diǎn)的對(duì)應(yīng)關(guān)系11343332312423222114131211wiwiwiiiizyxaaaaaaaaaaaavuw2 從上式中解出DLT矩陣參數(shù)3411aa0La nnwnnwnnwnwnwnwnnnwnnwnnwnwnwnwni

34、iwiiwiiwiwiwiwiiiwiiwiiwiwiwiwiwwwwwwwwwwwwvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxLaaaaaaaaa1000000001100000000110000000010,11111111111111111111333231232221131211aInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接線性變換法(DLT)在

35、最小二乘意義下可求出參數(shù) ，為了避免解很小的 ，必須使用合適的正則化方法，一種方法是令 然后用偽逆法求解，但當(dāng) ， 這種正則化可能產(chǎn)生奇異性，為了克服這個(gè)缺點(diǎn)，可以令3411aa3411aa034a134a1234232231aaa3 矩陣參數(shù)分解：RQ 方法FMTBVA11100100100vuV10001011221bbbbB1000000ffF矩陣M和T分別表示靶標(biāo)世界坐標(biāo)系相對(duì)于攝像機(jī)坐標(biāo)系的旋轉(zhuǎn)矢量與平移矢量（b1 b2) 為仿射失真系數(shù)Institute of Opto-mechatronics Engineering, Hefei University of Technology

36、, Hefei, Anhui, 230009, China3 RAC 兩步法徑向準(zhǔn)直約束：1）徑向畸變不影響矢量 的方向2）有效焦距不影響矢量 的方向3）光軸沿Z軸平移不影響矢量 的方向uOPuOPuOP該方法第一步采用最小二乘方法求解超定方程組求出外部參數(shù)，第二步求解內(nèi)部參數(shù) , 91rrxyx, s ,tt1k , f , zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 計(jì)算：0PPOP0d0yxYXddyxYXddywwwxww

37、wtzryrxrytzryrxrx654321)()(00vvdYYuudXsXyddxdxddndidyyyyxxyxyxyxdnwndnwndnwndndnwndnwndnwndiwidiwidiwididiwidiwidwidwdwdwddwdwdwXXXtrtrtrttstrstrstrsYzYyYxYYzYyYxYzYyYxYYzYyYxYzYyYxYYzYyYx165432111111111111111/, 91rrxyx, s ,tt（1）（2）yddwdwdwxddwdwdwTXrYzrYyrYxtYrYzrYyrYx654321（3）令把（2）代入（1）Institute o

38、f Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China傳感器坐標(biāo)系與攝像機(jī)外部參數(shù)的標(biāo)定Laser Light planepPXYcxcyImage planesxsyszCCSSCSczuizwwwywwwuizwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfuudYuudsX98765498732100)()()1 ()1 ()()(00yduixduiiydixxdiYYXXvvdYuudsX00011198653297

39、6431875421292623282522272421rrrrrrrrrrrrrrrrrrrrrrrrrrrInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 測(cè)量系統(tǒng)的全局標(biāo)定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget1 標(biāo)定的基本原理標(biāo)定的基本原理TCSTheo

40、dolitesGCSMeasured object SCSVisual sensorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget標(biāo)定的基本原理標(biāo)定的基本原理TCSLaser TrackerGCSMeasured object SCSVisual sensorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China經(jīng)緯儀的標(biāo)定經(jīng)緯儀的標(biāo)定abRulerTCSABSensor 2Sensor 1Sensor 4Sensor 3Sensor 6Sensor 5SCS (WCS)targetlight planeGCSInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接變換標(biāo)定法直接變換標(biāo)定法 sgssggTPRP987