stereo matching立体匹配图片3_lua_其他

<html xmlns="http://www.w3.org/1999/xhtml"><head><meta charset="utf-8"><meta name="generator" content="pdf2htmlEX"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><link rel="stylesheet" href="https://csdnimg.cn/release/download_crawler_static/css/base.min.css"><link rel="stylesheet" href="https://csdnimg.cn/release/download_crawler_static/css/fancy.min.css"><link rel="stylesheet" href="https://csdnimg.cn/release/download_crawler_static/9325663/raw.css"><script src="https://csdnimg.cn/release/download_crawler_static/js/compatibility.min.js"></script><script src="https://csdnimg.cn/release/download_crawler_static/js/pdf2htmlEX.min.js"></script><script>try{pdf2htmlEX.defaultViewer = new pdf2htmlEX.Viewer({});}catch(e){}</script><title></title></head><body><div id="sidebar" style="display: none"><div id="outline"></div></div><div id="pf1" class="pf w0 h0" data-page-no="1"><div class="pc pc1 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://csdnimg.cn/release/download_crawler_static/9325663/bg1.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">Evaluation of Cost Functions for Stereo Matching</div><div class="t m0 x2 h3 y2 ff2 fs1 fc0 sc0 ls0 ws0">Heiko Hirschm¨uller</div><div class="t m0 x3 h3 y3 ff2 fs1 fc0 sc0 ls0 ws0">Institute of Roboti cs and Mechatronics Oberpfaffenhofen</div><div class="t m0 x4 h3 y4 ff2 fs1 fc0 sc0 ls0 ws0">German Aerospace Center (DLR)</div><div class="t m1 x5 h4 y5 ff3 fs2 fc0 sc0 ls0 ws0">heiko.hirschmueller@dlr.de</div><div class="t m0 x6 h3 y6 ff2 fs1 fc0 sc0 ls0 ws0">Daniel Scharstein</div><div class="t m0 x7 h3 y7 ff2 fs1 fc0 sc0 ls0 ws0">Middlebury College</div><div class="t m0 x8 h3 y8 ff2 fs1 fc0 sc0 ls0 ws0">Middlebury, VT, USA</div><div class="t m1 x9 h4 y5 ff3 fs2 fc0 sc0 ls0 ws0">schar@middlebury.edu</div><div class="t m0 xa h5 y9 ff1 fs1 fc0 sc0 ls0 ws0">Abstract</div><div class="t m0 xb h6 ya ff4 fs3 fc0 sc0 ls0 ws0">Stereo correspondence methods rely on matching costs</div><div class="t m0 xc h6 yb ff4 fs3 fc0 sc0 ls0 ws0">for computing the similarity of image locations. In this pa-</div><div class="t m0 xc h6 yc ff4 fs3 fc0 sc0 ls0 ws0">per we evaluate the insensitivity of differen t matching costs</div><div class="t m0 xc h6 yd ff4 fs3 fc0 sc0 ls0 ws0">with respect to radiometric variations of the input images.</div><div class="t m0 xc h6 ye ff4 fs3 fc0 sc0 ls0 ws0">We consider both pixel-based and window-based variants</div><div class="t m0 xc h6 yf ff4 fs3 fc0 sc0 ls0 ws0">and measure their perfo rmance in the presence of global</div><div class="t m0 xc h6 y10 ff4 fs3 fc0 sc0 ls0 ws0">intensity changes (e.g., due to gain and exposure differ-</div><div class="t m0 xc h6 y11 ff4 fs3 fc0 sc0 ls0 ws0">ences), loc al intensity changes (e.g., due to vignetting, non-</div><div class="t m0 xc h6 y12 ff4 fs3 fc0 sc0 ls0 ws0">Lambertian surfaces, and varying lighting), and noise. Us-</div><div class="t m0 xc h6 y13 ff4 fs3 fc0 sc0 ls0 ws0">ing existing stereo datasets with ground-truth disparities as</div><div class="t m0 xc h6 y14 ff4 fs3 fc0 sc0 ls0 ws0">well as six new datasets taken under controlled changes of</div><div class="t m0 xc h6 y15 ff4 fs3 fc0 sc0 ls0 ws0">exposure and lighting, we evaluate the different costs with a</div><div class="t m0 xc h6 y16 ff4 fs3 fc0 sc0 ls0 ws0">local, a semi-global, and a global stereo method.</div><div class="t m0 xc h5 y17 ff1 fs1 fc0 sc0 ls0 ws0">1. Introduction and Related Work</div><div class="t m0 xb h7 y18 ff2 fs3 fc0 sc0 ls0 ws0">All stereo correspondence algorithms have a way of</div><div class="t m0 xc h7 y19 ff2 fs3 fc0 sc0 ls0 ws0">measuring the similarity of image locations. Typically, a</div><div class="t m0 xc h7 y1a ff4 fs3 fc0 sc0 ls0 ws0">matching cost is computed at each pixel for all dispari-</div><div class="t m0 xc h7 y1b ff2 fs3 fc0 sc0 ls0 ws0">ties under con sideration. The simplest matching c osts as-</div><div class="t m0 xc h7 y1c ff2 fs3 fc0 sc0 ls0 ws0">sume constant intensities at matching image lo c ations, but</div><div class="t m0 xc h7 y1d ff2 fs3 fc0 sc0 ls0 ws0">more robust costs model (explicitly or implicitly) certain</div><div class="t m0 xc h7 y1e ff2 fs3 fc0 sc0 ls0 ws0">radiometric c hanges and/o r noise. Common pixel-based</div><div class="t m0 xc h7 y1f ff2 fs3 fc0 sc0 ls0 ws0">matching c osts inc lude absolute differences, sq uared dif -</div><div class="t m0 xc h7 y20 ff2 fs3 fc0 sc0 ls0 ws0">ferences, sampling-insensitive absolute differences [2], or</div><div class="t m0 xc h7 y21 ff2 fs3 fc0 sc0 ls0 ws0">truncated versions, both on gray and color images. Com-</div><div class="t m0 xc h7 y22 ff2 fs3 fc0 sc0 ls0 ws0">mon window-based matching costs inc lude the sum of abso-</div><div class="t m0 xc h7 y23 ff2 fs3 fc0 sc0 ls0 ws0">lute or squared differences (SAD / SSD), normalized cross-</div><div class="t m0 xc h7 y24 ff2 fs3 fc0 sc0 ls0 ws0">correlation (NCC), and rank and census transforms [23].</div><div class="t m0 xc h7 y25 ff2 fs3 fc0 sc0 ls0 ws0">Some wind ow-based costs can be implemented efﬁciently</div><div class="t m0 xc h7 y26 ff2 fs3 fc0 sc0 ls0 ws0">using ﬁlters. For example, the rank transform can be com-</div><div class="t m0 xc h7 y27 ff2 fs3 fc0 sc0 ls0 ws0">puted using a rank ﬁlter followed by absolute differences of</div><div class="t m0 xc h7 y28 ff2 fs3 fc0 sc0 ls0 ws0">the ﬁlter results. Similarly, there are other ﬁlters that try to</div><div class="t m0 xc h7 y29 ff2 fs3 fc0 sc0 ls0 ws0">remove bias or gain changes, e.g., LoG and mean ﬁlters.</div><div class="t m0 xb h7 y2a ff2 fs3 fc0 sc0 ls0 ws0">More co mplicated similarity m e asures are p ossible, in-</div><div class="t m0 xc h7 y2b ff2 fs3 fc0 sc0 ls0 ws0">cluding mutual information [7, 9, 11] and approximative</div><div class="t m0 xc h7 y2c ff2 fs3 fc0 sc0 ls0 ws0">segment-wise mutual information as used in the layered</div><div class="t m0 xc h7 y2d ff2 fs3 fc0 sc0 ls0 ws0">stereo approach of Zitnick et al. [24].</div><div class="t m0 xd h7 y9 ff2 fs3 fc0 sc0 ls0 ws0">Recent stereo surveys [5, 17] and the Middlebury online</div><div class="t m0 xe h7 y2e ff2 fs3 fc0 sc0 ls0 ws0">evaluation [14] compar e state-of-the-a rt stereo method s on</div><div class="t m0 xe h7 ya ff2 fs3 fc0 sc0 ls0 ws0">test data with complex ge ometries an d varied texture. Other</div><div class="t m0 xe h7 yb ff2 fs3 fc0 sc0 ls0 ws0">evaluations focus on certain aspects like aggregation meth-</div><div class="t m0 xe h7 yc ff2 fs3 fc0 sc0 ls0 ws0">ods for real-time matching [21]. However, the insensitivity</div><div class="t m0 xe h7 yd ff2 fs3 fc0 sc0 ls0 ws0">of m atching costs is not evaluated since the stereo test sets</div><div class="t m0 xe h7 ye ff2 fs3 fc0 sc0 ls0 ws0">are typically pa irs of radiometr ically very similar images.</div><div class="t m0 xd h7 yf ff2 fs3 fc0 sc0 ls0 ws0">The term radiometrically similar means that pixels that</div><div class="t m0 xe h7 y10 ff2 fs3 fc0 sc0 ls0 ws0">correspo nd to the same scene point have similar or ideally</div><div class="t m0 xe h7 y11 ff2 fs3 fc0 sc0 ls0 ws0">the sam e values in the images. Radiometric differences can</div><div class="t m0 xe h7 y12 ff2 fs3 fc0 sc0 ls0 ws0">be caused by the ca mera(s) due to slightly different settings,</div><div class="t m0 xe h7 y13 ff2 fs3 fc0 sc0 ls0 ws0">vignetting, image noise, etc. Further differences may be due</div><div class="t m0 xe h7 y14 ff2 fs3 fc0 sc0 ls0 ws0">to non- Lambertian surfaces, which make the amount of re-</div><div class="t m0 xe h7 y15 ff2 fs3 fc0 sc0 ls0 ws0">ﬂected light dependent on the viewing angle. Finally, the</div><div class="t m0 xe h7 y16 ff2 fs3 fc0 sc0 ls0 ws0">strength or positions of the light sources may change when</div><div class="t m0 xe h7 y2f ff2 fs3 fc0 sc0 ls0 ws0">images of a static scene are acquired at different times, as</div><div class="t m0 xe h7 y30 ff2 fs3 fc0 sc0 ls0 ws0">is the case when matching aerial or satellite images. In a ll</div><div class="t m0 xe h7 y31 ff2 fs3 fc0 sc0 ls0 ws0">cases, methods are required that can handle radiometric dif-</div><div class="t m0 xe h7 y32 ff2 fs3 fc0 sc0 ls0 ws0">ferences.</div><div class="t m0 xd h7 y33 ff2 fs3 fc0 sc0 ls0 ws0">The scope of this paper is the evaluation and compari-</div><div class="t m0 xe h7 y34 ff2 fs3 fc0 sc0 ls0 ws0">son of some widely used stereo matching costs on images</div><div class="t m0 xe h7 y35 ff2 fs3 fc0 sc0 ls0 ws0">with several common radiometric differences. The focus</div><div class="t m0 xe h7 y36 ff2 fs3 fc0 sc0 ls0 ws0">is o n matching costs that explicitely or implicitly handle</div><div class="t m0 xe h7 y37 ff2 fs3 fc0 sc0 ls0 ws0">radiometric differences. Th is excludes popular methods</div><div class="t m0 xe h7 y38 ff2 fs3 fc0 sc0 ls0 ws0">like the correlation- based weighting according to proxim-</div><div class="t m0 xe h7 y39 ff2 fs3 fc0 sc0 ls0 ws0">ity and color similarity [22], as this is an a ggregation ap-</div><div class="t m0 xe h7 y3a ff2 fs3 fc0 sc0 ls0 ws0">proach that uses th e truncated absolute difference as match-</div><div class="t m0 xe h7 y3b ff2 fs3 fc0 sc0 ls0 ws0">ing cost. Furthermore, only methods that work on a single</div><div class="t m0 xe h7 y3c ff2 fs3 fc0 sc0 ls0 ws0">stereo pair with unkn own radiometric distortions and light</div><div class="t m0 xe h7 y3d ff2 fs3 fc0 sc0 ls0 ws0">sources are evaluated, according to the considered applica-</div><div class="t m0 xe h7 y3e ff2 fs3 fc0 sc0 ls0 ws0">tions. This excludes methods that explicitly handle non-</div><div class="t m0 xe h7 y3f ff2 fs3 fc0 sc0 ls0 ws0">Lambertian sur faces by taking at least two stereo images</div><div class="t m0 xe h7 y40 ff2 fs3 fc0 sc0 ls0 ws0">with different illuminations [6] or methods that require cal-</div><div class="t m0 xe h7 y41 ff2 fs3 fc0 sc0 ls0 ws0">ibrated lig ht sources.</div><div class="t m0 xe h5 y42 ff1 fs1 fc0 sc0 ls0 ws0">2. Matching Costs and Stereo Methods</div><div class="t m0 xd h7 y2a ff2 fs3 fc0 sc0 ls0 ws0">It is important to distinguish between matching costs and</div><div class="t m0 xe h7 y2b ff2 fs3 fc0 sc0 ls0 ws0">methods that use these costs. In this paper we compare 6</div><div class="t m0 xe h7 y2c ff2 fs3 fc0 sc0 ls0 ws0">costs and 3 stereo methods. We consider all possible com-</div><div class="t m0 xe h7 y2d ff2 fs3 fc0 sc0 ls0 ws0">binations to fully evaluate the insensitivity of each cost.</div></div><div class="pi" data-data='{"ctm":[1.611639,0.000000,0.000000,1.611639,0.000000,0.000000]}'></div></div></body></html>

<div id="pf2" class="pf w0 h0" data-page-no="2"><div class="pc pc2 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://csdnimg.cn/release/download_crawler_static/9325663/bg2.jpg"><div class="t m0 xc h8 y43 ff1 fs4 fc0 sc0 ls0 ws0">2.1. Matching Costs</div><div class="t m0 xb h7 y44 ff2 fs3 fc0 sc0 ls0 ws0">Our ﬁrst cost function is the com monly-used absolute</div><div class="t m0 xc h7 y45 ff4 fs3 fc0 sc0 ls0 ws0">difference, which assumes brightness constancy for corre-</div><div class="t m0 xc h7 y46 ff2 fs3 fc0 sc0 ls0 ws0">sponding pixels, and which serves as a baseline perfor-</div><div class="t m0 xc h7 y47 ff2 fs3 fc0 sc0 ls0 ws0">mance measure of our evaluation. Local stereo me thods</div><div class="t m0 xc h7 y48 ff2 fs3 fc0 sc0 ls0 ws0">usually aggregate the sum of absolute differences (SAD)</div><div class="t m0 xc h7 y49 ff2 fs3 fc0 sc0 ls0 ws0">over a window, while global methods use the differences</div><div class="t m0 xc h7 y4a ff2 fs3 fc0 sc0 ls0 ws0">pixel-wise. In both cases we use the sampling-insensitive</div><div class="t m0 xc h7 y4b ff2 fs3 fc0 sc0 ls0 ws0">calculation of Birchﬁeld and Tomasi ( BT) [2].</div><div class="t m0 xb h7 y4c ff2 fs3 fc0 sc0 ls0 ws0">Our next three cost fu nctions can be implemented as ﬁl-</div><div class="t m0 xc h7 y4d ff2 fs3 fc0 sc0 ls0 ws0">ters that are applied separately to the input images. The</div><div class="t m0 xc h7 y4e ff2 fs3 fc0 sc0 ls0 ws0">transformed images are then matched using the absolute dif-</div><div class="t m0 xc h7 y4f ff2 fs3 fc0 sc0 ls0 ws0">ference. The ﬁrst ﬁlter is the Laplacian of Gaussian (LoG),</div><div class="t m0 xc h7 y50 ff2 fs3 fc0 sc0 ls0 ws0">which is often u sed in local m ethods for removing noise and</div><div class="t m0 xc h7 y51 ff2 fs3 fc0 sc0 ls0 ws0">changes in bias [10, 13]. Here we use a LoG ﬁlter with a</div><div class="t m0 xc h7 y52 ff2 fs3 fc0 sc0 ls0 ws0">standard deviation of 1 p ixel, which is a pplied by convolu-</div><div class="t m0 xc h9 y53 ff2 fs3 fc0 sc0 ls0 ws0">tion with a 5× 5 kernel. The second ﬁlter is th e rank ﬁlter,</div><div class="t m0 xc h7 y54 ff2 fs3 fc0 sc0 ls0 ws0">which replaces the intensity of a pixel with its rank among</div><div class="t m0 xc h7 y55 ff2 fs3 fc0 sc0 ls0 ws0">all pixels within a certain neighborhood. It was originally</div><div class="t m0 xc h7 y56 ff2 fs3 fc0 sc0 ls0 ws0">proposed [23] for robustness to outliers within the neigh-</div><div class="t m0 xc h7 y57 ff2 fs3 fc0 sc0 ls0 ws0">borhood, which typically occur near depth discontinuities</div><div class="t m0 xc h7 y58 ff2 fs3 fc0 sc0 ls0 ws0">and leads to blurred object borders. Since the method only</div><div class="t m0 xc h7 y59 ff2 fs3 fc0 sc0 ls0 ws0">depends on the ordering of intensities and not their values,</div><div class="t m0 xc h7 y5a ff2 fs3 fc0 sc0 ls0 ws0">it compensates for all radiom e tric distortions that p reserve</div><div class="t m0 xc h7 y5b ff2 fs3 fc0 sc0 ls0 ws0">this ordering. Here we use a rank ﬁlter with a square win-</div><div class="t m0 xc h9 y5c ff2 fs3 fc0 sc0 ls0 ws0">dow of 15×15 pixels centere d at the pixel of interest. While</div><div class="t m0 xc h7 y5d ff2 fs3 fc0 sc0 ls0 ws0">there are other rank-based matching methods [1, 16], we</div><div class="t m0 xc h7 y5e ff2 fs3 fc0 sc0 ls0 ws0">chose the ran k tr ansform since it can be efﬁciently imple-</div><div class="t m0 xc h7 y5f ff2 fs3 fc0 sc0 ls0 ws0">mented as ﬁlter, without changing the ster e o method itself.</div><div class="t m0 xc h7 y60 ff2 fs3 fc0 sc0 ls0 ws0">The third ﬁlter is a mean ﬁ lter, which aims to com pensate</div><div class="t m0 xc h7 y61 ff2 fs3 fc0 sc0 ls0 ws0">a change in bias by subtr acting the mean intensity of a cer-</div><div class="t m0 xc h7 y62 ff2 fs3 fc0 sc0 ls0 ws0">tain neighborhood. We again use a squa re window of size</div><div class="t m0 xc h9 y63 ff2 fs3 fc0 sc0 ls0 ws0">15×15 th at is centered at the pixel of interest.</div><div class="t m0 xb h7 y64 ff2 fs3 fc0 sc0 ls0 ws0">Our next matching c ost is mutual information (MI), a</div><div class="t m0 xc h7 y65 ff2 fs3 fc0 sc0 ls0 ws0">powerful method for handling complex radiometric rela-</div><div class="t m0 xc h7 y66 ff2 fs3 fc0 sc0 ls0 ws0">tionships between two ima ges [20]. The MI of two images</div><div class="t m0 xc h7 y67 ff2 fs3 fc0 sc0 ls0 ws0">is calculated by sum ming the entro py of the histograms of</div><div class="t m0 xc h7 y68 ff2 fs3 fc0 sc0 ls0 ws0">the overlapping parts of each image and subtracting the en-</div><div class="t m0 xc h7 y69 ff2 fs3 fc0 sc0 ls0 ws0">tropy of the join t histogram of pixel-wise correspondences.</div><div class="t m0 xc h7 y6a ff2 fs3 fc0 sc0 ls0 ws0">The MI value directly expresses how well images are regis-</div><div class="t m0 xc h7 y6b ff2 fs3 fc0 sc0 ls0 ws0">tered. This follows from the observation that the joint his-</div><div class="t m0 xc h7 y6c ff2 fs3 fc0 sc0 ls0 ws0">togram of well-registered images has just a few high peaks</div><div class="t m0 xc h7 y6d ff2 fs3 fc0 sc0 ls0 ws0">in contrast to poorly registered images where th e joint his-</div><div class="t m0 xc h7 y6e ff2 fs3 fc0 sc0 ls0 ws0">togram is rather ﬂat. Thus, for well-registered images, the</div><div class="t m0 xc h7 y6f ff2 fs3 fc0 sc0 ls0 ws0">entropy of the joint histogram is low, while the entropy of</div><div class="t m0 xc h7 y70 ff2 fs3 fc0 sc0 ls0 ws0">the individual histog rams changes little. MI has been used</div><div class="t m0 xc h7 y71 ff2 fs3 fc0 sc0 ls0 ws0">for local [7] and global [11] stereo methods. In the lat-</div><div class="t m0 xc h7 y72 ff2 fs3 fc0 sc0 ls0 ws0">ter case, its calculation is changed by Taylor expansion for</div><div class="t m0 xc h7 y73 ff2 fs3 fc0 sc0 ls0 ws0">getting a pixel-wise ma tc hing cost. The costs are stored</div><div class="t m0 xc h7 y74 ff2 fs3 fc0 sc0 ls0 ws0">for each combination of intensities in a cost matrix. This</div><div class="t m0 xc h7 y75 ff2 fs3 fc0 sc0 ls0 ws0">lookup table is required for matching, but can only be cre-</div><div class="t m0 xc h7 y76 ff2 fs3 fc0 sc0 ls0 ws0">ated from known correspondences. The solution is an itera-</div><div class="t m0 xc h7 y77 ff2 fs3 fc0 sc0 ls0 ws0">tive design in which the disparity image of the previous loop</div><div class="t m0 xe h7 y78 ff2 fs3 fc0 sc0 ls0 ws0">serves for creating the cost matr ix for matching intensities</div><div class="t m0 xe h7 y79 ff2 fs3 fc0 sc0 ls0 ws0">in the next loop [11]. The p rocess is started with a random</div><div class="t m0 xe h7 y7a ff2 fs3 fc0 sc0 ls0 ws0">disparity image and r e quires typically only 3 to 4 iterations.</div><div class="t m0 xd h7 y7b ff2 fs3 fc0 sc0 ls0 ws0">In this paper we use the efﬁcient Hierarchical MI (HMI)</div><div class="t m0 xe h7 y7c ff2 fs3 fc0 sc0 ls0 ws0">method of [9], which works as follows. First, both input</div><div class="t m0 xe h7 y7d ff2 fs3 fc0 sc0 ls0 ws0">images are downscaled by factor 16 and MI is calculated</div><div class="t m0 xe h7 y7e ff2 fs3 fc0 sc0 ls0 ws0">by matching the stereo images using a random disparity im-</div><div class="t m0 xe h7 y7f ff2 fs3 fc0 sc0 ls0 ws0">age. The process is itera te d a few times before the disparity</div><div class="t m0 xe h7 y80 ff2 fs3 fc0 sc0 ls0 ws0">is upscaled for serving as initial guess for matching at</div><div class="t m0 xf ha y81 ff2 fs5 fc0 sc0 ls0 ws0">1</div><div class="t m0 xf ha y82 ff2 fs5 fc0 sc0 ls0 ws0">8</div><div class="t m0 x10 h7 y83 ff2 fs3 fc0 sc0 ls0 ws0">th of</div><div class="t m0 xe h7 y84 ff2 fs3 fc0 sc0 ls0 ws0">the full resolution. Upscaling and matching is repeated un-</div><div class="t m0 xe h7 y85 ff2 fs3 fc0 sc0 ls0 ws0">til the full resolutio n is reached. It should be noted that the</div><div class="t m0 xe h7 y86 ff2 fs3 fc0 sc0 ls0 ws0">disparity image of the lower-resolution level is used only</div><div class="t m0 xe h7 y87 ff2 fs3 fc0 sc0 ls0 ws0">for calculating the matching costs of the highe r-resolution</div><div class="t m0 xe h7 y88 ff2 fs3 fc0 sc0 ls0 ws0">level, but not for restricting the disparity range. The h ie r-</div><div class="t m0 xe h7 y89 ff2 fs3 fc0 sc0 ls0 ws0">archical calculation has a runtime overhead of just 14% if</div><div class="t m0 xe h7 y8a ff2 fs3 fc0 sc0 ls0 ws0">the runtime of the stereo meth od depends linearly on the</div><div class="t m0 xe h7 y8b ff2 fs3 fc0 sc0 ls0 ws0">number of p ixels and d isp arities [9].</div><div class="t m0 xd h7 y8c ff2 fs3 fc0 sc0 ls0 ws0">Finally, we also include normalized cross-correlatio n</div><div class="t m0 xe h7 y8d ff2 fs3 fc0 sc0 ls0 ws0">(NCC) in our evaluation. NCC is a standard method for</div><div class="t m0 xe h7 y8e ff2 fs3 fc0 sc0 ls0 ws0">matching two windows around a pixel of interest. The nor-</div><div class="t m0 xe h7 y8f ff2 fs3 fc0 sc0 ls0 ws0">malization within the window compensates differences in</div><div class="t m0 xe h7 y90 ff2 fs3 fc0 sc0 ls0 ws0">gain and bias. NCC is statistically the optimal method for</div><div class="t m0 xe h7 y91 ff2 fs3 fc0 sc0 ls0 ws0">compen sating G aussian noise. However, NCC tends to blur</div><div class="t m0 xe h7 y92 ff2 fs3 fc0 sc0 ls0 ws0">depth discon tinuities more than many other matching costs,</div><div class="t m0 xe h7 y93 ff2 fs3 fc0 sc0 ls0 ws0">because outliers lead to high errors within the NCC calcu-</div><div class="t m0 xe h7 y94 ff2 fs3 fc0 sc0 ls0 ws0">lation. MNCC has be en introduce d as a common variant</div><div class="t m0 xe h7 y95 ff2 fs3 fc0 sc0 ls0 ws0">by Moravec [15]. We selected th e standard NCC as MNCC</div><div class="t m0 xe h7 y96 ff2 fs3 fc0 sc0 ls0 ws0">gave slightly inferior results in our exper iments. In contrast</div><div class="t m0 xe h7 y97 ff2 fs3 fc0 sc0 ls0 ws0">to all other matching costs we consider here, NCC can only</div><div class="t m0 xe h7 y98 ff2 fs3 fc0 sc0 ls0 ws0">be used with local methods due to its window-based design.</div><div class="t m0 xd h7 y99 ff2 fs3 fc0 sc0 ls0 ws0">In all of the above costs, we only use the image inten sity</div><div class="t m0 xe h7 y9a ff2 fs3 fc0 sc0 ls0 ws0">(luminance) and not the color for matchin g. The reason is</div><div class="t m0 xe h7 y9b ff2 fs3 fc0 sc0 ls0 ws0">that several of the considered costs (e.g., rank and MI) are</div><div class="t m0 xe h7 y9c ff2 fs3 fc0 sc0 ls0 ws0">naturally deﬁned on intensity images, and for fairness we</div><div class="t m0 xe h7 y9d ff2 fs3 fc0 sc0 ls0 ws0">want to compare a ll costs on the same input data. However,</div><div class="t m0 xe h7 y9e ff2 fs3 fc0 sc0 ls0 ws0">we also found that that those costs that easily extend to colo r</div><div class="t m0 xe h7 y9f ff2 fs3 fc0 sc0 ls0 ws0">only perform marginally better on our data sets. Clearly,</div><div class="t m0 xe h7 ya0 ff2 fs3 fc0 sc0 ls0 ws0">future research is needed on robust color matching.</div><div class="t m0 xd h7 ya1 ff2 fs3 fc0 sc0 ls0 ws0">To summarize, we compare six costs: sampling-</div><div class="t m0 xe hb ya2 ff2 fs3 fc0 sc0 ls0 ws0">insensitive absolute differences (BT), thr ee ﬁlter-based</div><div class="t m0 xe hb ya3 ff2 fs3 fc0 sc0 ls0 ws0">costs (LoG, Rank, and Mean), hierarchical mutual in for-</div><div class="t m0 xe hb ya4 ff2 fs3 fc0 sc0 ls0 ws0">mation (HMI), an d normalized cross-corr elation (NCC).</div><div class="t m0 xe h8 ya5 ff1 fs4 fc0 sc0 ls0 ws0">2.2. Stereo Algorithms</div><div class="t m0 xd h7 ya6 ff2 fs3 fc0 sc0 ls0 ws0">The performance of a match ing cost can depend on the</div><div class="t m0 xe h7 ya7 ff2 fs3 fc0 sc0 ls0 ws0">algorithm that uses the cost. We thus conside r th ree dif-</div><div class="t m0 xe h7 ya8 ff2 fs3 fc0 sc0 ls0 ws0">ferent stereo algorithms: a local, correlation-based method</div><div class="t m0 xe hb ya9 ff2 fs3 fc0 sc0 ls0 ws0">(Corr), the semi-global method of [9] (SGM), and a global</div><div class="t m0 xe hb yaa ff2 fs3 fc0 sc0 ls0 ws0">method using graph cuts [4] (GC). We implemented each</div><div class="t m0 xe h7 yab ff2 fs3 fc0 sc0 ls0 ws0">of the six matching costs for each stereo method, except for</div><div class="t m0 xe h7 yac ff2 fs3 fc0 sc0 ls0 ws0">NCC which is only used with the local method.</div><div class="t m0 xd h7 yad ff2 fs3 fc0 sc0 ls0 ws0">Our local stereo method (Corr) is a simple window-based</div><div class="t m0 xe h7 yae ff2 fs3 fc0 sc0 ls0 ws0">approa c h [10, 13, 17]. After aggregating the matching cost</div></div><div class="pi" data-data='{"ctm":[1.611639,0.000000,0.000000,1.611639,0.000000,0.000000]}'></div></div>

<div id="pf3" class="pf w0 h0" data-page-no="3"><div class="pc pc3 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://csdnimg.cn/release/download_crawler_static/9325663/bg3.jpg"><div class="t m0 x11 hc yaf ff2 fs2 fc0 sc0 ls0 ws0">Figure 1. The left images of the Tsukuba, Venus, Teddy, and C ones stereo pairs.</div><div class="t m0 xc h9 yb0 ff2 fs3 fc0 sc0 ls0 ws0">with a square window of 9×9 pixels, the disparity with the</div><div class="t m0 xc h7 yb1 ff2 fs3 fc0 sc0 ls0 ws0">lowest aggregated cost is selected ( w inner-takes-all). This is</div><div class="t m0 xc h7 yb2 ff2 fs3 fc0 sc0 ls0 ws0">followed by subpixel interpolation, a left-rig ht consistency</div><div class="t m0 xc h7 yb3 ff2 fs3 fc0 sc0 ls0 ws0">check for invalidating occlusions and mismatches, and in-</div><div class="t m0 xc h7 yb4 ff2 fs3 fc0 sc0 ls0 ws0">validation of disparity segments smaller than 160 pixels [8].</div><div class="t m0 xc h7 yb5 ff2 fs3 fc0 sc0 ls0 ws0">Invalid disparity areas are ﬁlled by propagating neighboring</div><div class="t m0 xc h7 yb6 ff2 fs3 fc0 sc0 ls0 ws0">small (i.e., backgroun d) disparity values. The reason we</div><div class="t m0 xc h7 yb7 ff2 fs3 fc0 sc0 ls0 ws0">perform these post-processing steps, as op posed to compar-</div><div class="t m0 xc h7 yb8 ff2 fs3 fc0 sc0 ls0 ws0">ing the “raw” results, is to reduce the overall errors, which</div><div class="t m0 xc h7 yb9 ff2 fs3 fc0 sc0 ls0 ws0">in turn yields improved discrimination be twe en costs.</div><div class="t m0 xb h7 yba ff2 fs3 fc0 sc0 ls0 ws0">Our second stereo algo rithm is the semi-global match-</div><div class="t m0 xc h7 ybb ff2 fs3 fc0 sc0 ls0 ws0">ing (SGM) method [9]. We selected it as an approach in-</div><div class="t m0 xc h7 ybc ff2 fs3 fc0 sc0 ls0 ws0">between local and global matchin g. There are oth e r ap-</div><div class="t m0 xc h7 ybd ff2 fs3 fc0 sc0 ls0 ws0">proach e s in this category, e.g., dynamic programming (DP),</div><div class="t m0 xc h7 ybe ff2 fs3 fc0 sc0 ls0 ws0">but SGM outperforms DP a nd yield s no streaking artefacts.</div><div class="t m0 xc h9 ybf ff2 fs3 fc0 sc0 ls0 ws0">SGM aims to minimize a global 2D energy function E(D)</div><div class="t m0 xc h7 yc0 ff2 fs3 fc0 sc0 ls0 ws0">by solving a large number of 1D minimizatio n problems.</div><div class="t m0 xc h7 yc1 ff2 fs3 fc0 sc0 ls0 ws0">Following [9], the actua l energy used is</div><div class="t m0 x12 h9 yc2 ff4 fs3 fc0 sc0 ls0 ws0">E(D) =</div><div class="t m0 x13 hd yc3 ff7 fs6 fc0 sc0 ls0 ws0">∑</div><div class="t m0 x14 he yc4 ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x15 hf yc5 ff8 fs3 fc0 sc0 ls0 ws0"></div><div class="t m0 x16 h9 yc2 ff4 fs3 fc0 sc0 ls0 ws0">C(p, D</div><div class="t m0 x17 he yc6 ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x18 h9 yc2 ff6 fs3 fc0 sc0 ls0 ws0">)</div><div class="t m0 x19 h9 yc7 ff6 fs3 fc0 sc0 ls0 ws0">+</div><div class="t m0 x4 hd yc8 ff7 fs6 fc0 sc0 ls0 ws0">∑</div><div class="t m0 x1 h10 yc9 ff1 fs5 fc0 sc0 ls0 ws0">q∈N</div><div class="t m0 x1a h11 yca ff1 fs7 fc0 sc0 ls0 ws0">p</div><div class="t m0 x1b h6 ycb ff4 fs3 fc0 sc0 ls0 ws0">P</div><div class="t m0 x5 ha ycc ff2 fs5 fc0 sc0 ls0 ws0">1</div><div class="t m0 x1c h9 ycb ff2 fs3 fc0 sc0 ls0 ws0">T[|D</div><div class="t m0 x1d he ycc ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x1e h9 ycb ff5 fs3 fc0 sc0 ls0 ws0">− D</div><div class="t m0 x1f he ycc ff1 fs5 fc0 sc0 ls0 ws0">q</div><div class="t m0 x20 h9 ycb ff5 fs3 fc0 sc0 ls0 ws0">| = 1]</div><div class="t m0 x19 h9 ycd ff6 fs3 fc0 sc0 ls0 ws0">+</div><div class="t m0 x4 hd yce ff7 fs6 fc0 sc0 ls0 ws0">∑</div><div class="t m0 x1 h10 ycf ff1 fs5 fc0 sc0 ls0 ws0">q∈N</div><div class="t m0 x1a h11 yd0 ff1 fs7 fc0 sc0 ls0 ws0">p</div><div class="t m0 x1b h6 yd1 ff4 fs3 fc0 sc0 ls0 ws0">P</div><div class="t m0 x5 ha yd2 ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 x1c h9 yd1 ff2 fs3 fc0 sc0 ls0 ws0">T[|D</div><div class="t m0 x1d he yd2 ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x1e h9 yd1 ff5 fs3 fc0 sc0 ls0 ws0">− D</div><div class="t m0 x1f he yd2 ff1 fs5 fc0 sc0 ls0 ws0">q</div><div class="t m0 x20 h9 yd1 ff5 fs3 fc0 sc0 ls0 ws0">| > 1]</div><div class="t m0 x21 hf yd3 ff8 fs3 fc0 sc0 ls0 ws0"></div><div class="t m0 x22 h12 yd1 ff9 fs3 fc0 sc0 ls0 ws0">.</div><div class="t m0 x23 h7 yd4 ff2 fs3 fc0 sc0 ls0 ws0">(1)</div><div class="t m0 xc h7 yd5 ff2 fs3 fc0 sc0 ls0 ws0">The ﬁrst term of (1) calculates the sum of a pixel-wise</div><div class="t m0 xc h9 yd6 ff2 fs3 fc0 sc0 ls0 ws0">matching cost C(p, D</div><div class="t m0 x24 he yd7 ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x25 h9 yd6 ff6 fs3 fc0 sc0 ls0 ws0">) (as deﬁned in Section 2.1) for all</div><div class="t m0 xc hb yd8 ff2 fs3 fc0 sc0 ls0 ws0">pixels p at their disparities D</div><div class="t m0 x26 he yd9 ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x27 h9 yd8 ff2 fs3 fc0 sc0 ls0 ws0">. The function T[] is deﬁned</div><div class="t m0 xc h7 yda ff2 fs3 fc0 sc0 ls0 ws0">to return 1 if its argument is true and 0 otherwise. Thus, the</div><div class="t m0 xc h7 ydb ff2 fs3 fc0 sc0 ls0 ws0">second term o f the energy func tion penalizes small dispar-</div><div class="t m0 xc h7 ydc ff2 fs3 fc0 sc0 ls0 ws0">ity differences of neighboring pixels N</div><div class="t m0 x28 he ydd ff1 fs5 fc0 sc0 ls0 ws0">p</div><div class="t m0 x29 hb ydc ff2 fs3 fc0 sc0 ls0 ws0">of p with the cost</div><div class="t m0 xc h6 yde ff4 fs3 fc0 sc0 ls0 ws0">P</div><div class="t m0 x2a ha ydf ff2 fs5 fc0 sc0 ls0 ws0">1</div><div class="t m0 x2b h7 yde ff2 fs3 fc0 sc0 ls0 ws0">. Similarly, the third term penalizes larger disparity steps</div><div class="t m0 xc h7 ye0 ff2 fs3 fc0 sc0 ls0 ws0">(i.e., discontinuities) with a higher penalty P</div><div class="t m0 x2c ha ye1 ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 x2d h7 ye0 ff2 fs3 fc0 sc0 ls0 ws0">. The value of</div><div class="t m0 xc h6 ye2 ff4 fs3 fc0 sc0 ls0 ws0">P</div><div class="t m0 x2a ha ye3 ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 xb h7 ye2 ff2 fs3 fc0 sc0 ls0 ws0">is adapted to the local intensity gradient by P</div><div class="t m0 x2e ha ye3 ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 x2f h9 ye2 ff6 fs3 fc0 sc0 ls0 ws0">=</div><div class="t m0 x30 h13 ye4 ff4 fs5 fc0 sc0 ls0 ws0">P</div><div class="t m0 x31 h14 ye5 ff5 fs7 fc0 sc0 ls0 ws0">′</div><div class="t m0 x32 h15 ye6 ff2 fs7 fc0 sc0 ls0 ws0">2</div><div class="t m0 x33 h10 ye7 ff5 fs5 fc0 sc0 ls0 ws0">|I</div><div class="t m0 x34 h11 ye8 ff4 fs7 fc0 sc0 ls0 ws0">bp</div><div class="t m0 x35 h10 ye7 ff5 fs5 fc0 sc0 ls0 ws0">−I</div><div class="t m0 x36 h11 ye8 ff4 fs7 fc0 sc0 ls0 ws0">bq</div><div class="t m0 x37 h10 ye7 ff5 fs5 fc0 sc0 ls0 ws0">|</div><div class="t m0 xc hb ye9 ff2 fs3 fc0 sc0 ls0 ws0">for the neighboring pixels p and q. This results in sharper</div><div class="t m0 xc h7 yea ff2 fs3 fc0 sc0 ls0 ws0">depth discontinuities as they mostly coincide with intensity</div><div class="t m0 xc h7 yeb ff2 fs3 fc0 sc0 ls0 ws0">variations.</div><div class="t m0 xb h9 yec ff2 fs3 fc0 sc0 ls0 ws0">SGM c alculates E(D) along 1D paths from 8 directions</div><div class="t m0 xc h7 yed ff2 fs3 fc0 sc0 ls0 ws0">towards each pixel of interest using dynamic programming.</div><div class="t m0 xc h7 yee ff2 fs3 fc0 sc0 ls0 ws0">The costs of all paths are summed for e ach pixel and dispar-</div><div class="t m0 xc h7 yef ff2 fs3 fc0 sc0 ls0 ws0">ity. The disparity is then determined by winner-takes-all.</div><div class="t m0 xc h7 yf0 ff2 fs3 fc0 sc0 ls0 ws0">Subpixel interpolation is perform e d as w ell as a left-right</div><div class="t m0 xe h7 yf1 ff2 fs3 fc0 sc0 ls0 ws0">consistency check. Disparity segments below the size of</div><div class="t m0 xe h7 yf2 ff2 fs3 fc0 sc0 ls0 ws0">20 pixels are invalidated for getting rid of small patches of</div><div class="t m0 xe h7 yf3 ff2 fs3 fc0 sc0 ls0 ws0">outliers. Invalid disparities are again interp olated.</div><div class="t m0 xd h7 yf4 ff2 fs3 fc0 sc0 ls0 ws0">Finally, we use a graph-cuts (GC) stereo algorithm as a</div><div class="t m0 xe h7 yf5 ff2 fs3 fc0 sc0 ls0 ws0">representative of a global me thod [ 3, 4, 12]. Our imp le men-</div><div class="t m0 xe h7 yf6 ff2 fs3 fc0 sc0 ls0 ws0">tation is based on the MRF library provided by [19]. We</div><div class="t m0 xe h9 yf7 ff2 fs3 fc0 sc0 ls0 ws0">tried to use the same energy function E(D) as for SGM.</div><div class="t m0 xe h7 yf8 ff2 fs3 fc0 sc0 ls0 ws0">However, we found that for GC it give s better resu lts to</div><div class="t m0 xe h7 yf9 ff2 fs3 fc0 sc0 ls0 ws0">adapt the cost P</div><div class="t m0 x38 ha yfa ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 x39 h7 yf9 ff2 fs3 fc0 sc0 ls0 ws0">not linearly with the intensity gradient, but</div><div class="t m0 xe h7 yfb ff2 fs3 fc0 sc0 ls0 ws0">rather to double the va lue of P</div><div class="t m0 x3a ha yfc ff2 fs5 fc0 sc0 ls0 ws0">2</div><div class="t m0 x3b h7 yfb ff2 fs3 fc0 sc0 ls0 ws0">for gradients below a given</div><div class="t m0 xe h7 yfd ff2 fs3 fc0 sc0 ls0 ws0">threshold. Like SGM, GC only approximates the global</div><div class="t m0 xe h9 yfe ff2 fs3 fc0 sc0 ls0 ws0">minimum of E(D), but it utilizes the full 2D connectivity</div><div class="t m0 xe h7 yff ff2 fs3 fc0 sc0 ls0 ws0">for the smoothness term in contrast to SGM, which opti-</div><div class="t m0 xe h7 y100 ff2 fs3 fc0 sc0 ls0 ws0">mizes separately along 1D paths. Our G C implementation,</div><div class="t m0 xe h7 y101 ff2 fs3 fc0 sc0 ls0 ws0">unlike Corr and SGM, ne ither includes subpixel interpola-</div><div class="t m0 xe h7 y102 ff2 fs3 fc0 sc0 ls0 ws0">tion nor accounts for occlusions.</div><div class="t m0 xd h7 y103 ff2 fs3 fc0 sc0 ls0 ws0">We manually tuned the smoothness paramete rs of SGM</div><div class="t m0 xe h7 y104 ff2 fs3 fc0 sc0 ls0 ws0">and GC individually for each cost using ima ges without ra-</div><div class="t m0 xe h7 y105 ff2 fs3 fc0 sc0 ls0 ws0">diometric differences. Afte r th e tuning phase, all parame-</div><div class="t m0 xe h7 y106 ff2 fs3 fc0 sc0 ls0 ws0">ters were kept constant for all imag e s and experiments. This</div><div class="t m0 xe h7 y107 ff2 fs3 fc0 sc0 ls0 ws0">approa c h allows to concentrate on the performance of the</div><div class="t m0 xe h7 y108 ff2 fs3 fc0 sc0 ls0 ws0">matching cost rather than the stereo method.</div><div class="t m0 xe h5 y109 ff1 fs1 fc0 sc0 ls0 ws0">3. Evaluation</div><div class="t m0 xd h7 y10a ff2 fs3 fc0 sc0 ls0 ws0">We tested all combinations of all matching costs with the</div><div class="t m0 xe h7 y10b ff2 fs3 fc0 sc0 ls0 ws0">local, semi-global, and global stereo algorithms on images</div><div class="t m0 xe h7 y10c ff2 fs3 fc0 sc0 ls0 ws0">with simulated a nd real radiometric changes.</div><div class="t m0 xe h8 y10d ff1 fs4 fc0 sc0 ls0 ws0">3.1. Simulated Radiometric Changes</div><div class="t m0 xd h7 y10e ff2 fs3 fc0 sc0 ls0 ws0">For our ﬁrst set of experiments, we use the standard Mid-</div><div class="t m0 xe h7 y10f ff2 fs3 fc0 sc0 ls0 ws0">dlebury stereo da ta sets Tsukuba, Venus, Teddy, an d Co nes</div><div class="t m0 xe h7 y110 ff2 fs3 fc0 sc0 ls0 ws0">[17, 18]. Figure 1 shows the left images of each set. All</div><div class="t m0 xe h7 y111 ff2 fs3 fc0 sc0 ls0 ws0">images were carefully taken in a laboratory with the same</div><div class="t m0 xe h7 y112 ff2 fs3 fc0 sc0 ls0 ws0">camera settings and under the same lighting con ditions.</div><div class="t m0 xe h7 y113 ff2 fs3 fc0 sc0 ls0 ws0">Therefore radiometric changes are expected to be minimal.</div><div class="t m0 xe h7 y114 ff2 fs3 fc0 sc0 ls0 ws0">We used a disparity range of 16 pixels for Tsukuba, 32 pix-</div><div class="t m0 xe h7 y115 ff2 fs3 fc0 sc0 ls0 ws0">els for Venus and 64 pixels for Teddy, and Cones.</div><div class="t m0 xd h7 y116 ff2 fs3 fc0 sc0 ls0 ws0">The ﬁrst experiments consist of artiﬁcially changing</div><div class="t m0 xe h7 y117 ff2 fs3 fc0 sc0 ls0 ws0">the global brightness line arly (i.e., gain change) and non-</div><div class="t m0 xe h7 y118 ff2 fs3 fc0 sc0 ls0 ws0">linearly (e.g., gamma change). Only the right stereo images</div><div class="t m0 xe h7 y119 ff2 fs3 fc0 sc0 ls0 ws0">were changed , while leaving the left images untouched.</div></div><div class="pi" data-data='{"ctm":[1.611639,0.000000,0.000000,1.611639,0.000000,0.000000]}'></div></div>

stereo matching立体匹配 图片3

stereo matching立体匹配图片3