Skip to main content
SpringerLink
Log in

Development and Evaluation of a Miniature Trinocular Camera System for Surgical Measurement Applications

  • Original Article
  • Published:
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science Aims and scope Submit manuscript

Abstract

Stereo endoscopes for minimally invasive surgery are well established in different medical applications. They may reduce the intervention time due to a better visual support of the surgeon by stereoscopic viewing. Due to rather difficult image analysis conditions in surgical environments, stereoscopic cameras are usually not used for 3D measurements during conventional operations. Especially for computer-assisted surgery, a highly reliable, accurate and dense surface real-time measurement could be of high importance. For that reason, a miniaturised trinocular camera system has been developed and is presented in this paper. To process the trinocular image sequences, a rectification approach for image triplets and an efficient trinocular semi-global matching approach have been implemented. The system is evaluated using two exemplary image triplets with very different matching conditions. The results from the developed trinocular matching are compared with reference models and against the results from OpenCV semi-global block matching using the left and right stereo images only. The proposed trinocular approach consistently provides more complete and error-free point clouds. The greatest improvement due to introducing the third image is achieved without any optimisations such as semi-global matching. The proposed trinocular approach achieves at least the same result quality as the binocular approach, but allows smaller matching windows. Due to fewer arithmetic operations in three small windows than in two large ones, the trinocular analysis outperforms the binocular analysis in terms of speed.

Zusammenfassung

Entwicklung und Evaluation eines miniaturisierten Dreikamerasystems für chirurgische Anwendungen. Stereoendoskope sind in einigen Bereichen der minimalinvasiven Chirurgie gut etabliert. Sie können die Interventionszeit durch eine bessere Wahrnehmung aufgrund des stereoskopischen Sehens verkürzen. Bedingt durch recht komplexe Aufnahmebedingungen im chirurgischen Umfeld werden die Stereobilder gewöhnlich nicht für 3D Messungen während konventioneller Operationen eingesetzt. Besonders für die computerassistierte Chirurgie sind eine hoch zuverlässige, genaue sowie dichte Oberflächenmessung in Echtzeit besonders wertvoll. Deshalb wurde ein miniaturisiertes Dreikamerasystem entwickelt, das in dieser Veröffentlichung vorgestellt wird. Zur Verarbeitung der Bildtripel wurden effiziente Verfahren zur Erstellung von Normalbildern und ein trinokulares Semi-Global Matching entwickelt. Mithilfe von zwei exemplarischen Bildtripeln mit sehr unterschiedlichen Bildanalysebedingungen wird das System näher untersucht. Die Ergebnisse des entwickelten trinokularen Matching werden mit Referenzmodellen und den Ergebnissen des OpenCV Semi-Global Block Matching unter Verwendung des linken und rechten Stereobildes verglichen. Der vorgeschlagene trinokulare Ansatz liefert durchweg vollständigere und störungsfreiere Punktwolken. Die größte Verbesserung durch die dritte Kamera wurde ohne Einsatz von Optimierungen, wie z.B. Semi-Global Matching, erzielt. Der vorgeschlagene trinokulare Ansatz erzielt mindestens dieselbe Ergebnisqualität wie der binokulare Ansatz und erlaubt dabei kleinere Matchingfenster. Durch eine verringerte Anzahl arithmetischer Operationen bei drei kleinen Matchingfenstern ergibt sich außerdem ein Geschwindigkeitsvorteil.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Atzpadin N, Kauff P, Schreer O (2004) Stereo analysis by hybrid recursive matching for real-time immersive video conferencing. IEEE Trans Circ Syst Video Technol 14(3):321–334

    Article  Google Scholar 

  • Ayache N, Hansen C (1988) Rectification of images for binocular and trinocular stereovision. Int Conf Pattern Recognit 9:11–16

    Google Scholar 

  • Banz C, Hesselbarth S, Flatt H, Blume H, Pirsch P (2010) Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: IEEE 2010 International Conference on Embedded Computer Systems, pp 93–101. doi:10.1109/ICSAMOS.2010.5642077

  • Birchfield S, Tomasi C (1998) A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans Pattern Anal Mach Intell 20(4):401–406

    Article  Google Scholar 

  • Bulatov D (2015) Temporal selection of images for a fast algorithm for depth-map extraction in multi-baseline configurations. In: Proceedings of the 10th international conference on computer vision theory and applications, vol 3, Berlin, Germany, 11–14 March, 2015, pp 395–402. doi:10.5220/0005239503950402

  • Conen N, Jepping C, Luhmann T, Maas H-G (2016) Rectification and robust matching using oriented image triplets for minimally invasive surgery. ISPRS international annals of the photogrammetry, remote sensing and spatial. Inf Sci 3(3):27–34

    Google Scholar 

  • Ernst I, Hirschmüller H (2008) Mutual information based semi-global stereo matching on the GPU. international symposium on visual. Computing 4:228–239

    Google Scholar 

  • Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2 edn. Cambridge University Press, Cambridge

  • Heinrichs M, Rodehorst V (2006) Trinocular rectification for various camera setups. In: ISPRS symposium photogrammetric computer vision. September 20–22, Bonn, vol 36, pp 43–48

  • Heinrichs M, Rodehorst V, Hellwich, O (2007) Efficient semi-global matching for trinocular stereo. Int Arch Photogramm Remote Sens Spatial Inf Sci 36(3/W49A):185–190

  • Hirschmüller H (2005) Accurate and efficient stereo processing by semi-global matching and mutual information. IEEE Conf Comput Vis Pattern Recognit 2:807–814

    Google Scholar 

  • Hirschmüller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341

    Article  Google Scholar 

  • Konolige K (1998) Small vision systems: hardware and implementation. Robotics research. Springer, London, pp 203–212

  • Liebold F, Maas H-G (2016) Advanced spatio-temporal filtering techniques for photogrammetric image sequence analysis in civil engineering material testing. ISPRS J Photogramm Remote Sens 111:13–21

    Article  Google Scholar 

  • Luhmann T, Robson S, Kyle S, Boehm J (2014) Close-range photogrammetry and 3D imaging, 2nd edn. Walter de Gruyter, Berlin

  • Maas HG (1992a) Complexity analysis for the establishment of image correspondences in dense spatial target fields. Int Arch Photogramm Remote Sens 29(B5):102–107

  • Maas HG (1992b) Robust automatic surface reconstruction with structured light. Int Arch Photogramm Remote Sens 29(B5):709–713

  • Maas H-G, Grün A (1995) Digital photogrammetric techniques for high-resolution 3-D flow velocity measurements. Opt Eng 34(7):1970–1976

    Article  Google Scholar 

  • Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computerassisted laparoscopic surgery. Med Image Anal 17(8):974–996

    Article  Google Scholar 

  • McInerney T, Terzopoulos D (1996) Deformable models in medical image analysis: a survey. Med Image Anal 1(2):91–108

    Article  Google Scholar 

  • Pollefeys M, Nistér D, Frahm JM et al (2008) Detailed real-time urban 3D reconstruction from video. Int J Comput Vis 78(2):143–167

    Article  Google Scholar 

  • Röhl S, Bodenstedt S, Suwelack S, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2011) Real-time surface reconstruction from stereo endoscopic images for intraoperative registration. Medical imaging, visualization, image-guided procedures, and modeling. SPIE 7964:796414

  • Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1):7–42

  • Smirnov S, Gotchev AP, Hannuksela M (2010) Comparative analysis of local binocular and trinocular depth estimation approaches. Real-time image and video processing. SPIE 7724:77240H

  • Spangenberg R, Langner T, Adfeldt S, Rojas R (2014) Large scale semi-global matching on the CPU. In: Proceedings of the 2014 IEEE intelligent vehicles symposium proceedings, Dearborn, MI, pp 195–201. doi:10.1109/IVS.2014.6856419

  • Steiner C, Wiggenhagen M, Heipke C (2016) Highspeed stereo-endoskopie für eng begrenzte Messvolumina. PFG Photogrammetrie, Fernerkundung, Geoinformation 2016(4):181–190

  • Terzopoulos D (1986) On matching deformable models to images. Technical report 60, Schlumberger Palo Alto Research. Topical meeting on machine vision. Technical digest series, vol 12. Optical Society of America, Washington, pp 160–167

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, pp 511–518. doi:10.1109/CVPR.2001.990517

  • Woo DM, Schultz H, Riseman E, Hanson A (2004) Performance of correlation-based stereo algorithm with respect to the change of the window size. In: Aizawa K, Nakamura Y, Satoh S (eds) Advances in multimedia information processing. PCM 2004. Lect Notes Comput Sci 3332:778–785

Download references

Author information

Authors and Affiliations

  1. Jade Hochschule Oldenburg, Institute for Applied Photogrammetry and Geoinformatics, 26121, Oldenburg, Germany

    Niklas Conen & Thomas Luhmann

  2. Technische Universität Dresden, Institute of Photogrammetry and Remote Sensing, 01062, Dresden, Germany

    Hans-Gerd Maas

Authors
  1. Niklas Conen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Luhmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hans-Gerd Maas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niklas Conen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Conen, N., Luhmann, T. & Maas, HG. Development and Evaluation of a Miniature Trinocular Camera System for Surgical Measurement Applications. PFG 85, 127–138 (2017). https://doi.org/10.1007/s41064-017-0014-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41064-017-0014-3

Keywords

Search

Navigation