3D scene and object reconstruction from digital images
Keywords:
Computer Vision, 3D Reconstruction, Epipolar Geometry, Structure from Motion, Multi-View StereoAbstract
New technologies, such as 3D printers, autonomous cars and robots for instance, originating from advances in Computer Vision and other fields, have been causing an increasingly high interest in robust 3D reconstruction pipelines, and particularly, scene reconstruction. Through use of these methods, it is possible to create an application that takes digital photographs of an object or environment as inputs and is capable of obtaining a 3D model that represents it. This model could then be used in a wide range of applications, such as game asset generation, video manipulation with special effects or parts replication with the use of a 3D printer for instance. In this paper, we address, present, discuss and implement methods that concern the different stages of a traditional 3D reconstruction pipeline using only digital images.
Downloads
References
ALCANTARILLA, P.; NUEVO, J.; BARTOLI, A. Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. Procedings Of The British Machine Vision Conference 2013, Bristol, Reino Unido, p. 13.1-13.11, 2013. British Machine Vision Association. https://dx.doi.org/10.5244/c.27.13.
BARNES, C.; SHECHTMAN, E.; FINKELSTEIN, A.; GOLDMAN, D. B. PatchMatch: a randomized correspondence algorithm for structural image editing. Acm Transactions On Graphics, [S.L.], v. 28, n. 3, p. 1-11, 27 jul. 2009. Association for Computing Machinery (ACM). https://dx.doi.org/10.1145/1531326.1531330.
BAY, H.; ESS, A.; TUYTELAARS, T.; VAN GOOL, L. Speeded-Up Robust Features (SURF). Computer Vision And Image Understanding, [S.L.], v. 110, n. 3, p. 346-359, jun. 2008. Elsevier BV. https://dx.doi.org/10.1016/j.cviu.2007.09.014.
BOOST. The Boost Graph Library (BGL). Disponível em: https://www.boost.org/doc/libs/1_79_0/libs/graph/doc/index.html. Acesso em 10 jun. 2022.
ESCRIVÁ, D. M.; JOSHI, P.; MENDONÇA, Vinícius G.; SHILKROT, R. Building Computer Vision Projects with OpenCV 4 and C++: implement complex computer vision algorithms and explore deep learning and face detection. [S. L.]: Packt Publishing, 2019.
FISCHLER, M. A.; BOLLES, R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications Of The Acm, [S.L.], v. 24, n. 6, p. 381-395, jun. 1981. Association for Computing Machinery (ACM). https://dx.doi.org/10.1145/358669.358692.
GREWENIG, S.; WEICKERT, J.; BRUHN, A. From Box Filtering to Fast Explicit Diffusion. Lecture Notes In Computer Science, [S.L.], p. 533-542, 2010. Springer Berlin Heidelberg. https://dx.doi.org/10.1007/978-3-642-15986-2_54.
HARTLEY, R.; ZISSERMAN, A. Multiple View Geometry in Computer Vision. 2. ed. Cambridge, Reino Unido: Cambridge University Press, 2004.
JANCOSEK, M.; PAJDLA, T. Exploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces. International Scholarly Research Notices, [S.L.], v. 2014, p. 1-20, 11 ago. 2014. Hindawi Limited. https://dx.doi.org/10.1155/2014/798595.
KAZHDAN, M.; BOLITHO, M.; HOPPE, H. Poisson surface reconstruction. Sgp '06: Proceedings of the fourth Eurographics symposium on Geometry processing, [S. L.], p. 61-70, jun. 2006. Association for Computing Machinery (ACM). https://dl.acm.org/doi/10.5555/1281957.1281965.
LEWIS, J. P. Fast Template Matching. Proceedings Of Vision Interface 95, Quebec, Canada, p. 120-123, 15 maio 1995.
LOWE, D. G. Distinctive Image Features from Scale-Invariant Keypoints. International Journal Of Computer Vision, [S.L.], v. 60, n. 2, p. 91-110, nov. 2004. Springer Science and Business Media LLC. https://dx.doi.org/10.1023/b:visi.0000029664.99615.94
MARTON, Z. C.; RUSU, R. B.; BEETZ, M. On fast surface reconstruction methods for large and noisy point clouds. 2009 Ieee International Conference On Robotics And Automation, [S. L.], p. 3218-3223, 12 maio 2009. Institute of Electrical and Electronics Engineers (IEEE). https://dx.doi.org/10.1109/ROBOT.2009.5152628
MCGLONE, J. C. Manual of Photogrammetry. 6. ed. [S. L.]: American Society For Photogrammetry And Remote Sensing (Asprs), 2013. 1372 p.
OPENCV. OpenCV. Disponível em: https://opencv.org. Acesso em 10 jun. 2022.
OPENMVS. OpenMVS. Disponível em: https://github.com/cdcseacave/openMVS. Acesso em 10 jun. 2022.
PCL. Point Cloud Library. Disponível em: https://pointclouds.org. Acesso em 10 jun. 2022.
SIMEK, K. Dissecting the Camera Matrix. Sightations. [S. L.], p. 0-0. 14 ago. 2012. Disponível em: https://ksimek.github.io/2012/08/14/decompose/. Acesso em: 10 jun. 2022
TUYTELAARS, T; MIKOLAJCZYK, K. Local Invariant Feature Detectors: a survey. Foundations And Trends® In Computer Graphics And Vision, [S.L.], v. 3, n. 3, p. 177-280, 15 jun. 2008. Now Publishers. https://dx.doi.org/10.1561/0600000017.
WAECHTER, M.; MOEHRLE, N.; GOESELE, M. Let There Be Color! Large-Scale Texturing of 3D Reconstructions. Computer Vision – Eccv 2014, [S.L.], p. 836-850, 2014. Springer International Publishing. https://dx.doi.org/10.1007/978-3-319-10602-1_54.
WIKIMEDIA FOUNDATION. Wikimedia Commons. 2022. Disponível em: https://commons.wikimedia.org/wiki/Main_Page. Acesso em: 10 jun. 2022.
YANG, X.; CHENG, K-T. LDB: an ultra-fast feature for scalable augmented reality on mobile devices. 2012 Ieee International Symposium On Mixed And Augmented Reality (Ismar), [S.L.], p. 49-57, nov. 2012. IEEE. https://dx.doi.org/10.1109/ismar.2012.6402537