Turning mobile phones into 3D scanners

The goal of this project is to develop a system which allows users to build interactively 3D models of real-world environments by means of mobile devices like smartphones and tablet computers.

The Computer Vision and Geometry Lab of the Swiss Federal Institute of Technology (ETH Zurich, Switzerland) announced on December 3, 2013 recent research results turning a normal smartphone into a mobile 3D scanner. 3D scanning aims to capture the geometry of the 3D world, however most existing solutions require a complicated setup, are often hard to use and might not always work outdoors. The technology was demonstrated publicly for the first time today at the International Conference on Computer Vision in Sydney, Australia.

Instead of taking a normal photograph, a user simply moves the phone around the object of interest and after a few motions, a 3D model appears on the screen. As the user keeps moving, additional images are recorded automatically and used to extend the 3D model. As all calculations are performed directly on the phone, the user gets immediate feedback and can select additional viewpoints to cover missing parts of the 3D model. This is an important advantage compared to solutions that batch process all the images in the cloud at a later time.

The app also makes it possible to visualize the state of the 3D scan from different viewing angles, allowing the user to cover all the areas of the object he is interested in. Having a solution on a mobile phone enables the acquisition of 3D scans on-the-fly anywhere. Our approach works in a wide variety of settings, including low-light conditions such as inside a museum. A user can capture a 3D model of a museum piece and interactively study it at home later. After reviewing the model, a user might decide to upload his 3D data to a cloud service to further refine the results.

By using the inertial sensors of the phone, the scanning process can be made simple, intuitive and robust. After the 3D capture is started, the system automatically determines the correct moments to extract camera images based on the user motion.

Contrary to image-only 3D capture solutions the app is able to determine the absolute size of the scanned 3D object, as well as the vertical direction. Because of the complexity of the calculations needed to reconstruct hundreds of thousands of points, the graphics co-processor (GPU) of the phone is used to enable a faster reconstruction and increase interactivity for the user.

The technology also allows the 3D capture of faces, giving a third dimension to portraits, profile pictures or images of loved ones.

Having a convenient way of getting 3D models of everyday objects, users will now be able to copy real-world objects by scanning a full 360 degree model of an object. The resulting 3D model can be used for visualization or augmented reality applications, or even be used for 3D printing, potentially at a remote location, effectively enabling the user to replicate an object.

The patent pending technology was developed exclusively by ETH Zurich and can run on a wide range of current smartphones.


About

The Computer Vision and Geometry Lab of ETH Zurich, the Swiss Federal Institute of Technology, is one of the leading research groups world-wide in extracting 3D geometric information from images. The group is currently working on projects including 3D city reconstruction, autonomous micro-aerial vehicles, video-based driver assistance, image-based geo-localization and dynamic scene reconstruction.



•Youtube Video: [www.youtube.com]