Prosthetic Knowledge Picks: Computational Photography


The latest in an ongoing series of themed collections of creative projects assembled by Prosthetic Knowledge. This edition brings together works dealing with computational photography, featuring new technologies which may alter the experience, relationship, and even definition of "the image."
The digital eye is an ubiquitous feature of current portable technology—webcams, DSLRs, mobile phones, tablets, even MP3 players. The Black Mirror-like ability to capture a moment and share it on social networks has shifted image recording from the creation of discrete analog mementos to an ongoing process of self-identification.

There are, however, new possibilities opening up around the next generation of mediated experiences. Of course, the artistic possibilities are tremendous, but the implications are far greater for many fields which may be struggling with their digital upkeep. From advertising to fashion, art to pornography, the photograph will not be "flat" anymore. The image can be seen from any angle, from the swipe of a touchscreen or drag from a mouse, or explored step-by-step with a headset and motion detector. "Photoshopping" will be 3D. It is not only industry-class endeavours that will change, as depth-sensing is now smaller and portable, and could give the (word-of-the-year contender) selfie an added dimension. Will the Facebooks or Flickrs support this new format? Or will another contender arise to facilitate a new process of creative self-identification?

Admittedly, the field of computational photography has grown tremendously over the past ten years (and many academic projects in this time are nowhere near mass-market availability); it would be impossible to cover the entire range of projects in a single submission. The focus on this piece, however, is to bring attention to examples that point to some of the artistic possibilities for these emerging technologies.

A downloadable demo by Infinite Realities put together in the Unity 3D engine features high resolution 3D scans of people in a virtual environment. Incredibly realistic, and can be viewed through an Occulus Rift headset.

Below are two videos which demonstrate the demo - the first showing the demo in normal view:


You can find out more about the demo (and download it) from Infinite Realities here. It is 300Mb, and there are some server issues which could prove to be slow.

The demo can be viewed normally on your desktop or with the headset, but either way, the high resolution detail demonstrated here is amazing.

Related: Infinite Realities demo creating Muybridge-style stop motion animations of the human body in 3D.

Structure Sensor  

A Kinect-like 3D sensor for mobile devices (which, incidently, successfully reached its Kickstarter target in a day):

From the product description:

Capture models of rooms, 3D scan objects, play augmented reality games, and develop mobile applications with 3D sensing.

The Structure Sensor gives mobile devices the ability to capture and understand the world in three dimensions.

With the Structure Sensor attached to your mobile device, you can walk around the world and instantly capture it in a digital form. This means you can capture 3D maps of indoor spaces and have every measurement in your pocket. You can instantly capture 3D models of objects and people for import into CAD and for 3D printing. You can play mind blowing augmented reality games where the real world is your game world. 

If you’re a developer, Structure gives you the ability to build mobile applications that interact with the three dimensional geometry of the real world for the very first time. You can even launch your app on the App Store!

Here is a brief demonstration of the utility with the latest iPhone 5: 

More can be found out about the product at its Kickstarter page here and at the project's website here.

STEM System - Virtual Camera Demo

Proof-of-concept demo of tech that can turn your handheld device into a camera for virtual spaces with accurate motion detection. It potentially inspires interesting possibilities with augmented real and recreated virtual spaces (such as a gallery space?)

From the project description:

We created this demo of virtual camera functionality using our STEM System prototype wireless motion tracker with an iPod Touch as the display. When we add Android and iOS support for the Sixense SDK, applications like the virtual camera will be able to run natively on mobile devices, with motion tracking provided by the STEM System.

More here.

3D Scanning and Printing Classic Art

PhD Student Tim Zaman takes a look at his and other projects reproducing classic fine art with contemporary 3D technology.

Here is a stop-motion video demonstrating Tim’s method of 3D capture using cameras which are commercially available:

He writes,
We used a hybrid system using stereo vision (2 camera’s) and fringe projection (using a projector). This system gives us unrivaled detail and capture speed, capturing 40 million points per capture, each point in 3-D space (XYZ) and in full color (RGB). Multiple captures allows us to capture the Jewish Bride for instance, a work that spans 160×120 cm; giving us more than a billion XYZ/RGB points. This is all done with proprietary camera’s that anyone can buy off-the shelf.
Here we can see a slightly yet purposely exaggerated 3D relief of a Van Gogh painting taken with this technology:

Tim also introduces a system put together by Canon called Océ which, using Tim’s data, can physically reproduce a scanned painting including physical relief. Essentially a 3D printer for impasto:

You can read more about Tim’s work and the technology at his blog here

Total Cinema 360 Oculus Rift Player

And finally, returning to the Oculus Rift. This demo allows you to watch panoramic 360-degree videos captured through an Oculus Rift headset:

From the product description: ""The Total Cinema 360 Oculus Player allows users to watch live action omni-directional video content specifically designed for the Oculus Rift."

You can download the demo here (Mac only for now. Windows version coming October 15, 2013.), and you can try an interactive 3D video in your browser here.

Other Picks

3-SweepImpressive demonstration of turning 2D objects in photographs into manipulable 3D objects, using a simple 3-point method at key areas.

Autodesk 123D. Around for over a year, but still a tool which is useful as an introduction to computational photography, turning several photos of an object into a 3D model.

Salted Perception. What happens when you combine an Oculus Rift headset with a Kinect? Something trippy...