The thrilling potential of SixthSense technology

I was always very curious about the idea of sixth sense and how it can be revealed or evaluated through the technological advancements we have in our hands today. And after studying and searching about this topic for quite some times I have at last found out about this MIT graduate Pranav Mistry from India who is actually working on this technology and succeeded in revealing its potential and application in the real world quite brilliantly. His work and way of thinking about this matter inspired me a lot to move into studying about this field of technology which I eventually did while I got admitted into “Intelligent Systems Design” program here at Chalmers. I think this subject will help me a lot to understand about developing Intelligent systems and their potentials in the real physical world which can possibly guide me to come up with some unique idea on my own like Pranav did.

As I will be blogging about this project that Pranav is currently working on now which is entitled “SixthSense”- integrating information with the real world, let’s watch and learn about this from the point of view of the inventor himself.

While presenting the idea on TED: IDEAS WORTH SPREADING”

video source “

let’s see what the inventor has to say about his project :

‘SixthSense’ is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

This slideshow requires JavaScript.

Slideshow of the product’s applications

picture source “

We’ve evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

Pattie Maes’s pesentation

video source “

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user’s hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

video source “

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognizes user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch.

The most impressive implication of this technology is that it can help us being more human and not carrying or sitting in front of another machine, thus becoming a machine ourselves as said by Pranav himself. and also this project’s source files and all will be uploaded as open source very soon. You just have to buy the hardware and can program your own gestures and make it behave as the way you want. Isn’t it awesome?

I am eagerly waiting for that day when this product will be available to consumer level for purchase.

– Md. Basharat Hasan Mollah (Shishir) / Group Y

via “SixthSense- integrating information with the real world”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s