I applied a patent to Japan patent Office a few weeks ago. It is about sensing of finger gestures on some object surface (palm, desk, wall, bag, and any). I may apply PCT patent too.
The background motivation:
I have a suspect about gesture in the air. It is socially strange if you do such gestures in air in front of the other persons. It is because human kind has not used such gestures in its long history (except pointing by index finger). It lacks the reality of human natural behaviors. Moreover it does not keep privacy. And, from the design point of view, a gesture decoding has the complexity to search in the 3D space.
On the other hand, human kind has a long history of use of finger gesture on surface of some objects – papers, and naturally uses it. Moreover it can hide gesture from someone else. And a gesture decoding in the 2D space is simpler.
I recently bought Tobii 4C, and has used with desktop PC.
Here are some observations.
- After simple calibration, the accuracy of eye tracking is quite high and robust. I had used ITU Gaze Tracker (Eye Tribe). The calibration was pains, and a head move easily destroy tracking. Tobii, leader of gaze tracker, is doing a good job.
- Tobii is doing a good job to use gaze tracking with current mainstream UI. Tobii core SDK bundles an utility for gaze tracking to use with Windows GUI. For instance, the Mouse warp feature helps the mouse pointer move fast beyond Hick’s law. It’s nice to use.
Moreover, they owned a lot of patents in the area of hardware design and applications with GUI/Touch UI.
So then I wonder why Tobii has not played a main role in main-stream of the human machine interfaces yet.
Some reasons may be:
- Tobii style gaze tracking has its limitations. With non-portable type, use must sit near and in front of PC monitor. With portable type, user must wear bothering glasses. Those would be overcome by some other players.
- Business strategy
- Tobii device was too expensive until Eye Tribe started to offer cheap device and Tobii was forced to follow it. Being expensive blocked a rapid creation of market of gaze tracking by Tobii.
- Tobii could provide technology of platform layers, or could partner with platform (OS) players. But they are currently focusing on gaming and user research consulting…
- Tobii patent portfolio blocks other players to realize the platform evolution, while Tobii does not have ability to play a platform provider…
I examined recently how NUI is described.
One description describes that AR is among NUI. I don’t think so.
Real world seen is already rich and dangerous enough. The AR, for instance with Head Mount Display, adds extra information for seeing, which leads further stress on human sight/brain. AR is practical for those who are used to exposed and manage spiritual tensions such as soldiers. However AR will be an extra stress to most people, until human being get used to it as part of life, and it make time time more than a few generations.
Fingers move finer than wrists. Wrists move finer than hands. The subtlety of human muscle control is Fingers>Wrists>Hands.
All the human great tools in the history, scissors, chopstickes, pens, etc rely on and take advantages of synthesis of finger/wristshands.
Traditional computer gadget, Mouse, failed to take advantage of human muscle natural powers, because
- Mouse uses wrists/hands for pointing and finger click for commanding but pointing requires more subtlety than commanding.
- Hands for point and finger for commanding are separately used and they are not the synthesis of each powers.
Microsoft Surface dial looks to try taking advantage of finger for pointing too. If it is true, it would be one of great progresses from the mouse.
I cleaned up all icons on my PC desktop window..
PC desktop metaphor has played an important role for productivity of people since two decades.
My desktop use has been a mixture of short cuts of applications and working documents.
After I upgraded my Windows to 10, its start-up menu has a duplicated role for application short cuts. I moved those short cuts on Desktop to start menu. Windows 8 tile start window could play the same role.
On the other hand, I use a Dropbox folder as my working document repository, because I access them from multiple devices. Dropbox synchronization is much faster and stable than One Drive or Google drive. I don’t need short cuts for them.
So I don’t need desktop window any more even of desktop PC which is not tablet.
Gaze tracking is beyond mouse/taop poiting in two senses.
1. Seeing is followed by muscle motors, i.e., finger pointing operations in the current human computer interaction. Seeing is a direct expression of pointing. Seeing is a far responsive pointer than the current finger based methods.
2. Seeing is 3D. Pointing in the current human computer interaction is mere 2D. Current computer has not evolved into 3D user experiences, it is easy to imagine future human machine/robot interaction is 3D. Imagine an autofocus camera captures what human is seeing in the world.