Tag Archives: Multi-modal

Car driving as a human machine interaction model

While you are driving a car, you do the followings in parallel.

  • watch forward of the read, get feedback of your action
  • hear outside sounds
  • manipulate handle by hand and brake/accelerator bu foot
  • talk with passenger seat person

You sight organ, audio organ, and body motor systems are working in parallel, even two parts of body motor systems are working in parallel, and you can achieve your goal to get some place.

How wonderful the power of the human being is!

On the contrarily,  current human computer interaction is very limited and naive. It doesn’t take advantage of the human power as above:  You give information only by hand, and get information only by sight of 2D, serially. Voice interaction is going to add another channel b/w human and machine, but the channel is isolated from the others.



HMD or wearable glasses are another miniature of Desktop

Using Smart-Phone is eye-busy and hand-busy. You can’t use it while doing other things. Using on the road is dangerous.

It is because Smart-Phone is a mere miniature of  desktop computer interactions. Human use hand/finger to input and get information by eye. It doesn’t change human machine interactions fundamentally.

Apple’s Watch is another desktop too. Using it is eye-busy and hand-busy.

Head Mount display or wearable glasses such as Google glass, Microsoft Hololens looks to me yet another miniature of desktop interaction. Using them is eye-busy, though hands may be free. Eye is already busy to catch information from the rich world and to guide reaction against the world. Why do we over-load eye more? Virtual Reality belongs to this eye-busy staff.

I think it is worth pursuing an approach which take information from eye (gaze tracker) without any interference in sight, take advantage of eye to react against world without any interference in sight again.

Fast messaging

Language is, to begin with, for communication. We find a number of inefficiency in communicating our messages to the others. For example, we can consume text messages in thinking speed through hearing and reading, but we cannot produce messages in the similar speed through writing or typing. Human beings can consume text very fast but produce it slow Imbalance in producing & consuming bandwidths since 6,000 years ago even in IT ages Human could talk each other (aka telepathy) and talk to atoms (aka remote control) in the speed of thought with future IT. Human computer interaction or human world interaction could change so that it takes advantage of more powers of human receptors and effectors. Steve Jobs used a metaphor of PC as “bicycle” which gives power to human. We could realize “automotive of words”. Take advantage of sight power more, Use effectors more effectively My Home Page