I understand why people want to write about the death of the mouse and keyboard - it's been around unchanged for a while and tech is now a hype-based "what's next" culture - but it's always just a bunch of hand-waving and name-dropping experimental technologies.<p>Mouse and keyboard is dead, "cannibalized" by touch, which is now also dying? This is ridiculous. The entire world of "getting things done" on a computer - business productivity, programming, content creation, science - is still firmly entrenched in KB-mouse. Nothing has even come within a longshot of challenging that yet.<p>I'm sure I'm somewhat biased by interacting mostly with tech-savvy people, but even people like my mom - who loves her iPad and is right in that target demographic - still use a mouse and keyboard every day. I just don't understand how anyone can claim that we are even close to replacing those tools.
I can't help but roll my eyes at the idea that "mouse and keyboard" are dead.<p>Yes, lots of touch devices are now being sold, but a lot of them are being sold into a channel in which nobody used a physical keyboard or mouse before (phones) and the rest are being sold into a device category that didn't quite exist before (tablets).<p>A lot of people seem to be misreading the decline of PC/laptop sales and the rise of tablets and smartphones as being totally interrelated when I don't believe they are. Certainly some percentage of tablets are purchases that otherwise would be have been netbooks, but IMO the bigger issue with PC/laptop sales dropping is people (granted, I'm only discussing First World people) generally already have one AND just don't need new ones as often anymore. Anything with a Core 2 Duo and 4+ gigabytes of RAM "ought to be enough for anyone" (and in my experience, for non-power-users it certainly is).<p>I'm a hardcore developer/gamer/"power-user" and even my system buying and major upgrade lifecycle has extended to about 3-4 years when it used to be 6 months to a year, combine that lifecycle extension with the fact that most people can get by with just a laptop (because the practical power difference between even a low-end laptop and a desktop are insignificant) and it isn't any wonder that PC/laptop sales have suffered for everyone but Apple who is one of the few smart enough to be selling systems with actual new-system differentiation ("Retina" screens).<p>tl; dr - I know lots of people (including myself) who have bought new smartphones and tablets over the past few years. I don't know a single one of them who doesn't use a "real" computer with mouse/keyboard daily and on average much more than their tablet/smartphone (if you exclude phone talking from the smartphone use). But their smartphones are probably like <1 year old (because significant practical hardware progress is still being made in this space) while their laptops might be years old and plenty fine for what they use them for.
Most of the "post touch" technologies discussed are, in my opinion, blind alleys. Touch has supplanted the mouse -- to some extent -- because it's less abstract and more convenient than the mouse. These are big wins. (Consider the finer-grained hierarchy: touch - stylus on display - stylus on tablet - mouse). Stylus on tablet is only barely less abstract and generally more inconvenient and expensive than mouse so mouse never really got supplanted by stylus on tablet. Stylus on display has disadvantages as well as advantages over mouse, and it's even more expensive, so again it didn't reach a tipping point.<p>Most of these "post touch" options are even more inconvenient, more abstract, and have greater shortcomings than touch or mouse, so I don't see them replacing touch. Now if an interface technology can win across the board: more convenient, more direct/less abstract, more definite/reliable -- then it will win. But these technologies lose across the board, at least in their current state.
I'm sorry, what was wrong with mouse and keyboard again?<p>If in any way possible, I much prefer mouse and keyboard over touch, touchscreen, or air (eyetoy, wii, kinect, etc.). It wouldn't be the first time I use my computer keyboard on my phone to be able to type decently. (USB keyboard, female usb -> male microusb converter.)<p>> <i>you could easily type something different from what you actually want</i><p>Lol, talk about touchscreen, or accidentally dragging something with a mouse. GUIs aren't much better than typing commands.<p>> <i>it'll be "move it where I'm thinking."</i><p>I agree with you on that. The problem is that I don't have the hardware to experiment with this (and I don't expect that I could develop anything better than other researchers are doing), so it's not possible yet. In the meantime, I prefer keyboard over everything, including mouse. The problem is that websites are optimized for mice, so many aren't very usable without. Touchscreen with a stylus is good for drawing, but that's the only application I can think of, and you might as well just use pen and paper for that.
It seems crazy that with all these innovations in input devices, so many of us control computers via a 80x24 window emulating a 1970's terminal - 80 columns to match IBM punch cards from 1928(!). And I type into these windows the exact same Unix commands I was typing in 1985. It just seems to me there are huge opportunities for progress that are being missed - surely 80-column text windows aren't a global maximum.<p>30 years from now will we be using neural interfaces to enter Unix commands into 80x24 text windows projected onto our retinas?
I've observed that change has its own sort of Newtonian laws that regulate it, and the one where people do the same thing they always did before unless acted upon by a very force is perhaps one of the stronger ones.<p>So to my mind you have to have some notion of what is 'forcing' the change before you can really say that things will change.<p>To use a current example, 'touch'. The force here is that two fold, one the amount of keyboard interaction you need to consume content is much less than the amount you need to create it, and two, keyboards take up space that could be filled with other features. Touch became credible when you could use it exclusively to operate the device in an acceptable way. Its why it failed to displace keyboards on the original Tablet PCs (you needed the keyboard too often) and its why the iPad without a keyboard is a lot less productive to process email on [1].<p>So 'post touch' needs, by my reasoning, some force behind it if it is going to displace touch. And we can look at those forces and see where they are coming from.<p>Clearly people talking to their devices is cool, but annoying to others on the train and potentially embarrassing. That being an example of a force which doesn't allow voice to display touch. But the Myo device seems to be operable reasonably privately if it is sensitive enough. The Leap lets you do gestures locally for action at a distance, I could see that as having some pull if people continue with large displays at a distance, but being less effective if the trend becomes many touchable displays close to you. I would say Kinect is a sort of mixed bag here, great for games, a huge win for Robotic vision, but less durable as new general purpose interaction method.<p>It will be fun to watch. Just hope my toy budget can keep up!<p>[1] This are clearly pretty arguable statements, but they are hear to serve as illustration of the force pushing change on people rather than a quantitative measure of that force.
I don't agree with Gabe's conclusion that "post-touch, and we’ll be stable for a really long time"<p>As you said: "where I'm looking", "where I'm gesturing", and "where I'm thinking" are all possible - but totally different. I think if post-touch gets critical mass, we'll see a much more diverse and dynamic space of interfaces. An implication is that which interfaces to support would be a much more important decision - right now you can just default to building a web app and a touch app.
The notion that touch has/is replacing the mouse and keyboard strikes me as silly. It will supplement, not replace.<p>Thought experiment: I offer to replace your /WORK/ computer with an equivalent computer that has a touch screen and voice recognition. (But no physical keyboard or mouse). Do you take this offer?<p>I'm guessing most of you would say "no", because you'd get less work done.<p>I think it's telling that touch interfaces are mostly being used in consumer devices. The "consume" in "consumer" is the key hint there.
The arrow of progress in human-computer interfaces has pointed toward steadily simpler interaction. Most people have no idea what a computer is capable of doing, much less how to make the computer do it. So the dual challenge has been to present the user with the available functionality and then make it as simple as possible to put that functionality into use. Thus, the modern smartphone. The command line is in a sense a much simpler interface, but absent a cumbersome series of menus it requires the user to know what <i>can</i> be done. (Touch has an analogous mode with complex gestures, but few people go beyond swipe and tap.) Even if we have thought-directed cursors, that only addresses half of the interface problem. Users still need to know what their options are, and there is no corresponding "thought-display" to go along with the thought-mouse.
It's interesting that when we think "post-touch" we still think about files and folders ("move the folder where I'm looking"). Maybe we won't need to think about organization -- the computer can handle that, and just play the songs we want, pull up the project we want, play the video we want.
Imagine a future where you play a video game in the first person, using virtual reality, controlling your avatar with your own movements. The most natural way for someone to walk, run, aim, and jump, is to actually do so. Angling control sticks, pressing buttons, and waving gestures in-air does not map one-to-one in the same way your brain maps to your muscles and limbs. Flow with the wiring of your brain rather than against it.<p>This system would require direct connection to signals from your brain, intercepting and consuming them, never reaching your muscles. Resting comfortably with REM-like paralysis, you control your game character naturally and intuitively, moving it and not yourself, all from the comfort of your couch.<p>Technology like this already exists in its infancy, such as providing moveable arms and hands to those who have lost them:<p><pre><code> "Move Arm" ---> Engage Robotics ---> "See Arm Move"
"Move Arm" ---> Animate Game ---> "Feel Arm Move"
</code></pre>
The current feedback loop in robotics is limited; we can't make Luke Skywalker's robotic hand flinch with needles yet. For things to work well in a video game, you'll need to feel your actions, just as a runner feels their foot-strike flick off the ground.<p>When will this technology be available? What bottlenecks need to be overcome for it to exist? How much will it cost? What other applications could it have? Will it be released before Episode 3? ;)
One thing that always surprises me is how little keyboards have changed in the past thirty years as computers became more mainstream. The majority of keyboards use cheap rubber dome keys in a horribly unergonomic setup. It's no wonder people dislike keyboards and thing of them as dated.<p>A personal aside — I use a Kinesis Advantage keyboard, as I have found that typing too quickly on other keyboards causes me wrist pain after a few minutes (I'm not affiliated with Kinesis or any other keyboard company for that matter.)<p>I don't think that keyboards will ever go away. I think that touch will complement keyboards, but will never replace them because keyboards allow you to type more quickly and efficiently and in a more comfortable fashion.
Perhaps when touch technology incorporates tactile feedback (by using electrovibration [1]), there will be touch based simulacrum of keyboards. However, there will still be the problem of having "Ipad neck" when typing if the keyboard is part of the screen.<p>For most people voice transcription software allows them to compose documents much faster than typing. But there are many problems with voice: besides mis-transcriptions, it is very annoying to listen to someone else dictating text and to try to work at the same time. Technologies using brain currents to create text or eye blinks are still a long way from competing with keyboards as a primary input device for healthy people.<p>[1]<a href="http://www.technologyreview.com/news/421191/a-touch-screen-with-texture/" rel="nofollow">http://www.technologyreview.com/news/421191/a-touch-screen-w...</a>
My issue with this post comes down to his last line "The abstractions between our intentions and how we execute them in computing are eroding. That's where the puck is going."<p>The question I pose is, how do we determine our intentions? I believe that the idea of a frictionless interaction with devices is impossible for anything worthwhile, because anything worthwhile requires actually thinking about what you want.
I think Bret Victor's article on the direction of interaction design is relevant here.<p><a href="http://worrydream.com/#!/ABriefRantOnTheFutureOfInteractionDesign" rel="nofollow">http://worrydream.com/#!/ABriefRantOnTheFutureOfInteractionD...</a><p>I agree with the author that we are heading towards interpreting the user's intentions in better ways. But physical inputs with tactile feedback are still to be beaten at their domain.
<i>>In my mind, it clicked that that we're always inching closer to computing being "do what I really mean."</i><p><i>>And here we are, not even a full year after Gabe's talk, and the first post-touch products are already landing on Planet Earth: Glass, Myo, Leap Motion, Kinect, on and on. </i><p>I don't think these devices/interfaces are really any better than touch with respect to "do what I really mean". It's still hand eye coordination at the level of our limbs and motor senses.<p>I think real "do what I really mean" post-touch interfaces are the ones where our minds become the actuators, instead of our limbs. Things like translating neural signals to actions. We <i>just</i> got the first Telepathic Mice [1], you know?<p>[1] <a href="http://jezebel.com/5987583/brain-implant-allows-rats-to-communicate-telepathically" rel="nofollow">http://jezebel.com/5987583/brain-implant-allows-rats-to-comm...</a>
There is a bit of neomania in this view. The keyboard has been with us for many years, which only strengthens that case that it will continue to be with us for many to come. Even touch screens use a tactile keyboard. If you take that further and think about the history of humans using their hands to express themselves, it goes back to all of written history (further if you include cave paintings, etc.). I'm sure non-tactile manipulation will be very important, but it doesn't have the history necessary to make solid predictions about its level of importance in the future.<p>"For the perishable, every additional day in its life translates into a shorter additional life expectancy. For the nonperishable, every additional day may imply a longer life expectancy. So the longer a technology lives, the longer it can be expected to live." -Taleb
Where did the 3D (VR) glove go? We've always had such alternative input devices, their existence doesn't mean that keyboard and touch are suddenly doomed. Keyboard and mouse are probably the main reason why PC gaming still thrives next to consoles.<p>The author also (mostly) missed the most promising post-touch/keyboard input device we've had lately: speech input in the form of Siri & clones, which has finally reached the masses after decades of development and use in some fields where the required extensive training of the speech recognition was feasible. It could be used more for games, in the age of MMOs, everyone is already used to shouting into their headset microphone.
"Now, in the touch-era, it's more like "move the folder where my finger goes," without any mouse-indirection happening. But with post-touch, the status quo will be "move the folder where I'm looking" or "where I say"; and one day (as Kurzweilian as this sounds) it'll be "move it where I'm thinking." Or simply "move it.""<p>A truly Kurzweilian scenario would be that you merely have to expect to find the folder somewhere, <i>and it is already there</i>.
I'm curious as to whether Apple without Jobs will continue to be the company that redefines* computing (PC, GUI, touch, tablet). Or will change come from another giant like Google or from an upstart like LeapMotion?<p>*Yes, I know they didn't "invent" these things. But they were instrumental in refining them and positioning them so as to catalyze the industry toward sea changes.
Current users of a paradigm tend to stick with it. Takes a generation to raise a new set of users open to something new. My father is a hunt-and-peck typist obsessed with knowing where his files are; my daughter's second word was "iPad" and will have little notion of files.
The dominant control interface for any device has to be incredibly robust. It has to be useable in almost every conceivable situation, and for almost every conceivable purpose the device is intended to be used for. The edge cases it doesn't cover have to be covered by an even more robust, but perhaps less sophisticated alternate.<p>Keyboard and mouse allow very rapid and precise text input, rich option and function choice, and very precise selection and movement control. Touch pads do a very good job or replacing the mouse for mobile devices like laptops.<p>For a long time touch wasn't up to scratch. Pens allowed more precise selection and motion, but were always a kludge because the pens themselves were too easy to misplace or drop while on the go. Once touch's early imprecision was overcome, it took over because you always have your fingers with you. Note that there is one case where touch isn't enough - controlling volume settings for your phone while in your pocket. In this edge case, physical buttons take up the slack. My point is not that touch has limitations (it does), but that you need to take a very long, hard look at any technology intending to replace it to be sure it is even more robust, and even more convenient and precise and has even fewer limitations in a huge range of situations.<p>Motion controllers like TouchMotion are extremely limited compared to touch. You can't use it in a relaxed posture, you have to have your hands raised and posed in the space that will accept gesture input. For precise selection, you need to have a cursor on screen like a mouse, because you don't have the directness of touch. Also while it's not ideal to use touch on a train or bus that's moving, trying to use something like TouchMotion would be a joke.<p>Voice control is highly problematic too. People actually find it extremely hard to be precise, to the level that many interactions with computers require, in verbal communication. That goes double for describing visual or spacial information verbally. Anyone that's ever worked phone tech support for computer users knows what I'm talking about.<p>Eye tracking has possibilities, but our eyes wander around and shift focus point all the time. Sometimes we want to look at something other than the thing we're controlling. Also I suspect that maintaining the disciplined and precise eye movements you'd need to replace touch or mouse/trackpad would be pretty onerous.<p>So I don't see touch going away for a very long time, if ever. I remember in the 90s pundits predicting that keyboards and mice were just placeholders and they'd be gone within a few years. The truth is you'd better get used to them because they're here to stay, and so is touch.