Believe it or not, this is a century-old problem.<p>The theremin is an electronic musical instrument, played by waving your hands in the air. It works by detecting RF capacitance between a pair of antennae and the player's body. You can see the Theremin being played at the YouTube link below.<p>Playing the theremin is incredibly difficult, due to the lack of tactile feedback. The human body is very poorly equipped to point precisely at an arbitrary position in free space. Only a handful of players can achieve anything better than squeaky science fiction noises and even virtuoso players struggle constantly with intonation. Modern theremin technique depends on a system of discrete hand gestures, which reduce the player's dependency upon coarse proprioception.<p>If the Leap Motion is to have any real utility, it will need <i>phenomenally</i> sophisticated software, to interpret intent from hand motion rather than simply passing the hand location as a raw input. The human body simply isn't capable of making the kind of movements that the designers of Leap Motion seem to expect, even with a great deal of practice.<p><a href="https://www.youtube.com/watch?v=Ptq_N-gjEpI" rel="nofollow">https://www.youtube.com/watch?v=Ptq_N-gjEpI</a>
I think the article is a bit unfair. I've been playing around with a leap for a few days and am suitably impressed.<p>What works very well with the device is coarse movements, especially relative hand movements. What doesn't work so well is finer gestures (1/100th of a millimeter motions of all 10 fingers? ha).<p>This app is a perfect example:
<a href="https://airspace.leapmotion.com/apps/cyber-science-motion" rel="nofollow">https://airspace.leapmotion.com/apps/cyber-science-motion</a><p>You can use your hands, kept flat, to spin around / zoom a 3-D rendering of a human skull. You can also point at specific elements on the skull. Both of these coarse gestures work great, and the experience is incredible.<p>However, the app also unfortunately has a "click" gesture to pick apart elements of the skull - you click by spreading out your thumb and then folding it back in. Works terribly, as this fine gesture is detected maybe 50% of the time. It should've simply been left out.<p>I showed this app to my dad, who's a doctor, and he was blown away. He was visibly excited about the potential for a device he can use to spin around CT and MRI scans in the operating room without having to touch a mouse/joystick - currently he has a person doing this for him to keep things sterile, and this can sometimes be frustrating.<p>The leap, at least in its current incarnation, reminds me a lot of Google glass. Both Google/LEAP and their proponents say the devices are going to change the world. Maybe, maybe not. Neither device works perfectly like what you see in the heavily edited demo videos. But both can be invaluable in certain specialized fields, <i>today</i>, as long as folks are realistic about what can be done with them.
There's a general assumption that the Minority Report interface is the interface of the future.[0] There are few reasons why I'm saying no.<p>1. First and foremost, gorilla arm.[1] My presumption with the "interface of the future" is that it's needed for prolonged use. So, first thing's first, the interface can't be one where our arms require our hands to be higher than our elbows. Unless of course our species got a whole lot stronger in the forearm to support such a feature. Don't see our species doing that anytime soon.<p>2. Feedback - Right now the feedback loop is eye->brain->hand->brain->eye (repeat) where the hand's pressure against a solid surface is the most important feedback response. With the minority report style interface we currently have a massive delay (comparatively speaking) between the brain->hand->brain loop. We also have to iterate the whole loop much more because we need to constantly assess with our eye where our hand is in 3D (not digital) space. Now let's say the technology gets much better and reduces this to 5ms. We are now bound by the differences of our synapses firing between touch and light. I could be wrong, but it's my assumption that due to the speed of light being the way that it is, that "touch" will always beat "sight" in performance.<p>For prolonged used applications my bet is on adaptive surfaces. For short term (turning an stove on, flicking a light switch, etc) interfaces I potentially see this Minority Report style interface happening. But does the benefit cost of innovation? Personally I think we are fooling ourselves.<p>[0] - <a href="http://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture.html" rel="nofollow">http://www.ted.com/talks/john_underkoffler_drive_3d_data_wit...</a><p>[1] - <a href="http://en.wikipedia.org/wiki/Touchscreen#.22Gorilla_arm.22" rel="nofollow">http://en.wikipedia.org/wiki/Touchscreen#.22Gorilla_arm.22</a>
A coworker and I have written some gesture recognizers for Leap in C# and Objective-C hoping to make the thing more usable and easier for other developers to write software using it. Hate to do a shameless plug, but both can be found here:<p><a href="https://github.com/uacaps/MotionGestureRecognizers-ObjC" rel="nofollow">https://github.com/uacaps/MotionGestureRecognizers-ObjC</a><p><a href="https://github.com/uacaps/MotionGestureRecognizers-CSharp" rel="nofollow">https://github.com/uacaps/MotionGestureRecognizers-CSharp</a><p>---<p>We're hoping to start the community foundation for making tools that help make Leap extremely usable from both a development and from a user experience standpoint. The Leap is awesome, beautiful and we think can be used in a myriad of applications.
If all you do is browse the web, a touchpad is often all you need.<p>For text editing / word processing, a good keyboard is often all what's needed, and the use of mouse is often discouraged by gurus.<p>I still can easily imagine using the Leap Motion device while editing images and especially 3D models. Even more I can imagine using it in games, especially games written with this device in mind.<p>I don't own the device but I tried it. What's great is that you don't need to wave your hands in the air, Minority report-style; moving your fingers us enough. I wish it was built into a keyboard; it would replace a touchpad / trackpoint easily, adding much more capabilities.<p>BTW does anyone here remember how clumsy were mice on PCs in, say, 1992?
I find the Leap part of BTT (BetterTouchTool) is actually... err.. use-worthy..? Neither useful nor useless.<p>I've got some really cool (still a big part for me) and useful stuff working, augmenting my mouse/keyboard use. For example, a finger to the left minimises and two fingers to the right opens a list of recently used apps.<p>Yet I'm very conscious that everything would just be better suited to a keyboard shortcut..<p>I never bothered with Touchless and mouse emulation things; years of 2D GUI design isn't suited to this kind of interface. "Midnight" is a my favourite Leap app but I think that's just an iPad app that lends itself very well to the leap input too.
I did some investigative work to look at if we could use the leap motions to replace the touch-sensitive overlays that we strap to 60" TV's for our on-air traffic folks to use during their segments -- the overlays run about 2500, and if we could replace them with a leap motion and get the same functionality with less cost, this could allow us to roll out the traffic application to more stations than the six or so that are currently using our in-house traffic application.<p>The first thing that I noticed was that it couldn't take the range of the 60" television that we had hooked up to the traffic software, so I scaled this down to a Thunderbolt display, and tried again. In my tests, the recognition for the thumb was sporadic, if not completely missing -- in both my (fat guy) test as well as during the testing of the local personality (non-fat guy).<p>I then made some changes to our software to try to minimize the effects of the natural movements of the hand -- I turned down the sensitivity to attempt to compensate for the normal shakes and jitters that you have with your hands. This gave it a better feel, but, the traffic reporters still missed the feeling of touching the display and watching that display interact with your touch.<p>They're still neat devices (I really wanted to say neat toys, but, I don't want to cheapen the work that the Leap Motion folks put into this thing), but, I'm having a hard time implementing them in a way that would work for us...so they're sitting on my shelf, waiting for a project that could use them (or, take them to my local hackerspace should I not find a good project for them shortly)...
Amazing, Revolutionary, Useless... and let's not forget buggy! And with horrible support... My device was not able to recalibrate, a problem shared with many others should I believe the forum. A week or two has passed, and no reply from the makers of the device, neither on the forum or on my bug report.<p>I guess I just got another hunk of junk to put in the failed-devices-closet... :-(
This is why I decided to back Mycestro [1] and skipped on the LeapMotion.
While it won't support multi-hand (unless you have 2 devices) and multi-finger stuff, at least it should be able to easily recognize any motion I make with it on.<p>[1] <a href="http://www.mycestro.com/" rel="nofollow">http://www.mycestro.com/</a>
It's obviously not a replacement for a mouse and keyboard and never will be. I could see some useful gesture based macros, like "throw your hands up in total frustration" to rage-quit an app or open a distraction-free full screen editor.
I find Leap Motion quite accurate. You will quickly get used to the convenience of gestures with BetterTouchTool, and wish you had it on computers which don't have Leap Motion.<p>Otherwise, The gestures used by apps is something that needs to be carefully crafted. For instance Touchless, the mouse replacement, simply doesn't cut it; you'll find yourself reaching out for the mouse/trackball/trackpad within the first 10 seconds.<p>The leap gets effected by strong light sources on the ceiling. You might want to use it facing downwards if that is an issue. Also, if you are wearing a watch or a ring, it might get confused with the reflection.
<i>Disclaimer: I haven't tried Leap Motion, yet.</i><p>Did I get this right? Leap Motion vs. Kinect:<p><pre><code> - LM is smaller (significantly);
- LM is cheaper (significantly);
- LM is more accurate (significantly);
- LM has almost no real apps (mostly concept demos).
</code></pre>
If these are all correct I find Scott's post nothing more than a "normal", "the competition sucks, too", Microsoft type of post.
What surprised me after little digging up was that Leap Motion does not support point cloud. That means you can't get 3D world as points in space from Leap Motion. Their founders says Leap Motion isn't designed for this purpose. This means you can't use Leap Motion for applications such as 3D scanning. Personally I think that would be much more exciting then ability to move windows by waving.
The biggest problem with the leap that I have seen (there is one in my lab) is that it sees hands spread out evenly on the surface really well, and that's about it. If you do a thumbs up gesture (or a rude gesture when it doesn't work), the fingers get occluded by the bottom of your hand.<p>Think about you hands as five friends trying to play connect at the same time and you can imagine the kinds of occlusion problems you might face.<p>Still, I, and probably others, like the leap. It's not useless. You just have to exploit it the right way, looking for natural interface design beyond a Tom Cruse movie.<p>The biggest free air interaction problems are (1) making visible what the available gestures are, and (2) providing tangible or visible feedback. You don't get to see and feel the the interaction like you can with a keyboard or less digitally inclined tools.
I've been experimenting with the LeapMotion at work today, and some initial observations:<p>- There's no apps yet that have made me go wow.
- The range is quite small
- The motion of hovering an arm in front of you is extremely tiring after more than 10-15minutes. Try holding your arm out in front of you for that long without moving and you'll see why.<p>The reason Kinect was a success is that you can take real-world activities such as dancing, jumping over obstacles, jogging (on the spot), and translate them into an interactive digital version.<p>With the Leap, I've yet to think of a real world scenario where I would be waving my hands in front of my chest, that would translate well into a digital experience. Conducting an orchestra would be one good application for this, perhaps training conductors, but I couldn't think of anything else.
I think the "useless" comment misses that its sweet spot utility is unknown at this point. And I'm not sure "general purpose computer input device" is going to be such a sweet spot.<p>The precision "problem" can obviously be addressed with software.<p>That said, I do believe the absence of killer utility is a problem for Leap right now since it came out with a decent bang and now the less than favorable reviews are dripping in. I think they would have done themselves a significant favor by having a killer app ready from the outset. I also think they need to encourage people to look beyond simple human-computer interaction. Apparently these things could map a whole football game or count the number of people at a concert. Things like that. I also think the commercial angles will be better for business.
I've got my Leap doing what I bought it for: controlling iTunes when my hands are covered in hair dye. [1]<p>If I can get it to do more, awesome. But that's enough for now.<p>1: <a href="http://www.youtube.com/watch?v=_O6sR0PKofc" rel="nofollow">http://www.youtube.com/watch?v=_O6sR0PKofc</a>
I think that the newer Kinect 2.0 will be able to take care of all these more sophisticated gestures. When compared to the Leapmotion it has much better range and its accuracy will improve now too.
Four students at RIT are using this to develop a sign to text app.<p><a href="http://motionsavvy.com/" rel="nofollow">http://motionsavvy.com/</a><p>So it does have potential/use.
I also have one, and after seeing what it sees during the orientation I wondered how many of its problems could be cured by having a second sensor that one could position about a foot away from the other to give the Leap stereo vision. I am speaking totally out of school because I don't know if that would create more problems than it solves, though.<p>On my phone, to apologies if this is a duplicate of another thread.
Nice example of someone adding a leap motion controller to their site, GoSquared:
<a href="https://www.gosquared.com/blog/playing-around-with-the-new-leap-motion-controller-on-the-web" rel="nofollow">https://www.gosquared.com/blog/playing-around-with-the-new-l...</a>
I'm not going to argue this demonstrates utility, but it does show some good recognition of smaller gestures.
Looks like a much better alternative just revealed itself: <a href="http://www.kickstarter.com/projects/haptix/haptix-multitouch-reinvented" rel="nofollow">http://www.kickstarter.com/projects/haptix/haptix-multitouch...</a>
I think combining a kinnect and several leaps with an Oculus Rift might be great. That way you can use your own body motions within the rift. The kinnect would be for gross motions and the leaps would be for detecting fine motions.<p>What do you guys think?
Can someone build something like this but only have it recognize a small set simple gestures? Using it to simply browse the web would be cool. If you could throw in a way to click build in Xcode and IntelliJ then I'm sold.
This is exactly my experience with the Leap unit; I played with it for a couple of days. Then uninstalled all the drivers and put it back in the box, will try again in a few months to see if things have improved.