Show HN: Flutter - Control Spotify or iTunes Using Gestures thru Webcam

Amazing, works very well.<p>I assume it is looking for 5 fingers. I noticed if i folded my thumb in, it did not recognize the gesture so maybe an edge case where people missing fingers can't use this.<p>Also when I scratched my face briefly it did recognize it... funny.<p>I would strongly recommend trying to build a web demo somewhere down the line. Even if it requires flash to get the webcam working, I'm sure your conversions would go through the roof if you could show how well it worked instantly on your landing page.<p>Small note -- the blurring of the image is distracting rather than focusing. Down the line maybe think about a better landing page image.

We hated using the keyboard and mouse for controlling music in the background or watching movies on our computers from a few feet away. So we decided to solve this problem by using hand gestures to control music and videos via the built-in webcam in your computer.<p>Please download and try it. Do let us know what you think in your comments below or by emailing us at feedback [at] flutter.io.<p>Currently, it works for Mac OSX 10.6 and up.

I hardly ever get excited after downloading something, maybe a, wow that's cool, but when I installed this (which was very easy) and it just worked, I actually clapped and said 'this is fucking awesome.'

I believe one of the co-founders is Navneet Dalal, who, along with Bill Triggs, developed the "Histograms of Oriented Gradients" (HOG) features that are widely used across all of computer vision†. Thus, I'm guessing that there's some serious vision in use for this app, which is pretty awesome!<p>(My apologies to the other co-founders, who are likely just as talented!)<p>† <a href="http://en.wikipedia.org/wiki/Histogram_of_oriented_gradients" rel="nofollow">http://en.wikipedia.org/wiki/Histogram_of_oriented_gradients</a>

This is awesome.<p>However, music control isn't a pain point for me. What I <i>would</i> pay for is a start/pause gesture for Netflix/Hulu/Amazon Video. Worst thing about TV on a computer.

I just ask my very non-technical girlfriend to come see this new app I downloaded. I showed her flutter.io working in action. She shrieked and said, "that's awesome, how do I get it?". That's a very good sign for you guys...nice job.

Though I know there's a chance you'd not be willing to share the information, I'd love to learn a little bit about how you built this product.<p>It appears as though you're using ML to recognize the gestures—anything else you could share would be fascinating. I'll be sending in a job application, as well!

Interesting/cool project, though I do wonder how you plan on keeping it simple when you expand and add additional actions?<p>The UX of controlling media using the Xbox Kinect leaves a lot to be desired - it's cumbersome.<p>I've had my mac mini connected to my tv for awhile and am very interested in the Internet TV space in re: what will become the standard way we control such technology? Will it be thru gestures, airplay on iphone/ipad type technology, voice driven or a mix of all?<p>For many years I've been using a wireless gyration mouse/keyboard (demo of Internet TV setup <a href="http://www.youtube.com/watch?v=l-IryK4wsUM" rel="nofollow">http://www.youtube.com/watch?v=l-IryK4wsUM</a>) and recently I started to use Chrome's voice search to quickly bring things up especially if my wireless keyboard isn't reachable.<p>After seeing this and my years of enjoying the Internet on my TV I'm thinking a few short gestures like this app uses and mix of voice driven actions might provide the best UX.<p>Based on the views and comments of video I link to above people want the Internet on their TV and all it provides. We just haven't figured out the best UX for it.

This is amazingly useful... just not on the pc.<p>Implement it for Android and you can charge me 30€ so i can easily control the mp3 player in my car.

"The music swirled and dived for a moment. Another voice broke in, presumably Halfrunt. He said "Vell, Zapbod's just zis guy, you know?" but got no further because an electric pencil flew across the cabin and through the radio's on/off-sensitive airspace. Zaphod turned and glared at Trillian -- she had thrown the pencil. "<p>HHGTTG - Douglas Adams

Flutter in action: <a href="http://www.youtube.com/watch?v=IxsGgW6sQHI&#38;feature=youtu.be&#38;a" rel="nofollow">http://www.youtube.com/watch?v=IxsGgW6sQHI&#38;feature=youtu...</a>

This is very cool. It worked perfectly even in the dimly lit room I am sitting in.<p>What I've been wanting and have thought about developing myself is a universal play/pause control that pauses whatever you happen to be listening to at the moment, whether it be a desktop client like iTunes, Spotify, Rdio, etc, or if it is something running in a web browser like Pandora, Soundcloud, Youtube, etc.<p>This app takes a step towards this in the way it handles iTunes and Spotify at the same time, by controlling whichever app had focus last.<p>I imagine for web browser control a browser extension would necessary. And ideally a plugin architecture would exist to allow 3rd parties to add support for additional apps.

Really cool. I was just playing with it and my girlfriend walked over to ask me a question. I held up my hand to my computer as if to say "Hold on, computer."<p>It was a bit surreal. Very cool. Can't wait for more gestures or a generalized way to program my own gestures. I would love to be able to gesture "Switch to iTerm.app" by doing an OK sign or something. Sign language letters maybe?<p>Anyway, thanks. Really enjoy it already, can't wait for more. Also, I would pay money for this. If it had all of the play/pause/forward/back functionality I think I'd give you 4.99 if it was accurate. Just sayin'.

What's nice about this is you don't need to switch screens to pause the music if you find yourself writing/coding and need some quiet when you're in the zone. Granted you can do this with the mac play/pause physical buttons but that would require your eyes to look down and pause your thought process.<p>This happened to me just now after I installed Flutter, and it seemed to me that physical gestures didn't interrupt my higher brain functions. Of course when I noticed this it made me interrupt my work and come to HN to leave this comment. The irony.

Flutter seems to keep the camera on 100% of the time which prevents apps like Skype from using the camera for video chats. I'm not sure if there is a good workaround for that?

This is fantastic! I can't wait until it's extended with more gestures for next track, etc, but this initial version is so awesome! Nice work.

There is another, more fully-featured app called FluidTunes which I used quite a while ago. Not sure if it still works on the latest version of iTunes: <a href="http://majicjungle.com/fluidtunes.html" rel="nofollow">http://majicjungle.com/fluidtunes.html</a>

This is awesome, but one question/suggestion: when a user is going through the "How to Use Flutter" tutorial, do the gestures work then, or only once the tutorial is closed? I kept trying to do it, but there wasn't a response until I closed the tutorial.

One of the few occasions that I wish I had a mac.<p>Few questions: Aren't you entering a field which has patents covering gesture controls on media exactly like this?<p>A comparative advantage of mice and keyboards is that a minor amount of energy, (but large amounts of co-ordination) are leveraged into larger changes. Wont this limit the number of the gesture controls you can incorporate?<p>(OTOH - if you had enough fidelity, you would just need a dummy keyboard. You would just have to map finger movements with key presses. you could just create a webcam of your own as well.)

Very cool although with CPU usage at about 30% on my Air I won't be able to use it much. If you can figure out a way to drop average CPU usage down to reduce heat &#38; power usage that would be a big deal.

Soooooo. Doesn't this open up a whole new realm of privacy concerns? Who cares if Path has my contacts list, I'm more concerned that you guys are watching me surf for porn!

Hi guys. Since there seems to be an ML component, I was wondering how this was built. Do you start with a basic hypothesis and see if it holds for some training set and then refine it or right off the bat go with a generic classifier like SVM (with some features in mind) or something like that and then tweak it? Also, how did you choose the dataset for this ?<p>This project is very cool. Congratulations and good luck!

Would be nice if we could get this to work with other media players by emulating the media keys (really want to use this with vlc on my macbook)

I was blown away when I saw a demo of Flutter a couple of months ago. Their Spotify and iTunes integration only scratches the surface of all the gestures that they supported in that demo. I can't wait for them to add some additional gestures, such as skipping to the next track, and PC support so that I can get rid of my keyboard with the dedicated music player controls.

This must be amazing app to download as it is clear from the feedback received. Please don't forget that we are a large enough community who uses windows. Especially in India. We are already excited even before you have our version. the kind of capabilities it gives, would be amazing to download. Congrats in advance to the team.

Fantastic app, does a fantastic job of recognizing my palm - unfortunately, it seems to also really like my face. After the tenth or so time that it had paused my music without me raising a hand, I realized that every time I turn my right cheek to the camera, Flutter pauses my music. Which is a little troublesome :)

This is fantastic - absolutely the most satisfying method of interaction with an app that I've experienced in a long time. Expand the gesture set to include skip forward/back and it's a winning consumer app. Personally, I'd love (and would pay for) a library allowing me to add gesture control to my own apps.

Now that's what I call a very awesome MVP!

Amazing, the single gesture is <i>very</i> well implemented.<p>Can't wait to swish my hands around to skip songs! :) And once you have that, you should build a futuristic minority report kind of a demo video ;)<p>Also, if you're looking to hire, you should definitely put a "Team" page up on your site.

I did a similar hand gesture recognition software to control image viewer as part of my computer vision course project. It was really fun to do. I believe, presentations and lectures are the places where this kind of technology really makes a lot of difference. Good luck with your startup!

Blown away.<p>How far are you guys from full Minority Report-ization of my Mac?<p>I mentioned in another post that I tend to work in very dimly lit spaces due to painfully photosensitive eyes. A non-mandatory calibration dialogue would probably be a handy thing for folks like me.

Neat and simple, but as an end user I would use this a lot more on my TV than on my MAC/PC. Samsung built in cam below.<p><a href="http://www.youtube.com/watch?v=Szr5hGU3EK0&#38;feature=related" rel="nofollow">http://www.youtube.com/watch?v=Szr5hGU3EK0&#38;feature=relat...</a>

Wow - I originally didn't download because I've used something similar before that worked only kind of well.<p>This works fantastically and is usable enough for daily use as a simple way to control iTunes or Spotify. Well done!<p>I'll look forward to more gestures.

I saw Samsung demoing some similar technology during SXSW (using a kinect that was hooked up to a mac mini and being displayed on one of their LED tv's), and it didn't work anywhere near as well. Congrats.

Please note that this seems to be available for Mac users only right now.

Looks awesome, can't wait for a Linux version, so that I can try it out!

Works great once I realized that I couldn't be so close to the camera, but does anybody have any tips on disabling the green iSight LED on my macbook pro (short of covering it)?

A user just created a video for us - Thank You So Much!! :) <a href="http://www.youtube.com/watch?v=lhnlUMHUlNc" rel="nofollow">http://www.youtube.com/watch?v=lhnlUMHUlNc</a>

Quick note. Your site assumes I am using Chrome - FF doesn't save downloads in the bottom left hand corner. Easy for me to figure out, impossible for someone like my mom.

Brilliant work! truly awesome . Shall i expect more in future for WEB like this ? <a href="http://depthjs.media.mit.edu/" rel="nofollow">http://depthjs.media.mit.edu/</a>

Hey guys this is some really cool stuff. If you can integrate this with browser gestures to scroll, refresh, and move back and forward on pages. This would be awesome.

I <i>so</i> hope this is an art project and in a few days we'll have a website of people making funny gestures to the camera to pause and play Justin Bieber songs ;)

In case anyone was wondering, I can confirm "mirror stops" work: try facing your laptop towards a mirror and doing your gestures from behind the computer- wow!

On Lion it seems to register itself automatically in startup items. There is no Preferences or Settings window to change this either.<p>Very awesome overall.

This works incredibly well. I wasn't sure how accurate this could be without the kinect-style depth sensor but it's surprisingly accurate. Great work guys!

So this is Mac only?

I've been sitting here for like five minutes just starting and stopping. About every third time I laugh and say (out loud) "that's so awesome."

It would be interesting to know what piece of technology, or framework, they're using for recognizing gestures using the webcam.

It seems like the Minority Report future of gesture based control is not that far off. Great app!

You have built something people will want to talk about but why limit to spotify or iTunes :)

Please allow me to control a presentation in Keynote. I've wanted this feature for so long.

Looks very interesting.<p>The download link seems be down though (internal server error).

This is so freakishly awesome !!!

This is so good that I'm putting VLC away to use iTunes to listen to my music library.

issue is that even when i am not controlling the playlist and randomly moving my hand it recognizes it as gesture... seems like lot needs to be done to differentiate between noise and actual gesture. anyways nice try.

Great app. Looking forward to support for more gestures!

That's amazing.<p>I... wow.<p>Take my money.

Wonderful job guys!!! very very impressive

Desperately want a skip gesture.

i imagine you guys are already working on the skip/back functions. Awesome initiative!

Makes me feel like a jedi !!!

This is amazing!!

Great job guys!

Brilliant work!

this is sick, like @zigfu

It would be cool if this also let me clap as a gesture. And it would be cool if you sold an arduino kit that let me hook up my light switches.

Congrats guys, this is very cool.

Surveillance in the name of convenience has gone too far. I can see a future where companies offer services as wrappers around visually analyzing people's behavior in their own homes. They will sell that data just as they do now. That's not a future I encourage.