It seems like all "code exercise" sites like this have the same annoying problems which I'll generally describe as lack of attention to detail in the exercise descriptions.<p>Take their Fibonacci exercise as an example. In the first section they say, "The first line of the input will be an integer N (1 <= N <= 100)" which specifies how many test cases follow. Then in the next section they say, "The first line of the input will be an integer N (1 <= N <= 10000), specifying the number of test cases"<p>So besides the fact that they aren't even consistent in specifying the expected range of N it isn't even clear why you would need to specify the number of test cases in the first place vs just reading one from each line until EOF. That makes you think maybe they want you to put in some basic error checking which again since they aren't consistent with N turns into a trial and error exercise if it even matters at all.<p>There is also no indication about the version of interpreters or compilers being used to check the submissions. If I choose to write my solution in python is that python 2 or 3?<p>Combine all that with an implied scoring system based on speed or number of tries and you have to wonder if this is really measuring anything relevant or if it is just filtering for people who's default assumptions happen to be the same as the person who wrote the lame exercise description.
A number of start-ups have either already failed with a very similar business model or failed to go exponential growth. There was a good "Lessons Learned" post by one of them recently here on hacker news that got comments by both hiring managers and hackers. Why the rehash, again? What is this start-up really doing differently?<p>I personally wouldn't be motivated to use it either to find work or to hire. I think, when hiring, you really need to do a face to face interview. I've found resumes work well enough for pre-screening that it's not a pain point for me. And, it's not like there are so many candidates out there looking for work that it's impractical to do face to face, or at least telephone/skype, interviews with all the good candidates. I learned the lesson that face to face hiring is needed by hiring someone based on a co-worker's recommendation (even though he did badly in an interview, I looked passed it based on the recommendation). It's so costly and painful to make a hiring mistake that there's no way I would deviate from the current model and try a new method or service.
I wish this company well, and perhaps they'll be successful, but I don't think they've nailed down some ultimate formula for ranking and hiring programmers. This system sounds very easy for low-quality programmers with lots of time on their hands to manipulate - you sign up with a throwaway account with some fake name, get the list of questions, take your time searching for the answers online, and then sign up with your real information and have a carefully prepared way of typing out answers that get you high scores. If this achieves any measure of traction, you can bet that this will happen a lot.<p>This isn't an easy problem to solve and is something that companies have been struggling with for a long time, so no disrespect intended. But based on my understanding of how this works I don't think it will be very likely to yield great results.
As a younger developer, I struggle to imagine our senior developers - who have been doing complicated things for 5-10+ years - ever stoop to a number to quantify their experience. I suppose that senior developers are probably not the target audience, but I would never want a company to expect this, or a score to define my experiences as a developer.<p>A technical screening can be handled in a ten-minute phone call for free - with the dignity of the applicant intact.
Best of luck, I recently received an email that WorkforPie.com was shutting down (<a href="http://thenextweb.com/insider/2013/08/02/developer-hiring-startup-work-for-pie-to-shut-down-on-august-31-points-users-to-coderwall-and-pitchbox/" rel="nofollow">http://thenextweb.com/insider/2013/08/02/developer-hiring-st...</a>) and they had the same sort of model.<p>I get we always complain about the developer interview process, but are these business models actually solving the problem?
"...Lack of coder personality" That's more that just a mere gripe.<p>Many a job has been won or lost based on the character of a prospect. In fact, I've lost count of how many times I've seen otherwise very technically skilled people lose out to basic inter-personal communication. Above all else, a seeming lack of tact.
Coding is a small part of a developer's job. Given a simple task, a programmer can code to it. A software engineer can create a proper design that expands out to solving a wide range of problems, a design that is easy to maintain and flexible enough to deal with problems.<p>I don't want to see someone's first solution to a problem. I want to know the solution they came up with after 5 mental iterations and a few hallway discussions with peers.
Only people like software engineers would stoop so low to subject themselves to measures like these.<p>Aren't technical interviews (which are just politically correct way to test IQ) enough of a humiliation one has to go through?
In South Australia (and potentially elsewhere) university admissions for medicine have started to involve something of a personality test rather than purely academic aptitude. The workplace, and especially dealing with patients (colleagues or clients in this case?), is more than a calculated diagnosis.
I was disappointed to see that they limit the password length to 30 characters. Perhaps one of the exercises should be to hash passwords that are longer than 30 characters.<p>Bcrypt will support up to 72 characters. If you are going to limit the password length I'd expect it to be something closer to 72 than to 30.
It seems like as time goes on YC is funding far more startups that are direct competitors with each other. Hackermeter's pitch looks almost identical to that of InterviewStreet (YC S11).
Sounds fun, but everyone solves the same problems (ok, ideally not: "sets of custom-built, rapid fire coding challenges built by each employer. You can use Hackermeter’s pre-provided challenges, or build your own")? How long until copied solutions corrupt the scores? Better check performance on a held-out set before interviewing/hiring.<p>Let's not get into plagiarism-detection arms race; consider automatic refactorings / pretty printers with variations, and human input to vary variable names.
Oh Yay, another useless measurement of a programmer's abilities, very much as useless as the puzzle interview made popular by Microsoft and then Google. So you can measure somebody in an artificial sprint environment that has nothing to do with actually delivering a product. The real measurement is how somebody does delivering actual products to market.
Hey xianshou, I've been trying to figure out how to privately message you guys about this, but your info@hackermeter.com email doesn't seem to be working. Your online code editor is leaking environment variables pretty severely; stuff like API keys.
I don't know if this is common knowledge or something, but how are all these companies implementing a secure way to execute arbitrary code? Sounds like a non-trivial thing to get right.
Check out <a href="http://codewars.com" rel="nofollow">http://codewars.com</a> (only supports JS, and Ruby). They offer problems with a voting system for solutions.
Have you considered adding Scala? You're already running jvms for Java, and you could do incremental compilation with SBT to keep everything responsive.