Internally here at Guidewire, we have a declarative widget-based web framework for our apps, and we test by automatically generating a Page Object model based on the declarative widget definitions (using metaprogramming techniques in Gosu). The rigidness of defining your application in terms of declarative widgets is not for most applications, but the resulting improvement in testability is probably the biggest benefit. Separating out a logical model of "widgets" on the page and writing against a Page Object model, whether or not your web framework has widgets, is clearly the right way forward to me. There's nothing worse than debugging hundreds of test failures because no DOM element with id "User_Search" was found on the page. Did the page not render? Did the test run too fast? Did someone change the ID? Did someone move the search button to a different page? Did they hide it behind some div I need to expand first? Did the page just break? There goes two days of development time . . .<p>Prior to that approach, we tried everything: we actually wrote our own test framework that used javascript to drive the browser from inside the browser, then we moved to using selenium, but they all suffered from that same problem of incredibly fragile tests. Over time, test maintenance is easily 10x the cost of initial test development, so it's a huge, huge problem.<p>So I see this as a promising direction, but building and maintaining that (essentially) Page Object model by hand is going to result in its own set of maintenance problems. How do you know when the model is up to date? If someone changes the model, how do you know which test that affects if you're in a dynamic language like Ruby? It solves one set of horrific problems, and replaces it with another set of problems that are hopefully, with enough developer discipline applied (and guidance from the framework), ultimately slightly less horrific. Being able to automatically generate the POM via metaprogramming by statically analyzing your application is awesome, but it's not a technique that really works with any standard web framework that I know of.<p>Web app testing really still has a long, long, long way to go.
I think the goal of "impossible to write bad tests" is unreachable. It's like "idiot-proof": impossible because the idiots are too resourceful. I'll be really happy, though, if Test::Right accomplishes "easy to write good tests." Right now, nothing out there does.<p>I especially like the decoupling of intentions from mechanics. It lets us improve the mechanics without breaking or rewriting all the tests. Maybe the mechanics are implemented with JS DOM-traversal code, maybe it's CSS selectors, maybe it's SIKULI-like image recognition (<a href="http://sikuli.org/" rel="nofollow">http://sikuli.org/</a>).<p>Looking forward to trying it.
I see a big problems in web testing caused trying to test drive from outside of the browser. I'm currently working on a solution that has tests written in javascript that run <i>inside</i> the browser. The test runner puts your web app into an iframe, so the test code can use jQuery, for example, to easily automate and assert over the UI.<p>JavaScript also makes creating a very clean testing-focused DSL easy.<p>I'm hoping to open source the project soon. If anyone would like to know more give me a shout (@andrewdavey).
I'm having a hard time pinning down the feeling, but there's a lot more declaration here rather than evidence.<p>In other words, shut up and show me. (This isn't an attack on saucelabs)<p>The github page is sparse on documentation, and the wiki page is basically a copy of the README.<p>So don't make vacuous claims. Write an article demonstrating how this system works, how it compares to X, and why it's better than X.
tldr;
"I hate browser testing. Did I fix the problems? No. I started writing opinionated software though, because my opinion should matter. I wrote it in Ruby because I know Ruby."