OP struggles with XPaths in complex DOMs. This is a solved problem. Use Helium [1] to write test code that acts on what the user sees:<p><pre><code> start_chrome('github.com/login')
write('user', into='Username')
write('pw', into='Password')
click('Sign in')
</code></pre>
I am the author of Helium. It's a wrapper around Selenium. It's fully open source. Under the hood, it uses Selenium 3, which is old. But boy does it work beautifully.<p>1: <a href="https://github.com/mherrmann/selenium-python-helium" rel="nofollow">https://github.com/mherrmann/selenium-python-helium</a>
You're going to get a lot of responses that say record-replay testing doesn't scale, is unmaintainable, etc. They're totally right.<p>But the thing is, early on when your app is in flux, neither does writing Selenium code. There's a pretty big truism in UI automation that writing UI tests before a UI freeze is a recipe for a shit-ton of pain. Coding to ids or xpaths only gets you yay far if the UI flow itself fundamentally changes.<p>But re-recording might be easy.<p>Don't use stuff like this for long-standing tests. Unless you architect your app just right so that IDs are always stable, and you never change the interface, it'll break and break in ways that you can only fix by a complete re-record. Plus the tests tend to be extremely timing-fragile because recording uses delays instead of synchronization points, so they just won't work in CI.<p>But <i>do</i> use stuff like this at your desk during bring-up, when the cost of a re-record is lower than the cost of a test rewrite and it's ok to re-run the thing if the first try craps out due to a timing glitch.<p>And from there, keep an open mind.<p>I went to a GTAC (Google testing conference) where a presentation made a very good argument--with numbers and everything--that for smaller projects with simple and more or less static UIs, and where the tests were all fundamentally scripted, there was almost no advantage to coding the tests later. Record-replay was the best way to go.<p>But I definitely don't think a system like PlayWright fully replaces remote stubs like WebDriver and coding tests in the languages that can talk to it.<p>At some point you hit the issue that the login screen changed and now every single test is invalid. It's awfully nice if you used Page Object Model and you have a single-point-of-truth to fix it.<p>More to the point, test automation can do more than just execute a script repeatably. Randomized monkey testing is something you can really only do in code, ditto scenarios that need to talk to testing interfaces exposed from the app itself.<p>Glad you found a tool that resonates with you!
> Selenium just couldn't parse the over-engineered Javascript framework we had, and I had to pile on hacks on hacks.<p>Excuse me, but isn't selenium actually a browser robot and not a javascript parser? I assume this is just that DOM is mutating so much that they could or could not find that element or always have to wait for it and comes with unreliability and such.<p>I haven't used alternatives, but using selenium feels like solid "Engine" to make abstractions on.
I’ve tried writing e2e tests for web apps using browser drivers on-and-off for the last decade, and it feels like with playwright that the tech has finally matured enough to make the tests reliable enough to be worth the effort of maintaining them.
OP suggests that having a large company behind a software tool (Microsoft in full control of Playright) is a positive thing and is an advantage over Selenium.<p>I tend to see that negatively. Community driven projects (or non profit organisation behind them) like Python, Postgres, Firefox always are more trustworthy and likely to last longer in general.
> So much so, I started looking at Javascript testing frameworks like Chai, Mocha, Cypress etc. The problem I found is that they require a completely different setup, and aren't easy to get started for someone from a Python background.<p>A shame to see Cypress lumped in and dismissed like that. It really is a fantastic way to test.<p>> The killer feature of Playwright is: You can automatically generate tests by opening a web browser and manually running through the steps you want. It saves the hassle I faced with Selenium, where you were opening developer tools and finding the Xpath or similar. Yuck<p>This is absolutely <i>not</i> the primary hassle with testing. Recording steps can help kick start your testing, sure. But very quickly you start to realize that it's only saving you from the easiest work at best, and is creating flaky tests at worst.
As a long time test automation dev, I would never want to be tasked with e2e browser automation again. I did that for a brief period in my career. It didn't feel like real coding. The tests always seemed like they were barely less maintenance than manual testing. I'm curious how big the e2e web tests suites are for best in breed web software. My opinion has been to minimize the amount of e2e browser tests and do app integration tests solely against the API (assuming there is one!), but I've never specialized in web app testing.
My journey has been Selenium, then Cypress, then Playwright.<p>This was the first time I was happy writing ui tests, Playwright is great!<p>(Though I don't use the recording tool at all this article focuses on I rather write the tests manually)
I'm always surprised to see posts like this, and no one mentioning testcafe. It's a super solid e2e testing library! I've been using it for quite a while and it runs reliably and consistently. The API design can be a little messy, but it was (and still is!) a top contender when I was evaluating tools in this space. And open source and contributor friendly. Not sure why I so rarely see it mentioned!
Playwright is the bee knees. At my last tech job I did maintenance on about 400 wordpress sites, and I used playwright to automate a bunch of monotonous clicky tasks normally done via the wordpress backend gui. It was a lot of fun and toward the end I got playwright running on heroku and made a gui so that my not very tech coworkers could upload a csv of different sites and then and a select a dropdown list of different scrapers, tests, etc.
Selenium, Playwright, and Puppeteer all make use of Chrome DevTools Protocol (CDP) to access the browser, even Firefox. CDP is the current de facto standard to remotely access the browser. It actually starts an HTTP server in the browser that you send requests to. The W3C is working on actual standard for this space that will likely be more mature and durable.<p>My learning from playing CDP and the test automation applications mentioned is that this is way harder (and slower) than test automation should be and it’s a huge pain in the ass for authoring tests. The experience is so bad that it only seems to appeal to professional testers whose jobs depend upon it.<p>To solve for that I wrote my own test automation solution that does not require remote access to the browser. In my personal application where I use this form of testing I am able to execute about 30 user events per second in the browser. That performance speed is completely dependent upon other performance conditions of your application, hardware, and transmission handling. The test cases are simple data structures defined as TypeScript interfaces in a Node application communicating to the browser over WebSockets. The events are executed by creating a new Event object in the browser and executed with the event’s dispatchEvent property.<p>When your automation gets really fast (less than 30 seconds for a major test suite) everybody will make use of it including the developers, product owners, and even the business leaders. It becomes a faster way to setup and access various features of a browser application than doing it manually.
I just spent half an hour wrestling with Selenium for Python and that API has NOT aged well.<p><pre><code> content = driver.find_element(By.CSS_SELECTOR, "p.content")
</code></pre>
In Playwright Python:<p><pre><code> content = page.locator("p.content")
</code></pre>
Playwright is just nicer to use. Selenium feels very Java-ish.
> I found weird issues (when using the recorder) where I couldn't scroll down to the bottom of the screen in Playwright's Chromium, but could in a normal Chrome, forcing me to use other tricks to click the button.<p>You may consider trying this extension - <a href="https://chrome.google.com/webstore/detail/devraven-recorder/fonaldkpejilcpapnkcfecppoognbcah" rel="nofollow">https://chrome.google.com/webstore/detail/devraven-recorder/...</a>. This extension can record the tests using your Chrome browser instead of launching a separate Playwright chromium browser. Full disclosure: I am the developer of this extension.
For those of you who are as confused as I am about Playwright's growth over Selenium: <a href="https://star-history.com/#microsoft/playwright&SeleniumHQ/selenium&Date" rel="nofollow">https://star-history.com/#microsoft/playwright&SeleniumHQ/se...</a><p>My company uses Selenium for some of our projects, but I evaluated the browser automation space some time ago and found that it was the best-in-class solution at that time.<p>It looks like things have since changed.
I’m here just to say how much I love Playwright :)<p>Best features for me:<p>- Auto-waiting - it is a BRILLIANT feature!<p>- Shared authentication;<p>- Supported browsers list;<p>- API testing;<p>- Tests generator (with recognition of data-test-id attributes);<p>- Flaky tests detection;<p>- Flexible config for recording failed tests;<p>- Ease of installation and updates;<p>- Great performance.<p>Thank you, Playwright team!
Playwright is good, but stop thinking auto generated code is worthwhile.<p>Eventually you'll have to write your own scripts. I will admit Playwright can spit out code that will at least serve as a starting point.
Unfortunately, the .NET library doesn't support ARM processors. Thus, I can't use it on the Raspberry Pi. But PuppeteerSharp works great on the Pi.
Weird question. The trouble with web automation tools such as Selenium and beautiful soup is that they require specific instructions to parse html. So it's difficult to auto scrape websites if several websites have a different html layout or any website simply decides to change it's html. Would it be possible to create a neural network model that could be trained to parse html as a human would? Or an AI for that matter that could break Google's captcha. I'm surprised that that still works given the advances in image recognition. If both are possible, the security on the web may quickly become a thing of the past.
<i>None</i> of these solutions are robust wrt highly dynamic web sites or - worse - in-depth changes to the HTML implemented over time server-side.<p>What all of these tools actually need is to be bundled with some AI-powered recognition tools - computer vision, semantic understanding, etc ... - that can sort of "understand" the web page served before interacting with it.<p>All people do right now is a combo of XPATH-based search for an element in the DOM, or if that fails some sort of regex based pattern matching on the HTML.<p>These are weak and brittle techniques that are pretty much guaranteed to fail at some point.
Can someone provide an in depth comparison between Cypress and Playwright? It seems that both of them could be good choice compared to Selenium.<p>For example I keep seeing auto-wait being talked about for Playwright but doesn't cypress have that too? It seems playwright has better browser support and multi-window/concurrency story and cypress have better built-in tools (like a test-runner UI).<p>Basically looking for recommendation for test-automation for an old complex app mostly written using server-side tech but has potential for react and such moving forward.
Using TLS client certificates automatically is still not possible in Playwright [1]. That's our main reason why we keep using Selenium (with pytest-selenium) and Selenium Grid [2].<p>[1] <a href="https://github.com/microsoft/playwright/issues/1799" rel="nofollow">https://github.com/microsoft/playwright/issues/1799</a><p>[2] <a href="https://www.selenium.dev/documentation/grid/" rel="nofollow">https://www.selenium.dev/documentation/grid/</a>
Actually what is stopping you from generating automation selenium script by recording browser action? I recently wrote a tiny browser extension to do just that. Intercept event and translate the action into a selenium script. Currently the supported features are few. But would love to learn what am I expecting down the path.
I find that Hero[<a href="https://ulixee.org/docs/hero" rel="nofollow">https://ulixee.org/docs/hero</a>] has been a good replacement for my selenium needs
What’s the best way of monitoring suites? The built in report tool is amazing, but would love a solution that makes for easier analysis and maybe a closer integration with jira.
Beyond perhaps a smoke/availability test, I don’t know why you’d perform UI testing. Too brittle, too slow, too much of a pain to maintain compared to the value.