TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Launch HN: Reflect (YC S20) – No-code test automation for web apps

262 pointsby tmcnealalmost 5 years ago
We&#x27;re Fitz and Todd, co-founders of Reflect (<a href="https:&#x2F;&#x2F;reflect.run" rel="nofollow">https:&#x2F;&#x2F;reflect.run</a>) - we’re excited to share our no-code tool for automated web testing.<p>We worked together for 5+ years at a tech start-up that deployed multiple times a week. After every deployment, a bunch of our developers would manually smoke test the application by running through all of the critical user experiences. This manual testing was expensive in terms of our time. To speed up the tests’ run time, we dedicated developer resources to writing and managing Selenium scripts. That was expensive at “compile time” due to authoring and maintenance. At a high-level, we believe that the problem with automated end-to-end testing comes down to two things: tests are too hard to create, and they take too much time to maintain. These are the two issues we are trying to solve with Reflect.<p>Reflect lets you create end-to-end tests just by using your web application, and then executes that test definition whenever you want: on a schedule, via API trigger or simply on-demand. It emails you whenever the test fails and provides a video and the browser logs of the execution.<p>One knee-jerk reaction we’re well aware of: record-and-playback testing tools, where the user creates a test automatically by interacting with their web application, have traditionally not worked very well. We’re taking a new approach by loading the site-under-test inside of a VM in the cloud rather than rely on a locally installed browser extension. This eliminates a class of recording errors due to existing cookies, proxies or other extensions introducing state that the test executor is not aware of, and unifies the test creation environment with the test execution environment. By completely controlling the test environment we can also expose a better UX for certain actions. For example, to do visual testing you just click-and-drag over the element(s) you want to test. For recording file uploads we intercept the upload request in the VM, prompt you to upload a file from your local file system, and then store that file in the cloud and inject it into the running test. If you want to add additional steps to an existing test, we’ll put you back into the recording experience and fast-forward you to that point in the test, where again all you need to do is use your site and we’ll add those actions to your existing test. Controlling the environment also allows us to reduce the problem space by blocking actions which you typically wouldn’t want to test, but which are hard to replicate and thus could lead to failed recordings (e.g. changing browser dimensions mid-recording). As an added bonus, our approach requires no installation whatsoever!<p>We capture nearly every browser action from hovers to file uploads, and drag-and-drops to iframes, while building a repeatable, machine-executable test definition. We support variables for dynamic inputs, and test composition so your test suite is DRY. The API provides flexible integration with your CI&#x2F;CD out of the box and supports creating tests in prod and running them in staging on the fly. You don’t need to use a separate test grid, as all Reflect tests run on our own infrastructure. Parallel execution of your tests is a two click config change and we don’t charge you extra for it.<p>Some technical details that folks might find interesting:<p>- For every action you take we’ll generate multiple selectors targeting the element you interacted with. We wrote a custom algorithm that generates a diverse set of selectors (so that if you delete a class in the future your tests won’t break), and ranks them by specificity (i.e. [data-test-id] &gt; img[alt=”foo”] &gt; #bar &gt; .baz).<p>- To detect certain actions we have to track DOM mutations across async boundaries. So for example we can detect if a hover ended up mutating an element you clicked on and thus should be captured as a test step, even if the hover occurred within a requestAnimationFrame, XHR&#x2F;fetch callback, setTimeout&#x2F;setInterval, etc.<p>- We detect and ignore auto-genned classes from libraries like Styled Components. We use a heuristic to do this so it’s not perfect, but this approach allows us to generate higher quality selectors than if we didn’t ignore them.<p>- One feature in beta that we’re really excited about: For React apps we have the ability to target React component names as if they were DOM elements (e.g. if you click on a button you might get a selector like “&lt;NotificationPopupMenu&gt; button”). We think this is the best solution for the auto-genned classes problem described in the bullet above, as selectors containing component names should be very stable.<p>We tried to make Reflect as easy as possible to get started with - we have a free tier (no credit card required) and there’s nothing to install. Our paid plans start at $99 and you pay primarily for test execution time. Thanks and we look forward to your feedback!

35 comments

cmehdyalmost 5 years ago
Interesting tool! And you&#x27;ve definition given a lot of thought to many of the problems encountered while attempting to do UI testing automation.<p>I&#x27;ve written my fair share of extensive selenium stuff (and appium, but.. let&#x27;s forget about those painful memories) and one thing that I found fairly &quot;easy&#x27; to add to the suites I had written was accessibility testing (using Deque&#x27;s Axe[0] tools). It literally took a few lines of code and a flag at runtime to enable the accessibility report on my Jenkins&#x2F;TestNG&#x2F;Selenium combo. However WCAG is constantly changing and it&#x27;s hard to keep up, and even Deque is not always up to date AFAIK. Do you have plans to have accessibility testing with your tool ? (even a subset of WCAG&#x27;s rules)<p>Another thing I&#x27;ve noticed is the jump in pricing from Free to 100 USD&#x2F;month which goes from 30mins to 6hours. This might be steep for a team attempting to test out the validity of the tool against the competition - perhaps offering a first-month discounted trial or something like that would be appealing.<p>I also haven&#x27;t really seen if it is possible to enable concurrency (for example, is testing on three platforms at the same time for 10 minutes on that free tier possible? I would imagine so if one is doing the lifting with CI like Jenkins - but perhaps you have your own solutions). Tangentially related but you say your tests integrate with various CI solutions, does this mean one can extract the test results in a way that allows further processing and integration into other tools? (I&#x27;m thinking of the XMLs coming out of TestNG there).<p>Lastly, I don&#x27;t know if the time used is counted from the moment you start your VMs or browser starts, or the page is loaded, or the first step is done or something else. Clarifying that might help with a team&#x27;s estimates (I had internal tests where the struggle was entirely on having the virtual environment&#x2F;device&#x2F;browser ready and the test was then a breeze, so the significant metric was the boot-up time).<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;dequelabs&#x2F;axe-core" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dequelabs&#x2F;axe-core</a>
评论 #23900482 未加载
shay_keralmost 5 years ago
&gt; One feature in beta that we’re really excited about: For React apps we have the ability to target React component names as if they were DOM elements (e.g. if you click on a button you might get a selector like “&lt;NotificationPopupMenu&gt; button”). We think this is the best solution for the auto-genned classes problem described in the bullet above, as selectors containing component names should be very stable.<p>I remember looking into this exact idea. Theoretically, if you&#x27;re able to capture the React state, and you&#x27;re working with a &quot;pure&quot; React app, you should be able to auto-gen readable tests from human interaction. And, if you&#x27;re capturing the state in a granular enough fashion, you should be able to &quot;time-travel&quot;, but for non-technical users.<p>IMO the biggest use case for E2E tests are for critical things like auth &amp; checkout. If you&#x27;re able to auto-gen, maybe you can get even deeper than that.<p>Congrats on the launch, it looks cool!
评论 #23900442 未加载
yeldarbalmost 5 years ago
We are using Reflect.run at Roboflow and it really is very slick. We have it testing our core flow on a schedule so we know if any of our 3rd party services or dependencies go down and&#x2F;or if we introduce a regression.<p>Our app&#x27;s core workflow involves signing in, uploading images and annotation files, parsing those files, using HTML5 canvas to render previews and thumbnails, uploading files to GCS, and kicking off several async functions and ajax requests.<p>That Reflect.run was able to handle all of those complex operations makes me pretty confident it can effectively test any web app.
评论 #23904230 未加载
评论 #23899781 未加载
karussellalmost 5 years ago
Really nice tool, thanks! I&#x27;m more a backend developer so I might have some stupid questions: What is your competition and what are you doing better?<p>Update: reading through this thread and a bit web search resulted in the following list:<p><a href="https:&#x2F;&#x2F;ghostinspector.com" rel="nofollow">https:&#x2F;&#x2F;ghostinspector.com</a> <a href="https:&#x2F;&#x2F;www.testim.io" rel="nofollow">https:&#x2F;&#x2F;www.testim.io</a> <a href="https:&#x2F;&#x2F;www.katalon.com" rel="nofollow">https:&#x2F;&#x2F;www.katalon.com</a> <a href="https:&#x2F;&#x2F;www.cypress.io" rel="nofollow">https:&#x2F;&#x2F;www.cypress.io</a> <a href="https:&#x2F;&#x2F;github.com&#x2F;dequelabs&#x2F;axe-core" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dequelabs&#x2F;axe-core</a> <a href="https:&#x2F;&#x2F;www.mabl.com" rel="nofollow">https:&#x2F;&#x2F;www.mabl.com</a> <a href="https:&#x2F;&#x2F;preflight.com" rel="nofollow">https:&#x2F;&#x2F;preflight.com</a><p>&gt; We’re taking a new approach by loading the site-under-test inside of a VM in the cloud rather than rely on a locally installed browser extension.<p>But how would I record and test things locally? I.e. I would need a public setup right at the beginning.
评论 #23900929 未加载
amw-zeroalmost 5 years ago
I&#x27;ve largely moved away from UI testing. The two main reasons are:<p>1) Flakiness &#x2F; non-determinism 2) The constant change of the UI<p>Both of these are absolute killers to productivity on a large team. Note, I&#x27;m not against UI testing in theory. I think if you could, you would have full end to end tests for every single edge case so that you could be sure that the actual product presented to users work. But in practice, end to end testing an asynchronous distributed system (which a simple client-server application still is) is full of non-determinism.<p>Re the constant changing of the UI. This is just also true in my experience. I&#x27;ve worked on a navigation &#x2F; global UI redesign at every company I&#x27;ve ever worked at. It happens like once every 3-5 years. Within the redesigns, it&#x27;s still extremely common to subtly change the UI for UX reasons all the time. When this happens, be prepared to spend half of the time updating all of your tests.
评论 #23900305 未加载
评论 #23900651 未加载
评论 #23906646 未加载
lohalmost 5 years ago
Congrats on launching Reflect. Looks solid!<p>I soft launched something eerily similar ~6 months ago and got zero feedback (probably because no one was able to actually try it since Google took forever to approve the extension).<p>TestFront.io: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22130590" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22130590</a><p>Ran out of money though and had to pursue other things so I put it on hold. Maybe I&#x27;ll resume work on TestFront at some point and you&#x27;ll have some competition. ;)
mayankalmost 5 years ago
&gt; We’re taking a new approach by loading the site-under-test inside of a VM in the cloud rather than rely on a locally installed browser extension.<p>So while you do get the benefit of clean, repeatable tests, this requirement also puts you out of reach for most enterprise-y applications, where your pre-production environments (i.e., what you actually want to run your tests against) exist in a tightly locked down network environment that you can&#x27;t reach from the outside. In stricter environments, even a reverse port forwarding setup to allow you to reach inside the LAN would be out of the question.<p>I think this is a problem you&#x27;re going to run into pretty soon if you want to do enterprise sales, and one that is not trivial to solve without an open-source flavor of the test runner (e.g. like Selenium).
评论 #23903557 未加载
rococodealmost 5 years ago
This is super cool! I&#x27;m at a startup and we&#x27;ve finally started building up our testing infrastructure lately, this seems like a great time-saver. Just tried it out and it worked great. Two questions:<p>1) Is it possible to set the screen resolution? I see device profiles but don&#x27;t see if it&#x27;s possible to manually create profiles. The default Desktop res is lower than what we normally target.<p>2) How exactly does the visual testing work? I tried dragging over a div and it got the text, but parts of the text were formatted differently with spans. So does it just validate the text, or does it actually compare the visuals?
评论 #23903496 未加载
catchmeifyoucanalmost 5 years ago
One thing I like about this is that it supports drag and drop support. That&#x27;s an item that I haven&#x27;t seen to be super straight-forward with other suites like Cypress and is very-much a user initiated action.
chris_stalmost 5 years ago
For selecting things, I&#x27;ve found that allowing a &quot;data-test=&#x27;user-name-input&#x27;&quot; type attribute is useful for a lot of cases, when doing Gauge&#x2F;Taiko tests.<p>It might make sense to allow&#x2F;recognize these to save trying to find things that way, rather than via CSS-type selectors that may change.
评论 #23898220 未加载
评论 #23903773 未加载
jaequeryalmost 5 years ago
How does this compare with qawolf.com which uses Playwright, open-source and free?
p17almost 5 years ago
I want to use Reflect to test my game with many concurrent, simulated players. For context, the game needs to properly handle 50 players at once.<p>The pricing (6 hours of testing for $99) means that if I want to do a 1 minute test with 50 concurrent players, I can only test 6 times a month. A big benefit of testing is to ensure we ship reliable software, and we plan to ship much more often than 6 times a month.<p>Is there a way you could price by unique tests instead of hours tested?
评论 #23898404 未加载
idreynalmost 5 years ago
I maintain a web app that would really benefit from an E2E suite, but we don&#x27;t have the developer capacity to write one right now, so this looks like it hits a potential sweet spot for me. To use Reflect, I think we would need to understand the plan for what happens if the SaaS goes under -- ideally, we&#x27;d be left holding our test definitions and some open-source version of the test runner that we can instantiate on our own VM.
评论 #23899826 未加载
评论 #23900356 未加载
评论 #23917605 未加载
futheyalmost 5 years ago
This looks amazing. First test went incredibly well, easy to set up. Enjoying the VM approach vs. extension.<p>Will monitor this to see how our tests perform over time.
mritchie712almost 5 years ago
How do you handle logging in to the app that needs to be tested? We use Google OAuth.
评论 #23898173 未加载
评论 #23900419 未加载
评论 #23898184 未加载
start123almost 5 years ago
The registration is buggy - The form is misaligned - at least on Firefox, and why does Google sign up say &quot;continue to amazoncognito.com&quot;?
评论 #23902832 未加载
评论 #23900762 未加载
tribeca18almost 5 years ago
Makes me think of <a href="http:&#x2F;&#x2F;waldo.io" rel="nofollow">http:&#x2F;&#x2F;waldo.io</a>, but for web apps! Really excited about the new wave of no-code test automation tools - definitely helps semi-technical team members take on more responsibility on the testing side vs. just writing specs.
评论 #23898905 未加载
mchusmaalmost 5 years ago
How does this compare to recent YC alum Preflight?
评论 #23897972 未加载
评论 #23897957 未加载
mmckelvyalmost 5 years ago
Looks great. How much easier is this than say, writing E2E tests with Cypress?
评论 #23900377 未加载
评论 #23899568 未加载
UweSchmidtalmost 5 years ago
The true test for such a tool are usually the edge cases; in my opinion the web is simply too finnicky, and all those well-meaning custom algorithms I&#x27;ve tried fell way short.<p>I would recommend Ranorex, which combines a comfortable record-replay functionality (which creates real C# code) and a code-first approach and everything in-between. A powerful &quot;spy&quot; assists in creating the most sophisticated locators and turns them into first-class Ranorex objects; a shortcut jumps from code to the repository and back; duplicate elements are detected on the fly.
IAmNotAFixalmost 5 years ago
This assumes the URL is publicly available, or is there an on-premise offer?
评论 #23898845 未加载
chrisweeklyalmost 5 years ago
Wow, this looks really useful and well-thought out. Also, kudos for the compelling summary writeup &#x2F; pitch here. Bookmarked, will def be giving this a try.
评论 #23903093 未加载
YPCrumblealmost 5 years ago
Similar product I’ve had a great experience with is <a href="https:&#x2F;&#x2F;ghostinspector.com" rel="nofollow">https:&#x2F;&#x2F;ghostinspector.com</a>
评论 #23900714 未加载
o_____________oalmost 5 years ago
Took a look, nice job!<p>Would be nice:<p>1. Clearer setting for notification email<p>2. The ability to target an area for an element change without knowing what the elements will be. Example: filter for recent items without foreknowledge of which items will appear.<p>3. Ability to traverse up DOM (to select parent&#x2F;s) based on a selector that was too specific. Encountering this quite a bit.
评论 #23903104 未加载
jameslkalmost 5 years ago
How does this compare to other existing &quot;no code&quot; SaaS regression testing tools such as screenster.io?
foreigneralmost 5 years ago
Looks neato. Are tests versioned? I&#x27;d like to be able to see a textual diff to changes in tests over time.
评论 #23899184 未加载
Trufaalmost 5 years ago
Hey! Product looks great! My main question would be, for a rapidly evolving product would it generate a lot of false positives?<p>I really enjoy the idea of business&#x2F;marketing people collaborating with tests. Congrats.<p>I also really like your business model so will give it a try.
评论 #23898095 未加载
ggregoirealmost 5 years ago
Looks good. A small suggestion for the website: perhaps I missed it, but I watched both videos &quot;record a test&quot; and &quot;view test results&quot; on the front page and I didn&#x27;t see Reflect detects an actual regression.
评论 #23898970 未加载
catchmeifyoucanalmost 5 years ago
Are there any plans to support electron apps. I have a react-based Electron App, and also would love to test a few things that are relevant to the app loading and getting data from disk. Just another use case to consider.
评论 #23900531 未加载
d0malmost 5 years ago
Congrats on the launch, it looks awesome. I wished I had that a few years ago and will definitely give it a try
ck_onealmost 5 years ago
I assume it&#x27;s not possible to clean up after a test since you directly interact with the web app, right?
评论 #23898093 未加载
cbenincasaalmost 5 years ago
This is awesome, congrats guys!
gfodoralmost 5 years ago
How well would this work with a hybrid DOM&#x2F;WebGL application?
评论 #23898863 未加载
browsergapalmost 5 years ago
I also had the idea to run browsers in the cloud and intercept the interaction stream to record and replay interactions, also for the reasons you say, because it reduces a large class of bugs. But not for test automation, rather for web scraping. I applied to YC with this around 9 times. Great to see you convince them this is a good thing!<p>Are you using the Chrome DevTools protocol? That&#x27;s what I built on top of. You can see my open-source code for this here: <a href="https:&#x2F;&#x2F;github.com&#x2F;dosyago&#x2F;BrowserGap" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dosyago&#x2F;BrowserGap</a><p>And a demo running right now of the interactive cloud browsers (without automation, it&#x27;s WIP):<p><a href="https:&#x2F;&#x2F;start.cloudbrowser.xyz" rel="nofollow">https:&#x2F;&#x2F;start.cloudbrowser.xyz</a> [0]<p>[0]: Currently on Show <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=23904243" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=23904243</a><p>Fitz and Todd, wanna connect?
评论 #23906222 未加载
评论 #23906208 未加载
ingloralmost 5 years ago
Hey congrats, I work for Testim ( <a href="https:&#x2F;&#x2F;testim.io" rel="nofollow">https:&#x2F;&#x2F;testim.io</a> ) which I assume is somewhat competition?<p>Excited to see more players in this space - good luck! Most of the market is still doing manual QA and that has to change.