>We launched our Pay Later Program at the end of December 2021. As we usually do with new features, we A/B tested a few different variants, with the intent of increasing enrollment as much as possible each time.<p>Another example to add to the annals of A/B horror stories. Sample size too small, test length time too short, extremely relevant KPIs like chargebacks or fraud complaints not included.<p>A/B the background color of the home page all you want, but A/B testing legal disclaimers is going to be, uh, risky.<p>And this is a very tired and boring thing to say, but startups are not AAAMM. If you're on a product team at Meta you can do ten A/B tests a week because tens of millions of people are using your product. If you're an early stage startup looking for market fit, a A/B test really will authentically take weeks or months to get enough data. If you're on a short runway I could imagine it would be painful and unpleasant to just sit on your hands the whole time, but otherwise you wander down blind alleys like the above.
The fact that a service exists to make you better at interviews implies that interviews don't evaluate competency. If interviews evaluated competency then the way to get better at interviews would be the same as the way to get better at software development (books, projects, etc.). I feel like improving my competency is worthwhile but improving my interviewing is rent seeking.
I've never heard of this organisation, and maybe it's because I don't work in tech but I don't really understand why they need to exist. But in any case, I'm entirely unsurprised to see the customer response to these changes. It comes off as incredibly predatory - AB testing the perfect, most "frictionless" experience which is obviously going to lead people to signing up without understanding they're committing to hundreds of dollars of future expenses. Gross.
I appreciate the candor that Aline has shown in this write up.<p>>Practically speaking, I’ve learned that creating some amount of friction is necessary when you’re asking people to promise to pay you ~$1000 in the future. Removing that friction can create short-term wins but may hurt you (and disappoint your users) in the long run.<p>But people are going to be confused from a "click this button and you owe us a grand" UI seems blindingly obvious. I suspect they got greedy and thought it wouldn't going to be as big of a deal as it turned out to be.
> Then hiring basically froze when COVID-19 happened, and to survive, we started charging engineers<p>Err, this doesn’t line up with any of my experience, nor that of any of my peers or the multiple employers I worked for throughout the pandemic. Maybe things slowed down for a month or so, but there was so much job mobility and choice it was bonkers!
The screenshots of the deal are in gray-on-white text, which looks like the writer doesn't want the reader to read it.<p>Additionally, they wrap up the gray/small print with the final word, on what the reader is supposed to think, in boldface, and delimited by triple asterisks: "94% of our users find a job within 4 months of starting to practice, and of those, the majority get offers from FAANG".<p>There seems to be a conscious component of persuasion in that, so I think no surprise if some customers later felt they'd been misled or manipulated.
I'm really amazed they got a 90% collection rate...<p>If you'd have asked me to guess, I would reckon only 30% will pay.<p>A chunk will have used credit cards with no balance, or cards about to expire. A chunk will dispute charges. A chunk will never see your emails because they land in spam.
I wonder how many of these users thought "It's just a web page, it's not going to send around a debt collector, I won't ever need to pay".<p>Those types of users tend to be put off by PDF documents you have to print, sign and return, because then they feel it's more likely that they will get a court summons for not paying...