Launch HN: MutableAI (YC W22) – Automatically clean Jupyter notebooks using AI

82 点作者 oshams超过 3 年前

Hi HN, I’m Omar the Founder and CEO of MutableAI (YC W22) (<a href="https://mutable.ai" rel="nofollow">https://mutable.ai</a>). We transform Jupyter notebook code into production-quality Python code using a combination of AI (OpenAI codex) and PL metaprogramming techniques.I'm obsessed with clean code because I've written so much terrible code in the past. I went from being a theoretical physics PhD dropout -> data scientist -> software engineer at Google -> research engineer at DeepMind -> ML engineer at Apple. In that time I've grown to tremendously value code quality. Clean code is not only more maintainable but also more extensible as you can more readily add new features. It even enables you to think thoughts that you may have never considered before.I want to reduce the cost of clean, production-quality code using AI, and am starting with a niche I'm intimately familiar with (Jupyter), because it's particularly prone to bad code. Jupyter notebooks are beloved by data scientists, but notorious for having spaghetti code that is low on readability, hard to maintain, and hard to move into a production codebase or even share with a colleague. That’s why a Kaggle Grandmaster shocked his audience and recommended that they do not use Jupyter notebooks [1].MutableAI allows developers to get the best of both worlds: Jupyter’s easy prototyping and visualization, plus greatly improved quality with our AI product. We also offer a full featured AI autocomplete to help prototyping go faster. I think the quadrant of "easy to develop in" and "easy to create high quality code" has been almost empty, and AI can help fill this gap.Right now there are two ways of manipulating programs: PL techniques for program analysis and transformation, and large scale transformers from OpenAI/DeepMind, which are trained on code treated as text (tokens) and don't look at the tree structure of code (ASTs). MutableAI combines OpenAI Codex / Copilot with traditional PL analysis (variable lifetimes, scopes, etc.) and statistical filters to identify AST transformations that, when successively applied, produce cleaner code.We use OpenAI's Codex to document and type the code, and for AI autocomplete. We use PL techniques to refactor the code (e.g. extract methods), remove zombie code, and normalize formatting (e.g. remove weird spacing). We use statistical filters to detect opportunities for refactoring, for example when a large grouping of variable lifetimes are suddenly created and destroyed, which can be an opportunity to extract a function.Some of the PL techniques are similar to traditional refactoring tools, but those tools don’t help you decide when and how to refactor. We use AI and stats to do that, as well as to generate names when the new code needs them.A tool that reduces the time to productionize code can be compared to having an extra engineer on staff. If you take this seriously, that’s a pretty big market. Stripe Research claims that developer inefficiency is a $300B problem [2]. Just about every tech company would become more efficient through increased velocity, fewer errors, and the ability to tackle more complex problems. It may even become unthinkable to write software without this sort of tool, the same way most people don't write assembly and use a compiler.You can try the product by visiting our website, <a href="https://mutable.ai" rel="nofollow">https://mutable.ai</a> and creating an account on the setup page <a href="https://mutable.ai/setup.html" rel="nofollow">https://mutable.ai/setup.html</a>. License keys are on the setup page once you’ve signed up (check your mailbox for an email verification link). I’ve bumped up the budget for free accounts temporarily for the day, I hope you enjoy the product !In addition to inviting the HN community to try out the product, I’d love it if you would share any tips for reducing code complexity you’ve come across and of course to hear your ideas about this problem and tools to address it.[1] <a href="https://youtu.be/tsGGpe-onZI?t=1067" rel="nofollow">https://youtu.be/tsGGpe-onZI?t=1067</a>[2] <a href="https://stripe.com/files/reports/the-developer-coefficient.pdf" rel="nofollow">https://stripe.com/files/reports/the-developer-coefficient.p...</a>

20 条评论

FridgeSeal超过 3 年前

Impressive product, but honestly, how is this the solution we’ve come to?“Should we get our data scientists to make the merest amount of effort to stop writing bad code?”“No, we’ll use an AI to fix their code”Just what? Notebooks are nice for exploratory code, but that’s it. Got problems with data scientists writing terrible code in notebooks? Make them stop then. Stop accepting PR’s, stop letting them do it.I’m not saying this as an outsider, I’m a data scientist by trade too, I don’t think we should be enabling bad practices and habits that we wouldn’t tolerate anywhere else in software development.

评论 #30461578 未加载

version_five超过 3 年前

This looks cool*I can't tell from the website, but what I would absolutely love is if this could turn the notebook into a "train.py" with argparse to parse the arguments and model / program output saved on completion. This is a definite pain point for me, there are things I'd like to test interactively, but I default to trying to get out of a jupyter notebook as quickly as possible because all the code ends up having to be rewritten "properly".For ML in particular, I bet there are cool integrations you could do with pytorch lightning (and other frameworks) to take me from trying a forward pass in the jupyter notebook to having the datamodel with dataloaders declared and then set up in a main() . There is lots that could be automated.*(except the SaaS model, I have a really hard time supporting "api call" limited services that seem to just be charging me a subscription fee for the priviledge of running code, but I don't want to get into that.)

评论 #30459402 未加载

hervature超过 3 年前

I think bad code in Jupyter is a symptom of both the tool and the user. Jupyter's flexibility for running cells out of order should never have been allowed. Code developed in this way will not be able to be transformed into a sequential program. One example of this is illustrated in your demo, how did the algorithm decide input_shape was an input but num_classes should remain a global? For me, it is obvious that the user's intent was that these are the variables that define the model.Personally, I wouldn't pay those prices to hide the data scientist's problems. However, if you developed extensions to prevent and encourage the user to overcome the tool's problems, I would buy that for myself.Also, you probably don't a demo that would fail to run in the second cell (in both unclean and clean code) .

评论 #30459211 未加载

评论 #30458970 未加载

writablemkv超过 3 年前

I own a paid license for MutableAI tool.A few notes:1. The progress that was made by Omar and the team (from alpha to working version that is useful) is astounding. I am eagerly waiting for more improvements that will make my regular daily workflow even more fun.2. The tool is useful for both junior and senior developers and data scientists. They get different things from it though: junior developers get cleaner code, fixes for simple mistakes, structure and demonstration of how things “should be done”. Senior developers get to skip mundane tasks, go from prototype to production code quicker and can concentrate on more complex things/details.3. I especially like that there is an option for “on prem” installation. My current employer is very strict about using “online” tools that can leak our code outside. “cloud only” tool would be a deal breaker for us.4. Considering the amount of time this tool can potentially safe, it will pay for itself in no time. I am mean really – if it saves me just a few minutes of mindless reformatting/editing/adding comments – it already paid for itself.

edublancas超过 3 年前

Congrats on the launch! As a former data scientist, it pumps me up to see more notebook-centric tooling, as I believe it is the best environment for data exploration and rapid iterations.We're working on notebook tooling as well (<a href="https://github.com/ploomber/ploomber" rel="nofollow">https://github.com/ploomber/ploomber</a>), but our focus is at the macro level so to speak (how to develop projects that are made up of several notebooks). In recent conversations with data teams, the question "how do you ensure the code quality of each notebook?" has come up a lot, and it's great to see you are tackling that problem. It'll be exciting to see people using both MutableAI and Ploomber! Very cool stuff! I'll give it a try!

评论 #30460589 未加载

elmalto超过 3 年前

I get some of the sentiment that people should write clean code and I have been very adamant with my teams to ideally use best practices and make their code (re)usable and production ready. But the truth is: nobody likes to clean up. Including me. To use a real life example: I don’t use a broom at home unless my roomba is unable to get to the spot. This is not going to solve all problems and I will probably have to fix some edge cases, but I love the solution you have come up with and will definitely give it a shot next time I come across an ugly notebook!

评论 #30462547 未加载

chse_cake超过 3 年前

Not to gripe on your work, But I am extremely skeptical about tools like these. How do you ensure the equivalence of the computation ? What are the SLA's on equivalence and how do you even verify if the programs are equivalent?The solution to bad code written by data scientists is not more AI tools that write non-verifiable code. Its ergonomic API frameworks which can relieve the pain-points / bad practices via throughly testable / verifiable / deterministic code.The site even promises test-case generation in the future. This seems very flaky at best.

评论 #30460554 未加载

Darmani超过 3 年前

I saw a private demo of Mutable.ai last month. It was extremely impressive, and even more impressive was the pace of development. This is a clear must for anyone doing Jupyter.

评论 #30465252 未加载

tpoacher大约 3 年前

Nice project.But the fact you need AI to convert a 'notebook' into 'proper code' shows you just how bad jupyter notebooks are.(I don't mean 'notebooks' in general. I mean 'jupyter notebooks' specifically.)It used to be that programmers were taught that "your main function should read like a table of contents, and delegate details appropriately". Now it's "Y U NO FIT EVRYTHIN IN ONE PAEG?"

p1esk超过 3 年前

I don't use Jupyter, can you take my sometimes ugly, research-grade Pytorch code, and turn it into a clean, production quality code?

评论 #30459852 未加载

rbmattis超过 3 年前

How do you think about running on-prem versus in the cloud? I can see some users preferring the ease and simplicity of using MutableAI in the cloud, but I can see larger corporations preferring to keep all code in-house. Have you thought about that?

评论 #30459837 未加载

ricklamers超过 3 年前

This is awesome! Notebook centric static analysis is definitely going to help data scientists level up their game. Tools like this help you improve your coding style simply by observing the changes that are proposed by it.We have customers running batch data pipelines in production that contain notebooks. It’s because of the first class integration of notebooks in the orchestration tool we’ve built (<a href="https://github.com/orchest/orchest" rel="nofollow">https://github.com/orchest/orchest</a>). It would be great to help our users catch errors before they bite them in prod. Let’s have a chat sometime :-)!

评论 #30461899 未加载

mhartl超过 3 年前

I’ve been following this project closely (oshams is a friend and I’m an investor in the company), and to echo Darmani’s comment both the product and the pace of development have been extremely impressive. Congrats on the launch!

forgotmyoldacc超过 3 年前

The documentation in the screenshot seems pretty meaningless, they're already being described by the function name and parameters.Are there more examples that we can see?

评论 #30463015 未加载

bravura超过 3 年前

Can you explain, in your pricing plan, what is an "API call"? It isn't very clear from the landing page what that entails.

评论 #30460043 未加载

morganslaw超过 3 年前

Is this just prettier (<a href="https://prettier.io/" rel="nofollow">https://prettier.io/</a>) for Jupyter combined with a bit of hoisting?

评论 #30459647 未加载

hallqv超过 3 年前

Excellent, much neeeded product!

评论 #30461095 未加载

dmicah超过 3 年前

I think the real trick is to never use Jupyter / notebooks to begin with.

评论 #30460879 未加载

fudged71大约 3 年前

Awesome demo, congrats on the launch!

Ozzie-D大约 3 年前

Good luck!