AutoChain, lightweight and testable alternative to LangChain

AutoChain takes inspiration from LangChain and AutoGPT and aims to solve both problems by providing a lightweight and extensible framework for developers to build their own agents using LLMs with custom tools and automatically evaluating different user scenarios with simulated conversations. Experienced user of LangChain would find AutoChain is easy to navigate since they share similar but simpler concepts.<p>The goal is to enable rapid iteration on generative agents, both by simplifying agent customization and evaluation.<p>If you have any questions, please feel free to reach out to Yi Lu yi.lu@forethought.ai

Why do people need LangChain or AutoChain? I don’t really get it in the first place. Genuinely want to get more information.<p>It seems to me that this can be easily added ad-hoc by programmers in their projects. But the biggest reason I see may be standardization of some kind? For interoperable building blocks maybe

I think a lot of these libraries are made by people who don&#x27;t really design systems.<p>At its core, what chatgpt and the like really are, are ways to transform natural language into structured data and vice versa. All you really need is an interface that allows you to plug in these APIs to _normal_ workflow engines that lots of companies are using every day to do work, both fully automated workflow engines like airflow or whatever, and also just plugging them into the _exact same systems_ that human engineers use -- it&#x27;s perfectly possible to get chatgpt to use Jira and submit PRs or whatever.

How about haystack? It exists for long and much better architectured than langchain

One of the valuable things AutoChain provides is simulated conversation evaluation. Testing and tuning those agent is very hard and expensive. checkout the evaluation output from AutoChain, for example.<p>&quot;conversation_history&quot;:<p><pre><code> &quot;user: I want to change the shipping address for my order&quot;, &quot;assistant: Can you please provide the order ID?&quot;, &quot;user: The order ID is 456&quot;, &quot;assistant: The order with ID 456 has not yet shipped. I can update the shipping address for you. Please provide the new shipping address.&quot;, &quot;user: The new shipping address is 234 Spear St, San Francisco&quot;, &quot;assistant: The shipping address for your order with ID 456 has been successfully updated to 234 Spear St, San Francisco.&quot;, &quot;user: Thank you for updating the shipping address for my order&quot;, &quot;assistant: Thank your for contacting&quot; &quot;is_agent_helpful&quot;: { &quot;rating&quot;: 5, &quot;reason&quot;: &quot;The conversation reaches the expected outcome. The user successfully provides the order ID and the new shipping address, and the assistant updates the shipping address for the order with ID 456. The assistant confirms the successful update and thanks the user for contacting.&quot; }, &quot;actions_took&quot;: { &quot;tool&quot;: &quot;check_order_status&quot;, &quot;tool_input&quot;: { &quot;order_id&quot;: &quot;456&quot; }, &quot;tool_output&quot;: { &quot;status_code&quot;: 200, &quot;order_id&quot;: &quot;456&quot;, &quot;order_status&quot;: &quot;not_shipped&quot;, &quot;tracking_url&quot;: &quot;example.com&#x2F;456&quot;, &quot;shipping_address&quot;: &quot;301 ivy street san francisco ca&quot; } }, { &quot;tool&quot;: &quot;change_shipping_address&quot;, &quot;tool_input&quot;: { &quot;order_id&quot;: &quot;456&quot;, &quot;new_address&quot;: &quot;234 Spear St, San Francisco&quot; }, &quot;tool_output&quot;: { &quot;status_code&quot;: 200, &quot;order_id&quot;: &quot;456&quot;, &quot;shipping_address&quot;: &quot;234 Spear St, San Francisco&quot; } } &quot;num_turns&quot;: 8, &quot;expected_outcome&quot;: &quot;found order status and changed shipping address&quot;</code></pre>

I’d really like libraries to put the language they’re for in the description.<p>Especially anything AI-related seems to just assume everybody uses Python.