Why We Built ArchGW?<p>Traditional application architectures separate routing, security, and observability from business logic - so that developers can move faster without the tax of reinventing the wheel. LLM applications should be no different. ArchGW applies these patterns to prompts, providing a structured approach to building LLM applications.<p>How It Works<p>ArchGW runs as a separate process alongside application servers. It intercepts prompts before they reach the backend and applies transformations based on predefined rules and models:<p><pre><code> Preprocessing: Normalizes and analyzes prompt structure.
Security Filtering: Rejects jailbreak attempts and unsafe inputs.
Intent Mapping: Determines if a request maps to an API function.
Function Invocation: Extracts arguments and calls backend APIs.
LLM Routing: Chooses the right LLM provider based on latency/cost constraints.
Tracing & Metrics: Adds W3C Trace Context headers, tracks errors, token usage, and request latency.
</code></pre>
Why a Dedicated Proxy?<p>Traditional application architectures separate routing, security, and observability from business logic—LLM applications should be no different. ArchGW applies these patterns to prompt processing, providing a structured approach to LLM integration.