AcruxCore

AcruxCore is an open-source LLMOps platform that helps engineering teams build, manage, monitor, and improve production AI applications from one unified control plane. Instead of combining separate products for prompt management, model routing, tracing, tool calling, cost controls, and evaluation, teams can manage the complete LLM lifecycle through AcruxCore. ## What can you do with AcruxCore? * **Version prompts:** Create immutable prompt versions, compare changes, and preserve a complete history of every update. * **Update without redeploying:** Move production or staging aliases between prompt versions. Applications receive the selected version without requiring a new deployment. * **Build dynamic templates:** Use variables, conditionals, loops, and filters through Jinja-style prompt templating. * **Route model requests:** Send LLM calls through an OpenAI-compatible gateway connected to your preferred providers. * **Bring your own keys:** Continue using your existing model-provider accounts and API keys. * **Configure fallbacks:** Automatically move requests to another model if the preferred option becomes unavailable. * **Reduce costs with caching:** Reuse eligible responses for identical requests instead of paying for repeated model calls. * **Control usage:** Apply budgets, virtual keys, and RPM or TPM rate limits before requests reach the model provider. * **Trace every call:** Inspect models, tokens, latency, cost, inputs, outputs, and individual spans from one dashboard. * **Replay agent sessions:** Follow a complete multi-step agent run and identify precisely where an error occurred. * **Manage tools:** Create reusable, versioned tools, attach them to prompts, execute them through the gateway, and analyze their performance. * **Evaluate improvements:** Test prompt versions and models against datasets before promoting changes to production. * **Monitor live quality:** Use rule-based evaluations to score matching production traces and reveal recurring problems. * **Improve from feedback:** Move from a failed response to its trace and exact prompt version, draft fixes, test candidates, and promote the winner. * **Use flexible integrations:** Connect through TypeScript, Python, REST, an OpenAI-compatible endpoint, or OpenTelemetry. * **Self-host everything:** Run the API, gateway, dashboard, and related services on your own infrastructure. ## Why is AcruxCore different? * **It operates in the request path:** Many observability tools record information only after an application has contacted the provider. AcruxCore can route, cache, limit, and trace the request before and during execution. * **It unifies the LLM workflow:** Prompt management, gateway routing, observability, tools, evaluation, and feedback-driven improvement are available in one platform. * **It provides a dedicated Tool Catalog:** Many alternatives only store temporary playground schemas or display tool calls as traces. AcruxCore treats tools as persistent, versioned, measurable assets. * **It supports a complete improvement loop:** Teams can identify a poor response, inspect the trace, find the responsible prompt, create a revision, evaluate it, and move the successful version into production. * **It supports advanced prompt logic:** Unlike platforms limited to basic variable replacement, AcruxCore supports conditional and loop-based prompt templates. * **It traces gateway calls automatically:** Developers do not need to manually wrap every provider request to capture essential operational data. * **It delivers competitive gateway performance:** AcruxCore’s published hands-on tests measured approximately 4–51 ms of gateway overhead across 15 runs, compared with 135–225 ms for MLflow’s gateway in the same test series. * **It is genuinely open source:** AcruxCore uses the permissive Apache License 2.0, and its API, gateway, dashboard, and SDKs are publicly available without a separate enterprise-only directory. * **It supports self-hosting:** Organizations can retain greater control over their provider keys, data, network, and deployment environment. * **It is free during public beta:** Teams can get started using their own provider keys without a credit card or temporary product trial. ## Who should use AcruxCore? * Engineering teams building production LLM applications * AI startups that want an integrated LLMOps stack * Developers building agents and tool-calling workflows * Teams comparing model quality, latency, usage, and cost * Organizations seeking open-source and self-hosted infrastructure * Developers who want to improve prompts without redeploying code AcruxCore shortens the journey from discovering a weak AI response to testing a better prompt and safely releasing that improvement into production.