Curtly vs traditional compression architectures
Compare latency, token savings efficiency, and syntax reliability against LLMLingua, secondary LLM summarizers, and manual prompt engineering.
< 15ms total proxy overhead vs 300ms+ local inference.
Zero dedicated GPU requirements or infrastructure management.
Dual-stage architecture covers 5x output generation prices.
The production engineering challenge
Existing compression approaches like LLMLingua require running secondary local neural models with 150-500ms latency overhead, while basic string cleaners destroy code syntax and lack output generation governance.
Technical architecture & mechanics
Zero GPU/LLM dependency for compression
Executes via optimized in-memory Rust and TypeScript token engines, avoiding the high compute costs and latency of secondary models.
Integrated dual-stage pipeline
Addresses both input token context and output generation rates in a single unified proxy gateway.
Edge streaming proxy infrastructure
Deployed globally on sub-15ms edge runtimes with zero cold start delays.
Compare Gateway Response Headers
Architecture comparison
| Evaluation metric | Curtly.dev gateway | Traditional / naive approach |
|---|---|---|
| Architecture Model | In-memory deterministic edge proxy | Heavy secondary neural model (LLMLingua) |
| Proxy Latency | < 15ms end-to-end | 150ms - 500ms processing delay |
| Output Token Governance | Yes (Dual-stage 30-50% reduction) | No (Input only) |
| Zero Code Changes | Yes (1-line baseURL proxy swap) | No (Requires custom SDK wrapper & Python environment) |
Frequently asked questions
Explore related platform features
Ready to optimize your AI infrastructure?
Start optimizing in 60 seconds with a 1-line baseURL change. Free tier available with zero credit card required.