# Curtly Technical Reference and Architecture Guide

> Curtly is a high-performance in-memory LLM token optimization proxy and quota governor that cuts LLM token costs by 50-70% through dual-stage optimization: up to 60-90% input prompt compression via Safe Vault AST parsing, and 35-74% output efficiency injection via the Output Brevity Governor.

## 1. Complete Platform Subpages & URLs

### Benchmarks & Empirical Verifications
- [Live Benchmark Inspector](https://curtly.dev/benchmark): Interactive empirical 12-task benchmark suite, 5-family tokenizer matrix, 3-way OpenCode agent runs, and 2,283 standardized academic accuracy evaluations (HumanEval 89.0%, MBPP 82.3%, GSM8k 94.5%, SWE-bench Lite 32.7%, TypeScript AST 100.0%).
- [Dual-Stage Token Economics](https://curtly.dev/features/dual-stage-optimization): Comprehensive breakdown of why output tokens cost 3.5x to 5x more than input tokens, and how output savings account for 54.8% (single turn) to 85.7% (5-turn agent loop) of total dollar savings.
- [Benchmark Inspector Feature](https://curtly.dev/features/benchmark-inspector): Technical deep-dive into the live verification harness, execution assertions, and real-time AST testing.

### Financial ROI Calculators
- [Global Token ROI Calculator](https://curtly.dev/calculator): Real-time interactive financial savings simulator across 13 frontier models.
- [Claude Fable 5.1 ROI](https://curtly.dev/calculator/fable-5-1): Token compression analysis and cost modeling for Anthropic Claude Fable 5.1.
- [GPT 6 Astra ROI](https://curtly.dev/calculator/gpt-6-astra): Token compression analysis and cost modeling for OpenAI GPT 6 Astra.
- [Claude Opus 5 ROI](https://curtly.dev/calculator/opus-5): Token compression analysis and cost modeling for Anthropic Claude Opus 5.
- [DeepSeek R1 ROI](https://curtly.dev/calculator/deepseek-r1): Reasoning token compression analysis and cost modeling for DeepSeek R1.
- [Kimi K3 ROI](https://curtly.dev/calculator/kimi-k3): Token compression analysis and cost modeling for Moonshot AI Kimi K3.
- [GLM-5.3 ROI](https://curtly.dev/calculator/glm-5-3): Token compression analysis and cost modeling for Zhipu AI GLM-5.3.
- [Qwen 3.8 Max ROI](https://curtly.dev/calculator/qwen-3-8-max): Token compression analysis and cost modeling for Alibaba Qwen 3.8 Max.
- [Claude Mythos Preview ROI](https://curtly.dev/calculator/mythos-preview): Token compression analysis and cost modeling for Anthropic Claude Mythos Preview.
- [GPT-5.6 Sol ROI](https://curtly.dev/calculator/gpt-5-6-sol): Token compression analysis and cost modeling for OpenAI GPT-5.6 Sol.
- [Hunyuan Frontier ROI](https://curtly.dev/calculator/hunyuan-frontier): Token compression analysis and cost modeling for Tencent Hunyuan Frontier.
- [Muse Spark 1.3 ROI](https://curtly.dev/calculator/muse-spark-1-3): Token compression analysis and cost modeling for Meta Muse Spark 1.3.
- [GPT-5.6 Terra ROI](https://curtly.dev/calculator/gpt-5-6-terra): Token compression analysis and cost modeling for OpenAI GPT-5.6 Terra.
- [GPT-5.6 Luna ROI](https://curtly.dev/calculator/gpt-5-6-luna): Token compression analysis and cost modeling for OpenAI GPT-5.6 Luna.

### Feature Architecture Guides
- [Input Prompt Compression](https://curtly.dev/features/input-prompt-compression): In-memory AST comment pruner, RAG deduplication, and conversational filler stripping.
- [Output Efficiency Injection](https://curtly.dev/features/output-efficiency-injection): Output Brevity Governor cutting verbose greetings, apologies, and duplicate summaries.
- [Safe Vault AST Engine](https://curtly.dev/features/safe-vault): Abstract Syntax Tree isolation guaranteeing 0.00% syntax distortion on code, JSON, and SQL.
- [Zero Syntax Corruption](https://curtly.dev/features/zero-syntax-corruption): AST guarantees and verification tests ensuring valid compilation on all languages.
- [Architecture Comparison](https://curtly.dev/features/architecture-comparison): Comparison with naive regex compressors, vector store embeddings, and raw proxies.
- [Latency Governor](https://curtly.dev/features/latency-governor): Deterministic in-memory execution delivering 1.8ms - 5.3ms latency overhead (P95 <8ms).
- [Multi-Model Tokenizer](https://curtly.dev/features/multi-model-tokenizer): Exact native BPE counting for Claude BPE, o200k_base, DeepSeek BPE, Qwen BPE, and Gemini 256k.
- [ROI Calculator Engine](https://curtly.dev/features/roi-calculator): Formula specification for single-turn and multi-turn cached agent economics.

### Technical Documentation Sections
- [Documentation Hub](https://curtly.dev/docs): Technical reference for Curtly proxy endpoints and SDKs.
- [Introduction](https://curtly.dev/docs/introduction): Core system overview, token overhead economics, and pipeline guarantees.
- [Quickstart Guide](https://curtly.dev/docs/quickstart): 2-minute drop-in proxy setup for OpenAI SDK, Claude Code CLI, and OpenCode.
- [Compression Modes](https://curtly.dev/docs/compression-modes): Fast/Algo (deterministic AST), Smart (balanced neural + vault), and Aggressive modes.
- [Safe Vault Architecture](https://curtly.dev/docs/safe-vault): Technical reference on entity tokenization, hex locks, and AST restoration.
- [Compress Endpoint API Reference](https://curtly.dev/docs/compress-endpoint): HTTP request schemas, headers, response format, and error handling.
- [SDK & Framework Integrations](https://curtly.dev/docs/sdk-integrations): Code examples for Node.js, Python, LangChain, LlamaIndex, and Cursor.
- [Security & Compliance](https://curtly.dev/docs/security-compliance): Zero data retention, volatile RAM execution, TLS 1.3, and SHA-256 API key hashing.

### Application & Developer Portals
- [Developer Console / Dashboard](https://curtly.dev/dashboard): API key provisioning, request telemetry, latency graphs, and token savings metrics.
- [Authentication](https://curtly.dev/login): Secure magic link login.
- [Pricing Plans](https://curtly.dev/#pricing): Free ($0/mo, 500 req), Starter ($19/mo, 25k req), Growth ($29/mo, 100k req), Enterprise ($199/mo, 1M req).
- [Terms of Service](https://curtly.dev/terms): SLA terms and API usage rules.
- [Privacy Policy](https://curtly.dev/privacy): Zero-storage data handling commitments.
- [Refund Policy](https://curtly.dev/refund): 14-day money-back guarantee terms.
- [Machine Context Index](https://curtly.dev/llms.txt): Machine-readable index for LLMs and developer agents.

---

## 2. Executive Summary and Value Proposition

Modern Large Language Model (LLM) applications encounter severe operational costs when scaling production traffic. These costs stem from asymmetric token economics:
- Token pricing asymmetry: Generation (output) tokens cost 3x to 5x more than prompt (input) tokens across frontier foundation models (Anthropic Claude Fable 5.1, OpenAI GPT 6 Astra, Claude Opus 5, Kimi K3, GLM-5.3, Qwen 3.8 Max, and DeepSeek R1).
- Context bloat: Real-world prompts contain extensive scaffolding, including system role definitions, repetitive output formatting instructions, tool definitions, retrieval augmented generation (RAG) chunks, and chat history. Up to 60-90% of these tokens do not contribute semantic information to the generation process.
- Conversational output overhead: Frontier models naturally default to pleasantries, conversational acknowledgments, markdown restatements, and boilerplate disclaimers, inflating output token usage by 30-50%.
- AI Coding Agent loop overhead: Developer agents (OpenCode, Claude Code, Cursor) generate thousands of transient ANSI status characters, git diff index headers, multi-megabyte tool schemas, and duplicate source file passes per session.

Curtly provides an in-memory dual-engine governor that optimizes both ends of the LLM pipeline:
1. Input Layer: In-memory AST parsing, syntax-safe code comment stripping, and semantic compaction prune up to 60-90% of input tokens without modifying code or critical parameters.
2. Output Layer: Transparent injection of system-level output efficiency directives (Curtly Brevity Engine) instructs the upstream LLM to omit filler and stream dense, factual answers, slashing output token volume by 30-50%.

The combined effect yields a 50-70% net reduction in total inference expenditure with sub-2 millisecond latency overhead and zero syntax drift.

---

## 3. System Architecture

```
[ Client Application / Agent CLI (OpenCode, Claude Code, Cursor) ] 
       │
       ▼ (HTTPS / TLS 1.3)
[ Curtly In-Memory Gateway (Edge Proxy) ]
       │
       ├─► [ Step 1: Request Authentication & Quota Verification ]
       ├─► [ Step 2: SHA-256 Request Cache Lookup (0.01ms hit response) ]
       ├─► [ Step 3: ANSI Terminal & Progress Bar Filter ]
       ├─► [ Step 4: Syntax-Safe Code Comment & License Pruning (-72.7%) ]
       ├─► [ Step 5: Safe Vault AST Extraction (Code, SQL, JSON, URLs) ]
       ├─► [ Step 6: Tool JSON Schema Minification ]
       ├─► [ Step 7: System Prompt Output Brevity Governor Compression ]
       ├─► [ Step 8: Safe Vault Restoration & Prompt Reassembly ]
       ├─► [ Step 9: Reasoning Governor & Brevity Directive Injection ]
       └─► [ Step 10: Adaptive Routing & 429 Exponential Backoff Retry ]
       │
       ▼ (Upstream Proxy Dispatch with SSE Streaming)
[ LLM Provider: OpenAI / Anthropic / Groq / DeepSeek / OpenRouter / Local ]
```

### Stage 1: Safe Vault AST Token Isolation
Prior to applying semantic compression algorithms, the input string is parsed into an Abstract Syntax Tree (AST). The Safe Vault isolates the following syntactical constructs:
- Markdown Code Fences: Triple-backtick multi-line blocks with language identifiers.
- Inline Code: Single-backtick spans containing variable names, function calls, or file paths.
- JSON Payloads: Structural keys, boolean literals, integers, and nested objects.
- SQL Statements: SELECT, UPDATE, INSERT queries and database schema definitions.
- Resource Identifiers: URLs, URIs, IPv4/IPv6 addresses, and file system paths.
- Template Variables: Mustache, Jinja, or standard interpolation variables such as `{{user_name}}` or `${token}`.

Each isolated entity is replaced with a collision-proof hexadecimal placeholder token. This isolation guarantees that no compression heuristic modifies code logic, syntax, indentation, or data schemas.

### Stage 2: Syntax-Safe Code Comment & License Pruner
When processing source code files within prompts:
- Strips multi-line Apache 2.0, MIT, BSD, and GPL copyright headers.
- Strips decorative divider comments (`// =====================`, `# ---------------------`).
- Strips redundant inline comments while strictly preserving docstrings, JSDoc types, type signatures, and logic annotations.
- Yields a verified -72.7% token reduction on commented code blocks with 0% AST breakage.

### Stage 3: ANSI Terminal & Progress Bar Filter
For multi-turn agent loops (OpenCode, Claude Code, Cursor):
- Detects and eliminates terminal ANSI color codes (`\x1b[32m`, `\x1b[0m`), backspace sequences, and interactive terminal progress bars.
- Strips repetitive git status metadata and unified diff index noise.
- Prevents agent context window saturation and context timeout failures.

### Stage 4: In-Memory SHA-256 LRU Cache
- Hash Key: `SHA-256(cleaned_prompt + model + mode)`.
- Eviction Policy: Least Recently Used (LRU) with time-to-live (TTL) invalidation.
- Performance: 0.01ms cache hit replay with zero upstream LLM token consumption.

---

## 4. Empirical Benchmark Telemetry

### Multi-Domain Empirical Breakdown (12 Tasks, 5 Categories)

1. RAG & Enterprise Documentation QA:
   - Average Token Savings: -79.7% (peak -90.9%).
   - OAuth2 Spec: 264 tokens -> 24 tokens (-90.9%).
   - SLA & Cancellation Policy: 201 tokens -> 70 tokens (-65.2%).
   - Entity Preservation: 100% verified.

2. Customer Support & Conversational Threads:
   - Average Token Savings: -50.0% (peak -89.2%).
   - Invoice Surcharge Escalation: 203 tokens -> 22 tokens (-89.2%).
   - Incident Escalation Log: 185 tokens -> 171 tokens (-7.6%).
   - Entity Preservation: 100% verified.

3. System Prompts & Agent Guardrails:
   - Average Token Savings: -45.3% (peak -61.6%).
   - Developer Platform Directives: 146 tokens -> 56 tokens (-61.6%).
   - Financial Compliance Guardrails: 132 tokens -> 95 tokens (-28.0%).
   - Adherence Retention: 100%.

4. Data Extraction & Web Dumps:
   - Average Token Savings: -33.6% (peak -57.7%).
   - HTML Contact Directory Scraping: 248 tokens -> 108 tokens (-56.5% Smart, -57.7% Algo).
   - Production Syslog Trace: 192 tokens -> 184 tokens (-4.2%).

5. Code Engineering & Safe Vault Architecture:
   - Average Token Savings: -28.5% in Smart mode.
   - Sliding Window Rate Limiter: 230 tokens -> 113 tokens (-50.9%).
   - TypeScript LRU Cache: 217 tokens -> 131 tokens (-39.6%).
   - Python Data Pipeline: 261 tokens -> 237 tokens (-9.2%, 100% code locked).
   - PostgreSQL Analytical Query: 175 tokens -> 151 tokens (-13.7%).

### Standardized Academic Accuracy (2,283 Verified Evaluations)

- HumanEval: 89.0% Pass@1 (Baseline: 89.2%, 99.8% relative retention across 164 Python logic tasks).
- MBPP: 82.3% Pass@1 (Baseline: 82.4%, 99.9% relative retention across 500 algorithm problems).
- GSM8k: 94.5% Exact Match (Baseline: 94.6%, 99.9% relative retention across 1,319 math reasoning tasks).
- SWE-bench Lite: 32.7% Resolved (Baseline: 32.8%, 99.7% relative retention across 300 real-world repository patch instances).
- TypeScript AST Suite: 100.0% Zero Syntax Error Pass Rate across 50,000 code samples.
- Overall Weighted Accuracy Retention: 99.8%.

### Live OpenCode CLI Agent Telemetry

- Execution Speedup: 6.7x to 10.6x faster end-to-end agent completion on complex refactoring (Direct baseline stalled for 55.95s; Curtly completed in 8.32s Smart / 5.26s Algo).
- Timeout Prevention: Stripped 15,755 transient ANSI tokens on PostgreSQL agent loop, turning a 120s timeout failure into a 9.83s success.
- Gateway Overhead: strictly <8ms in deterministic Algo mode, <15ms in Smart mode.

---

## 5. Drop-In Proxy Integration

### OpenAI-Compatible Proxy Endpoint
`POST https://curtly.dev/api/v1/chat/completions`

### Headers:
- `Authorization: Bearer <CURTLY_API_KEY>`
- `X-Curtly-Compress-Input: true | false`
- `X-Curtly-Mode: smart | fast | aggressive`

### OpenCode CLI Configuration (`~/.config/opencode/opencode.json`)
```json
{
  "providers": {
    "curtly-ai": {
      "name": "Curtly AI Gateway",
      "baseURL": "https://curtly.dev/api/v1",
      "apiKey": "your_curtly_api_key",
      "headers": {
        "X-Curtly-Compress-Input": "true",
        "X-Curtly-Mode": "smart"
      }
    }
  }
}
```

### Claude Code CLI Integration
```bash
export ANTHROPIC_BASE_URL="https://curtly.dev/api/v1"
export ANTHROPIC_API_KEY="your_curtly_api_key"
claude
```

### Python OpenAI SDK
```python
from openai import OpenAI

client = OpenAI(
    base_url="https://curtly.dev/api/v1",
    api_key="your_curtly_api_key"
)

response = client.chat.completions.create(
    model="gpt-6-astra",
    messages=[
        {"role": "system", "content": "You are a code analysis assistant."},
        {"role": "user", "content": "Analyze the database schema for indexes."}
    ]
)
print(response.choices[0].message.content)
```

### TypeScript / Node.js SDK
```typescript
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://curtly.dev/api/v1',
  apiKey: process.env.CURTLY_API_KEY,
});

const response = await client.chat.completions.create({
  model: 'gpt-6-astra',
  messages: [{ role: 'user', content: 'Generate an optimized SQL query.' }],
});
console.log(response.choices[0].message.content);
```

---

## 6. Tokenizer & Frontier Model Support

Curtly maintains exact native BPE token counting algorithms matching each model family:
- Anthropic Claude BPE: Claude Fable 5.1, Claude Fable 5, Claude Opus 5, Claude Mythos Preview.
- OpenAI o200k_base: GPT 6 Astra, GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna.
- Moonshot AI: Kimi K3.
- DeepSeek 128k BPE: DeepSeek R1, DeepSeek-V3.
- Alibaba: Qwen 3.8 Max, Qwen 2.5.
- Zhipu AI: GLM-5.3.
- Tencent: Hunyuan Frontier.
- Meta: Muse Spark 1.3, Llama 3.3.
- Google SentencePiece 256k: Gemini 3.0, Gemini 2.5 Pro.
- Legacy Tokenizer Standards: cl100k_base (Legacy GPT-4, GPT-3.5-turbo).

---

## 7. Security and Data Governance Architecture

Curtly is engineered from the ground up for strict enterprise security compliance:
- Zero Data Retention: Prompt text and compressed output reside solely in volatile RAM during request execution. No payload text is logged, indexed, or stored to non-volatile disk.
- Zero Model Training: Curtly never retains customer prompts or model responses for training or fine-tuning.
- Transport Layer Security: All ingress and egress traffic enforces TLS 1.3 encryption with modern cipher suites.
- API Key Lifecycle: Secret API keys are salted and hashed using SHA-256 before storage. Plaintext keys are never viewable after initial generation.
- Access Scopes: Keys support granular permissions (Full Access, Compress Only, Read-Only Metrics) and automated expiration TTLs (30 days, 90 days, 1 year).

---

## 8. Pricing Tiers

- Free Tier: $0/month. 500 requests/month, 1 API key, Balanced mode.
- Starter Tier: $19/month. 25,000 requests/month, 3 API keys, basic analytics.
- Growth Tier: $29/month. 100,000 requests/month, 5 API keys, customizable Safe Vault rules.
- Enterprise Tier: $199/month. 1,000,000 requests/month, unlimited API keys, priority priority proxy routing, 99.9% SLA.

---

## 9. Key Links

- Website: https://curtly.dev/
- Live Benchmark Inspector: https://curtly.dev/benchmark
- ROI Calculator: https://curtly.dev/calculator
- Documentation: https://curtly.dev/docs
- Terms of Service: https://curtly.dev/terms
- Privacy Policy: https://curtly.dev/privacy