chopratejas
|
7a05808e0f
|
Add cache optimization module with scalable dynamic content detection
Implements a comprehensive cache optimization layer for LLM providers:
- Provider-specific optimizers (Anthropic, OpenAI, Google) with distinct
caching strategies: explicit breakpoints, prefix stabilization, and
CachedContent API respectively
- Scalable dynamic content detector using three strategies:
1. Structural detection: "Label: value" patterns (language-agnostic)
2. Entropy-based detection: high-entropy strings (IDs, tokens, hashes)
3. Universal patterns: ISO 8601, UUIDs, JWTs, hex hashes
- NO hardcoded locale-specific patterns (no month names, etc.)
- Semantic caching layer with LRU eviction and TTL support
- Plugin registry for provider selection and custom optimizers
- 131 tests, real-world benchmarks showing 20-55% compression at <0.3ms
|
2026-01-07 14:07:49 -08:00 |
|