Jurisdiction-scoped prefix caching
Prefixes are cached separately per jurisdiction.
- Per-jurisdiction
- Cache partition
- Logged
- Hit or miss, per decision
- Bounded
- Cost uplift of partition
Measured, not estimated
01
Design
How this part works
Cache partitions
Prefixes cached in one jurisdiction cannot be reused in another.
Shared system prompts
A long system prompt shared by a whole product is cached once per jurisdiction. Prefix reuse reduces cost within each jurisdiction without crossing its boundary.
Cache hit records
The selection record states whether a hit was available in the eligible set.
02
Comparison
Boundary-first routing
Global cache
- Highest hit rate across jurisdictions
- A warm prefix can pull work into another region
- Hit rate measured per jurisdiction
- Warm prefixes never cross a boundary
- Cost optimised inside the policy
03
Elsewhere on the platform
Other platform features
Start