State Store Cache Benchmark

Experimenting with various caching solutions to evaluate their performance for the State Store service. Detailed benchmark results are available in the results.txt file.

Usage

Run All Benchmarks

make bench          # 5-second benchmarks for all systems

Deterministic Testing

make deterministic  # Verify reproducible behavior across systems

Docker Services

make up            # Start Valkey container
make down          # Stop and cleanup

Storage Systems Tested

PebbleDB Configurations

• Default: Standard PebbleDB with 8MB cache
• DisableWAL: No write-ahead logging for faster writes
• LargeMemtable: 128MB memtable vs default ~4MB
• LargeCache: 4GB cache for memory-intensive workloads
• LargeCacheDisableWAL: Combined large cache + no WAL
• LargeMemtableAndCacheWithNoWAL: All optimizations combined
• OptimizedForWrites: Aggressive write optimization with large memtables, increased compaction thresholds, and concurrent compactions

Redis/Valkey with Caching

• RedisWithRistretto: Redis backend with Ristretto LRU cache (4GB, 100M counters)
• RedisWithHashicorpLRU: Redis backend with Hashicorp LRU cache (1M entries, can likely cause OOMs with large payloads)
• Rueidis: Direct Redis access without additional caching

Benchmark Assumptions

Workload Distribution

• Read Operations: ~70% of total operations
• Write Operations: 20% of total operations (configurable via updateRatio)
• Delete Operations: 10% of total operations (configurable via deleteRatio)

Concurrency Model

• Workers: 20 concurrent goroutines (conservative setting for PebbleDB; Redis benchmarks would benefit from higher concurrency due to being IO-bound)
• Key Distribution: Zipf distribution with s=1.005 (mild hotspot bias)
• Key Space: 500,000 unique keys with 16-byte identifiers

Value Size Distribution

• Range: 512B to 4MB per value
• Small Values (90.0%): 512B - 8KB
• Medium Values (9.9%): 8KB - 512KB
• Large Values (0.1%): 512KB - 4MB

Expected average value size: ~33KB

Key Configuration Parameters

Metrics Reported

Performance Metrics

• ops/sec: Operations per second (primary throughput metric)
• write-MB/s: Write throughput in megabytes per second
• read-ms: Average read latency in milliseconds
• write-ms: Average write latency in milliseconds
• delete-ms: Average delete latency in milliseconds

Storage-Specific Metrics

• pebble-hit-%: PebbleDB block cache hit ratio
• pebble-hits/miss: Absolute cache hit/miss counts
• lru-hit-%: LRU cache hit ratio for Redis+cache benchmarks
• lru-hits/miss: LRU cache absolute hit/miss counts