Why Memory Optimization Matters

Redis performance depends on keeping data in RAM. When memory fills, eviction policies delete keys or writes fail — both impact application behavior. Memory is often the primary scaling constraint before CPU or network.

Goals:

  • Fit more data per node (delay Cluster expansion)
  • Reduce cost (smaller instances)
  • Prevent unexpected evictions
  • Maintain predictable latency
  INFO memory
MEMORY STATS
MEMORY DOCTOR

Understanding Memory Usage

Metric Description
used_memory Redis allocator view
used_memory_rss OS physical memory
used_memory_dataset Approximate user data size
used_memory_overhead Keys, metadata, buffers
mem_fragmentation_ratio RSS / used_memory

Healthy fragmentation ratio: 1.0–1.5. Above 1.5 suggests wasted RAM — consider restart or MEMORY PURGE.

  MEMORY USAGE user:1001
MEMORY USAGE user:1001 SAMPLES 5
OBJECT ENCODING user:1001

Key Design for Memory Efficiency

Shorter Keys

Key names consume memory — every byte in the key is stored:

  # Bad — 40+ byte key names × millions of keys
SET application:production:users:1001:profile:data "..."

# Good — concise but readable
SET u:1001:prof "..."

Rule of thumb: shorter keys save megabytes at millions of keys scale.

Right Data Structure

Scenario Memory-Efficient Choice
Object with fields Hash (listpack encoding for small)
Unique IDs Set (intset for integer-only small sets)
Approximate UV count HyperLogLog (~12 KB fixed)
Boolean flags for millions of IDs Bitmap
JSON blob read as whole String with compression 
  # HyperLogLog: ~12 KB vs Set: ~8 bytes × N members
PFADD uv:2024-06-13 user:1001 user:1002
PFCOUNT uv:2024-06-13

Hash Field Limits for Listpack Encoding

Small hashes use compact listpack encoding:

  hash-max-listpack-entries 512
hash-max-listpack-value 64

Keep hash fields small and count ≤512 for optimal encoding. Check with OBJECT ENCODING.

Compression

Compress large string values before storing:

  import zlib
import json

def set_compressed(r, key, data, ttl=3600):
    compressed = zlib.compress(json.dumps(data).encode(), level=6)
    r.setex(key, ttl, compressed)

def get_compressed(r, key):
    raw = r.get(key)
    if raw:
        return json.loads(zlib.decompress(raw))
    return None

Trade-off: CPU for compression/decompression vs RAM savings. Worth it for values > 1 KB.

TTL and Key Lifecycle

Keys without TTL grow memory indefinitely:

  # Find keys without expiry (sample)
redis-cli --scan --pattern 'cache:*' | head -100 | while read k; do
  ttl=$(redis-cli TTL "$k")
  [ "$ttl" = "-1" ] && echo "$k"
done

Set TTL at creation. Audit periodically for orphan keys.

Eviction Policy Selection 

  maxmemory 8gb
maxmemory-policy allkeys-lfu
maxmemory-samples 10
Workload Recommended Policy
Pure cache allkeys-lru or allkeys-lfu
Cache + TTL sessions volatile-lru on shared instance, or separate instances
No data loss tolerance noeviction + alerting at 80% memory

LFU (Least Frequently Used) retains hot keys better than LRU for skewed access patterns.

Data Type Specific Optimizations

Strings 

  # Store integers as strings — Redis optimizes int encoding
SET counter:views 1000000
OBJECT ENCODING counter:views
# "int"

Sets

Integer-only sets with ≤512 members use intset encoding:

  SADD user:ids:batch1 1001 1002 1003
OBJECT ENCODING user:ids:batch1
# "listpack" or "intset"

Sorted Sets 

  zset-max-listpack-entries 128
zset-max-listpack-value 64

Small ZSETs use listpack — much smaller than skip list.

Streams

Trim streams to prevent unbounded growth:

  XADD events * field value
XTRIM events MAXLEN ~ 50000

Fragmentation Management

High mem_fragmentation_ratio wastes RAM:

  INFO memory
# mem_fragmentation_ratio:1.8

MEMORY PURGE          # Redis 4+ — ask allocator to release pages
# Or planned restart during maintenance window

Causes: frequent updates/deletes, RDB forks, changing value sizes.

Prevention:

  • Avoid frequent size-changing updates on same keys
  • Use consistent value sizes where possible
  • Restart during maintenance if ratio sustained > 1.5

Active Defragmentation (Redis 4+) 

  activedefrag yes
active-defrag-ignore-bytes 100mb
active-defrag-threshold-lower 10
active-defrag-threshold-upper 100

Redis actively defragments during idle cycles — monitor CPU impact.

Capacity Planning 

  # Estimate memory for N keys
redis-cli DEBUG POPULATE 1000000 user: 10
INFO memory
# used_memory / 1000000 = bytes per key (approximate)
DEBUG POPULATE 0   # clean up test data

Plan headroom:

  • 20% buffer below maxmemory for AOF rewrite / RDB fork copy-on-write
  • Monitor used_memory_peak after traffic spikes

Best Practices

  1. Audit key patterns — remove unused prefixes
  2. Set TTL on all ephemeral data
  3. Use hashes over JSON strings for small objects
  4. Use HyperLogLog/bitmaps for analytics when exact counts unnecessary
  5. Separate instances by workload (cache vs sessions) with appropriate policies
  6. Monitor memory weekly — trend analysis predicts capacity needs

Common Mistakes

Mistake Impact
Storing full API responses with redundant data 10× memory vs normalized hashes
No maxmemory set Host OOM
Long descriptive key names at scale Millions of wasted bytes
Keeping debug/test keys in production Silent memory drain
allkeys-lru on session store Users logged out randomly

Troubleshooting

Memory growing despite TTL:

  INFO keyspace
# Compare keys over time
redis-cli --bigkeys
# Find largest keys

Sudden memory spike:

  INFO memory
# Check used_memory_rss vs used_memory — fork COW during BGSAVE?
LASTSAVE
INFO persistence

Evictions increasing:

  INFO stats
# evicted_keys counter rising
# Increase maxmemory, optimize keys, or add nodes

Performance Tips

  • Run redis-cli --bigkeys and redis-cli --memkeys (Redis 4+) monthly
  • Pipeline UNLINK (not DEL) when bulk-deleting large keys
  • Prefer SCAN + batch delete over FLUSHDB in production
  • Use Redis 7 client-side caching (tracking) to reduce duplicate data in app + Redis

Production Scenario

A SaaS platform’s 16 GB Redis instance hit 94% memory with evictions impacting cache hit ratio (91% → 74%). Analysis via --bigkeys found 2M session keys averaging 4 KB (stored full user objects). Migration: hashes with user_id + role only (avg 180 bytes), gzip on cached API responses > 2 KB. Memory dropped to 6.2 GB. Hit ratio recovered to 93%. Deferred Cluster migration by 8 months, saving $4K/month.

Memory optimization is continuous — measure per-key cost, choose structures deliberately, and audit key lifecycles before adding hardware.