v1.0.0

Recovery Resync

Last updated


This page explains how data from the fallback period flows from local files through memory back to Redis once Redis recovers, and which entries are not written back.

Steps


changeToNormalMode (sync.go:50-92) runs in order:
StepActionOn failure
1Walk every .json file under {DBPath}/{DB}Errors such as a missing directory → the whole flow aborts and isHealth stays false
2Read and parse each file as Cache, store it in the memory cache under Cache.KeyA file that fails to read or parse → error logged, file skipped
3syncMemoryToRedis: iterate the entire memory cache and pipeline entries to RedisPipeline execution errors are not checked
4cleanupLocalFile: delete every .json file and empty subdirectory (keeping {DBPath}/{DB} itself)A failed delete → error logged, continue
5isHealth = true—

Write-Back Rules


syncMemoryToRedis (sync.go:94-133):
ItemRule
ScopeEvery non-expired entry in the memory cache, including entries that were in memory before fallback
Redis TTLTimestamp + TTL − now seconds (remaining time)
Remaining TTL ≤ 0Not written; entries without a TTL (ttl = 0) also land here
BatchingOne Exec per 100 non-expired entries, plus a final Exec for the remainder
FormatJSON of Data (strings have quotes trimmed), same as Set
Concurrency guardisRecovering uses CAS so only one resync runs at a time; a concurrent call returns immediately

Observed Results


Two keys written while Redis was down, then Redis restored:
KeyWriteRedis after recoveryLocal file after recovery
ttlkeySet("ttlkey", "t", time.Hour)Present, TTL 59m59sDeleted
foreverSet("forever", "f", 0)AbsentDeleted

forever is still in the memory cache; the first read in normal mode writes it back to Redis through Read-Repair.

When Data Survives Only in Memory


Step 4 deletes local files whether or not step 3 succeeded. After any of the following, the data lives only in the memory cache and is lost when the process exits:
CaseReason
Pipeline Exec fails (for example Redis drops again during recovery)Errors are unchecked and files are deleted anyway
A resync is already runningThe second syncMemoryToRedis call returns immediately, but the file cleanup still runs
ttl = 0 entriesNot in the batch; only read-repair writes them back

Memory Cache After Recovery


Recovery does not clear the memory cache. Every loaded entry stays in memory until it expires and is removed by the background sweep; in normal mode, Get always hits memory for these keys and never asks Redis.
中文