v1.0.0

Write Path

Last updated


This page explains how Set and Del behave in each mode, and how writes during fallback are batched to disk through a queue.

Set


ModeBehaviorSource
NormalWrite Redis with SET (no expiry when TTL is 0), up to MaxRetry times; on success, store in the memory cacheset.go:33-48
Normal (retries exhausted)Switch to fallback; the same call continues with a fallback writeset.go:50-55
FallbackStore in the memory cache, then send a WriteRequest to the queueset.go:58-68

When Set returns nil in fallback mode, the data is already in memory and Get can read it immediately; the file write happens later.

Batch File Writes


graph LR
    Set[Set] --> Q{Queue has room?}
    Q -->|Yes| Queue[queue of length MaxQueue]
    Q -->|No| Sync[Synchronous writeToFile]
    Queue --> Pending[pending map merged by key]
    Timer[Every TimeToWrite] --> Flush[Take and reset pending]
    Pending --> Flush
    Flush --> Par[One goroutine per key writes its file]
  1. graph LR
  2. Set[Set] --> Q{Queue has room?}
  3. Q -->|Yes| Queue[queue of length MaxQueue]
  4. Q -->|No| Sync[Synchronous writeToFile]
  5. Queue --> Pending[pending map merged by key]
  6. Timer[Every TimeToWrite] --> Flush[Take and reset pending]
  7. Pending --> Flush
  8. Flush --> Par[One goroutine per key writes its file]

ItemBehaviorSource
QueueLength MaxQueue (default 1000)instance.go:36
MergingMultiple writes to one key within an interval keep only the lastwriter.go:13-16
Flush intervalTimeToWrite (default 3 seconds)writer.go:17-18
ParallelismOne goroutine per key; the round ends when all finishwriter.go:39-49
Full queueDoes not block; writes the file synchronously and returns its errorset.go:61-65
File contentJSON of the whole Cache structure, mode 0644writer.go:52-65

Del


Del (del.go:7-23) always deletes the memory entry and the local file; in normal mode it also runs Redis DEL and returns an error if that fails (no mode switch, no retry). Deleting a missing key returns no error.
ModeMemoryLocal fileRedis
NormalDeletedDeletedDEL
FallbackDeletedDeletedUntouched

A key deleted during fallback still exists in Redis; the recovery resync does not replay deletes, so the key stays in Redis until it expires on its own.

Example


  1. if err := rf.Set("user:1", map[string]any{"name": "pardn"}, 30*time.Minute); err != nil {
  2. // only when serialization fails, or the fallback queue is full and the synchronous file write fails
  3. log.Println(err)
  4. }
  5. if err := rf.Del("user:1"); err != nil {
  6. // only when Redis DEL fails in normal mode
  7. log.Println(err)
  8. }
中文