v1.0.0

Health Check and Mode Switch

Last updated


This page explains what switches the instance into fallback mode, how the health check detects that Redis is back, and what happens between detection and returning to normal mode.

Fallback Triggers


TriggerLocationResult
Startup ping in New failsinstance.go:43-46Enters fallback directly
Get fails MaxRetry Redis readsget.go:55-58The same call continues on the local read path
Set fails MaxRetry Redis writesset.go:50-55The same call writes to memory and file instead

A failed Redis delete in Del only returns an error; it does not switch modes.

The Switch


changeToFallbackMode (sync.go:25-48):
  1. Sets isHealth to false.
  2. Returns immediately if a health-check ticker already exists, so it never starts twice.
  3. Otherwise creates a ticker at TimeToCheck intervals (default 1 minute) that sends PING to Redis on each tick.

Detecting Recovery


sequenceDiagram
    participant T as Health-Check Ticker
    participant R as Redis
    participant RF as RedisFallback
    loop Every TimeToCheck
        T->>R: PING
        R--xT: Failure
    end
    T->>R: PING
    R-->>T: PONG
    T->>RF: go changeToNormalMode
    T->>T: Stop ticker, clear checker
    Note over RF: isHealth = true after resync completes
  1. sequenceDiagram
  2. participant T as Health-Check Ticker
  3. participant R as Redis
  4. participant RF as RedisFallback
  5. loop Every TimeToCheck
  6. T->>R: PING
  7. R--xT: Failure
  8. end
  9. T->>R: PING
  10. R-->>T: PONG
  11. T->>RF: go changeToNormalMode
  12. T->>T: Stop ticker, clear checker
  13. Note over RF: isHealth = true after resync completes

ItemBehavior
Maximum detection delayTimeToCheck; after Redis comes back it can take up to one interval to notice
How recovery runsIn a separate goroutine; the ticker stops immediately
Reads and writes during recoveryisHealth is still false, so Get / Set keep using the local path until the resync finishes
Failing againThe next read or write that exhausts its retries re-enters fallback and creates a new ticker

Recovery at Startup


When the startup ping in New succeeds, changeToNormalMode runs synchronously (instance.go:47-50): JSON files left by a previous process during fallback are loaded and resynced to Redis on this start.
The first step of changeToNormalMode walks {DBPath}/{DB}; if that directory does not exist it returns an error, isHealth is never set to true, no health check is running, and the instance stays in local mode for good. Create the directory before the first deployment; see Getting Started.

Resync Details


The full steps for loading files, writing back to Redis, and removing local files are in Recovery Resync.
中文