Add persistence design document

2026-05-15 01:39:55 -07:00
parent 5490b325be
commit a0eea829fd
2 changed files with 416 additions and 4 deletions
@@ -586,8 +586,8 @@ Target deliverable: A small group can join a server, spawn into one biome, gathe
 ## 1.13 Persistence MVP
- [ ] Decide MVP persistence scope.
+- [x] Decide MVP persistence scope.
- [ ] Decide what tile metadata is stored in save data vs external tile registry.
+- [x] Decide what tile metadata is stored in save data vs external tile registry.
 - [ ] Save player identity.
 - [ ] Save player stats.
 - [ ] Save player inventory.
@@ -1432,7 +1432,7 @@ Earliest incomplete foundation items:
 - [x] Create the core design document.
 - [x] Create the technical design document.
 - [x] Create the multiplayer/networking design document.
- [ ] Create the persistence design document.
+- [x] Create the persistence design document.
 - [ ] Create the Earth-scale terrain/tile streaming design document.
 - [x] Launch near-term MVP map-tile serving cloud VM and prove Ground Zero tile lookup/download/cache flow.
 - [ ] Create economy and AGR design document.
@@ -1443,4 +1443,4 @@ Earliest incomplete foundation items:
 Immediate next item:
- [ ] Create the persistence design document.
+- [ ] Create the Earth-scale terrain/tile streaming design document.
@@ -0,0 +1,412 @@
 # Agrarian Persistence Design Document
 ## Purpose
 This document defines Agrarian's persistence direction for the MVP foundation.
 It separates what the game saves, what remains external metadata, how save data
 should be versioned, and what must be deferred until later persistence systems
 are needed.
 The design goal is to preserve meaningful player and world progress without
 locking the project into brittle prototype data.
 ## Persistence Principles
 ### Save Meaningful State
 Persist things that change gameplay or player progress:
 - player identity;
 - survival stats;
 - inventory;
 - placed structures;
 - world time and weather state;
 - resource depletion where needed;
 - active tile/package version.
 Do not persist static terrain files, generated caches, editor-only assets, or
 pure presentation state.
 ### Version Everything That Can Outlive A Build
 Save data is a compatibility contract. Every persistent record should carry
 enough version/context to migrate or safely reject it later.
 Minimum version fields:
 - save format version;
 - game build/version;
 - active map/tile ID;
 - active tile package version;
 - record type version where needed.
 ### External Metadata Stays External
 Tile registry and terrain package metadata should remain external authoritative
 metadata. Save data should reference tile IDs/package versions, not copy entire
 tile registries into saves.
 ### Server Owns Writes
 Only the server writes authoritative persistence records. Clients display
 replicated state and may request actions, but they do not write authoritative
 save state directly.
 ## MVP Persistence Scope
 The MVP persistence scope is intentionally narrow:
 - player identity;
 - player survival snapshot;
 - player inventory snapshot;
 - placed campfire/shelter/basic build pieces;
 - resource depletion state where needed for shared gameplay;
 - world time;
 - weather state/seed/source metadata;
 - active Ground Zero tile ID;
 - active Ground Zero tile package version.
 Containers should be included once they become part of the playable loop. If
 containers are not in the first playable loop yet, they remain a near-term
 implementation item rather than a blocker for this design document.
 ## Player Identity
 MVP player identity can be simple.
 Acceptable early identity options:
 - local platform/user identifier;
 - deterministic test profile ID;
 - server-assigned temporary player ID;
 - future account ID once account services exist.
 Persistence should avoid using player display names as primary keys. Display
 names can change.
 Minimum player record:
 ```text
 PlayerId
 DisplayName
 LastKnownTileId
 LastKnownTransform
 SurvivalSnapshot
 InventorySnapshot
 UpdatedAt
 RecordVersion
 ```
 ## Survival Snapshot
 Persist the survival state needed to resume a player:
 - health;
 - stamina;
 - hunger;
 - thirst;
 - body temperature;
 - injury severity;
 - alive/dead state if death persistence is active.
 Survival rates and tuning values should not be duplicated into the save unless
 there is a specific compatibility reason. They belong in code/config/data
 assets.
 ## Inventory Snapshot
 Persist inventory as item definition IDs and stack counts.
 Do persist:
 - item ID;
 - stack count;
 - durability/condition when added;
 - container/location ID when containers exist.
 Do not persist full item definition data. Item definitions belong in Data
 Assets and are loaded by ID.
 ## World State
 MVP world state should include:
 - world/gameplay time;
 - weather state;
 - weather seed or source timestamp when available;
 - active tile ID;
 - active tile package version;
 - save timestamp;
 - save format version.
 World time uses the MVP calendar target:
 ```text
 4 real hours = 1 in-game day
 ```
 Day/night presentation can later use real-region solar metadata, but the save
 should persist the gameplay calendar state and enough tile context to recreate
 presentation.
 ## Structure And Placement State
 Placed gameplay structures should persist when they are part of the survival
 loop.
 Minimum placed object record:
 ```text
 ObjectId
 ObjectTypeId
 TileId
 Transform
 OwnerPlayerId
 HealthOrCondition
 CustomState
 CreatedAt
 UpdatedAt
 RecordVersion
 ```
 For MVP, this includes:
 - campfire;
 - primitive shelter;
 - basic build pieces once available.
 Future structure persistence should support decay, ownership, permissions,
 settlement membership, repair state, and destruction history.
 ## Resource Depletion State
 Not every resource needs persistent depletion in the MVP.
 Persist depletion when:
 - multiple players can observe the changed state;
 - the depleted state affects survival;
 - immediate respawn would undermine the loop;
 - the actor is intentionally part of a shared world memory test.
 Avoid persisting every foliage or minor gatherable actor until performance and
 world-density rules are clear.
 Minimum resource state:
 ```text
 ResourceId
 ResourceTypeId
 TileId
 RemainingAmount
 DepletedAt
 RespawnAt
 RecordVersion
 ```
 ## Tile Metadata In Save Data
 Save data should reference external tile metadata, not duplicate it.
 Persist in save:
 - active tile ID;
 - active tile package version;
 - tile registry schema version;
 - tile package checksum or manifest hash if needed for validation.
 Keep external:
 - full tile registry;
 - terrain metadata;
 - heightmap metadata;
 - landform analysis;
 - water/shoreline analysis;
 - neighbor edge verification;
 - tile package files.
 Reason:
 - terrain packages may improve over time;
 - tile registry can grow independently;
 - save files should stay small and migrate safely;
 - player/world state should not corrupt when tile metadata updates.
 ## Containers
 Containers are a near-term persistence item once storage gameplay exists.
 Minimum container record:
 ```text
 ContainerId
 ContainerTypeId
 TileId
 Transform
 OwnerPlayerId
 InventorySnapshot
 LockOrPermissionState
 RecordVersion
 ```
 Until containers are implemented, inventory can remain player-carried and
 placed structure state can remain separate.
 ## Save Timing
 MVP save timing should include:
 - manual admin save command;
 - load on server start;
 - periodic server-side save interval once server state is changing regularly;
 - explicit save before planned server shutdown.
 Recommended initial interval:
 ```text
 every 5 minutes during active play
 ```
 The interval should be tunable. It should not block gameplay on slow disk I/O.
 ## Load On Server Start
 Server startup should:
 1. Load save metadata.
 2. Validate save format version.
 3. Validate active tile ID and package version.
 4. Load world state.
 5. Load placed structures/resource state.
 6. Load known player records.
 7. Spawn or restore runtime actors.
 8. Log skipped or migrated records.
 If save validation fails, the server should fail clearly or start a new world
 only when explicitly configured to do so.
 ## Migration Strategy
 Persistence must assume data formats will change.
 Rules:
 - never silently reinterpret old records as new formats;
 - add migration functions when changing saved record shape;
 - keep unknown fields harmless where possible;
 - log migrations;
 - reject unsupported future save versions;
 - back up saves before destructive migrations.
 MVP can use simple file saves, but the data shape should still include versions
 so future database migration is possible.
 ## Storage Backend
 MVP storage can be file-based while the system is small.
 Acceptable MVP options:
 - Unreal `USaveGame` records;
 - JSON records for inspectability;
 - hybrid approach where gameplay uses `USaveGame` and tile metadata remains
  JSON/external.
 Long-term candidates:
 - SQLite for local/server structured state;
 - PostgreSQL for persistent multiplayer services;
 - object storage/CDN for tile packages;
 - append-only event logs for economy/governance where auditability matters.
 Do not choose the long-term database before the MVP proves the shape of the
 state that needs to persist.
 ## Backup And Recovery
 Persistence is not a backup by itself.
 MVP operations should keep:
 - repository backup;
 - VM backup;
 - save data backup;
 - tile package backup;
 - tile registry backup.
 Before changing save formats, take a manual backup of current save data.
 ## Security
 Persistent data should never store secrets.
 Do not store:
 - passwords;
 - API tokens;
 - private keys;
 - admin reset tokens;
 - mail credentials;
 - DigitalOcean tokens.
 Player records should store stable IDs and gameplay state, not sensitive
 account credentials.
 ## Testing Gates
 Minimum persistence smoke test:
 1. Start server.
 2. Join as a player.
 3. Change survival/inventory state.
 4. Place a campfire or primitive shelter.
 5. Deplete one resource node if resource persistence is enabled.
 6. Save manually.
 7. Stop server.
 8. Start server.
 9. Confirm player/world state restored as expected.
 Minimum migration test:
 1. Load a previous save format fixture.
 2. Run migration or compatibility load.
 3. Confirm unsupported data is rejected or logged clearly.
 4. Confirm supported data survives.
 Minimum tile validation test:
 1. Save active Ground Zero tile ID/package version.
 2. Restart server.
 3. Confirm the server validates the active package version.
 4. Confirm clients can still download the required package from the tile
   endpoint.
 ## Current Roadmap Decisions
 This document defines:
 - MVP persistence scope;
 - save data vs external tile registry boundary;
 - player identity save direction;
 - player stats save direction;
 - player inventory save direction;
 - resource depletion persistence rule;
 - world time save direction;
 - weather seed/state save direction;
 - container persistence direction;
 - server-side save interval direction;
 - load-on-server-start direction;
 - initial tile registry persistence direction.
 Implementation work remains tracked separately in the roadmap.
 ## Open Questions
 - Should the first playable MVP use `USaveGame`, JSON, or a hybrid save
  backend?
 - What is the first stable `SaveFormatVersion` value?
 - Which resource nodes should persist depletion in Ground Zero?
 - Should disconnected players remain in world physically or be removed?
 - How much inventory should death/respawn preserve?
 - When do saves move from file-based records to database-backed records?
 - How should family/generation records attach to player identity later?