From a0eea829fd10b8dc18e821557f871cbddcd7d714 Mon Sep 17 00:00:00 2001 From: nathan Date: Fri, 15 May 2026 01:39:55 -0700 Subject: [PATCH] Add persistence design document --- AGRARIAN_DEVELOPMENT_ROADMAP.md | 8 +- Docs/PersistenceDesignDocument.md | 412 ++++++++++++++++++++++++++++++ 2 files changed, 416 insertions(+), 4 deletions(-) create mode 100644 Docs/PersistenceDesignDocument.md diff --git a/AGRARIAN_DEVELOPMENT_ROADMAP.md b/AGRARIAN_DEVELOPMENT_ROADMAP.md index 562964f..113e507 100644 --- a/AGRARIAN_DEVELOPMENT_ROADMAP.md +++ b/AGRARIAN_DEVELOPMENT_ROADMAP.md @@ -586,8 +586,8 @@ Target deliverable: A small group can join a server, spawn into one biome, gathe ## 1.13 Persistence MVP -- [ ] Decide MVP persistence scope. -- [ ] Decide what tile metadata is stored in save data vs external tile registry. +- [x] Decide MVP persistence scope. +- [x] Decide what tile metadata is stored in save data vs external tile registry. - [ ] Save player identity. - [ ] Save player stats. - [ ] Save player inventory. @@ -1432,7 +1432,7 @@ Earliest incomplete foundation items: - [x] Create the core design document. - [x] Create the technical design document. - [x] Create the multiplayer/networking design document. -- [ ] Create the persistence design document. +- [x] Create the persistence design document. - [ ] Create the Earth-scale terrain/tile streaming design document. - [x] Launch near-term MVP map-tile serving cloud VM and prove Ground Zero tile lookup/download/cache flow. - [ ] Create economy and AGR design document. @@ -1443,4 +1443,4 @@ Earliest incomplete foundation items: Immediate next item: -- [ ] Create the persistence design document. +- [ ] Create the Earth-scale terrain/tile streaming design document. diff --git a/Docs/PersistenceDesignDocument.md b/Docs/PersistenceDesignDocument.md new file mode 100644 index 0000000..ff4dfd0 --- /dev/null +++ b/Docs/PersistenceDesignDocument.md @@ -0,0 +1,412 @@ +# Agrarian Persistence Design Document + +## Purpose + +This document defines Agrarian's persistence direction for the MVP foundation. +It separates what the game saves, what remains external metadata, how save data +should be versioned, and what must be deferred until later persistence systems +are needed. + +The design goal is to preserve meaningful player and world progress without +locking the project into brittle prototype data. + +## Persistence Principles + +### Save Meaningful State + +Persist things that change gameplay or player progress: + +- player identity; +- survival stats; +- inventory; +- placed structures; +- world time and weather state; +- resource depletion where needed; +- active tile/package version. + +Do not persist static terrain files, generated caches, editor-only assets, or +pure presentation state. + +### Version Everything That Can Outlive A Build + +Save data is a compatibility contract. Every persistent record should carry +enough version/context to migrate or safely reject it later. + +Minimum version fields: + +- save format version; +- game build/version; +- active map/tile ID; +- active tile package version; +- record type version where needed. + +### External Metadata Stays External + +Tile registry and terrain package metadata should remain external authoritative +metadata. Save data should reference tile IDs/package versions, not copy entire +tile registries into saves. + +### Server Owns Writes + +Only the server writes authoritative persistence records. Clients display +replicated state and may request actions, but they do not write authoritative +save state directly. + +## MVP Persistence Scope + +The MVP persistence scope is intentionally narrow: + +- player identity; +- player survival snapshot; +- player inventory snapshot; +- placed campfire/shelter/basic build pieces; +- resource depletion state where needed for shared gameplay; +- world time; +- weather state/seed/source metadata; +- active Ground Zero tile ID; +- active Ground Zero tile package version. + +Containers should be included once they become part of the playable loop. If +containers are not in the first playable loop yet, they remain a near-term +implementation item rather than a blocker for this design document. + +## Player Identity + +MVP player identity can be simple. + +Acceptable early identity options: + +- local platform/user identifier; +- deterministic test profile ID; +- server-assigned temporary player ID; +- future account ID once account services exist. + +Persistence should avoid using player display names as primary keys. Display +names can change. + +Minimum player record: + +```text +PlayerId +DisplayName +LastKnownTileId +LastKnownTransform +SurvivalSnapshot +InventorySnapshot +UpdatedAt +RecordVersion +``` + +## Survival Snapshot + +Persist the survival state needed to resume a player: + +- health; +- stamina; +- hunger; +- thirst; +- body temperature; +- injury severity; +- alive/dead state if death persistence is active. + +Survival rates and tuning values should not be duplicated into the save unless +there is a specific compatibility reason. They belong in code/config/data +assets. + +## Inventory Snapshot + +Persist inventory as item definition IDs and stack counts. + +Do persist: + +- item ID; +- stack count; +- durability/condition when added; +- container/location ID when containers exist. + +Do not persist full item definition data. Item definitions belong in Data +Assets and are loaded by ID. + +## World State + +MVP world state should include: + +- world/gameplay time; +- weather state; +- weather seed or source timestamp when available; +- active tile ID; +- active tile package version; +- save timestamp; +- save format version. + +World time uses the MVP calendar target: + +```text +4 real hours = 1 in-game day +``` + +Day/night presentation can later use real-region solar metadata, but the save +should persist the gameplay calendar state and enough tile context to recreate +presentation. + +## Structure And Placement State + +Placed gameplay structures should persist when they are part of the survival +loop. + +Minimum placed object record: + +```text +ObjectId +ObjectTypeId +TileId +Transform +OwnerPlayerId +HealthOrCondition +CustomState +CreatedAt +UpdatedAt +RecordVersion +``` + +For MVP, this includes: + +- campfire; +- primitive shelter; +- basic build pieces once available. + +Future structure persistence should support decay, ownership, permissions, +settlement membership, repair state, and destruction history. + +## Resource Depletion State + +Not every resource needs persistent depletion in the MVP. + +Persist depletion when: + +- multiple players can observe the changed state; +- the depleted state affects survival; +- immediate respawn would undermine the loop; +- the actor is intentionally part of a shared world memory test. + +Avoid persisting every foliage or minor gatherable actor until performance and +world-density rules are clear. + +Minimum resource state: + +```text +ResourceId +ResourceTypeId +TileId +RemainingAmount +DepletedAt +RespawnAt +RecordVersion +``` + +## Tile Metadata In Save Data + +Save data should reference external tile metadata, not duplicate it. + +Persist in save: + +- active tile ID; +- active tile package version; +- tile registry schema version; +- tile package checksum or manifest hash if needed for validation. + +Keep external: + +- full tile registry; +- terrain metadata; +- heightmap metadata; +- landform analysis; +- water/shoreline analysis; +- neighbor edge verification; +- tile package files. + +Reason: + +- terrain packages may improve over time; +- tile registry can grow independently; +- save files should stay small and migrate safely; +- player/world state should not corrupt when tile metadata updates. + +## Containers + +Containers are a near-term persistence item once storage gameplay exists. + +Minimum container record: + +```text +ContainerId +ContainerTypeId +TileId +Transform +OwnerPlayerId +InventorySnapshot +LockOrPermissionState +RecordVersion +``` + +Until containers are implemented, inventory can remain player-carried and +placed structure state can remain separate. + +## Save Timing + +MVP save timing should include: + +- manual admin save command; +- load on server start; +- periodic server-side save interval once server state is changing regularly; +- explicit save before planned server shutdown. + +Recommended initial interval: + +```text +every 5 minutes during active play +``` + +The interval should be tunable. It should not block gameplay on slow disk I/O. + +## Load On Server Start + +Server startup should: + +1. Load save metadata. +2. Validate save format version. +3. Validate active tile ID and package version. +4. Load world state. +5. Load placed structures/resource state. +6. Load known player records. +7. Spawn or restore runtime actors. +8. Log skipped or migrated records. + +If save validation fails, the server should fail clearly or start a new world +only when explicitly configured to do so. + +## Migration Strategy + +Persistence must assume data formats will change. + +Rules: + +- never silently reinterpret old records as new formats; +- add migration functions when changing saved record shape; +- keep unknown fields harmless where possible; +- log migrations; +- reject unsupported future save versions; +- back up saves before destructive migrations. + +MVP can use simple file saves, but the data shape should still include versions +so future database migration is possible. + +## Storage Backend + +MVP storage can be file-based while the system is small. + +Acceptable MVP options: + +- Unreal `USaveGame` records; +- JSON records for inspectability; +- hybrid approach where gameplay uses `USaveGame` and tile metadata remains + JSON/external. + +Long-term candidates: + +- SQLite for local/server structured state; +- PostgreSQL for persistent multiplayer services; +- object storage/CDN for tile packages; +- append-only event logs for economy/governance where auditability matters. + +Do not choose the long-term database before the MVP proves the shape of the +state that needs to persist. + +## Backup And Recovery + +Persistence is not a backup by itself. + +MVP operations should keep: + +- repository backup; +- VM backup; +- save data backup; +- tile package backup; +- tile registry backup. + +Before changing save formats, take a manual backup of current save data. + +## Security + +Persistent data should never store secrets. + +Do not store: + +- passwords; +- API tokens; +- private keys; +- admin reset tokens; +- mail credentials; +- DigitalOcean tokens. + +Player records should store stable IDs and gameplay state, not sensitive +account credentials. + +## Testing Gates + +Minimum persistence smoke test: + +1. Start server. +2. Join as a player. +3. Change survival/inventory state. +4. Place a campfire or primitive shelter. +5. Deplete one resource node if resource persistence is enabled. +6. Save manually. +7. Stop server. +8. Start server. +9. Confirm player/world state restored as expected. + +Minimum migration test: + +1. Load a previous save format fixture. +2. Run migration or compatibility load. +3. Confirm unsupported data is rejected or logged clearly. +4. Confirm supported data survives. + +Minimum tile validation test: + +1. Save active Ground Zero tile ID/package version. +2. Restart server. +3. Confirm the server validates the active package version. +4. Confirm clients can still download the required package from the tile + endpoint. + +## Current Roadmap Decisions + +This document defines: + +- MVP persistence scope; +- save data vs external tile registry boundary; +- player identity save direction; +- player stats save direction; +- player inventory save direction; +- resource depletion persistence rule; +- world time save direction; +- weather seed/state save direction; +- container persistence direction; +- server-side save interval direction; +- load-on-server-start direction; +- initial tile registry persistence direction. + +Implementation work remains tracked separately in the roadmap. + +## Open Questions + +- Should the first playable MVP use `USaveGame`, JSON, or a hybrid save + backend? +- What is the first stable `SaveFormatVersion` value? +- Which resource nodes should persist depletion in Ground Zero? +- Should disconnected players remain in world physically or be removed? +- How much inventory should death/respawn preserve? +- When do saves move from file-based records to database-backed records? +- How should family/generation records attach to player identity later?