Compare commits
20 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 716a5e63da | |||
| ff88b0451a | |||
| 40add02ac2 | |||
| f2227dce21 | |||
| 14f89262ab | |||
| de81084560 | |||
| 4914f6df70 | |||
| 410e6f7536 | |||
| d768737629 | |||
| a97837b832 | |||
| daab30fcd0 | |||
| 8b0fbf831e | |||
| 19712aa7a6 | |||
| ec053c98ab | |||
| 9a737ef57f | |||
| 540b1bc305 | |||
| 05cdb4e2ab | |||
| b0deefddb7 | |||
| c7a6df859d | |||
| 415342b590 |
12
CHANGELOG.md
12
CHANGELOG.md
@@ -1,8 +1,18 @@
|
|||||||
<!-- file: CHANGELOG.md -->
|
<!-- file: CHANGELOG.md -->
|
||||||
<!-- version: 35 -->
|
<!-- version: 36 -->
|
||||||
|
|
||||||
# Changelog KSP
|
# Changelog KSP
|
||||||
|
|
||||||
|
## 0.3.16 — fondations multi-variantes RAW, promotion canonique et quarantaine durable — 2026-09-22
|
||||||
|
|
||||||
|
`0.3.16` stabilise la fondation Store nécessaire à la convergence de plusieurs représentations `RawTransaction` pour une même identité `(network, signature)`. `ksp-store-api` porte désormais le comparateur backend-neutral `Exact / CompatibleLessComplete / CompatibleMoreComplete / Conflict / Incomparable` et les outcomes de variante associés. La dominance reste volontairement fail-closed : seule la troncature `logMessages` explicitement prouvée établit aujourd'hui une relation plus/moins complète ; les autres divergences ne sont jamais fusionnées ou départagées par provider. La migration PostgreSQL V003 est additive et conserve l'historique V000/V001/V002 inchangé ; elle introduit les variantes, le sélecteur canonique, le lien observation-variante et les conflict cases minimaux.
|
||||||
|
|
||||||
|
Le backend PostgreSQL sérialise la décision sous le verrou de l'identité V001. Une représentation `CompatibleMoreComplete` est promue atomiquement tout en conservant l'ancienne variante et en gardant la projection V001 cohérente avec le sélecteur canonique. Une divergence `Conflict` ou `Incomparable` conserve la variante entrante, ouvre ou réouvre un conflict case durable et enregistre l'observation sans muter le canonique. Le Worker ne possède plus d'arbitrage run-local plus fort que le Store : chaque acquisition atteint le Store, `QuarantinedConflict` est un succès non terminal, le Worker reste `Running` avec health `Degraded`, et le Job Backfill projette ce même résultat vers son état `Conflict` tout en conservant l'idempotence de l'observation.
|
||||||
|
|
||||||
|
Le hardening final couvre concurrence, rollback transactionnel, non-régression Store/Backfill/Worker et invariants de sélecteur/projection. La preuve live `postgres_raw_transaction_live` a été exécutée avec succès sur PostgreSQL 17, y compris les scénarios concurrents et récupérables. Les gates opérateur de `pre.008-fix.001` puis de la réconciliation documentaire `pre.009` passent audits Rust/Markdown, `cargo check --workspace`, Clippy workspace/all-targets/all-features avec `-D warnings` et les suites ciblées Store API, Store façade, PostgreSQL, Backfill et Worker sans échec.
|
||||||
|
|
||||||
|
La release se ferme volontairement avant la résolution opérateur complète, la rétention/pinning des variantes, le retry Store/backpressure `Blocked`, la reconnexion Transport configurable et l'UI de résolution Store Desk. `prompts/036-V0_3_17_START_PROMPT.md` ouvre `0.3.17` depuis le futur tag stable `v0.3.16` pour ces responsabilités backend/runtime et le cycle de vie complet des conflits/variantes ; `0.3.18` est réservé à l'inspection/résolution dans `ksp-app-store-desk`, puis le Backfill multi-route/multi-stratégie et son adaptation Desk sont repoussés respectivement à `0.3.19` et `0.3.20`.
|
||||||
|
|
||||||
## 0.3.15 — Raw Transaction Ingest Desk multi-route et fermeture live Mainnet — 2026-09-19
|
## 0.3.15 — Raw Transaction Ingest Desk multi-route et fermeture live Mainnet — 2026-09-19
|
||||||
|
|
||||||
`0.3.15` livre `ksp-app-raw-transaction-ingest-desk` comme Desk Tauri KSP de composition et supervision des routes live `RawTransaction`. La Desk raisonne en routes complètes plutôt qu’en endpoints : une route n’est composable que lorsque toutes ses capabilities Config/Transport requises existent sur le même réseau, puis le Start reconstruit les ressources backend et lance une instance `ksp-worker-raw-transaction-ingest-lib` indépendante par route sélectionnée. Plusieurs routes peuvent partager le même Store sans partager implicitement lifecycle, gaps, `TargetCoverage` ou réparation. L’application expose inventaire, Start/Stop ciblé, lifecycle, health, activité, backpressure, reconnect/replay, gaps/repair et compteurs sûrs sans déplacer Transport, Store ou logique d’ingestion dans le frontend.
|
`0.3.15` livre `ksp-app-raw-transaction-ingest-desk` comme Desk Tauri KSP de composition et supervision des routes live `RawTransaction`. La Desk raisonne en routes complètes plutôt qu’en endpoints : une route n’est composable que lorsque toutes ses capabilities Config/Transport requises existent sur le même réseau, puis le Start reconstruit les ressources backend et lance une instance `ksp-worker-raw-transaction-ingest-lib` indépendante par route sélectionnée. Plusieurs routes peuvent partager le même Store sans partager implicitement lifecycle, gaps, `TargetCoverage` ou réparation. L’application expose inventaire, Start/Stop ciblé, lifecycle, health, activité, backpressure, reconnect/replay, gaps/repair et compteurs sûrs sans déplacer Transport, Store ou logique d’ingestion dans le frontend.
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
# file: Cargo.toml
|
# file: Cargo.toml
|
||||||
# version: 637
|
# version: 651
|
||||||
|
|
||||||
[workspace]
|
[workspace]
|
||||||
resolver = "3"
|
resolver = "3"
|
||||||
members = ["crates/ksp-app-backfill-desk", "crates/ksp-app-config-desk", "crates/ksp-app-raw-transaction-ingest-desk", "crates/ksp-app-solprices-desk", "crates/ksp-app-store-desk", "crates/ksp-app-wallet-desk", "crates/ksp-config-lib", "crates/ksp-core-lib", "crates/ksp-interface-lib", "crates/ksp-job-api", "crates/ksp-job-backfill-lib", "crates/ksp-logging-lib", "crates/ksp-offchain-transport-lib", "crates/ksp-onchain-transport-lib", "crates/ksp-program-api", "crates/ksp-raw-transaction-lib", "crates/ksp-store-api", "crates/ksp-store-lib", "crates/ksp-store-postgres-lib", "crates/ksp-wallet-lib", "crates/ksp-worker-api", "crates/ksp-worker-raw-transaction-ingest-lib"]
|
members = ["crates/ksp-app-backfill-desk", "crates/ksp-app-config-desk", "crates/ksp-app-raw-transaction-ingest-desk", "crates/ksp-app-solprices-desk", "crates/ksp-app-store-desk", "crates/ksp-app-wallet-desk", "crates/ksp-config-lib", "crates/ksp-core-lib", "crates/ksp-interface-lib", "crates/ksp-job-api", "crates/ksp-job-backfill-lib", "crates/ksp-logging-lib", "crates/ksp-offchain-transport-lib", "crates/ksp-onchain-transport-lib", "crates/ksp-program-api", "crates/ksp-raw-transaction-lib", "crates/ksp-store-api", "crates/ksp-store-lib", "crates/ksp-store-postgres-lib", "crates/ksp-wallet-lib", "crates/ksp-worker-api", "crates/ksp-worker-raw-transaction-ingest-lib"]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "0.3.15"
|
version = "0.3.16"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
license = "MIT"
|
license = "MIT"
|
||||||
repository = "https://git.sasedev.com/Sasedev/khadhroony-solana-project"
|
repository = "https://git.sasedev.com/Sasedev/khadhroony-solana-project"
|
||||||
|
|||||||
12
ROADMAP.md
12
ROADMAP.md
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: ROADMAP.md -->
|
<!-- file: ROADMAP.md -->
|
||||||
<!-- version: 114 -->
|
<!-- version: 115 -->
|
||||||
|
|
||||||
# Roadmap KSP
|
# Roadmap KSP
|
||||||
|
|
||||||
@@ -108,9 +108,11 @@ RAW -> STRUCTURAL -> DECODED -> DOMAIN
|
|||||||
- [X] `0.3.13` — Convergence live multi-source du Worker RAW livrée : composition caller-owned `1..32` sources d’un même réseau, cinq familles Yellowstone / Standard Logs / Standard Block / Helius Transaction / HTTP Block Polling, hydration globale des trois voies reference-bearing, RAW-direct Legacy/V0/V1 pour les deux voies blocs, observations multiples, content conflict explicite, coalescence cross-source, fairness/backpressure/health source-neutral et shutdown hardening sans second actor Transport. Gate workspace complet vert (`1 850` PASS / `0` échec / `15` ignored) et smokes keyless HTTP + WebSocket Devnet verts ; les preuves provider/resource-gated non exécutées restent explicitement non revendiquées.
|
- [X] `0.3.13` — Convergence live multi-source du Worker RAW livrée : composition caller-owned `1..32` sources d’un même réseau, cinq familles Yellowstone / Standard Logs / Standard Block / Helius Transaction / HTTP Block Polling, hydration globale des trois voies reference-bearing, RAW-direct Legacy/V0/V1 pour les deux voies blocs, observations multiples, content conflict explicite, coalescence cross-source, fairness/backpressure/health source-neutral et shutdown hardening sans second actor Transport. Gate workspace complet vert (`1 850` PASS / `0` échec / `15` ignored) et smokes keyless HTTP + WebSocket Devnet verts ; les preuves provider/resource-gated non exécutées restent explicitement non revendiquées.
|
||||||
- [X] `0.3.14` — Gap repair/hardening multi-source du Worker RAW livré : gaps run-local bornés, distinction reconnect/replay/delivery/coverage, TargetCoverage conservative, coverage redondante prouvée, discovery HTTP bornée, known-reference hydration, réconciliation source-loss, health gap-aware, fairness nominal/repair, snapshots publics source-neutral et shutdown/drain renforcé. Le pipeline Common RAW/Store reste unique, Worker/Backfill restent indépendants, EARLY n’est pas ajouté. Gate workspace complet vert (`1 931` PASS / `0` échec / `15` ignored) et smokes keyless HTTP + WebSocket Devnet verts ; preuves provider/resource-gated non provisionnées explicitement non revendiquées.
|
- [X] `0.3.14` — Gap repair/hardening multi-source du Worker RAW livré : gaps run-local bornés, distinction reconnect/replay/delivery/coverage, TargetCoverage conservative, coverage redondante prouvée, discovery HTTP bornée, known-reference hydration, réconciliation source-loss, health gap-aware, fairness nominal/repair, snapshots publics source-neutral et shutdown/drain renforcé. Le pipeline Common RAW/Store reste unique, Worker/Backfill restent indépendants, EARLY n’est pas ajouté. Gate workspace complet vert (`1 931` PASS / `0` échec / `15` ignored) et smokes keyless HTTP + WebSocket Devnet verts ; preuves provider/resource-gated non provisionnées explicitement non revendiquées.
|
||||||
- [X] `0.3.15` — `ksp-app-raw-transaction-ingest-desk` livrée comme Desk Tauri KSP multi-route : composition capability-driven par réseau, une instance Worker indépendante par route sélectionnée, Store partagé, Start/Stop ciblé et supervision lifecycle/health/backpressure/reconnect/replay/gaps/repair sans réimplémenter Transport ni persistence. Les cinq familles live sont projetées selon les capacités réellement configurées ; Yellowstone Mainnet utilise le trigger gRPC Block + HTTP `getBlock` et applique une backpressure bornée sans overflow local prolongé. Le Stop Yellowstone post-half-close est coopératif, et Store accepte le cas strict `canonique complet + incoming logMessages explicitement tronqué compatible` sans remplacer le canonique ni arrêter le Worker. Gate workspace/Clippy/tests/graphes/bundles Linux vert et live Mainnet Yellowstone + HTTP polling d’environ 19 minutes fermé `Stopped/Healthy`. La généralisation variantes/conflits/promotions/retry/reconnect reste explicitement `0.3.16`.
|
- [X] `0.3.15` — `ksp-app-raw-transaction-ingest-desk` livrée comme Desk Tauri KSP multi-route : composition capability-driven par réseau, une instance Worker indépendante par route sélectionnée, Store partagé, Start/Stop ciblé et supervision lifecycle/health/backpressure/reconnect/replay/gaps/repair sans réimplémenter Transport ni persistence. Les cinq familles live sont projetées selon les capacités réellement configurées ; Yellowstone Mainnet utilise le trigger gRPC Block + HTTP `getBlock` et applique une backpressure bornée sans overflow local prolongé. Le Stop Yellowstone post-half-close est coopératif, et Store accepte le cas strict `canonique complet + incoming logMessages explicitement tronqué compatible` sans remplacer le canonique ni arrêter le Worker. Gate workspace/Clippy/tests/graphes/bundles Linux vert et live Mainnet Yellowstone + HTTP polling d’environ 19 minutes fermé `Stopped/Healthy`. La généralisation variantes/conflits/promotions/retry/reconnect reste explicitement `0.3.16`.
|
||||||
- [ ] `0.3.16` — **RAW resilience / conflict management** : faire converger Store API/façade/PostgreSQL, Worker live et Store Desk autour de variantes RAW durables, résolution automatique `Exact / CompatibleLessComplete / CompatibleMoreComplete / Conflict`, quarantaine des conflits non résolus sans arrêt du Worker, historique réversible des promotions/résolutions, retry Store sans perte silencieuse et reconnexion Transport configurable/bornée. Une provenance provider n'est jamais une priorité canonique en soi ; la représentation prouvée la plus complète gagne.
|
- [X] `0.3.16` — **Fondations RAW multi-variantes et conflits durables** livrées : V003 additive avec variantes/sélecteur/mappings/conflict cases, comparateur backend-neutral fail-closed `Exact / CompatibleLessComplete / CompatibleMoreComplete / Conflict / Incomparable`, promotion canonique atomique de la représentation strictement prouvée plus complète, conservation de l'ancien canonique et quarantaine durable des divergences. Le Worker transmet toute acquisition au Store et traite `QuarantinedConflict` comme succès non terminal `Running/Degraded`; le Backfill conserve une projection `Conflict`. Hardening concurrence/rollback vert et preuve PostgreSQL 17 réelle passée. Résolution complète, rétention/pins, retry Store, reconnect configurable et UI sont explicitement reportés.
|
||||||
- [ ] `0.3.17` — Étendre `ksp-job-backfill-lib` au **backfill multi-route/multi-stratégie** à partir de la matrice d'acquisition auditée en `0.3.9` et de la convergence Store `0.3.16`. Conserver `getSignaturesForAddress + getTransaction` comme première voie valide puis ajouter les stratégies historiques/catch-up réellement pertinentes sans edge Job ↔ Worker.
|
- [ ] `0.3.17` — **Résilience opérationnelle et cycle de vie des variantes** : contrats Store API backend-neutral d'inspection/résolution et classification `Transient/Terminal`, conflict cases complets avec participants/historique/reopen/races, rétention/pins/purge guards, façade Store d'inspection/actions, retry Store borné avec état `Blocked`, puis reconnexion WebSocket/Yellowstone configurable sans inventer de preuve de coverage.
|
||||||
- [ ] `0.3.18` — Adapter `ksp-app-backfill-desk` au Job multi-route `0.3.17` : inventaire/composition de routes, sélection/supervision et progression sur le modèle structurel de `ksp-app-raw-transaction-ingest-desk`, sans réimplémenter la logique du Job.
|
- [ ] `0.3.18` — **Store Desk conflits/variantes** : exposer via `ksp-store-lib` uniquement les listes, détails, historique et actions opérateur `promote/keep/restore/reopen` avec revision attendue, pagination bornée, stale-action explicite, redaction et preuves Tauri/end-to-end pertinentes.
|
||||||
|
- [ ] `0.3.19` — Étendre `ksp-job-backfill-lib` au **backfill multi-route/multi-stratégie** à partir de la matrice d'acquisition auditée en `0.3.9` et des contrats Store stabilisés en `0.3.16`–`0.3.18`, sans edge Job ↔ Worker.
|
||||||
|
- [ ] `0.3.20` — Adapter `ksp-app-backfill-desk` au Job multi-route `0.3.19` : inventaire/composition de routes, sélection/supervision et progression sur le modèle structurel de `ksp-app-raw-transaction-ingest-desk`, sans réimplémenter la logique du Job.
|
||||||
|
|
||||||
### TODO/IDEAS — applications spécialisées et control plane
|
### TODO/IDEAS — applications spécialisées et control plane
|
||||||
|
|
||||||
@@ -120,7 +122,7 @@ RAW -> STRUCTURAL -> DECODED -> DOMAIN
|
|||||||
### TODO/IDEAS — taxonomie D1, processing et rétention
|
### TODO/IDEAS — taxonomie D1, processing et rétention
|
||||||
|
|
||||||
- [ ] **TODO** — maintenir la matrice d’admission HTTP/WS/gRPC/provider lors de toute nouvelle famille D1 : plusieurs sources ne convergent vers un même struct que si elles satisfont la même sémantique sans perte.
|
- [ ] **TODO** — maintenir la matrice d’admission HTTP/WS/gRPC/provider lors de toute nouvelle famille D1 : plusieurs sources ne convergent vers un même struct que si elles satisfont la même sémantique sans perte.
|
||||||
- [X] **TODO `0.3.9`** — matrice RAW Transaction unique produite et consolidée dans `docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md`, avec séparation possibilités/support/preuve, capabilities orthogonales aux producteurs, provenance/déduplication/continuité et handoffs désormais scindés `0.3.11`–`0.3.14` Worker live / `0.3.16` résilience RAW / `0.3.17`–`0.3.18` Backfill multi-route.
|
- [X] **TODO `0.3.9`** — matrice RAW Transaction unique produite et consolidée dans `docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md`, avec séparation possibilités/support/preuve, capabilities orthogonales aux producteurs, provenance/déduplication/continuité et handoffs désormais scindés `0.3.11`–`0.3.14` Worker live / `0.3.16` fondations de résilience RAW / `0.3.17` résilience opérationnelle + cycle de vie des variantes / `0.3.18` Store Desk conflits / `0.3.19`–`0.3.20` Backfill multi-route et Desk.
|
||||||
- [X] **TODO Helius `0.3.13`** — la voie Worker Helius `transactionSubscribe` réutilise la surface Transport et les profils Config Helius déjà existants avec `KSP_SECRET_HELIUS_API_KEY`; aucun nouveau profil, secret, SDK provider, tier codé ou second client parallèle n’a été nécessaire. Le payload riche reste Transport-owned et la voie Worker hydrate via `getTransaction observed` tant que le RAW complet n’est pas prouvé directement.
|
- [X] **TODO Helius `0.3.13`** — la voie Worker Helius `transactionSubscribe` réutilise la surface Transport et les profils Config Helius déjà existants avec `KSP_SECRET_HELIUS_API_KEY`; aucun nouveau profil, secret, SDK provider, tier codé ou second client parallèle n’a été nécessaire. Le payload riche reste Transport-owned et la voie Worker hydrate via `getTransaction observed` tant que le RAW complet n’est pas prouvé directement.
|
||||||
- [X] **TODO réseau** — identité KSP canonique fixée à `mainnet` dans Config/Store/Transport/tests ; `mainnet-beta` reste uniquement un alias legacy/externe lorsque la frontière provider l'exige. Les données D1 RAW antérieures restent non autoritaires pendant cette phase et aucune compatibilité de base de test n'impose l'ancien libellé.
|
- [X] **TODO réseau** — identité KSP canonique fixée à `mainnet` dans Config/Store/Transport/tests ; `mainnet-beta` reste uniquement un alias legacy/externe lorsque la frontière provider l'exige. Les données D1 RAW antérieures restent non autoritaires pendant cette phase et aucune compatibilité de base de test n'impose l'ancien libellé.
|
||||||
- [X] `RawAccountState` + observation — contrat commun stabilisé en `0.3.1` avec bytes complets + slot, provenance séparée et enrichissements source-specific optionnels ; la persistence PostgreSQL physique est complétée en `0.3.4` avec les quatre capabilities account et la conformance RAW 10/10.
|
- [X] `RawAccountState` + observation — contrat commun stabilisé en `0.3.1` avec bytes complets + slot, provenance séparée et enrichissements source-specific optionnels ; la persistence PostgreSQL physique est complétée en `0.3.4` avec les quatre capabilities account et la conformance RAW 10/10.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-job-backfill-lib/README.md -->
|
<!-- file: crates/ksp-job-backfill-lib/README.md -->
|
||||||
<!-- version: 2 -->
|
<!-- version: 3 -->
|
||||||
|
|
||||||
# ksp-job-backfill-lib
|
# ksp-job-backfill-lib
|
||||||
|
|
||||||
@@ -56,7 +56,7 @@ ksp-job-backfill-lib
|
|||||||
-X-> backend imposé par le job
|
-X-> backend imposé par le job
|
||||||
```
|
```
|
||||||
|
|
||||||
Le chemin d'écriture est l'acquisition atomique `RawTransaction + RawTransactionObservation` en mode normal. La crate distingue insertion, déjà présent, tombstone purgé, observation nouvelle/existante, missing et conflit. Un conflit de contenu n'est jamais converti en succès idempotent.
|
Le chemin d'écriture est l'acquisition atomique `RawTransaction + RawTransactionObservation` en mode normal. La crate distingue insertion, déjà présent, tombstone purgé, observation nouvelle/existante, missing et conflit. Depuis `0.3.16`, un outcome Store `QuarantinedConflict` est projeté en `BackfillEntityPersistence::Conflict` tout en conservant l'outcome réel de l'observation (`Inserted` ou `AlreadyPresent`) ; il n'est jamais aplati en simple succès idempotent. Le code d'erreur legacy `ERROR_CODE_RAW_CONFLICT` reste pris en charge pour compatibilité.
|
||||||
|
|
||||||
## Concurrence et checkpoint
|
## Concurrence et checkpoint
|
||||||
|
|
||||||
@@ -112,7 +112,7 @@ La crate ne possède pas :
|
|||||||
- Worker API / worker live ;
|
- Worker API / worker live ;
|
||||||
- retry, pacing ou sélection d'endpoint Transport ;
|
- retry, pacing ou sélection d'endpoint Transport ;
|
||||||
- backend Store concret ;
|
- backend Store concret ;
|
||||||
- decoding Program ou matérialisation CORE/DECODE/SPECIALIZED ;
|
- decoding Program ou matérialisation STRUCTURAL/DECODED/DOMAIN ;
|
||||||
- table dédiée de checkpoint ;
|
- table dédiée de checkpoint ;
|
||||||
- control plane Job générique.
|
- control plane Job générique.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-job-backfill-lib/USAGE.md -->
|
<!-- file: crates/ksp-job-backfill-lib/USAGE.md -->
|
||||||
<!-- version: 2 -->
|
<!-- version: 3 -->
|
||||||
|
|
||||||
# Utilisation de ksp-job-backfill-lib
|
# Utilisation de ksp-job-backfill-lib
|
||||||
|
|
||||||
@@ -191,7 +191,7 @@ entity: Inserted | AlreadyPresent | SkippedPurged | Conflict
|
|||||||
observation: Inserted | AlreadyPresent | NotRecorded | NotApplicable
|
observation: Inserted | AlreadyPresent | NotRecorded | NotApplicable
|
||||||
```
|
```
|
||||||
|
|
||||||
`Missing` ne provoque aucune écriture Store. Un conflit de contenu reste observable comme conflit et ne doit pas être traité comme une relance idempotente réussie.
|
`Missing` ne provoque aucune écriture Store. Un `QuarantinedConflict` durable reste observable comme `Conflict`, mais l'outcome de l'observation Store est conservé : une réobservation peut donc produire `Conflict + AlreadyPresent`. Ce cas ne doit jamais être aplati en simple relance idempotente réussie. Un backend legacy qui renvoie encore le code Store de conflit est également projeté comme conflit de compatibilité.
|
||||||
|
|
||||||
## Frontières à respecter
|
## Frontières à respecter
|
||||||
|
|
||||||
@@ -203,4 +203,4 @@ Le caller ne doit pas :
|
|||||||
- utiliser provider/endpoint comme identité transactionnelle ;
|
- utiliser provider/endpoint comme identité transactionnelle ;
|
||||||
- inventer une provenance pour `getTransaction = null` ;
|
- inventer une provenance pour `getTransaction = null` ;
|
||||||
- interpréter un checkpoint caller-owned comme une garantie de persistence crash-safe automatique ;
|
- interpréter un checkpoint caller-owned comme une garantie de persistence crash-safe automatique ;
|
||||||
- utiliser le job RAW comme decoder Program ou processor CORE.
|
- utiliser le job RAW comme decoder Program ou processor STRUCTURAL.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-job-backfill-lib/src/persistence.rs
|
// file: crates/ksp-job-backfill-lib/src/persistence.rs
|
||||||
// version: 3
|
// version: 4
|
||||||
|
|
||||||
/// Canonical entity disposition produced by one Backfill Store persistence attempt.
|
/// Canonical entity disposition produced by one Backfill Store persistence attempt.
|
||||||
#[non_exhaustive]
|
#[non_exhaustive]
|
||||||
@@ -13,7 +13,7 @@ pub enum BackfillEntityPersistence {
|
|||||||
SkippedPurged,
|
SkippedPurged,
|
||||||
/// `getTransaction` returned JSON `null`, so no canonical Store write was attempted.
|
/// `getTransaction` returned JSON `null`, so no canonical Store write was attempted.
|
||||||
Missing,
|
Missing,
|
||||||
/// Store reported divergent canonical content for the same network-scoped transaction identity.
|
/// Store durably quarantined divergent/incomparable content, or a legacy backend reported a conflict error.
|
||||||
Conflict,
|
Conflict,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -25,7 +25,7 @@ pub enum BackfillObservationPersistence {
|
|||||||
Inserted,
|
Inserted,
|
||||||
/// The same deterministic acquisition observation was already durable.
|
/// The same deterministic acquisition observation was already durable.
|
||||||
AlreadyPresent,
|
AlreadyPresent,
|
||||||
/// Store intentionally recorded no observation because normal persistence skipped a purged entity or detected a conflict.
|
/// Store intentionally recorded no observation because normal persistence skipped a purged entity or a legacy conflict error predated durable quarantine.
|
||||||
NotRecorded,
|
NotRecorded,
|
||||||
/// No observation existed because hydration returned `Missing` before persistence.
|
/// No observation existed because hydration returned `Missing` before persistence.
|
||||||
NotApplicable,
|
NotApplicable,
|
||||||
@@ -75,9 +75,9 @@ impl crate::BackfillPersistenceOutcome {
|
|||||||
///
|
///
|
||||||
/// Available acquisitions use the existing atomic transaction-plus-observation Store contract in
|
/// Available acquisitions use the existing atomic transaction-plus-observation Store contract in
|
||||||
/// `Normal` mode. A durable purge is therefore respected and never rehydrated implicitly.
|
/// `Normal` mode. A durable purge is therefore respected and never rehydrated implicitly.
|
||||||
/// `Missing` performs no Store write. Stable Store content conflicts are projected explicitly as
|
/// `Missing` performs no Store write. Durable quarantined conflicts are projected explicitly as
|
||||||
/// [`BackfillEntityPersistence::Conflict`] rather than being silently treated as idempotent skips.
|
/// [`BackfillEntityPersistence::Conflict`] while preserving the actual observation disposition;
|
||||||
/// Other Store failures remain errors.
|
/// legacy Store conflict errors remain compatible and other Store failures remain errors.
|
||||||
pub async fn persist_backfill_hydration(
|
pub async fn persist_backfill_hydration(
|
||||||
store: &ksp_store_lib::Store,
|
store: &ksp_store_lib::Store,
|
||||||
hydration: crate::BackfillHydrationOutcome,
|
hydration: crate::BackfillHydrationOutcome,
|
||||||
@@ -170,6 +170,17 @@ fn map_store_outcome(
|
|||||||
) -> ksp_core_lib::Result<crate::BackfillPersistenceOutcome> {
|
) -> ksp_core_lib::Result<crate::BackfillPersistenceOutcome> {
|
||||||
let entity = outcome.entity();
|
let entity = outcome.entity();
|
||||||
let observation = outcome.observation();
|
let observation = outcome.observation();
|
||||||
|
if outcome.transaction_variant() == std::option::Option::Some(ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict) {
|
||||||
|
let projected_observation = match observation {
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::Inserted => crate::BackfillObservationPersistence::Inserted,
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent => crate::BackfillObservationPersistence::AlreadyPresent,
|
||||||
|
_ => return std::result::Result::Err(persistence_error("store.conflict_observation")),
|
||||||
|
};
|
||||||
|
if entity != ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent {
|
||||||
|
return std::result::Result::Err(persistence_error("store.conflict_entity"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(crate::BackfillPersistenceOutcome::new(reference, crate::BackfillEntityPersistence::Conflict, projected_observation));
|
||||||
|
}
|
||||||
if entity == ksp_store_lib::RawEntityWriteOutcome::Inserted && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
if entity == ksp_store_lib::RawEntityWriteOutcome::Inserted && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
||||||
return std::result::Result::Ok(crate::BackfillPersistenceOutcome::new(
|
return std::result::Result::Ok(crate::BackfillPersistenceOutcome::new(
|
||||||
reference,
|
reference,
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-job-backfill-lib/unit_tests/persistence.rs
|
// file: crates/ksp-job-backfill-lib/unit_tests/persistence.rs
|
||||||
// version: 1
|
// version: 3
|
||||||
|
|
||||||
#[derive(Clone, Copy)]
|
#[derive(Clone, Copy)]
|
||||||
enum FakeResponse {
|
enum FakeResponse {
|
||||||
@@ -143,6 +143,14 @@ fn store_outcome(
|
|||||||
return ksp_store_lib::RawAcquisitionWriteOutcome::new(entity, observation);
|
return ksp_store_lib::RawAcquisitionWriteOutcome::new(entity, observation);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn store_variant_outcome(
|
||||||
|
entity: ksp_store_lib::RawEntityWriteOutcome,
|
||||||
|
observation: ksp_store_lib::RawObservationWriteOutcome,
|
||||||
|
transaction_variant: ksp_store_lib::RawTransactionVariantWriteOutcome,
|
||||||
|
) -> ksp_store_lib::RawAcquisitionWriteOutcome {
|
||||||
|
return ksp_store_lib::RawAcquisitionWriteOutcome::with_transaction_variant(entity, observation, transaction_variant);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn pre_007_missing_skips_store_and_preserves_network_scoped_identity() {
|
async fn pre_007_missing_skips_store_and_preserves_network_scoped_identity() {
|
||||||
let reference = match raw_reference("devnet", 1) {
|
let reference = match raw_reference("devnet", 1) {
|
||||||
@@ -266,7 +274,58 @@ async fn pre_007_normal_backfill_respects_purged_tombstone_without_observation()
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn pre_007_store_content_conflict_is_explicit_and_not_idempotent_success() {
|
async fn v0_3_16_pre_007_durable_quarantined_conflict_preserves_observation_projection() {
|
||||||
|
let (reference, transaction, observation) = match raw_acquisition_parts("devnet", 6, 16) {
|
||||||
|
std::option::Option::Some(parts) => parts,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let network = match raw_network("devnet") {
|
||||||
|
std::option::Option::Some(network) => network,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let response = store_variant_outcome(
|
||||||
|
ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::Inserted,
|
||||||
|
ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
|
);
|
||||||
|
let port = FakePersistencePort::new(network, &[FakeResponse::Outcome(response)]);
|
||||||
|
let result = super::persist_available_with_port(&port, reference, transaction, observation).await;
|
||||||
|
assert!(result.is_ok());
|
||||||
|
if let std::result::Result::Ok(result) = result {
|
||||||
|
assert_eq!(result.entity(), crate::BackfillEntityPersistence::Conflict);
|
||||||
|
assert_eq!(result.observation(), crate::BackfillObservationPersistence::Inserted);
|
||||||
|
assert_ne!(result.entity(), crate::BackfillEntityPersistence::AlreadyPresent);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn v0_3_16_pre_008_reobserved_quarantined_conflict_preserves_idempotent_observation() {
|
||||||
|
let (reference, transaction, observation) = match raw_acquisition_parts("devnet", 6, 18) {
|
||||||
|
std::option::Option::Some(parts) => parts,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let network = match raw_network("devnet") {
|
||||||
|
std::option::Option::Some(network) => network,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let response = store_variant_outcome(
|
||||||
|
ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
|
);
|
||||||
|
let port = FakePersistencePort::new(network, &[FakeResponse::Outcome(response)]);
|
||||||
|
let result = super::persist_available_with_port(&port, reference, transaction, observation).await;
|
||||||
|
assert!(result.is_ok());
|
||||||
|
if let std::result::Result::Ok(result) = result {
|
||||||
|
assert_eq!(result.entity(), crate::BackfillEntityPersistence::Conflict);
|
||||||
|
assert_eq!(result.observation(), crate::BackfillObservationPersistence::AlreadyPresent);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn pre_007_legacy_store_content_conflict_error_is_explicit_and_not_idempotent_success() {
|
||||||
let (reference, transaction, observation) = match raw_acquisition_parts("devnet", 6, 16) {
|
let (reference, transaction, observation) = match raw_acquisition_parts("devnet", 6, 16) {
|
||||||
std::option::Option::Some(parts) => parts,
|
std::option::Option::Some(parts) => parts,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/src/capability/raw_transaction.rs
|
// file: crates/ksp-store-api/src/capability/raw_transaction.rs
|
||||||
// version: 4
|
// version: 5
|
||||||
|
|
||||||
/// Read capability for canonical RAW transactions.
|
/// Read capability for canonical RAW transactions.
|
||||||
///
|
///
|
||||||
@@ -42,8 +42,9 @@ pub trait RawTransactionInspectionRead: std::marker::Send + std::marker::Sync {
|
|||||||
///
|
///
|
||||||
/// The transaction and its acquisition observation form one logical persistence
|
/// The transaction and its acquisition observation form one logical persistence
|
||||||
/// operation. An implementation must not leave one side durable if the other
|
/// operation. An implementation must not leave one side durable if the other
|
||||||
/// side fails. Divergent content for one logical identity is a stable conflict
|
/// side fails. Divergent or incomparable content must be preserved as a durable
|
||||||
/// error, never a silent overwrite.
|
/// variant/conflict outcome rather than silently overwriting the canonical transaction
|
||||||
|
/// or being projected as a terminal persistence failure.
|
||||||
pub trait RawTransactionWrite: std::marker::Send + std::marker::Sync {
|
pub trait RawTransactionWrite: std::marker::Send + std::marker::Sync {
|
||||||
/// Persists one complete RAW transaction together with one observation atomically.
|
/// Persists one complete RAW transaction together with one observation atomically.
|
||||||
fn persist_raw_transaction_acquisition<'a>(
|
fn persist_raw_transaction_acquisition<'a>(
|
||||||
|
|||||||
@@ -1,10 +1,8 @@
|
|||||||
// file: crates/ksp-store-api/src/lib.rs
|
// file: crates/ksp-store-api/src/lib.rs
|
||||||
// version: 7
|
// version: 9
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
#![forbid(unsafe_code)]
|
#![forbid(unsafe_code)]
|
||||||
|
|
||||||
//! Backend-agnostic persistence contracts for KSP Store implementations.
|
//! Backend-agnostic persistence contracts for KSP Store implementations.
|
||||||
//!
|
//!
|
||||||
//! `ksp-store-api` owns persistent models plus the contracts that operate on
|
//! `ksp-store-api` owns persistent models plus the contracts that operate on
|
||||||
@@ -15,11 +13,9 @@
|
|||||||
//! Models and capabilities deliberately have separate private module homes.
|
//! Models and capabilities deliberately have separate private module homes.
|
||||||
//! Backend implementations, SQL, migrations, Config, Transport and runtime
|
//! Backend implementations, SQL, migrations, Config, Transport and runtime
|
||||||
//! dispatch remain outside this crate.
|
//! dispatch remain outside this crate.
|
||||||
|
|
||||||
mod capability;
|
mod capability;
|
||||||
mod error;
|
mod error;
|
||||||
mod model;
|
mod model;
|
||||||
|
|
||||||
/// Boxed async operation returned by object-safe Store capability contracts.
|
/// Boxed async operation returned by object-safe Store capability contracts.
|
||||||
pub use self::capability::StoreApiFuture;
|
pub use self::capability::StoreApiFuture;
|
||||||
/// Read capability for random-access RAW account-observation inspection.
|
/// Read capability for random-access RAW account-observation inspection.
|
||||||
@@ -94,6 +90,8 @@ pub use self::model::raw_outcome::RawAcquisitionWriteOutcome;
|
|||||||
pub use self::model::raw_outcome::RawEntityWriteOutcome;
|
pub use self::model::raw_outcome::RawEntityWriteOutcome;
|
||||||
/// Outcome for one deterministic acquisition observation write.
|
/// Outcome for one deterministic acquisition observation write.
|
||||||
pub use self::model::raw_outcome::RawObservationWriteOutcome;
|
pub use self::model::raw_outcome::RawObservationWriteOutcome;
|
||||||
|
/// Durable Store outcome for one variant-aware RAW transaction acquisition.
|
||||||
|
pub use self::model::raw_outcome::RawTransactionVariantWriteOutcome;
|
||||||
/// Maximum opaque query cursor length admitted by the Store API.
|
/// Maximum opaque query cursor length admitted by the Store API.
|
||||||
pub use self::model::raw_pagination::MAX_RAW_PAGE_CURSOR_BYTES;
|
pub use self::model::raw_pagination::MAX_RAW_PAGE_CURSOR_BYTES;
|
||||||
/// Backend-independent list query for complete canonical RAW account states.
|
/// Backend-independent list query for complete canonical RAW account states.
|
||||||
@@ -154,10 +152,26 @@ pub use self::model::raw_retention::RawTransactionRetentionTransition;
|
|||||||
pub use self::model::raw_retention::RawTransactionTombstone;
|
pub use self::model::raw_retention::RawTransactionTombstone;
|
||||||
/// Canonical source-independent N1 RAW transaction persisted by Store backends.
|
/// Canonical source-independent N1 RAW transaction persisted by Store backends.
|
||||||
pub use self::model::raw_transaction::RawTransaction;
|
pub use self::model::raw_transaction::RawTransaction;
|
||||||
|
/// Durable lifecycle status of one RAW transaction conflict case.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionConflictStatus;
|
||||||
/// Persistable acquisition observation linked to one canonical RAW transaction.
|
/// Persistable acquisition observation linked to one canonical RAW transaction.
|
||||||
pub use self::model::raw_transaction::RawTransactionObservation;
|
pub use self::model::raw_transaction::RawTransactionObservation;
|
||||||
/// Durable backend-independent identity of one canonical RAW transaction.
|
/// Durable backend-independent identity of one canonical RAW transaction.
|
||||||
pub use self::model::raw_transaction::RawTransactionReference;
|
pub use self::model::raw_transaction::RawTransactionReference;
|
||||||
|
/// Validated backend-neutral result of comparing canonical and incoming RAW transaction variants.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantComparison;
|
||||||
|
/// Stable non-zero Store identity of one persisted RAW transaction variant.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantId;
|
||||||
|
/// Origin of one persisted RAW transaction variant.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantOrigin;
|
||||||
|
/// Durable reference to one persisted RAW transaction variant.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantReference;
|
||||||
|
/// Backend-neutral quality relation between canonical and incoming RAW transaction variants.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantRelation;
|
||||||
|
/// Stable backend-neutral reason explaining one RAW transaction variant relation.
|
||||||
|
pub use self::model::raw_transaction::RawTransactionVariantRelationReason;
|
||||||
|
/// Compares one incoming RAW transaction variant against the currently canonical variant.
|
||||||
|
pub use self::model::raw_transaction::compare_raw_transaction_variants;
|
||||||
/// Common KSP error type used by Store-facing contracts.
|
/// Common KSP error type used by Store-facing contracts.
|
||||||
pub use ksp_core_lib::Error;
|
pub use ksp_core_lib::Error;
|
||||||
/// Stable structured code identifying a KSP error category and condition.
|
/// Stable structured code identifying a KSP error category and condition.
|
||||||
|
|||||||
@@ -1,6 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/src/model/raw_outcome.rs
|
// file: crates/ksp-store-api/src/model/raw_outcome.rs
|
||||||
// version: 1
|
// version: 4
|
||||||
|
|
||||||
/// Outcome for one canonical RAW entity in an idempotent persistence operation.
|
/// Outcome for one canonical RAW entity in an idempotent persistence operation.
|
||||||
#[non_exhaustive]
|
#[non_exhaustive]
|
||||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
@@ -32,13 +31,24 @@ pub enum RawObservationWriteOutcome {
|
|||||||
pub struct RawAcquisitionWriteOutcome {
|
pub struct RawAcquisitionWriteOutcome {
|
||||||
entity: crate::RawEntityWriteOutcome,
|
entity: crate::RawEntityWriteOutcome,
|
||||||
observation: crate::RawObservationWriteOutcome,
|
observation: crate::RawObservationWriteOutcome,
|
||||||
|
transaction_variant: std::option::Option<crate::RawTransactionVariantWriteOutcome>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl RawAcquisitionWriteOutcome {
|
impl RawAcquisitionWriteOutcome {
|
||||||
/// Creates one backend-independent atomic acquisition outcome.
|
/// Creates one backend-independent atomic acquisition outcome.
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub const fn new(entity: crate::RawEntityWriteOutcome, observation: crate::RawObservationWriteOutcome) -> Self {
|
pub const fn new(entity: crate::RawEntityWriteOutcome, observation: crate::RawObservationWriteOutcome) -> Self {
|
||||||
return Self { entity, observation };
|
return Self { entity, observation, transaction_variant: std::option::Option::None };
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Creates one backend-independent atomic RAW transaction acquisition outcome with variant convergence classification.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn with_transaction_variant(
|
||||||
|
entity: crate::RawEntityWriteOutcome,
|
||||||
|
observation: crate::RawObservationWriteOutcome,
|
||||||
|
transaction_variant: crate::RawTransactionVariantWriteOutcome,
|
||||||
|
) -> Self {
|
||||||
|
return Self { entity, observation, transaction_variant: std::option::Option::Some(transaction_variant) };
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Returns the canonical RAW entity write outcome.
|
/// Returns the canonical RAW entity write outcome.
|
||||||
@@ -52,4 +62,34 @@ impl RawAcquisitionWriteOutcome {
|
|||||||
pub const fn observation(&self) -> crate::RawObservationWriteOutcome {
|
pub const fn observation(&self) -> crate::RawObservationWriteOutcome {
|
||||||
return self.observation;
|
return self.observation;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Returns the variant-aware transaction outcome when the persistence operation was a RAW transaction acquisition.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn transaction_variant(&self) -> std::option::Option<crate::RawTransactionVariantWriteOutcome> {
|
||||||
|
return self.transaction_variant;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durable Store outcome for one RAW transaction acquisition after variant-aware convergence.
|
||||||
|
///
|
||||||
|
/// Every variant represents a successful Store-domain result. In particular,
|
||||||
|
/// [`Self::QuarantinedConflict`] is durable success and must not be reinterpreted as a
|
||||||
|
/// transport failure or terminal persistence error by callers.
|
||||||
|
#[non_exhaustive]
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub enum RawTransactionVariantWriteOutcome {
|
||||||
|
/// First durable variant became the initial canonical transaction.
|
||||||
|
InsertedCanonical,
|
||||||
|
/// Incoming content exactly matched an already durable variant/canonical representation.
|
||||||
|
ObservedExact,
|
||||||
|
/// Incoming content was durably observed while the current canonical variant remained strictly more complete.
|
||||||
|
ObservedCompatibleLessComplete,
|
||||||
|
/// Incoming content was durably persisted and atomically promoted because it was strictly more complete.
|
||||||
|
PromotedCompatibleMoreComplete,
|
||||||
|
/// Divergent or incomparable content was durably preserved without changing the current canonical variant.
|
||||||
|
QuarantinedConflict,
|
||||||
|
/// Explicit force-rehydration restored a previously purged logical transaction into variant-aware storage.
|
||||||
|
Rehydrated,
|
||||||
|
/// Normal acquisition respected an existing purged tombstone and performed no rehydration.
|
||||||
|
SkippedPurged,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/src/model/raw_transaction.rs
|
// file: crates/ksp-store-api/src/model/raw_transaction.rs
|
||||||
// version: 1
|
// version: 5
|
||||||
|
|
||||||
/// Durable backend-independent identity of one canonical RAW transaction.
|
/// Durable backend-independent identity of one canonical RAW transaction.
|
||||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||||
@@ -28,6 +28,247 @@ impl RawTransactionReference {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Stable non-zero Store identity of one persisted RAW transaction variant.
|
||||||
|
///
|
||||||
|
/// The identifier is a backend-neutral surrogate. It deliberately does not encode
|
||||||
|
/// `content_hash`, provider identity, canonicality or any physical database key layout.
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
|
||||||
|
pub struct RawTransactionVariantId(u64);
|
||||||
|
|
||||||
|
impl RawTransactionVariantId {
|
||||||
|
/// Creates one non-zero stable variant identifier.
|
||||||
|
pub fn try_new(value: u64) -> crate::Result<Self> {
|
||||||
|
if value == 0 {
|
||||||
|
return std::result::Result::Err(raw_variant_model_error("variant_id"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(Self(value));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the opaque numeric surrogate unchanged.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn get(&self) -> u64 {
|
||||||
|
return self.0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durable reference to one persisted RAW transaction variant.
|
||||||
|
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub struct RawTransactionVariantReference {
|
||||||
|
transaction: crate::RawTransactionReference,
|
||||||
|
variant_id: crate::RawTransactionVariantId,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionVariantReference {
|
||||||
|
/// Creates one durable variant reference from logical transaction identity and surrogate id.
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(transaction: crate::RawTransactionReference, variant_id: crate::RawTransactionVariantId) -> Self {
|
||||||
|
return Self { transaction, variant_id };
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the logical transaction identity owning this variant.
|
||||||
|
#[must_use]
|
||||||
|
pub fn transaction(&self) -> &crate::RawTransactionReference {
|
||||||
|
return &self.transaction;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the stable surrogate variant identifier.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn variant_id(&self) -> crate::RawTransactionVariantId {
|
||||||
|
return self.variant_id;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Origin of one persisted RAW transaction variant.
|
||||||
|
#[non_exhaustive]
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub enum RawTransactionVariantOrigin {
|
||||||
|
/// Exact canonical RAW representation received from one acquisition source.
|
||||||
|
Native,
|
||||||
|
/// Explicitly synthesized canonical RAW representation derived from preserved parent variants.
|
||||||
|
Synthetic,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionVariantOrigin {
|
||||||
|
/// Returns the stable persistence code for this origin.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn code(&self) -> &'static str {
|
||||||
|
return match self {
|
||||||
|
Self::Native => "native",
|
||||||
|
Self::Synthetic => "synthetic",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Backend-neutral quality relation between the current canonical RAW transaction and one incoming variant.
|
||||||
|
#[non_exhaustive]
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub enum RawTransactionVariantRelation {
|
||||||
|
/// Canonical transaction content is exactly equal after exact comparison.
|
||||||
|
Exact,
|
||||||
|
/// Incoming content is compatible but provably less complete than the current canonical variant.
|
||||||
|
CompatibleLessComplete,
|
||||||
|
/// Incoming content is compatible and provably more complete than the current canonical variant.
|
||||||
|
CompatibleMoreComplete,
|
||||||
|
/// A canonical transaction invariant is contradicted by the incoming variant.
|
||||||
|
Conflict,
|
||||||
|
/// No contradiction must be asserted, but no safe dominance relation is proved.
|
||||||
|
Incomparable,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionVariantRelation {
|
||||||
|
/// Returns the stable persistence/diagnostic code for this relation.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn code(&self) -> &'static str {
|
||||||
|
return match self {
|
||||||
|
Self::Exact => "exact",
|
||||||
|
Self::CompatibleLessComplete => "compatible_less_complete",
|
||||||
|
Self::CompatibleMoreComplete => "compatible_more_complete",
|
||||||
|
Self::Conflict => "conflict",
|
||||||
|
Self::Incomparable => "incomparable",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stable backend-neutral reason explaining one RAW transaction variant relation.
|
||||||
|
///
|
||||||
|
/// These codes describe Store-domain comparison evidence only. They do not encode a
|
||||||
|
/// provider, route, backend implementation or retry classification.
|
||||||
|
#[non_exhaustive]
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub enum RawTransactionVariantRelationReason {
|
||||||
|
/// Exact canonical transaction content equality was proved.
|
||||||
|
ExactCanonicalContent,
|
||||||
|
/// The incoming variant differs only by explicitly proved truncated `logMessages`.
|
||||||
|
IncomingLogMessagesTruncated,
|
||||||
|
/// The current canonical variant differs only by explicitly proved truncated `logMessages`.
|
||||||
|
CanonicalLogMessagesTruncated,
|
||||||
|
/// The transaction slot contradicts the current canonical transaction.
|
||||||
|
SlotMismatch,
|
||||||
|
/// The optional block timestamp contradicts the current canonical transaction.
|
||||||
|
BlockTimeMismatch,
|
||||||
|
/// Canonical RAW format identity/version differs and no safe equivalence contract applies.
|
||||||
|
PayloadFormatMismatch,
|
||||||
|
/// Canonical RAW payload content contains a proved contradiction.
|
||||||
|
CanonicalPayloadConflict,
|
||||||
|
/// Equal content digests were observed for canonical representations that are not byte-identical.
|
||||||
|
ContentHashCollision,
|
||||||
|
/// Canonical representations differ but the current comparator cannot prove equality, dominance or contradiction safely.
|
||||||
|
UnsupportedCanonicalDifference,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionVariantRelationReason {
|
||||||
|
/// Returns the stable persistence/diagnostic code for this reason.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn code(&self) -> &'static str {
|
||||||
|
return match self {
|
||||||
|
Self::ExactCanonicalContent => "exact_canonical_content",
|
||||||
|
Self::IncomingLogMessagesTruncated => "incoming_log_messages_truncated",
|
||||||
|
Self::CanonicalLogMessagesTruncated => "canonical_log_messages_truncated",
|
||||||
|
Self::SlotMismatch => "slot_mismatch",
|
||||||
|
Self::BlockTimeMismatch => "block_time_mismatch",
|
||||||
|
Self::PayloadFormatMismatch => "payload_format_mismatch",
|
||||||
|
Self::CanonicalPayloadConflict => "canonical_payload_conflict",
|
||||||
|
Self::ContentHashCollision => "content_hash_collision",
|
||||||
|
Self::UnsupportedCanonicalDifference => "unsupported_canonical_difference",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Validated backend-neutral result of comparing the current canonical transaction with one incoming variant.
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub struct RawTransactionVariantComparison {
|
||||||
|
reason: crate::RawTransactionVariantRelationReason,
|
||||||
|
relation: crate::RawTransactionVariantRelation,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionVariantComparison {
|
||||||
|
/// Creates one relation/reason pair and rejects semantically inconsistent combinations.
|
||||||
|
pub fn try_new(relation: crate::RawTransactionVariantRelation, reason: crate::RawTransactionVariantRelationReason) -> crate::Result<Self> {
|
||||||
|
if !valid_variant_relation_reason(relation, reason) {
|
||||||
|
return std::result::Result::Err(raw_variant_model_error("relation_reason"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(Self { reason, relation });
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the stable reason code explaining the comparison.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn reason(&self) -> crate::RawTransactionVariantRelationReason {
|
||||||
|
return self.reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the backend-neutral quality relation.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn relation(&self) -> crate::RawTransactionVariantRelation {
|
||||||
|
return self.relation;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Compares one incoming RAW transaction variant against the currently canonical variant.
|
||||||
|
///
|
||||||
|
/// The comparator is pure and backend-neutral. Automatic dominance is intentionally limited to
|
||||||
|
/// a strictly proved `logMessages` truncation relation. Every other payload divergence fails closed
|
||||||
|
/// as either [`RawTransactionVariantRelation::Conflict`] or [`RawTransactionVariantRelation::Incomparable`].
|
||||||
|
pub fn compare_raw_transaction_variants(
|
||||||
|
canonical: &crate::RawTransaction,
|
||||||
|
incoming: &crate::RawTransaction,
|
||||||
|
) -> crate::Result<crate::RawTransactionVariantComparison> {
|
||||||
|
if canonical.reference() != incoming.reference() {
|
||||||
|
return std::result::Result::Err(raw_variant_model_error("comparison.reference"));
|
||||||
|
}
|
||||||
|
if canonical.slot() != incoming.slot() {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::SlotMismatch,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if canonical.block_time() != incoming.block_time() {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::BlockTimeMismatch,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if canonical.payload().format_id() != incoming.payload().format_id() || canonical.payload().format_version() != incoming.payload().format_version() {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Incomparable,
|
||||||
|
crate::RawTransactionVariantRelationReason::PayloadFormatMismatch,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if canonical.payload().bytes() == incoming.payload().bytes() {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Exact,
|
||||||
|
crate::RawTransactionVariantRelationReason::ExactCanonicalContent,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if canonical.payload().content_hash() == incoming.payload().content_hash() {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::ContentHashCollision,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return compare_raw_variant_payloads(canonical.payload().bytes(), incoming.payload().bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durable lifecycle status of one RAW transaction conflict case.
|
||||||
|
#[non_exhaustive]
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||||
|
pub enum RawTransactionConflictStatus {
|
||||||
|
/// At least one preserved variant still requires explicit or future automatic reconciliation.
|
||||||
|
Open,
|
||||||
|
/// The conflict currently has an explicit durable resolution while history remains preserved.
|
||||||
|
Resolved,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RawTransactionConflictStatus {
|
||||||
|
/// Returns the stable persistence code for this status.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn code(&self) -> &'static str {
|
||||||
|
return match self {
|
||||||
|
Self::Open => "open",
|
||||||
|
Self::Resolved => "resolved",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Canonical source-independent N1 RAW transaction persisted by Store backends.
|
/// Canonical source-independent N1 RAW transaction persisted by Store backends.
|
||||||
///
|
///
|
||||||
/// The payload must contain the complete KSP canonical transaction representation required
|
/// The payload must contain the complete KSP canonical transaction representation required
|
||||||
@@ -108,6 +349,408 @@ impl RawTransactionObservation {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const MAX_RAW_VARIANT_JSON_DEPTH: usize = 128;
|
||||||
|
const RAW_LOG_MESSAGES_KEY: &[u8] = b"\"logMessages\"";
|
||||||
|
const RAW_META_KEY: &[u8] = b"\"meta\"";
|
||||||
|
const RAW_TRUNCATION_MARKER_JSON: &[u8] = b"\"Log truncated\"";
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct RawVariantJsonMember<'a> {
|
||||||
|
key: &'a [u8],
|
||||||
|
value: &'a [u8],
|
||||||
|
}
|
||||||
|
|
||||||
|
fn compare_raw_variant_payloads(canonical: &[u8], incoming: &[u8]) -> crate::Result<crate::RawTransactionVariantComparison> {
|
||||||
|
let canonical_members = raw_variant_json_object_members(canonical);
|
||||||
|
let incoming_members = raw_variant_json_object_members(incoming);
|
||||||
|
let (canonical_members, incoming_members) = match (canonical_members, incoming_members) {
|
||||||
|
(std::option::Option::Some(canonical_members), std::option::Option::Some(incoming_members)) => (canonical_members, incoming_members),
|
||||||
|
_ => return raw_variant_incomparable(),
|
||||||
|
};
|
||||||
|
if canonical_members.len() != incoming_members.len() {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
let mut canonical_meta = std::option::Option::None;
|
||||||
|
let mut incoming_meta = std::option::Option::None;
|
||||||
|
for (canonical_member, incoming_member) in canonical_members.iter().zip(incoming_members.iter()) {
|
||||||
|
if canonical_member.key != incoming_member.key {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
if canonical_member.key == RAW_META_KEY {
|
||||||
|
canonical_meta = std::option::Option::Some(canonical_member.value);
|
||||||
|
incoming_meta = std::option::Option::Some(incoming_member.value);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if canonical_member.value != incoming_member.value {
|
||||||
|
return raw_variant_concrete_difference(canonical_member.value, incoming_member.value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let (canonical_meta, incoming_meta) = match (canonical_meta, incoming_meta) {
|
||||||
|
(std::option::Option::Some(canonical_meta), std::option::Option::Some(incoming_meta)) => (canonical_meta, incoming_meta),
|
||||||
|
_ => return raw_variant_incomparable(),
|
||||||
|
};
|
||||||
|
return compare_raw_variant_meta(canonical_meta, incoming_meta);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn compare_raw_variant_meta(canonical: &[u8], incoming: &[u8]) -> crate::Result<crate::RawTransactionVariantComparison> {
|
||||||
|
if canonical == incoming {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
if canonical == b"null" || incoming == b"null" {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
let canonical_members = raw_variant_json_object_members(canonical);
|
||||||
|
let incoming_members = raw_variant_json_object_members(incoming);
|
||||||
|
let (canonical_members, incoming_members) = match (canonical_members, incoming_members) {
|
||||||
|
(std::option::Option::Some(canonical_members), std::option::Option::Some(incoming_members)) => (canonical_members, incoming_members),
|
||||||
|
_ => return raw_variant_concrete_difference(canonical, incoming),
|
||||||
|
};
|
||||||
|
if canonical_members.len() != incoming_members.len() {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
let mut canonical_logs = std::option::Option::None;
|
||||||
|
let mut incoming_logs = std::option::Option::None;
|
||||||
|
for (canonical_member, incoming_member) in canonical_members.iter().zip(incoming_members.iter()) {
|
||||||
|
if canonical_member.key != incoming_member.key {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
if canonical_member.key == RAW_LOG_MESSAGES_KEY {
|
||||||
|
canonical_logs = std::option::Option::Some(canonical_member.value);
|
||||||
|
incoming_logs = std::option::Option::Some(incoming_member.value);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if canonical_member.value != incoming_member.value {
|
||||||
|
return raw_variant_concrete_difference(canonical_member.value, incoming_member.value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let (canonical_logs, incoming_logs) = match (canonical_logs, incoming_logs) {
|
||||||
|
(std::option::Option::Some(canonical_logs), std::option::Option::Some(incoming_logs)) => (canonical_logs, incoming_logs),
|
||||||
|
_ => return raw_variant_incomparable(),
|
||||||
|
};
|
||||||
|
if canonical_logs == incoming_logs {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
let canonical_items = raw_variant_json_array_items(canonical_logs);
|
||||||
|
let incoming_items = raw_variant_json_array_items(incoming_logs);
|
||||||
|
let (canonical_items, incoming_items) = match (canonical_items, incoming_items) {
|
||||||
|
(std::option::Option::Some(canonical_items), std::option::Option::Some(incoming_items)) => (canonical_items, incoming_items),
|
||||||
|
_ => return raw_variant_concrete_difference(canonical_logs, incoming_logs),
|
||||||
|
};
|
||||||
|
if raw_variant_log_messages_truncated(incoming_items.as_slice(), canonical_items.as_slice()) {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::CompatibleLessComplete,
|
||||||
|
crate::RawTransactionVariantRelationReason::IncomingLogMessagesTruncated,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if raw_variant_log_messages_truncated(canonical_items.as_slice(), incoming_items.as_slice()) {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::CompatibleMoreComplete,
|
||||||
|
crate::RawTransactionVariantRelationReason::CanonicalLogMessagesTruncated,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return raw_variant_concrete_difference(canonical_logs, incoming_logs);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_concrete_difference(canonical: &[u8], incoming: &[u8]) -> crate::Result<crate::RawTransactionVariantComparison> {
|
||||||
|
if canonical == b"null" || incoming == b"null" {
|
||||||
|
return raw_variant_incomparable();
|
||||||
|
}
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::CanonicalPayloadConflict,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_incomparable() -> crate::Result<crate::RawTransactionVariantComparison> {
|
||||||
|
return crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Incomparable,
|
||||||
|
crate::RawTransactionVariantRelationReason::UnsupportedCanonicalDifference,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_log_messages_truncated(truncated: &[&[u8]], full: &[&[u8]]) -> bool {
|
||||||
|
if truncated.is_empty() || full.is_empty() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if !truncated.iter().all(|value| return raw_variant_json_is_string(value)) || !full.iter().all(|value| return raw_variant_json_is_string(value)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let marker_count = truncated.iter().filter(|value| return **value == RAW_TRUNCATION_MARKER_JSON).count();
|
||||||
|
if marker_count != 1 || truncated.last().copied() != std::option::Option::Some(RAW_TRUNCATION_MARKER_JSON) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if full.iter().any(|value| return *value == RAW_TRUNCATION_MARKER_JSON) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let prefix_len = truncated.len() - 1;
|
||||||
|
if prefix_len >= full.len() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return truncated[..prefix_len] == full[..prefix_len];
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_object_members(value: &[u8]) -> std::option::Option<std::vec::Vec<RawVariantJsonMember<'_>>> {
|
||||||
|
if std::str::from_utf8(value).is_err() {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let start = raw_variant_json_skip_whitespace(value, 0);
|
||||||
|
if value.get(start) != std::option::Option::Some(&b'{') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let end = match raw_variant_json_value_end(value, start, 0) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
if raw_variant_json_skip_whitespace(value, end) != value.len() {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let mut index = raw_variant_json_skip_whitespace(value, start + 1);
|
||||||
|
let mut members = std::vec::Vec::new();
|
||||||
|
if value.get(index) == std::option::Option::Some(&b'}') {
|
||||||
|
return std::option::Option::Some(members);
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
let key_start = index;
|
||||||
|
let key_end = match raw_variant_json_string_end(value, key_start) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
index = raw_variant_json_skip_whitespace(value, key_end);
|
||||||
|
if value.get(index) != std::option::Option::Some(&b':') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
index = raw_variant_json_skip_whitespace(value, index + 1);
|
||||||
|
let value_start = index;
|
||||||
|
let value_end = match raw_variant_json_value_end(value, value_start, 1) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
members.push(RawVariantJsonMember { key: &value[key_start..key_end], value: &value[value_start..value_end] });
|
||||||
|
index = raw_variant_json_skip_whitespace(value, value_end);
|
||||||
|
match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(b',') => index = raw_variant_json_skip_whitespace(value, index + 1),
|
||||||
|
std::option::Option::Some(b'}') => return std::option::Option::Some(members),
|
||||||
|
_ => return std::option::Option::None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_array_items(value: &[u8]) -> std::option::Option<std::vec::Vec<&[u8]>> {
|
||||||
|
let start = raw_variant_json_skip_whitespace(value, 0);
|
||||||
|
if value.get(start) != std::option::Option::Some(&b'[') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let end = match raw_variant_json_value_end(value, start, 0) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
if raw_variant_json_skip_whitespace(value, end) != value.len() {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let mut index = raw_variant_json_skip_whitespace(value, start + 1);
|
||||||
|
let mut items = std::vec::Vec::new();
|
||||||
|
if value.get(index) == std::option::Option::Some(&b']') {
|
||||||
|
return std::option::Option::Some(items);
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
let item_start = index;
|
||||||
|
let item_end = match raw_variant_json_value_end(value, item_start, 1) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
items.push(&value[item_start..item_end]);
|
||||||
|
index = raw_variant_json_skip_whitespace(value, item_end);
|
||||||
|
match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(b',') => index = raw_variant_json_skip_whitespace(value, index + 1),
|
||||||
|
std::option::Option::Some(b']') => return std::option::Option::Some(items),
|
||||||
|
_ => return std::option::Option::None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_value_end(value: &[u8], start: usize, depth: usize) -> std::option::Option<usize> {
|
||||||
|
if depth > MAX_RAW_VARIANT_JSON_DEPTH {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let start = raw_variant_json_skip_whitespace(value, start);
|
||||||
|
return match value.get(start).copied() {
|
||||||
|
std::option::Option::Some(b'"') => raw_variant_json_string_end(value, start),
|
||||||
|
std::option::Option::Some(b'{') => raw_variant_json_object_end(value, start, depth + 1),
|
||||||
|
std::option::Option::Some(b'[') => raw_variant_json_array_end(value, start, depth + 1),
|
||||||
|
std::option::Option::Some(b't') => raw_variant_json_literal_end(value, start, b"true"),
|
||||||
|
std::option::Option::Some(b'f') => raw_variant_json_literal_end(value, start, b"false"),
|
||||||
|
std::option::Option::Some(b'n') => raw_variant_json_literal_end(value, start, b"null"),
|
||||||
|
std::option::Option::Some(b'-' | b'0'..=b'9') => raw_variant_json_number_end(value, start),
|
||||||
|
_ => std::option::Option::None,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_object_end(value: &[u8], start: usize, depth: usize) -> std::option::Option<usize> {
|
||||||
|
let mut index = raw_variant_json_skip_whitespace(value, start + 1);
|
||||||
|
if value.get(index) == std::option::Option::Some(&b'}') {
|
||||||
|
return std::option::Option::Some(index + 1);
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
index = match raw_variant_json_string_end(value, index) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
index = raw_variant_json_skip_whitespace(value, index);
|
||||||
|
if value.get(index) != std::option::Option::Some(&b':') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
index = match raw_variant_json_value_end(value, index + 1, depth) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
index = raw_variant_json_skip_whitespace(value, index);
|
||||||
|
match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(b',') => index = raw_variant_json_skip_whitespace(value, index + 1),
|
||||||
|
std::option::Option::Some(b'}') => return std::option::Option::Some(index + 1),
|
||||||
|
_ => return std::option::Option::None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_array_end(value: &[u8], start: usize, depth: usize) -> std::option::Option<usize> {
|
||||||
|
let mut index = raw_variant_json_skip_whitespace(value, start + 1);
|
||||||
|
if value.get(index) == std::option::Option::Some(&b']') {
|
||||||
|
return std::option::Option::Some(index + 1);
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
index = match raw_variant_json_value_end(value, index, depth) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
index = raw_variant_json_skip_whitespace(value, index);
|
||||||
|
match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(b',') => index = raw_variant_json_skip_whitespace(value, index + 1),
|
||||||
|
std::option::Option::Some(b']') => return std::option::Option::Some(index + 1),
|
||||||
|
_ => return std::option::Option::None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_string_end(value: &[u8], start: usize) -> std::option::Option<usize> {
|
||||||
|
if value.get(start) != std::option::Option::Some(&b'"') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
let mut index = start + 1;
|
||||||
|
while index < value.len() {
|
||||||
|
match value[index] {
|
||||||
|
b'"' => return std::option::Option::Some(index + 1),
|
||||||
|
b'\\' => {
|
||||||
|
index += 1;
|
||||||
|
let escape = match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
if escape == b'u' {
|
||||||
|
if index + 4 >= value.len() || !value[index + 1..=index + 4].iter().all(|byte| return byte.is_ascii_hexdigit()) {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
index += 5;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if !matches!(escape, b'"' | b'\\' | b'/' | b'b' | b'f' | b'n' | b'r' | b't') {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
index += 1;
|
||||||
|
},
|
||||||
|
0x00..=0x1f => return std::option::Option::None,
|
||||||
|
_ => index += 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_literal_end(value: &[u8], start: usize, literal: &[u8]) -> std::option::Option<usize> {
|
||||||
|
let end = match start.checked_add(literal.len()) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
if value.get(start..end) == std::option::Option::Some(literal) {
|
||||||
|
return std::option::Option::Some(end);
|
||||||
|
}
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_number_end(value: &[u8], start: usize) -> std::option::Option<usize> {
|
||||||
|
let mut index = start;
|
||||||
|
if value.get(index) == std::option::Option::Some(&b'-') {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
match value.get(index).copied() {
|
||||||
|
std::option::Option::Some(b'0') => index += 1,
|
||||||
|
std::option::Option::Some(b'1'..=b'9') => {
|
||||||
|
index += 1;
|
||||||
|
while matches!(value.get(index).copied(), std::option::Option::Some(b'0'..=b'9')) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => return std::option::Option::None,
|
||||||
|
}
|
||||||
|
if value.get(index) == std::option::Option::Some(&b'.') {
|
||||||
|
index += 1;
|
||||||
|
let fraction_start = index;
|
||||||
|
while matches!(value.get(index).copied(), std::option::Option::Some(b'0'..=b'9')) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
if index == fraction_start {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if matches!(value.get(index).copied(), std::option::Option::Some(b'e' | b'E')) {
|
||||||
|
index += 1;
|
||||||
|
if matches!(value.get(index).copied(), std::option::Option::Some(b'+' | b'-')) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
let exponent_start = index;
|
||||||
|
while matches!(value.get(index).copied(), std::option::Option::Some(b'0'..=b'9')) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
if index == exponent_start {
|
||||||
|
return std::option::Option::None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::option::Option::Some(index);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_skip_whitespace(value: &[u8], mut index: usize) -> usize {
|
||||||
|
while matches!(value.get(index).copied(), std::option::Option::Some(b' ' | b'\n' | b'\r' | b'\t')) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
return index;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_json_is_string(value: &[u8]) -> bool {
|
||||||
|
return raw_variant_json_string_end(value, 0) == std::option::Option::Some(value.len());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn raw_variant_model_error(field: &'static str) -> crate::Error {
|
||||||
|
return crate::Error::new(crate::ERROR_CODE_RAW_MODEL_INVALID, "invalid RAW transaction variant model").with_context("field", field);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn valid_variant_relation_reason(relation: crate::RawTransactionVariantRelation, reason: crate::RawTransactionVariantRelationReason) -> bool {
|
||||||
|
return matches!(
|
||||||
|
(relation, reason),
|
||||||
|
(crate::RawTransactionVariantRelation::Exact, crate::RawTransactionVariantRelationReason::ExactCanonicalContent)
|
||||||
|
| (crate::RawTransactionVariantRelation::CompatibleLessComplete, crate::RawTransactionVariantRelationReason::IncomingLogMessagesTruncated)
|
||||||
|
| (crate::RawTransactionVariantRelation::CompatibleMoreComplete, crate::RawTransactionVariantRelationReason::CanonicalLogMessagesTruncated)
|
||||||
|
| (
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::SlotMismatch
|
||||||
|
| crate::RawTransactionVariantRelationReason::BlockTimeMismatch
|
||||||
|
| crate::RawTransactionVariantRelationReason::CanonicalPayloadConflict
|
||||||
|
| crate::RawTransactionVariantRelationReason::ContentHashCollision
|
||||||
|
)
|
||||||
|
| (
|
||||||
|
crate::RawTransactionVariantRelation::Incomparable,
|
||||||
|
crate::RawTransactionVariantRelationReason::PayloadFormatMismatch | crate::RawTransactionVariantRelationReason::UnsupportedCanonicalDifference
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
#[path = "../../unit_tests/model/raw_transaction.rs"]
|
#[path = "../../unit_tests/model/raw_transaction.rs"]
|
||||||
mod tests;
|
mod tests;
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/tests/dependency_boundary.rs
|
// file: crates/ksp-store-api/tests/dependency_boundary.rs
|
||||||
// version: 7
|
// version: 8
|
||||||
|
|
||||||
//! Dependency canaries for the Store API RAW foundation.
|
//! Dependency canaries for the Store API RAW foundation.
|
||||||
|
|
||||||
@@ -97,6 +97,7 @@ fn pre_006_source_boundary_keeps_models_and_capabilities_backend_free() {
|
|||||||
"ksp_onchain_transport_lib",
|
"ksp_onchain_transport_lib",
|
||||||
"ksp_program_api",
|
"ksp_program_api",
|
||||||
"serde::",
|
"serde::",
|
||||||
|
"serde_json::",
|
||||||
"sqlx::",
|
"sqlx::",
|
||||||
"tokio::",
|
"tokio::",
|
||||||
"tokio_postgres::",
|
"tokio_postgres::",
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/tests/public_api.rs
|
// file: crates/ksp-store-api/tests/public_api.rs
|
||||||
// version: 9
|
// version: 11
|
||||||
|
|
||||||
//! Integration canaries for the public `ksp-store-api` surface.
|
//! Integration canaries for the public `ksp-store-api` surface.
|
||||||
|
|
||||||
@@ -170,6 +170,13 @@ fn public_pre_006_query_outcome_and_retention_contracts_are_available_from_crate
|
|||||||
ksp_store_api::RawAcquisitionWriteOutcome::new(ksp_store_api::RawEntityWriteOutcome::Inserted, ksp_store_api::RawObservationWriteOutcome::Inserted);
|
ksp_store_api::RawAcquisitionWriteOutcome::new(ksp_store_api::RawEntityWriteOutcome::Inserted, ksp_store_api::RawObservationWriteOutcome::Inserted);
|
||||||
assert_eq!(outcome.entity(), ksp_store_api::RawEntityWriteOutcome::Inserted);
|
assert_eq!(outcome.entity(), ksp_store_api::RawEntityWriteOutcome::Inserted);
|
||||||
assert_eq!(outcome.observation(), ksp_store_api::RawObservationWriteOutcome::Inserted);
|
assert_eq!(outcome.observation(), ksp_store_api::RawObservationWriteOutcome::Inserted);
|
||||||
|
assert_eq!(outcome.transaction_variant(), std::option::Option::None);
|
||||||
|
let variant_outcome = ksp_store_api::RawAcquisitionWriteOutcome::with_transaction_variant(
|
||||||
|
ksp_store_api::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_api::RawObservationWriteOutcome::Inserted,
|
||||||
|
ksp_store_api::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
|
);
|
||||||
|
assert_eq!(variant_outcome.transaction_variant(), std::option::Option::Some(ksp_store_api::RawTransactionVariantWriteOutcome::QuarantinedConflict));
|
||||||
let retention_read: std::option::Option<&dyn ksp_store_api::RawTransactionRetentionRead> = std::option::Option::None;
|
let retention_read: std::option::Option<&dyn ksp_store_api::RawTransactionRetentionRead> = std::option::Option::None;
|
||||||
let retention_write: std::option::Option<&dyn ksp_store_api::RawTransactionRetentionWrite> = std::option::Option::None;
|
let retention_write: std::option::Option<&dyn ksp_store_api::RawTransactionRetentionWrite> = std::option::Option::None;
|
||||||
assert!(retention_read.is_none());
|
assert!(retention_read.is_none());
|
||||||
@@ -208,3 +215,10 @@ fn public_v0_3_8_pre_003_inspection_contracts_are_backend_neutral_and_dyn_compat
|
|||||||
assert!(account_observation_inspection.is_none());
|
assert!(account_observation_inspection.is_none());
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn public_v0_3_16_pre_005_variant_comparator_is_available_from_crate_root() {
|
||||||
|
let comparator = ksp_store_api::compare_raw_transaction_variants;
|
||||||
|
let _ = comparator;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,8 +1,7 @@
|
|||||||
// file: crates/ksp-store-api/tests/release_completeness.rs
|
// file: crates/ksp-store-api/tests/release_completeness.rs
|
||||||
// version: 3
|
// version: 6
|
||||||
|
|
||||||
//! Release-level boundary and completeness canaries for the backend-neutral Store API RAW surface.
|
//! Release-level boundary and completeness canaries for the backend-neutral Store API RAW surface.
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn v0_3_8_pre_003_exact_crate_root_export_inventory_is_stable() {
|
fn v0_3_8_pre_003_exact_crate_root_export_inventory_is_stable() {
|
||||||
let crate_root = include_str!("../src/lib.rs");
|
let crate_root = include_str!("../src/lib.rs");
|
||||||
@@ -57,6 +56,7 @@ fn v0_3_8_pre_003_exact_crate_root_export_inventory_is_stable() {
|
|||||||
"pub use self::model::raw_outcome::RawAcquisitionWriteOutcome;",
|
"pub use self::model::raw_outcome::RawAcquisitionWriteOutcome;",
|
||||||
"pub use self::model::raw_outcome::RawEntityWriteOutcome;",
|
"pub use self::model::raw_outcome::RawEntityWriteOutcome;",
|
||||||
"pub use self::model::raw_outcome::RawObservationWriteOutcome;",
|
"pub use self::model::raw_outcome::RawObservationWriteOutcome;",
|
||||||
|
"pub use self::model::raw_outcome::RawTransactionVariantWriteOutcome;",
|
||||||
"pub use self::model::raw_pagination::MAX_RAW_PAGE_CURSOR_BYTES;",
|
"pub use self::model::raw_pagination::MAX_RAW_PAGE_CURSOR_BYTES;",
|
||||||
"pub use self::model::raw_pagination::RawAccountStateQuery;",
|
"pub use self::model::raw_pagination::RawAccountStateQuery;",
|
||||||
"pub use self::model::raw_pagination::RawPage;",
|
"pub use self::model::raw_pagination::RawPage;",
|
||||||
@@ -87,8 +87,16 @@ fn v0_3_8_pre_003_exact_crate_root_export_inventory_is_stable() {
|
|||||||
"pub use self::model::raw_retention::RawTransactionRetentionTransition;",
|
"pub use self::model::raw_retention::RawTransactionRetentionTransition;",
|
||||||
"pub use self::model::raw_retention::RawTransactionTombstone;",
|
"pub use self::model::raw_retention::RawTransactionTombstone;",
|
||||||
"pub use self::model::raw_transaction::RawTransaction;",
|
"pub use self::model::raw_transaction::RawTransaction;",
|
||||||
|
"pub use self::model::raw_transaction::compare_raw_transaction_variants;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionConflictStatus;",
|
||||||
"pub use self::model::raw_transaction::RawTransactionObservation;",
|
"pub use self::model::raw_transaction::RawTransactionObservation;",
|
||||||
"pub use self::model::raw_transaction::RawTransactionReference;",
|
"pub use self::model::raw_transaction::RawTransactionReference;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantComparison;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantId;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantOrigin;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantReference;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantRelation;",
|
||||||
|
"pub use self::model::raw_transaction::RawTransactionVariantRelationReason;",
|
||||||
];
|
];
|
||||||
expected.sort_unstable();
|
expected.sort_unstable();
|
||||||
assert_eq!(actual, expected);
|
assert_eq!(actual, expected);
|
||||||
@@ -133,6 +141,7 @@ fn pre_007_public_evolutive_enums_remain_non_exhaustive() {
|
|||||||
include_str!("../src/model/raw_pagination.rs"),
|
include_str!("../src/model/raw_pagination.rs"),
|
||||||
include_str!("../src/model/raw_primitives.rs"),
|
include_str!("../src/model/raw_primitives.rs"),
|
||||||
include_str!("../src/model/raw_retention.rs"),
|
include_str!("../src/model/raw_retention.rs"),
|
||||||
|
include_str!("../src/model/raw_transaction.rs"),
|
||||||
];
|
];
|
||||||
for enum_name in [
|
for enum_name in [
|
||||||
"RawAcquisitionOrigin",
|
"RawAcquisitionOrigin",
|
||||||
@@ -142,6 +151,11 @@ fn pre_007_public_evolutive_enums_remain_non_exhaustive() {
|
|||||||
"RawRetentionWriteOutcome",
|
"RawRetentionWriteOutcome",
|
||||||
"RawSortDirection",
|
"RawSortDirection",
|
||||||
"RawTransactionAcquisitionMode",
|
"RawTransactionAcquisitionMode",
|
||||||
|
"RawTransactionConflictStatus",
|
||||||
|
"RawTransactionVariantOrigin",
|
||||||
|
"RawTransactionVariantRelation",
|
||||||
|
"RawTransactionVariantRelationReason",
|
||||||
|
"RawTransactionVariantWriteOutcome",
|
||||||
] {
|
] {
|
||||||
assert_non_exhaustive(&sources, enum_name);
|
assert_non_exhaustive(&sources, enum_name);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-api/unit_tests/model/raw_transaction.rs
|
// file: crates/ksp-store-api/unit_tests/model/raw_transaction.rs
|
||||||
// version: 2
|
// version: 5
|
||||||
|
|
||||||
fn network() -> std::option::Option<crate::RawNetworkId> {
|
fn network() -> std::option::Option<crate::RawNetworkId> {
|
||||||
return match crate::RawNetworkId::new("mainnet".to_owned()) {
|
return match crate::RawNetworkId::new("mainnet".to_owned()) {
|
||||||
@@ -83,3 +83,178 @@ fn transaction_observation_is_separate_from_canonical_raw_payload() {
|
|||||||
assert_eq!(observation.provenance().provider().as_str(), "publicnode");
|
assert_eq!(observation.provenance().provider().as_str(), "publicnode");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn transaction_variant_identity_is_non_zero_and_keeps_transaction_scope() {
|
||||||
|
assert!(crate::RawTransactionVariantId::try_new(0).is_err());
|
||||||
|
let variant_id = match crate::RawTransactionVariantId::try_new(7) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
let network = match network() {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let transaction = crate::RawTransactionReference::new(network, crate::RawTransactionSignature::new([13_u8; 64]));
|
||||||
|
let variant = crate::RawTransactionVariantReference::new(transaction, variant_id);
|
||||||
|
assert_eq!(variant.variant_id().get(), 7);
|
||||||
|
assert_eq!(variant.transaction().network().as_str(), "mainnet");
|
||||||
|
assert_eq!(crate::RawTransactionVariantOrigin::Native.code(), "native");
|
||||||
|
assert_eq!(crate::RawTransactionVariantOrigin::Synthetic.code(), "synthetic");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn transaction_variant_relation_reason_pairs_are_validated() {
|
||||||
|
let exact = crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Exact,
|
||||||
|
crate::RawTransactionVariantRelationReason::ExactCanonicalContent,
|
||||||
|
);
|
||||||
|
assert!(exact.is_ok());
|
||||||
|
let invalid = crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Exact,
|
||||||
|
crate::RawTransactionVariantRelationReason::IncomingLogMessagesTruncated,
|
||||||
|
);
|
||||||
|
assert!(invalid.is_err());
|
||||||
|
let conflict = crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Conflict,
|
||||||
|
crate::RawTransactionVariantRelationReason::ContentHashCollision,
|
||||||
|
);
|
||||||
|
assert!(conflict.is_ok());
|
||||||
|
let incomparable = crate::RawTransactionVariantComparison::try_new(
|
||||||
|
crate::RawTransactionVariantRelation::Incomparable,
|
||||||
|
crate::RawTransactionVariantRelationReason::PayloadFormatMismatch,
|
||||||
|
);
|
||||||
|
assert!(incomparable.is_ok());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn transaction_variant_codes_and_outcomes_are_stable_backend_neutral_contracts() {
|
||||||
|
assert_eq!(crate::RawTransactionVariantRelation::CompatibleLessComplete.code(), "compatible_less_complete");
|
||||||
|
assert_eq!(crate::RawTransactionVariantRelationReason::IncomingLogMessagesTruncated.code(), "incoming_log_messages_truncated");
|
||||||
|
assert_eq!(crate::RawTransactionVariantRelationReason::ContentHashCollision.code(), "content_hash_collision");
|
||||||
|
assert_eq!(crate::RawTransactionConflictStatus::Open.code(), "open");
|
||||||
|
assert_eq!(crate::RawTransactionConflictStatus::Resolved.code(), "resolved");
|
||||||
|
assert_ne!(crate::RawTransactionVariantWriteOutcome::InsertedCanonical, crate::RawTransactionVariantWriteOutcome::ObservedExact);
|
||||||
|
assert_ne!(crate::RawTransactionVariantWriteOutcome::QuarantinedConflict, crate::RawTransactionVariantWriteOutcome::PromotedCompatibleMoreComplete);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn comparison_transaction(payload_bytes: &[u8], hash_byte: u8) -> std::option::Option<crate::RawTransaction> {
|
||||||
|
let network = match network() {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let format = match crate::RawFormatId::new("ksp.solana.raw_transaction".to_owned()) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let payload = match crate::RawPayload::try_new(format, 1, payload_bytes.to_vec().into_boxed_slice(), crate::RawContentHash::new([hash_byte; 32])) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let reference = crate::RawTransactionReference::new(network, crate::RawTransactionSignature::new([21_u8; 64]));
|
||||||
|
return std::option::Option::Some(crate::RawTransaction::new(reference, 88, std::option::Option::None, payload));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn comparison_relation(
|
||||||
|
canonical_payload: &[u8],
|
||||||
|
canonical_hash: u8,
|
||||||
|
incoming_payload: &[u8],
|
||||||
|
incoming_hash: u8,
|
||||||
|
) -> std::option::Option<crate::RawTransactionVariantComparison> {
|
||||||
|
let canonical = match comparison_transaction(canonical_payload, canonical_hash) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let incoming = match comparison_transaction(incoming_payload, incoming_hash) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
return crate::compare_raw_transaction_variants(&canonical, &incoming).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_005_comparator_proves_bidirectional_log_messages_truncation_only() {
|
||||||
|
let full = br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#;
|
||||||
|
let truncated = br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated"],"rewards":[]},"version":0,"transactionIndex":7}"#;
|
||||||
|
let less = match comparison_relation(full, 31, truncated, 32) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(less.relation(), crate::RawTransactionVariantRelation::CompatibleLessComplete);
|
||||||
|
assert_eq!(less.reason(), crate::RawTransactionVariantRelationReason::IncomingLogMessagesTruncated);
|
||||||
|
let more = match comparison_relation(truncated, 32, full, 31) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(more.relation(), crate::RawTransactionVariantRelation::CompatibleMoreComplete);
|
||||||
|
assert_eq!(more.reason(), crate::RawTransactionVariantRelationReason::CanonicalLogMessagesTruncated);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_005_comparator_rejects_unproved_log_shortening_and_marker_shapes() {
|
||||||
|
let full = br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["a","b","c"],"rewards":[]},"version":0}"#;
|
||||||
|
let shorter = br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["a","b"],"rewards":[]},"version":0}"#;
|
||||||
|
let marker_not_last =
|
||||||
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["a","Log truncated","c"],"rewards":[]},"version":0}"#;
|
||||||
|
let marker_not_exact =
|
||||||
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["a","b","Log truncated by provider"],"rewards":[]},"version":0}"#;
|
||||||
|
let bad_prefix =
|
||||||
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["a","changed","Log truncated"],"rewards":[]},"version":0}"#;
|
||||||
|
let candidates: [&[u8]; 4] = [shorter, marker_not_last, marker_not_exact, bad_prefix];
|
||||||
|
for (index, candidate) in candidates.iter().enumerate() {
|
||||||
|
let hash_offset = match u8::try_from(index) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
let comparison = match comparison_relation(full, 40, candidate, 41 + hash_offset) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(comparison.relation(), crate::RawTransactionVariantRelation::Conflict);
|
||||||
|
assert_eq!(comparison.reason(), crate::RawTransactionVariantRelationReason::CanonicalPayloadConflict);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_005_comparator_is_fail_closed_for_optional_and_other_payload_differences() {
|
||||||
|
let canonical = br#"{"transaction":["AQID","base64"],"meta":{"fee":5000,"logMessages":["a"],"returnData":null},"version":0}"#;
|
||||||
|
let optional_value =
|
||||||
|
br#"{"transaction":["AQID","base64"],"meta":{"fee":5000,"logMessages":["a"],"returnData":{"data":["","base64"],"programId":"Vote111"}},"version":0}"#;
|
||||||
|
let concrete_conflict = br#"{"transaction":["AQID","base64"],"meta":{"fee":5001,"logMessages":["a"],"returnData":null},"version":0}"#;
|
||||||
|
let optional = match comparison_relation(canonical, 50, optional_value, 51) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(optional.relation(), crate::RawTransactionVariantRelation::Incomparable);
|
||||||
|
assert_eq!(optional.reason(), crate::RawTransactionVariantRelationReason::UnsupportedCanonicalDifference);
|
||||||
|
let conflict = match comparison_relation(canonical, 50, concrete_conflict, 52) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(conflict.relation(), crate::RawTransactionVariantRelation::Conflict);
|
||||||
|
assert_eq!(conflict.reason(), crate::RawTransactionVariantRelationReason::CanonicalPayloadConflict);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_005_comparator_uses_exact_bytes_and_detects_hash_collision_without_hash_equality_shortcut() {
|
||||||
|
let canonical = br#"{"transaction":["AQID","base64"],"meta":{"logMessages":["a"]}}"#;
|
||||||
|
let same_bytes = match comparison_relation(canonical, 60, canonical, 61) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(same_bytes.relation(), crate::RawTransactionVariantRelation::Exact);
|
||||||
|
let different = br#"{"transaction":["AQID","base64"],"meta":{"logMessages":["b"]}}"#;
|
||||||
|
let collision = match comparison_relation(canonical, 62, different, 62) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
assert_eq!(collision.relation(), crate::RawTransactionVariantRelation::Conflict);
|
||||||
|
assert_eq!(collision.reason(), crate::RawTransactionVariantRelationReason::ContentHashCollision);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-store-lib/README.md -->
|
<!-- file: crates/ksp-store-lib/README.md -->
|
||||||
<!-- version: 5 -->
|
<!-- version: 6 -->
|
||||||
|
|
||||||
# ksp-store-lib
|
# ksp-store-lib
|
||||||
|
|
||||||
@@ -19,7 +19,7 @@ Elle expose aux consumers une surface backend-neutral, réexporte les contrats R
|
|||||||
- `Store::health().await` pour la readiness portable et bornée ;
|
- `Store::health().await` pour la readiness portable et bornée ;
|
||||||
- `Store::close(self).await` pour la fermeture explicite bornée ;
|
- `Store::close(self).await` pour la fermeture explicite bornée ;
|
||||||
- le mapping des erreurs backend vers des codes Store stables sans exposer les erreurs physiques ;
|
- le mapping des erreurs backend vers des codes Store stables sans exposer les erreurs physiques ;
|
||||||
- les dix capabilities RAW de `ksp-store-api` dispatchées vers le backend compilé : six `RawTransaction*` et quatre `RawAccount*` ;
|
- les quatorze capabilities RAW de `ksp-store-api` dispatchées vers le backend compilé, dont les quatre capabilities d'inspection entity/observation ;
|
||||||
- une validation réseau backend-neutral avant dispatch pour toutes les opérations qui portent explicitement un réseau ;
|
- une validation réseau backend-neutral avant dispatch pour toutes les opérations qui portent explicitement un réseau ;
|
||||||
- les réexports crate-root de `ksp-store-api` nécessaires aux consumers ordinaires.
|
- les réexports crate-root de `ksp-store-api` nécessaires aux consumers ordinaires.
|
||||||
|
|
||||||
@@ -65,7 +65,7 @@ Store ne lit ni `.env`, ni variables `KSP_*` / `KSPB_*`, ni variables/fichiers i
|
|||||||
|
|
||||||
## Surface RAW
|
## Surface RAW
|
||||||
|
|
||||||
`Store` implémente exactement dix capabilities RAW backend-neutral :
|
`Store` implémente exactement quatorze capabilities RAW backend-neutral :
|
||||||
|
|
||||||
```text
|
```text
|
||||||
RawTransactionRead
|
RawTransactionRead
|
||||||
@@ -74,13 +74,17 @@ RawTransactionObservationRead
|
|||||||
RawTransactionObservationWrite
|
RawTransactionObservationWrite
|
||||||
RawTransactionRetentionRead
|
RawTransactionRetentionRead
|
||||||
RawTransactionRetentionWrite
|
RawTransactionRetentionWrite
|
||||||
|
RawTransactionInspectionRead
|
||||||
|
RawTransactionObservationInspectionRead
|
||||||
RawAccountStateRead
|
RawAccountStateRead
|
||||||
RawAccountStateWrite
|
RawAccountStateWrite
|
||||||
RawAccountObservationRead
|
RawAccountObservationRead
|
||||||
RawAccountObservationWrite
|
RawAccountObservationWrite
|
||||||
|
RawAccountStateInspectionRead
|
||||||
|
RawAccountObservationInspectionRead
|
||||||
```
|
```
|
||||||
|
|
||||||
Pour `RawTransaction`, la façade fournit la lecture canonique et des observations, l'acquisition atomique transaction+observation, l'ajout idempotent d'observations, la pagination keyset et les transitions de rétention demandées par le caller.
|
Pour `RawTransaction`, la façade fournit la lecture canonique et des observations, l'acquisition atomique transaction+observation, l'ajout idempotent d'observations, les inspections random-access, la pagination keyset et les transitions de rétention demandées par le caller. Depuis `0.3.16`, l'outcome d'acquisition peut aussi exposer la classification de variante (`InsertedCanonical`, `ObservedExact`, `ObservedCompatibleLessComplete`, `PromotedCompatibleMoreComplete`, `QuarantinedConflict`, `Rehydrated`, `SkippedPurged`). `QuarantinedConflict` est un succès durable : la façade ne le transforme pas en erreur terminale.
|
||||||
|
|
||||||
Pour `RawAccountState`, elle fournit :
|
Pour `RawAccountState`, elle fournit :
|
||||||
|
|
||||||
@@ -112,4 +116,6 @@ Sont volontairement hors de cette surface :
|
|||||||
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — plan `RawTransaction` ;
|
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — plan `RawTransaction` ;
|
||||||
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction` ;
|
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction` ;
|
||||||
- [`../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md`](../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md) — design `RawAccountState` et complétude RAW ;
|
- [`../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md`](../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md) — design `RawAccountState` et complétude RAW ;
|
||||||
- [`../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md`](../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md) — validation `RawAccountState` et conformance 10/10.
|
- [`../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md`](../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md) — validation `RawAccountState` ;
|
||||||
|
- [`../../docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md`](../../docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md) — variantes, sélecteur canonique et conflit durable ;
|
||||||
|
- [`../../docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md`](../../docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md) — validation `0.3.16` et preuve PostgreSQL live.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-store-lib/USAGE.md -->
|
<!-- file: crates/ksp-store-lib/USAGE.md -->
|
||||||
<!-- version: 5 -->
|
<!-- version: 6 -->
|
||||||
|
|
||||||
# Utilisation de ksp-store-lib
|
# Utilisation de ksp-store-lib
|
||||||
|
|
||||||
@@ -195,7 +195,7 @@ async fn persist_acquisition(
|
|||||||
|
|
||||||
Le mode `Normal` respecte un tombstone `Purged`. `ForceRehydrate` doit être choisi explicitement lorsqu'un caller veut restaurer un payload purgé et que l'identité retenue est compatible.
|
Le mode `Normal` respecte un tombstone `Purged`. `ForceRehydrate` doit être choisi explicitement lorsqu'un caller veut restaurer un payload purgé et que l'identité retenue est compatible.
|
||||||
|
|
||||||
Un contenu divergent sous la même identité produit `ERROR_CODE_RAW_CONFLICT`; Store ne remplace jamais silencieusement le contenu gagnant.
|
Depuis `0.3.16`, le caller doit inspecter `RawAcquisitionWriteOutcome::transaction_variant()` pour une acquisition `RawTransaction`. Un contenu divergent conservable ne produit plus une erreur terminale : il retourne `QuarantinedConflict`, la variante reçue est durable et le canonique reste inchangé. `ObservedCompatibleLessComplete` conserve le canonique courant ; `PromotedCompatibleMoreComplete` signale une promotion atomique. `ERROR_CODE_RAW_CONFLICT` reste pertinent pour des chemins legacy/directs qui ne disposent pas de cette quarantaine durable.
|
||||||
|
|
||||||
## 8. Lire et ajouter une observation
|
## 8. Lire et ajouter une observation
|
||||||
|
|
||||||
@@ -379,6 +379,6 @@ Les conflits et queries invalides utilisent les codes backend-neutral `store_api
|
|||||||
|
|
||||||
## 15. Limites de la façade
|
## 15. Limites de la façade
|
||||||
|
|
||||||
La façade ne fournit pas d'accès public au SQL, au pool, aux clients ou transactions PostgreSQL. Elle dispatch les dix capabilities RAW de l'API commune, mais ne fournit aucune capability de rétention, archivage, purge, delete ou compaction account.
|
La façade ne fournit pas d'accès public au SQL, au pool, aux clients ou transactions PostgreSQL. Elle dispatch les quatorze capabilities RAW de l'API commune, dont quatre capabilities d'inspection, mais ne fournit aucune capability de rétention, archivage, purge, delete ou compaction account.
|
||||||
|
|
||||||
La taille de page est une primitive de navigation. Les décisions de batch, priorité, backlog et scheduling appartiennent aux workers/jobs, pas à Store.
|
La taille de page est une primitive de navigation. Les décisions de batch, priorité, backlog et scheduling appartiennent aux workers/jobs, pas à Store.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-lib/src/lib.rs
|
// file: crates/ksp-store-lib/src/lib.rs
|
||||||
// version: 12
|
// version: 13
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -229,6 +229,8 @@ pub use ksp_store_api::RawTransactionSignature;
|
|||||||
pub use ksp_store_api::RawTransactionSummary;
|
pub use ksp_store_api::RawTransactionSummary;
|
||||||
/// Minimal durable identity retained after a canonical RAW transaction payload is purged.
|
/// Minimal durable identity retained after a canonical RAW transaction payload is purged.
|
||||||
pub use ksp_store_api::RawTransactionTombstone;
|
pub use ksp_store_api::RawTransactionTombstone;
|
||||||
|
/// Durable Store outcome for one variant-aware RAW transaction acquisition.
|
||||||
|
pub use ksp_store_api::RawTransactionVariantWriteOutcome;
|
||||||
/// Write capability for canonical RAW transaction acquisitions.
|
/// Write capability for canonical RAW transaction acquisitions.
|
||||||
pub use ksp_store_api::RawTransactionWrite;
|
pub use ksp_store_api::RawTransactionWrite;
|
||||||
/// Common KSP result alias using [`Error`].
|
/// Common KSP result alias using [`Error`].
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-lib/tests/hardening_completeness.rs
|
// file: crates/ksp-store-lib/tests/hardening_completeness.rs
|
||||||
// version: 10
|
// version: 11
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -209,6 +209,7 @@ fn pre_009_facade_modules_and_crate_root_exports_are_exact() {
|
|||||||
"RawTransactionSignature",
|
"RawTransactionSignature",
|
||||||
"RawTransactionSummary",
|
"RawTransactionSummary",
|
||||||
"RawTransactionTombstone",
|
"RawTransactionTombstone",
|
||||||
|
"RawTransactionVariantWriteOutcome",
|
||||||
"RawTransactionWrite",
|
"RawTransactionWrite",
|
||||||
"Result",
|
"Result",
|
||||||
"Store",
|
"Store",
|
||||||
@@ -222,7 +223,7 @@ fn pre_009_facade_modules_and_crate_root_exports_are_exact() {
|
|||||||
];
|
];
|
||||||
expected.sort_unstable();
|
expected.sort_unstable();
|
||||||
assert_eq!(actual.as_slice(), expected.as_slice());
|
assert_eq!(actual.as_slice(), expected.as_slice());
|
||||||
assert_eq!(actual.len(), 105);
|
assert_eq!(actual.len(), 106);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-store-postgres-lib/README.md -->
|
<!-- file: crates/ksp-store-postgres-lib/README.md -->
|
||||||
<!-- version: 11 -->
|
<!-- version: 12 -->
|
||||||
|
|
||||||
# ksp-store-postgres-lib
|
# ksp-store-postgres-lib
|
||||||
|
|
||||||
@@ -63,7 +63,18 @@ VerifyFull
|
|||||||
|
|
||||||
Le moteur de migrations embarqué vérifie version logique, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime.
|
Le moteur de migrations embarqué vérifie version logique, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime.
|
||||||
|
|
||||||
Le bootstrap metadata est conservé comme migration V000. La migration logique V001 matérialise le schéma `RawTransaction` et V002 le schéma `RawAccountState`, chacune en ressources séparées `tables/`, `constraints/` et `indexes/` afin que le backend puisse vérifier leur compatibilité effective sans transformer les fichiers SQL en parser généraliste.
|
Le bootstrap metadata est conservé comme migration V000. La migration logique V001 matérialise le schéma `RawTransaction`, V002 le schéma `RawAccountState` et V003 ajoute le sidecar multi-variantes `RawTransaction`. V000/V001/V002 restent byte/checksum-identiques. Les migrations sont découpées en ressources `tables/`, `constraints/` et `indexes/` afin que le backend puisse vérifier leur compatibilité effective sans transformer les fichiers SQL en parser généraliste.
|
||||||
|
|
||||||
|
V003 ajoute exactement les quatre tables métier suivantes :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ksp_raw_transaction_variants
|
||||||
|
ksp_raw_transaction_canonical_selectors
|
||||||
|
ksp_raw_transaction_observation_variants
|
||||||
|
ksp_raw_transaction_conflicts
|
||||||
|
```
|
||||||
|
|
||||||
|
Le conflict case V003 de `0.3.16` est volontairement minimal (`Open`/réouverture idempotente, relation/reason/revision). Le lifecycle complet de résolution/historique est reporté à `0.3.17`.
|
||||||
|
|
||||||
La base est liée à un seul `RawNetworkId` via `ksp_store_identity`. Une migration enregistrée mais physiquement divergente est un mismatch ; les réparations additives sûres dépendent de `schema_autoupdate`.
|
La base est liée à un seul `RawNetworkId` via `ksp_store_identity`. Une migration enregistrée mais physiquement divergente est un mismatch ; les réparations additives sûres dépendent de `schema_autoupdate`.
|
||||||
|
|
||||||
@@ -108,9 +119,9 @@ persist_raw_transaction_acquisition
|
|||||||
record_raw_transaction_observation
|
record_raw_transaction_observation
|
||||||
```
|
```
|
||||||
|
|
||||||
L'acquisition canonique et son observation initiale sont commises dans une seule transaction PostgreSQL. Les clés uniques physiques fournissent l'admission idempotente ; après un conflit unique, le backend verrouille la ligne gagnante et compare le contenu réel avant de conclure `AlreadyPresent` ou `Conflict`.
|
L'acquisition et son observation sont commises dans une seule transaction PostgreSQL. Après collision d'identité, le backend verrouille la ligne V001 avec `FOR UPDATE`, compare le contenu via le comparateur partagé puis persiste/réutilise la variante reçue. `Exact` et `CompatibleLessComplete` conservent le canonique, `CompatibleMoreComplete` met à jour atomiquement selector V003 et projection V001, tandis que `Conflict`/`Incomparable` ouvrent ou mettent à jour un conflict case durable sans modifier le canonique. La nouvelle observation est rattachée à la variante réellement reçue avant `COMMIT`.
|
||||||
|
|
||||||
Un tombstone `Purged` compatible produit `SkippedPurged/NotRecorded` en mode normal. `ForceRehydrate` restaure explicitement le payload `Full` et l'observation dans la même transaction.
|
Un tombstone `Purged` compatible produit `SkippedPurged/NotRecorded` en mode normal. `ForceRehydrate` restaure explicitement le payload `Full` et l'observation dans la même transaction. `content_hash` n'est qu'un préfiltre : les bytes disponibles restent l'autorité d'égalité.
|
||||||
|
|
||||||
## Pagination RAW transaction
|
## Pagination RAW transaction
|
||||||
|
|
||||||
@@ -181,4 +192,6 @@ La crate ne contient :
|
|||||||
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — design `RawTransaction` ;
|
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — design `RawTransaction` ;
|
||||||
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction` ;
|
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction` ;
|
||||||
- [`../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md`](../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md) — design `RawAccountState` et complétude RAW ;
|
- [`../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md`](../../docs/plans/025-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT_PLAN.md) — design `RawAccountState` et complétude RAW ;
|
||||||
- [`../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md`](../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md) — validation `RawAccountState` et conformance RAW.
|
- [`../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md`](../../docs/validation/021-V0_3_4_STORE_POSTGRES_RAW_ACCOUNT.md) — validation `RawAccountState` et conformance RAW ;
|
||||||
|
- [`../../docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md`](../../docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md) — V003, convergence et conflit durable ;
|
||||||
|
- [`../../docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md`](../../docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md) — validation `0.3.16`, concurrence/rollback et live PostgreSQL 17.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-store-postgres-lib/USAGE.md -->
|
<!-- file: crates/ksp-store-postgres-lib/USAGE.md -->
|
||||||
<!-- version: 11 -->
|
<!-- version: 12 -->
|
||||||
|
|
||||||
# Utilisation de ksp-store-postgres-lib
|
# Utilisation de ksp-store-postgres-lib
|
||||||
|
|
||||||
@@ -155,7 +155,7 @@ async fn persist_acquisition(
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
L'opération est atomique : le canonique et son observation initiale sont tous deux durables ou aucun ne l'est. Une identité déjà présente avec un contenu identique est idempotente ; un contenu divergent retourne `PostgresBackendErrorKind::Conflict` sans overwrite silencieux.
|
L'opération est atomique : variante reçue, observation, mapping observation->variante, selector/projection éventuels et conflict case sont tous durables ou aucun ne l'est. Une identité déjà présente avec un contenu exact est idempotente. Une divergence conservable retourne un succès `RawAcquisitionWriteOutcome` dont `transaction_variant()` vaut `QuarantinedConflict`; le canonique n'est pas écrasé. Une représentation strictement plus complète selon la seule preuve `logMessages` admise peut retourner `PromotedCompatibleMoreComplete` et promouvoir atomiquement le selector/projection. Les conflits d'observation directe et autres contrats legacy peuvent encore produire `PostgresBackendErrorKind::Conflict`.
|
||||||
|
|
||||||
Pour un tombstone purgé compatible, le mode `Normal` ne restaure pas le payload. `ForceRehydrate` doit être demandé explicitement pour rétablir un payload `Full`.
|
Pour un tombstone purgé compatible, le mode `Normal` ne restaure pas le payload. `ForceRehydrate` doit être demandé explicitement pour rétablir un payload `Full`.
|
||||||
|
|
||||||
@@ -289,4 +289,4 @@ fn classify(error: &ksp_store_postgres_lib::PostgresBackendError) {
|
|||||||
|
|
||||||
Le backend ne lit aucune variable d'environnement et ne possède aucune sélection de target Config. Les applications, jobs et workers doivent normalement passer par `ksp-store-lib`.
|
Le backend ne lit aucune variable d'environnement et ne possède aucune sélection de target Config. Les applications, jobs et workers doivent normalement passer par `ksp-store-lib`.
|
||||||
|
|
||||||
Les dix capabilities RAW communes sont implémentées par ce backend. Les décisions de batch, priorité, backlog, scheduling et policy de rétention restent hors de sa responsabilité ; aucune rétention/archivage/purge account n'est fournie.
|
Les quatorze capabilities RAW communes sont implémentées par ce backend, dont les quatre capabilities d'inspection entity/observation. Les décisions de batch, priorité, backlog, scheduling et policy de rétention restent hors de sa responsabilité ; aucune rétention/archivage/purge account n'est fournie.
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'pk_ksp_raw_transaction_variants'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT pk_ksp_raw_transaction_variants PRIMARY KEY (transaction_signature, variant_id);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_variants_identity'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT ck_ksp_raw_transaction_variants_identity CHECK (octet_length(transaction_signature) = 64 AND variant_id >= 1 AND variant_id <= 18446744073709551615);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_variants_origin'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT ck_ksp_raw_transaction_variants_origin CHECK (origin_kind = 'native' OR origin_kind = 'synthetic');
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_variants_slot_time'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT ck_ksp_raw_transaction_variants_slot_time CHECK (slot >= 0 AND slot <= 18446744073709551615 AND (block_time_unix_millis IS NULL OR (block_time_unix_millis >= 0 AND block_time_unix_millis <= 253402300799999)) AND created_at_unix_millis >= 0 AND created_at_unix_millis <= 253402300799999);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_variants_format'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT ck_ksp_raw_transaction_variants_format CHECK (char_length(format_id) >= 1 AND char_length(format_id) <= 128 AND format_version >= 1 AND format_version <= 4294967295 AND octet_length(content_hash) = 32);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_variants_payload'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT ck_ksp_raw_transaction_variants_payload CHECK ((retention_state = 'full' AND payload IS NOT NULL AND octet_length(payload) >= 1 AND octet_length(payload) <= 16777216) OR ((retention_state = 'archived' OR retention_state = 'purged') AND payload IS NULL));
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_variants_transaction'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD CONSTRAINT fk_ksp_raw_transaction_variants_transaction FOREIGN KEY (transaction_signature) REFERENCES ksp_raw_transactions(signature) ON DELETE RESTRICT;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'pk_ksp_raw_transaction_canonical_selectors'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_canonical_selectors')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD CONSTRAINT pk_ksp_raw_transaction_canonical_selectors PRIMARY KEY (transaction_signature);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_canonical_selectors_bounds'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_canonical_selectors')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD CONSTRAINT ck_ksp_raw_transaction_canonical_selectors_bounds CHECK (octet_length(transaction_signature) = 64 AND canonical_variant_id >= 1 AND canonical_variant_id <= 18446744073709551615 AND canonical_revision >= 1 AND canonical_revision <= 18446744073709551615 AND updated_at_unix_millis >= 0 AND updated_at_unix_millis <= 253402300799999);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_canonical_selectors_variant'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_canonical_selectors')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD CONSTRAINT fk_ksp_raw_transaction_canonical_selectors_variant FOREIGN KEY (transaction_signature, canonical_variant_id) REFERENCES ksp_raw_transaction_variants(transaction_signature, variant_id) ON DELETE RESTRICT;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'pk_ksp_raw_transaction_observation_variants'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_observation_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD CONSTRAINT pk_ksp_raw_transaction_observation_variants PRIMARY KEY (observation_key);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_observation_variants_bounds'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_observation_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD CONSTRAINT ck_ksp_raw_transaction_observation_variants_bounds CHECK (octet_length(observation_key) = 32 AND octet_length(transaction_signature) = 64 AND variant_id >= 1 AND variant_id <= 18446744073709551615 AND linked_at_unix_millis >= 0 AND linked_at_unix_millis <= 253402300799999);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_observation_variants_observation'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_observation_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD CONSTRAINT fk_ksp_raw_transaction_observation_variants_observation FOREIGN KEY (observation_key) REFERENCES ksp_raw_transaction_observations(observation_key) ON DELETE CASCADE;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_observation_variants_variant'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_observation_variants')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD CONSTRAINT fk_ksp_raw_transaction_observation_variants_variant FOREIGN KEY (transaction_signature, variant_id) REFERENCES ksp_raw_transaction_variants(transaction_signature, variant_id) ON DELETE RESTRICT;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'pk_ksp_raw_transaction_conflicts'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_conflicts')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD CONSTRAINT pk_ksp_raw_transaction_conflicts PRIMARY KEY (transaction_signature);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'ck_ksp_raw_transaction_conflicts_bounds'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_conflicts')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD CONSTRAINT ck_ksp_raw_transaction_conflicts_bounds CHECK (octet_length(transaction_signature) = 64 AND (status = 'open' OR status = 'resolved') AND revision >= 1 AND revision <= 18446744073709551615 AND canonical_variant_id >= 1 AND canonical_variant_id <= 18446744073709551615 AND incoming_variant_id >= 1 AND incoming_variant_id <= 18446744073709551615 AND canonical_variant_id <> incoming_variant_id AND (latest_relation = 'conflict' OR latest_relation = 'incomparable') AND ((latest_relation = 'conflict' AND (latest_reason_code = 'slot_mismatch' OR latest_reason_code = 'block_time_mismatch' OR latest_reason_code = 'canonical_payload_conflict' OR latest_reason_code = 'content_hash_collision')) OR (latest_relation = 'incomparable' AND (latest_reason_code = 'payload_format_mismatch' OR latest_reason_code = 'unsupported_canonical_difference'))) AND created_at_unix_millis >= 0 AND created_at_unix_millis <= 253402300799999 AND updated_at_unix_millis >= created_at_unix_millis AND updated_at_unix_millis <= 253402300799999);
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_conflicts_canonical_variant'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_conflicts')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD CONSTRAINT fk_ksp_raw_transaction_conflicts_canonical_variant FOREIGN KEY (transaction_signature, canonical_variant_id) REFERENCES ksp_raw_transaction_variants(transaction_signature, variant_id) ON DELETE RESTRICT;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
DO $ksp$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM pg_constraint
|
||||||
|
WHERE conname = 'fk_ksp_raw_transaction_conflicts_incoming_variant'
|
||||||
|
AND conrelid = to_regclass('ksp_raw_transaction_conflicts')
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD CONSTRAINT fk_ksp_raw_transaction_conflicts_incoming_variant FOREIGN KEY (transaction_signature, incoming_variant_id) REFERENCES ksp_raw_transaction_variants(transaction_signature, variant_id) ON DELETE RESTRICT;
|
||||||
|
END IF;
|
||||||
|
END
|
||||||
|
$ksp$;
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
CREATE INDEX IF NOT EXISTS ix_ksp_raw_transaction_variants_signature_hash_variant
|
||||||
|
ON ksp_raw_transaction_variants (transaction_signature, content_hash, variant_id);
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
CREATE INDEX IF NOT EXISTS ix_ksp_raw_transaction_observation_variants_variant
|
||||||
|
ON ksp_raw_transaction_observation_variants (transaction_signature, variant_id, observation_key);
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
CREATE INDEX IF NOT EXISTS ix_ksp_raw_transaction_conflicts_status_updated_signature
|
||||||
|
ON ksp_raw_transaction_conflicts (status, updated_at_unix_millis, transaction_signature);
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_variants (
|
||||||
|
transaction_signature BYTEA NOT NULL,
|
||||||
|
variant_id NUMERIC(20, 0) NOT NULL,
|
||||||
|
origin_kind TEXT NOT NULL,
|
||||||
|
slot NUMERIC(20, 0) NOT NULL,
|
||||||
|
block_time_unix_millis BIGINT NULL,
|
||||||
|
format_id TEXT NOT NULL,
|
||||||
|
format_version BIGINT NOT NULL,
|
||||||
|
content_hash BYTEA NOT NULL,
|
||||||
|
payload BYTEA NULL,
|
||||||
|
retention_state TEXT NOT NULL,
|
||||||
|
created_at_unix_millis BIGINT NOT NULL
|
||||||
|
);
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS transaction_signature BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS variant_id NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS origin_kind TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS slot NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS block_time_unix_millis BIGINT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS format_id TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS format_version BIGINT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS content_hash BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS payload BYTEA NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS retention_state TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_variants ADD COLUMN IF NOT EXISTS created_at_unix_millis BIGINT NOT NULL;
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_canonical_selectors (
|
||||||
|
transaction_signature BYTEA NOT NULL,
|
||||||
|
canonical_variant_id NUMERIC(20, 0) NOT NULL,
|
||||||
|
canonical_revision NUMERIC(20, 0) NOT NULL,
|
||||||
|
updated_at_unix_millis BIGINT NOT NULL
|
||||||
|
);
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD COLUMN IF NOT EXISTS transaction_signature BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD COLUMN IF NOT EXISTS canonical_variant_id NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD COLUMN IF NOT EXISTS canonical_revision NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_canonical_selectors ADD COLUMN IF NOT EXISTS updated_at_unix_millis BIGINT NOT NULL;
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_observation_variants (
|
||||||
|
observation_key BYTEA NOT NULL,
|
||||||
|
transaction_signature BYTEA NOT NULL,
|
||||||
|
variant_id NUMERIC(20, 0) NOT NULL,
|
||||||
|
linked_at_unix_millis BIGINT NOT NULL
|
||||||
|
);
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD COLUMN IF NOT EXISTS observation_key BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD COLUMN IF NOT EXISTS transaction_signature BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD COLUMN IF NOT EXISTS variant_id NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_observation_variants ADD COLUMN IF NOT EXISTS linked_at_unix_millis BIGINT NOT NULL;
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_conflicts (
|
||||||
|
transaction_signature BYTEA NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
revision NUMERIC(20, 0) NOT NULL,
|
||||||
|
canonical_variant_id NUMERIC(20, 0) NOT NULL,
|
||||||
|
incoming_variant_id NUMERIC(20, 0) NOT NULL,
|
||||||
|
latest_relation TEXT NOT NULL,
|
||||||
|
latest_reason_code TEXT NOT NULL,
|
||||||
|
created_at_unix_millis BIGINT NOT NULL,
|
||||||
|
updated_at_unix_millis BIGINT NOT NULL
|
||||||
|
);
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS transaction_signature BYTEA NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS status TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS revision NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS canonical_variant_id NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS incoming_variant_id NUMERIC(20, 0) NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS latest_relation TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS latest_reason_code TEXT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS created_at_unix_millis BIGINT NOT NULL;
|
||||||
|
ALTER TABLE ksp_raw_transaction_conflicts ADD COLUMN IF NOT EXISTS updated_at_unix_millis BIGINT NOT NULL;
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/src/health.rs
|
// file: crates/ksp-store-postgres-lib/src/health.rs
|
||||||
// version: 1
|
// version: 2
|
||||||
|
|
||||||
const DEFAULT_HEALTH_TIMEOUT_MS: u64 = 5_000;
|
const DEFAULT_HEALTH_TIMEOUT_MS: u64 = 5_000;
|
||||||
const MIGRATION_VERSION_SQL: &str = "SELECT COALESCE(MAX(version), -1)::BIGINT FROM ksp_store_schema_migrations";
|
const MIGRATION_VERSION_SQL: &str = "SELECT COALESCE(MAX(version), -1)::BIGINT FROM ksp_store_schema_migrations";
|
||||||
@@ -61,7 +61,7 @@ async fn probe_health_inner(pool: &deadpool_postgres::Pool, runtime: crate::Post
|
|||||||
return crate::PostgresBackendHealthSnapshot::not_ready(runtime, std::option::Option::None, 0, crate::PostgresBackendErrorKind::HealthFailed);
|
return crate::PostgresBackendHealthSnapshot::not_ready(runtime, std::option::Option::None, 0, crate::PostgresBackendErrorKind::HealthFailed);
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
let expected = crate::current_migration_version();
|
let expected = crate::current_migration_version_v003();
|
||||||
if migration_version < 0 || migration_version < expected {
|
if migration_version < 0 || migration_version < expected {
|
||||||
let observed = nonnegative_version(migration_version);
|
let observed = nonnegative_version(migration_version);
|
||||||
let pending = pending_migration_count(migration_version, expected);
|
let pending = pending_migration_count(migration_version, expected);
|
||||||
|
|||||||
@@ -1,10 +1,8 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/src/lib.rs
|
// file: crates/ksp-store-postgres-lib/src/lib.rs
|
||||||
// version: 24
|
// version: 28
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
#![forbid(unsafe_code)]
|
#![forbid(unsafe_code)]
|
||||||
|
|
||||||
//! Official PostgreSQL backend implementation for KSP Store.
|
//! Official PostgreSQL backend implementation for KSP Store.
|
||||||
//!
|
//!
|
||||||
//! The backend owns the physical `tokio-postgres` connection, bounded Deadpool
|
//! The backend owns the physical `tokio-postgres` connection, bounded Deadpool
|
||||||
@@ -38,19 +36,24 @@
|
|||||||
//! with exact counts while preserving both keyset traversal families unchanged.
|
//! with exact counts while preserving both keyset traversal families unchanged.
|
||||||
//! `0.3.8-pre.009` adds safe random-access observation inspection for both RAW
|
//! `0.3.8-pre.009` adds safe random-access observation inspection for both RAW
|
||||||
//! families without changing the physical schema or read-by-key contracts.
|
//! families without changing the physical schema or read-by-key contracts.
|
||||||
|
//! `0.3.16-pre.003` layers additive V003 transaction-variant resources over the
|
||||||
|
//! frozen V000-V002 registry without changing any prior migration resource bytes.
|
||||||
|
//! `0.3.16-pre.004` wires acquisition writes into the V003 variant ledger with
|
||||||
|
//! lazy canonical bootstrap, exact native-variant reuse and exact observation linkage.
|
||||||
//!
|
//!
|
||||||
//! This crate depends on `ksp-store-api` and never on `ksp-store-lib`. The
|
//! This crate depends on `ksp-store-api` and never on `ksp-store-lib`. The
|
||||||
//! common facade consumes only this crate's narrow backend bridge and never
|
//! common facade consumes only this crate's narrow backend bridge and never
|
||||||
//! exposes PostgreSQL pool, client, row or statement types.
|
//! exposes PostgreSQL pool, client, row or statement types.
|
||||||
|
|
||||||
mod constants;
|
mod constants;
|
||||||
mod error;
|
mod error;
|
||||||
mod health;
|
mod health;
|
||||||
mod migration;
|
mod migration;
|
||||||
|
mod migration_v003;
|
||||||
mod raw_account;
|
mod raw_account;
|
||||||
mod raw_transaction;
|
mod raw_transaction;
|
||||||
mod runtime;
|
mod runtime;
|
||||||
mod schema;
|
mod schema;
|
||||||
|
mod schema_v003;
|
||||||
|
|
||||||
/// Stable KSP error code for unsupported PostgreSQL retention compaction.
|
/// Stable KSP error code for unsupported PostgreSQL retention compaction.
|
||||||
pub use self::error::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED;
|
pub use self::error::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED;
|
||||||
@@ -73,10 +76,14 @@ pub use self::runtime::PostgresBackendTlsMode;
|
|||||||
pub(crate) use self::constants::TRACING_TARGET;
|
pub(crate) use self::constants::TRACING_TARGET;
|
||||||
/// Private bounded health probe consumed by the physical backend runtime.
|
/// Private bounded health probe consumed by the physical backend runtime.
|
||||||
pub(crate) use self::health::probe_health;
|
pub(crate) use self::health::probe_health;
|
||||||
/// Private migration/bootstrap runner consumed by the physical backend runtime.
|
/// Frozen V000-V002 migration/bootstrap runner consumed only by the V003 extension.
|
||||||
pub(crate) use self::migration::bootstrap;
|
pub(crate) use self::migration::bootstrap;
|
||||||
/// Current embedded migration version consumed by the private health probe.
|
/// Frozen V000-V002 latest migration version consumed only by the V003 extension.
|
||||||
pub(crate) use self::migration::current_migration_version;
|
pub(crate) use self::migration::current_migration_version;
|
||||||
|
/// Current V003 migration/bootstrap runner consumed by the physical backend runtime.
|
||||||
|
pub(crate) use self::migration_v003::bootstrap_v003;
|
||||||
|
/// Current embedded V003 migration version consumed by the private health probe.
|
||||||
|
pub(crate) use self::migration_v003::current_migration_version_v003;
|
||||||
/// Private RAW account cursor decoder consumed by the physical RAW account module.
|
/// Private RAW account cursor decoder consumed by the physical RAW account module.
|
||||||
pub(crate) use self::raw_account::cursor::decode_raw_account_cursor;
|
pub(crate) use self::raw_account::cursor::decode_raw_account_cursor;
|
||||||
/// Private RAW account cursor encoder consumed by the physical RAW account module.
|
/// Private RAW account cursor encoder consumed by the physical RAW account module.
|
||||||
@@ -127,23 +134,33 @@ pub(crate) use self::raw_transaction::transition_raw_transaction_retention;
|
|||||||
pub(crate) use self::runtime::map_pool_error;
|
pub(crate) use self::runtime::map_pool_error;
|
||||||
/// Private Deadpool status projector shared with the health probe.
|
/// Private Deadpool status projector shared with the health probe.
|
||||||
pub(crate) use self::runtime::runtime_snapshot_from_status;
|
pub(crate) use self::runtime::runtime_snapshot_from_status;
|
||||||
/// Private physical schema resource descriptor consumed by the migration engine.
|
/// Private physical schema resource descriptor consumed by the frozen migration engine.
|
||||||
pub(crate) use self::schema::SchemaResource;
|
pub(crate) use self::schema::SchemaResource;
|
||||||
/// Private physical schema resource compatibility state consumed by the migration engine.
|
/// Private physical schema resource compatibility state consumed by the frozen migration engine.
|
||||||
pub(crate) use self::schema::SchemaResourceState;
|
pub(crate) use self::schema::SchemaResourceState;
|
||||||
/// Private V000 schema resource inventory consumed by the migration engine.
|
/// Private V000 schema resource inventory consumed by the migration engines.
|
||||||
pub(crate) use self::schema::V000_RESOURCES;
|
pub(crate) use self::schema::V000_RESOURCES;
|
||||||
/// Private V001 schema resource inventory consumed by the migration engine.
|
/// Private V001 schema resource inventory consumed by the migration engines.
|
||||||
pub(crate) use self::schema::V001_RESOURCES;
|
pub(crate) use self::schema::V001_RESOURCES;
|
||||||
/// Private V002 schema resource inventory consumed by the migration engine.
|
/// Private V002 schema resource inventory consumed by the migration engines.
|
||||||
pub(crate) use self::schema::V002_RESOURCES;
|
pub(crate) use self::schema::V002_RESOURCES;
|
||||||
/// Private physical schema resource inspector consumed by the migration engine.
|
/// Private physical schema resource inspector consumed by the migration engines.
|
||||||
pub(crate) use self::schema::inspect_resource;
|
pub(crate) use self::schema::inspect_resource;
|
||||||
/// Private managed-schema adoption probe consumed by the migration engine.
|
/// Private managed-schema adoption probe consumed by the frozen migration engine.
|
||||||
pub(crate) use self::schema::managed_schema_objects_exist;
|
pub(crate) use self::schema::managed_schema_objects_exist;
|
||||||
/// Private V001 external-schema compatibility gate consumed by the migration engine.
|
/// Private V001 external-schema compatibility gate consumed by the migration engines.
|
||||||
pub(crate) use self::schema::verify_v001_external_compatibility;
|
pub(crate) use self::schema::verify_v001_external_compatibility;
|
||||||
/// Private V002 external-schema compatibility gate consumed by the migration engine.
|
/// Private V002 external-schema compatibility gate consumed by the migration engines.
|
||||||
pub(crate) use self::schema::verify_v002_external_compatibility;
|
pub(crate) use self::schema::verify_v002_external_compatibility;
|
||||||
|
/// Private V003 schema resource inventory consumed by the V003 migration extension.
|
||||||
|
pub(crate) use self::schema_v003::V003_RESOURCES;
|
||||||
|
/// Private V003 schema resource descriptor consumed by the V003 migration extension.
|
||||||
|
pub(crate) use self::schema_v003::V003SchemaResource;
|
||||||
|
/// Private V003 schema compatibility state consumed by the V003 migration extension.
|
||||||
|
pub(crate) use self::schema_v003::V003SchemaResourceState;
|
||||||
|
/// Private V003 schema inspector consumed by the V003 migration extension.
|
||||||
|
pub(crate) use self::schema_v003::inspect_v003_resource;
|
||||||
|
/// Private V003 external-schema compatibility gate consumed by the V003 migration extension.
|
||||||
|
pub(crate) use self::schema_v003::verify_v003_external_compatibility;
|
||||||
|
|
||||||
const _: &str = crate::TRACING_TARGET;
|
const _: &str = crate::TRACING_TARGET;
|
||||||
|
|||||||
499
crates/ksp-store-postgres-lib/src/migration_v003.rs
Normal file
499
crates/ksp-store-postgres-lib/src/migration_v003.rs
Normal file
@@ -0,0 +1,499 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/src/migration_v003.rs
|
||||||
|
// version: 3
|
||||||
|
|
||||||
|
use sha2::Digest; // rust-rules: trait-import
|
||||||
|
|
||||||
|
const ADVISORY_LOCK_KEY: i64 = 0x4b53_5053_544f_5245;
|
||||||
|
const HISTORY_INSERT_SQL: &str = "INSERT INTO ksp_store_schema_migrations (version, name, checksum, applied_at) VALUES ($1, $2, $3, CURRENT_TIMESTAMP)";
|
||||||
|
const HISTORY_LOAD_SQL: &str = "SELECT version, name, checksum FROM ksp_store_schema_migrations ORDER BY version";
|
||||||
|
const IDENTITY_LOAD_SQL: &str = "SELECT singleton, network FROM ksp_store_identity ORDER BY singleton LIMIT 2";
|
||||||
|
const LATEST_VERSION_SQL: &str = "SELECT MAX(version) FROM ksp_store_schema_migrations";
|
||||||
|
const LOCK_POLL_INTERVAL_MS: u64 = 25;
|
||||||
|
const METADATA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||||
|
SELECT 1 FROM information_schema.tables
|
||||||
|
WHERE table_schema = current_schema()
|
||||||
|
AND table_name = 'ksp_store_schema_migrations'
|
||||||
|
AND table_type = 'BASE TABLE'
|
||||||
|
)"#;
|
||||||
|
const SET_STATEMENT_TIMEOUT_SQL: &str = "SELECT set_config('statement_timeout', $1, true)";
|
||||||
|
const V000_CHECKSUM: &str = "d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450";
|
||||||
|
const V001_CHECKSUM: &str = "31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51";
|
||||||
|
const V002_CHECKSUM: &str = "ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e";
|
||||||
|
const V002_VERSION: i64 = 2;
|
||||||
|
const V003_NAME: &str = "raw_transaction_variants";
|
||||||
|
const V003_VERSION: i64 = 3;
|
||||||
|
|
||||||
|
struct AppliedMigration {
|
||||||
|
checksum: std::string::String,
|
||||||
|
name: std::string::String,
|
||||||
|
version: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the latest migration version embedded by the physical PostgreSQL backend.
|
||||||
|
#[must_use]
|
||||||
|
pub(crate) const fn current_migration_version_v003() -> i64 {
|
||||||
|
return V003_VERSION;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Runs the frozen V000-V002 bootstrap followed by additive V003 under the KSP schema lock.
|
||||||
|
pub(crate) async fn bootstrap_v003(
|
||||||
|
client: &mut deadpool_postgres::Client,
|
||||||
|
network: &ksp_store_api::RawNetworkId,
|
||||||
|
schema_autocreate: bool,
|
||||||
|
schema_autoupdate: bool,
|
||||||
|
migration_timeout: std::time::Duration,
|
||||||
|
migration_lock_timeout: std::time::Duration,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
if crate::current_migration_version() != 2 {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "v003_legacy_registry_version"));
|
||||||
|
}
|
||||||
|
let metadata_result = metadata_exists_client(client).await;
|
||||||
|
let metadata_preexisting = match metadata_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let latest_before = if metadata_preexisting {
|
||||||
|
let latest_result = latest_version_client(client).await;
|
||||||
|
match latest_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
std::option::Option::None
|
||||||
|
};
|
||||||
|
if let std::option::Option::Some(version) = latest_before
|
||||||
|
&& version > V003_VERSION
|
||||||
|
{
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||||
|
}
|
||||||
|
if legacy_bootstrap_required(latest_before) {
|
||||||
|
let legacy_result = crate::bootstrap(client, network, schema_autocreate, schema_autoupdate, migration_timeout, migration_lock_timeout).await;
|
||||||
|
if let std::result::Result::Err(error) = legacy_result {
|
||||||
|
if error.kind() != crate::PostgresBackendErrorKind::SchemaNewer {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let latest_result = latest_version_client(client).await;
|
||||||
|
let latest = match latest_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(latest_error) => return std::result::Result::Err(latest_error),
|
||||||
|
};
|
||||||
|
if !legacy_schema_newer_is_completed_v003(latest) {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let allow_v003_apply = v003_apply_allowed(metadata_preexisting, schema_autoupdate);
|
||||||
|
let bounded = tokio::time::timeout(
|
||||||
|
migration_timeout,
|
||||||
|
bootstrap_v003_inner(client, network, schema_autoupdate, allow_v003_apply, migration_timeout, migration_lock_timeout),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
return match bounded {
|
||||||
|
std::result::Result::Ok(result) => result,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_timeout"))
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn bootstrap_v003_inner(
|
||||||
|
client: &mut deadpool_postgres::Client,
|
||||||
|
network: &ksp_store_api::RawNetworkId,
|
||||||
|
schema_autoupdate: bool,
|
||||||
|
allow_v003_apply: bool,
|
||||||
|
migration_timeout: std::time::Duration,
|
||||||
|
migration_lock_timeout: std::time::Duration,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
let transaction_result = client.transaction().await;
|
||||||
|
let transaction = match transaction_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_begin"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let lock_result = acquire_advisory_lock(&transaction, migration_lock_timeout).await;
|
||||||
|
if let std::result::Result::Err(error) = lock_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let timeout_result = set_statement_timeout(&transaction, migration_timeout).await;
|
||||||
|
if let std::result::Result::Err(error) = timeout_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let history_result = load_history(&transaction).await;
|
||||||
|
let history = match history_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let next_version_result = validate_history(history.as_slice());
|
||||||
|
let next_version = match next_version_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let legacy_result = verify_or_repair_legacy_resources(&transaction, schema_autoupdate).await;
|
||||||
|
if let std::result::Result::Err(error) = legacy_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let identity_result = validate_store_identity(&transaction, network).await;
|
||||||
|
if let std::result::Result::Err(error) = identity_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let v003_applied = next_version > V003_VERSION;
|
||||||
|
let v003_result = verify_or_apply_v003_resources(&transaction, schema_autoupdate, allow_v003_apply, v003_applied).await;
|
||||||
|
if let std::result::Result::Err(error) = v003_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let compatibility_result = crate::verify_v003_external_compatibility(&transaction).await;
|
||||||
|
if let std::result::Result::Err(error) = compatibility_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
if !v003_applied {
|
||||||
|
let checksum = migration_checksum_v003();
|
||||||
|
let insert_result = transaction.execute(HISTORY_INSERT_SQL, &[&V003_VERSION, &V003_NAME, &checksum]).await;
|
||||||
|
match insert_result {
|
||||||
|
std::result::Result::Ok(1) => {},
|
||||||
|
std::result::Result::Ok(_) | std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_insert"));
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let commit_result = transaction.commit().await;
|
||||||
|
return match commit_result {
|
||||||
|
std::result::Result::Ok(()) => std::result::Result::Ok(()),
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_commit"))
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn acquire_advisory_lock(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
timeout: std::time::Duration,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
let started = tokio::time::Instant::now();
|
||||||
|
let deadline = match started.checked_add(timeout) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_lock_timeout"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
loop {
|
||||||
|
let row_result = transaction.query_one("SELECT pg_try_advisory_xact_lock($1)", &[&ADVISORY_LOCK_KEY]).await;
|
||||||
|
let row = match row_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_lock"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let acquired_result = row.try_get::<usize, bool>(0);
|
||||||
|
let acquired = match acquired_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_lock_decode"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
if acquired {
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
}
|
||||||
|
let now = tokio::time::Instant::now();
|
||||||
|
if now >= deadline {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_lock_timeout"));
|
||||||
|
}
|
||||||
|
let candidate = now + std::time::Duration::from_millis(LOCK_POLL_INTERVAL_MS);
|
||||||
|
let wake = if candidate < deadline { candidate } else { deadline };
|
||||||
|
tokio::time::sleep_until(wake).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn verify_or_repair_legacy_resources(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
schema_autoupdate: bool,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
for resources in [crate::V000_RESOURCES, crate::V001_RESOURCES, crate::V002_RESOURCES] {
|
||||||
|
for resource in resources {
|
||||||
|
let state_result = crate::inspect_resource(transaction, resource).await;
|
||||||
|
let state = match state_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
match state {
|
||||||
|
crate::SchemaResourceState::Compatible => {},
|
||||||
|
crate::SchemaResourceState::Incompatible => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_resource_incompatible",
|
||||||
|
));
|
||||||
|
},
|
||||||
|
crate::SchemaResourceState::Missing => {
|
||||||
|
if !schema_autoupdate {
|
||||||
|
return std::result::Result::Err(schema_autoupdate_disabled_error());
|
||||||
|
}
|
||||||
|
if !resource.repair_existing {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_resource_missing",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let execute_result = transaction.batch_execute(resource.sql).await;
|
||||||
|
if execute_result.is_err() {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationFailed,
|
||||||
|
"schema_resource_apply",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let verify_result = crate::inspect_resource(transaction, resource).await;
|
||||||
|
match verify_result {
|
||||||
|
std::result::Result::Ok(crate::SchemaResourceState::Compatible) => {},
|
||||||
|
std::result::Result::Ok(crate::SchemaResourceState::Missing | crate::SchemaResourceState::Incompatible) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_resource_post_apply",
|
||||||
|
));
|
||||||
|
},
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let v001_result = crate::verify_v001_external_compatibility(transaction).await;
|
||||||
|
if let std::result::Result::Err(error) = v001_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
return crate::verify_v002_external_compatibility(transaction).await;
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn verify_or_apply_v003_resources(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
schema_autoupdate: bool,
|
||||||
|
allow_v003_apply: bool,
|
||||||
|
applied_history: bool,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
for resource in crate::V003_RESOURCES {
|
||||||
|
let state_result = crate::inspect_v003_resource(transaction, resource).await;
|
||||||
|
let state = match state_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
match state {
|
||||||
|
crate::V003SchemaResourceState::Compatible => {},
|
||||||
|
crate::V003SchemaResourceState::Incompatible => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_v003_resource_incompatible",
|
||||||
|
));
|
||||||
|
},
|
||||||
|
crate::V003SchemaResourceState::Missing => {
|
||||||
|
if applied_history && !schema_autoupdate {
|
||||||
|
return std::result::Result::Err(schema_autoupdate_disabled_error());
|
||||||
|
}
|
||||||
|
if !applied_history && !allow_v003_apply {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_pending"));
|
||||||
|
}
|
||||||
|
if applied_history && !resource.repair_existing {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_v003_resource_missing",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let execute_result = transaction.batch_execute(resource.sql).await;
|
||||||
|
if execute_result.is_err() {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationFailed,
|
||||||
|
"schema_v003_resource_apply",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let verify_result = crate::inspect_v003_resource(transaction, resource).await;
|
||||||
|
match verify_result {
|
||||||
|
std::result::Result::Ok(crate::V003SchemaResourceState::Compatible) => {},
|
||||||
|
std::result::Result::Ok(crate::V003SchemaResourceState::Missing | crate::V003SchemaResourceState::Incompatible) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||||
|
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||||
|
"schema_v003_resource_post_apply",
|
||||||
|
));
|
||||||
|
},
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn validate_store_identity(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
network: &ksp_store_api::RawNetworkId,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
let rows_result = transaction.query(IDENTITY_LOAD_SQL, &[]).await;
|
||||||
|
let rows = match rows_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_read"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
if rows.len() != 1 {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_count"));
|
||||||
|
}
|
||||||
|
let row = &rows[0];
|
||||||
|
let singleton_result = row.try_get::<usize, i16>(0);
|
||||||
|
let network_result = row.try_get::<usize, std::string::String>(1);
|
||||||
|
let (singleton, stored_network) = match (singleton_result, network_result) {
|
||||||
|
(std::result::Result::Ok(singleton), std::result::Result::Ok(stored_network)) => (singleton, stored_network),
|
||||||
|
_ => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_decode"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
if singleton != 1 || stored_network != network.as_str() {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_network"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn metadata_exists_client(client: &deadpool_postgres::Client) -> std::result::Result<bool, crate::PostgresBackendError> {
|
||||||
|
let result = client.query_one(METADATA_EXISTS_SQL, &[]).await;
|
||||||
|
let row = match result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return match row.try_get::<usize, bool>(0) {
|
||||||
|
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe_decode"))
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn latest_version_client(client: &deadpool_postgres::Client) -> std::result::Result<std::option::Option<i64>, crate::PostgresBackendError> {
|
||||||
|
let result = client.query_one(LATEST_VERSION_SQL, &[]).await;
|
||||||
|
let row = match result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_latest"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return match row.try_get::<usize, std::option::Option<i64>>(0) {
|
||||||
|
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_latest_decode"))
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn load_history(transaction: &deadpool_postgres::Transaction<'_>) -> std::result::Result<std::vec::Vec<AppliedMigration>, crate::PostgresBackendError> {
|
||||||
|
let result = transaction.query(HISTORY_LOAD_SQL, &[]).await;
|
||||||
|
let rows = match result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_load"));
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let mut history = std::vec::Vec::with_capacity(rows.len());
|
||||||
|
for row in rows {
|
||||||
|
let version = row.try_get::<usize, i64>(0);
|
||||||
|
let name = row.try_get::<usize, std::string::String>(1);
|
||||||
|
let checksum = row.try_get::<usize, std::string::String>(2);
|
||||||
|
match (version, name, checksum) {
|
||||||
|
(std::result::Result::Ok(version), std::result::Result::Ok(name), std::result::Result::Ok(checksum)) => {
|
||||||
|
history.push(AppliedMigration { checksum, name, version });
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_decode"));
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(history);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn validate_history(history: &[AppliedMigration]) -> std::result::Result<i64, crate::PostgresBackendError> {
|
||||||
|
if history.len() < 3 {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_missing"));
|
||||||
|
}
|
||||||
|
let expected = [(0_i64, "bootstrap", V000_CHECKSUM), (1_i64, "raw_transaction", V001_CHECKSUM), (2_i64, "raw_account_state", V002_CHECKSUM)];
|
||||||
|
for (index, (version, name, checksum)) in expected.iter().enumerate() {
|
||||||
|
let applied = &history[index];
|
||||||
|
if applied.version != *version || applied.name != *name || applied.checksum != *checksum {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_diverged"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if history.len() == 3 {
|
||||||
|
return std::result::Result::Ok(3);
|
||||||
|
}
|
||||||
|
let v003 = &history[3];
|
||||||
|
if v003.version > V003_VERSION {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||||
|
}
|
||||||
|
if v003.version != V003_VERSION || v003.name != V003_NAME || v003.checksum != migration_checksum_v003() {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_diverged"));
|
||||||
|
}
|
||||||
|
if history.len() > 4 {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(4);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn v003_apply_allowed(metadata_preexisting: bool, schema_autoupdate: bool) -> bool {
|
||||||
|
return !metadata_preexisting || schema_autoupdate;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn legacy_bootstrap_required(latest_before: std::option::Option<i64>) -> bool {
|
||||||
|
let legacy_version = match latest_before {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => -1,
|
||||||
|
};
|
||||||
|
return legacy_version < V002_VERSION;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn legacy_schema_newer_is_completed_v003(latest: std::option::Option<i64>) -> bool {
|
||||||
|
return latest == std::option::Option::Some(V003_VERSION);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn migration_checksum_v003() -> std::string::String {
|
||||||
|
let mut hasher = sha2::Sha256::new();
|
||||||
|
hasher.update(b"ksp-migration-resources-v1\0");
|
||||||
|
for resource in crate::V003_RESOURCES {
|
||||||
|
update_v003_checksum(&mut hasher, resource);
|
||||||
|
}
|
||||||
|
return encode_digest(hasher.finalize().as_slice());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn update_v003_checksum(hasher: &mut sha2::Sha256, resource: &crate::V003SchemaResource) {
|
||||||
|
hasher.update(resource.id.as_bytes());
|
||||||
|
hasher.update([0]);
|
||||||
|
hasher.update(resource.sql.as_bytes());
|
||||||
|
hasher.update([0]);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn encode_digest(bytes: &[u8]) -> std::string::String {
|
||||||
|
const HEX_LOWER: &[u8; 16] = b"0123456789abcdef";
|
||||||
|
let mut encoded = std::string::String::with_capacity(bytes.len() * 2);
|
||||||
|
for byte in bytes {
|
||||||
|
let value = *byte;
|
||||||
|
encoded.push(char::from(HEX_LOWER[(value >> 4) as usize]));
|
||||||
|
encoded.push(char::from(HEX_LOWER[(value & 0x0f) as usize]));
|
||||||
|
}
|
||||||
|
return encoded;
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set_statement_timeout(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
timeout: std::time::Duration,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
let timeout_value = format!("{}ms", timeout.as_millis());
|
||||||
|
let result = transaction.query_one(SET_STATEMENT_TIMEOUT_SQL, &[&timeout_value]).await;
|
||||||
|
return match result {
|
||||||
|
std::result::Result::Ok(_) => std::result::Result::Ok(()),
|
||||||
|
std::result::Result::Err(_) => {
|
||||||
|
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "statement_timeout"))
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_autoupdate_disabled_error() -> crate::PostgresBackendError {
|
||||||
|
return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "schema_autoupdate_disabled");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "../unit_tests/migration_v003.rs"]
|
||||||
|
mod tests;
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/src/runtime.rs
|
// file: crates/ksp-store-postgres-lib/src/runtime.rs
|
||||||
// version: 18
|
// version: 19
|
||||||
|
|
||||||
const APPLICATION_NAME: &str = "ksp-store";
|
const APPLICATION_NAME: &str = "ksp-store";
|
||||||
const MAX_CONNECTION_URI_BYTES: usize = 4_096;
|
const MAX_CONNECTION_URI_BYTES: usize = 4_096;
|
||||||
@@ -298,7 +298,7 @@ impl PostgresBackend {
|
|||||||
tls_mode = settings.tls_mode().code(),
|
tls_mode = settings.tls_mode().code(),
|
||||||
"PostgreSQL Store backend established initial physical connection"
|
"PostgreSQL Store backend established initial physical connection"
|
||||||
);
|
);
|
||||||
let bootstrap_result = crate::bootstrap(
|
let bootstrap_result = crate::bootstrap_v003(
|
||||||
&mut client,
|
&mut client,
|
||||||
settings.network(),
|
settings.network(),
|
||||||
settings.schema_autocreate,
|
settings.schema_autocreate,
|
||||||
|
|||||||
659
crates/ksp-store-postgres-lib/src/schema_v003.rs
Normal file
659
crates/ksp-store-postgres-lib/src/schema_v003.rs
Normal file
@@ -0,0 +1,659 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/src/schema_v003.rs
|
||||||
|
// version: 2
|
||||||
|
|
||||||
|
const COLUMN_LOAD_SQL: &str = r#"SELECT column_name, udt_name, is_nullable = 'YES', numeric_precision, numeric_scale, column_default, is_identity, is_generated
|
||||||
|
FROM information_schema.columns
|
||||||
|
WHERE table_schema = current_schema() AND table_name = $1
|
||||||
|
ORDER BY ordinal_position"#;
|
||||||
|
const CONSTRAINT_LOAD_SQL: &str = r#"SELECT c.contype::TEXT, c.convalidated, c.condeferrable, c.condeferred, pg_get_constraintdef(c.oid, true)
|
||||||
|
FROM pg_constraint c
|
||||||
|
JOIN pg_class t ON t.oid = c.conrelid
|
||||||
|
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||||
|
WHERE n.nspname = current_schema() AND t.relname = $1 AND c.conname = $2"#;
|
||||||
|
const INDEX_LOAD_SQL: &str = r#"SELECT i.indisunique, am.amname, pg_get_indexdef(i.indexrelid), pg_get_expr(i.indpred, i.indrelid)
|
||||||
|
FROM pg_index i
|
||||||
|
JOIN pg_class idx ON idx.oid = i.indexrelid
|
||||||
|
JOIN pg_class tbl ON tbl.oid = i.indrelid
|
||||||
|
JOIN pg_namespace n ON n.oid = tbl.relnamespace
|
||||||
|
JOIN pg_am am ON am.oid = idx.relam
|
||||||
|
WHERE n.nspname = current_schema() AND tbl.relname = $1 AND idx.relname = $2 AND i.indisvalid"#;
|
||||||
|
const TABLE_LOAD_SQL: &str = r#"SELECT c.relkind::TEXT, c.relrowsecurity, c.relforcerowsecurity
|
||||||
|
FROM pg_class c
|
||||||
|
JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||||
|
WHERE n.nspname = current_schema() AND c.relname = $1"#;
|
||||||
|
|
||||||
|
/// Compatibility state of one additive V003 physical schema resource.
|
||||||
|
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||||
|
pub(crate) enum V003SchemaResourceState {
|
||||||
|
/// Resource does not exist or is missing one additive expected column.
|
||||||
|
Missing,
|
||||||
|
/// Resource exactly satisfies the owned V003 physical contract.
|
||||||
|
Compatible,
|
||||||
|
/// Existing resource contradicts the owned V003 physical contract.
|
||||||
|
Incompatible,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Additive V003 physical schema resource descriptor consumed by the V003 migration extension.
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
pub(crate) struct V003SchemaResource {
|
||||||
|
/// Stable resource identifier included in the V003 migration checksum domain.
|
||||||
|
pub(crate) id: &'static str,
|
||||||
|
object: V003SchemaObjectContract,
|
||||||
|
/// Whether an already-applied history may repair a missing additive resource.
|
||||||
|
pub(crate) repair_existing: bool,
|
||||||
|
/// Immutable SQL bytes owned by this V003 physical resource.
|
||||||
|
pub(crate) sql: &'static str,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
enum V003SchemaObjectContract {
|
||||||
|
Constraint(ConstraintContract),
|
||||||
|
Index(IndexContract),
|
||||||
|
Table(TableContract),
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct ConstraintContract {
|
||||||
|
kind: &'static str,
|
||||||
|
name: &'static str,
|
||||||
|
table: &'static str,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct IndexContract {
|
||||||
|
access_method: &'static str,
|
||||||
|
key_fragment: &'static str,
|
||||||
|
name: &'static str,
|
||||||
|
predicate_fragment: std::option::Option<&'static str>,
|
||||||
|
table: &'static str,
|
||||||
|
unique: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct TableContract {
|
||||||
|
columns: &'static [ColumnContract],
|
||||||
|
name: &'static str,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct ColumnContract {
|
||||||
|
name: &'static str,
|
||||||
|
nullable: bool,
|
||||||
|
numeric_precision: std::option::Option<i32>,
|
||||||
|
numeric_scale: std::option::Option<i32>,
|
||||||
|
udt_name: &'static str,
|
||||||
|
}
|
||||||
|
|
||||||
|
const VARIANT_COLUMNS: &[ColumnContract] = &[
|
||||||
|
column("transaction_signature", false, None, None, "bytea"),
|
||||||
|
column("variant_id", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("origin_kind", false, None, None, "text"),
|
||||||
|
column("slot", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("block_time_unix_millis", true, None, None, "int8"),
|
||||||
|
column("format_id", false, None, None, "text"),
|
||||||
|
column("format_version", false, None, None, "int8"),
|
||||||
|
column("content_hash", false, None, None, "bytea"),
|
||||||
|
column("payload", true, None, None, "bytea"),
|
||||||
|
column("retention_state", false, None, None, "text"),
|
||||||
|
column("created_at_unix_millis", false, None, None, "int8"),
|
||||||
|
];
|
||||||
|
const SELECTOR_COLUMNS: &[ColumnContract] = &[
|
||||||
|
column("transaction_signature", false, None, None, "bytea"),
|
||||||
|
column("canonical_variant_id", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("canonical_revision", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("updated_at_unix_millis", false, None, None, "int8"),
|
||||||
|
];
|
||||||
|
const OBSERVATION_VARIANT_COLUMNS: &[ColumnContract] = &[
|
||||||
|
column("observation_key", false, None, None, "bytea"),
|
||||||
|
column("transaction_signature", false, None, None, "bytea"),
|
||||||
|
column("variant_id", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("linked_at_unix_millis", false, None, None, "int8"),
|
||||||
|
];
|
||||||
|
const CONFLICT_COLUMNS: &[ColumnContract] = &[
|
||||||
|
column("transaction_signature", false, None, None, "bytea"),
|
||||||
|
column("status", false, None, None, "text"),
|
||||||
|
column("revision", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("canonical_variant_id", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("incoming_variant_id", false, Some(20), Some(0), "numeric"),
|
||||||
|
column("latest_relation", false, None, None, "text"),
|
||||||
|
column("latest_reason_code", false, None, None, "text"),
|
||||||
|
column("created_at_unix_millis", false, None, None, "int8"),
|
||||||
|
column("updated_at_unix_millis", false, None, None, "int8"),
|
||||||
|
];
|
||||||
|
|
||||||
|
/// Immutable V003 physical resource inventory.
|
||||||
|
pub(crate) const V003_RESOURCES: &[V003SchemaResource] = &[
|
||||||
|
table_resource(
|
||||||
|
"tables/001_ksp_raw_transaction_variants.sql",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
VARIANT_COLUMNS,
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/tables/001_ksp_raw_transaction_variants.sql"),
|
||||||
|
),
|
||||||
|
table_resource(
|
||||||
|
"tables/002_ksp_raw_transaction_canonical_selectors.sql",
|
||||||
|
"ksp_raw_transaction_canonical_selectors",
|
||||||
|
SELECTOR_COLUMNS,
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/tables/002_ksp_raw_transaction_canonical_selectors.sql"),
|
||||||
|
),
|
||||||
|
table_resource(
|
||||||
|
"tables/003_ksp_raw_transaction_observation_variants.sql",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
OBSERVATION_VARIANT_COLUMNS,
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/tables/003_ksp_raw_transaction_observation_variants.sql"),
|
||||||
|
),
|
||||||
|
table_resource(
|
||||||
|
"tables/004_ksp_raw_transaction_conflicts.sql",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
CONFLICT_COLUMNS,
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/tables/004_ksp_raw_transaction_conflicts.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/001_pk_ksp_raw_transaction_variants.sql",
|
||||||
|
"p",
|
||||||
|
"pk_ksp_raw_transaction_variants",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/001_pk_ksp_raw_transaction_variants.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/002_ck_ksp_raw_transaction_variants_identity.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_variants_identity",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/002_ck_ksp_raw_transaction_variants_identity.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/003_ck_ksp_raw_transaction_variants_origin.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_variants_origin",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/003_ck_ksp_raw_transaction_variants_origin.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/004_ck_ksp_raw_transaction_variants_slot_time.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_variants_slot_time",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/004_ck_ksp_raw_transaction_variants_slot_time.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/005_ck_ksp_raw_transaction_variants_format.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_variants_format",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/005_ck_ksp_raw_transaction_variants_format.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/006_ck_ksp_raw_transaction_variants_payload.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_variants_payload",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/006_ck_ksp_raw_transaction_variants_payload.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/007_fk_ksp_raw_transaction_variants_transaction.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_variants_transaction",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/007_fk_ksp_raw_transaction_variants_transaction.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/008_pk_ksp_raw_transaction_canonical_selectors.sql",
|
||||||
|
"p",
|
||||||
|
"pk_ksp_raw_transaction_canonical_selectors",
|
||||||
|
"ksp_raw_transaction_canonical_selectors",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/008_pk_ksp_raw_transaction_canonical_selectors.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/009_ck_ksp_raw_transaction_canonical_selectors_bounds.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_canonical_selectors_bounds",
|
||||||
|
"ksp_raw_transaction_canonical_selectors",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/009_ck_ksp_raw_transaction_canonical_selectors_bounds.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/010_fk_ksp_raw_transaction_canonical_selectors_variant.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_canonical_selectors_variant",
|
||||||
|
"ksp_raw_transaction_canonical_selectors",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/010_fk_ksp_raw_transaction_canonical_selectors_variant.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/011_pk_ksp_raw_transaction_observation_variants.sql",
|
||||||
|
"p",
|
||||||
|
"pk_ksp_raw_transaction_observation_variants",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/011_pk_ksp_raw_transaction_observation_variants.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/012_ck_ksp_raw_transaction_observation_variants_bounds.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_observation_variants_bounds",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/012_ck_ksp_raw_transaction_observation_variants_bounds.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/013_fk_ksp_raw_transaction_observation_variants_observation.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_observation_variants_observation",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/013_fk_ksp_raw_transaction_observation_variants_observation.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/014_fk_ksp_raw_transaction_observation_variants_variant.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_observation_variants_variant",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/014_fk_ksp_raw_transaction_observation_variants_variant.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/015_pk_ksp_raw_transaction_conflicts.sql",
|
||||||
|
"p",
|
||||||
|
"pk_ksp_raw_transaction_conflicts",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/015_pk_ksp_raw_transaction_conflicts.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/016_ck_ksp_raw_transaction_conflicts_bounds.sql",
|
||||||
|
"c",
|
||||||
|
"ck_ksp_raw_transaction_conflicts_bounds",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/016_ck_ksp_raw_transaction_conflicts_bounds.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/017_fk_ksp_raw_transaction_conflicts_canonical_variant.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_conflicts_canonical_variant",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/017_fk_ksp_raw_transaction_conflicts_canonical_variant.sql"),
|
||||||
|
),
|
||||||
|
constraint_resource(
|
||||||
|
"constraints/018_fk_ksp_raw_transaction_conflicts_incoming_variant.sql",
|
||||||
|
"f",
|
||||||
|
"fk_ksp_raw_transaction_conflicts_incoming_variant",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/constraints/018_fk_ksp_raw_transaction_conflicts_incoming_variant.sql"),
|
||||||
|
),
|
||||||
|
index_resource(
|
||||||
|
"indexes/001_ix_ksp_raw_transaction_variants_signature_hash_variant.sql",
|
||||||
|
"(transaction_signature, content_hash, variant_id)",
|
||||||
|
"ix_ksp_raw_transaction_variants_signature_hash_variant",
|
||||||
|
"ksp_raw_transaction_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/indexes/001_ix_ksp_raw_transaction_variants_signature_hash_variant.sql"),
|
||||||
|
),
|
||||||
|
index_resource(
|
||||||
|
"indexes/002_ix_ksp_raw_transaction_observation_variants_variant.sql",
|
||||||
|
"(transaction_signature, variant_id, observation_key)",
|
||||||
|
"ix_ksp_raw_transaction_observation_variants_variant",
|
||||||
|
"ksp_raw_transaction_observation_variants",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/indexes/002_ix_ksp_raw_transaction_observation_variants_variant.sql"),
|
||||||
|
),
|
||||||
|
index_resource(
|
||||||
|
"indexes/003_ix_ksp_raw_transaction_conflicts_status_updated_signature.sql",
|
||||||
|
"(status, updated_at_unix_millis, transaction_signature)",
|
||||||
|
"ix_ksp_raw_transaction_conflicts_status_updated_signature",
|
||||||
|
"ksp_raw_transaction_conflicts",
|
||||||
|
include_str!("../migrations/v003_raw_transaction_variants/indexes/003_ix_ksp_raw_transaction_conflicts_status_updated_signature.sql"),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
|
||||||
|
const fn column(
|
||||||
|
name: &'static str,
|
||||||
|
nullable: bool,
|
||||||
|
numeric_precision: std::option::Option<i32>,
|
||||||
|
numeric_scale: std::option::Option<i32>,
|
||||||
|
udt_name: &'static str,
|
||||||
|
) -> ColumnContract {
|
||||||
|
return ColumnContract { name, nullable, numeric_precision, numeric_scale, udt_name };
|
||||||
|
}
|
||||||
|
|
||||||
|
const fn table_resource(id: &'static str, name: &'static str, columns: &'static [ColumnContract], sql: &'static str) -> V003SchemaResource {
|
||||||
|
return V003SchemaResource {
|
||||||
|
id,
|
||||||
|
object: V003SchemaObjectContract::Table(TableContract { columns, name }),
|
||||||
|
repair_existing: true,
|
||||||
|
sql,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const fn constraint_resource(id: &'static str, kind: &'static str, name: &'static str, table: &'static str, sql: &'static str) -> V003SchemaResource {
|
||||||
|
return V003SchemaResource {
|
||||||
|
id,
|
||||||
|
object: V003SchemaObjectContract::Constraint(ConstraintContract { kind, name, table }),
|
||||||
|
repair_existing: true,
|
||||||
|
sql,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const fn index_resource(id: &'static str, key_fragment: &'static str, name: &'static str, table: &'static str, sql: &'static str) -> V003SchemaResource {
|
||||||
|
return V003SchemaResource {
|
||||||
|
id,
|
||||||
|
object: V003SchemaObjectContract::Index(IndexContract {
|
||||||
|
access_method: "btree",
|
||||||
|
key_fragment,
|
||||||
|
name,
|
||||||
|
predicate_fragment: std::option::Option::None,
|
||||||
|
table,
|
||||||
|
unique: false,
|
||||||
|
}),
|
||||||
|
repair_existing: true,
|
||||||
|
sql,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Inspects one V003 resource against the active PostgreSQL schema.
|
||||||
|
pub(crate) async fn inspect_v003_resource(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
resource: &V003SchemaResource,
|
||||||
|
) -> std::result::Result<V003SchemaResourceState, crate::PostgresBackendError> {
|
||||||
|
return match resource.object {
|
||||||
|
V003SchemaObjectContract::Constraint(contract) => inspect_constraint(transaction, contract, resource.sql).await,
|
||||||
|
V003SchemaObjectContract::Index(contract) => inspect_index(transaction, contract).await,
|
||||||
|
V003SchemaObjectContract::Table(contract) => inspect_table(transaction, contract).await,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verifies that every owned V003 object is compatible after migration/repair.
|
||||||
|
pub(crate) async fn verify_v003_external_compatibility(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||||
|
for resource in V003_RESOURCES {
|
||||||
|
let state_result = inspect_v003_resource(transaction, resource).await;
|
||||||
|
let state = match state_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
if state != V003SchemaResourceState::Compatible {
|
||||||
|
return schema_incompatible("schema_v003_external_contract");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn inspect_constraint(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
contract: ConstraintContract,
|
||||||
|
resource_sql: &str,
|
||||||
|
) -> std::result::Result<V003SchemaResourceState, crate::PostgresBackendError> {
|
||||||
|
let expected = match expected_constraint_definition(resource_sql, contract.name) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return schema_query_error("schema_v003_constraint_contract"),
|
||||||
|
};
|
||||||
|
let result = transaction.query_opt(CONSTRAINT_LOAD_SQL, &[&contract.table, &contract.name]).await;
|
||||||
|
let row = match result {
|
||||||
|
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||||
|
std::result::Result::Ok(std::option::Option::None) => return std::result::Result::Ok(V003SchemaResourceState::Missing),
|
||||||
|
std::result::Result::Err(_) => return schema_query_error("schema_v003_constraint_probe"),
|
||||||
|
};
|
||||||
|
let kind = row.try_get::<usize, std::string::String>(0);
|
||||||
|
let validated = row.try_get::<usize, bool>(1);
|
||||||
|
let deferrable = row.try_get::<usize, bool>(2);
|
||||||
|
let deferred = row.try_get::<usize, bool>(3);
|
||||||
|
let definition = row.try_get::<usize, std::string::String>(4);
|
||||||
|
let (kind, validated, deferrable, deferred, definition) = match (kind, validated, deferrable, deferred, definition) {
|
||||||
|
(
|
||||||
|
std::result::Result::Ok(kind),
|
||||||
|
std::result::Result::Ok(validated),
|
||||||
|
std::result::Result::Ok(deferrable),
|
||||||
|
std::result::Result::Ok(deferred),
|
||||||
|
std::result::Result::Ok(definition),
|
||||||
|
) => (kind, validated, deferrable, deferred, definition),
|
||||||
|
_ => return schema_query_error("schema_v003_constraint_probe_decode"),
|
||||||
|
};
|
||||||
|
if kind != contract.kind || !validated || deferrable || deferred || normalize_catalog_sql(definition.as_str()) != expected {
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Incompatible);
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Compatible);
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn inspect_index(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
contract: IndexContract,
|
||||||
|
) -> std::result::Result<V003SchemaResourceState, crate::PostgresBackendError> {
|
||||||
|
let result = transaction.query_opt(INDEX_LOAD_SQL, &[&contract.table, &contract.name]).await;
|
||||||
|
let row = match result {
|
||||||
|
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||||
|
std::result::Result::Ok(std::option::Option::None) => return std::result::Result::Ok(V003SchemaResourceState::Missing),
|
||||||
|
std::result::Result::Err(_) => return schema_query_error("schema_v003_index_probe"),
|
||||||
|
};
|
||||||
|
let unique = row.try_get::<usize, bool>(0);
|
||||||
|
let access_method = row.try_get::<usize, std::string::String>(1);
|
||||||
|
let definition = row.try_get::<usize, std::string::String>(2);
|
||||||
|
let predicate = row.try_get::<usize, std::option::Option<std::string::String>>(3);
|
||||||
|
let (unique, access_method, definition, predicate) = match (unique, access_method, definition, predicate) {
|
||||||
|
(std::result::Result::Ok(unique), std::result::Result::Ok(access_method), std::result::Result::Ok(definition), std::result::Result::Ok(predicate)) => {
|
||||||
|
(unique, access_method, definition, predicate)
|
||||||
|
},
|
||||||
|
_ => return schema_query_error("schema_v003_index_probe_decode"),
|
||||||
|
};
|
||||||
|
let definition = normalize_catalog_sql(definition.as_str());
|
||||||
|
let predicate = predicate.map(|value| return normalize_catalog_sql(value.as_str()));
|
||||||
|
let predicate_matches = match (contract.predicate_fragment, predicate.as_deref()) {
|
||||||
|
(std::option::Option::Some(expected), std::option::Option::Some(value)) => value.contains(expected),
|
||||||
|
(std::option::Option::None, std::option::Option::None) => true,
|
||||||
|
_ => false,
|
||||||
|
};
|
||||||
|
if unique != contract.unique
|
||||||
|
|| access_method != contract.access_method
|
||||||
|
|| !definition.contains(&normalize_catalog_sql(contract.key_fragment))
|
||||||
|
|| !predicate_matches
|
||||||
|
{
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Incompatible);
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Compatible);
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn inspect_table(
|
||||||
|
transaction: &deadpool_postgres::Transaction<'_>,
|
||||||
|
contract: TableContract,
|
||||||
|
) -> std::result::Result<V003SchemaResourceState, crate::PostgresBackendError> {
|
||||||
|
let table_result = transaction.query_opt(TABLE_LOAD_SQL, &[&contract.name]).await;
|
||||||
|
let table_row = match table_result {
|
||||||
|
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||||
|
std::result::Result::Ok(std::option::Option::None) => return std::result::Result::Ok(V003SchemaResourceState::Missing),
|
||||||
|
std::result::Result::Err(_) => return schema_query_error("schema_v003_table_probe"),
|
||||||
|
};
|
||||||
|
let relkind = table_row.try_get::<usize, std::string::String>(0);
|
||||||
|
let row_security = table_row.try_get::<usize, bool>(1);
|
||||||
|
let force_row_security = table_row.try_get::<usize, bool>(2);
|
||||||
|
let (relkind, row_security, force_row_security) = match (relkind, row_security, force_row_security) {
|
||||||
|
(std::result::Result::Ok(relkind), std::result::Result::Ok(row_security), std::result::Result::Ok(force_row_security)) => {
|
||||||
|
(relkind, row_security, force_row_security)
|
||||||
|
},
|
||||||
|
_ => return schema_query_error("schema_v003_table_probe_decode"),
|
||||||
|
};
|
||||||
|
if relkind != "r" || row_security || force_row_security {
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Incompatible);
|
||||||
|
}
|
||||||
|
let rows_result = transaction.query(COLUMN_LOAD_SQL, &[&contract.name]).await;
|
||||||
|
let rows = match rows_result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return schema_query_error("schema_v003_column_probe"),
|
||||||
|
};
|
||||||
|
let mut actual = std::vec::Vec::<ActualColumn>::with_capacity(rows.len());
|
||||||
|
for row in rows {
|
||||||
|
let decoded = ActualColumn::from_row(&row);
|
||||||
|
match decoded {
|
||||||
|
std::result::Result::Ok(value) => actual.push(value),
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let mut missing = false;
|
||||||
|
for expected in contract.columns {
|
||||||
|
let found = actual.iter().find(|candidate| return candidate.name == expected.name);
|
||||||
|
match found {
|
||||||
|
std::option::Option::Some(value) if value.matches(expected) => {},
|
||||||
|
std::option::Option::Some(_) => return std::result::Result::Ok(V003SchemaResourceState::Incompatible),
|
||||||
|
std::option::Option::None => missing = true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for column in &actual {
|
||||||
|
if contract.columns.iter().any(|expected| return expected.name == column.name) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if !column.is_non_blocking_extra() {
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Incompatible);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if missing {
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Missing);
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(V003SchemaResourceState::Compatible);
|
||||||
|
}
|
||||||
|
|
||||||
|
struct ActualColumn {
|
||||||
|
default: std::option::Option<std::string::String>,
|
||||||
|
generated: std::string::String,
|
||||||
|
identity: std::string::String,
|
||||||
|
name: std::string::String,
|
||||||
|
nullable: bool,
|
||||||
|
numeric_precision: std::option::Option<i32>,
|
||||||
|
numeric_scale: std::option::Option<i32>,
|
||||||
|
udt_name: std::string::String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ActualColumn {
|
||||||
|
fn from_row(row: &tokio_postgres::Row) -> std::result::Result<Self, crate::PostgresBackendError> {
|
||||||
|
let name = row.try_get::<usize, std::string::String>(0);
|
||||||
|
let udt_name = row.try_get::<usize, std::string::String>(1);
|
||||||
|
let nullable = row.try_get::<usize, bool>(2);
|
||||||
|
let numeric_precision = row.try_get::<usize, std::option::Option<i32>>(3);
|
||||||
|
let numeric_scale = row.try_get::<usize, std::option::Option<i32>>(4);
|
||||||
|
let default = row.try_get::<usize, std::option::Option<std::string::String>>(5);
|
||||||
|
let identity = row.try_get::<usize, std::string::String>(6);
|
||||||
|
let generated = row.try_get::<usize, std::string::String>(7);
|
||||||
|
return match (name, udt_name, nullable, numeric_precision, numeric_scale, default, identity, generated) {
|
||||||
|
(
|
||||||
|
std::result::Result::Ok(name),
|
||||||
|
std::result::Result::Ok(udt_name),
|
||||||
|
std::result::Result::Ok(nullable),
|
||||||
|
std::result::Result::Ok(numeric_precision),
|
||||||
|
std::result::Result::Ok(numeric_scale),
|
||||||
|
std::result::Result::Ok(default),
|
||||||
|
std::result::Result::Ok(identity),
|
||||||
|
std::result::Result::Ok(generated),
|
||||||
|
) => std::result::Result::Ok(Self { default, generated, identity, name, nullable, numeric_precision, numeric_scale, udt_name }),
|
||||||
|
_ => schema_query_error("schema_v003_column_probe_decode"),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_non_blocking_extra(&self) -> bool {
|
||||||
|
return self.nullable && self.default.is_none() && self.identity == "NO" && self.generated == "NEVER";
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches(&self, expected: &ColumnContract) -> bool {
|
||||||
|
let numeric_matches = if expected.udt_name == "numeric" {
|
||||||
|
self.numeric_precision == expected.numeric_precision && self.numeric_scale == expected.numeric_scale
|
||||||
|
} else {
|
||||||
|
true
|
||||||
|
};
|
||||||
|
return self.udt_name == expected.udt_name
|
||||||
|
&& self.nullable == expected.nullable
|
||||||
|
&& numeric_matches
|
||||||
|
&& self.identity == "NO"
|
||||||
|
&& self.generated == "NEVER";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn expected_constraint_definition(resource_sql: &str, name: &str) -> std::option::Option<std::string::String> {
|
||||||
|
let marker = std::format!("ADD CONSTRAINT {name} ");
|
||||||
|
let marker_start = match resource_sql.find(marker.as_str()) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let start = match marker_start.checked_add(marker.len()) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let tail = match resource_sql.get(start..) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let end = match tail.find(';') {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
let definition = match tail.get(..end) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::option::Option::None,
|
||||||
|
};
|
||||||
|
return std::option::Option::Some(normalize_catalog_sql(definition));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn normalize_catalog_sql(value: &str) -> std::string::String {
|
||||||
|
let value = normalize_quoted_integral_cast_literals(value);
|
||||||
|
return value
|
||||||
|
.chars()
|
||||||
|
.filter(|character| return !character.is_whitespace() && *character != '"' && *character != '(' && *character != ')')
|
||||||
|
.collect::<std::string::String>()
|
||||||
|
.replace("::text", "")
|
||||||
|
.replace("::numeric", "")
|
||||||
|
.replace("::bigint", "")
|
||||||
|
.replace("::smallint", "")
|
||||||
|
.replace("::integer", "")
|
||||||
|
.replace("::int8", "")
|
||||||
|
.replace("::int4", "")
|
||||||
|
.replace("::int2", "")
|
||||||
|
.to_ascii_lowercase();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn normalize_quoted_integral_cast_literals(value: &str) -> std::string::String {
|
||||||
|
const INTEGRAL_CASTS: &[&str] = &["::bigint", "::int2", "::int4", "::int8", "::integer", "::numeric", "::smallint"];
|
||||||
|
let mut normalized = std::string::String::with_capacity(value.len());
|
||||||
|
let mut remaining = value;
|
||||||
|
loop {
|
||||||
|
let (before_quote, after_quote) = match remaining.split_once('\'') {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => {
|
||||||
|
normalized.push_str(remaining);
|
||||||
|
break;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
normalized.push_str(before_quote);
|
||||||
|
let (literal, after_literal) = match after_quote.split_once('\'') {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => {
|
||||||
|
normalized.push('\'');
|
||||||
|
normalized.push_str(after_quote);
|
||||||
|
break;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let has_digit = literal.chars().any(|character| return character.is_ascii_digit());
|
||||||
|
let integral_literal = !literal.is_empty()
|
||||||
|
&& has_digit
|
||||||
|
&& literal
|
||||||
|
.chars()
|
||||||
|
.enumerate()
|
||||||
|
.all(|(offset, character)| return character.is_ascii_digit() || (offset == 0 && (character == '+' || character == '-')));
|
||||||
|
let mut after_integral_cast = std::option::Option::None;
|
||||||
|
if integral_literal {
|
||||||
|
for cast in INTEGRAL_CASTS {
|
||||||
|
if let std::option::Option::Some(value) = after_literal.strip_prefix(*cast) {
|
||||||
|
after_integral_cast = std::option::Option::Some(value);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let std::option::Option::Some(value) = after_integral_cast {
|
||||||
|
normalized.push_str(literal);
|
||||||
|
remaining = value;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
normalized.push('\'');
|
||||||
|
normalized.push_str(literal);
|
||||||
|
normalized.push('\'');
|
||||||
|
remaining = after_literal;
|
||||||
|
}
|
||||||
|
return normalized;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_incompatible<T>(phase: &'static str) -> std::result::Result<T, crate::PostgresBackendError> {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, phase));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_query_error<T>(phase: &'static str) -> std::result::Result<T, crate::PostgresBackendError> {
|
||||||
|
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, phase));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "../unit_tests/schema_v003.rs"]
|
||||||
|
mod tests;
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
// file: crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||||
// version: 27
|
// version: 30
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -284,6 +284,9 @@ fn pre_009_live_raw_transaction_proof_is_opt_in_isolated_and_secret_safe() {
|
|||||||
"prove_schema_update_policy",
|
"prove_schema_update_policy",
|
||||||
"prove_concurrent_identical_insert",
|
"prove_concurrent_identical_insert",
|
||||||
"prove_concurrent_divergent_insert",
|
"prove_concurrent_divergent_insert",
|
||||||
|
"prove_conflict_case_rollback",
|
||||||
|
"QuarantinedConflict",
|
||||||
|
"conflict_state_is_exact",
|
||||||
"prove_pagination",
|
"prove_pagination",
|
||||||
"prove_inspection",
|
"prove_inspection",
|
||||||
"prove_retention_and_rehydrate",
|
"prove_retention_and_rehydrate",
|
||||||
@@ -594,7 +597,7 @@ fn v0_3_15_pre_012_fix_004_content_conflict_diagnostic_is_field_only_bounded_and
|
|||||||
let Some(start) = start else {
|
let Some(start) = start else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let end = source[start..].find("fn raw_transactions_equal(");
|
let end = source[start..].find("async fn rehydrate_transaction(");
|
||||||
assert!(end.is_some(), "missing RAW conflict logger end marker");
|
assert!(end.is_some(), "missing RAW conflict logger end marker");
|
||||||
let Some(end) = end else {
|
let Some(end) = end else {
|
||||||
return;
|
return;
|
||||||
@@ -645,33 +648,43 @@ fn v0_3_15_pre_014_content_conflict_diagnostic_exposes_only_block_slot_and_bound
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn v0_3_15_pre_014_fix_001_truncated_log_compatibility_is_narrow_and_keeps_canonical_content() {
|
fn v0_3_16_pre_007_postgres_uses_shared_fail_closed_variant_comparator_with_durable_conflict() {
|
||||||
let source = include_str!("../src/raw_transaction.rs");
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
for required in [
|
for required in [
|
||||||
"ActiveIncomingTruncatedLogs",
|
"ksp_store_api::compare_raw_transaction_variants",
|
||||||
|
"RawTransactionVariantRelation::Exact",
|
||||||
|
"RawTransactionVariantRelation::CompatibleLessComplete",
|
||||||
|
"RawTransactionVariantRelation::CompatibleMoreComplete",
|
||||||
|
"RawTransactionVariantRelation::Conflict",
|
||||||
|
"RawTransactionVariantRelation::Incomparable",
|
||||||
|
"ExistingTransactionMatch::ActiveCompatibleLessComplete",
|
||||||
|
"ExistingTransactionMatch::ActiveConflict",
|
||||||
|
"RawTransactionVariantWriteOutcome::QuarantinedConflict",
|
||||||
|
] {
|
||||||
|
assert!(source.contains(required), "missing shared comparator integration guard: {required}");
|
||||||
|
}
|
||||||
|
for forbidden in [
|
||||||
"raw_transaction_incoming_truncated_log_messages_compatible",
|
"raw_transaction_incoming_truncated_log_messages_compatible",
|
||||||
"raw_meta_other_than_log_messages_mismatch",
|
"raw_meta_other_than_log_messages_mismatch",
|
||||||
"raw_log_messages_incoming_truncated_compatible",
|
"raw_log_messages_incoming_truncated_compatible",
|
||||||
"raw_log_message_is_exact_truncation_marker",
|
"raw_log_message_is_exact_truncation_marker",
|
||||||
"marker_index >= stored.len()",
|
|
||||||
"incoming[..marker_index] != stored[..marker_index]",
|
|
||||||
"PostgreSQL Store accepted compatible truncated RAW transaction logs without replacing canonical content",
|
|
||||||
] {
|
] {
|
||||||
assert!(source.contains(required), "missing narrow truncated-log compatibility guard: {required}");
|
assert!(!source.contains(forbidden), "backend-private convergence heuristic survived pre.005: {forbidden}");
|
||||||
}
|
}
|
||||||
let start = source.find("fn compare_existing_transaction(");
|
let start = source.find("fn compare_existing_transaction(");
|
||||||
assert!(start.is_some(), "missing fix.001 existing-transaction comparison path");
|
assert!(start.is_some(), "missing pre.005 existing-transaction comparison path");
|
||||||
let Some(start) = start else {
|
let Some(start) = start else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let end = source[start..].find("fn raw_transaction_incoming_truncated_log_messages_compatible(");
|
let end = source[start..].find("fn raw_transaction_content_conflict_diagnostic(");
|
||||||
assert!(end.is_some(), "missing fix.001 compatibility helper boundary");
|
assert!(end.is_some(), "missing pre.005 comparison boundary");
|
||||||
let Some(end) = end else {
|
let Some(end) = end else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let comparison = &source[start..start + end];
|
let comparison = &source[start..start + end];
|
||||||
assert!(comparison.contains("ExistingTransactionMatch::ActiveIncomingTruncatedLogs"));
|
assert!(comparison.contains("CompatibleLessComplete"));
|
||||||
|
assert!(comparison.contains("CompatibleMoreComplete"));
|
||||||
assert!(!comparison.contains("UPDATE ksp_raw_transactions SET payload"));
|
assert!(!comparison.contains("UPDATE ksp_raw_transactions SET payload"));
|
||||||
assert!(!comparison.contains("DELETE FROM ksp_raw_transactions"));
|
assert!(!comparison.contains("canonical_revision"));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/tests/postgres_foundation_live.rs
|
// file: crates/ksp-store-postgres-lib/tests/postgres_foundation_live.rs
|
||||||
// version: 4
|
// version: 5
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -15,7 +15,11 @@
|
|||||||
const LIVE_BOOTSTRAP_SQL: &str = include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql");
|
const LIVE_BOOTSTRAP_SQL: &str = include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql");
|
||||||
const LIVE_BROKEN_CHECKSUM_A: &str = "0000000000000000000000000000000000000000000000000000000000000000";
|
const LIVE_BROKEN_CHECKSUM_A: &str = "0000000000000000000000000000000000000000000000000000000000000000";
|
||||||
const LIVE_BROKEN_CHECKSUM_B: &str = "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff";
|
const LIVE_BROKEN_CHECKSUM_B: &str = "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff";
|
||||||
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_transaction_conflicts;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_observation_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_canonical_selectors;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transactions;
|
DROP TABLE IF EXISTS ksp_raw_transactions;
|
||||||
DROP TABLE IF EXISTS ksp_store_identity;
|
DROP TABLE IF EXISTS ksp_store_identity;
|
||||||
@@ -28,7 +32,11 @@ const LIVE_MANAGED_SCHEMA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
|||||||
'ksp_store_identity',
|
'ksp_store_identity',
|
||||||
'ksp_raw_transactions',
|
'ksp_raw_transactions',
|
||||||
'ksp_raw_transaction_observations',
|
'ksp_raw_transaction_observations',
|
||||||
'ksp_raw_transaction_archive_payloads'
|
'ksp_raw_transaction_archive_payloads',
|
||||||
|
'ksp_raw_transaction_variants',
|
||||||
|
'ksp_raw_transaction_canonical_selectors',
|
||||||
|
'ksp_raw_transaction_observation_variants',
|
||||||
|
'ksp_raw_transaction_conflicts'
|
||||||
)
|
)
|
||||||
AND table_type = 'BASE TABLE'
|
AND table_type = 'BASE TABLE'
|
||||||
)"#;
|
)"#;
|
||||||
@@ -153,7 +161,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
|||||||
}
|
}
|
||||||
*owns_schema = true;
|
*owns_schema = true;
|
||||||
let initial_health = initial.health().await;
|
let initial_health = initial.health().await;
|
||||||
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(2) || initial_health.pending_migration_count() != 0 {
|
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(3) || initial_health.pending_migration_count() != 0 {
|
||||||
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
||||||
}
|
}
|
||||||
let initial_close = close_backend(initial).await;
|
let initial_close = close_backend(initial).await;
|
||||||
@@ -167,7 +175,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
|||||||
};
|
};
|
||||||
let idempotent_health = idempotent.health().await;
|
let idempotent_health = idempotent.health().await;
|
||||||
if !idempotent_health.is_ready()
|
if !idempotent_health.is_ready()
|
||||||
|| idempotent_health.migration_version() != std::option::Option::Some(2)
|
|| idempotent_health.migration_version() != std::option::Option::Some(3)
|
||||||
|| idempotent_health.pending_migration_count() != 0
|
|| idempotent_health.pending_migration_count() != 0
|
||||||
{
|
{
|
||||||
return std::result::Result::Err(LiveFailure::new("idempotent_health"));
|
return std::result::Result::Err(LiveFailure::new("idempotent_health"));
|
||||||
@@ -250,7 +258,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
|||||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
};
|
};
|
||||||
let final_health = final_backend.health().await;
|
let final_health = final_backend.health().await;
|
||||||
if !final_health.is_ready() || final_health.migration_version() != std::option::Option::Some(2) || final_health.pending_migration_count() != 0 {
|
if !final_health.is_ready() || final_health.migration_version() != std::option::Option::Some(3) || final_health.pending_migration_count() != 0 {
|
||||||
return std::result::Result::Err(LiveFailure::new("final_health"));
|
return std::result::Result::Err(LiveFailure::new("final_health"));
|
||||||
}
|
}
|
||||||
return close_backend(final_backend).await;
|
return close_backend(final_backend).await;
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/tests/postgres_raw_account_live.rs
|
// file: crates/ksp-store-postgres-lib/tests/postgres_raw_account_live.rs
|
||||||
// version: 2
|
// version: 3
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -20,7 +20,11 @@ const LIVE_ACCOUNT_STATE_EXISTS_SQL: &str =
|
|||||||
"SELECT EXISTS (SELECT 1 FROM ksp_raw_account_states WHERE pubkey = $1 AND slot = $2::TEXT::NUMERIC AND state_hash = $3)";
|
"SELECT EXISTS (SELECT 1 FROM ksp_raw_account_states WHERE pubkey = $1 AND slot = $2::TEXT::NUMERIC AND state_hash = $3)";
|
||||||
const LIVE_CANCEL_WAIT: std::time::Duration = std::time::Duration::from_millis(300);
|
const LIVE_CANCEL_WAIT: std::time::Duration = std::time::Duration::from_millis(300);
|
||||||
const LIVE_LOCK_ACCOUNT_OBSERVATION_SQL: &str = "SELECT observation_key FROM ksp_raw_account_observations WHERE observation_key = $1 FOR UPDATE";
|
const LIVE_LOCK_ACCOUNT_OBSERVATION_SQL: &str = "SELECT observation_key FROM ksp_raw_account_observations WHERE observation_key = $1 FOR UPDATE";
|
||||||
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_account_observations;
|
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_transaction_conflicts;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_observation_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_canonical_selectors;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_account_observations;
|
||||||
DROP TABLE IF EXISTS ksp_raw_account_states;
|
DROP TABLE IF EXISTS ksp_raw_account_states;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
||||||
@@ -36,6 +40,10 @@ const LIVE_MANAGED_SCHEMA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
|||||||
'ksp_raw_transactions',
|
'ksp_raw_transactions',
|
||||||
'ksp_raw_transaction_observations',
|
'ksp_raw_transaction_observations',
|
||||||
'ksp_raw_transaction_archive_payloads',
|
'ksp_raw_transaction_archive_payloads',
|
||||||
|
'ksp_raw_transaction_variants',
|
||||||
|
'ksp_raw_transaction_canonical_selectors',
|
||||||
|
'ksp_raw_transaction_observation_variants',
|
||||||
|
'ksp_raw_transaction_conflicts',
|
||||||
'ksp_raw_account_states',
|
'ksp_raw_account_states',
|
||||||
'ksp_raw_account_observations'
|
'ksp_raw_account_observations'
|
||||||
)
|
)
|
||||||
@@ -152,7 +160,7 @@ async fn run_raw_account_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
|||||||
};
|
};
|
||||||
*owns_schema = true;
|
*owns_schema = true;
|
||||||
let initial_health = initial.health().await;
|
let initial_health = initial.health().await;
|
||||||
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(2) || initial_health.pending_migration_count() != 0 {
|
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(3) || initial_health.pending_migration_count() != 0 {
|
||||||
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
||||||
}
|
}
|
||||||
let wrong_network_result = open_backend_result(uri, "testnet", true, true).await;
|
let wrong_network_result = open_backend_result(uri, "testnet", true, true).await;
|
||||||
@@ -211,7 +219,7 @@ async fn run_raw_account_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
|||||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
};
|
};
|
||||||
let reopened_health = reopened.health().await;
|
let reopened_health = reopened.health().await;
|
||||||
if !reopened_health.is_ready() || reopened_health.migration_version() != std::option::Option::Some(2) {
|
if !reopened_health.is_ready() || reopened_health.migration_version() != std::option::Option::Some(3) {
|
||||||
return std::result::Result::Err(LiveFailure::new("reopen_health"));
|
return std::result::Result::Err(LiveFailure::new("reopen_health"));
|
||||||
}
|
}
|
||||||
let reference = match account_reference(10, 100, 10) {
|
let reference = match account_reference(10, 100, 10) {
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
// file: crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||||
// version: 7
|
// version: 10
|
||||||
|
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
#![deny(unreachable_pub)]
|
#![deny(unreachable_pub)]
|
||||||
@@ -12,13 +12,79 @@
|
|||||||
//! drops only the isolated schema it proved absent before the run.
|
//! drops only the isolated schema it proved absent before the run.
|
||||||
|
|
||||||
const LIVE_CANCEL_WAIT: std::time::Duration = std::time::Duration::from_millis(300);
|
const LIVE_CANCEL_WAIT: std::time::Duration = std::time::Duration::from_millis(300);
|
||||||
|
const LIVE_CONFLICT_STATE_SQL: &str = r#"SELECT
|
||||||
|
(SELECT COUNT(*) = 2 FROM ksp_raw_transaction_variants WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_canonical_selectors WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 2 FROM ksp_raw_transaction_observations WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 2 FROM ksp_raw_transaction_observation_variants WHERE transaction_signature = $1)
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM ksp_raw_transaction_conflicts AS conflict
|
||||||
|
INNER JOIN ksp_raw_transaction_canonical_selectors AS selector
|
||||||
|
ON selector.transaction_signature = conflict.transaction_signature
|
||||||
|
WHERE conflict.transaction_signature = $1
|
||||||
|
AND conflict.status = 'open'
|
||||||
|
AND conflict.revision = 1
|
||||||
|
AND conflict.canonical_variant_id = selector.canonical_variant_id
|
||||||
|
AND conflict.incoming_variant_id <> conflict.canonical_variant_id
|
||||||
|
)
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM ksp_raw_transactions AS canonical
|
||||||
|
INNER JOIN ksp_raw_transaction_canonical_selectors AS selector
|
||||||
|
ON selector.transaction_signature = canonical.signature
|
||||||
|
INNER JOIN ksp_raw_transaction_variants AS variant
|
||||||
|
ON variant.transaction_signature = selector.transaction_signature
|
||||||
|
AND variant.variant_id = selector.canonical_variant_id
|
||||||
|
WHERE canonical.signature = $1
|
||||||
|
AND variant.slot = canonical.slot
|
||||||
|
AND variant.block_time_unix_millis IS NOT DISTINCT FROM canonical.block_time_unix_millis
|
||||||
|
AND variant.format_id = canonical.format_id
|
||||||
|
AND variant.format_version = canonical.format_version
|
||||||
|
AND variant.content_hash = canonical.content_hash
|
||||||
|
AND variant.payload = canonical.payload
|
||||||
|
)
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1 FROM ksp_raw_transaction_observation_variants
|
||||||
|
WHERE observation_key = $2 AND transaction_signature = $1
|
||||||
|
)
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1 FROM ksp_raw_transaction_observation_variants
|
||||||
|
WHERE observation_key = $3 AND transaction_signature = $1
|
||||||
|
)"#;
|
||||||
|
const LIVE_CONFLICT_ROLLBACK_STATE_SQL: &str = r#"SELECT
|
||||||
|
(SELECT COUNT(*) = 1 FROM ksp_raw_transaction_variants WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_canonical_selectors WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 0 FROM ksp_raw_transaction_conflicts WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_observations WHERE transaction_signature = $1)
|
||||||
|
AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_observation_variants WHERE transaction_signature = $1)
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM ksp_raw_transactions AS canonical
|
||||||
|
INNER JOIN ksp_raw_transaction_canonical_selectors AS selector
|
||||||
|
ON selector.transaction_signature = canonical.signature
|
||||||
|
INNER JOIN ksp_raw_transaction_variants AS variant
|
||||||
|
ON variant.transaction_signature = selector.transaction_signature
|
||||||
|
AND variant.variant_id = selector.canonical_variant_id
|
||||||
|
WHERE canonical.signature = $1
|
||||||
|
AND variant.slot = canonical.slot
|
||||||
|
AND variant.block_time_unix_millis IS NOT DISTINCT FROM canonical.block_time_unix_millis
|
||||||
|
AND variant.format_id = canonical.format_id
|
||||||
|
AND variant.format_version = canonical.format_version
|
||||||
|
AND variant.content_hash = canonical.content_hash
|
||||||
|
AND variant.payload = canonical.payload
|
||||||
|
)"#;
|
||||||
const LIVE_INDEX_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
const LIVE_INDEX_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||||
SELECT 1 FROM pg_indexes
|
SELECT 1 FROM pg_indexes
|
||||||
WHERE schemaname = current_schema()
|
WHERE schemaname = current_schema()
|
||||||
AND indexname = 'ix_ksp_raw_transactions_slot_signature'
|
AND indexname = 'ix_ksp_raw_transactions_slot_signature'
|
||||||
)"#;
|
)"#;
|
||||||
const LIVE_LOCK_OBSERVATION_SQL: &str = "SELECT observation_key FROM ksp_raw_transaction_observations WHERE observation_key = $1 FOR UPDATE";
|
const LIVE_LOCK_OBSERVATION_SQL: &str = "SELECT observation_key FROM ksp_raw_transaction_observations WHERE observation_key = $1 FOR UPDATE";
|
||||||
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_account_observations;
|
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_transaction_conflicts;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_observation_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_canonical_selectors;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_transaction_variants;
|
||||||
|
DROP TABLE IF EXISTS ksp_raw_account_observations;
|
||||||
DROP TABLE IF EXISTS ksp_raw_account_states;
|
DROP TABLE IF EXISTS ksp_raw_account_states;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
||||||
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
||||||
@@ -34,14 +100,18 @@ const LIVE_MANAGED_SCHEMA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
|||||||
'ksp_raw_transactions',
|
'ksp_raw_transactions',
|
||||||
'ksp_raw_transaction_observations',
|
'ksp_raw_transaction_observations',
|
||||||
'ksp_raw_transaction_archive_payloads',
|
'ksp_raw_transaction_archive_payloads',
|
||||||
|
'ksp_raw_transaction_variants',
|
||||||
|
'ksp_raw_transaction_canonical_selectors',
|
||||||
|
'ksp_raw_transaction_observation_variants',
|
||||||
|
'ksp_raw_transaction_conflicts',
|
||||||
'ksp_raw_account_states',
|
'ksp_raw_account_states',
|
||||||
'ksp_raw_account_observations'
|
'ksp_raw_account_observations'
|
||||||
)
|
)
|
||||||
AND table_type = 'BASE TABLE'
|
AND table_type = 'BASE TABLE'
|
||||||
)"#;
|
)"#;
|
||||||
const LIVE_MAX_URI_BYTES: usize = 4_096;
|
const LIVE_MAX_URI_BYTES: usize = 4_096;
|
||||||
const LIVE_OBSERVATION_EXISTS_SQL: &str = "SELECT EXISTS (SELECT 1 FROM ksp_raw_transaction_observations WHERE observation_key = $1)";
|
|
||||||
const LIVE_TRANSACTION_EXISTS_SQL: &str = "SELECT EXISTS (SELECT 1 FROM ksp_raw_transactions WHERE signature = $1)";
|
const LIVE_TRANSACTION_EXISTS_SQL: &str = "SELECT EXISTS (SELECT 1 FROM ksp_raw_transactions WHERE signature = $1)";
|
||||||
|
const LIVE_VARIANT_IDEMPOTENCE_SQL: &str = "SELECT (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_variants WHERE transaction_signature = $1) AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_canonical_selectors WHERE transaction_signature = $1) AND (SELECT COUNT(*) = 1 FROM ksp_raw_transaction_observation_variants WHERE transaction_signature = $1) AND EXISTS (SELECT 1 FROM ksp_raw_transaction_observation_variants AS mapping INNER JOIN ksp_raw_transaction_canonical_selectors AS selector ON selector.transaction_signature = mapping.transaction_signature AND selector.canonical_variant_id = mapping.variant_id WHERE mapping.observation_key = $2 AND mapping.transaction_signature = $1)";
|
||||||
|
|
||||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||||
struct LiveFailure {
|
struct LiveFailure {
|
||||||
@@ -60,7 +130,7 @@ impl LiveFailure {
|
|||||||
|
|
||||||
#[derive(Debug)]
|
#[derive(Debug)]
|
||||||
enum LivePersistResult {
|
enum LivePersistResult {
|
||||||
BackendError(ksp_store_postgres_lib::PostgresBackendErrorKind),
|
BackendError,
|
||||||
Outcome(ksp_store_api::RawAcquisitionWriteOutcome),
|
Outcome(ksp_store_api::RawAcquisitionWriteOutcome),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -157,7 +227,7 @@ async fn run_raw_transaction_scenario(admin: &mut tokio_postgres::Client, uri: &
|
|||||||
};
|
};
|
||||||
*owns_schema = true;
|
*owns_schema = true;
|
||||||
let initial_health = initial.health().await;
|
let initial_health = initial.health().await;
|
||||||
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(2) || initial_health.pending_migration_count() != 0 {
|
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(3) || initial_health.pending_migration_count() != 0 {
|
||||||
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
||||||
}
|
}
|
||||||
let wrong_network_result = open_backend_result(uri, "testnet", true, true).await;
|
let wrong_network_result = open_backend_result(uri, "testnet", true, true).await;
|
||||||
@@ -197,7 +267,7 @@ async fn run_raw_transaction_scenario(admin: &mut tokio_postgres::Client, uri: &
|
|||||||
if let std::result::Result::Err(error) = backend_close {
|
if let std::result::Result::Err(error) = backend_close {
|
||||||
return std::result::Result::Err(error);
|
return std::result::Result::Err(error);
|
||||||
}
|
}
|
||||||
let identical_result = prove_concurrent_identical_insert(uri).await;
|
let identical_result = prove_concurrent_identical_insert(admin, uri).await;
|
||||||
if let std::result::Result::Err(error) = identical_result {
|
if let std::result::Result::Err(error) = identical_result {
|
||||||
return std::result::Result::Err(error);
|
return std::result::Result::Err(error);
|
||||||
}
|
}
|
||||||
@@ -205,6 +275,10 @@ async fn run_raw_transaction_scenario(admin: &mut tokio_postgres::Client, uri: &
|
|||||||
if let std::result::Result::Err(error) = divergent_result {
|
if let std::result::Result::Err(error) = divergent_result {
|
||||||
return std::result::Result::Err(error);
|
return std::result::Result::Err(error);
|
||||||
}
|
}
|
||||||
|
let conflict_rollback_result = prove_conflict_case_rollback(admin, uri).await;
|
||||||
|
if let std::result::Result::Err(error) = conflict_rollback_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
let race_result = prove_retention_races(uri).await;
|
let race_result = prove_retention_races(uri).await;
|
||||||
if let std::result::Result::Err(error) = race_result {
|
if let std::result::Result::Err(error) = race_result {
|
||||||
return std::result::Result::Err(error);
|
return std::result::Result::Err(error);
|
||||||
@@ -219,7 +293,7 @@ async fn run_raw_transaction_scenario(admin: &mut tokio_postgres::Client, uri: &
|
|||||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
};
|
};
|
||||||
let reopened_health = reopened.health().await;
|
let reopened_health = reopened.health().await;
|
||||||
if !reopened_health.is_ready() || reopened_health.migration_version() != std::option::Option::Some(2) {
|
if !reopened_health.is_ready() || reopened_health.migration_version() != std::option::Option::Some(3) {
|
||||||
return std::result::Result::Err(LiveFailure::new("reopen_health"));
|
return std::result::Result::Err(LiveFailure::new("reopen_health"));
|
||||||
}
|
}
|
||||||
let reference_result = raw_reference(10);
|
let reference_result = raw_reference(10);
|
||||||
@@ -361,7 +435,7 @@ async fn prove_additional_observation_and_atomic_rollback(backend: &ksp_store_po
|
|||||||
return std::result::Result::Ok(());
|
return std::result::Result::Ok(());
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn prove_concurrent_identical_insert(uri: &str) -> std::result::Result<(), LiveFailure> {
|
async fn prove_concurrent_identical_insert(admin: &tokio_postgres::Client, uri: &str) -> std::result::Result<(), LiveFailure> {
|
||||||
let first = tokio::spawn(persist_once(uri.to_owned(), 20, 2_000, 20, 20));
|
let first = tokio::spawn(persist_once(uri.to_owned(), 20, 2_000, 20, 20));
|
||||||
let second = tokio::spawn(persist_once(uri.to_owned(), 20, 2_000, 20, 20));
|
let second = tokio::spawn(persist_once(uri.to_owned(), 20, 2_000, 20, 20));
|
||||||
let first_result = match joined_persist(first.await, "concurrent_identical_first") {
|
let first_result = match joined_persist(first.await, "concurrent_identical_first") {
|
||||||
@@ -378,6 +452,17 @@ async fn prove_concurrent_identical_insert(uri: &str) -> std::result::Result<(),
|
|||||||
if inserted != 1 || already != 1 {
|
if inserted != 1 || already != 1 {
|
||||||
return std::result::Result::Err(LiveFailure::new("concurrent_identical_outcome"));
|
return std::result::Result::Err(LiveFailure::new("concurrent_identical_outcome"));
|
||||||
}
|
}
|
||||||
|
let reference = match raw_reference(20) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let key = ksp_store_api::RawObservationKey::new([20; 32]);
|
||||||
|
let projection_result = variant_projection_is_exact(admin, &reference, &key).await;
|
||||||
|
match projection_result {
|
||||||
|
std::result::Result::Ok(true) => {},
|
||||||
|
std::result::Result::Ok(false) => return std::result::Result::Err(LiveFailure::new("concurrent_identical_variant_projection")),
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
return std::result::Result::Ok(());
|
return std::result::Result::Ok(());
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -394,23 +479,61 @@ async fn prove_concurrent_divergent_insert(admin: &tokio_postgres::Client, uri:
|
|||||||
};
|
};
|
||||||
let pair = [first_result, second_result];
|
let pair = [first_result, second_result];
|
||||||
let inserted = pair.iter().filter(|value| return matches_inserted(value)).count();
|
let inserted = pair.iter().filter(|value| return matches_inserted(value)).count();
|
||||||
let conflicts = pair
|
let quarantined = pair.iter().filter(|value| return matches_quarantined_conflict(value)).count();
|
||||||
.iter()
|
if inserted != 1 || quarantined != 1 {
|
||||||
.filter(|value| matches!(value, LivePersistResult::BackendError(ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict)))
|
|
||||||
.count();
|
|
||||||
if inserted != 1 || conflicts != 1 {
|
|
||||||
return std::result::Result::Err(LiveFailure::new("concurrent_divergent_outcome"));
|
return std::result::Result::Err(LiveFailure::new("concurrent_divergent_outcome"));
|
||||||
}
|
}
|
||||||
|
let reference = match raw_reference(30) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
let first_key = ksp_store_api::RawObservationKey::new([31; 32]);
|
let first_key = ksp_store_api::RawObservationKey::new([31; 32]);
|
||||||
let second_key = ksp_store_api::RawObservationKey::new([32; 32]);
|
let second_key = ksp_store_api::RawObservationKey::new([32; 32]);
|
||||||
let first_exists = observation_exists(admin, &first_key).await;
|
let state = conflict_state_is_exact(admin, &reference, &first_key, &second_key).await;
|
||||||
let second_exists = observation_exists(admin, &second_key).await;
|
match state {
|
||||||
let count = match (first_exists, second_exists) {
|
std::result::Result::Ok(true) => {},
|
||||||
(std::result::Result::Ok(first_value), std::result::Result::Ok(second_value)) => usize::from(first_value) + usize::from(second_value),
|
std::result::Result::Ok(false) => return std::result::Result::Err(LiveFailure::new("concurrent_divergent_state")),
|
||||||
_ => return std::result::Result::Err(LiveFailure::new("concurrent_divergent_observation_probe")),
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn prove_conflict_case_rollback(admin: &tokio_postgres::Client, uri: &str) -> std::result::Result<(), LiveFailure> {
|
||||||
|
let seed = persist_once(uri.to_owned(), 82, 8_200, 82, 82).await;
|
||||||
|
match seed {
|
||||||
|
std::result::Result::Ok(LivePersistResult::Outcome(value)) if value.entity() == ksp_store_api::RawEntityWriteOutcome::Inserted => {},
|
||||||
|
_ => return std::result::Result::Err(LiveFailure::new("conflict_rollback_seed")),
|
||||||
|
}
|
||||||
|
let backend = match open_backend(uri, "devnet", true, true).await {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
};
|
};
|
||||||
if count != 1 {
|
let transaction = match raw_transaction(82, 8_200, 83) {
|
||||||
return std::result::Result::Err(LiveFailure::new("concurrent_divergent_rollback"));
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let observation = match raw_observation(10, 82, 83) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let result = backend.persist_raw_transaction_acquisition(transaction, observation, ksp_store_api::RawTransactionAcquisitionMode::Normal).await;
|
||||||
|
match result {
|
||||||
|
std::result::Result::Err(error) if error.kind() == ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict => {},
|
||||||
|
_ => return std::result::Result::Err(LiveFailure::new("conflict_rollback_expected_failure")),
|
||||||
|
}
|
||||||
|
let close_result = close_backend(backend).await;
|
||||||
|
if let std::result::Result::Err(error) = close_result {
|
||||||
|
return std::result::Result::Err(error);
|
||||||
|
}
|
||||||
|
let reference = match raw_reference(82) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
let state = conflict_rollback_state_is_exact(admin, &reference).await;
|
||||||
|
match state {
|
||||||
|
std::result::Result::Ok(true) => {},
|
||||||
|
std::result::Result::Ok(false) => return std::result::Result::Err(LiveFailure::new("conflict_rollback_state")),
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
}
|
}
|
||||||
return std::result::Result::Ok(());
|
return std::result::Result::Ok(());
|
||||||
}
|
}
|
||||||
@@ -883,7 +1006,7 @@ async fn persist_once(
|
|||||||
let persisted = backend.persist_raw_transaction_acquisition(transaction, observation, ksp_store_api::RawTransactionAcquisitionMode::ForceRehydrate).await;
|
let persisted = backend.persist_raw_transaction_acquisition(transaction, observation, ksp_store_api::RawTransactionAcquisitionMode::ForceRehydrate).await;
|
||||||
let result = match persisted {
|
let result = match persisted {
|
||||||
std::result::Result::Ok(value) => LivePersistResult::Outcome(value),
|
std::result::Result::Ok(value) => LivePersistResult::Outcome(value),
|
||||||
std::result::Result::Err(error) => LivePersistResult::BackendError(error.kind()),
|
std::result::Result::Err(_) => LivePersistResult::BackendError,
|
||||||
};
|
};
|
||||||
let close_result = close_backend(backend).await;
|
let close_result = close_backend(backend).await;
|
||||||
if let std::result::Result::Err(error) = close_result {
|
if let std::result::Result::Err(error) = close_result {
|
||||||
@@ -951,6 +1074,16 @@ fn matches_inserted(value: &LivePersistResult) -> bool {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn matches_quarantined_conflict(value: &LivePersistResult) -> bool {
|
||||||
|
return matches!(
|
||||||
|
value,
|
||||||
|
LivePersistResult::Outcome(outcome)
|
||||||
|
if outcome.entity() == ksp_store_api::RawEntityWriteOutcome::AlreadyPresent
|
||||||
|
&& outcome.observation() == ksp_store_api::RawObservationWriteOutcome::Inserted
|
||||||
|
&& outcome.transaction_variant() == std::option::Option::Some(ksp_store_api::RawTransactionVariantWriteOutcome::QuarantinedConflict)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
fn matches_already_present(value: &LivePersistResult) -> bool {
|
fn matches_already_present(value: &LivePersistResult) -> bool {
|
||||||
return matches!(
|
return matches!(
|
||||||
value,
|
value,
|
||||||
@@ -1245,14 +1378,56 @@ async fn transaction_exists(client: &tokio_postgres::Client, reference: &ksp_sto
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn observation_exists(client: &tokio_postgres::Client, key: &ksp_store_api::RawObservationKey) -> std::result::Result<bool, LiveFailure> {
|
async fn conflict_state_is_exact(
|
||||||
let key_bytes: &[u8] = key.as_bytes();
|
client: &tokio_postgres::Client,
|
||||||
let row = match client.query_one(LIVE_OBSERVATION_EXISTS_SQL, &[&key_bytes]).await {
|
reference: &ksp_store_api::RawTransactionReference,
|
||||||
|
first_key: &ksp_store_api::RawObservationKey,
|
||||||
|
second_key: &ksp_store_api::RawObservationKey,
|
||||||
|
) -> std::result::Result<bool, LiveFailure> {
|
||||||
|
let signature = reference.signature();
|
||||||
|
let signature_bytes: &[u8] = signature.as_bytes();
|
||||||
|
let first_key_bytes: &[u8] = first_key.as_bytes();
|
||||||
|
let second_key_bytes: &[u8] = second_key.as_bytes();
|
||||||
|
let row = match client.query_one(LIVE_CONFLICT_STATE_SQL, &[&signature_bytes, &first_key_bytes, &second_key_bytes]).await {
|
||||||
std::result::Result::Ok(value) => value,
|
std::result::Result::Ok(value) => value,
|
||||||
std::result::Result::Err(_) => return std::result::Result::Err(LiveFailure::new("observation_exists_probe")),
|
std::result::Result::Err(_) => return std::result::Result::Err(LiveFailure::new("conflict_state_probe")),
|
||||||
};
|
};
|
||||||
return match row.try_get::<usize, bool>(0) {
|
return match row.try_get::<usize, bool>(0) {
|
||||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||||
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("observation_exists_decode")),
|
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("conflict_state_decode")),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn conflict_rollback_state_is_exact(
|
||||||
|
client: &tokio_postgres::Client,
|
||||||
|
reference: &ksp_store_api::RawTransactionReference,
|
||||||
|
) -> std::result::Result<bool, LiveFailure> {
|
||||||
|
let signature = reference.signature();
|
||||||
|
let signature_bytes: &[u8] = signature.as_bytes();
|
||||||
|
let row = match client.query_one(LIVE_CONFLICT_ROLLBACK_STATE_SQL, &[&signature_bytes]).await {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return std::result::Result::Err(LiveFailure::new("conflict_rollback_state_probe")),
|
||||||
|
};
|
||||||
|
return match row.try_get::<usize, bool>(0) {
|
||||||
|
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||||
|
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("conflict_rollback_state_decode")),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn variant_projection_is_exact(
|
||||||
|
client: &tokio_postgres::Client,
|
||||||
|
reference: &ksp_store_api::RawTransactionReference,
|
||||||
|
key: &ksp_store_api::RawObservationKey,
|
||||||
|
) -> std::result::Result<bool, LiveFailure> {
|
||||||
|
let signature = reference.signature();
|
||||||
|
let signature_bytes: &[u8] = signature.as_bytes();
|
||||||
|
let key_bytes: &[u8] = key.as_bytes();
|
||||||
|
let row = match client.query_one(LIVE_VARIANT_IDEMPOTENCE_SQL, &[&signature_bytes, &key_bytes]).await {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return std::result::Result::Err(LiveFailure::new("variant_projection_probe")),
|
||||||
|
};
|
||||||
|
return match row.try_get::<usize, bool>(0) {
|
||||||
|
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||||
|
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("variant_projection_decode")),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,58 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/tests/v003_migration_completeness.rs
|
||||||
|
// version: 2
|
||||||
|
|
||||||
|
#![warn(missing_docs)]
|
||||||
|
#![deny(unreachable_pub)]
|
||||||
|
#![forbid(unsafe_code)]
|
||||||
|
|
||||||
|
//! V003 additive migration and frozen-history canaries.
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_extension_is_wired_without_modifying_frozen_registry_sources() {
|
||||||
|
let crate_root = include_str!("../src/lib.rs");
|
||||||
|
let legacy_migration = include_str!("../src/migration.rs");
|
||||||
|
let legacy_schema = include_str!("../src/schema.rs");
|
||||||
|
let extension = include_str!("../src/migration_v003.rs");
|
||||||
|
assert!(crate_root.contains("mod migration_v003;"));
|
||||||
|
assert!(crate_root.contains("mod schema_v003;"));
|
||||||
|
assert!(crate_root.contains("pub(crate) use self::migration::bootstrap;"));
|
||||||
|
assert!(crate_root.contains("pub(crate) use self::migration::current_migration_version;"));
|
||||||
|
assert!(crate_root.contains("pub(crate) use self::migration_v003::bootstrap_v003;"));
|
||||||
|
assert!(crate_root.contains("pub(crate) use self::migration_v003::current_migration_version_v003;"));
|
||||||
|
assert!(legacy_migration.starts_with("// file: crates/ksp-store-postgres-lib/src/migration.rs\n// version: 10\n"));
|
||||||
|
assert!(legacy_schema.starts_with("// file: crates/ksp-store-postgres-lib/src/schema.rs\n// version: 9\n"));
|
||||||
|
assert!(!legacy_migration.contains("V003_RESOURCES"));
|
||||||
|
assert!(!legacy_schema.contains("v003_raw_transaction_variants"));
|
||||||
|
for checksum in [
|
||||||
|
"d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450",
|
||||||
|
"31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51",
|
||||||
|
"ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e",
|
||||||
|
] {
|
||||||
|
assert!(extension.contains(checksum), "missing frozen history checksum canary: {checksum}");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_sources_remain_backend_private_secret_safe_and_without_facade_edges() {
|
||||||
|
let migration = include_str!("../src/migration_v003.rs");
|
||||||
|
let schema = include_str!("../src/schema_v003.rs");
|
||||||
|
let combined = std::format!("{migration}\n{schema}");
|
||||||
|
for forbidden in ["std::env", "dotenv", "KSP_SECRET_", "PGPASSWORD", "ksp_store_lib", "ksp_config_lib", "sqlx::", "unwrap()", "expect(", "unsafe "] {
|
||||||
|
assert!(!combined.contains(forbidden), "forbidden V003 migration dependency/material detected: {forbidden}");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_migration_is_schema_only_and_defers_business_backfill() {
|
||||||
|
let schema = include_str!("../src/schema_v003.rs");
|
||||||
|
assert!(schema.contains("ksp_raw_transaction_variants"));
|
||||||
|
assert!(schema.contains("ksp_raw_transaction_canonical_selectors"));
|
||||||
|
assert!(schema.contains("ksp_raw_transaction_observation_variants"));
|
||||||
|
assert!(schema.contains("ksp_raw_transaction_conflicts"));
|
||||||
|
for forbidden in ["INSERT INTO ksp_raw_transaction_variants", "INSERT INTO ksp_raw_transaction_observation_variants", "UPDATE ksp_raw_transactions"] {
|
||||||
|
assert!(!schema.contains(forbidden), "V003 source unexpectedly performs business backfill: {forbidden}");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
194
crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
Normal file
194
crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
Normal file
@@ -0,0 +1,194 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
|
||||||
|
// version: 5
|
||||||
|
|
||||||
|
#![warn(missing_docs)]
|
||||||
|
#![deny(unreachable_pub)]
|
||||||
|
#![forbid(unsafe_code)]
|
||||||
|
|
||||||
|
//! Static completeness canaries for the first V003 variant-aware PostgreSQL write slice.
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_acquisition_orders_identity_lock_bootstrap_observation_and_variant_link_atomically() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let start = source.find("pub(crate) async fn persist_raw_transaction_acquisition(");
|
||||||
|
let end = source.find("pub(crate) async fn record_raw_transaction_observation(");
|
||||||
|
let (start, end) = match (start, end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.004 acquisition function markers are missing"),
|
||||||
|
};
|
||||||
|
let body = &source[start..end];
|
||||||
|
let lock = body.find("load_locked_transaction_row");
|
||||||
|
let bootstrap = body.find("ensure_v003_transaction_variant_bootstrap");
|
||||||
|
let compare = body.find("compare_existing_transaction");
|
||||||
|
let observation = body.find("persist_observation_row");
|
||||||
|
let link = body.find("persist_transaction_variant_link");
|
||||||
|
assert!(matches!((lock, bootstrap, compare), (Some(lock), Some(bootstrap), Some(compare)) if lock < bootstrap && bootstrap < compare));
|
||||||
|
assert!(matches!((observation, link), (Some(observation), Some(link)) if observation < link));
|
||||||
|
assert!(body.contains("sql_transaction.commit().await"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_native_variant_reuse_is_exact_and_hash_is_only_a_prefilter() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let exact_sql = source.find("const SELECT_EXACT_NATIVE_VARIANT_SQL");
|
||||||
|
let exact_helper = source.find("async fn persist_or_reuse_native_transaction_variant(");
|
||||||
|
assert!(exact_sql.is_some());
|
||||||
|
assert!(exact_helper.is_some());
|
||||||
|
assert!(source.contains("content_hash = $6 AND payload = $7"));
|
||||||
|
assert!(source.contains("block_time_unix_millis IS NOT DISTINCT FROM $3"));
|
||||||
|
assert!(source.contains("origin_kind = 'native'"));
|
||||||
|
assert!(!source.contains("UNIQUE (transaction_signature, content_hash)"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_legacy_observation_without_mapping_is_not_fabricated() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let start = source.find("async fn persist_transaction_variant_link(");
|
||||||
|
let end = source.find("async fn rehydrate_canonical_transaction_variant(");
|
||||||
|
let (start, end) = match (start, end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.004 observation-variant link markers are missing"),
|
||||||
|
};
|
||||||
|
let body = &source[start..end];
|
||||||
|
assert!(body.contains("RawObservationWriteOutcome::Inserted"));
|
||||||
|
assert!(body.contains("RawObservationWriteOutcome::AlreadyPresent"));
|
||||||
|
assert!(body.contains("Ok(std::option::Option::None) => return std::result::Result::Ok(())"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_007_shared_comparator_promotes_more_complete_and_quarantines_conflict() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
assert!(source.contains("ksp_store_api::compare_raw_transaction_variants"));
|
||||||
|
assert!(source.contains("ExistingTransactionMatch::ActiveCompatibleLessComplete"));
|
||||||
|
assert!(source.contains("ExistingTransactionMatch::ActiveCompatibleMoreComplete"));
|
||||||
|
assert!(source.contains("promote_more_complete_transaction_variant"));
|
||||||
|
assert!(source.contains("UPDATE ksp_raw_transaction_canonical_selectors SET canonical_variant_id"));
|
||||||
|
assert!(source.contains("canonical_revision = $5::TEXT::NUMERIC"));
|
||||||
|
assert!(source.contains("UPDATE ksp_raw_transactions SET slot = $2::TEXT::NUMERIC"));
|
||||||
|
assert!(source.contains("PostgreSQL Store committed a proved RAW transaction canonical promotion"));
|
||||||
|
assert!(!source.contains("log_raw_transaction_compatible_more_complete_log_messages"));
|
||||||
|
assert!(source.contains("INSERT INTO ksp_raw_transaction_conflicts"));
|
||||||
|
assert!(source.contains("ExistingTransactionMatch::ActiveConflict"));
|
||||||
|
assert!(source.contains("RawTransactionVariantWriteOutcome::QuarantinedConflict"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_007_conflict_is_persisted_inside_acquisition_transaction_before_observation_and_commit() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let start = source.find("pub(crate) async fn persist_raw_transaction_acquisition(");
|
||||||
|
let end = source.find("pub(crate) async fn record_raw_transaction_observation(");
|
||||||
|
let (start, end) = match (start, end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.007 acquisition function markers are missing"),
|
||||||
|
};
|
||||||
|
let body = &source[start..end];
|
||||||
|
let persist_variant = body.find("persist_or_reuse_native_transaction_variant");
|
||||||
|
let conflict = body.find("open_or_update_raw_transaction_conflict");
|
||||||
|
let observation = body.find("persist_observation_row");
|
||||||
|
let commit = body.rfind("sql_transaction.commit().await");
|
||||||
|
let transition_log = body.find("log_raw_transaction_conflict_quarantined_committed");
|
||||||
|
assert!(matches!(
|
||||||
|
(persist_variant, conflict, observation, commit, transition_log),
|
||||||
|
(Some(persist_variant), Some(conflict), Some(observation), Some(commit), Some(transition_log))
|
||||||
|
if persist_variant < conflict && conflict < observation && observation < commit && commit < transition_log
|
||||||
|
));
|
||||||
|
assert!(body.contains("RawTransactionVariantWriteOutcome::QuarantinedConflict"));
|
||||||
|
let conflict_start = source.find("async fn open_or_update_raw_transaction_conflict(");
|
||||||
|
let conflict_end = source.find("async fn persist_observation_row(");
|
||||||
|
let (conflict_start, conflict_end) = match (conflict_start, conflict_end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.007 conflict helper markers are missing"),
|
||||||
|
};
|
||||||
|
let conflict_helper = &source[conflict_start..conflict_end];
|
||||||
|
assert!(conflict_helper.contains("LOCK_TRANSACTION_CONFLICT_SQL"));
|
||||||
|
assert!(conflict_helper.contains("INSERT_TRANSACTION_CONFLICT_SQL"));
|
||||||
|
assert!(conflict_helper.contains("UPDATE_TRANSACTION_CONFLICT_SQL"));
|
||||||
|
assert!(conflict_helper.contains("checked_add(1)"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_006_promotion_is_inside_acquisition_transaction_and_preserves_previous_canonical() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let start = source.find("pub(crate) async fn persist_raw_transaction_acquisition(");
|
||||||
|
let end = source.find("pub(crate) async fn record_raw_transaction_observation(");
|
||||||
|
let (start, end) = match (start, end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.006 acquisition function markers are missing"),
|
||||||
|
};
|
||||||
|
let body = &source[start..end];
|
||||||
|
let persist_variant = body.find("persist_or_reuse_native_transaction_variant");
|
||||||
|
let promote = body.find("promote_more_complete_transaction_variant");
|
||||||
|
let observation = body.find("persist_observation_row");
|
||||||
|
let commit = body.rfind("sql_transaction.commit().await");
|
||||||
|
let transition_log = body.find("log_raw_transaction_canonical_promotion_committed");
|
||||||
|
assert!(matches!(
|
||||||
|
(persist_variant, promote, observation, commit, transition_log),
|
||||||
|
(Some(persist_variant), Some(promote), Some(observation), Some(commit), Some(transition_log))
|
||||||
|
if persist_variant < promote && promote < observation && observation < commit && commit < transition_log
|
||||||
|
));
|
||||||
|
let promotion_start = source.find("async fn promote_more_complete_transaction_variant(");
|
||||||
|
let promotion_end = source.find("async fn persist_transaction_variant_link(");
|
||||||
|
let (promotion_start, promotion_end) = match (promotion_start, promotion_end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.006 promotion helper markers are missing"),
|
||||||
|
};
|
||||||
|
let promotion = &source[promotion_start..promotion_end];
|
||||||
|
assert!(promotion.contains("preserve_previous_canonical_variant"));
|
||||||
|
assert!(promotion.contains("DELETE_ARCHIVE_PAYLOAD_SQL"));
|
||||||
|
assert!(promotion.contains("UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL"));
|
||||||
|
assert!(promotion.contains("UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL"));
|
||||||
|
assert!(promotion.contains("next_transaction_canonical_revision"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_008_conflict_path_never_mutates_canonical_projection_or_selector() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let start = source.find("ExistingTransactionMatch::ActiveConflict(variant_comparison) => {");
|
||||||
|
let end = source.find("ExistingTransactionMatch::Purged => {");
|
||||||
|
let (start, end) = match (start, end) {
|
||||||
|
(std::option::Option::Some(start), std::option::Option::Some(end)) if start < end => (start, end),
|
||||||
|
_ => panic!("pre.008 conflict branch markers are missing"),
|
||||||
|
};
|
||||||
|
let branch = &source[start..end];
|
||||||
|
assert!(branch.contains("persist_or_reuse_native_transaction_variant"));
|
||||||
|
assert!(branch.contains("open_or_update_raw_transaction_conflict"));
|
||||||
|
assert!(branch.contains("RawTransactionVariantWriteOutcome::QuarantinedConflict"));
|
||||||
|
assert!(!branch.contains("promote_more_complete_transaction_variant"));
|
||||||
|
assert!(!branch.contains("UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL"));
|
||||||
|
assert!(!branch.contains("UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_008_variant_transition_helpers_cannot_commit_independently() {
|
||||||
|
let source = include_str!("../src/raw_transaction.rs");
|
||||||
|
let promotion_start = source.find("async fn promote_more_complete_transaction_variant(");
|
||||||
|
let promotion_end = source.find("async fn preserve_previous_canonical_variant(");
|
||||||
|
let conflict_start = source.find("async fn open_or_update_raw_transaction_conflict(");
|
||||||
|
let conflict_end = source.find("async fn promote_more_complete_transaction_variant(");
|
||||||
|
let (promotion_start, promotion_end, conflict_start, conflict_end) = match (promotion_start, promotion_end, conflict_start, conflict_end) {
|
||||||
|
(Some(promotion_start), Some(promotion_end), Some(conflict_start), Some(conflict_end))
|
||||||
|
if promotion_start < promotion_end && conflict_start < conflict_end =>
|
||||||
|
{
|
||||||
|
(promotion_start, promotion_end, conflict_start, conflict_end)
|
||||||
|
},
|
||||||
|
_ => panic!("pre.008 transition helper markers are missing"),
|
||||||
|
};
|
||||||
|
let promotion = &source[promotion_start..promotion_end];
|
||||||
|
let conflict = &source[conflict_start..conflict_end];
|
||||||
|
for helper in [promotion, conflict] {
|
||||||
|
assert!(helper.contains("&deadpool_postgres::Transaction<'_>"));
|
||||||
|
assert!(!helper.contains(".commit().await"));
|
||||||
|
}
|
||||||
|
assert!(source.contains("const LOCK_TRANSACTION_SQL: &str = \"SELECT"));
|
||||||
|
assert!(source.contains("WHERE signature = $1 FOR UPDATE\";"));
|
||||||
|
assert!(source.contains("canonical_variant_id = $2::TEXT::NUMERIC AND canonical_revision = $3::TEXT::NUMERIC"));
|
||||||
|
assert!(source.contains("AND revision = $8::TEXT::NUMERIC"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
// file: crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
||||||
// version: 8
|
// version: 10
|
||||||
|
|
||||||
fn applied(version: i64, name: &str, checksum: &str) -> super::AppliedMigration {
|
fn applied(version: i64, name: &str, checksum: &str) -> super::AppliedMigration {
|
||||||
return super::AppliedMigration { checksum: checksum.to_owned(), name: name.to_owned(), version };
|
return super::AppliedMigration { checksum: checksum.to_owned(), name: name.to_owned(), version };
|
||||||
@@ -52,6 +52,7 @@ fn pre_003_v002_registry_is_complete_and_keeps_v000_v001_checksums_stable() {
|
|||||||
assert_eq!(super::migration_checksum(v002), "ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e");
|
assert_eq!(super::migration_checksum(v002), "ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e");
|
||||||
assert!(super::validate_embedded_registry(super::EMBEDDED_MIGRATIONS).is_ok());
|
assert!(super::validate_embedded_registry(super::EMBEDDED_MIGRATIONS).is_ok());
|
||||||
assert_eq!(crate::current_migration_version(), 2);
|
assert_eq!(crate::current_migration_version(), 2);
|
||||||
|
assert_eq!(crate::current_migration_version_v003(), 3);
|
||||||
let full = [
|
let full = [
|
||||||
applied(0, v000.name, super::migration_checksum(v000).as_str()),
|
applied(0, v000.name, super::migration_checksum(v000).as_str()),
|
||||||
applied(1, v001.name, super::migration_checksum(v001).as_str()),
|
applied(1, v001.name, super::migration_checksum(v001).as_str()),
|
||||||
|
|||||||
83
crates/ksp-store-postgres-lib/unit_tests/migration_v003.rs
Normal file
83
crates/ksp-store-postgres-lib/unit_tests/migration_v003.rs
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/unit_tests/migration_v003.rs
|
||||||
|
// version: 3
|
||||||
|
|
||||||
|
fn applied(version: i64, name: &str, checksum: &str) -> super::AppliedMigration {
|
||||||
|
return super::AppliedMigration { checksum: checksum.to_owned(), name: name.to_owned(), version };
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_extension_preserves_frozen_history_and_advances_current_version() {
|
||||||
|
assert_eq!(crate::current_migration_version_v003(), 3);
|
||||||
|
assert_eq!(super::V000_CHECKSUM, "d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450");
|
||||||
|
assert_eq!(super::V001_CHECKSUM, "31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51");
|
||||||
|
assert_eq!(super::V002_CHECKSUM, "ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e");
|
||||||
|
let prefix = [
|
||||||
|
applied(0, "bootstrap", super::V000_CHECKSUM),
|
||||||
|
applied(1, "raw_transaction", super::V001_CHECKSUM),
|
||||||
|
applied(2, "raw_account_state", super::V002_CHECKSUM),
|
||||||
|
];
|
||||||
|
assert_eq!(super::validate_history(&prefix).ok(), std::option::Option::Some(3));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_resource_checksum_and_history_are_stable() {
|
||||||
|
let checksum = super::migration_checksum_v003();
|
||||||
|
assert_eq!(checksum, "3c9cf41877c96944a378abb41ee7a8d8ab7f99fc6dc2bb7a26c295cf8f9fc914");
|
||||||
|
let full = [
|
||||||
|
applied(0, "bootstrap", super::V000_CHECKSUM),
|
||||||
|
applied(1, "raw_transaction", super::V001_CHECKSUM),
|
||||||
|
applied(2, "raw_account_state", super::V002_CHECKSUM),
|
||||||
|
applied(3, super::V003_NAME, checksum.as_str()),
|
||||||
|
];
|
||||||
|
assert_eq!(super::validate_history(&full).ok(), std::option::Option::Some(4));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_history_rejects_divergence_and_future_versions() {
|
||||||
|
let checksum = super::migration_checksum_v003();
|
||||||
|
let divergent = [
|
||||||
|
applied(0, "bootstrap", super::V000_CHECKSUM),
|
||||||
|
applied(1, "raw_transaction", super::V001_CHECKSUM),
|
||||||
|
applied(2, "raw_account_state", super::V002_CHECKSUM),
|
||||||
|
applied(3, super::V003_NAME, "changed"),
|
||||||
|
];
|
||||||
|
let divergent_result = super::validate_history(&divergent);
|
||||||
|
assert_eq!(divergent_result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::MigrationMismatch));
|
||||||
|
let future = [
|
||||||
|
applied(0, "bootstrap", super::V000_CHECKSUM),
|
||||||
|
applied(1, "raw_transaction", super::V001_CHECKSUM),
|
||||||
|
applied(2, "raw_account_state", super::V002_CHECKSUM),
|
||||||
|
applied(3, super::V003_NAME, checksum.as_str()),
|
||||||
|
applied(4, "future", "future"),
|
||||||
|
];
|
||||||
|
let future_result = super::validate_history(&future);
|
||||||
|
assert_eq!(future_result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::SchemaNewer));
|
||||||
|
let direct_future = [
|
||||||
|
applied(0, "bootstrap", super::V000_CHECKSUM),
|
||||||
|
applied(1, "raw_transaction", super::V001_CHECKSUM),
|
||||||
|
applied(2, "raw_account_state", super::V002_CHECKSUM),
|
||||||
|
applied(4, "future", "future"),
|
||||||
|
];
|
||||||
|
let direct_future_result = super::validate_history(&direct_future);
|
||||||
|
assert_eq!(direct_future_result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::SchemaNewer),);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_bootstrap_uses_legacy_engine_only_before_v002_and_absorbs_completed_v003_race() {
|
||||||
|
assert!(super::legacy_bootstrap_required(std::option::Option::None));
|
||||||
|
assert!(super::legacy_bootstrap_required(std::option::Option::Some(0)));
|
||||||
|
assert!(super::legacy_bootstrap_required(std::option::Option::Some(1)));
|
||||||
|
assert!(!super::legacy_bootstrap_required(std::option::Option::Some(2)));
|
||||||
|
assert!(!super::legacy_bootstrap_required(std::option::Option::Some(3)));
|
||||||
|
assert!(super::legacy_schema_newer_is_completed_v003(std::option::Option::Some(3)));
|
||||||
|
assert!(!super::legacy_schema_newer_is_completed_v003(std::option::Option::Some(2)));
|
||||||
|
assert!(!super::legacy_schema_newer_is_completed_v003(std::option::Option::Some(4)));
|
||||||
|
assert!(super::v003_apply_allowed(false, false));
|
||||||
|
assert!(super::v003_apply_allowed(false, true));
|
||||||
|
assert!(super::v003_apply_allowed(true, true));
|
||||||
|
assert!(!super::v003_apply_allowed(true, false));
|
||||||
|
return;
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
// file: crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||||
// version: 10
|
// version: 15
|
||||||
|
|
||||||
fn network() -> ksp_store_api::RawNetworkId {
|
fn network() -> ksp_store_api::RawNetworkId {
|
||||||
return match ksp_store_api::RawNetworkId::new("devnet") {
|
return match ksp_store_api::RawNetworkId::new("devnet") {
|
||||||
@@ -346,7 +346,7 @@ fn pre_005_existing_full_content_requires_exact_payload_equality_not_hash_only()
|
|||||||
let network = network();
|
let network = network();
|
||||||
let incoming = raw_transaction(9, &[1, 2, 3, 4], 7);
|
let incoming = raw_transaction(9, &[1, 2, 3, 4], 7);
|
||||||
let matching = super::compare_existing_transaction(&network, transaction_row("full"), &incoming);
|
let matching = super::compare_existing_transaction(&network, transaction_row("full"), &incoming);
|
||||||
assert!(matching.is_err(), "fixture intentionally differs in slot/content and must conflict");
|
assert!(matches!(matching, std::result::Result::Ok(super::ExistingTransactionMatch::ActiveConflict(_))));
|
||||||
let mut exact_row = transaction_row("full");
|
let mut exact_row = transaction_row("full");
|
||||||
exact_row.signature = vec![9; 64];
|
exact_row.signature = vec![9; 64];
|
||||||
exact_row.slot_text = "42".to_owned();
|
exact_row.slot_text = "42".to_owned();
|
||||||
@@ -362,7 +362,7 @@ fn pre_005_existing_full_content_requires_exact_payload_equality_not_hash_only()
|
|||||||
same_hash_different_bytes.format_version = 1;
|
same_hash_different_bytes.format_version = 1;
|
||||||
same_hash_different_bytes.payload = std::option::Option::Some(vec![9, 9, 9, 9]);
|
same_hash_different_bytes.payload = std::option::Option::Some(vec![9, 9, 9, 9]);
|
||||||
let conflict = super::compare_existing_transaction(&network, same_hash_different_bytes, &incoming);
|
let conflict = super::compare_existing_transaction(&network, same_hash_different_bytes, &incoming);
|
||||||
assert_eq!(conflict.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::Conflict));
|
assert!(matches!(conflict, std::result::Result::Ok(super::ExistingTransactionMatch::ActiveConflict(_))));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -458,7 +458,7 @@ fn v0_3_15_pre_014_log_messages_conflict_diagnostic_distinguishes_prefix_truncat
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn v0_3_15_pre_014_fix_001_incoming_truncated_logs_are_compatible_only_when_missing_middle_is_proven() {
|
fn v0_3_16_pre_005_shared_comparator_accepts_only_proved_incoming_truncation_in_postgres_path() {
|
||||||
let stored = raw_transaction(
|
let stored = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Program log: detail-a","Program log: detail-b","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Program log: detail-a","Program log: detail-b","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
@@ -466,10 +466,15 @@ fn v0_3_15_pre_014_fix_001_incoming_truncated_logs_are_compatible_only_when_miss
|
|||||||
);
|
);
|
||||||
let incoming = raw_transaction(
|
let incoming = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(super::raw_transaction_incoming_truncated_log_messages_compatible(&stored, &incoming));
|
let relation = ksp_store_api::compare_raw_transaction_variants(&stored, &incoming);
|
||||||
|
let relation = match relation {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
assert_eq!(relation.relation(), ksp_store_api::RawTransactionVariantRelation::CompatibleLessComplete);
|
||||||
let mut stored_row = transaction_row("full");
|
let mut stored_row = transaction_row("full");
|
||||||
stored_row.signature = stored.reference().signature().as_bytes().to_vec();
|
stored_row.signature = stored.reference().signature().as_bytes().to_vec();
|
||||||
stored_row.slot_text = stored.slot().to_string();
|
stored_row.slot_text = stored.slot().to_string();
|
||||||
@@ -479,12 +484,12 @@ fn v0_3_15_pre_014_fix_001_incoming_truncated_logs_are_compatible_only_when_miss
|
|||||||
stored_row.content_hash = stored.payload().content_hash().as_bytes().to_vec();
|
stored_row.content_hash = stored.payload().content_hash().as_bytes().to_vec();
|
||||||
stored_row.payload = std::option::Option::Some(stored.payload().bytes().to_vec());
|
stored_row.payload = std::option::Option::Some(stored.payload().bytes().to_vec());
|
||||||
let comparison = super::compare_existing_transaction(&network(), stored_row, &incoming);
|
let comparison = super::compare_existing_transaction(&network(), stored_row, &incoming);
|
||||||
assert!(matches!(comparison, std::result::Result::Ok(super::ExistingTransactionMatch::ActiveIncomingTruncatedLogs)));
|
assert!(matches!(comparison, std::result::Result::Ok(super::ExistingTransactionMatch::ActiveCompatibleLessComplete)));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn v0_3_15_pre_014_fix_001_truncated_log_compatibility_rejects_unproven_or_reverse_changes() {
|
fn v0_3_16_pre_006_shared_comparator_rejects_unproved_shapes_and_accepts_more_complete_promotion() {
|
||||||
let stored = raw_transaction(
|
let stored = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Program log: detail-a","Program log: detail-b","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Program log: detail-a","Program log: detail-b","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
@@ -492,34 +497,52 @@ fn v0_3_15_pre_014_fix_001_truncated_log_compatibility_rejects_unproven_or_rever
|
|||||||
);
|
);
|
||||||
let bad_prefix = raw_transaction(
|
let bad_prefix = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: changed","Log truncated","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: changed","Log truncated"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(!super::raw_transaction_incoming_truncated_log_messages_compatible(&stored, &bad_prefix));
|
|
||||||
let post_marker_material = raw_transaction(
|
let post_marker_material = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated","Program Other111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated","Program Other111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(super::raw_transaction_incoming_truncated_log_messages_compatible(&stored, &post_marker_material));
|
|
||||||
let other_meta = raw_transaction(
|
let other_meta = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5001,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5001,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(!super::raw_transaction_incoming_truncated_log_messages_compatible(&stored, &other_meta));
|
|
||||||
let non_exact_marker = raw_transaction(
|
let non_exact_marker = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated by provider"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated by provider"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(!super::raw_transaction_incoming_truncated_log_messages_compatible(&stored, &non_exact_marker));
|
for candidate in [&bad_prefix, &post_marker_material, &other_meta, &non_exact_marker] {
|
||||||
|
let relation = match ksp_store_api::compare_raw_transaction_variants(&stored, candidate) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
assert_ne!(relation.relation(), ksp_store_api::RawTransactionVariantRelation::CompatibleLessComplete);
|
||||||
|
assert_ne!(relation.relation(), ksp_store_api::RawTransactionVariantRelation::CompatibleMoreComplete);
|
||||||
|
}
|
||||||
let stored_truncated = raw_transaction(
|
let stored_truncated = raw_transaction(
|
||||||
9,
|
9,
|
||||||
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated","Program Vote111 success"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
br#"{"transaction":["AQID","base64"],"meta":{"err":null,"fee":5000,"logMessages":["Program Vote111 invoke [1]","Program log: stable","Log truncated"],"rewards":[]},"version":0,"transactionIndex":7}"#,
|
||||||
8,
|
8,
|
||||||
);
|
);
|
||||||
assert!(!super::raw_transaction_incoming_truncated_log_messages_compatible(&stored_truncated, &stored));
|
let more = match ksp_store_api::compare_raw_transaction_variants(&stored_truncated, &stored) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
assert_eq!(more.relation(), ksp_store_api::RawTransactionVariantRelation::CompatibleMoreComplete);
|
||||||
|
let mut stored_row = transaction_row("full");
|
||||||
|
stored_row.signature = stored_truncated.reference().signature().as_bytes().to_vec();
|
||||||
|
stored_row.slot_text = stored_truncated.slot().to_string();
|
||||||
|
stored_row.block_time_unix_millis = stored_truncated.block_time().and_then(|value| return i64::try_from(value.unix_millis()).ok());
|
||||||
|
stored_row.format_id = stored_truncated.payload().format_id().as_str().to_owned();
|
||||||
|
stored_row.format_version = i64::from(stored_truncated.payload().format_version());
|
||||||
|
stored_row.content_hash = stored_truncated.payload().content_hash().as_bytes().to_vec();
|
||||||
|
stored_row.payload = std::option::Option::Some(stored_truncated.payload().bytes().to_vec());
|
||||||
|
let comparison = super::compare_existing_transaction(&network(), stored_row, &stored);
|
||||||
|
assert!(matches!(comparison, std::result::Result::Ok(super::ExistingTransactionMatch::ActiveCompatibleMoreComplete)));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -817,3 +840,76 @@ fn pre_007_retained_payload_shape_rejects_empty_and_oversized_bytes() {
|
|||||||
assert_eq!(oversized.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
assert_eq!(oversized.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_variant_id_allocation_is_monotonic_and_bounded() {
|
||||||
|
assert_eq!(super::next_transaction_variant_id("0"), std::result::Result::Ok(1));
|
||||||
|
assert_eq!(super::next_transaction_variant_id("1"), std::result::Result::Ok(2));
|
||||||
|
let exhausted = super::next_transaction_variant_id(u64::MAX.to_string().as_str()).err();
|
||||||
|
assert_eq!(exhausted.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||||
|
assert_eq!(exhausted.as_ref().map(|value| return value.phase()), std::option::Option::Some("raw_variant_id_exhausted"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_variant_sql_uses_exact_content_and_identity_locking() {
|
||||||
|
assert!(super::LOCK_TRANSACTION_SQL.contains("FOR UPDATE"));
|
||||||
|
assert!(super::SELECT_EXACT_NATIVE_VARIANT_SQL.contains("content_hash = $6"));
|
||||||
|
assert!(super::SELECT_EXACT_NATIVE_VARIANT_SQL.contains("payload = $7"));
|
||||||
|
assert!(super::SELECT_EXACT_NATIVE_VARIANT_SQL.contains("origin_kind = 'native'"));
|
||||||
|
assert!(super::SELECT_MAX_VARIANT_ID_SQL.contains("MAX(variant_id)"));
|
||||||
|
assert!(!super::INSERT_TRANSACTION_VARIANT_SQL.contains("ON CONFLICT"));
|
||||||
|
assert!(!super::INSERT_TRANSACTION_VARIANT_SQL.contains("UNIQUE"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_004_variant_link_contract_keeps_legacy_observations_unfabricated() {
|
||||||
|
assert!(super::INSERT_TRANSACTION_VARIANT_LINK_SQL.contains("ksp_raw_transaction_observation_variants"));
|
||||||
|
assert!(super::SELECT_TRANSACTION_VARIANT_LINK_SQL.contains("WHERE observation_key = $1"));
|
||||||
|
assert!(!super::INSERT_TRANSACTION_VARIANT_LINK_SQL.contains("ON CONFLICT"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_006_canonical_revision_is_monotonic_and_bounded() {
|
||||||
|
assert_eq!(super::next_transaction_canonical_revision(1), std::result::Result::Ok(2));
|
||||||
|
let exhausted = super::next_transaction_canonical_revision(u64::MAX).err();
|
||||||
|
assert_eq!(exhausted.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||||
|
assert_eq!(exhausted.as_ref().map(|value| return value.phase()), std::option::Option::Some("raw_variant_canonical_revision_exhausted"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_006_promotion_sql_is_compare_guarded_and_projection_complete() {
|
||||||
|
assert!(super::SELECT_CANONICAL_VARIANT_SELECTOR_SQL.contains("canonical_revision::TEXT"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL.contains("canonical_variant_id = $2::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL.contains("canonical_revision = $3::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL.contains("canonical_variant_id = $4::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL.contains("slot = $2::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL.contains("content_hash = $6"));
|
||||||
|
assert!(super::UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL.contains("payload = $7"));
|
||||||
|
assert!(super::UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL.contains("retention_state = 'full'"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("slot = $3::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("block_time_unix_millis IS NOT DISTINCT FROM $4"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("format_id = $5"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("format_version = $6"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("content_hash = $7"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("retention_state = 'archived'"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("retention_state = 'full' AND payload = $8"));
|
||||||
|
assert!(!super::UPDATE_TRANSACTION_VARIANT_TO_FULL_SQL.contains("DELETE"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn v0_3_16_pre_007_conflict_sql_is_locked_revision_guarded_and_reopenable() {
|
||||||
|
assert!(super::LOCK_TRANSACTION_CONFLICT_SQL.contains("FOR UPDATE"));
|
||||||
|
assert!(super::INSERT_TRANSACTION_CONFLICT_SQL.contains("status, revision"));
|
||||||
|
assert!(super::INSERT_TRANSACTION_CONFLICT_SQL.contains("'open', 1"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_CONFLICT_SQL.contains("status = 'open'"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_CONFLICT_SQL.contains("revision = $2::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_CONFLICT_SQL.contains("AND revision = $8::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_CONFLICT_SQL.contains("canonical_variant_id = $3::TEXT::NUMERIC"));
|
||||||
|
assert!(super::UPDATE_TRANSACTION_CONFLICT_SQL.contains("incoming_variant_id = $4::TEXT::NUMERIC"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|||||||
54
crates/ksp-store-postgres-lib/unit_tests/schema_v003.rs
Normal file
54
crates/ksp-store-postgres-lib/unit_tests/schema_v003.rs
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
// file: crates/ksp-store-postgres-lib/unit_tests/schema_v003.rs
|
||||||
|
// version: 2
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_resource_inventory_is_exact_and_ordered() {
|
||||||
|
let resources = crate::V003_RESOURCES;
|
||||||
|
assert_eq!(resources.len(), 25);
|
||||||
|
let ids = resources.iter().map(|resource| return resource.id).collect::<std::vec::Vec<_>>();
|
||||||
|
assert_eq!(ids.iter().filter(|id| return id.starts_with("tables/")).count(), 4);
|
||||||
|
assert_eq!(ids.iter().filter(|id| return id.starts_with("constraints/")).count(), 18);
|
||||||
|
assert_eq!(ids.iter().filter(|id| return id.starts_with("indexes/")).count(), 3);
|
||||||
|
assert!(ids[..4].iter().all(|id| return id.starts_with("tables/")));
|
||||||
|
assert!(ids[4..22].iter().all(|id| return id.starts_with("constraints/")));
|
||||||
|
assert!(ids[22..].iter().all(|id| return id.starts_with("indexes/")));
|
||||||
|
let mut unique = std::collections::BTreeSet::<&str>::new();
|
||||||
|
for id in ids {
|
||||||
|
assert!(unique.insert(id), "duplicate V003 resource id: {id}");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_schema_is_additive_payload_bounded_and_hash_non_unique() {
|
||||||
|
let sql = crate::V003_RESOURCES.iter().map(|resource| return resource.sql).collect::<std::vec::Vec<_>>().concat();
|
||||||
|
for required in [
|
||||||
|
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_variants",
|
||||||
|
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_canonical_selectors",
|
||||||
|
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_observation_variants",
|
||||||
|
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_conflicts",
|
||||||
|
"PRIMARY KEY (transaction_signature, variant_id)",
|
||||||
|
"FOREIGN KEY (transaction_signature) REFERENCES ksp_raw_transactions(signature)",
|
||||||
|
"FOREIGN KEY (observation_key) REFERENCES ksp_raw_transaction_observations(observation_key)",
|
||||||
|
"octet_length(payload) <= 16777216",
|
||||||
|
"origin_kind = 'native' OR origin_kind = 'synthetic'",
|
||||||
|
"latest_relation = 'conflict' OR latest_relation = 'incomparable'",
|
||||||
|
"latest_reason_code = 'content_hash_collision'",
|
||||||
|
"latest_reason_code = 'unsupported_canonical_difference'",
|
||||||
|
] {
|
||||||
|
assert!(sql.contains(required), "missing V003 schema invariant: {required}");
|
||||||
|
}
|
||||||
|
for forbidden in ["UNIQUE (content_hash", "UNIQUE(content_hash", "BIGSERIAL", "INSERT INTO", "UPDATE ksp_raw_transactions"] {
|
||||||
|
assert!(!sql.contains(forbidden), "forbidden V003 migration behavior: {forbidden}");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pre_003_v003_observation_mapping_does_not_fabricate_legacy_rows() {
|
||||||
|
let sql = crate::V003_RESOURCES.iter().map(|resource| return resource.sql).collect::<std::vec::Vec<_>>().concat();
|
||||||
|
assert!(sql.contains("ksp_raw_transaction_observation_variants"));
|
||||||
|
assert!(!sql.contains("SELECT observation_key FROM ksp_raw_transaction_observations"));
|
||||||
|
assert!(!sql.contains("INSERT INTO ksp_raw_transaction_observation_variants"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/README.md -->
|
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/README.md -->
|
||||||
<!-- version: 17 -->
|
<!-- version: 19 -->
|
||||||
|
|
||||||
# ksp-worker-raw-transaction-ingest-lib
|
# ksp-worker-raw-transaction-ingest-lib
|
||||||
|
|
||||||
@@ -187,7 +187,7 @@ source hydratante active => quota pending >= 1 et quota in-flight >= 1
|
|||||||
|
|
||||||
Pour garantir simultanément ces bornes et l'absence de starvation structurelle, le démarrage échoue avant spawn si le nombre de sources hydratantes dépasse `admission_queue_capacity` ou `persistence_concurrency`. Le registre des hydrations transactionnelles protège en plus l'ouverture effective des `getTransaction`; Yellowstone Block reste borné par sa partition déterministe et son `JoinSet` possédé.
|
Pour garantir simultanément ces bornes et l'absence de starvation structurelle, le démarrage échoue avant spawn si le nombre de sources hydratantes dépasse `admission_queue_capacity` ou `persistence_concurrency`. Le registre des hydrations transactionnelles protège en plus l'ouverture effective des `getTransaction`; Yellowstone Block reste borné par sa partition déterministe et son `JoinSet` possédé.
|
||||||
|
|
||||||
Les signaux partageant le même `(network, signature, commitment)` sont coalescés cross-source avant le fan-out HTTP. La publication du résultat partagé notifie les followers sous le verrou de registry avant de retirer la clé : une nouvelle génération de leader ne peut donc pas s'intercaler entre retrait et notification. Après canonicalisation, une cache run-local bornée sérialise les acquisitions de même `(network, signature)` : la première passe par l'écriture atomique entity + observation, les suivantes de contenu canonique identique ajoutent uniquement leur observation déterministe. Le Store conserve le guard durable final. En `0.3.15`, il accepte aussi un entrant strictement moins complet lorsque l'unique divergence prouvée est `meta.logMessages` avec exactement un marqueur `Log truncated` après un préfixe identique alors que le canonique stocké n'est pas tronqué ; le canonique complet reste inchangé et l'observation est conservée. Toute autre divergence, ainsi que le sens tronqué -> complet, reste un `content_conflict` terminal. Aucune majorité, préférence provider ou overwrite n'est appliqué.
|
Les signaux partageant le même `(network, signature, commitment)` sont coalescés cross-source avant le fan-out HTTP. La publication du résultat partagé notifie les followers sous le verrou de registry avant de retirer la clé : une nouvelle génération de leader ne peut donc pas s'intercaler entre retrait et notification. Après canonicalisation, une cache run-local bornée sérialise uniquement les acquisitions de même `(network, signature)` ; elle ne décide jamais du contenu. Chaque acquisition repasse par l'écriture atomique du Store, qui reste l'autorité durable finale. Le comparateur partagé accepte un entrant strictement moins complet lorsque l'unique divergence prouvée est `meta.logMessages` tronqué, promeut atomiquement un entrant prouvé plus complet, et conserve toute divergence conflictuelle/incomparable comme variante durable avec un conflict case ouvert. Aucune majorité, préférence provider ou overwrite n'est appliqué.
|
||||||
|
|
||||||
Les retries/reroutages HTTP appartiennent à `ksp-onchain-transport-lib`. Le Worker ne possède pas une seconde boucle de retry autour de `getTransaction`.
|
Les retries/reroutages HTTP appartiennent à `ksp-onchain-transport-lib`. Le Worker ne possède pas une seconde boucle de retry autour de `getTransaction`.
|
||||||
|
|
||||||
@@ -283,11 +283,13 @@ entity skipped purged
|
|||||||
observation inserted
|
observation inserted
|
||||||
observation already present
|
observation already present
|
||||||
observation not recorded for purged entity
|
observation not recorded for purged entity
|
||||||
content conflict
|
variant inserted canonical / exact / compatible less complete
|
||||||
|
variant promoted compatible more complete
|
||||||
|
variant quarantined conflict
|
||||||
store failure
|
store failure
|
||||||
```
|
```
|
||||||
|
|
||||||
Un content conflict réel reste terminal en `0.3.15` et n'est jamais converti en succès idempotent. L'exception `logMessages` tronqués décrite ci-dessus est classée avant le conflit comme observation compatible moins complète ; elle ne remplace pas le canonique et n'incrémente pas le terminal `content_conflict`. La conservation de variantes conflictuelles et leur résolution sans arrêt du Worker appartiennent à `0.3.16`.
|
Depuis `0.3.16`, un conflit de contenu classifié par le Store n'est plus une faute terminale : la variante entrante est conservée, le conflict case durable est ouvert ou mis à jour idempotemment et l'outcome `QuarantinedConflict` maintient le Worker `Running` avec une health `Degraded`. Le hardening final prouve qu'une identité suivante continue d'atteindre le Store après cette quarantaine. Un ancien backend qui renvoie encore `ERROR_CODE_RAW_CONFLICT` reste traité comme erreur terminale de compatibilité. La résolution explicite du conflict case et le retry Store complet appartiennent à `0.3.17`.
|
||||||
|
|
||||||
## Snapshots et erreurs
|
## Snapshots et erreurs
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md -->
|
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md -->
|
||||||
<!-- version: 15 -->
|
<!-- version: 17 -->
|
||||||
|
|
||||||
# Utilisation de ksp-worker-raw-transaction-ingest-lib
|
# Utilisation de ksp-worker-raw-transaction-ingest-lib
|
||||||
|
|
||||||
@@ -266,7 +266,7 @@ somme de leurs tâches d'hydration actives <= persistence_concurrency
|
|||||||
|
|
||||||
Ces contraintes garantissent au moins une part à chaque source reference-bearing sans introduire de scheduler pondéré. Si les settings ne permettent pas cette répartition, le démarrage échoue avant spawn avec `runtime_invalid`; le caller doit augmenter la capacité concernée ou réduire le nombre de sources nécessitant une hydration.
|
Ces contraintes garantissent au moins une part à chaque source reference-bearing sans introduire de scheduler pondéré. Si les settings ne permettent pas cette répartition, le démarrage échoue avant spawn avec `runtime_invalid`; le caller doit augmenter la capacité concernée ou réduire le nombre de sources nécessitant une hydration.
|
||||||
|
|
||||||
Pour une même identité `(network, signature)`, le Worker ne choisit ni majorité ni provider préféré et n'écrase pas un contenu divergent ; le Store reste l'autorité durable finale. En `0.3.15`, un canonique complet peut accepter comme observation supplémentaire un entrant identique hors `meta.logMessages` lorsque celui-ci contient exactement un marqueur `Log truncated` après un préfixe identique. Le canonique n'est pas réécrit. Le sens tronqué -> complet et toute autre divergence non prouvée restent des `content_conflict` terminaux ; leur stockage en variantes et leur résolution sont réservés à `0.3.16`.
|
Pour une même identité `(network, signature)`, le Worker ne choisit ni majorité ni provider préféré et n'écrase pas un contenu divergent ; le Store reste l'autorité durable finale. Le cache run-local ne fait que sérialiser cette identité et chaque acquisition atteint le Store. Un entrant prouvé moins complet reste une observation compatible, un entrant prouvé plus complet peut devenir canonique atomiquement, et une divergence conflictuelle/incomparable est conservée comme variante avec un conflict case durable. L'outcome `QuarantinedConflict` n'arrête pas le Worker : il reste `Running`, sa health devient `Degraded` et les identités suivantes continuent à être persistées. Le retry Store/backoff configurable n'est pas encore fourni par `0.3.16`.
|
||||||
|
|
||||||
## Observer le snapshot concret
|
## Observer le snapshot concret
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/persistence.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/persistence.rs
|
||||||
// version: 4
|
// version: 5
|
||||||
|
|
||||||
/// Canonical entity disposition produced by one Worker Store persistence attempt.
|
/// Canonical entity disposition produced by one Worker Store persistence attempt.
|
||||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||||
@@ -28,6 +28,7 @@ pub(crate) enum RawTransactionIngestObservationPersistence {
|
|||||||
pub(crate) struct RawTransactionIngestPersistenceOutcome {
|
pub(crate) struct RawTransactionIngestPersistenceOutcome {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence,
|
entity: crate::RawTransactionIngestEntityPersistence,
|
||||||
observation: crate::RawTransactionIngestObservationPersistence,
|
observation: crate::RawTransactionIngestObservationPersistence,
|
||||||
|
quarantined_conflict: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl crate::RawTransactionIngestPersistenceOutcome {
|
impl crate::RawTransactionIngestPersistenceOutcome {
|
||||||
@@ -42,6 +43,12 @@ impl crate::RawTransactionIngestPersistenceOutcome {
|
|||||||
pub(crate) const fn observation(self) -> crate::RawTransactionIngestObservationPersistence {
|
pub(crate) const fn observation(self) -> crate::RawTransactionIngestObservationPersistence {
|
||||||
return self.observation;
|
return self.observation;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Reports whether Store durably quarantined divergent or incomparable content without failing the acquisition.
|
||||||
|
#[must_use]
|
||||||
|
pub(crate) const fn quarantined_conflict(self) -> bool {
|
||||||
|
return self.quarantined_conflict;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Private backend-neutral Store persistence port used by the Worker and deterministic tests.
|
/// Private backend-neutral Store persistence port used by the Worker and deterministic tests.
|
||||||
@@ -56,12 +63,6 @@ pub(crate) trait RawTransactionIngestPersistencePort: std::marker::Send + std::m
|
|||||||
observation: ksp_store_lib::RawTransactionObservation,
|
observation: ksp_store_lib::RawTransactionObservation,
|
||||||
mode: ksp_store_lib::RawTransactionAcquisitionMode,
|
mode: ksp_store_lib::RawTransactionAcquisitionMode,
|
||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawAcquisitionWriteOutcome>>;
|
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawAcquisitionWriteOutcome>>;
|
||||||
|
|
||||||
/// Records one additional observation for an already durable canonical RAW transaction.
|
|
||||||
fn record_observation<'a>(
|
|
||||||
&'a self,
|
|
||||||
observation: ksp_store_lib::RawTransactionObservation,
|
|
||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawObservationWriteOutcome>>;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl crate::RawTransactionIngestPersistencePort for ksp_store_lib::Store {
|
impl crate::RawTransactionIngestPersistencePort for ksp_store_lib::Store {
|
||||||
@@ -77,13 +78,6 @@ impl crate::RawTransactionIngestPersistencePort for ksp_store_lib::Store {
|
|||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawAcquisitionWriteOutcome>> {
|
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawAcquisitionWriteOutcome>> {
|
||||||
return ksp_store_lib::RawTransactionWrite::persist_raw_transaction_acquisition(self, transaction, observation, mode);
|
return ksp_store_lib::RawTransactionWrite::persist_raw_transaction_acquisition(self, transaction, observation, mode);
|
||||||
}
|
}
|
||||||
|
|
||||||
fn record_observation<'a>(
|
|
||||||
&'a self,
|
|
||||||
observation: ksp_store_lib::RawTransactionObservation,
|
|
||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawObservationWriteOutcome>> {
|
|
||||||
return ksp_store_lib::RawTransactionObservationWrite::record_raw_transaction_observation(self, observation);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Clone, Eq, Ord, PartialEq, PartialOrd)]
|
#[derive(Clone, Eq, Ord, PartialEq, PartialOrd)]
|
||||||
@@ -92,24 +86,10 @@ struct RawTransactionIngestPersistenceKey {
|
|||||||
signature: ksp_store_lib::RawTransactionSignature,
|
signature: ksp_store_lib::RawTransactionSignature,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Clone, Eq, PartialEq)]
|
|
||||||
struct RawTransactionIngestCanonicalState {
|
|
||||||
block_time: std::option::Option<ksp_store_lib::RawTimestamp>,
|
|
||||||
content_hash: ksp_store_lib::RawContentHash,
|
|
||||||
format_id: ksp_store_lib::RawFormatId,
|
|
||||||
format_version: u32,
|
|
||||||
slot: u64,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Private bounded run-local cache serializing repeated canonical identities before Store writes.
|
/// Private bounded run-local cache serializing repeated canonical identities before Store writes.
|
||||||
pub(crate) struct RawTransactionIngestPersistenceConvergence {
|
pub(crate) struct RawTransactionIngestPersistenceConvergence {
|
||||||
max_entries: usize,
|
max_entries: usize,
|
||||||
entries: std::sync::Mutex<
|
entries: std::sync::Mutex<std::collections::BTreeMap<RawTransactionIngestPersistenceKey, std::sync::Arc<tokio::sync::Mutex<()>>>>,
|
||||||
std::collections::BTreeMap<
|
|
||||||
RawTransactionIngestPersistenceKey,
|
|
||||||
std::sync::Arc<tokio::sync::Mutex<std::option::Option<RawTransactionIngestCanonicalState>>>,
|
|
||||||
>,
|
|
||||||
>,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl crate::RawTransactionIngestPersistenceConvergence {
|
impl crate::RawTransactionIngestPersistenceConvergence {
|
||||||
@@ -119,10 +99,7 @@ impl crate::RawTransactionIngestPersistenceConvergence {
|
|||||||
return Self { max_entries, entries: std::sync::Mutex::new(std::collections::BTreeMap::new()) };
|
return Self { max_entries, entries: std::sync::Mutex::new(std::collections::BTreeMap::new()) };
|
||||||
}
|
}
|
||||||
|
|
||||||
fn entry(
|
fn entry(&self, key: RawTransactionIngestPersistenceKey) -> std::option::Option<std::sync::Arc<tokio::sync::Mutex<()>>> {
|
||||||
&self,
|
|
||||||
key: RawTransactionIngestPersistenceKey,
|
|
||||||
) -> std::option::Option<std::sync::Arc<tokio::sync::Mutex<std::option::Option<RawTransactionIngestCanonicalState>>>> {
|
|
||||||
let mut entries = match self.entries.lock() {
|
let mut entries = match self.entries.lock() {
|
||||||
std::result::Result::Ok(value) => value,
|
std::result::Result::Ok(value) => value,
|
||||||
std::result::Result::Err(poisoned) => poisoned.into_inner(),
|
std::result::Result::Err(poisoned) => poisoned.into_inner(),
|
||||||
@@ -144,7 +121,7 @@ impl crate::RawTransactionIngestPersistenceConvergence {
|
|||||||
if entries.len() >= self.max_entries {
|
if entries.len() >= self.max_entries {
|
||||||
return std::option::Option::None;
|
return std::option::Option::None;
|
||||||
}
|
}
|
||||||
let entry = std::sync::Arc::new(tokio::sync::Mutex::new(std::option::Option::None));
|
let entry = std::sync::Arc::new(tokio::sync::Mutex::new(()));
|
||||||
entries.insert(key, std::sync::Arc::clone(&entry));
|
entries.insert(key, std::sync::Arc::clone(&entry));
|
||||||
return std::option::Option::Some(entry);
|
return std::option::Option::Some(entry);
|
||||||
}
|
}
|
||||||
@@ -189,44 +166,14 @@ where
|
|||||||
if acquisition.observation().transaction() != &reference {
|
if acquisition.observation().transaction() != &reference {
|
||||||
return std::result::Result::Err(crate::runtime_error("persistence.acquisition_reference_mismatch"));
|
return std::result::Result::Err(crate::runtime_error("persistence.acquisition_reference_mismatch"));
|
||||||
}
|
}
|
||||||
let canonical_state = canonical_state(acquisition.transaction());
|
|
||||||
let key = RawTransactionIngestPersistenceKey { network: reference.network().clone(), signature: reference.signature() };
|
let key = RawTransactionIngestPersistenceKey { network: reference.network().clone(), signature: reference.signature() };
|
||||||
let entry = convergence.entry(key);
|
let entry = convergence.entry(key);
|
||||||
let entry = match entry {
|
let entry = match entry {
|
||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return crate::persist_raw_transaction_ingest_acquisition(port, acquisition).await,
|
std::option::Option::None => return crate::persist_raw_transaction_ingest_acquisition(port, acquisition).await,
|
||||||
};
|
};
|
||||||
let mut known_state = entry.lock().await;
|
let _identity_guard = entry.lock().await;
|
||||||
if let std::option::Option::Some(known_state_value) = known_state.as_ref() {
|
return crate::persist_raw_transaction_ingest_acquisition(port, acquisition).await;
|
||||||
if known_state_value != &canonical_state {
|
|
||||||
return std::result::Result::Err(crate::content_conflict_error());
|
|
||||||
}
|
|
||||||
let (_transaction, observation) = acquisition.into_parts();
|
|
||||||
let result = port.record_observation(observation).await;
|
|
||||||
return match result {
|
|
||||||
std::result::Result::Ok(outcome) => map_additional_observation_outcome(outcome),
|
|
||||||
std::result::Result::Err(error) => map_store_error(error),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
let outcome = crate::persist_raw_transaction_ingest_acquisition(port, acquisition).await;
|
|
||||||
let outcome = match outcome {
|
|
||||||
std::result::Result::Ok(value) => value,
|
|
||||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
|
||||||
};
|
|
||||||
if outcome.entity() != crate::RawTransactionIngestEntityPersistence::SkippedPurged {
|
|
||||||
*known_state = std::option::Option::Some(canonical_state);
|
|
||||||
}
|
|
||||||
return std::result::Result::Ok(outcome);
|
|
||||||
}
|
|
||||||
|
|
||||||
fn canonical_state(transaction: &ksp_store_lib::RawTransaction) -> RawTransactionIngestCanonicalState {
|
|
||||||
return RawTransactionIngestCanonicalState {
|
|
||||||
block_time: transaction.block_time(),
|
|
||||||
content_hash: transaction.payload().content_hash(),
|
|
||||||
format_id: transaction.payload().format_id().clone(),
|
|
||||||
format_version: transaction.payload().format_version(),
|
|
||||||
slot: transaction.slot(),
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn map_store_error(error: ksp_core_lib::Error) -> ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome> {
|
fn map_store_error(error: ksp_core_lib::Error) -> ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome> {
|
||||||
@@ -239,49 +186,74 @@ fn map_store_error(error: ksp_core_lib::Error) -> ksp_core_lib::Result<crate::Ra
|
|||||||
fn map_store_outcome(outcome: ksp_store_lib::RawAcquisitionWriteOutcome) -> ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome> {
|
fn map_store_outcome(outcome: ksp_store_lib::RawAcquisitionWriteOutcome) -> ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome> {
|
||||||
let entity = outcome.entity();
|
let entity = outcome.entity();
|
||||||
let observation = outcome.observation();
|
let observation = outcome.observation();
|
||||||
|
let quarantined_conflict = match validate_transaction_variant_outcome(outcome.transaction_variant(), entity, observation) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
if entity == ksp_store_lib::RawEntityWriteOutcome::Inserted && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
if entity == ksp_store_lib::RawEntityWriteOutcome::Inserted && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
||||||
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::Inserted,
|
entity: crate::RawTransactionIngestEntityPersistence::Inserted,
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::Inserted,
|
observation: crate::RawTransactionIngestObservationPersistence::Inserted,
|
||||||
|
quarantined_conflict,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted {
|
||||||
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::Inserted,
|
observation: crate::RawTransactionIngestObservationPersistence::Inserted,
|
||||||
|
quarantined_conflict,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && observation == ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent {
|
if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && observation == ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent {
|
||||||
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::AlreadyPresent,
|
observation: crate::RawTransactionIngestObservationPersistence::AlreadyPresent,
|
||||||
|
quarantined_conflict,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
if entity == ksp_store_lib::RawEntityWriteOutcome::SkippedPurged && observation == ksp_store_lib::RawObservationWriteOutcome::NotRecorded {
|
if entity == ksp_store_lib::RawEntityWriteOutcome::SkippedPurged && observation == ksp_store_lib::RawObservationWriteOutcome::NotRecorded {
|
||||||
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
return std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::SkippedPurged,
|
entity: crate::RawTransactionIngestEntityPersistence::SkippedPurged,
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::NotRecorded,
|
observation: crate::RawTransactionIngestObservationPersistence::NotRecorded,
|
||||||
|
quarantined_conflict,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
return std::result::Result::Err(crate::runtime_error("persistence.store_outcome_invalid"));
|
return std::result::Result::Err(crate::runtime_error("persistence.store_outcome_invalid"));
|
||||||
}
|
}
|
||||||
|
|
||||||
fn map_additional_observation_outcome(
|
fn validate_transaction_variant_outcome(
|
||||||
outcome: ksp_store_lib::RawObservationWriteOutcome,
|
transaction_variant: std::option::Option<ksp_store_lib::RawTransactionVariantWriteOutcome>,
|
||||||
) -> ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome> {
|
entity: ksp_store_lib::RawEntityWriteOutcome,
|
||||||
return match outcome {
|
observation: ksp_store_lib::RawObservationWriteOutcome,
|
||||||
ksp_store_lib::RawObservationWriteOutcome::Inserted => std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
) -> ksp_core_lib::Result<bool> {
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
let recorded_observation =
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::Inserted,
|
matches!(observation, ksp_store_lib::RawObservationWriteOutcome::Inserted | ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent);
|
||||||
}),
|
return match transaction_variant {
|
||||||
ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent => std::result::Result::Ok(crate::RawTransactionIngestPersistenceOutcome {
|
std::option::Option::None => std::result::Result::Ok(false),
|
||||||
entity: crate::RawTransactionIngestEntityPersistence::AlreadyPresent,
|
std::option::Option::Some(ksp_store_lib::RawTransactionVariantWriteOutcome::InsertedCanonical)
|
||||||
observation: crate::RawTransactionIngestObservationPersistence::AlreadyPresent,
|
if entity == ksp_store_lib::RawEntityWriteOutcome::Inserted && observation == ksp_store_lib::RawObservationWriteOutcome::Inserted =>
|
||||||
}),
|
{
|
||||||
ksp_store_lib::RawObservationWriteOutcome::NotRecorded => {
|
std::result::Result::Ok(false)
|
||||||
std::result::Result::Err(crate::runtime_error("persistence.additional_observation_not_recorded"))
|
|
||||||
},
|
},
|
||||||
_ => std::result::Result::Err(crate::runtime_error("persistence.additional_observation_outcome_invalid")),
|
std::option::Option::Some(
|
||||||
|
ksp_store_lib::RawTransactionVariantWriteOutcome::ObservedExact
|
||||||
|
| ksp_store_lib::RawTransactionVariantWriteOutcome::ObservedCompatibleLessComplete
|
||||||
|
| ksp_store_lib::RawTransactionVariantWriteOutcome::PromotedCompatibleMoreComplete,
|
||||||
|
) if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && recorded_observation => std::result::Result::Ok(false),
|
||||||
|
std::option::Option::Some(ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict)
|
||||||
|
if entity == ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent && recorded_observation =>
|
||||||
|
{
|
||||||
|
std::result::Result::Ok(true)
|
||||||
|
},
|
||||||
|
std::option::Option::Some(ksp_store_lib::RawTransactionVariantWriteOutcome::SkippedPurged)
|
||||||
|
if entity == ksp_store_lib::RawEntityWriteOutcome::SkippedPurged && observation == ksp_store_lib::RawObservationWriteOutcome::NotRecorded =>
|
||||||
|
{
|
||||||
|
std::result::Result::Ok(false)
|
||||||
|
},
|
||||||
|
std::option::Option::Some(ksp_store_lib::RawTransactionVariantWriteOutcome::Rehydrated) => {
|
||||||
|
std::result::Result::Err(crate::runtime_error("persistence.store_variant_rehydrated_in_normal_mode"))
|
||||||
|
},
|
||||||
|
_ => std::result::Result::Err(crate::runtime_error("persistence.store_variant_outcome_invalid")),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/snapshot.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/snapshot.rs
|
||||||
// version: 10
|
// version: 11
|
||||||
|
|
||||||
/// Runtime-neutral boxed future resolving to one newer concrete RAW transaction ingest Worker snapshot.
|
/// Runtime-neutral boxed future resolving to one newer concrete RAW transaction ingest Worker snapshot.
|
||||||
pub type RawTransactionIngestSnapshotFuture<'a> =
|
pub type RawTransactionIngestSnapshotFuture<'a> =
|
||||||
@@ -1132,6 +1132,13 @@ impl crate::RawTransactionIngestSnapshotPublisher {
|
|||||||
let mut entity_skipped_purged_total = self.snapshot.entity_skipped_purged_total;
|
let mut entity_skipped_purged_total = self.snapshot.entity_skipped_purged_total;
|
||||||
let mut observation_inserted_total = self.snapshot.observation_inserted_total;
|
let mut observation_inserted_total = self.snapshot.observation_inserted_total;
|
||||||
let mut observation_already_present_total = self.snapshot.observation_already_present_total;
|
let mut observation_already_present_total = self.snapshot.observation_already_present_total;
|
||||||
|
let mut content_conflict_total = self.snapshot.content_conflict_total;
|
||||||
|
if outcome.quarantined_conflict() {
|
||||||
|
content_conflict_total = match checked_counter(content_conflict_total, "content_conflict_total") {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||||
|
};
|
||||||
|
}
|
||||||
match outcome.entity() {
|
match outcome.entity() {
|
||||||
crate::RawTransactionIngestEntityPersistence::Inserted => {
|
crate::RawTransactionIngestEntityPersistence::Inserted => {
|
||||||
entity_inserted_total = match checked_counter(entity_inserted_total, "entity_inserted_total") {
|
entity_inserted_total = match checked_counter(entity_inserted_total, "entity_inserted_total") {
|
||||||
@@ -1173,6 +1180,7 @@ impl crate::RawTransactionIngestSnapshotPublisher {
|
|||||||
self.snapshot.entity_skipped_purged_total = entity_skipped_purged_total;
|
self.snapshot.entity_skipped_purged_total = entity_skipped_purged_total;
|
||||||
self.snapshot.observation_inserted_total = observation_inserted_total;
|
self.snapshot.observation_inserted_total = observation_inserted_total;
|
||||||
self.snapshot.observation_already_present_total = observation_already_present_total;
|
self.snapshot.observation_already_present_total = observation_already_present_total;
|
||||||
|
self.snapshot.content_conflict_total = content_conflict_total;
|
||||||
return self.publish(state, admission_queue_depth, in_flight_persistence);
|
return self.publish(state, admission_queue_depth, in_flight_persistence);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1279,6 +1287,9 @@ fn health_for_state(
|
|||||||
return ksp_worker_api::WorkerHealth::Unhealthy;
|
return ksp_worker_api::WorkerHealth::Unhealthy;
|
||||||
}
|
}
|
||||||
if source_total > 0 && snapshot.source_active == snapshot.source_total {
|
if source_total > 0 && snapshot.source_active == snapshot.source_total {
|
||||||
|
if snapshot.content_conflict_total > 0 {
|
||||||
|
return ksp_worker_api::WorkerHealth::Degraded;
|
||||||
|
}
|
||||||
return ksp_worker_api::WorkerHealth::Healthy;
|
return ksp_worker_api::WorkerHealth::Healthy;
|
||||||
}
|
}
|
||||||
if source_total > 0
|
if source_total > 0
|
||||||
@@ -1297,6 +1308,7 @@ fn health_for_state(
|
|||||||
ksp_worker_api::WorkerState::Running if snapshot.source_total > 0 && snapshot.source_active < snapshot.source_total => {
|
ksp_worker_api::WorkerState::Running if snapshot.source_total > 0 && snapshot.source_active < snapshot.source_total => {
|
||||||
ksp_worker_api::WorkerHealth::Degraded
|
ksp_worker_api::WorkerHealth::Degraded
|
||||||
},
|
},
|
||||||
|
ksp_worker_api::WorkerState::Running if snapshot.content_conflict_total > 0 => ksp_worker_api::WorkerHealth::Degraded,
|
||||||
ksp_worker_api::WorkerState::Running => ksp_worker_api::WorkerHealth::Healthy,
|
ksp_worker_api::WorkerState::Running => ksp_worker_api::WorkerHealth::Healthy,
|
||||||
ksp_worker_api::WorkerState::Stopping | ksp_worker_api::WorkerState::Stopped => previous,
|
ksp_worker_api::WorkerState::Stopping | ksp_worker_api::WorkerState::Stopped => previous,
|
||||||
ksp_worker_api::WorkerState::Faulted(_) => ksp_worker_api::WorkerHealth::Unhealthy,
|
ksp_worker_api::WorkerState::Faulted(_) => ksp_worker_api::WorkerHealth::Unhealthy,
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/cross_layer_completeness.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/cross_layer_completeness.rs
|
||||||
// version: 3
|
// version: 5
|
||||||
|
|
||||||
//! Cross-layer completeness and security canaries through `0.3.14-pre.013`.
|
//! Cross-layer completeness and security canaries through `0.3.14-pre.013`.
|
||||||
|
|
||||||
@@ -215,7 +215,12 @@ fn v0_3_13_pre_012_all_five_live_source_families_share_one_worker_common_raw_sto
|
|||||||
] {
|
] {
|
||||||
assert!(worker_resources.contains(required), "pre.012 Worker source/pipeline contract missing: {required}");
|
assert!(worker_resources.contains(required), "pre.012 Worker source/pipeline contract missing: {required}");
|
||||||
}
|
}
|
||||||
for required in ["persist_raw_transaction_ingest_converged_acquisition", "RawTransactionIngestPersistenceConvergence", "record_observation"] {
|
for required in [
|
||||||
|
"persist_raw_transaction_ingest_converged_acquisition",
|
||||||
|
"RawTransactionIngestPersistenceConvergence",
|
||||||
|
"let _identity_guard = entry.lock().await",
|
||||||
|
"persist_raw_transaction_ingest_acquisition(port, acquisition).await",
|
||||||
|
] {
|
||||||
assert!(worker_persistence.contains(required), "pre.012 Worker persistence contract missing: {required}");
|
assert!(worker_persistence.contains(required), "pre.012 Worker persistence contract missing: {required}");
|
||||||
}
|
}
|
||||||
for required in ["SolanaStandardWsSession", "HeliusLaserStreamWsSession"] {
|
for required in ["SolanaStandardWsSession", "HeliusLaserStreamWsSession"] {
|
||||||
@@ -247,6 +252,7 @@ fn v0_3_13_pre_012_legacy_v0_v1_is_proven_from_transport_through_worker_common_r
|
|||||||
let worker_persistence = include_str!("../src/persistence.rs");
|
let worker_persistence = include_str!("../src/persistence.rs");
|
||||||
let common_raw_wire = include_str!("../../ksp-raw-transaction-lib/unit_tests/wire.rs");
|
let common_raw_wire = include_str!("../../ksp-raw-transaction-lib/unit_tests/wire.rs");
|
||||||
let common_raw_canonical = include_str!("../../ksp-raw-transaction-lib/unit_tests/canonical.rs");
|
let common_raw_canonical = include_str!("../../ksp-raw-transaction-lib/unit_tests/canonical.rs");
|
||||||
|
let common_raw_canonical_source = include_str!("../../ksp-raw-transaction-lib/src/canonical.rs");
|
||||||
let store_api = include_str!("../../ksp-store-api/tests/public_api.rs");
|
let store_api = include_str!("../../ksp-store-api/tests/public_api.rs");
|
||||||
let store = include_str!("../../ksp-store-lib/src/store.rs");
|
let store = include_str!("../../ksp-store-lib/src/store.rs");
|
||||||
assert!(transport_grpc.contains("yellowstone_transaction_update_decodes_current_storage_wire_including_v1_config_and_meta"));
|
assert!(transport_grpc.contains("yellowstone_transaction_update_decodes_current_storage_wire_including_v1_config_and_meta"));
|
||||||
@@ -271,9 +277,16 @@ fn v0_3_13_pre_012_legacy_v0_v1_is_proven_from_transport_through_worker_common_r
|
|||||||
assert!(common_raw_wire.contains(required), "pre.012 Common RAW V1 proof missing: {required}");
|
assert!(common_raw_wire.contains(required), "pre.012 Common RAW V1 proof missing: {required}");
|
||||||
}
|
}
|
||||||
assert!(common_raw_canonical.contains("pre_002_canonical_raw_v1_golden_bytes_and_hash_are_exact"));
|
assert!(common_raw_canonical.contains("pre_002_canonical_raw_v1_golden_bytes_and_hash_are_exact"));
|
||||||
for required in ["block_time", "content_hash", "format_id", "format_version", "slot"] {
|
for required in [
|
||||||
assert!(worker_persistence.contains(required), "pre.012 canonical Store identity field missing: {required}");
|
"RawPayload::try_new(",
|
||||||
|
"RAW_TRANSACTION_FORMAT_VERSION",
|
||||||
|
"RawContentHash::new(hash)",
|
||||||
|
"RawTransaction::new(reference, material.slot, block_time, payload)",
|
||||||
|
] {
|
||||||
|
assert!(common_raw_canonical_source.contains(required), "pre.012 canonical Common RAW field assembly missing: {required}");
|
||||||
}
|
}
|
||||||
|
assert!(worker_persistence.contains("let (transaction, observation) = acquisition.into_parts();"));
|
||||||
|
assert!(worker_persistence.contains("port.persist_acquisition(transaction, observation, ksp_store_lib::RawTransactionAcquisitionMode::Normal).await"));
|
||||||
assert!(store_api.contains("public_pre_003_raw_transaction_and_observation_are_constructible_from_crate_root"));
|
assert!(store_api.contains("public_pre_003_raw_transaction_and_observation_are_constructible_from_crate_root"));
|
||||||
assert!(store_api.contains("RawPayload::try_new(format, 1"));
|
assert!(store_api.contains("RawPayload::try_new(format, 1"));
|
||||||
assert!(store.contains("persist_raw_transaction_acquisition"));
|
assert!(store.contains("persist_raw_transaction_acquisition"));
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
|
||||||
// version: 31
|
// version: 33
|
||||||
|
|
||||||
//! Dependency firewall canaries for the RAW transaction ingest Worker foundation.
|
//! Dependency firewall canaries for the RAW transaction ingest Worker foundation.
|
||||||
|
|
||||||
@@ -585,13 +585,16 @@ fn v0_3_13_pre_008_cross_source_convergence_reuses_store_and_transport_facades_o
|
|||||||
"RawTransactionIngestPersistenceConvergence",
|
"RawTransactionIngestPersistenceConvergence",
|
||||||
"persist_raw_transaction_ingest_converged_acquisition",
|
"persist_raw_transaction_ingest_converged_acquisition",
|
||||||
"RawTransactionIngestPersistencePort",
|
"RawTransactionIngestPersistencePort",
|
||||||
"record_raw_transaction_observation",
|
"let _identity_guard = entry.lock().await",
|
||||||
"RawTransactionObservationWrite",
|
"persist_raw_transaction_ingest_acquisition(port, acquisition).await",
|
||||||
"payload().content_hash()",
|
"RawTransactionVariantWriteOutcome::QuarantinedConflict",
|
||||||
"content_conflict_error()",
|
"content_conflict_error()",
|
||||||
] {
|
] {
|
||||||
assert!(persistence.contains(required) || root.contains(required), "required pre.008 persistence convergence contract missing: {required}");
|
assert!(persistence.contains(required) || root.contains(required), "required pre.008 persistence convergence contract missing: {required}");
|
||||||
}
|
}
|
||||||
|
for forbidden in ["record_raw_transaction_observation", "RawTransactionObservationWrite", "known_state_value", "canonical_state("] {
|
||||||
|
assert!(!persistence.contains(forbidden), "pre.007 Worker persistence retained stronger run-local arbitration: {forbidden}");
|
||||||
|
}
|
||||||
for required in [
|
for required in [
|
||||||
"RawTransactionIngestGlobalHydrationRegistry",
|
"RawTransactionIngestGlobalHydrationRegistry",
|
||||||
"fetch_hydration_shared",
|
"fetch_hydration_shared",
|
||||||
@@ -625,7 +628,7 @@ fn v0_3_13_pre_009_global_bounds_and_fairness_stay_inside_worker_facades() {
|
|||||||
"tokio::sync::Semaphore",
|
"tokio::sync::Semaphore",
|
||||||
"acquire_owned",
|
"acquire_owned",
|
||||||
"RawTransactionIngestHydrationLeaderGuard",
|
"RawTransactionIngestHydrationLeaderGuard",
|
||||||
"RawTransactionIngestCanonicalState",
|
"RawTransactionIngestPersistenceConvergence",
|
||||||
"content_conflict_error()",
|
"content_conflict_error()",
|
||||||
] {
|
] {
|
||||||
assert!(
|
assert!(
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/hardening.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/hardening.rs
|
||||||
// version: 40
|
// version: 41
|
||||||
|
|
||||||
//! External public, security, redaction and release-boundary hardening canaries through `v0.3.15-pre.004`.
|
//! External public, security, redaction and release-boundary hardening canaries through `v0.3.15-pre.004`.
|
||||||
|
|
||||||
@@ -764,11 +764,15 @@ fn v0_3_13_pre_008_cross_source_convergence_is_bounded_conflict_checked_and_priv
|
|||||||
"max_entries: usize",
|
"max_entries: usize",
|
||||||
"entries.len() >= self.max_entries",
|
"entries.len() >= self.max_entries",
|
||||||
"std::sync::Arc::strong_count(entry) == 1",
|
"std::sync::Arc::strong_count(entry) == 1",
|
||||||
"known_state_value != &canonical_state",
|
"let _identity_guard = entry.lock().await",
|
||||||
"record_observation(observation)",
|
"persist_raw_transaction_ingest_acquisition(port, acquisition).await",
|
||||||
"persistence.additional_observation_not_recorded",
|
"outcome.transaction_variant()",
|
||||||
|
"RawTransactionVariantWriteOutcome::QuarantinedConflict",
|
||||||
] {
|
] {
|
||||||
assert!(persistence.contains(required), "required pre.008 persistence hardening guard missing: {required}");
|
assert!(persistence.contains(required), "required persistence serialization/Store-authority guard missing: {required}");
|
||||||
|
}
|
||||||
|
for forbidden in ["known_state_value", "canonical_state(", "record_observation(", "RawTransactionObservationWrite"] {
|
||||||
|
assert!(!persistence.contains(forbidden), "run-local persistence cache still arbitrates before Store: {forbidden}");
|
||||||
}
|
}
|
||||||
for required in [
|
for required in [
|
||||||
"max_pending: usize",
|
"max_pending: usize",
|
||||||
@@ -803,9 +807,9 @@ fn v0_3_13_pre_009_duplicate_storm_disagreement_and_starvation_guards_are_explic
|
|||||||
"tokio::sync::Semaphore::new(max_in_flight)",
|
"tokio::sync::Semaphore::new(max_in_flight)",
|
||||||
"RawTransactionIngestHydrationLeaderGuard",
|
"RawTransactionIngestHydrationLeaderGuard",
|
||||||
"source.global_hydration_pending_saturated",
|
"source.global_hydration_pending_saturated",
|
||||||
"block_time: transaction.block_time()",
|
"network: ksp_store_lib::RawNetworkId",
|
||||||
"slot: transaction.slot()",
|
"signature: ksp_store_lib::RawTransactionSignature",
|
||||||
"content_hash: transaction.payload().content_hash()",
|
"let _identity_guard = entry.lock().await",
|
||||||
] {
|
] {
|
||||||
assert!(
|
assert!(
|
||||||
admission.contains(required) || persistence.contains(required) || resources.contains(required),
|
admission.contains(required) || persistence.contains(required) || resources.contains(required),
|
||||||
|
|||||||
@@ -1,9 +1,10 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/persistence.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/persistence.rs
|
||||||
// version: 3
|
// version: 4
|
||||||
|
|
||||||
#[derive(Clone, Copy)]
|
#[derive(Clone, Copy)]
|
||||||
enum PortResponse {
|
enum PortResponse {
|
||||||
Outcome(ksp_store_lib::RawEntityWriteOutcome, ksp_store_lib::RawObservationWriteOutcome),
|
Outcome(ksp_store_lib::RawEntityWriteOutcome, ksp_store_lib::RawObservationWriteOutcome),
|
||||||
|
VariantOutcome(ksp_store_lib::RawEntityWriteOutcome, ksp_store_lib::RawObservationWriteOutcome, ksp_store_lib::RawTransactionVariantWriteOutcome),
|
||||||
Conflict,
|
Conflict,
|
||||||
StoreFailure,
|
StoreFailure,
|
||||||
}
|
}
|
||||||
@@ -12,8 +13,6 @@ struct FakePersistencePort {
|
|||||||
calls: std::sync::atomic::AtomicUsize,
|
calls: std::sync::atomic::AtomicUsize,
|
||||||
network: ksp_store_lib::RawNetworkId,
|
network: ksp_store_lib::RawNetworkId,
|
||||||
normal_mode_seen: std::sync::atomic::AtomicBool,
|
normal_mode_seen: std::sync::atomic::AtomicBool,
|
||||||
observation_calls: std::sync::atomic::AtomicUsize,
|
|
||||||
observation_response: std::option::Option<ksp_store_lib::RawObservationWriteOutcome>,
|
|
||||||
response: PortResponse,
|
response: PortResponse,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -23,16 +22,9 @@ impl FakePersistencePort {
|
|||||||
calls: std::sync::atomic::AtomicUsize::new(0),
|
calls: std::sync::atomic::AtomicUsize::new(0),
|
||||||
network,
|
network,
|
||||||
normal_mode_seen: std::sync::atomic::AtomicBool::new(false),
|
normal_mode_seen: std::sync::atomic::AtomicBool::new(false),
|
||||||
observation_calls: std::sync::atomic::AtomicUsize::new(0),
|
|
||||||
observation_response: std::option::Option::None,
|
|
||||||
response,
|
response,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
fn with_observation_response(mut self, observation_response: ksp_store_lib::RawObservationWriteOutcome) -> Self {
|
|
||||||
self.observation_response = std::option::Option::Some(observation_response);
|
|
||||||
return self;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl crate::RawTransactionIngestPersistencePort for FakePersistencePort {
|
impl crate::RawTransactionIngestPersistencePort for FakePersistencePort {
|
||||||
@@ -52,28 +44,9 @@ impl crate::RawTransactionIngestPersistencePort for FakePersistencePort {
|
|||||||
return std::boxed::Box::pin(async move {
|
return std::boxed::Box::pin(async move {
|
||||||
return match response {
|
return match response {
|
||||||
PortResponse::Outcome(entity, observation) => std::result::Result::Ok(ksp_store_lib::RawAcquisitionWriteOutcome::new(entity, observation)),
|
PortResponse::Outcome(entity, observation) => std::result::Result::Ok(ksp_store_lib::RawAcquisitionWriteOutcome::new(entity, observation)),
|
||||||
PortResponse::Conflict => std::result::Result::Err(ksp_core_lib::Error::new(ksp_store_lib::ERROR_CODE_RAW_CONFLICT, "synthetic conflict")),
|
PortResponse::VariantOutcome(entity, observation, transaction_variant) => {
|
||||||
PortResponse::StoreFailure => std::result::Result::Err(ksp_core_lib::Error::new(
|
std::result::Result::Ok(ksp_store_lib::RawAcquisitionWriteOutcome::with_transaction_variant(entity, observation, transaction_variant))
|
||||||
ksp_core_lib::ErrorCode::new("synthetic_store", "write_failed"),
|
},
|
||||||
"synthetic remote failure text that must not escape",
|
|
||||||
)),
|
|
||||||
};
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
fn record_observation<'a>(
|
|
||||||
&'a self,
|
|
||||||
_observation: ksp_store_lib::RawTransactionObservation,
|
|
||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawObservationWriteOutcome>> {
|
|
||||||
self.observation_calls.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
|
||||||
let observation_response = self.observation_response;
|
|
||||||
let response = self.response;
|
|
||||||
return std::boxed::Box::pin(async move {
|
|
||||||
if let std::option::Option::Some(outcome) = observation_response {
|
|
||||||
return std::result::Result::Ok(outcome);
|
|
||||||
}
|
|
||||||
return match response {
|
|
||||||
PortResponse::Outcome(_, observation) => std::result::Result::Ok(observation),
|
|
||||||
PortResponse::Conflict => std::result::Result::Err(ksp_core_lib::Error::new(ksp_store_lib::ERROR_CODE_RAW_CONFLICT, "synthetic conflict")),
|
PortResponse::Conflict => std::result::Result::Err(ksp_core_lib::Error::new(ksp_store_lib::ERROR_CODE_RAW_CONFLICT, "synthetic conflict")),
|
||||||
PortResponse::StoreFailure => std::result::Result::Err(ksp_core_lib::Error::new(
|
PortResponse::StoreFailure => std::result::Result::Err(ksp_core_lib::Error::new(
|
||||||
ksp_core_lib::ErrorCode::new("synthetic_store", "write_failed"),
|
ksp_core_lib::ErrorCode::new("synthetic_store", "write_failed"),
|
||||||
@@ -203,7 +176,36 @@ async fn pre_007_normal_mode_classifies_new_idempotent_and_purged_outcomes() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "current_thread")]
|
#[tokio::test(flavor = "current_thread")]
|
||||||
async fn pre_007_content_conflict_maps_to_terminal_worker_code_without_remote_text() {
|
async fn v0_3_16_pre_007_quarantined_conflict_is_successful_and_explicit() {
|
||||||
|
let network = match network("mainnet") {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let port = FakePersistencePort::new(
|
||||||
|
network,
|
||||||
|
PortResponse::VariantOutcome(
|
||||||
|
ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::Inserted,
|
||||||
|
ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
let acquisition = match acquisition("mainnet", 8) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let result = crate::persist_raw_transaction_ingest_acquisition(&port, acquisition).await;
|
||||||
|
let outcome = match result {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
assert_eq!(outcome.entity(), crate::RawTransactionIngestEntityPersistence::AlreadyPresent);
|
||||||
|
assert_eq!(outcome.observation(), crate::RawTransactionIngestObservationPersistence::Inserted);
|
||||||
|
assert!(outcome.quarantined_conflict());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "current_thread")]
|
||||||
|
async fn pre_007_legacy_store_conflict_error_maps_to_terminal_worker_code_without_remote_text() {
|
||||||
let network = match network("mainnet") {
|
let network = match network("mainnet") {
|
||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
@@ -303,14 +305,14 @@ async fn pre_007_store_network_mismatch_is_rejected_before_write() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "current_thread")]
|
#[tokio::test(flavor = "current_thread")]
|
||||||
async fn v0_3_13_pre_008_same_canonical_identity_uses_atomic_entity_once_then_additional_observation() {
|
async fn v0_3_16_pre_007_same_identity_is_serialized_but_every_acquisition_reaches_store() {
|
||||||
let network = match network("mainnet") {
|
let network = match network("mainnet") {
|
||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
};
|
};
|
||||||
let port = FakePersistencePort::new(
|
let port = FakePersistencePort::new(
|
||||||
network,
|
network,
|
||||||
PortResponse::Outcome(ksp_store_lib::RawEntityWriteOutcome::Inserted, ksp_store_lib::RawObservationWriteOutcome::Inserted),
|
PortResponse::Outcome(ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent, ksp_store_lib::RawObservationWriteOutcome::Inserted),
|
||||||
);
|
);
|
||||||
let convergence = crate::RawTransactionIngestPersistenceConvergence::new(8);
|
let convergence = crate::RawTransactionIngestPersistenceConvergence::new(8);
|
||||||
let first = match acquisition_with_observation_key("mainnet", 41, 1) {
|
let first = match acquisition_with_observation_key("mainnet", 41, 1) {
|
||||||
@@ -321,56 +323,14 @@ async fn v0_3_13_pre_008_same_canonical_identity_uses_atomic_entity_once_then_ad
|
|||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
};
|
};
|
||||||
let first_outcome = crate::persist_raw_transaction_ingest_converged_acquisition(&port, first, &convergence).await;
|
assert!(crate::persist_raw_transaction_ingest_converged_acquisition(&port, first, &convergence).await.is_ok());
|
||||||
let second_outcome = crate::persist_raw_transaction_ingest_converged_acquisition(&port, second, &convergence).await;
|
assert!(crate::persist_raw_transaction_ingest_converged_acquisition(&port, second, &convergence).await.is_ok());
|
||||||
assert!(first_outcome.is_ok());
|
assert_eq!(port.calls.load(std::sync::atomic::Ordering::Acquire), 2);
|
||||||
assert!(second_outcome.is_ok());
|
|
||||||
assert_eq!(port.calls.load(std::sync::atomic::Ordering::Acquire), 1);
|
|
||||||
assert_eq!(port.observation_calls.load(std::sync::atomic::Ordering::Acquire), 1);
|
|
||||||
let second_outcome = match second_outcome {
|
|
||||||
std::result::Result::Ok(value) => value,
|
|
||||||
std::result::Result::Err(_) => return,
|
|
||||||
};
|
|
||||||
assert_eq!(second_outcome.entity(), crate::RawTransactionIngestEntityPersistence::AlreadyPresent);
|
|
||||||
assert_eq!(second_outcome.observation(), crate::RawTransactionIngestObservationPersistence::Inserted);
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "current_thread")]
|
#[tokio::test(flavor = "current_thread")]
|
||||||
async fn v0_3_13_pre_008_replayed_same_observation_is_idempotent_without_raw_resubmission() {
|
async fn v0_3_16_pre_007_run_local_cache_never_rejects_divergent_shapes_before_store() {
|
||||||
let network = match network("mainnet") {
|
|
||||||
std::option::Option::Some(value) => value,
|
|
||||||
std::option::Option::None => return,
|
|
||||||
};
|
|
||||||
let port = FakePersistencePort::new(
|
|
||||||
network,
|
|
||||||
PortResponse::Outcome(ksp_store_lib::RawEntityWriteOutcome::Inserted, ksp_store_lib::RawObservationWriteOutcome::Inserted),
|
|
||||||
)
|
|
||||||
.with_observation_response(ksp_store_lib::RawObservationWriteOutcome::AlreadyPresent);
|
|
||||||
let convergence = crate::RawTransactionIngestPersistenceConvergence::new(8);
|
|
||||||
let first = match acquisition_with_observation_key("mainnet", 42, 3) {
|
|
||||||
std::option::Option::Some(value) => value,
|
|
||||||
std::option::Option::None => return,
|
|
||||||
};
|
|
||||||
let second = match acquisition_with_observation_key("mainnet", 42, 3) {
|
|
||||||
std::option::Option::Some(value) => value,
|
|
||||||
std::option::Option::None => return,
|
|
||||||
};
|
|
||||||
let first_result = crate::persist_raw_transaction_ingest_converged_acquisition(&port, first, &convergence).await;
|
|
||||||
assert!(first_result.is_ok());
|
|
||||||
let second_result = crate::persist_raw_transaction_ingest_converged_acquisition(&port, second, &convergence).await;
|
|
||||||
let second_outcome = match second_result {
|
|
||||||
std::result::Result::Ok(value) => value,
|
|
||||||
std::result::Result::Err(_) => return,
|
|
||||||
};
|
|
||||||
assert_eq!(port.calls.load(std::sync::atomic::Ordering::Acquire), 1);
|
|
||||||
assert_eq!(port.observation_calls.load(std::sync::atomic::Ordering::Acquire), 1);
|
|
||||||
assert_eq!(second_outcome.observation(), crate::RawTransactionIngestObservationPersistence::AlreadyPresent);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test(flavor = "current_thread")]
|
|
||||||
async fn v0_3_13_pre_009_source_skew_and_provider_disagreement_fail_before_additional_observation() {
|
|
||||||
let cases = [
|
let cases = [
|
||||||
(43_u64, std::option::Option::Some(1_700_000_000_i64), "AQID"),
|
(43_u64, std::option::Option::Some(1_700_000_000_i64), "AQID"),
|
||||||
(42_u64, std::option::Option::Some(1_700_000_001_i64), "AQID"),
|
(42_u64, std::option::Option::Some(1_700_000_001_i64), "AQID"),
|
||||||
@@ -383,7 +343,11 @@ async fn v0_3_13_pre_009_source_skew_and_provider_disagreement_fail_before_addit
|
|||||||
};
|
};
|
||||||
let port = FakePersistencePort::new(
|
let port = FakePersistencePort::new(
|
||||||
network,
|
network,
|
||||||
PortResponse::Outcome(ksp_store_lib::RawEntityWriteOutcome::Inserted, ksp_store_lib::RawObservationWriteOutcome::Inserted),
|
PortResponse::VariantOutcome(
|
||||||
|
ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
|
ksp_store_lib::RawObservationWriteOutcome::Inserted,
|
||||||
|
ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
let convergence = crate::RawTransactionIngestPersistenceConvergence::new(8);
|
let convergence = crate::RawTransactionIngestPersistenceConvergence::new(8);
|
||||||
let first = match acquisition_with_shape("mainnet", 51, 10, 42, std::option::Option::Some(1_700_000_000), "AQID") {
|
let first = match acquisition_with_shape("mainnet", 51, 10, 42, std::option::Option::Some(1_700_000_000), "AQID") {
|
||||||
@@ -394,15 +358,11 @@ async fn v0_3_13_pre_009_source_skew_and_provider_disagreement_fail_before_addit
|
|||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
};
|
};
|
||||||
assert!(crate::persist_raw_transaction_ingest_converged_acquisition(&port, first, &convergence).await.is_ok());
|
let first_outcome = crate::persist_raw_transaction_ingest_converged_acquisition(&port, first, &convergence).await;
|
||||||
let conflict = crate::persist_raw_transaction_ingest_converged_acquisition(&port, divergent, &convergence).await;
|
let second_outcome = crate::persist_raw_transaction_ingest_converged_acquisition(&port, divergent, &convergence).await;
|
||||||
let error = match conflict {
|
assert!(matches!(first_outcome, std::result::Result::Ok(value) if value.quarantined_conflict()));
|
||||||
std::result::Result::Ok(_) => return,
|
assert!(matches!(second_outcome, std::result::Result::Ok(value) if value.quarantined_conflict()));
|
||||||
std::result::Result::Err(value) => value,
|
assert_eq!(port.calls.load(std::sync::atomic::Ordering::Acquire), 2);
|
||||||
};
|
|
||||||
assert_eq!(error.code(), crate::ERROR_CODE_RAW_TRANSACTION_INGEST_CONTENT_CONFLICT);
|
|
||||||
assert_eq!(port.calls.load(std::sync::atomic::Ordering::Acquire), 1);
|
|
||||||
assert_eq!(port.observation_calls.load(std::sync::atomic::Ordering::Acquire), 0);
|
|
||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime.rs
|
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime.rs
|
||||||
// version: 12
|
// version: 15
|
||||||
|
|
||||||
struct ActiveTaskGuard {
|
struct ActiveTaskGuard {
|
||||||
active: std::sync::Arc<std::sync::atomic::AtomicUsize>,
|
active: std::sync::Arc<std::sync::atomic::AtomicUsize>,
|
||||||
@@ -249,6 +249,7 @@ async fn pre_005_supervisor_reaps_completed_source_task_and_still_joins_live_chi
|
|||||||
#[derive(Clone, Copy)]
|
#[derive(Clone, Copy)]
|
||||||
enum RuntimePortResponse {
|
enum RuntimePortResponse {
|
||||||
Conflict,
|
Conflict,
|
||||||
|
QuarantinedConflict,
|
||||||
StoreFailure,
|
StoreFailure,
|
||||||
StoreFailureBlocked,
|
StoreFailureBlocked,
|
||||||
SuccessBlocked,
|
SuccessBlocked,
|
||||||
@@ -337,35 +338,13 @@ impl crate::RawTransactionIngestPersistencePort for RuntimePersistencePort {
|
|||||||
));
|
));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if matches!(response, RuntimePortResponse::Conflict) {
|
if matches!(response, RuntimePortResponse::QuarantinedConflict) {
|
||||||
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
||||||
return std::result::Result::Err(ksp_core_lib::Error::new(ksp_store_lib::ERROR_CODE_RAW_CONFLICT, "synthetic conflict"));
|
return std::result::Result::Ok(ksp_store_lib::RawAcquisitionWriteOutcome::with_transaction_variant(
|
||||||
}
|
ksp_store_lib::RawEntityWriteOutcome::AlreadyPresent,
|
||||||
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
ksp_store_lib::RawObservationWriteOutcome::Inserted,
|
||||||
return std::result::Result::Err(ksp_core_lib::Error::new(
|
ksp_store_lib::RawTransactionVariantWriteOutcome::QuarantinedConflict,
|
||||||
ksp_core_lib::ErrorCode::new("synthetic_store", "write_failed"),
|
|
||||||
"synthetic store failure",
|
|
||||||
));
|
));
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
fn record_observation<'a>(
|
|
||||||
&'a self,
|
|
||||||
_observation: ksp_store_lib::RawTransactionObservation,
|
|
||||||
) -> ksp_store_lib::StoreApiFuture<'a, ksp_store_lib::Result<ksp_store_lib::RawObservationWriteOutcome>> {
|
|
||||||
let response = self.response;
|
|
||||||
return std::boxed::Box::pin(async move {
|
|
||||||
if matches!(response, RuntimePortResponse::StoreFailureBlocked | RuntimePortResponse::SuccessBlocked) {
|
|
||||||
let current = self.active.fetch_add(1, std::sync::atomic::Ordering::AcqRel) + 1;
|
|
||||||
let _active_guard = RuntimePortActiveGuard { active: &self.active };
|
|
||||||
self.update_max_active(current);
|
|
||||||
while !self.released.load(std::sync::atomic::Ordering::Acquire) {
|
|
||||||
self.notify.notified().await;
|
|
||||||
}
|
|
||||||
if matches!(response, RuntimePortResponse::SuccessBlocked) {
|
|
||||||
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
|
||||||
return std::result::Result::Ok(ksp_store_lib::RawObservationWriteOutcome::Inserted);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if matches!(response, RuntimePortResponse::Conflict) {
|
if matches!(response, RuntimePortResponse::Conflict) {
|
||||||
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
self.completed.fetch_add(1, std::sync::atomic::Ordering::AcqRel);
|
||||||
@@ -505,7 +484,94 @@ async fn pre_007_runtime_bounds_in_flight_store_persistence_to_configured_concur
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "current_thread")]
|
#[tokio::test(flavor = "current_thread")]
|
||||||
async fn pre_007_content_conflict_becomes_terminal_after_private_drain() {
|
async fn v0_3_16_pre_007_quarantined_conflict_keeps_worker_running_and_degraded() {
|
||||||
|
let settings = match settings_with_persistence_concurrency(1) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let network = settings.network().clone();
|
||||||
|
let port = std::sync::Arc::new(RuntimePersistencePort::new(network.clone(), RuntimePortResponse::QuarantinedConflict, true));
|
||||||
|
let runtime_port: super::PersistencePort = port.clone();
|
||||||
|
let handle = match super::start_foundation_with_port_and_source_spawner(
|
||||||
|
settings,
|
||||||
|
tokio::runtime::Handle::current(),
|
||||||
|
std::option::Option::Some(runtime_port),
|
||||||
|
move |children, _stop_receiver, admission_sender| {
|
||||||
|
let _abort_handle = children.spawn(async move {
|
||||||
|
let ingress = match runtime_ingress(&network, 10) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::result::Result::Err(crate::runtime_error("test.source_failed")),
|
||||||
|
};
|
||||||
|
if admission_sender.send(ingress).await.is_err() {
|
||||||
|
return std::result::Result::Err(crate::runtime_error("test.source_failed"));
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
});
|
||||||
|
},
|
||||||
|
) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
let source = handle.snapshot_source();
|
||||||
|
assert!(wait_for_completed(&port, 1).await);
|
||||||
|
let snapshot = source.current();
|
||||||
|
assert_eq!(snapshot.worker_snapshot().state(), ksp_worker_api::WorkerState::Running);
|
||||||
|
assert_eq!(snapshot.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Degraded);
|
||||||
|
assert_eq!(snapshot.persisted_total(), 1);
|
||||||
|
assert_eq!(snapshot.content_conflict_total(), 1);
|
||||||
|
assert_eq!(snapshot.store_failure_total(), 0);
|
||||||
|
assert!(handle.request_stop());
|
||||||
|
let terminal = handle.wait_terminal().await;
|
||||||
|
assert!(matches!(terminal, std::result::Result::Ok(ksp_worker_api::WorkerState::Stopped)));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "current_thread")]
|
||||||
|
async fn v0_3_16_pre_008_quarantined_conflict_does_not_block_later_identity() {
|
||||||
|
let settings = match settings_with_persistence_concurrency(1) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return,
|
||||||
|
};
|
||||||
|
let network = settings.network().clone();
|
||||||
|
let port = std::sync::Arc::new(RuntimePersistencePort::new(network.clone(), RuntimePortResponse::QuarantinedConflict, true));
|
||||||
|
let runtime_port: super::PersistencePort = port.clone();
|
||||||
|
let handle = match super::start_foundation_with_port_and_source_spawner(
|
||||||
|
settings,
|
||||||
|
tokio::runtime::Handle::current(),
|
||||||
|
std::option::Option::Some(runtime_port),
|
||||||
|
move |children, _stop_receiver, admission_sender| {
|
||||||
|
let _abort_handle = children.spawn(async move {
|
||||||
|
for signature_byte in [20_u8, 21_u8] {
|
||||||
|
let ingress = match runtime_ingress(&network, signature_byte) {
|
||||||
|
std::option::Option::Some(value) => value,
|
||||||
|
std::option::Option::None => return std::result::Result::Err(crate::runtime_error("test.source_failed")),
|
||||||
|
};
|
||||||
|
if admission_sender.send(ingress).await.is_err() {
|
||||||
|
return std::result::Result::Err(crate::runtime_error("test.source_failed"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::result::Result::Ok(());
|
||||||
|
});
|
||||||
|
},
|
||||||
|
) {
|
||||||
|
std::result::Result::Ok(value) => value,
|
||||||
|
std::result::Result::Err(_) => return,
|
||||||
|
};
|
||||||
|
let source = handle.snapshot_source();
|
||||||
|
assert!(wait_for_completed(&port, 2).await);
|
||||||
|
let snapshot = source.current();
|
||||||
|
assert_eq!(snapshot.worker_snapshot().state(), ksp_worker_api::WorkerState::Running);
|
||||||
|
assert_eq!(snapshot.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Degraded);
|
||||||
|
assert_eq!(snapshot.persisted_total(), 2);
|
||||||
|
assert_eq!(snapshot.content_conflict_total(), 2);
|
||||||
|
assert_eq!(snapshot.store_failure_total(), 0);
|
||||||
|
assert!(handle.request_stop());
|
||||||
|
assert!(matches!(handle.wait_terminal().await, std::result::Result::Ok(ksp_worker_api::WorkerState::Stopped)));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "current_thread")]
|
||||||
|
async fn pre_007_legacy_store_conflict_error_becomes_terminal_after_private_drain() {
|
||||||
let settings = match settings_with_persistence_concurrency(1) {
|
let settings = match settings_with_persistence_concurrency(1) {
|
||||||
std::option::Option::Some(value) => value,
|
std::option::Option::Some(value) => value,
|
||||||
std::option::Option::None => return,
|
std::option::Option::None => return,
|
||||||
|
|||||||
140
deltas/0.3.16/pre.001-fix.001.md
Normal file
140
deltas/0.3.16/pre.001-fix.001.md
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.001-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.001-fix.001` — redécoupage `0.3.16` -> `0.3.18`
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.001 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Ce fix est strictement documentaire.
|
||||||
|
|
||||||
|
## Objet
|
||||||
|
|
||||||
|
Corriger le sizing du plan 038 après validation opérateur de `0.3.16-pre.001`.
|
||||||
|
|
||||||
|
Le découpage initial prévoyait jusqu'à `0.3.16-pre.022` avant release stable. Avec les `fix.*` probables, cette trajectoire ne respecte pas suffisamment la règle opérationnelle KSP selon laquelle une version doit tenir dans au maximum une session normale.
|
||||||
|
|
||||||
|
Le programme RAW résilience/conflits est donc réparti sur :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16 : fondations RAW multi-variantes + promotion sûre + ingestion non terminale
|
||||||
|
0.3.17 : résilience opérationnelle + cycle de vie/rétention/résolution
|
||||||
|
0.3.18 : inspection/résolution opérateur Store Desk + hardening end-to-end
|
||||||
|
```
|
||||||
|
|
||||||
|
## Version workspace
|
||||||
|
|
||||||
|
Aucune modification.
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Conformément aux règles KSP, un fix strictement documentaire ne modifie ni `Cargo.toml`, ni package frontend, ni configuration Tauri/Android, ni autre version runtime/build.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.001-fix.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Nouveau découpage
|
||||||
|
|
||||||
|
### `0.3.16`
|
||||||
|
|
||||||
|
Huit prereleases prévues au maximum dans le plan courant :
|
||||||
|
|
||||||
|
```text
|
||||||
|
pre.001 gate architecture/sizing déjà appliqué
|
||||||
|
pre.002 Store API variantes/qualité/outcomes
|
||||||
|
pre.003 migration V003
|
||||||
|
pre.004 PostgreSQL variant ledger/observations
|
||||||
|
pre.005 comparateur et canaris logMessages
|
||||||
|
pre.006 sélecteur/promotion canonique atomique
|
||||||
|
pre.007 conflit durable + Worker non terminal
|
||||||
|
pre.008 hardening/gates/fermeture
|
||||||
|
rel.001 stable
|
||||||
|
```
|
||||||
|
|
||||||
|
### `0.3.17`
|
||||||
|
|
||||||
|
Huit prereleases prévues : inspection/résolution, cycle de vie des conflits, rétention, Store-lib, retry/backpressure Worker, reconnect Transport/Config, hardening/live et fermeture.
|
||||||
|
|
||||||
|
### `0.3.18`
|
||||||
|
|
||||||
|
Sept prereleases prévues : Store Desk backend, frontend, actions opérateur, fusion synthétique uniquement si prouvée, hardening, preuves end-to-end et fermeture documentaire.
|
||||||
|
|
||||||
|
Le nombre exact reste souple. Les `fix.*` ne forcent jamais la fusion artificielle de tranches suivantes.
|
||||||
|
|
||||||
|
## Déplacement des anciens scopes `0.3.17` / `0.3.18`
|
||||||
|
|
||||||
|
Les réservations précédentes :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.17 : Backfill multi-route/multi-stratégie
|
||||||
|
0.3.18 : adaptation Backfill Desk
|
||||||
|
```
|
||||||
|
|
||||||
|
sont annulées pour éviter une collision de roadmap avec le programme de résilience RAW.
|
||||||
|
|
||||||
|
Cibles de planning par défaut après ce programme :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.19 : Backfill multi-route/multi-stratégie
|
||||||
|
0.3.20 : adaptation Backfill Desk
|
||||||
|
```
|
||||||
|
|
||||||
|
Ces deux numéros restent des cibles de planning et pourront être réévalués à leur gate d'ouverture.
|
||||||
|
|
||||||
|
## Validation opérateur de `pre.001` enregistrée
|
||||||
|
|
||||||
|
Transcript fourni le 20 septembre 2026 :
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit : clean
|
||||||
|
Rust export completeness audit : 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit : clean
|
||||||
|
Markdown table audit : clean (352 table(s), 941 file(s))
|
||||||
|
cargo check --workspace : PASS / Finished dev profile
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings : PASS / Finished dev profile
|
||||||
|
cargo fmt --all -- --check : aucun diagnostic affiché
|
||||||
|
```
|
||||||
|
|
||||||
|
Toutes les crates affichées résolvent `v0.3.16-pre.1`.
|
||||||
|
|
||||||
|
## Validations du présent fix
|
||||||
|
|
||||||
|
Le présent delta ne modifie que du Markdown. Après application, le gate minimal requis est :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun `cargo fmt`, `cargo check`, `cargo clippy`, test Rust, build Tauri ou test live n'est requis par ce fix seul puisqu'aucun code, manifest, dépendance ou configuration runtime/build n'est modifié.
|
||||||
|
|
||||||
|
## Prochaine tranche
|
||||||
|
|
||||||
|
Après audit Markdown propre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.002
|
||||||
|
```
|
||||||
|
|
||||||
|
Objet : contrats Store API backend-neutral pour variantes, relations de qualité, outcomes de persistance, reason codes et contrats minimaux du conflit durable.
|
||||||
178
deltas/0.3.16/pre.001.md
Normal file
178
deltas/0.3.16/pre.001.md
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.001` — gate d'audit, architecture et sizing
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
v0.3.15
|
||||||
|
commit 7328be6997399c513306f2f4c8cdf00bd8b22367
|
||||||
|
workspace.package.version = 0.3.15
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne pas appliquer sur une `0.3.15-pre.*`.
|
||||||
|
|
||||||
|
## Objet
|
||||||
|
|
||||||
|
Exécuter le gate obligatoire de lecture/audit/brainstorming/sizing défini par `prompts/035-V0_3_16_START_PROMPT.md` avant toute migration ou implémentation lourde de la résilience RAW.
|
||||||
|
|
||||||
|
Cette tranche :
|
||||||
|
|
||||||
|
- réaudite la fermeture stable `0.3.15` ;
|
||||||
|
- réaudite les règles KSP applicables ;
|
||||||
|
- inventorie les contrats Store/Worker/Transport/Config/Store Desk actuels ;
|
||||||
|
- réaudite les sémantiques Solana RPC et PostgreSQL pertinentes ;
|
||||||
|
- compare plusieurs modèles physiques ;
|
||||||
|
- choisit le modèle conceptuel et les ownerships ;
|
||||||
|
- définit les invariants de concurrence/rétention/résolution ;
|
||||||
|
- recalibre le découpage des prereleases ;
|
||||||
|
- ne crée aucune migration SQL finale et ne modifie aucun Rust.
|
||||||
|
|
||||||
|
## Version workspace
|
||||||
|
|
||||||
|
`Cargo.toml` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
header version : 637 -> 638
|
||||||
|
workspace : 0.3.15 -> 0.3.16-pre.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune autre modification sémantique du `Cargo.toml` racine n'est prévue.
|
||||||
|
|
||||||
|
## Fichiers ajoutés
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
|
||||||
|
deltas/0.3.16/pre.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Décisions principales
|
||||||
|
|
||||||
|
### Variantes
|
||||||
|
|
||||||
|
Modèle retenu : ledger de variantes V003 à identifiant surrogate, sélecteur canonique sidecar et conservation de `ksp_raw_transactions` comme projection canonique compatible V001.
|
||||||
|
|
||||||
|
`content_hash` est un préfiltre/intégrité ; il n'est pas une preuve d'égalité à lui seul lorsque les bytes sont disponibles.
|
||||||
|
|
||||||
|
### Observations
|
||||||
|
|
||||||
|
Toute observation créée sous V003 devra pointer vers la variante réellement reçue.
|
||||||
|
|
||||||
|
Les observations historiques dont le payload entrant n'a pas été conservé sont qualifiées legacy/unknown et ne sont jamais faussement rattachées à une variante native reconstruite.
|
||||||
|
|
||||||
|
### Comparaison
|
||||||
|
|
||||||
|
Relations internes retenues :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact
|
||||||
|
CompatibleLessComplete
|
||||||
|
CompatibleMoreComplete
|
||||||
|
Conflict
|
||||||
|
Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
Seule la troncature `logMessages` strictement prouvée autorise une dominance automatique en première implémentation.
|
||||||
|
|
||||||
|
### Rétention
|
||||||
|
|
||||||
|
Les variantes nécessaires au canonique courant, aux conflits ouverts, au rollback ou à la lignée synthétique peuvent être archivées mais restent protégées contre une purge irréversible tant que l'invariant dépend de leurs bytes.
|
||||||
|
|
||||||
|
### Store retry
|
||||||
|
|
||||||
|
Le backend Store classe les fautes `Transient/Terminal` de manière backend-neutral ; le Worker orchestre retry, backpressure, health et exhaustion.
|
||||||
|
|
||||||
|
### Transport reconnect
|
||||||
|
|
||||||
|
La reconnexion reste propriété de `ksp-onchain-transport-lib` et étend les mécanismes existants. Elle reste distincte du retry Store et ne prouve jamais la coverage.
|
||||||
|
|
||||||
|
### Store Desk
|
||||||
|
|
||||||
|
Inspection et résolution passent exclusivement par `ksp-store-lib`. Les actions futures sont typées et protégées par revision attendue.
|
||||||
|
|
||||||
|
## Découpage recalibré
|
||||||
|
|
||||||
|
La release est étendue jusqu'à une prévision souple `pre.022` avant `rel.001` afin de ne pas fusionner artificiellement migration, backend, convergence, rétention, Worker, Transport, Desk et gates de fermeture.
|
||||||
|
|
||||||
|
Le détail est dans le plan `038`.
|
||||||
|
|
||||||
|
## Hors périmètre
|
||||||
|
|
||||||
|
Inchangé par rapport au prompt 035 :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Backfill multi-route/multi-stratégie
|
||||||
|
Backfill Desk multi-route
|
||||||
|
RAW -> STRUCTURAL
|
||||||
|
STRUCTURAL persistence
|
||||||
|
DECODED / DOMAIN
|
||||||
|
majority/provider authority
|
||||||
|
free JSON merge
|
||||||
|
Internet reconstruction as normal rollback
|
||||||
|
second acquisition pipeline
|
||||||
|
Worker <-> Backfill dependency
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations réellement exécutées lors de l'assemblage du delta
|
||||||
|
|
||||||
|
L'environnement d'assemblage possède uniquement les fichiers du delta, pas le checkout complet `v0.3.15`.
|
||||||
|
|
||||||
|
Sont exécutés sur l'archive produite :
|
||||||
|
|
||||||
|
```text
|
||||||
|
parse TOML du Cargo.toml delta
|
||||||
|
vérification workspace.package.version = 0.3.16-pre.1
|
||||||
|
vérification des headers file/version des quatre fichiers
|
||||||
|
vérification newline finale
|
||||||
|
vérification absence de lockfile/cache/secret dans le ZIP
|
||||||
|
vérification inventaire ZIP exact
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne sont pas déclarés PASS ici :
|
||||||
|
|
||||||
|
```text
|
||||||
|
cargo fmt/check/clippy/test workspace
|
||||||
|
scripts/audit_rust_workspace_rules.py
|
||||||
|
scripts/audit_markdown_tables.py du dépôt complet
|
||||||
|
byte-compare de l'archive source v0.3.15
|
||||||
|
```
|
||||||
|
|
||||||
|
L'archive binaire source stable n'était pas matérialisable dans l'environnement d'assemblage. L'arbre/tag canonique et le commit de base ont été vérifiés via le dépôt public.
|
||||||
|
|
||||||
|
## Gate demandé après application
|
||||||
|
|
||||||
|
À exécuter par l'opérateur sur le checkout exact `v0.3.15` après application du delta :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun test live/provider/PostgreSQL n'est requis par ce gate documentaire s'il n'est pas explicitement exécuté. Aucun `npm run build` n'est utilisé.
|
||||||
|
|
||||||
|
## Prochaine tranche
|
||||||
|
|
||||||
|
Après gate propre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.002
|
||||||
|
```
|
||||||
|
|
||||||
|
Objet prévu : contrats Store API backend-neutral pour variantes, relations de qualité, reason codes et outcomes de persistance, sans migration PostgreSQL anticipée.
|
||||||
134
deltas/0.3.16/pre.002-fix.001.md
Normal file
134
deltas/0.3.16/pre.002-fix.001.md
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.002-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.002-fix.001` — conformité `RUST-FMT-111`
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.002 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Motif du fix
|
||||||
|
|
||||||
|
Le gate manuel de `0.3.16-pre.002` confirme que le workspace compile, que Clippy passe et que les tests ciblés `ksp-store-api` passent, mais l'audit des règles Rust signale exactement 26 violations `RUST-FMT-111` : absence de l'unique ligne vide exigée entre certains items Rust.
|
||||||
|
|
||||||
|
Aucune erreur fonctionnelle, de compilation, de Clippy, d'export ou de test n'a été observée dans ce gate.
|
||||||
|
|
||||||
|
## Version workspace
|
||||||
|
|
||||||
|
`Cargo.toml` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
header version : 639 -> 640
|
||||||
|
workspace : 0.3.16-pre.2 -> 0.3.16-pre.2.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Ce fix modifie des fichiers Rust ; il incrémente donc la version workspace conformément aux règles du projet.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-api/src/model/raw_outcome.rs
|
||||||
|
crates/ksp-store-api/src/model/raw_transaction.rs
|
||||||
|
crates/ksp-store-api/tests/release_completeness.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.002-fix.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Correction
|
||||||
|
|
||||||
|
Le fix ajoute uniquement les 26 lignes vides requises par `RUST-FMT-111` aux emplacements signalés par l'audit :
|
||||||
|
|
||||||
|
```text
|
||||||
|
raw_outcome.rs : 4
|
||||||
|
raw_transaction.rs : 16
|
||||||
|
release_completeness.rs : 6
|
||||||
|
--
|
||||||
|
total : 26
|
||||||
|
```
|
||||||
|
|
||||||
|
Les séparations concernent des frontières entre items de module, `impl`, méthodes, tests et fonctions auxiliaires.
|
||||||
|
|
||||||
|
Aucun type, enum, variant, champ, reason code, outcome, méthode, corps de fonction, assertion, export ou comportement introduit par `pre.002` n'est modifié.
|
||||||
|
|
||||||
|
## Gate `pre.002` reçu avant correction
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit: 26 violation(s)
|
||||||
|
RUST-FMT-111: 26
|
||||||
|
Rust export completeness audit: 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit: clean
|
||||||
|
Markdown table audit: clean (352 table(s), 943 file(s))
|
||||||
|
cargo check --workspace: PASS
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings: PASS
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features: PASS
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests `ksp-store-api` observés :
|
||||||
|
|
||||||
|
```text
|
||||||
|
unit tests : 25 passed
|
||||||
|
dependency_boundary : 2 passed
|
||||||
|
external_backend : 1 passed
|
||||||
|
public_api : 8 passed
|
||||||
|
release_completeness : 5 passed
|
||||||
|
security_hardening : 5 passed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation de l'assemblage
|
||||||
|
|
||||||
|
L'environnement d'assemblage ne fournit ni `cargo`, ni `rustc`, ni `rustfmt`; aucun gate Rust n'est déclaré PASS localement.
|
||||||
|
|
||||||
|
Ont été vérifiés localement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
diff fonctionnel nul hors whitespace et headers de version
|
||||||
|
26 séparations manquantes ajoutées exactement
|
||||||
|
workspace.package.version = 0.3.16-pre.2.fix.1
|
||||||
|
headers file/version des trois fichiers Rust incrémentés
|
||||||
|
newline finale
|
||||||
|
inventaire exact du delta
|
||||||
|
absence de lockfile/cache/secret
|
||||||
|
archive ZIP valide
|
||||||
|
```
|
||||||
|
|
||||||
|
## Gate requis après application
|
||||||
|
|
||||||
|
Comme le fix modifie du Rust et `Cargo.toml`, exécuter :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Le résultat attendu est notamment :
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit: clean
|
||||||
|
Rust export completeness audit: 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit: clean
|
||||||
|
```
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre, poursuivre avec :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.003
|
||||||
|
```
|
||||||
|
|
||||||
|
Le scope fonctionnel et architectural de `pre.002` reste inchangé.
|
||||||
262
deltas/0.3.16/pre.002.md
Normal file
262
deltas/0.3.16/pre.002.md
Normal file
@@ -0,0 +1,262 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.002.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.002` — contrats Store API RAW multi-variantes
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.001 appliquée
|
||||||
|
0.3.16-pre.001-fix.001 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate Markdown du fix précédent a été fourni propre le 20 septembre 2026 :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Markdown table audit: clean (352 table(s), 942 file(s))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Objet
|
||||||
|
|
||||||
|
Figer les contrats backend-neutral indispensables à la persistance RAW transaction multi-variantes avant d'introduire la migration PostgreSQL V003.
|
||||||
|
|
||||||
|
Cette tranche ajoute uniquement des modèles/types Store API. Elle ne modifie encore :
|
||||||
|
|
||||||
|
- aucune migration SQL ;
|
||||||
|
- aucun backend PostgreSQL ;
|
||||||
|
- aucun `RawTransactionWrite` existant ;
|
||||||
|
- aucun Worker ;
|
||||||
|
- aucun comparateur de contenu `logMessages` ;
|
||||||
|
- aucune UI Store Desk.
|
||||||
|
|
||||||
|
## Version workspace
|
||||||
|
|
||||||
|
`Cargo.toml` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
header version : 638 -> 639
|
||||||
|
workspace : 0.3.16-pre.1 -> 0.3.16-pre.2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-api/src/lib.rs
|
||||||
|
crates/ksp-store-api/src/model/raw_outcome.rs
|
||||||
|
crates/ksp-store-api/src/model/raw_transaction.rs
|
||||||
|
crates/ksp-store-api/tests/release_completeness.rs
|
||||||
|
crates/ksp-store-api/unit_tests/model/raw_transaction.rs
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.002.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Contrats ajoutés
|
||||||
|
|
||||||
|
### Identité de variante
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariantId
|
||||||
|
RawTransactionVariantReference
|
||||||
|
```
|
||||||
|
|
||||||
|
`RawTransactionVariantId` est un surrogate backend-neutral strictement non nul. Il ne dérive ni de `content_hash`, ni du provider, ni d'une clé physique SQL.
|
||||||
|
|
||||||
|
`RawTransactionVariantReference` conserve explicitement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionReference
|
||||||
|
+
|
||||||
|
RawTransactionVariantId
|
||||||
|
```
|
||||||
|
|
||||||
|
Le hash reste donc un attribut de contenu et jamais l'identité métier de la variante.
|
||||||
|
|
||||||
|
### Origine de variante
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariantOrigin::Native
|
||||||
|
RawTransactionVariantOrigin::Synthetic
|
||||||
|
```
|
||||||
|
|
||||||
|
Les codes persistables stables sont respectivement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
native
|
||||||
|
synthetic
|
||||||
|
```
|
||||||
|
|
||||||
|
Une variante synthétique est seulement un contrat réservé ; aucune fusion synthétique n'est implémentée dans cette tranche.
|
||||||
|
|
||||||
|
### Relation de qualité
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariantRelation::Exact
|
||||||
|
RawTransactionVariantRelation::CompatibleLessComplete
|
||||||
|
RawTransactionVariantRelation::CompatibleMoreComplete
|
||||||
|
RawTransactionVariantRelation::Conflict
|
||||||
|
RawTransactionVariantRelation::Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
La direction du contrat est toujours :
|
||||||
|
|
||||||
|
```text
|
||||||
|
current canonical
|
||||||
|
vs
|
||||||
|
incoming variant
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune heuristique provider-majority ou source authority n'est introduite.
|
||||||
|
|
||||||
|
### Reason codes
|
||||||
|
|
||||||
|
`RawTransactionVariantRelationReason` fige les raisons backend-neutral suivantes :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ExactCanonicalContent
|
||||||
|
IncomingLogMessagesTruncated
|
||||||
|
CanonicalLogMessagesTruncated
|
||||||
|
SlotMismatch
|
||||||
|
BlockTimeMismatch
|
||||||
|
PayloadFormatMismatch
|
||||||
|
CanonicalPayloadConflict
|
||||||
|
ContentHashCollision
|
||||||
|
UnsupportedCanonicalDifference
|
||||||
|
```
|
||||||
|
|
||||||
|
Chaque raison possède un `code()` stable destiné à la persistance et aux diagnostics sûrs.
|
||||||
|
|
||||||
|
Les reason codes ne transportent ni URL, ni provider concret, ni secret, ni payload RAW.
|
||||||
|
|
||||||
|
### Couple relation/reason validé
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariantComparison
|
||||||
|
```
|
||||||
|
|
||||||
|
`try_new(...)` refuse les couples incohérents.
|
||||||
|
|
||||||
|
Contrats actuellement admis :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact <-> ExactCanonicalContent
|
||||||
|
CompatibleLessComplete <-> IncomingLogMessagesTruncated
|
||||||
|
CompatibleMoreComplete <-> CanonicalLogMessagesTruncated
|
||||||
|
Conflict <-> SlotMismatch | BlockTimeMismatch | CanonicalPayloadConflict | ContentHashCollision
|
||||||
|
Incomparable <-> PayloadFormatMismatch | UnsupportedCanonicalDifference
|
||||||
|
```
|
||||||
|
|
||||||
|
Cette validation ne constitue pas encore le comparateur de contenu. Le comparateur pur est prévu en `0.3.16-pre.005`.
|
||||||
|
|
||||||
|
### Outcome de persistance variant-aware
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariantWriteOutcome::InsertedCanonical
|
||||||
|
RawTransactionVariantWriteOutcome::ObservedExact
|
||||||
|
RawTransactionVariantWriteOutcome::ObservedCompatibleLessComplete
|
||||||
|
RawTransactionVariantWriteOutcome::PromotedCompatibleMoreComplete
|
||||||
|
RawTransactionVariantWriteOutcome::QuarantinedConflict
|
||||||
|
RawTransactionVariantWriteOutcome::Rehydrated
|
||||||
|
RawTransactionVariantWriteOutcome::SkippedPurged
|
||||||
|
```
|
||||||
|
|
||||||
|
`QuarantinedConflict` est explicitement un succès durable Store : une divergence conservée ne doit pas être transformée en faute Transport ou en erreur terminale Worker.
|
||||||
|
|
||||||
|
Aucune capability existante ne retourne encore ce nouvel outcome dans `pre.002`; son adoption backend intervient après V003.
|
||||||
|
|
||||||
|
### Statut minimal du conflict case
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionConflictStatus::Open
|
||||||
|
RawTransactionConflictStatus::Resolved
|
||||||
|
```
|
||||||
|
|
||||||
|
Le détail des actions opérateur, revisions attendues, reopen et historique append-only reste réservé au programme `0.3.17`.
|
||||||
|
|
||||||
|
## Compatibilité
|
||||||
|
|
||||||
|
Les contrats historiques restent intacts :
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransaction
|
||||||
|
RawTransactionObservation
|
||||||
|
RawTransactionReference
|
||||||
|
RawAcquisitionWriteOutcome
|
||||||
|
RawEntityWriteOutcome
|
||||||
|
RawObservationWriteOutcome
|
||||||
|
RawTransactionWrite
|
||||||
|
RawTransactionRead
|
||||||
|
```
|
||||||
|
|
||||||
|
`pre.002` n'impose donc aucune migration immédiate aux consumers existants.
|
||||||
|
|
||||||
|
Tous les nouveaux enums évolutifs sont `#[non_exhaustive]` et les exports restent disponibles uniquement depuis la racine `ksp-store-api` conformément aux règles KSP.
|
||||||
|
|
||||||
|
## Tests/canaris ajoutés ou étendus
|
||||||
|
|
||||||
|
Les tests couvrent notamment :
|
||||||
|
|
||||||
|
```text
|
||||||
|
variant_id = 0 rejeté
|
||||||
|
variant_id non nul conservé sans encoder content_hash
|
||||||
|
référence variante conserve le scope transaction
|
||||||
|
codes Native/Synthetic stables
|
||||||
|
couples relation/reason valides acceptés
|
||||||
|
couples incohérents rejetés
|
||||||
|
codes relation/reason/status stables
|
||||||
|
outcomes variant-aware distincts
|
||||||
|
inventaire exact des exports crate-root mis à jour
|
||||||
|
nouveaux enums evolutifs maintenus #[non_exhaustive]
|
||||||
|
absence de nouveau module production dans ksp-store-api
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation de l'assemblage
|
||||||
|
|
||||||
|
L'environnement d'assemblage ne fournit ni `cargo`, ni `rustc`, ni `rustfmt`. Aucun gate Rust n'est donc déclaré PASS localement.
|
||||||
|
|
||||||
|
Ont été vérifiés localement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
inventaire exact du delta
|
||||||
|
workspace.package.version = 0.3.16-pre.2
|
||||||
|
headers file/version des fichiers modifiés
|
||||||
|
newline finale
|
||||||
|
lignes Rust <= 160 caractères
|
||||||
|
absence de lockfile/cache/secret
|
||||||
|
cohérence statique exports <-> release_completeness
|
||||||
|
cohérence du plan 038 avec les noms Rust désormais figés
|
||||||
|
```
|
||||||
|
|
||||||
|
## Gate requis après application
|
||||||
|
|
||||||
|
Comme cette tranche modifie du Rust et le manifest workspace, exécuter :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne pas exécuter `npm run build` pour ce gate ; aucune application Desk n'est modifiée.
|
||||||
|
|
||||||
|
## Prochaine tranche
|
||||||
|
|
||||||
|
Après gate propre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.003
|
||||||
|
```
|
||||||
|
|
||||||
|
Objet prévu : migration PostgreSQL V003, registre/resources et schéma minimal variant ledger + selector + conflit, avec canaris garantissant l'immuabilité byte/checksum des migrations V000/V001/V002.
|
||||||
68
deltas/0.3.16/pre.003-fix.001.md
Normal file
68
deltas/0.3.16/pre.003-fix.001.md
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.003-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.003-fix.001` — conformité Rust de V003
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.003 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.3
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.003` confirme que `cargo check --workspace` et les suites ciblées `ksp-store-postgres-lib` passent, y compris les 82 tests unitaires, les tests de frontière, de hardening et les trois canaris V003. Le gate reste toutefois rouge avant ce fix : 31 violations de l'audit Rust et un `unused import` transformé en erreur par Clippy `-D warnings`.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette correction modifie du Rust :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.3.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Corrections
|
||||||
|
|
||||||
|
Le fix est strictement structurel et ne modifie ni le schéma SQL V003 ni son comportement métier :
|
||||||
|
|
||||||
|
- les blocs `pub use` et `pub(crate) use` de `ksp-store-postgres-lib` sont séparés et ordonnés selon les règles du workspace ;
|
||||||
|
- les fonctions legacy `migration::bootstrap` et `migration::current_migration_version` sont réexportées à la racine sous les alias privés `crate::legacy_bootstrap` et `crate::legacy_current_migration_version` ;
|
||||||
|
- `migration_v003.rs` ne traverse plus directement le module privé `crate::migration` ;
|
||||||
|
- `V003SchemaResource` reste réexporté à la racine et est réellement utilisé via `crate::V003SchemaResource` dans le calcul du checksum, au lieu d'être supprimé ou masqué par un `allow` ;
|
||||||
|
- les tests V003 utilisent `crate::current_migration_version` et `crate::V003_RESOURCES` lorsqu'ils ciblent des items `pub(crate)` ;
|
||||||
|
- les champs `pub(crate)` de `V003SchemaResource` reçoivent leur rustdoc adjacente ;
|
||||||
|
- les blocs homogènes de constantes sont remis au format exact attendu par `RUST-FMT-112` ;
|
||||||
|
- aucune annotation `#[allow(unused_imports)]` et aucun marquage artificiel `#[cfg(test)]` ne sont ajoutés.
|
||||||
|
|
||||||
|
## Invariants conservés
|
||||||
|
|
||||||
|
Aucune ressource sous `migrations/v003_raw_transaction_variants/` n'est incluse ou modifiée par ce fix.
|
||||||
|
|
||||||
|
Le checksum V003 reste donc :
|
||||||
|
|
||||||
|
```text
|
||||||
|
3c9cf41877c96944a378abb41ee7a8d8ab7f99fc6dc2bb7a26c295cf8f9fc914
|
||||||
|
```
|
||||||
|
|
||||||
|
Les checksums V000/V001/V002 et les sources historiques `src/migration.rs` / `src/schema.rs` restent inchangés.
|
||||||
|
|
||||||
|
Le fix ne modifie pas les quatre tables, les dix-huit contraintes, les trois index, la politique de backfill, la sémantique `content_hash`, ni le comportement de concurrence V002 -> V003.
|
||||||
|
|
||||||
|
## Validation demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les trois tests PostgreSQL live restent opt-in comme dans `pre.003`.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.004` — backend PostgreSQL V003, bootstrap paresseux de l'identité, ledger de variantes, rattachement exact des nouvelles observations, idempotence et concurrence d'insertion.
|
||||||
79
deltas/0.3.16/pre.003-fix.002.md
Normal file
79
deltas/0.3.16/pre.003-fix.002.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.003-fix.002.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.003-fix.002` — façade crate-root V003 conforme aux audits
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.003-fix.001 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.3.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.003-fix.001` confirme que `cargo check --workspace`, Clippy `-D warnings` et toutes les suites ciblées `ksp-store-postgres-lib` passent. Le fix précédent reste toutefois invalide au niveau des règles du workspace : 53 violations `RUST-DOC-102`, deux violations `RUST-IMPORT-104`, puis quatre candidats dérivés de l'audit d'exports.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette correction modifie du Rust :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.3.fix.2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Corrections
|
||||||
|
|
||||||
|
Le correctif reste structurel et ne modifie aucune ressource SQL V003 :
|
||||||
|
|
||||||
|
- les rustdocs adjacentes de tous les reexports crate-root de `ksp-store-postgres-lib` sont restaurées ;
|
||||||
|
- aucun alias de `use` ou de reexport n'est utilisé ;
|
||||||
|
- les fonctions historiques V000-V002 restent inchangées dans `migration.rs` et sont réexportées sous leurs noms réels `crate::bootstrap` et `crate::current_migration_version` ;
|
||||||
|
- les fonctions de l'extension V003 deviennent explicitement `crate::bootstrap_v003` et `crate::current_migration_version_v003` ;
|
||||||
|
- `runtime.rs` et `health.rs` consomment explicitement les helpers V003 courants ;
|
||||||
|
- `migration_v003.rs` consomme les helpers historiques uniquement via la façade crate-root ;
|
||||||
|
- les canaris de migration distinguent explicitement la version legacy V002 de la version courante V003 ;
|
||||||
|
- le test de complétude confirme que `migration.rs` reste à sa version historique `10` et sans ressource V003.
|
||||||
|
|
||||||
|
## Invariants conservés
|
||||||
|
|
||||||
|
`crates/ksp-store-postgres-lib/src/migration.rs`, `src/schema.rs` et toutes les ressources sous `migrations/v000_*`, `migrations/v001_*`, `migrations/v002_*` et `migrations/v003_raw_transaction_variants/` restent inchangés par ce fix.
|
||||||
|
|
||||||
|
Le checksum V003 reste :
|
||||||
|
|
||||||
|
```text
|
||||||
|
3c9cf41877c96944a378abb41ee7a8d8ab7f99fc6dc2bb7a26c295cf8f9fc914
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun changement n'est apporté aux quatre tables V003, aux dix-huit contraintes, aux trois index, à la politique de backfill, à la sémantique `content_hash` ni à la logique de concurrence V002 -> V003.
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Les scripts Python réels du checkout complet ont été exécutés après application du fix :
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit: clean
|
||||||
|
Rust export completeness audit: 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit: clean
|
||||||
|
Markdown table audit: clean (352 table(s), 947 file(s))
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas disponible dans l'environnement de génération ; les gates Cargo restent donc opérateur.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les trois tests PostgreSQL live restent opt-in comme dans `pre.003`.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.004` — backend PostgreSQL V003, bootstrap paresseux de l'identité, ledger de variantes, rattachement exact des nouvelles observations, idempotence et concurrence d'insertion.
|
||||||
75
deltas/0.3.16/pre.003.md
Normal file
75
deltas/0.3.16/pre.003.md
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.003.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.003` — PostgreSQL V003 RAW transaction variants
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.002-fix.001 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.2.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de la base est confirmé propre : audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings` et toutes les suites `ksp-store-api` passent.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette tranche contient du Rust et du schéma PostgreSQL :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.3
|
||||||
|
```
|
||||||
|
|
||||||
|
## Portée
|
||||||
|
|
||||||
|
Cette tranche livre uniquement la fondation physique V003. Elle ne branche encore aucune écriture métier multi-variante.
|
||||||
|
|
||||||
|
V003 ajoute :
|
||||||
|
|
||||||
|
- `ksp_raw_transaction_variants` ;
|
||||||
|
- `ksp_raw_transaction_canonical_selectors` ;
|
||||||
|
- `ksp_raw_transaction_observation_variants` ;
|
||||||
|
- `ksp_raw_transaction_conflicts` ;
|
||||||
|
- dix-huit contraintes physiques et trois index non uniques ;
|
||||||
|
- un checksum V003 stable calculé par le même domaine `ksp-migration-resources-v1` ;
|
||||||
|
- un bootstrap d'extension privé qui conserve le moteur V000-V002 et applique V003 avec la même clé et le même protocole d'advisory-lock ;
|
||||||
|
- une garde de concurrence qui n'appelle plus le moteur legacy lorsqu'une base est déjà V002 et absorbe uniquement la course où un pair vient de terminer V003 ;
|
||||||
|
- une politique `schema_autoupdate=false` qui valide d'abord l'historique existant avant de rendre `migration_pending`, tout en autorisant la création complète d'une base fraîche ;
|
||||||
|
- des canaris unitaires et d'intégration pour la version courante, le checksum, l'ordre des resources et la frontière de migration.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
`src/migration.rs`, `src/schema.rs` et les ressources SQL V000/V001/V002 ne sont pas modifiés par ce delta.
|
||||||
|
|
||||||
|
Les checksums historiques restent :
|
||||||
|
|
||||||
|
```text
|
||||||
|
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||||
|
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||||
|
V002 ff21605ed45f7ab4c0f92bbb692700b4118a9488b04d50a31d259ac59bdb550e
|
||||||
|
V003 3c9cf41877c96944a378abb41ee7a8d8ab7f99fc6dc2bb7a26c295cf8f9fc914
|
||||||
|
```
|
||||||
|
|
||||||
|
`content_hash` n'est jamais `UNIQUE` et ne constitue pas une preuve d'égalité métier.
|
||||||
|
|
||||||
|
La migration V003 est schema-only : aucun `INSERT` de variantes, aucun rattachement artificiel des observations historiques et aucun `UPDATE` de la projection V001 n'est réalisé. Le bootstrap métier des identités existantes appartient à `pre.004`.
|
||||||
|
|
||||||
|
## Validation demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les tests PostgreSQL live restent opt-in et ne sont pas requis pour ce gate de schéma statique ; `pre.004` ajoutera les preuves backend de bootstrap/écriture sur base réelle.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.004` — backend PostgreSQL V003, bootstrap paresseux de l'identité, ledger de variantes, rattachement exact des nouvelles observations, idempotence et concurrence d'insertion.
|
||||||
127
deltas/0.3.16/pre.004.md
Normal file
127
deltas/0.3.16/pre.004.md
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.004.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.004` — persistance PostgreSQL V003 des variantes RAW
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.003-fix.002 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.3.fix.2
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de la base est confirmé propre : audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings`, 82 tests unitaires `ksp-store-postgres-lib` et toutes les suites d'intégration ciblées passent. Les trois preuves PostgreSQL live restent opt-in.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette tranche modifie le runtime PostgreSQL :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.4
|
||||||
|
```
|
||||||
|
|
||||||
|
## Portée
|
||||||
|
|
||||||
|
Cette tranche branche la persistance RAW existante sur les surfaces V003 créées par `pre.003`, sans introduire encore le comparateur backend-neutral de `pre.005`, la promotion canonique de `pre.006` ni le dossier de conflit durable de `pre.007`.
|
||||||
|
|
||||||
|
Le backend PostgreSQL fournit désormais :
|
||||||
|
|
||||||
|
- un bootstrap paresseux V003 sous le verrou transactionnel de l'identité V001 ;
|
||||||
|
- une variante bootstrap `variant_id = 1` représentant l'état canonique réellement disponible lorsqu'aucun selector V003 n'existe encore ;
|
||||||
|
- un selector canonique initial de revision `1` créé atomiquement avec cette variante ;
|
||||||
|
- la réutilisation d'une variante native uniquement après comparaison exacte de `slot`, `block_time`, format, version, `content_hash` et payload ;
|
||||||
|
- `content_hash` comme préfiltre de recherche, jamais comme preuve d'égalité ;
|
||||||
|
- une allocation monotone et bornée du prochain `variant_id` sous le verrou de l'identité ;
|
||||||
|
- le rattachement atomique de toute nouvelle observation persistée à la variante réellement reçue ;
|
||||||
|
- le rattachement des observations supplémentaires sans payload au selector canonique courant ;
|
||||||
|
- la conservation explicite des observations legacy déjà présentes mais sans mapping V003 : aucune association historique n'est fabriquée ;
|
||||||
|
- la réhydratation de la variante canonique bootstrap lorsqu'un tombstone legacy V001 est réhydraté avec preuve d'identité/hash existante.
|
||||||
|
|
||||||
|
Le cas de compatibilité étroit déjà admis en `0.3.15`, `ActiveIncomingTruncatedLogs`, matérialise maintenant l'entrant tronqué comme variante native distincte et rattache son observation à cette variante sans modifier le canonique.
|
||||||
|
|
||||||
|
## Atomicité et concurrence
|
||||||
|
|
||||||
|
L'acquisition reste une seule transaction PostgreSQL :
|
||||||
|
|
||||||
|
```text
|
||||||
|
BEGIN
|
||||||
|
insert/collision V001
|
||||||
|
lock identité V001 FOR UPDATE
|
||||||
|
bootstrap selector/variant V003 si absent
|
||||||
|
comparaison canonique existante
|
||||||
|
réutilisation/insertion exacte de variante native si nécessaire
|
||||||
|
insertion/idempotence observation
|
||||||
|
rattachement observation -> variante
|
||||||
|
COMMIT
|
||||||
|
```
|
||||||
|
|
||||||
|
Le verrou de l'identité sérialise l'allocation de `variant_id` et empêche deux writers concurrents de créer deux variantes pour un même contenu exact.
|
||||||
|
|
||||||
|
Une preuve PostgreSQL live existante est étendue : après deux acquisitions identiques concurrentes, elle exige exactement une variante, un selector canonique, un mapping d'observation et l'égalité entre la variante mappée et le selector.
|
||||||
|
|
||||||
|
## Rétention
|
||||||
|
|
||||||
|
Cette tranche ne met pas encore le ledger V003 sous une politique de rétention indépendante.
|
||||||
|
|
||||||
|
Une variante native déjà matérialisée peut donc conserver ses bytes `full` même si la projection V001 canonique passe ensuite `archived` ou `purged`. Ce comportement est volontaire : la rétention, les pins, les purge guards et le rollback local des variantes sont réservés à `0.3.17-pre.003`.
|
||||||
|
|
||||||
|
Pour une identité legacy bootstrapée alors que V001 est déjà `archived` ou `purged`, la variante bootstrap reflète uniquement l'état réellement disponible et ne prétend pas reconstruire des bytes perdus.
|
||||||
|
|
||||||
|
## Hors scope maintenu
|
||||||
|
|
||||||
|
Cette tranche n'ajoute pas :
|
||||||
|
|
||||||
|
- `RawTransactionVariantRelation` dans le backend PostgreSQL ;
|
||||||
|
- dominance `CompatibleLessComplete` / `CompatibleMoreComplete` générique ;
|
||||||
|
- promotion ou remplacement du selector canonique ;
|
||||||
|
- écriture dans `ksp_raw_transaction_conflicts` ;
|
||||||
|
- dossier de conflit durable non terminal ;
|
||||||
|
- Store retry Worker ;
|
||||||
|
- inspection/résolution Store Desk ;
|
||||||
|
- rétention propre aux variantes.
|
||||||
|
|
||||||
|
Aucune ressource SQL V003 et aucun fichier historique `migration.rs` / `schema.rs` ne sont modifiés.
|
||||||
|
|
||||||
|
## Tests et canaris
|
||||||
|
|
||||||
|
Les tests unitaires ajoutés verrouillent :
|
||||||
|
|
||||||
|
- allocation `variant_id` monotone et bornée ;
|
||||||
|
- recherche exacte incluant le payload et ne reposant pas sur le hash seul ;
|
||||||
|
- absence de `ON CONFLICT` masquant une collision d'identité de variante ;
|
||||||
|
- absence de fabrication de mapping pour une observation legacy.
|
||||||
|
|
||||||
|
Le canari d'intégration `v003_variant_persistence.rs` verrouille :
|
||||||
|
|
||||||
|
- l'ordre lock -> bootstrap -> comparaison ;
|
||||||
|
- l'ordre observation -> mapping -> commit ;
|
||||||
|
- la comparaison exacte avant réutilisation ;
|
||||||
|
- l'absence de promotion canonique et de conflit durable avant leurs tranches dédiées.
|
||||||
|
|
||||||
|
Les trois tests PostgreSQL live existants sont alignés sur `migration_version = 3` et nettoient désormais aussi les quatre tables V003.
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Les scripts Python réels du checkout complet sont exécutés sur l'état final du delta avant archivage.
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas disponible dans l'environnement de génération ; les gates Cargo restent opérateur.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
La preuve PostgreSQL réelle reste opt-in et peut être exécutée séparément sur une base dédiée lorsque souhaité.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.005` — comparateur partagé backend-neutral `Exact | CompatibleLessComplete | CompatibleMoreComplete | Conflict | Incomparable`, avec canaris `logMessages` bidirectionnels et politique fail-closed sur les autres champs.
|
||||||
77
deltas/0.3.16/pre.005-fix.001.md
Normal file
77
deltas/0.3.16/pre.005-fix.001.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.005-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.005-fix.001` — conformité Clippy et canari de scope `pre.005`
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.005 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.5
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.005` confirme que les audits Rust/export/KSP/Markdown sont propres, que `cargo check --workspace` passe et que les suites ciblées exécutées sont fonctionnellement vertes à l'exception d'un canari PostgreSQL devenu obsolète. Clippy `-D warnings` refuse toutefois treize usages de l'opérateur `?` sous la règle workspace `clippy::question_mark_used` : dix dans le scanner JSON privé du comparateur et trois dans ses helpers de tests.
|
||||||
|
|
||||||
|
Le seul échec de test est `v0_3_16_pre_004_keeps_comparator_and_durable_conflict_scope_deferred`, qui interdit encore toute référence à `RawTransactionVariantRelation::` alors que `pre.005` a précisément introduit le comparateur partagé et ses relations dans le chemin PostgreSQL.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette correction modifie du Rust :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.5.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Corrections
|
||||||
|
|
||||||
|
Le correctif ne modifie aucune règle métier du comparateur :
|
||||||
|
|
||||||
|
- les dix propagations `Option` par `?` du scanner JSON privé sont remplacées par des `match` explicites qui retournent `None` sur le même chemin d'échec ;
|
||||||
|
- les trois usages `?` des helpers de tests Store API sont remplacés par les mêmes retours explicites ;
|
||||||
|
- aucune autorisation locale `#[allow(clippy::question_mark_used)]` n'est ajoutée ;
|
||||||
|
- aucune dépendance runtime n'est ajoutée à `ksp-store-api` ;
|
||||||
|
- le canari V003 hérité de `pre.004` est mis à jour pour exiger désormais l'appel à `ksp_store_api::compare_raw_transaction_variants` et les relations `CompatibleLessComplete` / `CompatibleMoreComplete` ;
|
||||||
|
- ce même canari continue d'interdire l'écriture dans `ksp_raw_transaction_conflicts` et toute promotion du selector canonique avant les tranches prévues.
|
||||||
|
|
||||||
|
## Invariants conservés
|
||||||
|
|
||||||
|
La sémantique de `pre.005` reste inchangée :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact -> idempotence canonique
|
||||||
|
CompatibleLessComplete -> canonique conservé, variante entrante persistée/rattachée
|
||||||
|
CompatibleMoreComplete -> reconnue mais fail-closed jusqu'à pre.006
|
||||||
|
Conflict / Incomparable -> conflit existant sans conflict case durable avant pre.007
|
||||||
|
```
|
||||||
|
|
||||||
|
La preuve `logMessages` reste strictement bidirectionnelle, avec marqueur exact `"Log truncated"`, unique et terminal, préfixe byte-identique et matériel supplémentaire obligatoire du côté complet.
|
||||||
|
|
||||||
|
Aucune migration SQL, ressource V003, promotion canonique, `canonical_revision`, journal de transition ou conflit durable n'est modifié par ce fix.
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Les scripts Python réels du checkout complet sont exécutés sur l'état final puis sur le ZIP rejoué sur une copie fraîche de `pre.005`.
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas disponible dans l'environnement de génération ; les gates Cargo restent opérateur.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les trois preuves PostgreSQL live restent opt-in.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.006` — selector canonique, promotion atomique de `CompatibleMoreComplete`, `canonical_revision`, projection V001 cohérente et conservation de l'ancien canonique.
|
||||||
173
deltas/0.3.16/pre.005.md
Normal file
173
deltas/0.3.16/pre.005.md
Normal file
@@ -0,0 +1,173 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.005.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.005` — comparateur backend-neutral des variantes RAW
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.004 appliquée
|
||||||
|
workspace.package.version = 0.3.16-pre.4
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.004` est confirmé propre : audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings`, 85 tests unitaires `ksp-store-postgres-lib` et toutes les suites d'intégration ciblées passent. Les trois preuves PostgreSQL live restent opt-in.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette tranche modifie le contrat Store API et son intégration PostgreSQL :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.5
|
||||||
|
```
|
||||||
|
|
||||||
|
## Portée
|
||||||
|
|
||||||
|
Cette tranche rend effectif le comparateur pur backend-neutral prévu depuis `pre.002`.
|
||||||
|
|
||||||
|
`ksp-store-api` expose désormais :
|
||||||
|
|
||||||
|
```text
|
||||||
|
compare_raw_transaction_variants(canonical, incoming)
|
||||||
|
-> RawTransactionVariantComparison
|
||||||
|
```
|
||||||
|
|
||||||
|
Le comparateur produit exclusivement les relations déjà figées :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact
|
||||||
|
CompatibleLessComplete
|
||||||
|
CompatibleMoreComplete
|
||||||
|
Conflict
|
||||||
|
Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
Le comparateur n'ajoute aucune dépendance runtime à `ksp-store-api`. En particulier, `serde_json` reste interdit par le canari de dépendances existant. L'analyse du payload RAW v1 utilise un scanner JSON privé, borné en profondeur et fondé uniquement sur `std`.
|
||||||
|
|
||||||
|
## Ordre de décision
|
||||||
|
|
||||||
|
La comparaison est fail-closed et applique l'ordre suivant :
|
||||||
|
|
||||||
|
1. les deux variantes doivent appartenir à la même identité `network + signature`, sinon l'appel est invalide ;
|
||||||
|
2. `slot` différent -> `Conflict / SlotMismatch` ;
|
||||||
|
3. `block_time` différent -> `Conflict / BlockTimeMismatch` ;
|
||||||
|
4. format id/version différent -> `Incomparable / PayloadFormatMismatch` ;
|
||||||
|
5. payload bytes exactement égaux -> `Exact / ExactCanonicalContent` ;
|
||||||
|
6. hash égal mais bytes différents -> `Conflict / ContentHashCollision` ;
|
||||||
|
7. seulement ensuite, analyse structurelle du RAW v1 pour une éventuelle preuve de troncature `logMessages` ;
|
||||||
|
8. toute autre divergence est `Conflict` ou `Incomparable` sans dominance automatique.
|
||||||
|
|
||||||
|
Le `content_hash` reste donc un marqueur d'intégrité/préfiltre, jamais une preuve d'égalité.
|
||||||
|
|
||||||
|
## Preuve `logMessages`
|
||||||
|
|
||||||
|
La seule dominance automatique admise reste la troncature SVM strictement prouvée.
|
||||||
|
|
||||||
|
Pour reconnaître une variante tronquée, toutes les conditions suivantes sont nécessaires :
|
||||||
|
|
||||||
|
- les autres composantes top-level sont identiques ;
|
||||||
|
- les autres champs `meta` sont identiques ;
|
||||||
|
- les deux `logMessages` sont des tableaux de chaînes JSON valides ;
|
||||||
|
- le côté tronqué contient exactement un marqueur exact `"Log truncated"` ;
|
||||||
|
- ce marqueur est le dernier élément ;
|
||||||
|
- le côté complet ne contient aucun marqueur exact ;
|
||||||
|
- le préfixe avant le marqueur est byte-identique ;
|
||||||
|
- le côté complet possède au moins une ligne au-delà de ce préfixe.
|
||||||
|
|
||||||
|
Ainsi :
|
||||||
|
|
||||||
|
```text
|
||||||
|
canonique complet + entrant tronqué -> CompatibleLessComplete
|
||||||
|
canonique tronqué + entrant complet -> CompatibleMoreComplete
|
||||||
|
```
|
||||||
|
|
||||||
|
Une liste plus courte sans marqueur, un préfixe divergent, un marqueur non exact ou du matériel placé après `"Log truncated"` ne prouve aucune dominance.
|
||||||
|
|
||||||
|
## Politique fail-closed des autres champs
|
||||||
|
|
||||||
|
Les différences de présence/omission ou `null` contre une valeur concrète ne sont pas transformées en score de qualité. Elles produisent `Incomparable / UnsupportedCanonicalDifference`.
|
||||||
|
|
||||||
|
Les valeurs concrètes présentes des deux côtés mais contradictoires produisent `Conflict / CanonicalPayloadConflict`.
|
||||||
|
|
||||||
|
Cette politique couvre notamment les champs optionnels documentés dans le plan (`innerInstructions`, `loadedAddresses`, `returnData`, unités consommées, token balances, rewards) sans inventer de hiérarchie de complétude.
|
||||||
|
|
||||||
|
## Intégration PostgreSQL
|
||||||
|
|
||||||
|
`ksp-store-postgres-lib` ne possède plus l'heuristique privée unidirectionnelle `raw_transaction_incoming_truncated_log_messages_compatible`.
|
||||||
|
|
||||||
|
`compare_existing_transaction` appelle désormais directement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ksp_store_api::compare_raw_transaction_variants(...)
|
||||||
|
```
|
||||||
|
|
||||||
|
Comportement de cette tranche :
|
||||||
|
|
||||||
|
- `Exact` -> idempotence canonique existante ;
|
||||||
|
- `CompatibleLessComplete` -> conservation du canonique et persistance/rattachement de la variante entrante comme en `pre.004` ;
|
||||||
|
- `CompatibleMoreComplete` -> relation reconnue mais encore fail-closed côté mutation, car la promotion atomique appartient à `pre.006` ;
|
||||||
|
- `Conflict` / `Incomparable` -> conflit PostgreSQL existant, sans ouverture de conflict case durable avant `pre.007`.
|
||||||
|
|
||||||
|
Aucune mise à jour du selector, de `canonical_revision` ou de la projection V001 n'est ajoutée ici.
|
||||||
|
|
||||||
|
## Canaris
|
||||||
|
|
||||||
|
Les tests Store API ajoutés verrouillent :
|
||||||
|
|
||||||
|
- la relation bidirectionnelle `LessComplete` / `MoreComplete` ;
|
||||||
|
- le marqueur exact, unique et terminal ;
|
||||||
|
- le rejet d'une simple liste plus courte ;
|
||||||
|
- le rejet d'un préfixe divergent ;
|
||||||
|
- le rejet d'un marqueur provider non exact ;
|
||||||
|
- le rejet de matériel après le marqueur ;
|
||||||
|
- la politique `Incomparable` sur optionalité `null`/valeur ;
|
||||||
|
- le `Conflict` sur valeurs concrètes contradictoires ;
|
||||||
|
- l'égalité par bytes même si les hashes fournis diffèrent ;
|
||||||
|
- la détection d'une collision de hash lorsque les bytes diffèrent.
|
||||||
|
|
||||||
|
Les canaris PostgreSQL verrouillent en plus :
|
||||||
|
|
||||||
|
- l'usage exclusif du comparateur Store API pour la décision de convergence ;
|
||||||
|
- l'absence des anciens helpers privés de compatibilité ;
|
||||||
|
- l'absence de promotion/écriture canonique dans `pre.005` ;
|
||||||
|
- le refus temporaire de `CompatibleMoreComplete` jusqu'à `pre.006`.
|
||||||
|
|
||||||
|
## Hors scope maintenu
|
||||||
|
|
||||||
|
Cette tranche n'ajoute pas :
|
||||||
|
|
||||||
|
- promotion du selector canonique ;
|
||||||
|
- incrément de `canonical_revision` ;
|
||||||
|
- remplacement atomique de la projection V001 ;
|
||||||
|
- journal de transition canonique ;
|
||||||
|
- ouverture de `ksp_raw_transaction_conflicts` ;
|
||||||
|
- outcome Worker non terminal pour les conflits ;
|
||||||
|
- Store Desk ;
|
||||||
|
- retry Store/Transport ;
|
||||||
|
- politique de rétention des variantes.
|
||||||
|
|
||||||
|
Aucune migration SQL ni ressource V003 n'est modifiée.
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Les scripts Python réels du checkout complet sont exécutés sur l'état final puis sur le ZIP rejoué sur une copie fraîche de `pre.004`.
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas disponible dans l'environnement de génération ; les gates Cargo restent opérateur.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.006` — selector canonique et promotion atomique avec revision, projection V001 cohérente, conservation de l'ancien canonique et journal minimal des transitions.
|
||||||
126
deltas/0.3.16/pre.006-fix.001.md
Normal file
126
deltas/0.3.16/pre.006-fix.001.md
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.006-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.006-fix.001` — réalignement du canari de promotion
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.006 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.6
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.006` est propre jusqu'aux tests `ksp-store-postgres-lib`. La suite unitaire exécute 87 tests et échoue sur un seul canari hérité de `pre.005` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
v0_3_16_pre_005_shared_comparator_rejects_unproved_shapes_and_defers_more_complete_promotion
|
||||||
|
```
|
||||||
|
|
||||||
|
Les 86 autres tests PostgreSQL exécutés avant l'arrêt sont verts. Les audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings` et toute la suite `ksp-store-api` sont également verts dans le gate fourni.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Le correctif modifie un test Rust et synchronise donc la version Cargo :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.6.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
Réaligner le canari PostgreSQL hérité de `pre.005` sur le contrat désormais implémenté par `pre.006` sans modifier le runtime.
|
||||||
|
|
||||||
|
`pre.005` avait volontairement reconnu :
|
||||||
|
|
||||||
|
```text
|
||||||
|
CompatibleMoreComplete
|
||||||
|
```
|
||||||
|
|
||||||
|
mais différé sa promotion et attendait encore un conflit côté backend PostgreSQL.
|
||||||
|
|
||||||
|
`pre.006` a précisément fermé ce différé : `compare_existing_transaction(...)` projette maintenant cette relation vers :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ExistingTransactionMatch::ActiveCompatibleMoreComplete
|
||||||
|
```
|
||||||
|
|
||||||
|
Le test ancien continuait pourtant d'exiger :
|
||||||
|
|
||||||
|
```text
|
||||||
|
raw_acquisition_content_conflict
|
||||||
|
```
|
||||||
|
|
||||||
|
Le défaut est donc exclusivement un canari obsolète, pas une régression de la promotion canonique.
|
||||||
|
|
||||||
|
## Correction
|
||||||
|
|
||||||
|
Le test concerné conserve toutes ses assertions fail-closed sur les formes non prouvées : mauvais préfixe, matériel après marqueur, différence d'un autre champ metadata et faux marqueur de troncature ne doivent toujours produire ni `CompatibleLessComplete` ni `CompatibleMoreComplete`.
|
||||||
|
|
||||||
|
Pour le seul cas bidirectionnel réellement prouvé — canonique tronqué et entrant strictement plus complet — le test exige désormais :
|
||||||
|
|
||||||
|
```text
|
||||||
|
std::result::Result::Ok(
|
||||||
|
ExistingTransactionMatch::ActiveCompatibleMoreComplete
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Le nom du test est avancé vers `pre.006` afin de ne plus documenter le comportement différé de `pre.005`.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.006-fix.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
## Validations opérateur demandées
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucune logique runtime de `pre.006` n'est modifiée ;
|
||||||
|
- les cas non prouvés restent fail-closed ;
|
||||||
|
- `CompatibleMoreComplete` doit être accepté par la comparaison backend afin que le chemin atomique de promotion de `pre.006` puisse s'exécuter ;
|
||||||
|
- le correctif reste strictement borné au canari obsolète qui bloquait le gate.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.007` — conflit durable minimal + intégration Worker non terminale.
|
||||||
217
deltas/0.3.16/pre.006.md
Normal file
217
deltas/0.3.16/pre.006.md
Normal file
@@ -0,0 +1,217 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.006.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.006` — promotion atomique du canonique RAW
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.005-fix.001 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.5.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur fourni pour la base est propre : audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings`, toutes les suites `ksp-store-api` et `ksp-store-postgres-lib` ciblées passent. Les preuves PostgreSQL live restent opt-in.
|
||||||
|
|
||||||
|
L'audit de l'archive source fournie a relevé un unique artefact ignoré, `scripts/__pycache__/audit_rust_general_rules.cpython-313.pyc`, alors que `.gitignore` exclut `__pycache__/`. Cet artefact n'appartient pas au source ni au delta et est explicitement exclu de l'archive `pre.006`. Aucun autre candidat correspondant aux familles ignorées de `.gitignore` n'a été trouvé.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette tranche modifie le runtime PostgreSQL :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.6
|
||||||
|
```
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
Rendre effectif le sens déjà reconnu par le comparateur partagé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
canonique logMessages tronqué
|
||||||
|
+
|
||||||
|
incoming logMessages complet strictement prouvé
|
||||||
|
=> CompatibleMoreComplete
|
||||||
|
=> variante entrante durable
|
||||||
|
=> promotion canonique atomique
|
||||||
|
=> ancien canonique conservé
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune règle de dominance nouvelle n'est ajoutée. `logMessages` reste la seule preuve automatique de complétude et les autres différences continuent à suivre la politique fail-closed de `pre.005`.
|
||||||
|
|
||||||
|
## Promotion atomique
|
||||||
|
|
||||||
|
Le chemin PostgreSQL `persist_raw_transaction_acquisition` traite désormais `CompatibleMoreComplete` dans la même transaction que l'acquisition :
|
||||||
|
|
||||||
|
```text
|
||||||
|
lock identité V001
|
||||||
|
ensure bootstrap V003
|
||||||
|
compare canonique / incoming
|
||||||
|
persist ou reuse variante incoming exacte
|
||||||
|
charger selector + revision
|
||||||
|
conserver l'ancien canonique
|
||||||
|
mettre à jour la projection V001
|
||||||
|
mettre à jour selector + canonical_revision
|
||||||
|
persist observation
|
||||||
|
lier observation -> variante incoming
|
||||||
|
COMMIT
|
||||||
|
```
|
||||||
|
|
||||||
|
Le selector est modifié avec une garde exacte sur :
|
||||||
|
|
||||||
|
```text
|
||||||
|
transaction_signature
|
||||||
|
canonical_variant_id attendu
|
||||||
|
canonical_revision attendue
|
||||||
|
```
|
||||||
|
|
||||||
|
La nouvelle révision est calculée par incrément borné `u64`; l'épuisement de la révision est fail-closed avant mutation canonique.
|
||||||
|
|
||||||
|
Toute cardinalité inattendue ou changement du selector observé pendant la transaction provoque une erreur et laisse PostgreSQL rollbacker l'ensemble de la promotion.
|
||||||
|
|
||||||
|
## Projection V001 cohérente
|
||||||
|
|
||||||
|
Lors d'une promotion, la projection de compatibilité `ksp_raw_transactions` est remplacée par la représentation entrante complète :
|
||||||
|
|
||||||
|
```text
|
||||||
|
slot
|
||||||
|
block_time_unix_millis
|
||||||
|
format_id
|
||||||
|
format_version
|
||||||
|
content_hash
|
||||||
|
payload
|
||||||
|
retention_state = full
|
||||||
|
```
|
||||||
|
|
||||||
|
Le selector V003 n'est déplacé qu'au sein de la même transaction. Il ne peut donc pas être durablement avancé si la projection V001 échoue, ni inversement.
|
||||||
|
|
||||||
|
## Conservation de l'ancien canonique
|
||||||
|
|
||||||
|
L'ancienne variante canonique n'est jamais supprimée ni écrasée par la nouvelle variante.
|
||||||
|
|
||||||
|
Cas `Full` : l'ancienne variante V003 est confirmée byte-exactement sur slot, block time, format, version, hash et payload avant tout remplacement de projection.
|
||||||
|
|
||||||
|
Cas projection V001 `Archived` : les bytes encore disponibles dans `ksp_raw_transaction_archive_payloads` sont d'abord copiés ou confirmés byte-exacts dans l'ancienne variante V003, avec la même vérification de métadonnées, puis l'archive V001 devenue obsolète est supprimée avant de remettre la projection V001 à `Full` sur le nouveau canonique.
|
||||||
|
|
||||||
|
Cette copie de sauvegarde est une nécessité de promotion/rollback local, pas l'ouverture de la politique générale de rétention des variantes prévue en `0.3.17`.
|
||||||
|
|
||||||
|
Aucune reconstruction réseau n'est utilisée.
|
||||||
|
|
||||||
|
## Révision et historique minimal
|
||||||
|
|
||||||
|
`canonical_revision` devient effectivement monotone lors d'une promotion automatique et constitue la garde durable du selector courant.
|
||||||
|
|
||||||
|
Un journal opérationnel minimal est émis uniquement après le `COMMIT` réussi de l'acquisition. Il porte seulement le réseau, le slot, l'ancien/nouveau `variant_id` et l'ancienne/nouvelle `canonical_revision`; aucun payload ni signature n'est journalisé. Une promotion rollbackée n'est donc jamais annoncée comme commitée.
|
||||||
|
|
||||||
|
Cette tranche n'ajoute pas de table append-only supplémentaire : le journal durable complet des actions/résolutions et les participants de conflit restent explicitement différés par le plan 038 à `0.3.17`. `pre.006` conserve donc le scope V003 déjà figé et n'altère aucun checksum de migration.
|
||||||
|
|
||||||
|
## Conflits toujours différés
|
||||||
|
|
||||||
|
Cette tranche ne modifie pas le comportement durable de :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Conflict
|
||||||
|
Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
Ils restent projetés vers `raw_acquisition_content_conflict` jusqu'à `0.3.16-pre.007`.
|
||||||
|
|
||||||
|
En particulier, `pre.006` n'écrit toujours pas dans :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ksp_raw_transaction_conflicts
|
||||||
|
```
|
||||||
|
|
||||||
|
et ne rend encore aucun conflit non terminal côté Worker.
|
||||||
|
|
||||||
|
## Tests et canaris
|
||||||
|
|
||||||
|
Les tests unitaires PostgreSQL ajoutés verrouillent :
|
||||||
|
|
||||||
|
- l'incrément borné de `canonical_revision` ;
|
||||||
|
- la garde selector par ancien `canonical_variant_id` et ancienne revision ;
|
||||||
|
- la projection V001 complète vers le nouveau canonique `Full` ;
|
||||||
|
- la conservation locale d'une ancienne variante `Archived` ou déjà `Full`.
|
||||||
|
|
||||||
|
Le canari `v003_variant_persistence.rs` est avancé de `pre.005` vers `pre.006` et exige désormais :
|
||||||
|
|
||||||
|
- le chemin `ActiveCompatibleMoreComplete` ;
|
||||||
|
- la persistance de la variante avant promotion ;
|
||||||
|
- la promotion avant l'observation et avant le commit final ;
|
||||||
|
- l'update du selector et de `canonical_revision` ;
|
||||||
|
- l'update de la projection V001 ;
|
||||||
|
- la conservation de l'ancien canonique ;
|
||||||
|
- le journal opérationnel de promotion placé après le commit réussi ;
|
||||||
|
- l'absence persistante d'écriture dans la table de conflits avant `pre.007`.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
|
||||||
|
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.006.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Sur le checkout complet final :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultat : audits Rust/export/KSP et Markdown propres.
|
||||||
|
|
||||||
|
Le ZIP minimal final est ensuite appliqué sur une copie fraîche de `0.3.16-pre.005-fix.001`. Les mêmes audits y restent propres, les cinq fichiers livrés sont byte-identiques au checkout de génération et les 98 ressources de migration PostgreSQL restent byte-identiques à la base.
|
||||||
|
|
||||||
|
Des canaris statiques supplémentaires ont été rejoués sur le source final et sur le replay pour confirmer l'ordre : variante -> promotion -> observation -> mapping -> commit -> journal opérationnel, l'existence des updates selector/projection, la conservation byte-exacte de l'ancien canonique et l'absence d'écriture de conflit.
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération ; `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici. Le gate Cargo propre de la base provient du log opérateur fourni avec `pre.005-fix.001`.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les preuves PostgreSQL live restent opt-in pour cette tranche ; elles pourront être rejouées séparément sur une base dédiée.
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- une promotion automatique n'est admise que pour `CompatibleMoreComplete` déjà prouvé par le comparateur partagé ;
|
||||||
|
- la projection V001 et le selector V003 restent une seule mutation transactionnelle ;
|
||||||
|
- l'ancien canonique est conservé localement avant tout remplacement de projection ;
|
||||||
|
- `canonical_revision` est incrémentée avec garde compare-and-set ;
|
||||||
|
- aucune migration existante n'est modifiée ;
|
||||||
|
- le journal append-only complet reste reporté selon le plan 038.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante pour cette tranche. Les conflict cases durables et l'outcome Worker non terminal appartiennent à `pre.007`.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.007` — conflit durable minimal + intégration Worker : variante divergente conservée, case `Open`, outcome Store non terminal, arbitrage run-local limité à l'autorité du Store et health `Degraded` lorsque requis.
|
||||||
115
deltas/0.3.16/pre.007-fix.001.md
Normal file
115
deltas/0.3.16/pre.007-fix.001.md
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.007-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.007-fix.001` — réalignement des canaris Worker et PostgreSQL
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.007 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.7
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.007` confirme que les audits Rust/export/KSP/Markdown sont propres, que `cargo check --workspace` passe et que les suites Store API et Store façade exécutées avant les défauts sont vertes.
|
||||||
|
|
||||||
|
Deux défauts de tests bloquent ensuite le gate :
|
||||||
|
|
||||||
|
1. un test Worker compare avec `assert_eq!` un `Result<WorkerState, ksp_core_lib::Error>`, alors que `ksp_core_lib::Error` n'implémente volontairement pas `PartialEq` ;
|
||||||
|
2. un test PostgreSQL hérité de `pre.005` attend encore qu'une divergence active soit retournée comme `PostgresBackendErrorKind::Conflict`, alors que `pre.007` la classe désormais comme `ExistingTransactionMatch::ActiveConflict(...)` afin de permettre sa quarantaine durable.
|
||||||
|
|
||||||
|
Aucun des deux défauts ne nécessite une modification runtime.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Le correctif modifie des tests Rust et synchronise donc la version Cargo :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Correction Worker
|
||||||
|
|
||||||
|
Le test `v0_3_16_pre_007_durable_conflict_is_non_terminal_and_degrades_health` conserve exactement la même exigence fonctionnelle : après une quarantaine durable, le Worker reste contrôlable et un stop explicite doit terminer en `WorkerState::Stopped`.
|
||||||
|
|
||||||
|
L'assertion n'utilise plus `assert_eq!` sur le `Result` complet. Elle vérifie la variante de succès avec `matches!`, ce qui ne requiert pas `PartialEq` sur le type d'erreur.
|
||||||
|
|
||||||
|
## Correction PostgreSQL
|
||||||
|
|
||||||
|
Le test `pre_005_existing_full_content_requires_exact_payload_equality_not_hash_only` conserve son objectif historique : un hash identique ne suffit jamais à déclarer deux payloads exacts.
|
||||||
|
|
||||||
|
Sous le contrat `pre.007`, une divergence d'une transaction active n'est toutefois plus une erreur backend terminale. Le comparateur PostgreSQL doit retourner :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ExistingTransactionMatch::ActiveConflict(_)
|
||||||
|
```
|
||||||
|
|
||||||
|
Le test exige désormais ce résultat pour :
|
||||||
|
|
||||||
|
- la fixture volontairement différente en contenu/slot ;
|
||||||
|
- le cas de même hash avec bytes de payload différents.
|
||||||
|
|
||||||
|
Le cas exact reste `ExistingTransactionMatch::Active`, et le comportement particulier des tombstones `Purged` n'est pas modifié.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.007-fix.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
## Validations opérateur demandées
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucune logique runtime de `pre.007` n'est modifiée ;
|
||||||
|
- `ksp_core_lib::Error` n'est pas rendu `PartialEq` uniquement pour satisfaire un test ;
|
||||||
|
- les divergences actives restent explicitement classées `ActiveConflict` afin d'alimenter le chemin de quarantaine durable ;
|
||||||
|
- le test de collision de hash continue d'exiger une comparaison byte-exacte ;
|
||||||
|
- les conflits portant sur une transaction `Purged` conservent leur comportement fail-closed existant.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.008` — hardening concurrence/rollback et régressions finales Store/Backfill/Worker.
|
||||||
95
deltas/0.3.16/pre.007-fix.002.md
Normal file
95
deltas/0.3.16/pre.007-fix.002.md
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.007-fix.002.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.007-fix.002` — réalignement du canari cross-layer Worker/Common RAW/Store
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.007-fix.001 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.007-fix.001` confirme que `cargo check --workspace`, Clippy et les suites Store API, Store façade, PostgreSQL, Backfill ainsi que les 162 tests unitaires Worker passent. Le seul échec restant est `v0_3_13_pre_012_legacy_v0_v1_is_proven_from_transport_through_worker_common_raw_to_store` dans `cross_layer_completeness`.
|
||||||
|
|
||||||
|
Le canari historique cherchait littéralement les chaînes `block_time`, `content_hash`, `format_id`, `format_version` et `slot` dans `worker/src/persistence.rs`. Depuis `pre.007`, le Worker ne reconstruit plus le contenu canonique champ par champ : il reçoit une `RawTransactionAcquisition`, extrait la `RawTransaction` typée avec `into_parts()` et la transmet telle quelle au Store. L'absence de ces noms de champs dans `persistence.rs` n'est donc pas une perte de données.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Correction
|
||||||
|
|
||||||
|
Le canari conserve son objectif de preuve cross-layer mais vérifie désormais les véritables frontières de propriété :
|
||||||
|
|
||||||
|
- Common RAW assemble toujours `RawPayload` avec le format, la version et le hash ;
|
||||||
|
- Common RAW construit toujours `RawTransaction::new(reference, material.slot, block_time, payload)` ;
|
||||||
|
- le Worker extrait `(transaction, observation)` depuis l'acquisition sans reconstruction locale ;
|
||||||
|
- le Worker transmet directement cette `transaction` au port Store en mode `Normal` ;
|
||||||
|
- les canaris existants Store API et façade continuent de vérifier la disponibilité et le dispatch du contrat.
|
||||||
|
|
||||||
|
Aucune logique runtime n'est modifiée.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/cross_layer_completeness.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.007-fix.002.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
## Validations opérateur demandées
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucune reconstruction champ par champ n'est réintroduite dans le Worker uniquement pour satisfaire un canari textuel ;
|
||||||
|
- la preuve est déplacée vers les frontières où les données sont réellement assemblées et transmises ;
|
||||||
|
- la `RawTransaction` typée reste l'unité de passage Common RAW → Worker persistence → Store ;
|
||||||
|
- aucun fichier de migration ni aucun code runtime n'est modifié.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.008`.
|
||||||
96
deltas/0.3.16/pre.007-fix.003.md
Normal file
96
deltas/0.3.16/pre.007-fix.003.md
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.007-fix.003.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.007-fix.003` — réalignement du canari de bornes globales Worker
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.007-fix.002 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.2
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.007-fix.002` confirme que les audits, `cargo check --workspace`, Clippy, Store API, Store façade, PostgreSQL, Backfill, les 162 tests unitaires Worker et les 12 tests `cross_layer_completeness` passent. Le seul échec restant est `v0_3_13_pre_009_global_bounds_and_fairness_stay_inside_worker_facades` dans `dependency_boundary`.
|
||||||
|
|
||||||
|
Le canari historique exigeait encore le symbole `RawTransactionIngestCanonicalState`. Ce type appartenait à l'ancien cache run-local qui conservait une représentation canonique et pouvait participer à l'arbitrage avant Store. `pre.007` a volontairement supprimé cette responsabilité : le cache courant `RawTransactionIngestPersistenceConvergence` est borné et ne fait que sérialiser les acquisitions partageant `(network, signature)` ; chaque acquisition atteint ensuite le Store, qui reste l'unique autorité durable.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.3
|
||||||
|
```
|
||||||
|
|
||||||
|
## Correction
|
||||||
|
|
||||||
|
Le canari de bornes/fairness conserve son objectif historique et vérifie désormais le contrat courant :
|
||||||
|
|
||||||
|
- les budgets d'hydratation restent bornés par les settings Worker ;
|
||||||
|
- les permits restent portés par `tokio::sync::Semaphore` ;
|
||||||
|
- le leader d'hydratation reste privé au Worker ;
|
||||||
|
- la convergence de persistence est représentée par `RawTransactionIngestPersistenceConvergence` ;
|
||||||
|
- `content_conflict_error()` reste le chemin de compatibilité pour un backend legacy ;
|
||||||
|
- aucune dépendance backend/transport physique interdite n'est introduite.
|
||||||
|
|
||||||
|
Aucune logique runtime n'est modifiée.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.007-fix.003.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
## Validations opérateur demandées
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucun type canonique run-local n'est réintroduit pour satisfaire un canari historique ;
|
||||||
|
- le test continue de prouver la boundedness/fairness via le cache de sérialisation réellement utilisé depuis `pre.007` ;
|
||||||
|
- le nom du test historique reste stable afin de ne pas casser les canaris de release completeness ;
|
||||||
|
- aucun code runtime ni fichier de migration n'est modifié.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.008`.
|
||||||
223
deltas/0.3.16/pre.007.md
Normal file
223
deltas/0.3.16/pre.007.md
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.007.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.007` — conflit RAW durable minimal et Worker non terminal
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.006-fix.001 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.6.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur fourni pour cette base est entièrement propre : audits Rust/export/KSP/Markdown, `cargo check --workspace`, Clippy `-D warnings`, les 29 tests unitaires `ksp-store-api`, les 87 tests unitaires `ksp-store-postgres-lib` et les suites d'intégration ciblées passent. Les trois preuves PostgreSQL live restent opt-in et sont ignorées par le gate déterministe normal.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
Cette tranche modifie les contrats Store, le runtime PostgreSQL, le Worker et la projection Backfill :
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.7
|
||||||
|
```
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
Fermer le comportement durable minimal prévu par le plan 038 pour `Conflict` et `Incomparable` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
incoming divergent/incomparable
|
||||||
|
-> variante entrante conservée
|
||||||
|
-> canonique courant inchangé
|
||||||
|
-> conflict case durable Open
|
||||||
|
-> observation liée à la variante entrante
|
||||||
|
-> COMMIT atomique
|
||||||
|
-> outcome QuarantinedConflict
|
||||||
|
-> Worker continue en Running
|
||||||
|
-> health Degraded
|
||||||
|
```
|
||||||
|
|
||||||
|
Cette tranche n'introduit ni résolution manuelle/automatique du conflict case, ni historique append-only complet, ni politique générale de retry Store. Ces responsabilités restent différées selon le plan 038.
|
||||||
|
|
||||||
|
## Contrat Store variant-aware
|
||||||
|
|
||||||
|
`RawAcquisitionWriteOutcome` conserve son constructeur historique `new(entity, observation)` pour les backends et consommateurs existants. Il expose désormais en plus une classification transactionnelle optionnelle :
|
||||||
|
|
||||||
|
```text
|
||||||
|
transaction_variant: Option<RawTransactionVariantWriteOutcome>
|
||||||
|
```
|
||||||
|
|
||||||
|
Le constructeur `with_transaction_variant(...)` permet à un backend transaction-aware de retourner explicitement le verdict de convergence. PostgreSQL renseigne cette classification pour toutes les acquisitions RAW Transaction :
|
||||||
|
|
||||||
|
```text
|
||||||
|
InsertedCanonical
|
||||||
|
ObservedExact
|
||||||
|
ObservedCompatibleLessComplete
|
||||||
|
PromotedCompatibleMoreComplete
|
||||||
|
QuarantinedConflict
|
||||||
|
Rehydrated
|
||||||
|
SkippedPurged
|
||||||
|
```
|
||||||
|
|
||||||
|
`QuarantinedConflict` reste un succès de durabilité Store et non une erreur terminale.
|
||||||
|
|
||||||
|
`ksp-store-lib` réexporte `RawTransactionVariantWriteOutcome` afin que les consumers restent dépendants de la façade et non de `ksp-store-api` directement.
|
||||||
|
|
||||||
|
## Conflict case PostgreSQL
|
||||||
|
|
||||||
|
Aucune migration V003 n'est modifiée : la table `ksp_raw_transaction_conflicts` et ses contraintes appartiennent déjà au schéma figé livré en amont.
|
||||||
|
|
||||||
|
Lorsqu'un comparateur partagé retourne `Conflict` ou `Incomparable`, `persist_raw_transaction_acquisition(...)` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
verrouille l'identité transactionnelle existante
|
||||||
|
persiste/reuse la variante entrante exacte
|
||||||
|
verrouille le conflict case existant FOR UPDATE
|
||||||
|
insère un case Open revision 1 s'il n'existe pas
|
||||||
|
ou valide/met à jour le case existant avec garde de revision
|
||||||
|
persiste l'observation
|
||||||
|
lie observation -> variante entrante
|
||||||
|
COMMIT
|
||||||
|
émet le journal opérationnel minimal après commit
|
||||||
|
retourne QuarantinedConflict
|
||||||
|
```
|
||||||
|
|
||||||
|
Le canonique courant et son selector ne sont pas modifiés par ce chemin.
|
||||||
|
|
||||||
|
Une réobservation strictement identique d'un même case déjà `Open` est idempotente : elle ne fabrique pas une nouvelle révision. Une modification réelle du couple canonique/entrant, de la relation ou du reason code incrémente la révision par `checked_add`; l'épuisement est fail-closed.
|
||||||
|
|
||||||
|
Le conflit reste dans la même transaction que la variante, l'observation et le lien observation-variante. Tout échec avant `COMMIT` laisse PostgreSQL rollbacker l'ensemble.
|
||||||
|
|
||||||
|
## Arbitrage Worker
|
||||||
|
|
||||||
|
La convergence run-local ne conserve plus de représentation locale du contenu canonique et ne peut plus rejeter une divergence avant Store.
|
||||||
|
|
||||||
|
La cache bornée `(network, signature)` reste uniquement un verrou de sérialisation run-local. Chaque acquisition, y compris une réobservation ou une forme divergente, repasse par `RawTransactionWrite::persist_raw_transaction_acquisition(...)`. Le Store reste donc l'unique autorité de comparaison et de convergence durable.
|
||||||
|
|
||||||
|
Le Worker valide la cohérence entre l'outcome entity/observation et l'outcome transactionnel. Pour :
|
||||||
|
|
||||||
|
```text
|
||||||
|
QuarantinedConflict
|
||||||
|
```
|
||||||
|
|
||||||
|
il considère la persistence comme réussie, incrémente `persisted_total` et `content_conflict_total`, reste `Running` et projette une health `Degraded`.
|
||||||
|
|
||||||
|
Un backend legacy qui retourne encore `ERROR_CODE_RAW_CONFLICT` reste converti vers l'erreur Worker terminale historique. Ce chemin est conservé uniquement pour compatibilité ; PostgreSQL `pre.007` n'utilise plus cette erreur pour un conflit actif variant-aware.
|
||||||
|
|
||||||
|
## Projection Backfill
|
||||||
|
|
||||||
|
Le Job Backfill reconnaît également `QuarantinedConflict` comme résultat Store durable. Il projette l'entité vers `BackfillEntityPersistence::Conflict` tout en conservant l'outcome réel de l'observation (`Inserted` ou `AlreadyPresent`).
|
||||||
|
|
||||||
|
L'ancien mapping d'un `ERROR_CODE_RAW_CONFLICT` vers `Conflict + NotRecorded` est conservé pour un backend legacy.
|
||||||
|
|
||||||
|
Le delta ne change pas la politique de progression/frontier Backfill ; le hardening cross-layer final appartient à `pre.008`.
|
||||||
|
|
||||||
|
## Documentation Worker
|
||||||
|
|
||||||
|
Le README et le USAGE du Worker ne décrivent plus la divergence comme nécessairement terminale ni la cache run-local comme autorité de contenu. Ils documentent le passage systématique par Store, la quarantaine durable et la health `Degraded`.
|
||||||
|
|
||||||
|
## Tests et canaris
|
||||||
|
|
||||||
|
Les tests ajoutés/révisés verrouillent notamment :
|
||||||
|
|
||||||
|
- l'exposition publique de la classification transactionnelle dans `RawAcquisitionWriteOutcome` ;
|
||||||
|
- le réexport façade de `RawTransactionVariantWriteOutcome` ;
|
||||||
|
- les SQL conflict `INSERT`, `FOR UPDATE` et update compare-and-set sur `revision` ;
|
||||||
|
- l'ordre transactionnel variante -> conflict case -> observation -> commit -> journal ;
|
||||||
|
- la conservation du canonique lors d'un conflit ;
|
||||||
|
- la suppression de l'arbitrage run-local par contenu dans le Worker ;
|
||||||
|
- le passage de chaque acquisition par Store malgré la cache de sérialisation ;
|
||||||
|
- `QuarantinedConflict` comme succès Worker explicite ;
|
||||||
|
- `Running + Degraded`, `persisted_total + 1`, `content_conflict_total + 1` et absence de `store_failure_total` pour la quarantaine ;
|
||||||
|
- la compatibilité terminale du legacy `ERROR_CODE_RAW_CONFLICT` ;
|
||||||
|
- la projection Backfill explicite du conflit durable avec conservation de l'observation.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-job-backfill-lib/src/persistence.rs
|
||||||
|
crates/ksp-job-backfill-lib/unit_tests/persistence.rs
|
||||||
|
crates/ksp-store-api/src/capability/raw_transaction.rs
|
||||||
|
crates/ksp-store-api/src/model/raw_outcome.rs
|
||||||
|
crates/ksp-store-api/tests/public_api.rs
|
||||||
|
crates/ksp-store-lib/src/lib.rs
|
||||||
|
crates/ksp-store-lib/tests/hardening_completeness.rs
|
||||||
|
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
|
||||||
|
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/README.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/src/persistence.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/src/snapshot.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/cross_layer_completeness.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/hardening.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/persistence.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.007.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation exécutée lors de la génération
|
||||||
|
|
||||||
|
Sur le checkout complet final :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération ; `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici. La base `pre.006-fix.001` dispose en revanche du gate Cargo opérateur entièrement vert fourni avant l'ouverture de cette tranche.
|
||||||
|
|
||||||
|
Le ZIP minimal doit être rejoué sur une copie fraîche de `pre.006-fix.001`, avec réexécution des audits Python, vérification byte-exacte du payload et confirmation que toutes les ressources de migration PostgreSQL restent inchangées.
|
||||||
|
|
||||||
|
## Validation opérateur demandée
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Les preuves PostgreSQL live restent opt-in et ne sont pas requises dans ce delta fonctionnel minimal ; `pre.008` porte le hardening ciblé et les régressions finales prévues par le plan.
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- `Conflict` et `Incomparable` actifs deviennent des résultats durables non terminaux ;
|
||||||
|
- le canonique courant n'est jamais remplacé par un conflit ;
|
||||||
|
- la variante entrante et son observation sont conservées dans la même transaction que le conflict case ;
|
||||||
|
- le conflict case minimal réutilise le schéma V003 existant sans migration ;
|
||||||
|
- la cache Worker ne possède plus d'autorité de contenu supérieure au Store ;
|
||||||
|
- `QuarantinedConflict` dégrade la health mais ne fault pas le Worker ;
|
||||||
|
- les backends legacy peuvent encore omettre l'outcome transactionnel et retourner l'ancien conflit terminal ;
|
||||||
|
- la résolution des conflits, l'historique append-only et le retry Store restent hors scope.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante pour cette tranche.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.008` — hardening ciblé de la vertical slice : concurrence, rollback transactionnel, régressions Store/Backfill/Worker, gates workspace, validation documentaire et préparation de `0.3.16-rel.001`.
|
||||||
132
deltas/0.3.16/pre.008-fix.001.md
Normal file
132
deltas/0.3.16/pre.008-fix.001.md
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.008-fix.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.008-fix.001` — fermeture Clippy du live proof PostgreSQL
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.008 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.8
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.008` confirme que les audits Rust/Markdown, `cargo check --workspace`, les suites Store API, Store façade, PostgreSQL, Job Backfill et Worker sont propres. Le live proof `postgres_raw_transaction_live` passe également sur PostgreSQL 17 avec un URI dédié fourni sur stdin.
|
||||||
|
|
||||||
|
Le seul blocage restant est Clippy workspace/all-targets/all-features sous `-D warnings`, qui détecte trois éléments morts dans le test live :
|
||||||
|
|
||||||
|
```text
|
||||||
|
LIVE_OBSERVATION_EXISTS_SQL
|
||||||
|
LivePersistResult::BackendError(PostgresBackendErrorKind)
|
||||||
|
observation_exists(...)
|
||||||
|
```
|
||||||
|
|
||||||
|
Ces éléments ne participent à aucun scénario `pre.008` et leur suppression ne modifie ni le runtime, ni les migrations, ni les invariants du live proof.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.8.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Correction
|
||||||
|
|
||||||
|
Le test live PostgreSQL est nettoyé comme suit :
|
||||||
|
|
||||||
|
- suppression de `LIVE_OBSERVATION_EXISTS_SQL`, inutilisé ;
|
||||||
|
- suppression du helper privé `observation_exists(...)`, inutilisé ;
|
||||||
|
- `LivePersistResult::BackendError(PostgresBackendErrorKind)` devient `LivePersistResult::BackendError`, car aucun scénario ne lit le `PostgresBackendErrorKind` capturé ;
|
||||||
|
- `persist_once(...)` conserve la distinction succès/erreur mais projette toute erreur backend vers le variant unitaire `BackendError`.
|
||||||
|
|
||||||
|
Les scénarios live ajoutés par `pre.008` restent inchangés, notamment :
|
||||||
|
|
||||||
|
```text
|
||||||
|
concurrence divergente -> InsertedCanonical + QuarantinedConflict
|
||||||
|
rollback tardif -> aucune variante/conflict case orphelin
|
||||||
|
selector -> cohérent avec la projection V001
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.008-fix.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation opérateur déjà observée avant ce fix
|
||||||
|
|
||||||
|
Le live proof suivant a été exécuté avec succès sur PostgreSQL 17 :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
read -rsp 'PostgreSQL URI: ' KSP_TEST_POSTGRES_URI
|
||||||
|
printf '\n'
|
||||||
|
printf '%s\n' "$KSP_TEST_POSTGRES_URI" \
|
||||||
|
| cargo test -p ksp-store-postgres-lib \
|
||||||
|
--test postgres_raw_transaction_live \
|
||||||
|
-- --ignored --nocapture
|
||||||
|
unset KSP_TEST_POSTGRES_URI
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultat observé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
KSP Store RawTransaction live proof: server major 17
|
||||||
|
test pre_009_real_postgres_raw_transaction_vertical_slice_is_atomic_concurrent_and_recoverable ... ok
|
||||||
|
1 passed; 0 failed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
## Gate opérateur demandé
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Le live proof PostgreSQL n'a pas besoin d'être rejoué pour corriger ces warnings, mais peut l'être sur une base dédiée vide si une seconde preuve opérateur est souhaitée.
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucun `#[allow(dead_code)]` ni `#[expect(dead_code)]` n'est ajouté pour masquer les warnings ;
|
||||||
|
- le code mort est retiré plutôt que toléré ;
|
||||||
|
- aucune logique runtime ni migration n'est modifiée ;
|
||||||
|
- le live proof déjà passé sur PostgreSQL 17 conserve ses scénarios et ses invariants.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question bloquante.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : fermeture de `0.3.16-pre.008`, puis `0.3.16-pre.009` pour la réconciliation documentaire finale.
|
||||||
236
deltas/0.3.16/pre.008.md
Normal file
236
deltas/0.3.16/pre.008.md
Normal file
@@ -0,0 +1,236 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.008.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.008` — hardening technique de la vertical slice variantes/conflits
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.007-fix.003 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.7.fix.3
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.007-fix.003` est entièrement propre : format, audits Rust/Markdown, `cargo check --workspace`, Clippy workspace/all-targets/all-features, Store API, Store façade, PostgreSQL, Job Backfill et l'intégralité des suites ciblées du Worker passent. `pre.007` est donc fermé avant ouverture de cette tranche.
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
`pre.008` est la tranche de hardening technique finale de la vertical slice `0.3.16` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
concurrence même identité
|
||||||
|
rollback transactionnel promotion/conflit
|
||||||
|
cohérence projection V001 / selector V003
|
||||||
|
conservation du canonique en conflit
|
||||||
|
régressions Job Backfill / Worker
|
||||||
|
preuves live PostgreSQL opt-in actualisées
|
||||||
|
gate workspace
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune nouvelle capacité fonctionnelle, résolution opérateur, retry Store, reconnexion Transport ou Store Desk n'est introduite.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.8
|
||||||
|
```
|
||||||
|
|
||||||
|
## Hardening PostgreSQL statique
|
||||||
|
|
||||||
|
Les canaris V003 verrouillent désormais explicitement deux invariants supplémentaires :
|
||||||
|
|
||||||
|
1. la branche `ActiveConflict` persiste/réutilise la variante entrante et ouvre/complète le conflict case, mais n'appelle jamais la promotion canonique et ne touche ni `UPDATE_CANONICAL_TRANSACTION_PROJECTION_SQL` ni `UPDATE_TRANSACTION_VARIANT_SELECTOR_SQL` ;
|
||||||
|
2. les helpers de promotion et de conflict case reçoivent la transaction PostgreSQL de l'acquisition et ne peuvent jamais effectuer leur propre `COMMIT`.
|
||||||
|
|
||||||
|
Les gardes existantes restent exigées :
|
||||||
|
|
||||||
|
```text
|
||||||
|
identity row -> FOR UPDATE
|
||||||
|
selector promotion -> expected canonical_variant_id + expected canonical_revision
|
||||||
|
conflict update -> expected revision
|
||||||
|
```
|
||||||
|
|
||||||
|
Ainsi, selector, projection V001, ledger V003, conflict case, observation et mapping restent bornés par la transaction d'acquisition externe.
|
||||||
|
|
||||||
|
## Preuve live PostgreSQL actualisée
|
||||||
|
|
||||||
|
Le test opt-in `postgres_raw_transaction_live.rs` est réaligné sur la sémantique `pre.007`.
|
||||||
|
|
||||||
|
Le scénario de divergence concurrente d'une même identité attend désormais :
|
||||||
|
|
||||||
|
```text
|
||||||
|
1 x InsertedCanonical
|
||||||
|
1 x QuarantinedConflict
|
||||||
|
2 variantes durables
|
||||||
|
2 observations durables
|
||||||
|
2 mappings observation -> variante
|
||||||
|
1 selector canonique
|
||||||
|
1 conflict case Open revision 1
|
||||||
|
projection V001 == variante désignée par le selector
|
||||||
|
```
|
||||||
|
|
||||||
|
L'ordre d'arrivée n'est pas supposé : l'une ou l'autre acquisition peut devenir le premier canonique, mais la seconde doit être quarantinée sans écraser ce canonique.
|
||||||
|
|
||||||
|
### Rollback après ouverture réelle d'un conflit
|
||||||
|
|
||||||
|
Un nouveau scénario live force une erreur après les écritures de divergence mais avant le commit :
|
||||||
|
|
||||||
|
```text
|
||||||
|
canonique existant
|
||||||
|
-> incoming divergent
|
||||||
|
-> variante entrante créée
|
||||||
|
-> conflict case créé
|
||||||
|
-> collision volontaire d'observation avec une autre identité
|
||||||
|
-> erreur avant COMMIT
|
||||||
|
-> rollback implicite de la transaction
|
||||||
|
```
|
||||||
|
|
||||||
|
Après l'échec, la preuve exige exactement l'état antérieur :
|
||||||
|
|
||||||
|
```text
|
||||||
|
1 seule variante pour l'identité
|
||||||
|
1 selector
|
||||||
|
0 conflict case
|
||||||
|
1 observation de seed
|
||||||
|
1 mapping de seed
|
||||||
|
projection V001 toujours cohérente avec le selector
|
||||||
|
```
|
||||||
|
|
||||||
|
Cela couvre le risque qu'un conflict case ou une variante orpheline survive à un échec tardif de l'acquisition.
|
||||||
|
|
||||||
|
Le test live reste `#[ignore]`, lit uniquement un URI PostgreSQL dédié depuis stdin, refuse un schéma KSP préexistant et ne devient pas un gate automatique sans ressource opérateur explicite.
|
||||||
|
|
||||||
|
## Régression Job Backfill
|
||||||
|
|
||||||
|
Le Job Backfill ajoute un canari sur une réobservation déjà idempotente d'un conflit durable :
|
||||||
|
|
||||||
|
```text
|
||||||
|
entity = AlreadyPresent
|
||||||
|
observation = AlreadyPresent
|
||||||
|
variant = QuarantinedConflict
|
||||||
|
```
|
||||||
|
|
||||||
|
La projection doit rester :
|
||||||
|
|
||||||
|
```text
|
||||||
|
BackfillEntityPersistence::Conflict
|
||||||
|
BackfillObservationPersistence::AlreadyPresent
|
||||||
|
```
|
||||||
|
|
||||||
|
Le conflit durable n'est donc jamais aplati en simple `AlreadyPresent`, même lorsque l'observation est elle-même idempotente.
|
||||||
|
|
||||||
|
## Régression Worker
|
||||||
|
|
||||||
|
Le Worker ajoute une preuve runtime avec deux identités successives alors que le Store retourne `QuarantinedConflict` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
première identité -> QuarantinedConflict
|
||||||
|
seconde identité -> QuarantinedConflict
|
||||||
|
```
|
||||||
|
|
||||||
|
Les deux acquisitions doivent atteindre le Store. Après les deux résultats :
|
||||||
|
|
||||||
|
```text
|
||||||
|
state = Running
|
||||||
|
health = Degraded
|
||||||
|
persisted_total = 2
|
||||||
|
content_conflict_total = 2
|
||||||
|
store_failure_total = 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Le Stop explicite doit ensuite terminer en `Stopped`. Cette preuve verrouille directement l'invariant « un conflit durable n'empêche pas la progression d'une autre identité ».
|
||||||
|
|
||||||
|
## Recalibrage du lifecycle de fermeture
|
||||||
|
|
||||||
|
Le plan 038 est ajusté pour respecter les règles `VER-LIFECYCLE` actuelles, qui sont plus strictes que le découpage historique `pre.008 -> rel.001` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
pre.008 = hardening / gate technique
|
||||||
|
pre.009 = réconciliation documentaire finale
|
||||||
|
pre.010 = préparation minimale de publication
|
||||||
|
rel.001 = mécanique de publication stable
|
||||||
|
```
|
||||||
|
|
||||||
|
`pre.008` ne modifie donc ni `CHANGELOG.md`, ni `ROADMAP.md`, ni le prompt de démarrage suivant, et ne fait pas la réconciliation README/USAGE finale.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
crates/ksp-job-backfill-lib/unit_tests/persistence.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||||
|
crates/ksp-store-postgres-lib/tests/v003_variant_persistence.rs
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime.rs
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.008.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations exécutées lors de la génération
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. `cargo fmt`, `cargo check`, Clippy et les tests Cargo ne sont donc pas déclarés exécutés ici.
|
||||||
|
|
||||||
|
La preuve PostgreSQL live reste volontairement opt-in et n'est pas déclarée exécutée sans URI isolé fourni par l'opérateur.
|
||||||
|
|
||||||
|
## Gate opérateur demandé
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Le live PostgreSQL peut être exécuté séparément sur une base dédiée vide :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture
|
||||||
|
```
|
||||||
|
|
||||||
|
Il ne doit jamais être lancé contre une base KSP contenant des données utiles.
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- aucune modification runtime n'est nécessaire pour fermer le hardening observé ;
|
||||||
|
- le conflit durable concurrent est un succès quarantiné, pas une erreur backend ;
|
||||||
|
- les deux observations concurrentes divergentes doivent être conservées ;
|
||||||
|
- un échec tardif avant commit doit annuler variante et conflict case nouvellement créés ;
|
||||||
|
- le canonique ne change jamais dans la branche de conflit ;
|
||||||
|
- Backfill conserve explicitement la distinction conflit / observation idempotente ;
|
||||||
|
- le Worker doit continuer à traiter les identités suivantes après une quarantaine ;
|
||||||
|
- le lifecycle final est séparé en `pre.008`, `pre.009`, `pre.010`, puis `rel.001` conformément aux règles actuelles.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question fonctionnelle bloquante pour `0.3.16`.
|
||||||
|
|
||||||
|
Le test PostgreSQL live reste conditionné à la disponibilité d'un PostgreSQL isolé ; son absence n'autorise pas à prétendre qu'il a été exécuté.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate technique propre : `0.3.16-pre.009` — réconciliation documentaire finale de `0.3.16`, sans nouveau comportement runtime.
|
||||||
181
deltas/0.3.16/pre.009.md
Normal file
181
deltas/0.3.16/pre.009.md
Normal file
@@ -0,0 +1,181 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.009.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.009` — réconciliation documentaire finale
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.008-fix.001 appliqué
|
||||||
|
workspace.package.version = 0.3.16-pre.8.fix.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.008-fix.001` est entièrement propre : format, audits Rust/Markdown, `cargo check --workspace`, Clippy workspace/all-targets/all-features avec `-D warnings`, Store API, Store façade, PostgreSQL, Job Backfill et Worker passent. Le live proof `postgres_raw_transaction_live` a également été exécuté séparément avec succès sur PostgreSQL 17.
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
`pre.009` est exclusivement la lane de réconciliation documentaire finale de `0.3.16`.
|
||||||
|
|
||||||
|
Elle aligne les références durables sur l'état réellement implémenté et validé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
V003 multi-variantes
|
||||||
|
selector canonique + promotion atomique
|
||||||
|
comparateur partagé fail-closed
|
||||||
|
conflict case minimal durable
|
||||||
|
QuarantinedConflict non terminal côté Worker
|
||||||
|
projection Conflict côté Backfill
|
||||||
|
hardening concurrence / rollback / PostgreSQL live
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun code runtime, test fonctionnel, migration, configuration, `CHANGELOG.md`, `ROADMAP.md` ou prompt de démarrage suivant n'est modifié.
|
||||||
|
|
||||||
|
## Version
|
||||||
|
|
||||||
|
```text
|
||||||
|
workspace.package.version = 0.3.16-pre.9
|
||||||
|
```
|
||||||
|
|
||||||
|
## Plan et validation
|
||||||
|
|
||||||
|
Le plan 038 est réconcilié avec les gates réellement observés :
|
||||||
|
|
||||||
|
- `pre.005` à `pre.008-fix.001` sont décrits avec leur état livré/validé ;
|
||||||
|
- `pre.008` enregistre le live PostgreSQL 17 comme PASS ;
|
||||||
|
- `0.3.16` ne revendique ni résolution complète, ni retry Store, ni reconnect configurable, ni Store Desk ;
|
||||||
|
- les lanes de fermeture `0.3.17` et `0.3.18` sont séparées conformément aux règles actuelles entre gate technique, réconciliation documentaire et préparation de publication.
|
||||||
|
|
||||||
|
La validation 033 ajoute la matrice finale `0.3.16`, le gate opérateur complet de `pre.008-fix.001`, la preuve PostgreSQL réelle, les scopes explicitement reportés et l'état de fermeture avant `pre.010`.
|
||||||
|
|
||||||
|
## Références d'architecture
|
||||||
|
|
||||||
|
Les références durables sont réalignées sur le découpage courant :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16 fondations RAW multi-variantes + conflit durable minimal + Worker non terminal
|
||||||
|
0.3.17 lifecycle/résolution/rétention variantes + résilience Store/Transport
|
||||||
|
0.3.18 Store Desk variantes/conflits/résolutions
|
||||||
|
0.3.19 cible Backfill multi-route/multi-stratégie
|
||||||
|
0.3.20 cible Backfill Desk multi-route
|
||||||
|
```
|
||||||
|
|
||||||
|
Le handoff historique `037` reste inchangé ; il est explicitement présenté comme historique et supersédé pour le sizing par le plan 038.
|
||||||
|
|
||||||
|
## Documentation Store
|
||||||
|
|
||||||
|
`ksp-store-lib` et `ksp-store-postgres-lib` documentent désormais :
|
||||||
|
|
||||||
|
- les 14 capabilities RAW réellement implémentées, dont les quatre inspections ;
|
||||||
|
- V003 et ses quatre tables ;
|
||||||
|
- les outcomes de variantes ;
|
||||||
|
- `QuarantinedConflict` comme succès durable ;
|
||||||
|
- la promotion atomique `CompatibleMoreComplete` ;
|
||||||
|
- le mapping observation -> variante réellement reçue ;
|
||||||
|
- la conservation du canonique sur conflit/incomparable ;
|
||||||
|
- la portée volontairement minimale du conflict case `0.3.16`.
|
||||||
|
|
||||||
|
## Documentation Worker / Backfill
|
||||||
|
|
||||||
|
Le Worker documente comme contrat stable `0.3.16` que la sérialisation run-local ne décide plus du contenu et que toute acquisition atteint le Store ; `QuarantinedConflict` laisse le Worker `Running/Degraded` et n'empêche pas l'identité suivante.
|
||||||
|
|
||||||
|
Le Job Backfill documente la projection `QuarantinedConflict -> Conflict` tout en conservant l'outcome réel de l'observation. Les deux références Backfill utilisent désormais la terminologie `STRUCTURAL` au lieu de l'ancien nom de couche `CORE`.
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
docs/000-README.md
|
||||||
|
docs/architecture/004-COMPONENT_INVENTORY.md
|
||||||
|
docs/architecture/009-ACQUISITION_WORKERS_AND_JOBS.md
|
||||||
|
docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
|
||||||
|
crates/ksp-store-lib/README.md
|
||||||
|
crates/ksp-store-lib/USAGE.md
|
||||||
|
crates/ksp-store-postgres-lib/README.md
|
||||||
|
crates/ksp-store-postgres-lib/USAGE.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/README.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md
|
||||||
|
crates/ksp-job-backfill-lib/README.md
|
||||||
|
crates/ksp-job-backfill-lib/USAGE.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichier ajouté
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/pre.009.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation opérateur de la base
|
||||||
|
|
||||||
|
Le gate `pre.008-fix.001` fourni par l'opérateur est entièrement vert. Les résultats détaillés sont désormais consignés dans `docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md`.
|
||||||
|
|
||||||
|
Le live PostgreSQL séparé a produit :
|
||||||
|
|
||||||
|
```text
|
||||||
|
server major 17
|
||||||
|
1 passed; 0 failed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations de génération
|
||||||
|
|
||||||
|
Exécutées dans l'environnement de génération :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultats :
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit: clean
|
||||||
|
Rust export completeness audit: 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit: clean
|
||||||
|
Markdown table audit: clean (352 table(s), 959 file(s))
|
||||||
|
```
|
||||||
|
|
||||||
|
Le toolchain Rust n'est pas installé dans l'environnement de génération. Le changement de version Cargo impose donc au gate opérateur de rejouer format/check/Clippy/tests, même si cette tranche ne modifie aucun source Rust.
|
||||||
|
|
||||||
|
## Gate opérateur demandé
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun live PostgreSQL supplémentaire n'est requis par `pre.009`, car cette tranche ne modifie aucun test/runtime et le live `pre.008` a déjà été exécuté avec succès.
|
||||||
|
|
||||||
|
## Décisions prises
|
||||||
|
|
||||||
|
- la documentation finale de `0.3.16` décrit uniquement les capacités effectivement livrées ;
|
||||||
|
- `0.3.17` et `0.3.18` conservent explicitement les responsabilités non implémentées ;
|
||||||
|
- les extensions Backfill restent reportées par défaut à `0.3.19`/`0.3.20` ;
|
||||||
|
- le handoff historique n'est pas réécrit ;
|
||||||
|
- `CHANGELOG.md`, `ROADMAP.md` et le prompt `0.3.17` restent réservés à `pre.010`.
|
||||||
|
|
||||||
|
## Questions ouvertes
|
||||||
|
|
||||||
|
Aucune question documentaire bloquante pour `0.3.16`.
|
||||||
|
|
||||||
|
## Suite
|
||||||
|
|
||||||
|
Après gate propre : `0.3.16-pre.010`, préparation de publication minimale uniquement (`CHANGELOG.md`, `ROADMAP.md`, prompt de démarrage `0.3.17`, version et delta).
|
||||||
100
deltas/0.3.16/pre.010.md
Normal file
100
deltas/0.3.16/pre.010.md
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
<!-- file: deltas/0.3.16/pre.010.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-pre.010`
|
||||||
|
|
||||||
|
## Base
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.009
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate opérateur de `pre.009` est entièrement vert : audits Rust/Markdown, `cargo check --workspace`, Clippy workspace/all-targets/all-features en `-D warnings`, suites Store API, Store façade, PostgreSQL, Job Backfill et Worker jusqu'à `release_completeness`.
|
||||||
|
|
||||||
|
## Objectif
|
||||||
|
|
||||||
|
Dernière prerelease de `0.3.16`, limitée strictement à la préparation de publication conformément à `VER-LIFECYCLE-003` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
CHANGELOG.md
|
||||||
|
ROADMAP.md
|
||||||
|
prompt de démarrage 0.3.17
|
||||||
|
delta pre.010
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun code, test fonctionnel, README/USAGE, plan, validation, architecture, règle, schema, config ou migration n'est modifié.
|
||||||
|
|
||||||
|
## Changements
|
||||||
|
|
||||||
|
### Version workspace
|
||||||
|
|
||||||
|
La version devient :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.10
|
||||||
|
```
|
||||||
|
|
||||||
|
### `CHANGELOG.md`
|
||||||
|
|
||||||
|
Ajout de l'entrée stable `0.3.16`, limitée au périmètre réellement livré :
|
||||||
|
|
||||||
|
- fondations RAW multi-variantes ;
|
||||||
|
- V003 additive ;
|
||||||
|
- comparateur fail-closed partagé ;
|
||||||
|
- sélecteur canonique et promotion atomique strictement prouvée ;
|
||||||
|
- conflit durable minimal et quarantaine non terminale ;
|
||||||
|
- continuité Worker `Running/Degraded` ;
|
||||||
|
- compatibilité Backfill ;
|
||||||
|
- hardening concurrence/rollback ;
|
||||||
|
- preuve PostgreSQL 17 réellement exécutée ;
|
||||||
|
- reports explicites vers `0.3.17`/`0.3.18`.
|
||||||
|
|
||||||
|
### `ROADMAP.md`
|
||||||
|
|
||||||
|
- `0.3.16` passe à `[X]` avec son périmètre réel ;
|
||||||
|
- `0.3.17` devient la résilience opérationnelle et le cycle de vie complet des variantes ;
|
||||||
|
- `0.3.18` devient la Store Desk d'inspection/résolution ;
|
||||||
|
- le Backfill multi-route et sa Desk sont repoussés à `0.3.19` et `0.3.20` ;
|
||||||
|
- le rappel de trajectoire issu de l'audit `0.3.9` est réaligné sur ce découpage.
|
||||||
|
|
||||||
|
### Prompt `0.3.17`
|
||||||
|
|
||||||
|
Création de :
|
||||||
|
|
||||||
|
```text
|
||||||
|
prompts/036-V0_3_17_START_PROMPT.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Le prompt impose une ouverture depuis le futur tag stable `v0.3.16` et réserve `pre.001` à l'audit/sizing des contrats d'inspection/résolution, du cycle de vie des conflict cases, des revisions/races, de la rétention/pinning, de la classification Store `Transient/Terminal`, du retry Worker et de la reconnexion Transport configurable avant toute implémentation lourde.
|
||||||
|
|
||||||
|
## Fichiers livrés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
CHANGELOG.md
|
||||||
|
ROADMAP.md
|
||||||
|
prompts/036-V0_3_17_START_PROMPT.md
|
||||||
|
deltas/0.3.16/pre.010.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations de génération
|
||||||
|
|
||||||
|
Exécutées sur la livraison assemblée :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate Cargo final reste à exécuter par l'opérateur après application du delta.
|
||||||
|
|
||||||
|
## Décisions de fermeture
|
||||||
|
|
||||||
|
`pre.010` ne rouvre aucun document durable réconcilié en `pre.009` et ne corrige aucun comportement. Toute anomalie fonctionnelle ou documentaire durable découverte après cette tranche doit repasser par un couloir de prerelease approprié avant `rel.001`, conformément à `VER-LIFECYCLE-010/012`.
|
||||||
|
|
||||||
|
Après gate propre de `pre.010`, la livraison suivante attendue est :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-rel.001
|
||||||
|
```
|
||||||
184
deltas/0.3.16/rel.001.md
Normal file
184
deltas/0.3.16/rel.001.md
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
<!-- file: deltas/0.3.16/rel.001.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Delta `0.3.16-rel.001` — publication stable
|
||||||
|
|
||||||
|
## Base requise
|
||||||
|
|
||||||
|
Base directe attendue :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.010
|
||||||
|
workspace.package.version = 0.3.16-pre.10
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate minimal opérateur de `pre.010`, exécuté le **22 septembre 2026 à 09:09–09:11**, est propre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
cargo fmt --all : PASS
|
||||||
|
cargo fmt --all -- --check : PASS
|
||||||
|
General Rust rule audit : clean
|
||||||
|
Rust export completeness audit : 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit : clean
|
||||||
|
Markdown table audit : clean (352 table(s), 961 file(s))
|
||||||
|
cargo check --workspace : PASS
|
||||||
|
```
|
||||||
|
|
||||||
|
Les couloirs de fermeture précédents sont fermés :
|
||||||
|
|
||||||
|
- `pre.008`, corrigée par `pre.008-fix.001` : hardening technique final, concurrence/rollback, non-régressions Store/Backfill/Worker et preuve live PostgreSQL 17 ;
|
||||||
|
- `pre.009` : réconciliation documentaire finale des surfaces `0.3.16` ;
|
||||||
|
- `pre.010` : préparation minimale de publication avec `CHANGELOG.md`, `ROADMAP.md` et `prompts/036-V0_3_17_START_PROMPT.md`.
|
||||||
|
|
||||||
|
## Objet
|
||||||
|
|
||||||
|
Publier mécaniquement la version stable :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16
|
||||||
|
```
|
||||||
|
|
||||||
|
Conformément à `VER-LIFECYCLE-012`, cette tranche ne corrige ni code, ni test, ni documentation durable, ni architecture, ni configuration, ni dépendance, ni prompt.
|
||||||
|
|
||||||
|
Tout défaut nouveau renvoie vers une prerelease appropriée ; `rel.001` n'est pas une tranche de rattrapage.
|
||||||
|
|
||||||
|
## Modification
|
||||||
|
|
||||||
|
### `Cargo.toml`
|
||||||
|
|
||||||
|
Le header est incrémenté parce que le fichier est réellement modifié :
|
||||||
|
|
||||||
|
```text
|
||||||
|
650 -> 651
|
||||||
|
```
|
||||||
|
|
||||||
|
La version workspace devient :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16-pre.10 -> 0.3.16
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune autre ligne du `Cargo.toml` racine n'est modifiée.
|
||||||
|
|
||||||
|
## Fichiers ajoutés
|
||||||
|
|
||||||
|
```text
|
||||||
|
deltas/0.3.16/rel.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers modifiés
|
||||||
|
|
||||||
|
```text
|
||||||
|
Cargo.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fichiers supprimés
|
||||||
|
|
||||||
|
```text
|
||||||
|
aucun
|
||||||
|
```
|
||||||
|
|
||||||
|
## Surfaces explicitement inchangées
|
||||||
|
|
||||||
|
```text
|
||||||
|
CHANGELOG.md
|
||||||
|
ROADMAP.md
|
||||||
|
README.md
|
||||||
|
RULES.md
|
||||||
|
prompts/036-V0_3_17_START_PROMPT.md
|
||||||
|
crates/**
|
||||||
|
docs/**
|
||||||
|
config/**
|
||||||
|
```
|
||||||
|
|
||||||
|
La surface préparée en `pre.010` reste byte-identique hors mécanique Cargo et ajout du présent delta.
|
||||||
|
|
||||||
|
## Surface stable publiée
|
||||||
|
|
||||||
|
`0.3.16` stabilise les fondations RAW multi-variantes et la convergence durable de plusieurs représentations `RawTransaction` pour une même identité `(network, signature)` :
|
||||||
|
|
||||||
|
- migration PostgreSQL V003 additive avec variantes, sélecteur canonique, lien observation-variante et conflict cases minimaux, tout en conservant V000/V001/V002 inchangés ;
|
||||||
|
- comparateur backend-neutral partagé `Exact / CompatibleLessComplete / CompatibleMoreComplete / Conflict / Incomparable` ;
|
||||||
|
- dominance fail-closed : seule la troncature `logMessages` explicitement prouvée établit aujourd'hui une relation de complétude ;
|
||||||
|
- égalité fondée sur les bytes lorsque disponibles, sans faire de `content_hash` une preuve suffisante ;
|
||||||
|
- décision sérialisée sous verrou de l'identité V001 ;
|
||||||
|
- promotion atomique d'une variante `CompatibleMoreComplete`, avec conservation de l'ancien canonique et cohérence entre projection V001 et sélecteur ;
|
||||||
|
- conservation durable des divergences `Conflict`/`Incomparable`, ouverture/réouverture d'un conflict case et enregistrement de l'observation sans mutation du canonique ;
|
||||||
|
- `QuarantinedConflict` comme succès durable non terminal côté Worker : le Worker reste `Running`, sa health devient `Degraded` et chaque acquisition continue d'atteindre le Store ;
|
||||||
|
- projection Backfill du conflit durable sans perdre l'idempotence réelle de l'observation ;
|
||||||
|
- hardening final des races, du rollback transactionnel et de la cohérence selector/projection ;
|
||||||
|
- preuve live PostgreSQL 17 des scénarios RawTransaction atomiques, concurrents et récupérables.
|
||||||
|
|
||||||
|
La release ne revendique pas encore la résolution opérateur complète, le journal/participants complets, la rétention/pinning des variantes nécessaires à une résolution, le retry Store/backpressure `Blocked`, la reconnexion Transport configurable ni les vues Store Desk de résolution. Ces responsabilités sont reportées conformément au `ROADMAP.md` et au prompt suivant.
|
||||||
|
|
||||||
|
## Suite préparée
|
||||||
|
|
||||||
|
La session suivante part exclusivement de la base stable :
|
||||||
|
|
||||||
|
```text
|
||||||
|
v0.3.16
|
||||||
|
```
|
||||||
|
|
||||||
|
et exécute :
|
||||||
|
|
||||||
|
```text
|
||||||
|
prompts/036-V0_3_17_START_PROMPT.md
|
||||||
|
```
|
||||||
|
|
||||||
|
pour ouvrir :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.17 — résilience opérationnelle et cycle de vie complet des variantes RAW
|
||||||
|
```
|
||||||
|
|
||||||
|
`0.3.17-pre.001` doit commencer par l'audit/sizing des contrats backend-neutral d'inspection/résolution, du cycle de vie revisionné des conflict cases, des races et invariants, de la rétention/pinning, de la classification Store `Transient / Terminal`, du retry Worker et de la reconnexion Transport configurable avant toute implémentation lourde.
|
||||||
|
|
||||||
|
La trajectoire publiée est :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.17 résilience opérationnelle + cycle de vie complet des variantes
|
||||||
|
0.3.18 Store Desk inspection / résolution des conflits
|
||||||
|
0.3.19 Backfill multi-route / multi-stratégie
|
||||||
|
0.3.20 adaptation correspondante de Backfill Desk
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validations d'assemblage
|
||||||
|
|
||||||
|
Sur l'état candidat `rel.001`, avant packaging :
|
||||||
|
|
||||||
|
```text
|
||||||
|
General Rust rule audit : clean
|
||||||
|
Rust export completeness audit : 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit : clean
|
||||||
|
Markdown table audit : clean
|
||||||
|
```
|
||||||
|
|
||||||
|
Les contrôles d'archive et la comparaison byte-exact sont exécutés après packaging avant livraison.
|
||||||
|
|
||||||
|
Aucun PASS Cargo post-bump stable n'est revendiqué localement faute de toolchain dans l'environnement d'assemblage.
|
||||||
|
|
||||||
|
## Gate demandé après application
|
||||||
|
|
||||||
|
La tranche est purement mécanique. Après application sur un `pre.010` dont le gate préalable est vert :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.16
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Après gate propre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
commit : v0.3.16-rel.001
|
||||||
|
tag : v0.3.16
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun tag de prerelease ou `rel.001` n'est requis.
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: docs/000-README.md -->
|
<!-- file: docs/000-README.md -->
|
||||||
<!-- version: 74 -->
|
<!-- version: 75 -->
|
||||||
|
|
||||||
# Documentation KSP
|
# Documentation KSP
|
||||||
|
|
||||||
@@ -133,7 +133,7 @@ Le plan historique de la phase fondatrice clôturée est conservé dans [`plans/
|
|||||||
|
|
||||||
La candidate `0.3.9 — Worker API générique + audit RAW Transaction` est réconciliée dans [`plans/030-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT_PLAN.md`](plans/030-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT_PLAN.md) avec [`validation/026-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT.md`](validation/026-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT.md). `ksp-worker-api` reste Core-only et runtime-neutral pour les services continus ; l'architecture RAW place Store/`RawTransaction` au centre et sépare strictement le Job Backfill historique paramétré du futur Worker Ingest continu start/stop. [`../crates/ksp-worker-api/README.md`](../crates/ksp-worker-api/README.md), [`../crates/ksp-worker-api/USAGE.md`](../crates/ksp-worker-api/USAGE.md), [`architecture/009-ACQUISITION_WORKERS_AND_JOBS.md`](architecture/009-ACQUISITION_WORKERS_AND_JOBS.md) et [`architecture/011-RAW_TRANSACTION_ACQUISITION.md`](architecture/011-RAW_TRANSACTION_ACQUISITION.md) constituent les références durables avant la lane de publication.
|
La candidate `0.3.9 — Worker API générique + audit RAW Transaction` est réconciliée dans [`plans/030-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT_PLAN.md`](plans/030-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT_PLAN.md) avec [`validation/026-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT.md`](validation/026-V0_3_9_WORKER_API_RAW_TRANSACTION_AUDIT.md). `ksp-worker-api` reste Core-only et runtime-neutral pour les services continus ; l'architecture RAW place Store/`RawTransaction` au centre et sépare strictement le Job Backfill historique paramétré du futur Worker Ingest continu start/stop. [`../crates/ksp-worker-api/README.md`](../crates/ksp-worker-api/README.md), [`../crates/ksp-worker-api/USAGE.md`](../crates/ksp-worker-api/USAGE.md), [`architecture/009-ACQUISITION_WORKERS_AND_JOBS.md`](architecture/009-ACQUISITION_WORKERS_AND_JOBS.md) et [`architecture/011-RAW_TRANSACTION_ACQUISITION.md`](architecture/011-RAW_TRANSACTION_ACQUISITION.md) constituent les références durables avant la lane de publication.
|
||||||
|
|
||||||
Les candidates `0.3.10` à `0.3.15` prolongent cette trajectoire RAW dans [`plans/031-V0_3_10_RAW_TRANSACTION_INGEST_PLAN.md`](plans/031-V0_3_10_RAW_TRANSACTION_INGEST_PLAN.md), [`plans/032-V0_3_11_RAW_TRANSACTION_INGEST_WORKER_FOUNDATION_PLAN.md`](plans/032-V0_3_11_RAW_TRANSACTION_INGEST_WORKER_FOUNDATION_PLAN.md), [`plans/033-V0_3_12_YELLOWSTONE_HYDRATION_CONTINUITY_PLAN.md`](plans/033-V0_3_12_YELLOWSTONE_HYDRATION_CONTINUITY_PLAN.md), [`plans/034-V0_3_13_MULTI_SOURCE_LIVE_CONVERGENCE_PLAN.md`](plans/034-V0_3_13_MULTI_SOURCE_LIVE_CONVERGENCE_PLAN.md), [`plans/035-V0_3_14_MULTI_SOURCE_GAP_REPAIR_HARDENING_PLAN.md`](plans/035-V0_3_14_MULTI_SOURCE_GAP_REPAIR_HARDENING_PLAN.md) et [`plans/036-V0_3_15_RAW_TRANSACTION_INGEST_DESK_PLAN.md`](plans/036-V0_3_15_RAW_TRANSACTION_INGEST_DESK_PLAN.md), avec leurs matrices [`validation/027-V0_3_10_RAW_TRANSACTION_INGEST.md`](validation/027-V0_3_10_RAW_TRANSACTION_INGEST.md) à [`validation/032-V0_3_15_RAW_TRANSACTION_INGEST_DESK.md`](validation/032-V0_3_15_RAW_TRANSACTION_INGEST_DESK.md). Cette séquence matérialise le Common RAW partagé, la fondation Worker, Yellowstone live, cinq familles multi-source, la continuité/gap repair puis le Desk multi-route ; `0.3.15` ferme en plus les anomalies Mainnet observées de convergence `logMessages`, backpressure Yellowstone soutenue et shutdown post-half-close. Le handoff [`plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md`](plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md) réserve explicitement à `0.3.16` les variantes RAW durables, la résolution des conflits, les retries Store/Transport et les vues correspondantes de Store Desk ; `0.3.17` et `0.3.18` portent ensuite le Backfill multi-route et son Desk adapté.
|
Les candidates `0.3.10` à `0.3.15` prolongent cette trajectoire RAW dans [`plans/031-V0_3_10_RAW_TRANSACTION_INGEST_PLAN.md`](plans/031-V0_3_10_RAW_TRANSACTION_INGEST_PLAN.md), [`plans/032-V0_3_11_RAW_TRANSACTION_INGEST_WORKER_FOUNDATION_PLAN.md`](plans/032-V0_3_11_RAW_TRANSACTION_INGEST_WORKER_FOUNDATION_PLAN.md), [`plans/033-V0_3_12_YELLOWSTONE_HYDRATION_CONTINUITY_PLAN.md`](plans/033-V0_3_12_YELLOWSTONE_HYDRATION_CONTINUITY_PLAN.md), [`plans/034-V0_3_13_MULTI_SOURCE_LIVE_CONVERGENCE_PLAN.md`](plans/034-V0_3_13_MULTI_SOURCE_LIVE_CONVERGENCE_PLAN.md), [`plans/035-V0_3_14_MULTI_SOURCE_GAP_REPAIR_HARDENING_PLAN.md`](plans/035-V0_3_14_MULTI_SOURCE_GAP_REPAIR_HARDENING_PLAN.md) et [`plans/036-V0_3_15_RAW_TRANSACTION_INGEST_DESK_PLAN.md`](plans/036-V0_3_15_RAW_TRANSACTION_INGEST_DESK_PLAN.md), avec leurs matrices [`validation/027-V0_3_10_RAW_TRANSACTION_INGEST.md`](validation/027-V0_3_10_RAW_TRANSACTION_INGEST.md) à [`validation/032-V0_3_15_RAW_TRANSACTION_INGEST_DESK.md`](validation/032-V0_3_15_RAW_TRANSACTION_INGEST_DESK.md). Cette séquence matérialise le Common RAW partagé, la fondation Worker, Yellowstone live, cinq familles multi-source, la continuité/gap repair puis le Desk multi-route ; `0.3.15` ferme en plus les anomalies Mainnet observées de convergence `logMessages`, backpressure Yellowstone soutenue et shutdown post-half-close. Le handoff historique [`plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md`](plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md) est supersédé pour le sizing par [`plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md`](plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md) et [`validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md`](validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md) : `0.3.16` ferme les fondations RAW multi-variantes, le sélecteur/promotion canonique, le conflit durable minimal et l'ingestion Worker non terminale ; `0.3.17` porte la résilience opérationnelle et le cycle de vie complet des variantes, `0.3.18` l'inspection/résolution Store Desk, puis les extensions Backfill multi-route et Backfill Desk sont reportées par défaut à `0.3.19` et `0.3.20`.
|
||||||
|
|
||||||
## Spécifications de formats
|
## Spécifications de formats
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: docs/architecture/004-COMPONENT_INVENTORY.md -->
|
<!-- file: docs/architecture/004-COMPONENT_INVENTORY.md -->
|
||||||
<!-- version: 43 -->
|
<!-- version: 44 -->
|
||||||
|
|
||||||
# Inventaire initial des composants KSP
|
# Inventaire initial des composants KSP
|
||||||
|
|
||||||
@@ -19,7 +19,7 @@ Ce document maintient l'inventaire synthétique des composants retenus ou presse
|
|||||||
## Inventaire synthétique
|
## Inventaire synthétique
|
||||||
|
|
||||||
| Domaine | Composant | Type | Statut | Première cible actuelle | Mission |
|
| Domaine | Composant | Type | Statut | Première cible actuelle | Mission |
|
||||||
|-------------------------|-----------------------------------------|-------------|--------------|----------------------------------|------------------------------------------------------------------------------------------------------|
|
|-------------------------|-----------------------------------------|-------------|--------------|---------------------------------|-------------------------------------------------------------------------------------------------------|
|
||||||
| Core | `ksp-core-lib` | lib | Stable | `0.1.1` | Error/Result, Program IDs et primitives fondamentales |
|
| Core | `ksp-core-lib` | lib | Stable | `0.1.1` | Error/Result, Program IDs et primitives fondamentales |
|
||||||
| Logging | `ksp-logging-lib` | lib | Stable | `0.1.2` | façade unique tracing KSP |
|
| Logging | `ksp-logging-lib` | lib | Stable | `0.1.2` | façade unique tracing KSP |
|
||||||
| Config | `ksp-config-lib` | lib | Stable | `0.1.3` | documents, profils, env et persistence Config |
|
| Config | `ksp-config-lib` | lib | Stable | `0.1.3` | documents, profils, env et persistence Config |
|
||||||
@@ -37,16 +37,16 @@ Ce document maintient l'inventaire synthétique des composants retenus ou presse
|
|||||||
| Program API | `ksp-program-api` | API | Stable | `0.2.14` | contrats extensibles Program |
|
| Program API | `ksp-program-api` | API | Stable | `0.2.14` | contrats extensibles Program |
|
||||||
| Program impl. | `ksp-program-lib` | lib | Retenu | vertical slices ultérieurs | implementations Program officielles |
|
| Program impl. | `ksp-program-lib` | lib | Retenu | vertical slices ultérieurs | implementations Program officielles |
|
||||||
| Program extension | `ksp-program-<name>-lib` | lib externe | À la demande | dès besoin | implementation externe de `ksp-program-api` |
|
| Program extension | `ksp-program-<name>-lib` | lib externe | À la demande | dès besoin | implementation externe de `ksp-program-api` |
|
||||||
| Store API | `ksp-store-api` | API | Stable | `0.3.1`, extension `0.3.16` | RAW transaction/account, observations ; variantes/conflits/résolutions ajoutés sans backend leak |
|
| Store API | `ksp-store-api` | API | Stable | `0.3.1`, ext. `0.3.16`/`0.3.17` | RAW transaction/account, observations ; variantes/comparateur en `0.3.16`, résolution en `0.3.17` |
|
||||||
| Store runtime | `ksp-store-lib` | lib | Stable | `0.3.2`–`0.3.4`, `0.3.16` | façade backend-neutral ; convergence, retry et résolution RAW partagés entre producteurs |
|
| Store runtime | `ksp-store-lib` | lib | Stable | `0.3.2`–`0.3.4`, ext. `0.3.16` | façade backend-neutral ; outcomes variantes partagés, inspection/résolution étendues en `0.3.17` |
|
||||||
| Store PostgreSQL | `ksp-store-postgres-lib` | lib | Stable | `0.3.2`–`0.3.4`, `0.3.16` | backend référence ; variantes conflictuelles et historique de résolution atomiques en `0.3.16` |
|
| Store PostgreSQL | `ksp-store-postgres-lib` | lib | Stable | `0.3.2`–`0.3.4`, ext. `0.3.16` | backend référence ; V003 variantes/selector/conflit minimal atomiques, lifecycle complet en `0.3.17` |
|
||||||
| Job lifecycle | `ksp-job-api` | API | Implémenté | `0.3.6` | identité/lifecycle/annulation/notifications latest-value runtime-neutral |
|
| Job lifecycle | `ksp-job-api` | API | Implémenté | `0.3.6` | identité/lifecycle/annulation/notifications latest-value runtime-neutral |
|
||||||
| Backfill | `ksp-job-backfill-lib` | lib | Implémenté | `0.3.6`, extension `0.3.17` | première verticale HTTP puis extension multi-route/historique sur convergence Store partagée |
|
| Backfill | `ksp-job-backfill-lib` | lib | Implémenté | `0.3.6`, cible ext. `0.3.19` | première verticale HTTP ; compatibilité variantes `0.3.16`, extension multi-route/historique reportée |
|
||||||
| Backfill Desk | `ksp-app-backfill-desk` | app | Implémenté | `0.3.7`, extension `0.3.18` | contrôle du backfill RAW ; sélection/supervision multi-route adaptée au Job `0.3.17` |
|
| Backfill Desk | `ksp-app-backfill-desk` | app | Implémenté | `0.3.7`, cible ext. `0.3.20` | contrôle du backfill RAW ; adaptation multi-route reportée après le programme résilience/Store Desk |
|
||||||
| Store Desk | `ksp-app-store-desk` | app | Implémenté | `0.3.8`, extension `0.3.16` | inspection RAW + vues conflits/variantes/historique et résolution manuelle réversible |
|
| Store Desk | `ksp-app-store-desk` | app | Implémenté | `0.3.8`, extension `0.3.18` | inspection RAW actuelle ; variantes/conflits/résolutions opérateur ajoutés après contrats `0.3.17` |
|
||||||
| RAW transaction common | `ksp-raw-transaction-lib` | lib | Implémenté | `0.3.10` | canonicalisation/wire RAW Transaction v1 source-neutral partagé entre producteurs |
|
| RAW transaction common | `ksp-raw-transaction-lib` | lib | Implémenté | `0.3.10` | canonicalisation/wire RAW Transaction v1 source-neutral partagé entre producteurs |
|
||||||
| Worker lifecycle | `ksp-worker-api` | API | Implémenté | `0.3.9` | lifecycle/health/progression génériques des services continus |
|
| Worker lifecycle | `ksp-worker-api` | API | Implémenté | `0.3.9` | lifecycle/health/progression génériques des services continus |
|
||||||
| RAW transaction worker | `ksp-worker-raw-transaction-ingest-lib` | worker/lib | Implémenté | `0.3.11`–`0.3.15`, ext. `0.3.16` | live cinq familles, continuité/repair et backpressure bornée ; conflits durables étendus en `0.3.16` |
|
| RAW transaction worker | `ksp-worker-raw-transaction-ingest-lib` | worker/lib | Implémenté | `0.3.11`–`0.3.16` | live multi-source borné ; Store arbitre les variantes, conflit durable non terminal en `0.3.16` |
|
||||||
| RAW ingest Desk | `ksp-app-raw-transaction-ingest-desk` | app | Implémenté | `0.3.15` | composition réseau-centrique et supervision simultanée de plusieurs routes live |
|
| RAW ingest Desk | `ksp-app-raw-transaction-ingest-desk` | app | Implémenté | `0.3.15` | composition réseau-centrique et supervision simultanée de plusieurs routes live |
|
||||||
| STRUCTURAL job | nom à fixer | job/lib | Retenu | couche STRUCTURAL | normalisation Solana générique RAW -> STRUCTURAL bornée/rejouable |
|
| STRUCTURAL job | nom à fixer | job/lib | Retenu | couche STRUCTURAL | normalisation Solana générique RAW -> STRUCTURAL bornée/rejouable |
|
||||||
| STRUCTURAL worker | nom à fixer | worker/lib | Retenu | fin couche STRUCTURAL | backlog RAW -> STRUCTURAL continu |
|
| STRUCTURAL worker | nom à fixer | worker/lib | Retenu | fin couche STRUCTURAL | backlog RAW -> STRUCTURAL continu |
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: docs/architecture/009-ACQUISITION_WORKERS_AND_JOBS.md -->
|
<!-- file: docs/architecture/009-ACQUISITION_WORKERS_AND_JOBS.md -->
|
||||||
<!-- version: 22 -->
|
<!-- version: 23 -->
|
||||||
|
|
||||||
# Acquisition, workers, jobs et pipelines spécialisés
|
# Acquisition, workers, jobs et pipelines spécialisés
|
||||||
|
|
||||||
@@ -90,7 +90,7 @@ Le job :
|
|||||||
|
|
||||||
Le caller desktop spécialisé actuel est `ksp-app-backfill-desk`. Il compose Config, le pool HTTP et Store, puis remet ces ressources au runtime Backfill. Il peut retenir le checkpoint terminal uniquement en mémoire Rust pour un Resume in-session ; cette rétention applicative ne transforme pas le checkpoint en garantie de reprise durable après redémarrage.
|
Le caller desktop spécialisé actuel est `ksp-app-backfill-desk`. Il compose Config, le pool HTTP et Store, puis remet ces ressources au runtime Backfill. Il peut retenir le checkpoint terminal uniquement en mémoire Rust pour un Resume in-session ; cette rétention applicative ne transforme pas le checkpoint en garantie de reprise durable après redémarrage.
|
||||||
|
|
||||||
Cette verticale `0.3.6`/`0.3.7` est **la première stratégie de backfill**, pas la définition générale du backfill KSP. Son discovery `getSignaturesForAddress` + hydration `getTransaction` est HTTP parce que cette méthode a été choisie pour le premier vertical slice. Après stabilisation du worker live et de sa Desk, `0.3.16` ferme d'abord la résilience RAW partagée : divergences compatibles, variantes conflictuelles durables, résolution/restauration canonique, retry Store et reconnexion Transport. `0.3.17` réaudite ensuite `ksp-job-backfill-lib` pour intégrer les autres stratégies historiques/catch-up pertinentes identifiées par l'audit RAW Transaction de `0.3.9`, puis `0.3.18` adapte `ksp-app-backfill-desk` au Job multi-route.
|
Cette verticale `0.3.6`/`0.3.7` est **la première stratégie de backfill**, pas la définition générale du backfill KSP. Son discovery `getSignaturesForAddress` + hydration `getTransaction` est HTTP parce que cette méthode a été choisie pour le premier vertical slice. Après stabilisation du worker live et de sa Desk, `0.3.16` ferme les fondations RAW multi-variantes et le conflit durable non terminal ; `0.3.17` complète le cycle de vie/résolution des variantes ainsi que la résilience Store/Transport, puis `0.3.18` expose ces contrats dans Store Desk. L'extension multi-route/multi-stratégie de `ksp-job-backfill-lib` est reportée par défaut à `0.3.19`, et l'adaptation correspondante de `ksp-app-backfill-desk` à `0.3.20`.
|
||||||
|
|
||||||
### Worker RAW Transaction live
|
### Worker RAW Transaction live
|
||||||
|
|
||||||
@@ -465,7 +465,7 @@ ksp-job-backfill-lib
|
|||||||
-> serde_json/sha2 # discovery/checkpoint/observation propres au Job
|
-> serde_json/sha2 # discovery/checkpoint/observation propres au Job
|
||||||
```
|
```
|
||||||
|
|
||||||
Le Job conserve seul ses scopes, campagnes, checkpoints et lifecycle. La canonicalisation transactionnelle partagée appartient désormais à `ksp-raw-transaction-lib`; `0.3.17` étendra les stratégies historiques sans déplacer ce rôle.
|
Le Job conserve seul ses scopes, campagnes, checkpoints et lifecycle. La canonicalisation transactionnelle partagée appartient désormais à `ksp-raw-transaction-lib`; l'extension multi-route/historique planifiée à partir de `0.3.19` ne déplacera pas ce rôle.
|
||||||
|
|
||||||
### RAW worker
|
### RAW worker
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<!-- file: docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md -->
|
<!-- file: docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md -->
|
||||||
<!-- version: 18 -->
|
<!-- version: 19 -->
|
||||||
|
|
||||||
# Acquisition et alimentation `RawTransaction`
|
# Acquisition et alimentation `RawTransaction`
|
||||||
|
|
||||||
@@ -840,16 +840,18 @@ Stop Yellowstone post-half-close : status distant tardif classé Closed, status
|
|||||||
compatibilité étroite : canonique complet + incoming logMessages avec marqueur exact Log truncated
|
compatibilité étroite : canonique complet + incoming logMessages avec marqueur exact Log truncated
|
||||||
```
|
```
|
||||||
|
|
||||||
Reste hors de cette verticale et nécessite une tranche dédiée :
|
Après fermeture technique de `0.3.16`, restent hors de cette verticale et nécessitent des versions dédiées :
|
||||||
|
|
||||||
```text
|
```text
|
||||||
sources EARLY / pre-execution
|
sources EARLY / pre-execution
|
||||||
campagne historique multi-source ou caller-paramétrable
|
campagne historique multi-source ou caller-paramétrable
|
||||||
variantes RAW durables et résolution générale des conflits
|
cycle de vie complet des conflits : résolution/reopen/historique/rétention des variantes
|
||||||
promotion canonique d'une représentation moins complète vers une plus complète
|
retry Store et reconnexion Transport configurables
|
||||||
retry Store et reconnexion Transport configurables au niveau prévu par 0.3.16
|
Store Desk conflits/variantes/résolutions opérateur
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Les variantes RAW durables, la promotion canonique strictement prouvée et le conflit durable minimal non terminal sont, eux, acquis par `0.3.16`.
|
||||||
|
|
||||||
### 14.3 Gaps Transport Worker
|
### 14.3 Gaps Transport Worker
|
||||||
|
|
||||||
| ID | Adaptation | Motif |
|
| ID | Adaptation | Motif |
|
||||||
@@ -882,9 +884,9 @@ La règle générale reste fail-closed : une divergence canonique non reconnue p
|
|||||||
|
|
||||||
Le sens inverse reste volontairement non résolu en `0.3.15` : un canonique tronqué suivi d'une représentation complète n'est pas promu automatiquement. Deux contenus complets divergents, un préfixe incompatible, une différence hors `logMessages` ou toute relation de troncature non prouvée restent des conflits. Cette asymétrie est précisément le point d'entrée du modèle de variantes/promotion/résolution de `0.3.16`.
|
Le sens inverse reste volontairement non résolu en `0.3.15` : un canonique tronqué suivi d'une représentation complète n'est pas promu automatiquement. Deux contenus complets divergents, un préfixe incompatible, une différence hors `logMessages` ou toute relation de troncature non prouvée restent des conflits. Cette asymétrie est précisément le point d'entrée du modèle de variantes/promotion/résolution de `0.3.16`.
|
||||||
|
|
||||||
## 14.5 Handoff `0.3.16` : résilience RAW et conflits durables
|
## 14.5 Résultat `0.3.16` : fondations multi-variantes et conflits durables
|
||||||
|
|
||||||
`0.3.16` doit déplacer la résolution des divergences RAW dans le contrat Store partagé, afin que Worker live et futurs Jobs historiques convergent avec la même politique. Une divergence de contenu n'est plus assimilée automatiquement à une panne d'acquisition.
|
`0.3.16` déplace l'arbitrage durable des divergences RAW dans le contrat Store partagé, afin que Worker live et Jobs historiques utilisent la même politique de convergence. Une divergence conservable n'est plus assimilée automatiquement à une panne d'acquisition.
|
||||||
|
|
||||||
Modèle cible :
|
Modèle cible :
|
||||||
|
|
||||||
@@ -897,17 +899,17 @@ RawTransaction identity (network, signature)
|
|||||||
-> historique immutable des promotions/résolutions
|
-> historique immutable des promotions/résolutions
|
||||||
```
|
```
|
||||||
|
|
||||||
Classification minimale : `Exact`, `CompatibleLessComplete`, `CompatibleMoreComplete`, `Conflict`. Une représentation prouvée moins complète ne remplace jamais le canonique ; une représentation prouvée plus complète peut le promouvoir atomiquement ; un vrai conflit conserve toutes les variantes sans arrêter l'acquisition. La provenance d'un provider n'établit jamais à elle seule une priorité canonique.
|
Classification livrée : `Exact`, `CompatibleLessComplete`, `CompatibleMoreComplete`, `Conflict`, `Incomparable`. Une représentation prouvée moins complète ne remplace jamais le canonique ; une représentation prouvée plus complète le promeut atomiquement ; `Conflict` et `Incomparable` conservent la variante entrante et ouvrent/mettent à jour un conflict case minimal sans arrêter l'acquisition. La provenance d'un provider n'établit jamais à elle seule une priorité canonique.
|
||||||
|
|
||||||
Le Worker doit distinguer les problèmes locaux durablement gérables des pannes externes. Un conflit RAW quarantiné laisse la route active et rend la santé au plus `Degraded`. Une indisponibilité Store impose backpressure/pause et retry sans perte silencieuse. Une indisponibilité Transport déclenche une politique de reconnexion bornée/configurable (`initial_delay`, `max_delay`, backoff, `max_attempts`, reset après stabilité). Un terminal `Faulted` reste réservé aux erreurs irréconciliables ou à l'épuisement explicite d'une politique de retry/reconnect.
|
Le Worker distingue désormais le conflit durable d'une panne externe : `QuarantinedConflict` laisse la route `Running` et projette `Degraded`, tandis qu'un ancien backend qui renvoie encore `ERROR_CODE_RAW_CONFLICT` reste traité comme erreur terminale de compatibilité. Les politiques complètes de retry Store/backpressure `Blocked` et de reconnexion Transport configurable sont reportées à `0.3.17` ; elles ne sont pas implicitement revendiquées par `0.3.16`.
|
||||||
|
|
||||||
`ksp-app-store-desk` doit ajouter des vues de conflits/variantes et d'historique. Une résolution manuelle peut promouvoir une variante, conserver le canonique, ou produire une variante de fusion assistée. Toute ancienne valeur canonique reste une variante durable et peut être restaurée ultérieurement ; aucun `UPDATE` destructif ne doit rendre une décision manuelle irréversible.
|
`ksp-app-store-desk` reste inchangé en `0.3.16`. Les contrats d'inspection/résolution complets sont préparés en `0.3.17`, puis les vues/actions opérateur de variantes, conflits et historique sont prévues pour `0.3.18`. Toute future décision manuelle doit rester réversible et ne jamais détruire les variantes nécessaires au rollback.
|
||||||
|
|
||||||
Le détail de handoff est fixé dans `docs/plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md`.
|
Le handoff historique est conservé dans `docs/plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md`; le sizing et la répartition actuels sont fixés par `docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md`.
|
||||||
|
|
||||||
## 15. Handoff `0.3.17` : Job Backfill multi-stratégie
|
## 15. Handoff `0.3.17` -> `0.3.20`
|
||||||
|
|
||||||
Le Job doit conserver son vertical slice existant puis ajouter des stratégies historiques choisies selon requête + capabilities + configuration.
|
`0.3.17` ferme le cycle de vie complet des variantes/conflits et la résilience opérationnelle Store/Transport. `0.3.18` expose ensuite inspection et résolution dans Store Desk. Le Job Backfill multi-stratégie est reporté par défaut à `0.3.19` : il doit conserver son vertical slice existant puis ajouter des stratégies historiques choisies selon requête + capabilities + configuration. L'adaptation Backfill Desk correspondante est ciblée par défaut sur `0.3.20`.
|
||||||
|
|
||||||
| Stratégie | Entrée de campagne typique | Acquisition | Admission |
|
| Stratégie | Entrée de campagne typique | Acquisition | Admission |
|
||||||
|----------------------------------------|-----------------------------------|----------------------------|------------------|
|
|----------------------------------------|-----------------------------------|----------------------------|------------------|
|
||||||
@@ -939,11 +941,12 @@ Ordre conseillé :
|
|||||||
0.3.14 P0 : gaps run-local, coverage prouvée, repair borné, source-loss et shutdown/fairness cross-layer
|
0.3.14 P0 : gaps run-local, coverage prouvée, repair borné, source-loss et shutdown/fairness cross-layer
|
||||||
0.3.15 P0 : Raw Transaction Ingest Desk multi-route + stabilité Mainnet Yellowstone/getBlock/polling
|
0.3.15 P0 : Raw Transaction Ingest Desk multi-route + stabilité Mainnet Yellowstone/getBlock/polling
|
||||||
0.3.15 P1 : diagnostic/convergence logMessages tronqués + backpressure gRPC soutenue + Stop post-half-close
|
0.3.15 P1 : diagnostic/convergence logMessages tronqués + backpressure gRPC soutenue + Stop post-half-close
|
||||||
0.3.16 P0 : variantes/conflits RAW durables, promotions/résolutions, retries et Store Desk conflicts
|
0.3.16 P0 : variantes RAW durables, promotion prouvée, conflit minimal durable, Worker non terminal
|
||||||
0.3.17 P0 : block scan historique
|
0.3.17 P0 : résolution/rétention des variantes + retry Store + reconnect Transport configurable
|
||||||
0.3.17 P0 : replay Yellowstone borné
|
0.3.18 P0 : Store Desk conflits/variantes/résolutions opérateur
|
||||||
0.3.17 P1 : provider history/archive
|
0.3.19 P0 : block scan historique + replay Yellowstone borné
|
||||||
0.3.17 P1 : Old Faithful
|
0.3.19 P1 : provider history/archive + Old Faithful
|
||||||
|
0.3.20 P0 : Backfill Desk multi-route
|
||||||
plus tard : substrats directs et sources vendor-specific sans accès actuel
|
plus tard : substrats directs et sources vendor-specific sans accès actuel
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -1065,9 +1068,11 @@ TR-C2 = adapter productif Transport DTO -> common réserv
|
|||||||
0.3.11 = fondation Worker source-neutral, persistence et observabilité
|
0.3.11 = fondation Worker source-neutral, persistence et observabilité
|
||||||
0.3.12 = première verticale Yellowstone + hydration/replay continuity
|
0.3.12 = première verticale Yellowstone + hydration/replay continuity
|
||||||
0.3.13 = cinq familles live + convergence/fairness/health/shutdown/completeness
|
0.3.13 = cinq familles live + convergence/fairness/health/shutdown/completeness
|
||||||
0.3.16 = résilience RAW, variantes/conflits/résolutions, retry/reconnect et Store Desk
|
0.3.16 = fondations RAW multi-variantes + conflit durable minimal + Worker non terminal
|
||||||
0.3.17 = Job Backfill multi-route/multi-stratégie historique
|
0.3.17 = lifecycle/résolution/rétention variantes + retry Store + reconnect Transport
|
||||||
0.3.18 = Backfill Desk multi-route
|
0.3.18 = Store Desk variantes/conflits/résolutions
|
||||||
|
0.3.19 cible = Job Backfill multi-route/multi-stratégie historique
|
||||||
|
0.3.20 cible = Backfill Desk multi-route
|
||||||
```
|
```
|
||||||
|
|
||||||
`0.3.9` n'implémente aucune nouvelle stratégie d'acquisition. Il ferme l'architecture et l'inventaire nécessaires aux releases suivantes.
|
`0.3.9` n'implémente aucune nouvelle stratégie d'acquisition. Il ferme l'architecture et l'inventaire nécessaires aux releases suivantes.
|
||||||
|
|||||||
1174
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
Normal file
1174
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
582
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
Normal file
582
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
Normal file
@@ -0,0 +1,582 @@
|
|||||||
|
<!-- file: docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md -->
|
||||||
|
<!-- version: 3 -->
|
||||||
|
|
||||||
|
# Validation `0.3.16` -> `0.3.18` — résilience RAW et gestion des conflits
|
||||||
|
|
||||||
|
## 1. Objet
|
||||||
|
|
||||||
|
Ce document suit le programme de résilience RAW depuis le gate `0.3.16-pre.001` jusqu'à la fermeture prévue de `0.3.18`. La présente révision réconcilie spécifiquement l'état réellement validé de `0.3.16` avant sa lane de préparation de publication.
|
||||||
|
|
||||||
|
La première tranche ne prétend pas valider une implémentation qui n'existe pas encore. Elle valide le point de départ, les contrats acquis, les décisions d'architecture et la liste des preuves à construire. Le fix documentaire `0.3.16-pre.001-fix.001` répartit ensuite cette matrice sur trois releases stables afin de respecter la contrainte opérationnelle d'une version par session au maximum.
|
||||||
|
|
||||||
|
## 2. Base stable
|
||||||
|
|
||||||
|
Base requise :
|
||||||
|
|
||||||
|
```text
|
||||||
|
v0.3.15
|
||||||
|
commit 7328be6997399c513306f2f4c8cdf00bd8b22367
|
||||||
|
```
|
||||||
|
|
||||||
|
Le delta stable `0.3.15-rel.001` documente :
|
||||||
|
|
||||||
|
```text
|
||||||
|
cargo fmt --all -- --check : PASS
|
||||||
|
General Rust rule audit : clean
|
||||||
|
Rust export completeness audit : 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit : clean
|
||||||
|
Markdown table audit : clean (318 table(s), 232 file(s))
|
||||||
|
cargo check --workspace : PASS
|
||||||
|
```
|
||||||
|
|
||||||
|
Il documente également le gate technique/live de `pre.016`, le live Mainnet Yellowstone + HTTP Block Polling d'environ dix-neuf minutes et la fermeture `Stopped/Healthy` des deux routes.
|
||||||
|
|
||||||
|
Ces résultats appartiennent à la fermeture `0.3.15` ; ils ne sont pas revendiqués comme réexécutés par `0.3.16-pre.001`.
|
||||||
|
|
||||||
|
## 3. Limitation de vérification de l'archive source dans l'environnement d'assemblage
|
||||||
|
|
||||||
|
L'arbre canonique du tag et le commit stable ont été vérifiés via le dépôt public.
|
||||||
|
|
||||||
|
L'archive binaire source `v0.3.15.zip` n'a pas pu être matérialisée dans l'environnement d'assemblage de `pre.001`. Aucun SHA/CRC ou byte-compare local de cette archive n'est donc déclaré PASS ici.
|
||||||
|
|
||||||
|
Cette limitation ne doit pas être masquée par les gates historiques du delta stable.
|
||||||
|
|
||||||
|
Après application du delta sur le checkout utilisateur taggé `v0.3.15`, les scripts et commandes du dépôt restent l'autorité de validation locale.
|
||||||
|
|
||||||
|
## 4. Vérification des règles
|
||||||
|
|
||||||
|
Le gate `pre.001` a réaudité les familles de règles requises par le prompt :
|
||||||
|
|
||||||
|
```text
|
||||||
|
RULES.md
|
||||||
|
RULES_GENERAL.md
|
||||||
|
RULES_KSP.md
|
||||||
|
RULES_RUST.md
|
||||||
|
RULES_DEPENDENCIES.md
|
||||||
|
RULES_DOCUMENTATION.md
|
||||||
|
FILE_CONTRACTS.md
|
||||||
|
VERSION_WORKFLOW.md
|
||||||
|
PROMPT_STRUCTURE.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Décisions directement dérivées de ces règles :
|
||||||
|
|
||||||
|
- première livraison `0.3.16-pre.001` -> Cargo `0.3.16-pre.1` ;
|
||||||
|
- aucun Rust ni SQL lourd dans le gate d'ouverture ;
|
||||||
|
- delta minimal ;
|
||||||
|
- migrations V000/V001/V002 immuables ;
|
||||||
|
- Store-lib façade unique ;
|
||||||
|
- aucune dépendance Worker <-> Job Backfill ;
|
||||||
|
- aucune queue non bornée ;
|
||||||
|
- aucun PASS inventé ;
|
||||||
|
- pas de `npm run build` comme gate Desk.
|
||||||
|
|
||||||
|
## 5. Inventaire de caractérisation `0.3.15`
|
||||||
|
|
||||||
|
### 5.1 Store RAW
|
||||||
|
|
||||||
|
Caractérisé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
identity = network + signature au contrat
|
||||||
|
backend PostgreSQL = une base liée à un network
|
||||||
|
canonical unique actuel
|
||||||
|
atomic canonical + observation
|
||||||
|
additional observation
|
||||||
|
content_conflict terminal hors cas narrow logs
|
||||||
|
retention Full / Archived / Purged / ForceRehydrate
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 Outcomes actuels
|
||||||
|
|
||||||
|
Caractérisés :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Inserted
|
||||||
|
AlreadyPresent
|
||||||
|
Rehydrated
|
||||||
|
SkippedPurged
|
||||||
|
|
||||||
|
Observation:
|
||||||
|
Inserted
|
||||||
|
AlreadyPresent
|
||||||
|
NotRecorded
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 PostgreSQL
|
||||||
|
|
||||||
|
Caractérisé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
V000/V001/V002
|
||||||
|
migration registry + checksum
|
||||||
|
transaction d'écriture
|
||||||
|
collision identité
|
||||||
|
SELECT ... FOR UPDATE
|
||||||
|
comparaison canonique
|
||||||
|
observation transactionnelle
|
||||||
|
retention sous verrou
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 Worker
|
||||||
|
|
||||||
|
Caractérisé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
convergence run-local existante
|
||||||
|
Store content_conflict -> faute persistence
|
||||||
|
faute persistence -> arrêt de source / route terminale
|
||||||
|
pas de Store retry policy dédiée comparable au reconnect Transport
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.5 Transport
|
||||||
|
|
||||||
|
Caractérisé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
HTTP request retry déjà Transport-owned
|
||||||
|
WebSocket reconnect borné avec backoff
|
||||||
|
Yellowstone reconnect borné avec backoff
|
||||||
|
Config projection déjà existante
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.6 Store Desk
|
||||||
|
|
||||||
|
Caractérisé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
inspection via ksp-store-lib
|
||||||
|
pas de SQL direct
|
||||||
|
pas encore de variants/conflicts/resolution actions
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. Preuves externes réauditées
|
||||||
|
|
||||||
|
### 6.1 Log truncation
|
||||||
|
|
||||||
|
Le collecteur SVM courant expose :
|
||||||
|
|
||||||
|
```text
|
||||||
|
LOG_MESSAGES_BYTES_LIMIT = 10 * 1000
|
||||||
|
```
|
||||||
|
|
||||||
|
et ajoute exactement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Log truncated
|
||||||
|
```
|
||||||
|
|
||||||
|
une seule fois lorsque la limite est atteinte/dépassée, puis cesse d'enregistrer les lignes suivantes.
|
||||||
|
|
||||||
|
Canari requis pour `0.3.16-pre.005` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
truncated exact prefix + marker vs full continuation
|
||||||
|
-> Less/More selon direction
|
||||||
|
|
||||||
|
different prefix before marker
|
||||||
|
-> Conflict
|
||||||
|
|
||||||
|
shorter list without marker
|
||||||
|
-> Conflict/Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.2 Autres champs RPC
|
||||||
|
|
||||||
|
La documentation Solana actuelle montre des différences de sérialisation/optionnalité pour :
|
||||||
|
|
||||||
|
```text
|
||||||
|
innerInstructions
|
||||||
|
loadedAddresses
|
||||||
|
returnData
|
||||||
|
computeUnitsConsumed
|
||||||
|
costUnits
|
||||||
|
rewards
|
||||||
|
preTokenBalances
|
||||||
|
postTokenBalances
|
||||||
|
```
|
||||||
|
|
||||||
|
Canari de politique : aucune de ces différences ne produit `CompatibleLessComplete` ou `CompatibleMoreComplete` sans nouvelle preuve normative explicite.
|
||||||
|
|
||||||
|
### 6.3 PostgreSQL
|
||||||
|
|
||||||
|
Les mécanismes retenus sont compatibles avec la stratégie :
|
||||||
|
|
||||||
|
- `SELECT ... FOR UPDATE` sérialise les writers/lockers concurrents d'une même ligne jusqu'à fin de transaction ;
|
||||||
|
- `INSERT ... ON CONFLICT` possède une sémantique atomique pour les conflits arbitrés ;
|
||||||
|
- certaines fautes d'isolation/serialization exigent le retry de la transaction complète.
|
||||||
|
|
||||||
|
Ces propriétés doivent être traduites en tests KSP et non seulement citées dans la documentation.
|
||||||
|
|
||||||
|
## 7. Décisions fermées par `pre.001`
|
||||||
|
|
||||||
|
### 7.1 Modèle physique
|
||||||
|
|
||||||
|
Décision : ledger V003 de variantes à `variant_id` surrogate + sélecteur canonique sidecar + projection V001 compatible.
|
||||||
|
|
||||||
|
### 7.2 Hash
|
||||||
|
|
||||||
|
Décision : `content_hash` est un préfiltre/intégrité, jamais la preuve unique d'égalité lorsque les bytes sont disponibles.
|
||||||
|
|
||||||
|
### 7.3 Observations
|
||||||
|
|
||||||
|
Décision : toute nouvelle observation V003 pointe vers la variante réellement reçue.
|
||||||
|
|
||||||
|
Les observations legacy impossibles à reconstruire sont qualifiées explicitement comme telles.
|
||||||
|
|
||||||
|
### 7.4 Rétention
|
||||||
|
|
||||||
|
Décision : toute variante requise pour conflit ouvert, canonique courant, restauration ou parent synthétique peut être archivée mais reste protégée contre une purge irréversible tant que l'invariant de rollback dépend de ses bytes.
|
||||||
|
|
||||||
|
### 7.5 Conflict case
|
||||||
|
|
||||||
|
Décision : état principal `Open | Resolved`, réouverture possible, journal append-only des transitions/résolutions.
|
||||||
|
|
||||||
|
### 7.6 Qualité
|
||||||
|
|
||||||
|
Décision : relation interne :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact
|
||||||
|
CompatibleLessComplete
|
||||||
|
CompatibleMoreComplete
|
||||||
|
Conflict
|
||||||
|
Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
`Incomparable` est persisté prudemment comme réconciliation ouverte.
|
||||||
|
|
||||||
|
### 7.7 Store retry
|
||||||
|
|
||||||
|
Décision :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Store backend -> classifie Transient/Terminal
|
||||||
|
Worker -> orchestre retry/backpressure/health
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.8 Transport reconnect
|
||||||
|
|
||||||
|
Décision : reste dans `ksp-onchain-transport-lib` ; extension du mécanisme existant, sans fusion conceptuelle avec Store retry ni coverage.
|
||||||
|
|
||||||
|
### 7.9 Store Desk
|
||||||
|
|
||||||
|
Décision : conflits/historique/actions via `ksp-store-lib` uniquement, bridge Tauri typé, frontend Vite/TypeScript, actions protégées par revision attendue.
|
||||||
|
|
||||||
|
## 8. Matrice de preuves à construire
|
||||||
|
|
||||||
|
### 8.1 API
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- roundtrip/égalité des DTO variant/conflict/history ;
|
||||||
|
- reason codes stables ;
|
||||||
|
- outcomes distincts ;
|
||||||
|
- classification Store transient/terminal sans type PostgreSQL exposé.
|
||||||
|
|
||||||
|
### 8.2 Migration V003
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- checksums V000/V001/V002 inchangés ;
|
||||||
|
- V003 idempotente dans le registry ;
|
||||||
|
- base vide ;
|
||||||
|
- base existante Full ;
|
||||||
|
- base existante Archived ;
|
||||||
|
- base existante Purged ;
|
||||||
|
- observation legacy non faussement attribuée ;
|
||||||
|
- rollback de migration selon contrat KSP si applicable.
|
||||||
|
|
||||||
|
### 8.3 Variants/convergence
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- exact concurrent -> une variante logique ;
|
||||||
|
- less complete logs -> canonique inchangé ;
|
||||||
|
- more complete logs -> promotion atomique ;
|
||||||
|
- conflict -> canonique inchangé + variante durable + case Open ;
|
||||||
|
- incomparable -> fail-closed durable ;
|
||||||
|
- hash identique + payload différent -> jamais fusionné par hash seul.
|
||||||
|
|
||||||
|
### 8.4 Promotions
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- selector et projection V001 cohérents après commit ;
|
||||||
|
- ancien canonique conservé ;
|
||||||
|
- restore local sans Internet ;
|
||||||
|
- stale revision rejetée ;
|
||||||
|
- crash/rollback ne laisse pas selector/projection divergents.
|
||||||
|
|
||||||
|
### 8.5 Rétention
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- archive d'une variante rollbackable conserve bytes exacts ;
|
||||||
|
- purge refusée lorsqu'une variante est épinglée ;
|
||||||
|
- ForceRehydrate exact ;
|
||||||
|
- ForceRehydrate payload différent -> nouvelle variante ;
|
||||||
|
- legacy Purged ne prétend pas disposer d'une égalité byte-exact perdue.
|
||||||
|
|
||||||
|
### 8.6 Worker
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- conflit durable -> route Running, health Degraded ;
|
||||||
|
- autre identité continue ;
|
||||||
|
- Store transient -> retry sans success prématuré ;
|
||||||
|
- queue bornée -> backpressure ;
|
||||||
|
- policy exhausted -> terminal explicite ;
|
||||||
|
- Stop pendant backoff -> cancellation/drain propre.
|
||||||
|
|
||||||
|
### 8.7 Transport
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- backoff initial/max/multiplier ;
|
||||||
|
- max attempts ;
|
||||||
|
- reset stable ;
|
||||||
|
- jitter borné ;
|
||||||
|
- reconnexion != coverage ;
|
||||||
|
- comportement HTTP existant non régressé.
|
||||||
|
|
||||||
|
### 8.8 Store Desk
|
||||||
|
|
||||||
|
À prouver :
|
||||||
|
|
||||||
|
- pagination bornée ;
|
||||||
|
- aucune dépendance backend direct ;
|
||||||
|
- promote/keep/restore/reopen typés ;
|
||||||
|
- stale action visible et recharge ;
|
||||||
|
- aucune donnée sensible dans tracing frontend/backend ;
|
||||||
|
- build via Tauri/Vite seulement.
|
||||||
|
|
||||||
|
## 9. Gate opérateur `0.3.16-pre.001` exécuté après application
|
||||||
|
|
||||||
|
Le 20 septembre 2026, l'opérateur a exécuté sur son checkout KSP après application de `pre.001` :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultats observés dans le transcript fourni :
|
||||||
|
|
||||||
|
```text
|
||||||
|
cargo fmt --all -- --check : aucun diagnostic affiché
|
||||||
|
General Rust rule audit : clean
|
||||||
|
Rust export completeness audit : 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit : clean
|
||||||
|
Markdown table audit : clean (352 table(s), 941 file(s))
|
||||||
|
cargo check --workspace : Finished dev profile (32.69 s)
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings : Finished dev profile (12.26 s)
|
||||||
|
```
|
||||||
|
|
||||||
|
Le workspace résout bien les crates sur `v0.3.16-pre.1`. Aucun `npm run build` n'a été utilisé et aucun test live/provider/PostgreSQL n'était requis par ce gate documentaire.
|
||||||
|
|
||||||
|
Ce résultat ferme le gate local demandé par `pre.001` et autorise la tranche d'implémentation suivante après application du présent fix documentaire.
|
||||||
|
|
||||||
|
## 10. Gate de sortie `pre.001`
|
||||||
|
|
||||||
|
État documentaire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
modèle conceptuel CLOSED
|
||||||
|
frontières de crates CLOSED avec vérification Cargo à chaque tranche
|
||||||
|
migration strategy CLOSED
|
||||||
|
logMessages quality semantics CLOSED
|
||||||
|
other field policy CLOSED / fail-closed
|
||||||
|
Store retry ownership CLOSED
|
||||||
|
Transport reconnect ownership CLOSED
|
||||||
|
Store Desk scope CLOSED
|
||||||
|
races/retention risks IDENTIFIED
|
||||||
|
prerelease sizing RECALIBRATED sur 0.3.16 -> 0.3.18
|
||||||
|
out-of-scope CLOSED
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate local `0.3.16-pre.001` est maintenant documenté comme propre. Après application de `0.3.16-pre.001-fix.001`, l'implémentation peut commencer en `0.3.16-pre.002` selon le découpage multi-version du plan 038.
|
||||||
|
|
||||||
|
## 11. Répartition des preuves par release
|
||||||
|
|
||||||
|
Le fix `0.3.16-pre.001-fix.001` ne supprime aucune preuve de la matrice ; il change uniquement leur release cible :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.16 : API variantes + V003 + persistance + comparateur + promotion + conflit durable + Worker non terminal
|
||||||
|
0.3.17 : résolution + rétention + Store-lib + retry Worker + reconnect Transport + preuves live de résilience
|
||||||
|
0.3.18 : Store Desk + actions opérateur + éventuelle fusion synthétique prouvée + hardening end-to-end
|
||||||
|
```
|
||||||
|
|
||||||
|
Chaque stable possède son propre gate de fermeture. Une preuve non nécessaire au périmètre de la stable courante n'est pas déclarée manquante : elle reste explicitement attendue par la version suivante du programme.
|
||||||
|
|
||||||
|
Les scopes Backfill initialement réservés à `0.3.17` et `0.3.18` sont déplacés après ce programme afin d'éviter une collision de roadmap ; les cibles de planning par défaut deviennent respectivement `0.3.19` et `0.3.20`.
|
||||||
|
|
||||||
|
## 12. Réconciliation finale `0.3.16-pre.009`
|
||||||
|
|
||||||
|
### 12.1 Périmètre réellement acquis par `0.3.16`
|
||||||
|
|
||||||
|
La vertical slice `0.3.16` ferme les éléments suivants :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Store API
|
||||||
|
RawTransactionVariantId / relation / reason code
|
||||||
|
comparateur partagé fail-closed
|
||||||
|
RawTransactionVariantWriteOutcome
|
||||||
|
RawAcquisitionWriteOutcome::transaction_variant()
|
||||||
|
|
||||||
|
PostgreSQL V003
|
||||||
|
ksp_raw_transaction_variants
|
||||||
|
ksp_raw_transaction_canonical_selectors
|
||||||
|
ksp_raw_transaction_observation_variants
|
||||||
|
ksp_raw_transaction_conflicts
|
||||||
|
|
||||||
|
Convergence
|
||||||
|
Exact -> observation/réutilisation de variante
|
||||||
|
CompatibleLessComplete -> variante observée, canonique inchangé
|
||||||
|
CompatibleMoreComplete -> promotion atomique
|
||||||
|
Conflict / Incomparable -> variante durable + conflict case Open, canonique inchangé
|
||||||
|
|
||||||
|
Worker
|
||||||
|
toute acquisition atteint le Store après sérialisation run-local par identité
|
||||||
|
QuarantinedConflict -> succès durable, Running + Degraded
|
||||||
|
identité suivante -> continue à être traitée
|
||||||
|
ERROR_CODE_RAW_CONFLICT legacy -> terminal de compatibilité
|
||||||
|
|
||||||
|
Job Backfill
|
||||||
|
QuarantinedConflict -> BackfillEntityPersistence::Conflict
|
||||||
|
outcome observation -> conservé, y compris AlreadyPresent
|
||||||
|
```
|
||||||
|
|
||||||
|
`content_hash` reste un préfiltre/intégrité et ne remplace jamais la comparaison des bytes lorsque ceux-ci sont disponibles. La seule dominance automatique admise en `0.3.16` reste la troncature `meta.logMessages` strictement prouvée ; les autres différences restent fail-closed.
|
||||||
|
|
||||||
|
### 12.2 Atomicité et rollback validés
|
||||||
|
|
||||||
|
Le hardening `pre.008` verrouille les invariants suivants :
|
||||||
|
|
||||||
|
- la branche de conflit ne modifie ni le sélecteur canonique ni la projection V001 ;
|
||||||
|
- les helpers de promotion et de conflict case restent dans la transaction d'acquisition externe ;
|
||||||
|
- un échec tardif après création de la variante divergente et du conflict case mais avant `COMMIT` annule ces écritures ;
|
||||||
|
- après rollback, selector et projection V001 restent cohérents avec le canonique antérieur ;
|
||||||
|
- deux écritures divergentes concurrentes produisent un canonique et une quarantaine durable, pas une erreur terminale de contenu.
|
||||||
|
|
||||||
|
### 12.3 Gate opérateur `0.3.16-pre.008-fix.001`
|
||||||
|
|
||||||
|
Le 22 septembre 2026, l'opérateur a exécuté :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
|
||||||
|
cargo test -p ksp-store-api --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-store-postgres-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-job-backfill-lib --all-targets --all-features
|
||||||
|
cargo test -p ksp-worker-raw-transaction-ingest-lib --all-targets --all-features
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultats observés :
|
||||||
|
|
||||||
|
```text
|
||||||
|
cargo fmt --all -- --check PASS
|
||||||
|
General Rust rule audit clean
|
||||||
|
Rust export completeness audit 0 candidate(s)
|
||||||
|
KSP workspace Rust rule audit clean
|
||||||
|
Markdown table audit clean (352 table(s), 958 file(s))
|
||||||
|
cargo check --workspace PASS
|
||||||
|
cargo clippy --workspace --all-targets --all-features -D warnings PASS
|
||||||
|
|
||||||
|
ksp-store-api
|
||||||
|
29 unit + 2 dependency + 1 external backend + 9 public API
|
||||||
|
+ 5 release completeness + 5 security hardening PASS
|
||||||
|
|
||||||
|
ksp-store-lib
|
||||||
|
10 unit + 3 dependency + 1 feature mismatch
|
||||||
|
+ 6 hardening completeness + 5 public API PASS
|
||||||
|
|
||||||
|
ksp-store-postgres-lib
|
||||||
|
88 unit + 13 dependency + 16 hardening completeness
|
||||||
|
+ 13 public API + 3 V003 migration + 8 V003 variant PASS
|
||||||
|
foundation/raw-account/raw-transaction live tests ignored in normal gate as designed
|
||||||
|
|
||||||
|
ksp-job-backfill-lib
|
||||||
|
53 unit + 4 dependency + 9 hardening + 1 HTTP block material
|
||||||
|
+ 6 public API + 3 release + 2 WS parity + 1 Yellowstone PASS
|
||||||
|
|
||||||
|
ksp-worker-raw-transaction-ingest-lib
|
||||||
|
163 unit + 12 cross-layer + 21 dependency + 41 hardening
|
||||||
|
+ 22 public API + 18 release completeness PASS
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun warning Clippy n'est toléré par ce gate.
|
||||||
|
|
||||||
|
### 12.4 Preuve PostgreSQL réelle
|
||||||
|
|
||||||
|
Le live RawTransaction a été exécuté séparément avec une URI dédiée fournie silencieusement sur stdin :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
read -rsp 'PostgreSQL URI: ' KSP_TEST_POSTGRES_URI
|
||||||
|
printf '\n'
|
||||||
|
printf '%s\n' "$KSP_TEST_POSTGRES_URI" \
|
||||||
|
| cargo test -p ksp-store-postgres-lib \
|
||||||
|
--test postgres_raw_transaction_live \
|
||||||
|
-- --ignored --nocapture
|
||||||
|
unset KSP_TEST_POSTGRES_URI
|
||||||
|
```
|
||||||
|
|
||||||
|
Résultat observé :
|
||||||
|
|
||||||
|
```text
|
||||||
|
KSP Store RawTransaction live proof: server major 17
|
||||||
|
test pre_009_real_postgres_raw_transaction_vertical_slice_is_atomic_concurrent_and_recoverable ... ok
|
||||||
|
1 passed; 0 failed
|
||||||
|
```
|
||||||
|
|
||||||
|
Cette preuve couvre notamment la concurrence divergente, la quarantaine durable, la cohérence selector/projection et le rollback tardif ajoutés au hardening `pre.008`.
|
||||||
|
|
||||||
|
### 12.5 Périmètre explicitement reporté
|
||||||
|
|
||||||
|
Ne sont **pas** revendiqués comme acquis par `0.3.16` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
résolution/reopen/historique complet des conflict cases
|
||||||
|
rétention/pins/purge guards propres aux variantes
|
||||||
|
classification Store Transient/Terminal complète
|
||||||
|
retry Store / Blocked / backoff Worker
|
||||||
|
reconnexion WebSocket/Yellowstone configurable étendue
|
||||||
|
inspection/résolution de variantes via ksp-store-lib
|
||||||
|
Store Desk conflits/variantes/résolutions
|
||||||
|
Backfill multi-route/multi-stratégie
|
||||||
|
Backfill Desk multi-route
|
||||||
|
```
|
||||||
|
|
||||||
|
Les six premiers groupes appartiennent à `0.3.17`, Store Desk à `0.3.18`, puis les extensions Backfill sont ciblées par défaut sur `0.3.19`/`0.3.20`.
|
||||||
|
|
||||||
|
### 12.6 État de fermeture avant publication
|
||||||
|
|
||||||
|
```text
|
||||||
|
architecture/sizing CLOSED
|
||||||
|
Store API variantes/comparateur CLOSED
|
||||||
|
migration V003 CLOSED
|
||||||
|
variant ledger + observation mapping CLOSED
|
||||||
|
promotion canonique atomique CLOSED
|
||||||
|
conflit durable minimal CLOSED
|
||||||
|
Worker non terminal sur quarantaine CLOSED
|
||||||
|
régression Backfill CLOSED
|
||||||
|
hardening concurrence/rollback CLOSED
|
||||||
|
PostgreSQL RawTransaction live PASS sur PostgreSQL 17
|
||||||
|
gate technique pre.008-fix.001 PASS
|
||||||
|
réconciliation documentaire pre.009 CURRENT LANE
|
||||||
|
préparation publication pre.010 PENDING
|
||||||
|
rel.001 PENDING
|
||||||
|
```
|
||||||
|
|
||||||
|
La lane `pre.009` ne doit modifier ni `CHANGELOG.md`, ni `ROADMAP.md`, ni le prompt de démarrage `0.3.17`. Ces trois responsabilités appartiennent exclusivement à `pre.010`.
|
||||||
|
|
||||||
692
prompts/036-V0_3_17_START_PROMPT.md
Normal file
692
prompts/036-V0_3_17_START_PROMPT.md
Normal file
@@ -0,0 +1,692 @@
|
|||||||
|
<!-- file: prompts/036-V0_3_17_START_PROMPT.md -->
|
||||||
|
<!-- version: 1 -->
|
||||||
|
|
||||||
|
# Prompt de démarrage `0.3.17` — résilience opérationnelle et cycle de vie des variantes RAW
|
||||||
|
|
||||||
|
## 1. Identité de la release et base exacte requise
|
||||||
|
|
||||||
|
Ouvrir **uniquement** `0.3.17` depuis la release stable/taggée :
|
||||||
|
|
||||||
|
```text
|
||||||
|
v0.3.16
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne pas démarrer depuis `0.3.16-pre.*`, depuis une archive intermédiaire ou depuis un état de travail non taggé.
|
||||||
|
|
||||||
|
La première livraison attendue est :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.17-pre.001
|
||||||
|
```
|
||||||
|
|
||||||
|
`pre.001` est obligatoirement un gate de **lecture + audit Store API/Store/PostgreSQL/Worker/Transport/Config + audit du cycle de vie durable des variantes/conflits + audit des erreurs transient/terminal + brainstorming + sizing + planification**.
|
||||||
|
|
||||||
|
Il est interdit de commencer directement par une UI Store Desk, un moteur de retry générique, une migration opportuniste ou une boucle de reconnexion parallèle avant fermeture de ce gate.
|
||||||
|
|
||||||
|
## 2. Mission et résultat attendu
|
||||||
|
|
||||||
|
`0.3.16` a livré les fondations multi-variantes : comparaison fail-closed, sélecteur canonique, promotion strictement prouvée, conservation des variantes et quarantaine durable minimale sans arrêt du Worker.
|
||||||
|
|
||||||
|
`0.3.17` doit transformer cette fondation en mécanisme opérable et résilient côté backend/runtime, sans ouvrir encore l'UI Store Desk.
|
||||||
|
|
||||||
|
Le résultat cible est :
|
||||||
|
|
||||||
|
```text
|
||||||
|
plusieurs représentations RAW durables
|
||||||
|
-> inspection backend-neutral
|
||||||
|
-> conflict case complet et revisionné
|
||||||
|
-> participants/historique durable
|
||||||
|
-> résolution explicite et concurrent-safe
|
||||||
|
-> restore/reopen possibles selon contrat
|
||||||
|
-> rétention/pins empêchant la perte d'une variante encore nécessaire
|
||||||
|
|
||||||
|
Store temporairement indisponible
|
||||||
|
-> classification Transient / Terminal
|
||||||
|
-> retry borné avec backpressure explicite
|
||||||
|
-> aucun drop silencieux
|
||||||
|
-> health/activity cohérentes
|
||||||
|
-> cancellation/drain pendant backoff
|
||||||
|
|
||||||
|
Transport live temporairement indisponible
|
||||||
|
-> mécanisme de reconnexion propriétaire réutilisé
|
||||||
|
-> configuration bornée
|
||||||
|
-> aucune confusion entre reconnexion, replay delivery et preuve de coverage
|
||||||
|
```
|
||||||
|
|
||||||
|
La release doit fournir les contrats nécessaires à `0.3.18`, où `ksp-app-store-desk` exposera ensuite l'inspection et les actions opérateur.
|
||||||
|
|
||||||
|
## 3. Principes directeurs acquis
|
||||||
|
|
||||||
|
Les principes suivants sont déjà décidés et ne doivent pas être redébattus sans preuve nouvelle :
|
||||||
|
|
||||||
|
```text
|
||||||
|
une divergence de données != une panne d'acquisition
|
||||||
|
Store est l'autorité de convergence durable
|
||||||
|
une provenance/provider n'est jamais une priorité canonique en soi
|
||||||
|
content_hash seul n'est jamais une preuve d'égalité lorsque les bytes sont disponibles
|
||||||
|
la dominance reste fail-closed
|
||||||
|
seule une complétude explicitement prouvée autorise une promotion automatique
|
||||||
|
un conflit non résolu conserve toutes les variantes nécessaires
|
||||||
|
reconnexion != coverage
|
||||||
|
retry != duplication silencieuse
|
||||||
|
```
|
||||||
|
|
||||||
|
Le Worker ne doit pas réintroduire un arbitre run-local plus fort que le Store.
|
||||||
|
|
||||||
|
## 4. Sources de vérité internes obligatoires — ordre de lecture
|
||||||
|
|
||||||
|
### 4.1 Gouvernance générale
|
||||||
|
|
||||||
|
Lire intégralement, dans cet ordre :
|
||||||
|
|
||||||
|
```text
|
||||||
|
RULES.md
|
||||||
|
ROADMAP.md
|
||||||
|
CHANGELOG.md
|
||||||
|
docs/000-README.md
|
||||||
|
|
||||||
|
docs/rules/RULES_GENERAL.md
|
||||||
|
docs/rules/RULES_KSP.md
|
||||||
|
docs/rules/RULES_RUST.md
|
||||||
|
docs/rules/RULES_DEPENDENCIES.md
|
||||||
|
docs/rules/RULES_DOCUMENTATION.md
|
||||||
|
docs/rules/FILE_CONTRACTS.md
|
||||||
|
docs/rules/VERSION_WORKFLOW.md
|
||||||
|
docs/rules/PROMPT_STRUCTURE.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Le présent prompt complète ces règles ; il ne les remplace pas.
|
||||||
|
|
||||||
|
Rappels bloquants :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Rust 2024
|
||||||
|
unsafe interdit
|
||||||
|
unwrap / expect / panic interdits selon les règles KSP
|
||||||
|
retours explicites ; clippy::implicit_return deny
|
||||||
|
#![warn(missing_docs)]
|
||||||
|
#![deny(unreachable_pub)]
|
||||||
|
#![forbid(unsafe_code)]
|
||||||
|
|
||||||
|
pas de pub mod
|
||||||
|
pub/pub(crate) partagés reexportés via crate root
|
||||||
|
accès partagés via crate::Item, y compris intra-crate
|
||||||
|
unit tests sous unit_tests/
|
||||||
|
integration tests sous tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 Handoff stable `0.3.16`
|
||||||
|
|
||||||
|
Lire intégralement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
prompts/035-V0_3_16_START_PROMPT.md
|
||||||
|
docs/plans/037-V0_3_16_RAW_RESILIENCE_CONFLICT_HANDOFF.md
|
||||||
|
docs/plans/038-V0_3_16_RAW_RESILIENCE_CONFLICT_PLAN.md
|
||||||
|
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
|
||||||
|
|
||||||
|
deltas/0.3.16/pre.001.md
|
||||||
|
...
|
||||||
|
deltas/0.3.16/pre.010.md
|
||||||
|
deltas/0.3.16/rel.001.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Les `pre.*-fix.*` réellement présents doivent également être lus lorsqu'ils corrigent une tranche citée.
|
||||||
|
|
||||||
|
Lorsque `rel.001.md` n'est pas encore présent dans l'archive fournie à l'ouverture de la session, **ne pas l'inventer** : vérifier d'abord que la base est bien le tag stable `v0.3.16` et utiliser le commit/tag comme autorité.
|
||||||
|
|
||||||
|
### 4.3 Store API et façade
|
||||||
|
|
||||||
|
Lire et auditer réellement :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-store-api/README.md
|
||||||
|
crates/ksp-store-api/USAGE.md
|
||||||
|
crates/ksp-store-api/src/
|
||||||
|
crates/ksp-store-api/tests/
|
||||||
|
|
||||||
|
crates/ksp-store-lib/README.md
|
||||||
|
crates/ksp-store-lib/USAGE.md
|
||||||
|
crates/ksp-store-lib/src/
|
||||||
|
crates/ksp-store-lib/tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Auditer en particulier :
|
||||||
|
|
||||||
|
```text
|
||||||
|
RawTransactionVariant*
|
||||||
|
RawTransactionVariantComparison
|
||||||
|
RawTransactionVariantWriteOutcome
|
||||||
|
RawAcquisitionWriteOutcome
|
||||||
|
capabilities RawTransaction existantes
|
||||||
|
surface d'inspection actuelle
|
||||||
|
codes d'erreur Store et backend-neutralité
|
||||||
|
object safety et stabilité des traits publics
|
||||||
|
rétention RawTransaction existante
|
||||||
|
```
|
||||||
|
|
||||||
|
`0.3.17-pre.001` doit décider les DTOs/capabilities backend-neutral nécessaires pour :
|
||||||
|
|
||||||
|
```text
|
||||||
|
lister/inspecter variantes
|
||||||
|
lister/inspecter conflict cases
|
||||||
|
lire l'historique de résolution
|
||||||
|
résoudre/promouvoir/conserver/restaurer/réouvrir avec revision attendue
|
||||||
|
classifier les erreurs Store Transient / Terminal sans exposer le backend physique
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucune API publique ne doit exposer `tokio-postgres`, SQL, row handles ou détails provider.
|
||||||
|
|
||||||
|
### 4.4 Backend PostgreSQL et migration V003
|
||||||
|
|
||||||
|
Lire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-store-postgres-lib/README.md
|
||||||
|
crates/ksp-store-postgres-lib/USAGE.md
|
||||||
|
crates/ksp-store-postgres-lib/src/
|
||||||
|
crates/ksp-store-postgres-lib/tests/
|
||||||
|
crates/ksp-store-postgres-lib/unit_tests/
|
||||||
|
crates/ksp-store-postgres-lib/resources/
|
||||||
|
```
|
||||||
|
|
||||||
|
État acquis à préserver :
|
||||||
|
|
||||||
|
```text
|
||||||
|
V000/V001/V002 figés
|
||||||
|
V003 additive
|
||||||
|
ksp_raw_transaction_variants
|
||||||
|
ksp_raw_transaction_canonical_selectors
|
||||||
|
ksp_raw_transaction_observation_variants
|
||||||
|
ksp_raw_transaction_conflicts
|
||||||
|
```
|
||||||
|
|
||||||
|
Le sélecteur canonique et la projection V001 doivent rester atomiquement cohérents.
|
||||||
|
|
||||||
|
Toute extension physique de `0.3.17` doit être additive, migrationnée et justifiée par un contrat métier réel. Ne pas modifier rétroactivement les ressources/checksums stables.
|
||||||
|
|
||||||
|
Auditer avant conception :
|
||||||
|
|
||||||
|
```text
|
||||||
|
FOR UPDATE / ordre des locks
|
||||||
|
revision compare-and-set
|
||||||
|
participants d'un conflict case
|
||||||
|
historique append-only
|
||||||
|
reopen après résolution
|
||||||
|
races ingestion <-> résolution
|
||||||
|
races résolution <-> rétention
|
||||||
|
rollback transactionnel
|
||||||
|
idempotence des actions opérateur
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.5 Common RAW et règles de comparaison
|
||||||
|
|
||||||
|
Lire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-raw-transaction-lib/README.md
|
||||||
|
crates/ksp-raw-transaction-lib/USAGE.md
|
||||||
|
crates/ksp-raw-transaction-lib/src/
|
||||||
|
crates/ksp-raw-transaction-lib/tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Le comparateur `0.3.16` reste l'autorité de classification automatique. Ne pas élargir la dominance pour faciliter l'UI ou réduire artificiellement le nombre de conflits.
|
||||||
|
|
||||||
|
État acquis :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Exact
|
||||||
|
CompatibleLessComplete
|
||||||
|
CompatibleMoreComplete
|
||||||
|
Conflict
|
||||||
|
Incomparable
|
||||||
|
```
|
||||||
|
|
||||||
|
La seule dominance actuellement prouvée concerne les formes strictes de troncature `logMessages` documentées. Les autres différences restent fail-closed tant qu'une preuve spécifique n'existe pas.
|
||||||
|
|
||||||
|
### 4.6 Worker live
|
||||||
|
|
||||||
|
Lire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-worker-api/
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/README.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/src/
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/tests/
|
||||||
|
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
État acquis :
|
||||||
|
|
||||||
|
```text
|
||||||
|
chaque acquisition atteint le Store
|
||||||
|
convergence run-local = sérialisation seulement
|
||||||
|
QuarantinedConflict = succès durable non terminal
|
||||||
|
Worker reste Running
|
||||||
|
health devient Degraded
|
||||||
|
legacy ERROR_CODE_RAW_CONFLICT reste terminal uniquement comme compatibilité backend ancien
|
||||||
|
```
|
||||||
|
|
||||||
|
`0.3.17` doit ajouter le comportement Store transient sans perdre cet invariant.
|
||||||
|
|
||||||
|
La politique de retry doit expliciter :
|
||||||
|
|
||||||
|
```text
|
||||||
|
quels codes/classes sont Transient
|
||||||
|
quels codes/classes sont Terminal
|
||||||
|
borne de tentatives ou budget temporel
|
||||||
|
backoff borné
|
||||||
|
jitter éventuel et ownership
|
||||||
|
backpressure amont
|
||||||
|
état Blocked/Degraded/Unhealthy attendu
|
||||||
|
cancellation pendant sleep/backoff
|
||||||
|
shutdown/drain avec tentative en vol
|
||||||
|
absence de double persistence après résultat durable
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.7 Transport et Config
|
||||||
|
|
||||||
|
Lire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-onchain-transport-lib/README.md
|
||||||
|
crates/ksp-onchain-transport-lib/USAGE.md
|
||||||
|
crates/ksp-onchain-transport-lib/src/
|
||||||
|
crates/ksp-onchain-transport-lib/tests/
|
||||||
|
|
||||||
|
crates/ksp-config-lib/README.md
|
||||||
|
crates/ksp-config-lib/USAGE.md
|
||||||
|
crates/ksp-config-lib/src/
|
||||||
|
crates/ksp-config-lib/tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Réutiliser les mécanismes propriétaires actuels HTTP/WebSocket/Yellowstone. Ne pas créer une seconde pile réseau dans le Worker.
|
||||||
|
|
||||||
|
La reconnexion configurable doit conserver la séparation :
|
||||||
|
|
||||||
|
```text
|
||||||
|
reconnect attempt
|
||||||
|
session resume / from_slot / ReplayInfo lorsque disponible
|
||||||
|
replay delivery
|
||||||
|
coverage proof
|
||||||
|
```
|
||||||
|
|
||||||
|
Aucun de ces éléments ne vaut automatiquement un autre.
|
||||||
|
|
||||||
|
### 4.8 Job Backfill existant
|
||||||
|
|
||||||
|
Lire au minimum :
|
||||||
|
|
||||||
|
```text
|
||||||
|
crates/ksp-job-backfill-lib/README.md
|
||||||
|
crates/ksp-job-backfill-lib/USAGE.md
|
||||||
|
crates/ksp-job-backfill-lib/src/persistence.rs
|
||||||
|
crates/ksp-job-backfill-lib/unit_tests/persistence.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
Le Job Backfill reste un producteur indépendant et doit continuer de fonctionner avec les évolutions Store. `0.3.17` n'est pas la release du Backfill multi-route/multi-stratégie.
|
||||||
|
|
||||||
|
### 4.9 Architecture et références durables
|
||||||
|
|
||||||
|
Lire :
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/architecture/004-COMPONENT_INVENTORY.md
|
||||||
|
docs/architecture/009-ACQUISITION_WORKERS_AND_JOBS.md
|
||||||
|
docs/architecture/011-RAW_TRANSACTION_ACQUISITION.md
|
||||||
|
docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Ces documents décrivent l'état réconcilié à la fermeture de `0.3.16` et doivent servir de baseline, pas les anciens objectifs devenus obsolètes du prompt `035`.
|
||||||
|
|
||||||
|
## 5. Sources externes normatives à réauditer en `pre.001`
|
||||||
|
|
||||||
|
Réauditer uniquement les sources officielles ou upstream réellement nécessaires :
|
||||||
|
|
||||||
|
```text
|
||||||
|
PostgreSQL : transactions, row locking, deadlocks, serialization failures et SQLSTATE
|
||||||
|
Tokio PostgreSQL : erreurs de connexion/IO/DB et possibilités de classification sûres
|
||||||
|
Solana JSON-RPC/WebSocket : comportement de session/subscription/reconnect applicable
|
||||||
|
Yellowstone gRPC upstream : lifecycle de stream, fermeture, reprise/from_slot/ReplayInfo applicable
|
||||||
|
Tokio/Tonic/Tungstenite utilisés par les crates actuelles : cancellation/reconnect lorsque pertinent
|
||||||
|
```
|
||||||
|
|
||||||
|
La réaudition doit distinguer :
|
||||||
|
|
||||||
|
```text
|
||||||
|
fait documenté upstream
|
||||||
|
comportement observé par KSP
|
||||||
|
politique KSP choisie
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne jamais transformer un message d'erreur provider arbitraire en contrat public KSP.
|
||||||
|
|
||||||
|
## 6. État validé de `0.3.16` à préserver
|
||||||
|
|
||||||
|
À l'ouverture de `0.3.17`, considérer comme acquis uniquement ce qui est présent dans le tag `v0.3.16` et confirmé par ses deltas/validation.
|
||||||
|
|
||||||
|
La baseline attendue comprend :
|
||||||
|
|
||||||
|
```text
|
||||||
|
Store API comparator backend-neutral
|
||||||
|
V003 additive et historique V000/V001/V002 intact
|
||||||
|
variants durables
|
||||||
|
canonical selector revisionné
|
||||||
|
observation -> variant mapping
|
||||||
|
conflict case minimal durable
|
||||||
|
promotion CompatibleMoreComplete atomique
|
||||||
|
ancien canonique conservé comme variante
|
||||||
|
Conflict/Incomparable quarantinés sans mutation canonique
|
||||||
|
Worker Running/Degraded après QuarantinedConflict
|
||||||
|
Backfill projection Conflict compatible
|
||||||
|
PostgreSQL live proof exécutée sur PostgreSQL 17
|
||||||
|
```
|
||||||
|
|
||||||
|
Le gate final de `0.3.16` doit être lu dans `docs/validation/033-V0_3_16_RAW_RESILIENCE_CONFLICT.md` et dans les deltas de fermeture ; ne pas inventer un résultat non enregistré.
|
||||||
|
|
||||||
|
## 7. Cycle de vie complet des conflict cases
|
||||||
|
|
||||||
|
Le modèle minimal `0.3.16` ne suffit pas à une résolution opérateur complète.
|
||||||
|
|
||||||
|
`pre.001` doit cadrer précisément :
|
||||||
|
|
||||||
|
```text
|
||||||
|
identité stable du conflict case
|
||||||
|
participants/variantes concernées
|
||||||
|
status Open / Resolved et éventuels états supplémentaires réellement nécessaires
|
||||||
|
reason/relation conservées
|
||||||
|
revision monotone
|
||||||
|
resolved canonical variant
|
||||||
|
actor/reason opérateur si un contrat KSP sûr le justifie
|
||||||
|
journal append-only de transitions
|
||||||
|
reopen quand une nouvelle divergence survient
|
||||||
|
idempotence d'une résolution répétée
|
||||||
|
stale revision explicite
|
||||||
|
```
|
||||||
|
|
||||||
|
Une résolution ne doit jamais détruire immédiatement la variante perdante si elle reste nécessaire à l'historique, au rollback ou à un conflict case ouvert.
|
||||||
|
|
||||||
|
## 8. Actions et concurrence
|
||||||
|
|
||||||
|
Les actions opérateur futures de `0.3.18` doivent être rendues sûres par les contrats backend de `0.3.17`.
|
||||||
|
|
||||||
|
Auditer et définir au minimum :
|
||||||
|
|
||||||
|
```text
|
||||||
|
promote candidate
|
||||||
|
keep current canonical
|
||||||
|
restore previous variant
|
||||||
|
resolve conflict
|
||||||
|
reopen conflict
|
||||||
|
```
|
||||||
|
|
||||||
|
Chaque action mutable doit avoir une sémantique de revision attendue ou équivalent compare-and-set. Une action stale doit échouer explicitement sans écraser une décision concurrente.
|
||||||
|
|
||||||
|
Les races à couvrir incluent :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ingestion -> nouvelle variante pendant résolution
|
||||||
|
promotion automatique -> résolution manuelle concurrente
|
||||||
|
résolution A -> résolution B concurrente
|
||||||
|
réouverture -> résolution précédente
|
||||||
|
rétention/purge -> variante encore référencée
|
||||||
|
force rehydrate -> identité existante avec bytes exacts/différents
|
||||||
|
```
|
||||||
|
|
||||||
|
## 9. Rétention, pins et rollback
|
||||||
|
|
||||||
|
La rétention doit être définie avant tout purge de variante.
|
||||||
|
|
||||||
|
Principe :
|
||||||
|
|
||||||
|
```text
|
||||||
|
une variante nécessaire à un selector, conflict case ouvert, historique de résolution ou rollback autorisé ne peut pas disparaître silencieusement
|
||||||
|
```
|
||||||
|
|
||||||
|
`pre.001` doit décider :
|
||||||
|
|
||||||
|
```text
|
||||||
|
quels objets pin une variante
|
||||||
|
quand un pin peut être relâché
|
||||||
|
archive vs purge
|
||||||
|
interaction avec RawRetentionState V001
|
||||||
|
interaction avec ForceRehydrate
|
||||||
|
restore depuis variante locale conservée
|
||||||
|
comportement si une ancienne variante est déjà physiquement absente
|
||||||
|
```
|
||||||
|
|
||||||
|
Le réseau n'est pas un mécanisme normal de rollback d'une variante précédemment connue.
|
||||||
|
|
||||||
|
## 10. Classification Store `Transient / Terminal`
|
||||||
|
|
||||||
|
La classification doit être backend-neutral à la frontière publique tout en restant suffisamment précise pour le runtime.
|
||||||
|
|
||||||
|
Ne pas classifier par texte libre d'erreur.
|
||||||
|
|
||||||
|
`pre.001` doit produire une matrice au moins pour :
|
||||||
|
|
||||||
|
```text
|
||||||
|
connexion indisponible / reset
|
||||||
|
pool temporairement indisponible
|
||||||
|
timeout/cancellation locale
|
||||||
|
deadlock/serialization retryable si applicable
|
||||||
|
constraint/data-invalid
|
||||||
|
migration/schema mismatch
|
||||||
|
network mismatch KSP
|
||||||
|
stale revision
|
||||||
|
retention conflict
|
||||||
|
secret/config invalid
|
||||||
|
```
|
||||||
|
|
||||||
|
La classe publique doit être stable et sûre ; le backend conserve les détails physiques privés.
|
||||||
|
|
||||||
|
## 11. Retry Store, backpressure et état `Blocked`
|
||||||
|
|
||||||
|
Le retry appartient au Worker/runtime consumer, pas au backend PostgreSQL sous forme de boucle cachée illimitée.
|
||||||
|
|
||||||
|
Le Store backend peut classifier et retourner ; le Worker décide une politique bornée selon les contrats KSP.
|
||||||
|
|
||||||
|
Le design doit éviter :
|
||||||
|
|
||||||
|
```text
|
||||||
|
queue mémoire non bornée
|
||||||
|
retry storm
|
||||||
|
sleep non cancellable
|
||||||
|
perte silencieuse après exhaustion
|
||||||
|
health Healthy pendant une impossibilité persistante de commit
|
||||||
|
double écriture après succès ambigu sans idempotence
|
||||||
|
```
|
||||||
|
|
||||||
|
La relation avec les limites actuelles `in_flight`/admission doit être explicitement testée.
|
||||||
|
|
||||||
|
## 12. Reconnexion Transport configurable
|
||||||
|
|
||||||
|
`0.3.17` étend les mécanismes existants ; il ne crée pas une abstraction universelle inventée au-dessus de toutes les routes.
|
||||||
|
|
||||||
|
La configuration doit être bornée et cohérente avec les contrats Config existants :
|
||||||
|
|
||||||
|
```text
|
||||||
|
attempt/backoff limits
|
||||||
|
connect/session deadlines
|
||||||
|
stop preemption
|
||||||
|
source supervision
|
||||||
|
replay/from_slot lorsque réellement supporté
|
||||||
|
```
|
||||||
|
|
||||||
|
Une reconnexion réussie ne ferme aucun gap à elle seule. Les contrats `0.3.14` de coverage/gap repair restent autoritaires.
|
||||||
|
|
||||||
|
## 13. Frontières de crates et hors UI
|
||||||
|
|
||||||
|
Frontières cibles :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ksp-store-api
|
||||||
|
DTOs/capabilities backend-neutral
|
||||||
|
|
||||||
|
ksp-store-postgres-lib
|
||||||
|
SQL, locks, transactions, migrations, classification physique
|
||||||
|
|
||||||
|
ksp-store-lib
|
||||||
|
façade et dispatch backend-neutral
|
||||||
|
|
||||||
|
ksp-worker-raw-transaction-ingest-lib
|
||||||
|
politique retry/backpressure/health/cancellation
|
||||||
|
|
||||||
|
ksp-onchain-transport-lib
|
||||||
|
sessions/reconnect/replay transport-owned
|
||||||
|
|
||||||
|
ksp-config-lib
|
||||||
|
configuration bornée des mécanismes existants
|
||||||
|
```
|
||||||
|
|
||||||
|
`ksp-app-store-desk` n'est pas à développer dans `0.3.17`. Seuls les contrats nécessaires à sa future utilisation doivent être stabilisés.
|
||||||
|
|
||||||
|
## 14. Sécurité et redaction
|
||||||
|
|
||||||
|
Ne jamais exposer dans erreurs/logs/snapshots publics :
|
||||||
|
|
||||||
|
```text
|
||||||
|
URI PostgreSQL
|
||||||
|
password/token/API key
|
||||||
|
endpoint complet lorsque sa publication n'est pas nécessaire
|
||||||
|
payload RAW
|
||||||
|
signature brute si les règles de surface l'interdisent
|
||||||
|
server error arbitraire
|
||||||
|
SQL complet
|
||||||
|
row/backend handle
|
||||||
|
```
|
||||||
|
|
||||||
|
Les diagnostics de conflits existants restent bornés et structurels.
|
||||||
|
|
||||||
|
Toute nouvelle action de résolution doit journaliser seulement les identifiants/codes sûrs nécessaires à l'audit.
|
||||||
|
|
||||||
|
## 15. Première mission `pre.001` — audit, brainstorming et sizing
|
||||||
|
|
||||||
|
Avant toute implémentation lourde :
|
||||||
|
|
||||||
|
1. vérifier que la base est exactement `v0.3.16` ;
|
||||||
|
2. lire toutes les sources internes de la section 4 ;
|
||||||
|
3. réauditer les sources externes pertinentes de la section 5 ;
|
||||||
|
4. inventorier les contrats publics actuels Store/Worker/Transport/Config ;
|
||||||
|
5. produire le modèle conceptuel complet conflict case / participants / history / resolution / reopen ;
|
||||||
|
6. produire la stratégie de revision/CAS et la matrice des races ;
|
||||||
|
7. produire la politique de rétention/pins/rollback/ForceRehydrate ;
|
||||||
|
8. produire la classification `Transient / Terminal` et son ownership ;
|
||||||
|
9. produire la politique Worker de retry/backpressure/Blocked/cancellation ;
|
||||||
|
10. produire la politique Transport reconnect/config sans confondre coverage ;
|
||||||
|
11. auditer la compatibilité du Job Backfill existant ;
|
||||||
|
12. dimensionner les migrations/tests/live proofs ;
|
||||||
|
13. créer ou réviser le plan `0.3.17` avec prereleases bornées ;
|
||||||
|
14. publier seulement ensuite `0.3.17-pre.001`.
|
||||||
|
|
||||||
|
Critères de sortie `pre.001` :
|
||||||
|
|
||||||
|
```text
|
||||||
|
contrats Store API proposés et bornés
|
||||||
|
modèle conflict lifecycle validé
|
||||||
|
revision/race semantics validées
|
||||||
|
rétention/pins validés
|
||||||
|
classification Transient/Terminal validée
|
||||||
|
retry ownership et bornes validés
|
||||||
|
reconnect ownership et bornes validés
|
||||||
|
compatibilité Backfill explicitée
|
||||||
|
migration strategy validée
|
||||||
|
preuves live nécessaires identifiées
|
||||||
|
prévision souple des prereleases publiée
|
||||||
|
hors-périmètre explicite
|
||||||
|
```
|
||||||
|
|
||||||
|
## 16. Prévision souple initiale des prereleases
|
||||||
|
|
||||||
|
Cette trajectoire provient du plan réconcilié `0.3.16`. Elle doit être réauditée en `pre.001` et peut être subdivisée si le sizing l'exige.
|
||||||
|
|
||||||
|
```text
|
||||||
|
pre.001 audit + contrats Store API inspection/résolution + classification Transient/Terminal + plan
|
||||||
|
pre.002 conflict cases complets : participants, résolution, reopen, historique append-only, races
|
||||||
|
pre.003 rétention des variantes : archive, pins, purge guards, rollback local, ForceRehydrate
|
||||||
|
pre.004 ksp-store-lib : inspection/historique/actions backend-neutral + compatibilité Backfill
|
||||||
|
pre.005 Worker : Store retry/backpressure/Blocked, exhaustion, cancellation et drain pendant backoff
|
||||||
|
pre.006 Transport/Config : reconnect WebSocket/Yellowstone configurable sur mécanismes existants
|
||||||
|
pre.007 hardening cross-layer + preuves live ciblées Store outage/recovery, retry, reconnect, concurrence
|
||||||
|
pre.008 réconciliation documentaire finale
|
||||||
|
pre.009 préparation de publication : CHANGELOG + ROADMAP + prompt 0.3.18
|
||||||
|
rel.001 publication stable mécanique
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne jamais fusionner artificiellement les couloirs de fermeture. Les règles `VER-LIFECYCLE-*` restent autoritaires si le nombre de tranches évolue.
|
||||||
|
|
||||||
|
## 17. Hors périmètre de `0.3.17`
|
||||||
|
|
||||||
|
Ne pas inclure :
|
||||||
|
|
||||||
|
```text
|
||||||
|
frontend/backend Tauri Store Desk de résolution -> 0.3.18
|
||||||
|
backfill multi-route/multi-stratégie -> 0.3.19
|
||||||
|
adaptation Backfill Desk -> 0.3.20
|
||||||
|
RAW -> STRUCTURAL
|
||||||
|
persistence STRUCTURAL
|
||||||
|
DECODED / DOMAIN
|
||||||
|
majority voting provider
|
||||||
|
priorité provider
|
||||||
|
fusion heuristique de JSON arbitraire
|
||||||
|
retry illimité caché dans le backend Store
|
||||||
|
nouvelle pile réseau Worker parallèle à Transport
|
||||||
|
nouvelle dépendance Worker <-> Backfill
|
||||||
|
```
|
||||||
|
|
||||||
|
## 18. Versionnement, deltas, validation et instruction d'ouverture
|
||||||
|
|
||||||
|
Respecter strictement `docs/rules/VERSION_WORKFLOW.md`.
|
||||||
|
|
||||||
|
Les deltas de `0.3.17` utilisent :
|
||||||
|
|
||||||
|
```text
|
||||||
|
ksp-general-0.3.17-pre.NNN.zip
|
||||||
|
ksp-general-0.3.17-pre.NNN-fix.NNN.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Après toute modification Rust :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --all
|
||||||
|
cargo fmt --all -- --check
|
||||||
|
python3 scripts/audit_rust_workspace_rules.py
|
||||||
|
cargo check --workspace
|
||||||
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
Pour tout Markdown touché :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas
|
||||||
|
```
|
||||||
|
|
||||||
|
Les tests ciblés sont préférés pendant le développement ; les gates complets et live proofs sont réservés aux tranches prévues et ne sont déclarés PASS que lorsqu'ils ont réellement été exécutés.
|
||||||
|
|
||||||
|
Les tests PostgreSQL live qui lisent une URI dédiée sur `stdin` doivent continuer à ne jamais l'échoer ni la conserver dans les logs/deltas.
|
||||||
|
|
||||||
|
Pour les applications Desk, ne pas utiliser `npm run build` comme gate manuel. `0.3.17` ne développe normalement pas la Store Desk ; si un smoke Tauri devenait nécessaire pour une compatibilité mécanique, utiliser le workflow Tauri/Vite KSP prévu par les règles.
|
||||||
|
|
||||||
|
### Instruction d'ouverture de la prochaine session
|
||||||
|
|
||||||
|
Commencer par :
|
||||||
|
|
||||||
|
```text
|
||||||
|
1. vérifier que la base est exactement le tag stable v0.3.16 ;
|
||||||
|
2. lire toutes les sources internes de la section 4 dans l'ordre ;
|
||||||
|
3. réauditer les sources externes pertinentes de la section 5 ;
|
||||||
|
4. exécuter la mission pre.001 de la section 15 ;
|
||||||
|
5. produire le plan/sizing avant toute migration ou développement lourd.
|
||||||
|
```
|
||||||
|
|
||||||
|
Ne pas commencer par l'UI Store Desk, par une boucle de retry ou par une migration avant d'avoir fermé le gate `pre.001`.
|
||||||
|
|
||||||
|
La trajectoire suivante est :
|
||||||
|
|
||||||
|
```text
|
||||||
|
0.3.17 résilience opérationnelle + cycle de vie des variantes
|
||||||
|
0.3.18 Store Desk inspection/résolution des variantes et conflits
|
||||||
|
0.3.19 Backfill multi-route/multi-stratégie
|
||||||
|
0.3.20 Backfill Desk correspondant
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user