Compare commits
28 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bcce89f076 | |||
| 66d5b03495 | |||
| 7af751c888 | |||
| 9f2d5ea704 | |||
| 8428b75b96 | |||
| 282b616a0e | |||
| 12eaf7a3de | |||
| f28e4d87f0 | |||
| ba2e52f841 | |||
| 5f1938c7c3 | |||
| 5e2f959c01 | |||
| 8da04717b6 | |||
| c094828704 | |||
| ce658e58f7 | |||
| 7f0a642972 | |||
| 84ab2b3651 | |||
| 1f0b202135 | |||
| aa56a12846 | |||
| 10996f11f7 | |||
| 1352a61d70 | |||
| 66a3926adb | |||
| 917e602a87 | |||
| 56fadb364a | |||
| 8fa1c8ce8d | |||
| c17e78c6a8 | |||
| 61bf7ba468 | |||
| 9712c7e1f7 | |||
| a560df80ce |
14
CHANGELOG.md
14
CHANGELOG.md
@@ -1,8 +1,20 @@
|
||||
<!-- file: CHANGELOG.md -->
|
||||
<!-- version: 20 -->
|
||||
<!-- version: 22 -->
|
||||
|
||||
# Changelog KSP
|
||||
|
||||
## 0.3.3 — Store/PostgreSQL RawTransaction vertical slice — 2026-08-30
|
||||
|
||||
`0.3.3` complète la première vertical slice RAW physique sur le couple `ksp-store-lib` / `ksp-store-postgres-lib` sans modifier les contrats backend-agnostic acquis dans `ksp-store-api`. `PostgresBackend` et la façade `Store` implémentent désormais les six capabilities `RawTransactionRead`, `RawTransactionWrite`, `RawTransactionObservationRead`, `RawTransactionObservationWrite`, `RawTransactionRetentionRead` et `RawTransactionRetentionWrite`. Une base PostgreSQL reste liée à un unique `RawNetworkId` par `ksp_store_identity`; le mauvais réseau est refusé avant I/O, les slots `u64` sont conservés exactement en `NUMERIC(20,0)`, et la migration logique V001 reste découpée en ressources tables/contraintes/indexes avec vérification de compatibilité du schéma effectif.
|
||||
|
||||
La persistence canonique utilise une transaction PostgreSQL unique pour `RawTransaction + observation`, tente l'insert sous contrainte d'unicité puis compare le contenu réel sous verrou en cas de collision : contenu identique -> idempotence, divergence -> `store_api.raw_conflict`. Les observations supplémentaires conservent leur provenance complète et ne créent jamais implicitement un canonical absent. La lecture reconstruit `Full` depuis le payload chaud et `Archived` depuis la relation archive ; `Purged` reste un tombstone minimal. La navigation est une keyset stricte `(slot, signature)` ASC/DESC sans `OFFSET`, avec cursor V1 opaque de 109 octets lié au réseau, à la direction, aux bornes et à la dernière clé. Aucun plafond métier de batch n'est introduit : seule la borne physique PostgreSQL de `LIMIT requested + 1` est appliquée.
|
||||
|
||||
La rétention physique supportée est `Full -> Archived -> Purged`, sérialisée par `FOR UPDATE`, avec outcomes `Applied`, `AlreadyAtTarget` et `ExpectedStateMismatch`. Le mode normal respecte le tombstone après purge ; `ForceRehydrate` reste explicite et atomique. `Compacted` demeure volontairement non supporté physiquement et retourne `store.postgres_retention_compaction_unsupported` plutôt que de présenter TOAST ou une compression implicite comme contrat KSP. Les erreurs backend sont réduites à des classes/phases statiques puis projetées vers des codes Store/API stables, sans URI, SQL, SQLSTATE, bind ni texte serveur.
|
||||
|
||||
La preuve PostgreSQL réelle a conduit à durcir l'introspection de schéma PostgreSQL 17 : canonicalisation ciblée des CHECK numériques reconstruits par le catalogue, conservation des littéraux texte, restauration des helpers de classification de schéma et distinction d'un drift d'une migration déjà enregistrée lorsque `schema_autoupdate=false`. Les ressources SQL V000/V001 et leurs checksums sont restés inchangés pendant ces corrections (`V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450`, `V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51`).
|
||||
|
||||
Le gate technique final `pre.011` passe audits Rust/Markdown, `cargo check --workspace`, Clippy all-targets, les tests ciblés Store/API/PostgreSQL/Config, les tests et checks façade avec `--no-default-features`, `cargo test --workspace` et les graphes Cargo. Le live `postgres_raw_transaction_live` est ensuite rejoué avec succès sur **PostgreSQL 17**, couvrant bootstrap/drift-repair, atomicité, concurrence identique/divergente, rollback sur collision et annulation, pagination/cursor, rétention/races, ForceRehydrate et réouverture durable. Aucun build Tauri supplémentaire n'est requis : `0.3.3` ne change ni resources applicatives ni packaging desktop. `RawAccountState` PostgreSQL et la complétude RAW restent réservés à `0.3.4`. `prompts/023-V0_3_4_START_PROMPT.md` ouvre cette slice suivante sur les quatre capabilities `RawAccount*`, une migration additive au-dessus de V000/V001, puis la conformance finale des dix capabilities RAW ; l'archive historique kbot3 y reste une source de comparaison ciblée account/observation, jamais une architecture à recopier.
|
||||
|
||||
## 0.3.2 — Store/PostgreSQL runtime foundation — 2026-08-30
|
||||
|
||||
`0.3.2` introduit ensemble `ksp-store-lib` et `ksp-store-postgres-lib` comme fondation runtime/backend PostgreSQL au-dessus des contrats backend-agnostic de `ksp-store-api`, sans encore implémenter de capability RAW métier. La façade `Store` conserve un backend connu `Postgres` activé par défaut via Cargo feature, reste compilable avec `--no-default-features`, lie chaque instance à exactement un `RawNetworkId` et n'expose aucun `Pool`, `Client`, `Row`, `Statement`, driver ou SQL physique. `ksp-config-lib` possède désormais `std.store` avec trois targets nommés `devnet`, `mainnet` et `testnet`, chacun associé à un réseau explicite et à une URI PostgreSQL Secret indépendante ; Store/backend ne lisent directement ni `.env`, ni `KSP_*`, ni `PG*`, ni `.pgpass`.
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# file: Cargo.toml
|
||||
# version: 347
|
||||
# version: 363
|
||||
|
||||
[workspace]
|
||||
resolver = "3"
|
||||
members = ["crates/ksp-app-config-desk", "crates/ksp-app-solprices-desk", "crates/ksp-app-wallet-desk", "crates/ksp-config-lib", "crates/ksp-core-lib", "crates/ksp-interface-lib", "crates/ksp-logging-lib", "crates/ksp-offchain-transport-lib", "crates/ksp-onchain-transport-lib", "crates/ksp-program-api", "crates/ksp-store-api", "crates/ksp-store-lib", "crates/ksp-store-postgres-lib", "crates/ksp-wallet-lib"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.3.2"
|
||||
version = "0.3.3"
|
||||
edition = "2024"
|
||||
license = "MIT"
|
||||
repository = "https://git.sasedev.com/Sasedev/khadhroony-solana-project"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: ROADMAP.md -->
|
||||
<!-- version: 95 -->
|
||||
<!-- version: 97 -->
|
||||
|
||||
# Roadmap KSP
|
||||
|
||||
@@ -95,8 +95,8 @@ RAW -> STRUCTURAL -> DECODED -> DOMAIN
|
||||
|
||||
- [X] `0.3.1` — `ksp-store-api` stable : modèles N1 RAW backend-agnostic `RawTransaction` et `RawAccountState` avec observations, provenance, payload/hash/timestamps bornés, 10 capabilities object-safe, queries cursorisées sans plafond métier arbitraire, outcomes idempotence/conflit et lifecycle logique rétention/tombstone/force-rehydrate ; aucun backend physique, Config, runtime Store, notification dédiée ni surface STRUCTURAL/DECODED/DOMAIN.
|
||||
- [X] `0.3.2` — `ksp-store-lib` + `ksp-store-postgres-lib` stables comme fondation runtime/backend PostgreSQL : feature `postgres` par défaut, `Store` lié à un unique `RawNetworkId`, Config `std.store` avec targets/bases `devnet`/`mainnet`/`testnet`, pool Deadpool borné, `tokio-postgres`, TLS Rustls `Disabled`/`VerifyFull`, moteur de migrations privé `V000` + SHA-256/advisory lock, health/readiness portable et close borné. Gate complet + PostgreSQL réel major 17 verts ; aucune table/capability `RawTransaction`/`RawAccountState` métier n'est encore ajoutée.
|
||||
- [ ] `0.3.3` — Étendre le même couple avec la vertical slice PostgreSQL `RawTransaction` complète : les six capabilities transaction/observation/rétention de `ksp-store-api`, persistence acquisition atomique, get/list cursorisé, idempotence/conflit, binding réseau, rétention/tombstone/force-rehydrate, concurrence et rollback validés sur PostgreSQL réel.
|
||||
- [ ] `0.3.4` — Étendre le même couple avec `RawAccountState` + observation, puis fermer la complétude/conformance RAW cross-family, les indexes/migrations physiques nécessaires et le hardening PostgreSQL final.
|
||||
- [X] `0.3.3` — Vertical slice PostgreSQL `RawTransaction` complète sur `ksp-store-lib` + `ksp-store-postgres-lib` : six capabilities transaction/observation/rétention, V001 physique liée à un réseau, acquisition canonical+observation atomique, idempotence/conflit, get/list keyset cursorisé, archive/purge/tombstone/ForceRehydrate, hardening des erreurs et du schéma, concurrence et rollback validés sur PostgreSQL 17.
|
||||
- [ ] `0.3.4` — Étendre le même couple avec `RawAccountState` + `RawAccountObservation` : quatre capabilities account, migration additive au-dessus de V000/V001, acquisition state+observation atomique, idempotence/conflit, get/list cursorisé, puis complétude des dix capabilities RAW, indexes justifiés par les queries et hardening PostgreSQL cross-family final.
|
||||
- [ ] `0.3.5` — Étendre `ksp-interface-lib` uniquement avec les modèles passifs/events réellement partagés par les premiers consumers d’acquisition, sans dupliquer les modèles persistants de `ksp-store-api`.
|
||||
- [ ] `0.3.6` — Introduire `ksp-job-api` et un premier job de backfill historique concret consommant `ksp-store-lib`, avec policy/batch-size/progression possédés par le job et non par Store.
|
||||
- [ ] `0.3.7` — Introduire une application spécialisée de backfill/inspection RAW.
|
||||
@@ -113,7 +113,8 @@ RAW -> STRUCTURAL -> DECODED -> DOMAIN
|
||||
- [ ] **REJET ACTUEL** — Yellowstone `Entry` : trop bas niveau et aucune destination replay/decomposition/event métier justifiant un modèle KSP n’est identifiée.
|
||||
- [ ] **TODO** — processing ledger : reprendre l’idée kbot2/kbot3 `stage + processor identity/version + input identity/hash + terminal status`, sans faire d’un `processed: bool` la preuve durable unique ; prévoir force replay/version upgrades lorsque les processors seront ouverts.
|
||||
- [X] lifecycle RAW logique — `RawRetentionState`, tombstone minimal, normal-skip et force-rehydrate sont stabilisés en `0.3.1` pour `RawTransaction`.
|
||||
- [ ] **TODO** — rétention physique : définir plus tard compression/archive backend, critères d’éligibilité fondés sur les preuves de processing et maintenance worker/job ; Store applique une transition demandée mais ne décide pas seul qu’un RAW peut être purgé.
|
||||
- [X] rétention physique `RawTransaction` PostgreSQL — `0.3.3` matérialise `Full -> Archived -> Purged`, tombstone et ForceRehydrate atomiques ; `Compacted` reste explicitement unsupported tant qu’aucune représentation compactée réelle n’existe.
|
||||
- [ ] **TODO** — policy de rétention/compaction : définir les critères d’éligibilité fondés sur les preuves de processing et la maintenance worker/job ; Store applique une transition demandée mais ne décide pas seul qu’un RAW peut être archivé/purgé, et la compaction physique ne sera ajoutée qu’avec un besoin réel.
|
||||
- [X] frontière `ksp-interface-lib` / `ksp-store-api` — ownership documenté et canaris de non-duplication stabilisés en `0.3.1`; les events passifs non persistés restent Interface, les modèles persistants/replayables restent Store API.
|
||||
- [ ] **IDEA** — réauditer la structure de processing/decode/materialization historique kbot2/kbot3 lors de l’ouverture de N2/N3 ; conserver l’isolation instruction/CPI et les statuts terminal/versionnés, sans reprendre automatiquement le schéma SQL historique.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"format_version": 2,
|
||||
"default_profile": "devnet",
|
||||
"profiles": [
|
||||
{
|
||||
@@ -19,7 +19,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -43,7 +44,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -67,7 +69,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "urn:ksp:schema:std.store:v1",
|
||||
"$id": "urn:ksp:schema:std.store:v2",
|
||||
"title": "KSP standard Store configuration",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
@@ -11,19 +11,45 @@
|
||||
],
|
||||
"properties": {
|
||||
"format_version": {
|
||||
"const": 1
|
||||
"enum": [
|
||||
1,
|
||||
2
|
||||
]
|
||||
},
|
||||
"default_profile": {
|
||||
"$ref": "#/$defs/profileId"
|
||||
},
|
||||
"profiles": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"minItems": 1
|
||||
}
|
||||
},
|
||||
"oneOf": [
|
||||
{
|
||||
"properties": {
|
||||
"format_version": {
|
||||
"const": 1
|
||||
},
|
||||
"profiles": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/profile"
|
||||
"$ref": "#/$defs/profileV1"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"format_version": {
|
||||
"const": 2
|
||||
},
|
||||
"profiles": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/profileV2"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"$defs": {
|
||||
"profileId": {
|
||||
"type": "string",
|
||||
@@ -40,7 +66,7 @@
|
||||
"minimum": 100,
|
||||
"maximum": 60000
|
||||
},
|
||||
"profile": {
|
||||
"profileV1": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
@@ -60,11 +86,35 @@
|
||||
"const": "postgres"
|
||||
},
|
||||
"postgres": {
|
||||
"$ref": "#/$defs/postgres"
|
||||
"$ref": "#/$defs/postgresV1"
|
||||
}
|
||||
}
|
||||
},
|
||||
"profileV2": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"profile_id",
|
||||
"network",
|
||||
"backend",
|
||||
"postgres"
|
||||
],
|
||||
"properties": {
|
||||
"profile_id": {
|
||||
"$ref": "#/$defs/profileId"
|
||||
},
|
||||
"network": {
|
||||
"$ref": "#/$defs/networkId"
|
||||
},
|
||||
"backend": {
|
||||
"const": "postgres"
|
||||
},
|
||||
"postgres": {
|
||||
"$ref": "#/$defs/postgresV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
"postgresV1": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
@@ -86,7 +136,38 @@
|
||||
"$ref": "#/$defs/tls"
|
||||
},
|
||||
"bootstrap": {
|
||||
"$ref": "#/$defs/bootstrap"
|
||||
"$ref": "#/$defs/bootstrapV1"
|
||||
},
|
||||
"shutdown_timeout_ms": {
|
||||
"type": "integer",
|
||||
"minimum": 100,
|
||||
"maximum": 30000
|
||||
}
|
||||
}
|
||||
},
|
||||
"postgresV2": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"connection_uri",
|
||||
"pool",
|
||||
"tls",
|
||||
"bootstrap",
|
||||
"shutdown_timeout_ms"
|
||||
],
|
||||
"properties": {
|
||||
"connection_uri": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"pool": {
|
||||
"$ref": "#/$defs/pool"
|
||||
},
|
||||
"tls": {
|
||||
"$ref": "#/$defs/tls"
|
||||
},
|
||||
"bootstrap": {
|
||||
"$ref": "#/$defs/bootstrapV2"
|
||||
},
|
||||
"shutdown_timeout_ms": {
|
||||
"type": "integer",
|
||||
@@ -140,7 +221,7 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"bootstrap": {
|
||||
"bootstrapV1": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
@@ -163,6 +244,34 @@
|
||||
"maximum": 120000
|
||||
}
|
||||
}
|
||||
},
|
||||
"bootstrapV2": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schema_autocreate",
|
||||
"schema_autoupdate",
|
||||
"migration_timeout_ms",
|
||||
"migration_lock_timeout_ms"
|
||||
],
|
||||
"properties": {
|
||||
"schema_autocreate": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"schema_autoupdate": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"migration_timeout_ms": {
|
||||
"type": "integer",
|
||||
"minimum": 1000,
|
||||
"maximum": 300000
|
||||
},
|
||||
"migration_lock_timeout_ms": {
|
||||
"type": "integer",
|
||||
"minimum": 100,
|
||||
"maximum": 120000
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"format_version": 2,
|
||||
"default_profile": "devnet",
|
||||
"profiles": [
|
||||
{
|
||||
@@ -19,7 +19,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -43,7 +44,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -67,7 +69,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-config-lib/src/store.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
/// Effective standard Store configuration mapped to `ksp_store_lib::StoreSettings`.
|
||||
pub struct ResolvedStoreConfig {
|
||||
@@ -153,9 +153,11 @@ struct EffectivePostgresTlsSource {
|
||||
#[derive(serde::Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct EffectivePostgresBootstrapSource {
|
||||
auto_migrate: bool,
|
||||
auto_migrate: std::option::Option<bool>,
|
||||
migration_lock_timeout_ms: u64,
|
||||
migration_timeout_ms: u64,
|
||||
schema_autocreate: std::option::Option<bool>,
|
||||
schema_autoupdate: std::option::Option<bool>,
|
||||
}
|
||||
|
||||
fn resolve_store_profile(profile: &crate::ResolvedConfigProfile, environment: &crate::ConfigEnvironment) -> ksp_core_lib::Result<ResolvedStoreConfig> {
|
||||
@@ -178,7 +180,7 @@ fn resolve_store_profile(profile: &crate::ResolvedConfigProfile, environment: &c
|
||||
);
|
||||
},
|
||||
};
|
||||
if source.format_version != 1 {
|
||||
if source.format_version != 1 && source.format_version != 2 {
|
||||
return std::result::Result::Err(effective_error(profile, "effective Store format_version is unsupported"));
|
||||
}
|
||||
if source.profile_id != profile.profile_id() {
|
||||
@@ -199,11 +201,34 @@ fn resolve_store_profile(profile: &crate::ResolvedConfigProfile, environment: &c
|
||||
std::time::Duration::from_millis(source.postgres.pool.create_timeout_ms),
|
||||
std::time::Duration::from_millis(source.postgres.pool.recycle_timeout_ms),
|
||||
);
|
||||
let bootstrap = ksp_store_lib::PostgresBootstrapSettings::new(
|
||||
source.postgres.bootstrap.auto_migrate,
|
||||
let bootstrap = match source.format_version {
|
||||
1 => {
|
||||
let auto_migrate =
|
||||
match (source.postgres.bootstrap.auto_migrate, source.postgres.bootstrap.schema_autocreate, source.postgres.bootstrap.schema_autoupdate) {
|
||||
(std::option::Option::Some(value), std::option::Option::None, std::option::Option::None) => value,
|
||||
_ => return std::result::Result::Err(effective_error(profile, "effective Store V1 bootstrap policy is invalid")),
|
||||
};
|
||||
ksp_store_lib::PostgresBootstrapSettings::new(
|
||||
auto_migrate,
|
||||
std::time::Duration::from_millis(source.postgres.bootstrap.migration_timeout_ms),
|
||||
std::time::Duration::from_millis(source.postgres.bootstrap.migration_lock_timeout_ms),
|
||||
);
|
||||
)
|
||||
},
|
||||
2 => {
|
||||
let (schema_autocreate, schema_autoupdate) =
|
||||
match (source.postgres.bootstrap.auto_migrate, source.postgres.bootstrap.schema_autocreate, source.postgres.bootstrap.schema_autoupdate) {
|
||||
(std::option::Option::None, std::option::Option::Some(autocreate), std::option::Option::Some(autoupdate)) => (autocreate, autoupdate),
|
||||
_ => return std::result::Result::Err(effective_error(profile, "effective Store V2 bootstrap policy is invalid")),
|
||||
};
|
||||
ksp_store_lib::PostgresBootstrapSettings::with_schema_policy(
|
||||
schema_autocreate,
|
||||
schema_autoupdate,
|
||||
std::time::Duration::from_millis(source.postgres.bootstrap.migration_timeout_ms),
|
||||
std::time::Duration::from_millis(source.postgres.bootstrap.migration_lock_timeout_ms),
|
||||
)
|
||||
},
|
||||
_ => return std::result::Result::Err(effective_error(profile, "effective Store format_version is unsupported")),
|
||||
};
|
||||
let network = ksp_store_lib::RawNetworkId::new(source.network);
|
||||
let network = match network {
|
||||
std::result::Result::Ok(value) => value,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"format_version": 2,
|
||||
"default_profile": "devnet",
|
||||
"profiles": [
|
||||
{
|
||||
@@ -19,7 +19,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -43,7 +44,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
@@ -67,7 +69,8 @@
|
||||
"mode": "verify_full"
|
||||
},
|
||||
"bootstrap": {
|
||||
"auto_migrate": true,
|
||||
"schema_autocreate": true,
|
||||
"schema_autoupdate": true,
|
||||
"migration_timeout_ms": 30000,
|
||||
"migration_lock_timeout_ms": 10000
|
||||
},
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-config-lib/unit_tests/store.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
#[test]
|
||||
fn committed_store_profile_maps_exact_runtime_settings_and_secret_fallback() {
|
||||
@@ -32,6 +32,8 @@ fn committed_store_profile_maps_exact_runtime_settings_and_secret_fallback() {
|
||||
assert_eq!(postgres.pool().recycle_timeout(), std::time::Duration::from_millis(5_000));
|
||||
assert_eq!(postgres.tls_mode(), ksp_store_lib::PostgresTlsMode::VerifyFull);
|
||||
assert!(postgres.bootstrap().auto_migrate());
|
||||
assert!(postgres.bootstrap().schema_autocreate());
|
||||
assert!(postgres.bootstrap().schema_autoupdate());
|
||||
assert_eq!(postgres.bootstrap().migration_timeout(), std::time::Duration::from_millis(30_000));
|
||||
assert_eq!(postgres.bootstrap().migration_lock_timeout(), std::time::Duration::from_millis(10_000));
|
||||
}
|
||||
@@ -140,6 +142,90 @@ fn named_store_targets_select_one_network_and_database_without_runtime_multiplex
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_v1_auto_migrate_remains_backward_readable_and_maps_both_schema_policies() {
|
||||
for auto_migrate in [false, true] {
|
||||
let fixture = tempfile::tempdir();
|
||||
assert!(fixture.is_ok());
|
||||
let fixture = match fixture {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
let source = committed_document_value();
|
||||
let mut source = match source {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
source["format_version"] = serde_json::Value::from(1);
|
||||
let profiles = source.get_mut("profiles").and_then(serde_json::Value::as_array_mut);
|
||||
if let std::option::Option::Some(profiles) = profiles {
|
||||
for profile in profiles {
|
||||
let bootstrap = profile.pointer_mut("/postgres/bootstrap").and_then(serde_json::Value::as_object_mut);
|
||||
if let std::option::Option::Some(bootstrap) = bootstrap {
|
||||
bootstrap.remove("schema_autocreate");
|
||||
bootstrap.remove("schema_autoupdate");
|
||||
bootstrap.insert("auto_migrate".to_owned(), serde_json::Value::Bool(auto_migrate));
|
||||
}
|
||||
}
|
||||
}
|
||||
let engine = fixture_engine_with_document(fixture.path(), &source);
|
||||
assert!(engine.is_ok());
|
||||
let engine = match engine {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
let environment = crate::ConfigEnvironment::from_maps(std::collections::BTreeMap::new(), std::collections::BTreeMap::new());
|
||||
let resolved = engine.load_resolved_store_config(std::option::Option::None, &environment);
|
||||
assert!(resolved.is_ok(), "Store Config V1 compatibility mapping failed: {resolved:?}");
|
||||
if let std::result::Result::Ok(resolved) = resolved
|
||||
&& let ksp_store_lib::StoreBackendSettings::Postgres(postgres) = resolved.settings().backend()
|
||||
{
|
||||
assert_eq!(postgres.bootstrap().schema_autocreate(), auto_migrate);
|
||||
assert_eq!(postgres.bootstrap().schema_autoupdate(), auto_migrate);
|
||||
assert_eq!(postgres.bootstrap().auto_migrate(), auto_migrate);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_v2_schema_creation_and_update_policies_map_independently() {
|
||||
let fixture = tempfile::tempdir();
|
||||
assert!(fixture.is_ok());
|
||||
let fixture = match fixture {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
let source = committed_document_value();
|
||||
let mut source = match source {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
let profiles = source.get_mut("profiles").and_then(serde_json::Value::as_array_mut);
|
||||
if let std::option::Option::Some(profiles) = profiles
|
||||
&& let std::option::Option::Some(profile) = profiles.first_mut()
|
||||
{
|
||||
profile["postgres"]["bootstrap"]["schema_autocreate"] = serde_json::Value::Bool(false);
|
||||
profile["postgres"]["bootstrap"]["schema_autoupdate"] = serde_json::Value::Bool(true);
|
||||
}
|
||||
let engine = fixture_engine_with_document(fixture.path(), &source);
|
||||
assert!(engine.is_ok());
|
||||
let engine = match engine {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return,
|
||||
};
|
||||
let environment = crate::ConfigEnvironment::from_maps(std::collections::BTreeMap::new(), std::collections::BTreeMap::new());
|
||||
let resolved = engine.load_resolved_store_config(std::option::Option::None, &environment);
|
||||
assert!(resolved.is_ok(), "Store Config V2 split schema policy mapping failed: {resolved:?}");
|
||||
if let std::result::Result::Ok(resolved) = resolved
|
||||
&& let ksp_store_lib::StoreBackendSettings::Postgres(postgres) = resolved.settings().backend()
|
||||
{
|
||||
assert!(!postgres.bootstrap().schema_autocreate());
|
||||
assert!(postgres.bootstrap().schema_autoupdate());
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
fn committed_engine() -> ksp_core_lib::Result<crate::ConfigDocumentEngine> {
|
||||
let workspace = workspace_root();
|
||||
let bootstrap = crate::ConfigBootstrapOptions::from_paths(workspace.join("config"), workspace.join("config/schemas"));
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: crates/ksp-store-lib/README.md -->
|
||||
<!-- version: 1 -->
|
||||
<!-- version: 4 -->
|
||||
|
||||
# ksp-store-lib
|
||||
|
||||
@@ -19,6 +19,8 @@ Elle expose aux consumers une surface backend-neutral, réexporte les contrats R
|
||||
- `Store::health().await` pour la readiness portable et bornée ;
|
||||
- `Store::close(self).await` pour la fermeture explicite bornée ;
|
||||
- le mapping des erreurs backend vers des codes Store stables sans exposer les erreurs physiques ;
|
||||
- les six capabilities `RawTransaction*` dispatchées vers le backend compilé ;
|
||||
- une validation réseau backend-neutral avant dispatch pour toutes les opérations qui portent explicitement un réseau ;
|
||||
- les réexports crate-root de `ksp-store-api` nécessaires aux consumers ordinaires.
|
||||
|
||||
## Une instance = un réseau
|
||||
@@ -29,7 +31,7 @@ Une instance `Store` représente exactement :
|
||||
1 Store = 1 RawNetworkId + 1 backend physique sélectionné
|
||||
```
|
||||
|
||||
Le runtime Store n'est pas un multiplexeur multi-database ou multi-réseau. La sélection d'un target nommé appartient à Config. Le document `std.store` peut donc définir plusieurs targets indépendants — par exemple Devnet, Mainnet et Testnet — mais un appel à `Store::open` reçoit les settings d'un seul target.
|
||||
Le runtime Store n'est pas un multiplexeur multi-database ou multi-réseau. La sélection d'un target nommé appartient à Config. Le document `std.store` peut donc définir plusieurs targets indépendants, mais un appel à `Store::open` reçoit les settings d'un seul target.
|
||||
|
||||
Cette séparation permet d'utiliser des bases PostgreSQL distinctes par réseau tout en conservant le réseau dans l'identité logique des données RAW.
|
||||
|
||||
@@ -61,35 +63,46 @@ Store ne lit ni `.env`, ni variables `KSP_*` / `KSPB_*`, ni variables/fichiers i
|
||||
|
||||
`ksp-config-lib` possède `std.store`, la résolution des secrets et la sélection du target. Il construit ensuite un `StoreSettings` backend-neutral. L'URI PostgreSQL reste nécessaire au runtime mais n'a aucun getter public dans `ksp-store-lib` et son `Debug` est redacted.
|
||||
|
||||
Les targets committed sont actuellement :
|
||||
## Surface RawTransaction
|
||||
|
||||
`Store` implémente les six capabilities transactionnelles acquises dans `ksp-store-api` :
|
||||
|
||||
```text
|
||||
devnet -> network devnet -> base indépendante
|
||||
mainnet -> network mainnet-beta -> base indépendante
|
||||
testnet -> network testnet -> base indépendante
|
||||
RawTransactionRead
|
||||
RawTransactionWrite
|
||||
RawTransactionObservationRead
|
||||
RawTransactionObservationWrite
|
||||
RawTransactionRetentionRead
|
||||
RawTransactionRetentionWrite
|
||||
```
|
||||
|
||||
Les credentials restent dans les variables `KSP_SECRET_STORE_*_POSTGRES_URI` ou le `.env` possédé par Config.
|
||||
Le consumer manipule uniquement les modèles et outcomes backend-neutral. Les erreurs physiques PostgreSQL sont projetées vers des codes Store stables sans exposer le backend.
|
||||
|
||||
## Surface actuelle et hors périmètre
|
||||
La façade fournit ainsi :
|
||||
|
||||
La fondation runtime ne fournit encore aucune implémentation PostgreSQL des capabilities métier RAW de `ksp-store-api`.
|
||||
- lecture d'une transaction canonique, de ses observations et de sa rétention ;
|
||||
- écriture atomique transaction + observation ;
|
||||
- ajout idempotent d'observations ;
|
||||
- pagination keyset déterministe par cursor opaque ;
|
||||
- application de transitions de rétention demandées par le caller ;
|
||||
- validation réseau avant dispatch lorsqu'un input porte explicitement son réseau.
|
||||
|
||||
## Hors périmètre
|
||||
|
||||
Sont volontairement hors de cette surface :
|
||||
|
||||
- persistence/query/rétention PostgreSQL de `RawTransaction` ;
|
||||
- persistence/query/rétention PostgreSQL de `RawAccountState` ;
|
||||
- batch-size, priorité, backlog ou policy de worker/job ;
|
||||
- transport d'acquisition, Program decoding et materialization ;
|
||||
- exposition publique de SQL, pool, client, row, statement ou transaction PostgreSQL.
|
||||
|
||||
Les premières vertical slices métier sont ajoutées séparément afin que la façade runtime reste stable et backend-neutral.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [`USAGE.md`](USAGE.md) — construction des settings, ouverture, health et fermeture ;
|
||||
- [`USAGE.md`](USAGE.md) — guide pratique de construction, lifecycle et capabilities Store ;
|
||||
- [`../ksp-store-postgres-lib/README.md`](../ksp-store-postgres-lib/README.md) — responsabilité du backend PostgreSQL physique ;
|
||||
- [`../../config/std.store.json`](../../config/std.store.json) — targets Store committed ;
|
||||
- [`../../docs/architecture/008-DATA_MATERIALIZATION_AND_STORE.md`](../../docs/architecture/008-DATA_MATERIALIZATION_AND_STORE.md) — architecture durable Store ;
|
||||
- [`../../docs/plans/023-V0_3_2_STORE_POSTGRES_FOUNDATION_PLAN.md`](../../docs/plans/023-V0_3_2_STORE_POSTGRES_FOUNDATION_PLAN.md) — plan de fondation ;
|
||||
- [`../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md`](../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md) — matrice de validation.
|
||||
- [`../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md`](../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md) — validation de fondation ;
|
||||
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — plan `RawTransaction` ;
|
||||
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction`.
|
||||
|
||||
@@ -1,36 +1,58 @@
|
||||
<!-- file: crates/ksp-store-lib/USAGE.md -->
|
||||
<!-- version: 1 -->
|
||||
<!-- version: 4 -->
|
||||
|
||||
# Utilisation de ksp-store-lib
|
||||
|
||||
## 1. Dépendance et features
|
||||
## 1. Dépendance et backend compilé
|
||||
|
||||
Le consumer runtime normal dépend uniquement de la façade :
|
||||
Le consumer runtime dépend de la façade commune :
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
ksp-store-lib = { path = "../ksp-store-lib" }
|
||||
```
|
||||
|
||||
La feature par défaut est :
|
||||
La feature par défaut compile le backend PostgreSQL :
|
||||
|
||||
```text
|
||||
postgres
|
||||
```
|
||||
|
||||
Pour construire un binaire sans backend physique :
|
||||
Pour compiler la façade sans backend physique :
|
||||
|
||||
```toml
|
||||
ksp-store-lib = { path = "../ksp-store-lib", default-features = false }
|
||||
```
|
||||
|
||||
Dans ce mode, le type PostgreSQL reste connu par la surface de settings mais `Store::open` retourne `ERROR_CODE_BACKEND_NOT_COMPILED` avant toute I/O si PostgreSQL est sélectionné.
|
||||
Dans ce mode, les settings PostgreSQL restent représentables, mais `Store::open` retourne `ERROR_CODE_BACKEND_NOT_COMPILED` avant toute I/O si PostgreSQL est sélectionné.
|
||||
|
||||
Un consumer ordinaire ne dépend pas directement de `ksp-store-postgres-lib`.
|
||||
Un consumer applicatif ordinaire ne dépend pas directement de `ksp-store-postgres-lib`.
|
||||
|
||||
## 2. Construire des settings PostgreSQL programmatiquement
|
||||
## 2. Obtenir les settings depuis Config
|
||||
|
||||
La construction directe est utile pour les tests, outils internes ou compositions qui n'utilisent pas `ksp-config-lib`.
|
||||
Le chemin applicatif recommandé passe par `ksp-config-lib`, propriétaire de `std.store`, de la résolution `.env` et des secrets.
|
||||
|
||||
```rust
|
||||
fn resolve_store_settings(
|
||||
engine: &ksp_config_lib::ConfigDocumentEngine,
|
||||
environment: &ksp_config_lib::ConfigEnvironment,
|
||||
target: std::option::Option<&str>,
|
||||
) -> ksp_core_lib::Result<ksp_store_lib::StoreSettings> {
|
||||
let resolved = engine.load_resolved_store_config(target, environment);
|
||||
let resolved = match resolved {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
|
||||
return std::result::Result::Ok(resolved.into_settings());
|
||||
}
|
||||
```
|
||||
|
||||
Chaque `StoreSettings` sélectionne exactement un réseau logique et un backend physique. La sélection d'un target nommé appartient à Config ; `Store` ne route pas automatiquement entre plusieurs targets.
|
||||
|
||||
## 3. Construire des settings programmatiquement
|
||||
|
||||
La construction directe est utile pour les tests et outils qui ne passent pas par Config.
|
||||
|
||||
```rust
|
||||
fn programmatic_store_settings(connection_uri: std::string::String) -> ksp_store_lib::Result<ksp_store_lib::StoreSettings> {
|
||||
@@ -52,8 +74,7 @@ fn programmatic_store_settings(connection_uri: std::string::String) -> ksp_store
|
||||
ksp_store_lib::StoreBackendSettings::Postgres(postgres),
|
||||
);
|
||||
|
||||
let validation = settings.validate();
|
||||
if let std::result::Result::Err(error) = validation {
|
||||
if let std::result::Result::Err(error) = settings.validate() {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
|
||||
@@ -61,11 +82,11 @@ fn programmatic_store_settings(connection_uri: std::string::String) -> ksp_store
|
||||
}
|
||||
```
|
||||
|
||||
`PostgresStoreSettings` ne fournit volontairement aucun getter public de l'URI. Son `Debug` remplace cette valeur par `<redacted>`.
|
||||
`PostgresStoreSettings` ne fournit aucun getter public de l'URI. Son `Debug` remplace cette valeur par `<redacted>`.
|
||||
|
||||
## 3. Ouvrir et fermer un Store
|
||||
## 4. Ouvrir, sonder et fermer un Store
|
||||
|
||||
`Store::open` est async et ne retourne un succès qu'après que le backend compilé a prouvé sa fondation runtime.
|
||||
`Store::open` est async et ne retourne un succès qu'après validation des settings, ouverture du backend compilé et bootstrap requis.
|
||||
|
||||
```rust
|
||||
async fn use_store(settings: ksp_store_lib::StoreSettings) -> ksp_store_lib::Result<()> {
|
||||
@@ -95,125 +116,178 @@ async fn use_store(settings: ksp_store_lib::StoreSettings) -> ksp_store_lib::Res
|
||||
}
|
||||
```
|
||||
|
||||
`Store::close(self)` consomme l'instance afin qu'une fermeture explicite ne puisse pas être suivie d'une nouvelle opération via la même valeur.
|
||||
`Store::close(self)` consomme l'instance. Une fermeture explicite ne peut donc pas être suivie d'une nouvelle opération via la même valeur.
|
||||
|
||||
## 4. Construire les settings depuis Config
|
||||
## 5. Lire une transaction RAW
|
||||
|
||||
Le chemin applicatif recommandé utilise `ksp-config-lib`, propriétaire du document `std.store`, de `.env` et des secrets.
|
||||
|
||||
Après construction du `ConfigDocumentEngine` :
|
||||
Importer le trait correspondant suffit pour utiliser la façade :
|
||||
|
||||
```rust
|
||||
fn resolve_store_settings(
|
||||
engine: &ksp_config_lib::ConfigDocumentEngine,
|
||||
environment: &ksp_config_lib::ConfigEnvironment,
|
||||
target: std::option::Option<&str>,
|
||||
) -> ksp_core_lib::Result<ksp_store_lib::StoreSettings> {
|
||||
let resolved = engine.load_resolved_store_config(target, environment);
|
||||
let resolved = match resolved {
|
||||
use ksp_store_lib::RawTransactionRead;
|
||||
|
||||
async fn read_transaction(
|
||||
store: &ksp_store_lib::Store,
|
||||
reference: &ksp_store_lib::RawTransactionReference,
|
||||
) -> ksp_store_lib::Result<std::option::Option<ksp_store_lib::RawTransaction>> {
|
||||
return store.get_raw_transaction(reference).await;
|
||||
}
|
||||
```
|
||||
|
||||
Une transaction absente retourne `None`. Une transaction `Purged` retourne également `None` pour le payload canonique ; son tombstone reste accessible via la capability de rétention.
|
||||
|
||||
## 6. Paginer les références de transactions
|
||||
|
||||
La pagination est keyset et utilise un cursor opaque. Le consumer ne doit pas interpréter ses bytes.
|
||||
|
||||
```rust
|
||||
use ksp_store_lib::RawTransactionRead;
|
||||
|
||||
async fn first_transaction_page(
|
||||
store: &ksp_store_lib::Store,
|
||||
network: ksp_store_lib::RawNetworkId,
|
||||
) -> ksp_store_lib::Result<ksp_store_lib::RawPage<ksp_store_lib::RawTransactionReference>> {
|
||||
let limit = ksp_store_lib::RawPageLimit::new(100);
|
||||
let limit = match limit {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
|
||||
return std::result::Result::Ok(resolved.into_settings());
|
||||
let slots = ksp_store_lib::RawSlotRange::new(std::option::Option::None, std::option::Option::None);
|
||||
let slots = match slots {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
|
||||
let query = ksp_store_lib::RawTransactionQuery::new(
|
||||
network,
|
||||
slots,
|
||||
ksp_store_lib::RawSortDirection::Ascending,
|
||||
ksp_store_lib::RawPageRequest::first(limit),
|
||||
);
|
||||
|
||||
return store.list_raw_transactions(&query).await;
|
||||
}
|
||||
```
|
||||
|
||||
Targets committed :
|
||||
Pour continuer, recopier le cursor retourné par `RawPage::next_cursor()` dans `RawPageRequest::after`. Le réseau, la direction et les bornes de slots doivent rester identiques à ceux de la query ayant produit le cursor.
|
||||
|
||||
```text
|
||||
devnet -> RawNetworkId("devnet")
|
||||
mainnet -> RawNetworkId("mainnet-beta")
|
||||
testnet -> RawNetworkId("testnet")
|
||||
## 7. Persister une acquisition canonique
|
||||
|
||||
La transaction canonique et son observation initiale forment une seule opération atomique.
|
||||
|
||||
```rust
|
||||
use ksp_store_lib::RawTransactionWrite;
|
||||
|
||||
async fn persist_acquisition(
|
||||
store: &ksp_store_lib::Store,
|
||||
transaction: ksp_store_lib::RawTransaction,
|
||||
observation: ksp_store_lib::RawTransactionObservation,
|
||||
) -> ksp_store_lib::Result<ksp_store_lib::RawAcquisitionWriteOutcome> {
|
||||
return store
|
||||
.persist_raw_transaction_acquisition(
|
||||
transaction,
|
||||
observation,
|
||||
ksp_store_lib::RawTransactionAcquisitionMode::Normal,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
```
|
||||
|
||||
Chaque target peut utiliser une URI PostgreSQL distincte. `default_profile` sélectionne un seul target ; `Store` ne route pas automatiquement entre plusieurs targets.
|
||||
Le mode `Normal` respecte un tombstone `Purged`. `ForceRehydrate` doit être choisi explicitement lorsqu'un caller veut restaurer un payload purgé et que l'identité retenue est compatible.
|
||||
|
||||
## 5. Settings disponibles
|
||||
Un contenu divergent sous la même identité produit `ERROR_CODE_RAW_CONFLICT`; Store ne remplace jamais silencieusement le contenu gagnant.
|
||||
|
||||
### `PostgresPoolSettings`
|
||||
## 8. Lire et ajouter une observation
|
||||
|
||||
Valeurs par défaut :
|
||||
Une observation supplémentaire référence une transaction canonique déjà durable.
|
||||
|
||||
```text
|
||||
max_connections 8
|
||||
connect_timeout 10 s
|
||||
wait_timeout 5 s
|
||||
create_timeout 10 s
|
||||
recycle_timeout 5 s
|
||||
```rust
|
||||
use ksp_store_lib::RawTransactionObservationRead;
|
||||
use ksp_store_lib::RawTransactionObservationWrite;
|
||||
|
||||
async fn use_observation(
|
||||
store: &ksp_store_lib::Store,
|
||||
key: &ksp_store_lib::RawObservationKey,
|
||||
observation: ksp_store_lib::RawTransactionObservation,
|
||||
) -> ksp_store_lib::Result<ksp_store_lib::RawObservationWriteOutcome> {
|
||||
let existing = store.get_raw_transaction_observation(key).await;
|
||||
if let std::result::Result::Err(error) = existing {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
|
||||
return store.record_raw_transaction_observation(observation).await;
|
||||
}
|
||||
```
|
||||
|
||||
Les getters sont :
|
||||
`record_raw_transaction_observation` ne crée pas implicitement le canonique. Une référence absente est signalée par `ERROR_CODE_RAW_REFERENCE_NOT_FOUND`.
|
||||
|
||||
```text
|
||||
max_connections()
|
||||
connect_timeout()
|
||||
wait_timeout()
|
||||
create_timeout()
|
||||
recycle_timeout()
|
||||
## 9. Lire la rétention et le tombstone
|
||||
|
||||
```rust
|
||||
use ksp_store_lib::RawTransactionRetentionRead;
|
||||
|
||||
async fn read_retention(
|
||||
store: &ksp_store_lib::Store,
|
||||
reference: &ksp_store_lib::RawTransactionReference,
|
||||
) -> ksp_store_lib::Result<std::option::Option<ksp_store_lib::RawRetentionState>> {
|
||||
let tombstone = store.get_raw_transaction_tombstone(reference).await;
|
||||
if let std::result::Result::Err(error) = tombstone {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
|
||||
return store.get_raw_transaction_retention_state(reference).await;
|
||||
}
|
||||
```
|
||||
|
||||
`validate()` vérifie les bornes sans I/O.
|
||||
Le tombstone minimal est utile uniquement lorsque le payload a été purgé ; il ne remplace pas le modèle canonique lorsqu'un payload est encore disponible.
|
||||
|
||||
### `PostgresBootstrapSettings`
|
||||
## 10. Appliquer une transition de rétention
|
||||
|
||||
Valeurs par défaut :
|
||||
La policy qui décide qu'une transition est autorisée appartient au caller. Store applique uniquement la transition demandée de manière atomique.
|
||||
|
||||
```text
|
||||
auto_migrate true
|
||||
migration_timeout 30 s
|
||||
migration_lock_timeout 10 s
|
||||
```rust
|
||||
use ksp_store_lib::RawTransactionRetentionWrite;
|
||||
|
||||
async fn archive_transaction(
|
||||
store: &ksp_store_lib::Store,
|
||||
reference: ksp_store_lib::RawTransactionReference,
|
||||
) -> ksp_store_lib::Result<ksp_store_lib::RawRetentionWriteOutcome> {
|
||||
let transition = ksp_store_lib::RawTransactionRetentionTransition::try_new(
|
||||
reference,
|
||||
ksp_store_lib::RawRetentionState::Full,
|
||||
ksp_store_lib::RawRetentionState::Archived,
|
||||
);
|
||||
let transition = match transition {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
|
||||
return store.transition_raw_transaction_retention(transition).await;
|
||||
}
|
||||
```
|
||||
|
||||
Getters :
|
||||
Le backend PostgreSQL supporte physiquement `Full -> Archived -> Purged`. Une transition impliquant `Compacted` est rejetée par ce backend tant qu'aucune représentation compactée réelle n'est implémentée.
|
||||
|
||||
## 11. Diagnostics et erreurs
|
||||
|
||||
Les snapshots et erreurs de façade n'exposent ni URI, host, user, database, SQL, handle backend, valeur de bind ni texte d'erreur PostgreSQL.
|
||||
|
||||
Les codes Store utiles incluent notamment :
|
||||
|
||||
```text
|
||||
auto_migrate()
|
||||
migration_timeout()
|
||||
migration_lock_timeout()
|
||||
store.wrong_network
|
||||
store.raw_reference_not_found
|
||||
store.postgres_read_failed
|
||||
store.postgres_write_failed
|
||||
store.postgres_data_invalid
|
||||
store.postgres_page_limit_unsupported
|
||||
store.postgres_retention_compaction_unsupported
|
||||
```
|
||||
|
||||
### `StoreSettings`
|
||||
Les conflits et queries invalides utilisent les codes backend-neutral `store_api.raw_conflict` et `store_api.raw_query_invalid`.
|
||||
|
||||
La surface expose :
|
||||
## 12. Limites de la façade
|
||||
|
||||
```text
|
||||
backend()
|
||||
backend_kind()
|
||||
network()
|
||||
shutdown_timeout()
|
||||
validate()
|
||||
```
|
||||
La façade ne fournit pas d'accès public au SQL, au pool, aux clients ou transactions PostgreSQL. Les capabilities `RawAccount*` réexportées par l'API commune ne sont pas encore dispatchées par `Store`.
|
||||
|
||||
`StoreSettings::new` permet de choisir explicitement le timeout de shutdown. `StoreSettings::with_default_shutdown` utilise la borne commune par défaut de 5 secondes.
|
||||
|
||||
## 6. Health et diagnostics
|
||||
|
||||
`StoreRuntimeSnapshot` est synchrone et ne déclenche aucune I/O. Il expose uniquement :
|
||||
|
||||
```text
|
||||
backend_kind
|
||||
network
|
||||
pool_capacity
|
||||
pool_size
|
||||
pool_available
|
||||
pool_waiting
|
||||
```
|
||||
|
||||
`StoreHealthSnapshot` ajoute une probe async bornée :
|
||||
|
||||
```text
|
||||
state = Ready | NotReady
|
||||
migration_version
|
||||
pending_migration_count
|
||||
last_error_code
|
||||
runtime snapshot
|
||||
```
|
||||
|
||||
Aucun snapshot n'expose URI, host, user, database, SQL, handle backend ou texte d'erreur PostgreSQL.
|
||||
|
||||
## 7. Limite fonctionnelle actuelle
|
||||
|
||||
`ksp-store-lib` réexporte les modèles et traits RAW de `ksp-store-api`, mais le backend PostgreSQL de la fondation n'implémente encore aucune capability `RawTransaction*` ou `RawAccount*`.
|
||||
|
||||
Les consumers ne doivent donc pas interpréter la disponibilité du runtime PostgreSQL comme une persistence métier déjà présente.
|
||||
La taille de page est une primitive de navigation. Les décisions de batch, priorité, backlog et scheduling appartiennent aux workers/jobs, pas à Store.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/src/error.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
/// Error code reserved for operations attempted after a Store backend has entered its closed state.
|
||||
pub const ERROR_CODE_BACKEND_CLOSED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "backend_closed");
|
||||
@@ -11,19 +11,34 @@ pub const ERROR_CODE_BACKEND_OPEN_FAILED: ksp_store_api::ErrorCode = ksp_store_a
|
||||
pub const ERROR_CODE_POSTGRES_CONFIG_INVALID: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_config_invalid");
|
||||
/// Error code used when PostgreSQL physical connection establishment fails without exposing remote or credential details.
|
||||
pub const ERROR_CODE_POSTGRES_CONNECT_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_connect_failed");
|
||||
/// Error code used when PostgreSQL returns persisted RAW data incompatible with the stable Store contract.
|
||||
pub const ERROR_CODE_POSTGRES_DATA_INVALID: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_data_invalid");
|
||||
/// Error code used when a lightweight PostgreSQL health/readiness probe fails safely.
|
||||
pub const ERROR_CODE_POSTGRES_HEALTH_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_health_failed");
|
||||
/// Error code used when PostgreSQL migration/bootstrap execution fails without exposing server text or SQL.
|
||||
pub const ERROR_CODE_POSTGRES_MIGRATION_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_migration_failed");
|
||||
/// Error code used when persisted PostgreSQL migration history diverges from the embedded immutable KSP history.
|
||||
pub const ERROR_CODE_POSTGRES_MIGRATION_MISMATCH: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_migration_mismatch");
|
||||
/// Error code used when the requested RAW page size exceeds the exact PostgreSQL LIMIT representation boundary.
|
||||
pub const ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_page_limit_unsupported");
|
||||
/// Error code used when a bounded PostgreSQL pool wait, create or recycle operation reaches its deadline.
|
||||
pub const ERROR_CODE_POSTGRES_POOL_TIMEOUT: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_pool_timeout");
|
||||
/// Error code used when a PostgreSQL RAW read fails without exposing SQL, bind values or server text.
|
||||
pub const ERROR_CODE_POSTGRES_READ_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_read_failed");
|
||||
/// Error code used when PostgreSQL cannot represent a requested RAW retention compaction state.
|
||||
pub const ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED: ksp_store_api::ErrorCode =
|
||||
ksp_store_api::ErrorCode::new("store", "postgres_retention_compaction_unsupported");
|
||||
/// Error code used when PostgreSQL history contains a migration newer than this Store runtime understands.
|
||||
pub const ERROR_CODE_POSTGRES_SCHEMA_NEWER: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_schema_newer");
|
||||
/// Error code used when verified PostgreSQL TLS setup or negotiation cannot be completed safely.
|
||||
pub const ERROR_CODE_POSTGRES_TLS_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_tls_failed");
|
||||
/// Error code used when a PostgreSQL RAW write fails without exposing SQL, bind values or server text.
|
||||
pub const ERROR_CODE_POSTGRES_WRITE_FAILED: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "postgres_write_failed");
|
||||
/// Error code used when a RAW write requires a canonical reference that is not durable.
|
||||
pub const ERROR_CODE_RAW_REFERENCE_NOT_FOUND: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "raw_reference_not_found");
|
||||
/// Error code used when backend-neutral Store settings violate runtime bounds or invariants.
|
||||
pub const ERROR_CODE_SETTINGS_INVALID: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "settings_invalid");
|
||||
/// Error code used when a Store cannot complete its explicit shutdown inside the configured bound.
|
||||
pub const ERROR_CODE_SHUTDOWN_TIMEOUT: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "shutdown_timeout");
|
||||
/// Error code used when a network-scoped Store operation targets a network different from the opened Store binding.
|
||||
pub const ERROR_CODE_WRONG_NETWORK: ksp_store_api::ErrorCode = ksp_store_api::ErrorCode::new("store", "wrong_network");
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/src/lib.rs
|
||||
// version: 6
|
||||
// version: 8
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -7,10 +7,10 @@
|
||||
|
||||
//! Common backend-neutral Store runtime facade for KSP.
|
||||
//!
|
||||
//! `0.3.2-pre.007` closes the physical PostgreSQL runtime composition with a
|
||||
//! portable safe runtime snapshot and lightweight health/readiness projection,
|
||||
//! while retaining the private migration/bootstrap foundation and no business
|
||||
//! persistence schema.
|
||||
//! The runtime facade owns backend selection, lifecycle, safe diagnostics and
|
||||
//! backend-neutral capability dispatch. `0.3.3-pre.008` completes the PostgreSQL
|
||||
//! `RawTransaction` vertical slice by implementing the six transaction capabilities
|
||||
//! on both the physical backend and this common facade without exposing physical types.
|
||||
//!
|
||||
//! The default `postgres` feature compiles the official PostgreSQL backend as
|
||||
//! an optional implementation dependency. No backend implementation type is
|
||||
@@ -32,22 +32,36 @@ pub use self::error::ERROR_CODE_BACKEND_OPEN_FAILED;
|
||||
pub use self::error::ERROR_CODE_POSTGRES_CONFIG_INVALID;
|
||||
/// Error code used when PostgreSQL physical connection establishment fails.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_CONNECT_FAILED;
|
||||
/// Error code used when PostgreSQL returns persisted RAW data incompatible with the Store contract.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_DATA_INVALID;
|
||||
/// Error code used when a lightweight PostgreSQL health/readiness probe fails safely.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_HEALTH_FAILED;
|
||||
/// Error code used when PostgreSQL migration/bootstrap execution fails safely.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_MIGRATION_FAILED;
|
||||
/// Error code used when PostgreSQL migration history diverges from the embedded immutable KSP history.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_MIGRATION_MISMATCH;
|
||||
/// Error code used when a requested RAW page size exceeds PostgreSQL's exact physical LIMIT boundary.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED;
|
||||
/// Error code used when a bounded PostgreSQL pool operation reaches its deadline.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_POOL_TIMEOUT;
|
||||
/// Error code used when a PostgreSQL RAW read statement fails safely.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_READ_FAILED;
|
||||
/// Error code used when PostgreSQL cannot represent a requested RAW retention compaction state.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED;
|
||||
/// Error code used when PostgreSQL schema history is newer than this Store runtime.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_SCHEMA_NEWER;
|
||||
/// Error code used when PostgreSQL verified TLS setup or negotiation fails.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_TLS_FAILED;
|
||||
/// Error code used when a PostgreSQL RAW write statement or transaction fails safely.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_WRITE_FAILED;
|
||||
/// Error code used when a RAW write requires a canonical reference that is not durable.
|
||||
pub use self::error::ERROR_CODE_RAW_REFERENCE_NOT_FOUND;
|
||||
/// Error code used when Store settings violate backend-neutral bounds or invariants.
|
||||
pub use self::error::ERROR_CODE_SETTINGS_INVALID;
|
||||
/// Error code used when explicit Store shutdown exceeds its configured deadline.
|
||||
pub use self::error::ERROR_CODE_SHUTDOWN_TIMEOUT;
|
||||
/// Error code used when a network-scoped operation targets a network different from the Store binding.
|
||||
pub use self::error::ERROR_CODE_WRONG_NETWORK;
|
||||
/// Portable Store health/readiness projection containing only safe diagnostics.
|
||||
pub use self::health::StoreHealthSnapshot;
|
||||
/// Portable Store health state independent from physical backend types.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/src/settings.rs
|
||||
// version: 3
|
||||
// version: 4
|
||||
|
||||
const DEFAULT_CONNECT_TIMEOUT_MS: u64 = 10_000;
|
||||
const DEFAULT_MAX_CONNECTIONS: u32 = 8;
|
||||
@@ -153,22 +153,49 @@ impl std::default::Default for PostgresPoolSettings {
|
||||
/// Bounded PostgreSQL bootstrap settings owned by the Store facade.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresBootstrapSettings {
|
||||
auto_migrate: bool,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
migration_timeout: std::time::Duration,
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
}
|
||||
|
||||
impl PostgresBootstrapSettings {
|
||||
/// Creates explicit bootstrap behavior and migration deadlines.
|
||||
/// Creates bootstrap settings using the legacy single migration switch for source compatibility.
|
||||
///
|
||||
/// The supplied value is mapped to both schema auto-creation and schema auto-update. New code should prefer
|
||||
/// [`Self::with_schema_policy`] when these policies need to differ.
|
||||
#[must_use]
|
||||
pub const fn new(auto_migrate: bool, migration_timeout: std::time::Duration, migration_lock_timeout: std::time::Duration) -> Self {
|
||||
return Self { auto_migrate, migration_lock_timeout, migration_timeout };
|
||||
return Self::with_schema_policy(auto_migrate, auto_migrate, migration_timeout, migration_lock_timeout);
|
||||
}
|
||||
|
||||
/// Returns whether pending KSP-owned migrations may be applied during Store opening.
|
||||
/// Creates explicit schema creation/update policy and migration deadlines.
|
||||
#[must_use]
|
||||
pub const fn with_schema_policy(
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
migration_timeout: std::time::Duration,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
) -> Self {
|
||||
return Self { migration_lock_timeout, migration_timeout, schema_autocreate, schema_autoupdate };
|
||||
}
|
||||
|
||||
/// Returns the legacy pending-migration switch, mapped to the schema auto-update policy.
|
||||
#[must_use]
|
||||
pub const fn auto_migrate(&self) -> bool {
|
||||
return self.auto_migrate;
|
||||
return self.schema_autoupdate;
|
||||
}
|
||||
|
||||
/// Returns whether an absent KSP-managed schema may be created or adopted during Store opening.
|
||||
#[must_use]
|
||||
pub const fn schema_autocreate(&self) -> bool {
|
||||
return self.schema_autocreate;
|
||||
}
|
||||
|
||||
/// Returns whether pending migrations and safe additive schema repairs may be applied during Store opening.
|
||||
#[must_use]
|
||||
pub const fn schema_autoupdate(&self) -> bool {
|
||||
return self.schema_autoupdate;
|
||||
}
|
||||
|
||||
/// Returns the bounded wait allowed for the private PostgreSQL migration lock.
|
||||
@@ -205,7 +232,8 @@ impl PostgresBootstrapSettings {
|
||||
|
||||
impl std::default::Default for PostgresBootstrapSettings {
|
||||
fn default() -> Self {
|
||||
return Self::new(
|
||||
return Self::with_schema_policy(
|
||||
true,
|
||||
true,
|
||||
std::time::Duration::from_millis(DEFAULT_MIGRATION_TIMEOUT_MS),
|
||||
std::time::Duration::from_millis(DEFAULT_MIGRATION_LOCK_TIMEOUT_MS),
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/src/store.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
/// Opaque common Store runtime facade.
|
||||
///
|
||||
@@ -116,6 +116,261 @@ impl std::fmt::Debug for Store {
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRead for Store {
|
||||
fn get_raw_transaction<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransaction>>> {
|
||||
let network_check = validate_operation_network(&self.network, reference.network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.get_raw_transaction(reference).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = reference;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
fn list_raw_transactions<'a>(
|
||||
&'a self,
|
||||
query: &'a ksp_store_api::RawTransactionQuery,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawPage<ksp_store_api::RawTransactionReference>>> {
|
||||
let network_check = validate_operation_network(&self.network, query.network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.list_raw_transactions(query).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = query;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionWrite for Store {
|
||||
fn persist_raw_transaction_acquisition<'a>(
|
||||
&'a self,
|
||||
transaction: ksp_store_api::RawTransaction,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
mode: ksp_store_api::RawTransactionAcquisitionMode,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawAcquisitionWriteOutcome>> {
|
||||
let transaction_network = validate_operation_network(&self.network, transaction.reference().network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = transaction_network {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
let observation_network = validate_operation_network(&self.network, observation.transaction().network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = observation_network {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.persist_raw_transaction_acquisition(transaction, observation, mode).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = transaction;
|
||||
let _ = observation;
|
||||
let _ = mode;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionObservationRead for Store {
|
||||
fn get_raw_transaction_observation<'a>(
|
||||
&'a self,
|
||||
observation_key: &'a ksp_store_api::RawObservationKey,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransactionObservation>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.get_raw_transaction_observation(observation_key).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = observation_key;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionObservationWrite for Store {
|
||||
fn record_raw_transaction_observation<'a>(
|
||||
&'a self,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawObservationWriteOutcome>> {
|
||||
let network_check = validate_operation_network(&self.network, observation.transaction().network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.record_raw_transaction_observation(observation).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = observation;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRetentionRead for Store {
|
||||
fn get_raw_transaction_retention_state<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawRetentionState>>> {
|
||||
let network_check = validate_operation_network(&self.network, reference.network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.get_raw_transaction_retention_state(reference).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = reference;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
fn get_raw_transaction_tombstone<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>>> {
|
||||
let network_check = validate_operation_network(&self.network, reference.network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.get_raw_transaction_tombstone(reference).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = reference;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRetentionWrite for Store {
|
||||
fn transition_raw_transaction_retention<'a>(
|
||||
&'a self,
|
||||
transition: ksp_store_api::RawTransactionRetentionTransition,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawRetentionWriteOutcome>> {
|
||||
let network_check = validate_operation_network(&self.network, transition.reference().network(), self.backend_kind);
|
||||
if let std::result::Result::Err(error) = network_check {
|
||||
return std::boxed::Box::pin(async move {
|
||||
return std::result::Result::Err(error);
|
||||
});
|
||||
}
|
||||
return std::boxed::Box::pin(async move {
|
||||
#[cfg(feature = "postgres")]
|
||||
{
|
||||
return match &self.runtime {
|
||||
StoreRuntime::Postgres(backend) => {
|
||||
let result = backend.transition_raw_transaction_retention(transition).await;
|
||||
result.map_err(|error| return map_postgres_error(error, self.backend_kind, self.network.as_str()))
|
||||
},
|
||||
};
|
||||
}
|
||||
#[cfg(not(feature = "postgres"))]
|
||||
{
|
||||
let _ = transition;
|
||||
return std::result::Result::Err(unavailable_runtime_error(self.backend_kind));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_operation_network(
|
||||
store_network: &ksp_store_api::RawNetworkId,
|
||||
operation_network: &ksp_store_api::RawNetworkId,
|
||||
backend_kind: crate::StoreBackendKind,
|
||||
) -> ksp_store_api::Result<()> {
|
||||
if store_network != operation_network {
|
||||
return std::result::Result::Err(
|
||||
ksp_store_api::Error::new(crate::ERROR_CODE_WRONG_NETWORK, "Store operation targeted a different logical network")
|
||||
.with_context("backend", backend_kind.code())
|
||||
.with_context("network", store_network.as_str()),
|
||||
);
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
#[cfg(feature = "postgres")]
|
||||
enum StoreRuntime {
|
||||
Postgres(ksp_store_postgres_lib::PostgresBackend),
|
||||
@@ -134,7 +389,7 @@ async fn open_postgres(
|
||||
crate::PostgresTlsMode::Disabled => ksp_store_postgres_lib::PostgresBackendTlsMode::Disabled,
|
||||
crate::PostgresTlsMode::VerifyFull => ksp_store_postgres_lib::PostgresBackendTlsMode::VerifyFull,
|
||||
};
|
||||
let backend_settings = ksp_store_postgres_lib::PostgresBackendSettings::new(
|
||||
let backend_settings = ksp_store_postgres_lib::PostgresBackendSettings::with_schema_policy(
|
||||
network.clone(),
|
||||
settings.connection_uri(),
|
||||
pool.max_connections(),
|
||||
@@ -143,7 +398,8 @@ async fn open_postgres(
|
||||
pool.create_timeout(),
|
||||
pool.recycle_timeout(),
|
||||
tls_mode,
|
||||
bootstrap.auto_migrate(),
|
||||
bootstrap.schema_autocreate(),
|
||||
bootstrap.schema_autoupdate(),
|
||||
bootstrap.migration_timeout(),
|
||||
bootstrap.migration_lock_timeout(),
|
||||
);
|
||||
@@ -175,7 +431,7 @@ async fn open_postgres(
|
||||
#[cfg(feature = "postgres")]
|
||||
fn map_postgres_error(error: ksp_store_postgres_lib::PostgresBackendError, backend_kind: crate::StoreBackendKind, network: &str) -> ksp_store_api::Error {
|
||||
let code = postgres_error_code(error.kind());
|
||||
return ksp_store_api::Error::new(code, "PostgreSQL Store backend lifecycle operation failed")
|
||||
return ksp_store_api::Error::new(code, "PostgreSQL Store backend operation failed")
|
||||
.with_context("backend", backend_kind.code())
|
||||
.with_context("network", network)
|
||||
.with_context("phase", error.phase());
|
||||
@@ -214,13 +470,22 @@ fn postgres_error_code(kind: ksp_store_postgres_lib::PostgresBackendErrorKind) -
|
||||
return match kind {
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid => crate::ERROR_CODE_POSTGRES_CONFIG_INVALID,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed => crate::ERROR_CODE_POSTGRES_CONNECT_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict => ksp_store_api::ERROR_CODE_RAW_CONFLICT,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid => crate::ERROR_CODE_POSTGRES_DATA_INVALID,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed => crate::ERROR_CODE_POSTGRES_HEALTH_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout => crate::ERROR_CODE_POSTGRES_POOL_TIMEOUT,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed => crate::ERROR_CODE_POSTGRES_MIGRATION_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch => crate::ERROR_CODE_POSTGRES_MIGRATION_MISMATCH,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported => crate::ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid => ksp_store_api::ERROR_CODE_RAW_QUERY_INVALID,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed => crate::ERROR_CODE_POSTGRES_READ_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound => crate::ERROR_CODE_RAW_REFERENCE_NOT_FOUND,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::RetentionCompactionUnsupported => crate::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer => crate::ERROR_CODE_POSTGRES_SCHEMA_NEWER,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout => crate::ERROR_CODE_SHUTDOWN_TIMEOUT,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed => crate::ERROR_CODE_POSTGRES_TLS_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WriteFailed => crate::ERROR_CODE_POSTGRES_WRITE_FAILED,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork => crate::ERROR_CODE_WRONG_NETWORK,
|
||||
_ => crate::ERROR_CODE_BACKEND_OPEN_FAILED,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/tests/dependency_boundary.rs
|
||||
// version: 6
|
||||
// version: 7
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -58,3 +58,36 @@ fn pre_005_facade_exposes_no_physical_postgres_types_or_environment_bypass() {
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_facade_dispatches_six_raw_transaction_capabilities_without_physical_leak() {
|
||||
let store = include_str!("../src/store.rs");
|
||||
for required in [
|
||||
"impl ksp_store_api::RawTransactionRead for Store",
|
||||
"impl ksp_store_api::RawTransactionWrite for Store",
|
||||
"impl ksp_store_api::RawTransactionObservationRead for Store",
|
||||
"impl ksp_store_api::RawTransactionObservationWrite for Store",
|
||||
"impl ksp_store_api::RawTransactionRetentionRead for Store",
|
||||
"impl ksp_store_api::RawTransactionRetentionWrite for Store",
|
||||
"validate_operation_network",
|
||||
"StoreRuntime::Postgres(backend)",
|
||||
"map_postgres_error",
|
||||
] {
|
||||
assert!(store.contains(required), "missing pre.008 Store capability dispatch contract: {required}");
|
||||
}
|
||||
for forbidden in [
|
||||
"impl ksp_store_api::RawAccountStateRead for Store",
|
||||
"impl ksp_store_api::RawAccountStateWrite for Store",
|
||||
"impl ksp_store_api::RawAccountObservationRead for Store",
|
||||
"impl ksp_store_api::RawAccountObservationWrite for Store",
|
||||
"tokio_postgres::",
|
||||
"deadpool_postgres::",
|
||||
"CREATE TABLE",
|
||||
"INSERT INTO",
|
||||
"UPDATE ksp_",
|
||||
"DELETE FROM",
|
||||
] {
|
||||
assert!(!store.contains(forbidden), "pre.008 facade leaked physical or RawAccount scope: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/tests/hardening_completeness.rs
|
||||
// version: 1
|
||||
// version: 4
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -98,20 +98,27 @@ fn pre_009_facade_modules_and_crate_root_exports_are_exact() {
|
||||
"ERROR_CODE_BACKEND_OPEN_FAILED",
|
||||
"ERROR_CODE_POSTGRES_CONFIG_INVALID",
|
||||
"ERROR_CODE_POSTGRES_CONNECT_FAILED",
|
||||
"ERROR_CODE_POSTGRES_DATA_INVALID",
|
||||
"ERROR_CODE_POSTGRES_HEALTH_FAILED",
|
||||
"ERROR_CODE_POSTGRES_MIGRATION_FAILED",
|
||||
"ERROR_CODE_POSTGRES_MIGRATION_MISMATCH",
|
||||
"ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED",
|
||||
"ERROR_CODE_POSTGRES_POOL_TIMEOUT",
|
||||
"ERROR_CODE_POSTGRES_READ_FAILED",
|
||||
"ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED",
|
||||
"ERROR_CODE_POSTGRES_SCHEMA_NEWER",
|
||||
"ERROR_CODE_POSTGRES_TLS_FAILED",
|
||||
"ERROR_CODE_POSTGRES_WRITE_FAILED",
|
||||
"ERROR_CODE_RAW_CONFLICT",
|
||||
"ERROR_CODE_RAW_MODEL_INVALID",
|
||||
"ERROR_CODE_RAW_PAYLOAD_INVALID",
|
||||
"ERROR_CODE_RAW_PROVENANCE_INVALID",
|
||||
"ERROR_CODE_RAW_QUERY_INVALID",
|
||||
"ERROR_CODE_RAW_REFERENCE_NOT_FOUND",
|
||||
"ERROR_CODE_RAW_RETENTION_INVALID",
|
||||
"ERROR_CODE_SETTINGS_INVALID",
|
||||
"ERROR_CODE_SHUTDOWN_TIMEOUT",
|
||||
"ERROR_CODE_WRONG_NETWORK",
|
||||
"Error",
|
||||
"ErrorCode",
|
||||
"ErrorContext",
|
||||
@@ -180,7 +187,7 @@ fn pre_009_facade_modules_and_crate_root_exports_are_exact() {
|
||||
];
|
||||
expected.sort_unstable();
|
||||
assert_eq!(actual.as_slice(), expected.as_slice());
|
||||
assert_eq!(actual.len(), 84);
|
||||
assert_eq!(actual.len(), 91);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -254,3 +261,29 @@ fn pre_009_facade_production_sources_keep_config_env_physical_sql_and_backend_ha
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_010_facade_raw_transaction_capability_inventory_is_exact_and_raw_account_scope_stays_closed() {
|
||||
let store = include_str!("../src/store.rs");
|
||||
let capability_impls = [
|
||||
"impl ksp_store_api::RawTransactionRead for Store",
|
||||
"impl ksp_store_api::RawTransactionWrite for Store",
|
||||
"impl ksp_store_api::RawTransactionObservationRead for Store",
|
||||
"impl ksp_store_api::RawTransactionObservationWrite for Store",
|
||||
"impl ksp_store_api::RawTransactionRetentionRead for Store",
|
||||
"impl ksp_store_api::RawTransactionRetentionWrite for Store",
|
||||
];
|
||||
for implementation in capability_impls {
|
||||
assert_eq!(store.matches(implementation).count(), 1, "unexpected Store capability implementation inventory: {implementation}");
|
||||
}
|
||||
for forbidden in [
|
||||
"impl ksp_store_api::RawAccountStateRead for Store",
|
||||
"impl ksp_store_api::RawAccountStateWrite for Store",
|
||||
"impl ksp_store_api::RawAccountObservationRead for Store",
|
||||
"impl ksp_store_api::RawAccountObservationWrite for Store",
|
||||
] {
|
||||
assert!(!store.contains(forbidden), "RawAccountState scope opened in Store during RawTransaction hardening: {forbidden}");
|
||||
}
|
||||
assert_eq!(store.matches("validate_operation_network(").count(), 9);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/tests/public_api.rs
|
||||
// version: 5
|
||||
// version: 7
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -38,14 +38,21 @@ fn pre_005_common_and_postgres_error_codes_are_stable_and_store_owned() {
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_BACKEND_OPEN_FAILED.code(), "backend_open_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_CONFIG_INVALID.code(), "postgres_config_invalid");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_CONNECT_FAILED.code(), "postgres_connect_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_DATA_INVALID.code(), "postgres_data_invalid");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_HEALTH_FAILED.code(), "postgres_health_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_POOL_TIMEOUT.code(), "postgres_pool_timeout");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_READ_FAILED.code(), "postgres_read_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED.code(), "postgres_retention_compaction_unsupported");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_MIGRATION_FAILED.code(), "postgres_migration_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_MIGRATION_MISMATCH.code(), "postgres_migration_mismatch");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED.code(), "postgres_page_limit_unsupported");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_SCHEMA_NEWER.code(), "postgres_schema_newer");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_TLS_FAILED.code(), "postgres_tls_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_POSTGRES_WRITE_FAILED.code(), "postgres_write_failed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_RAW_REFERENCE_NOT_FOUND.code(), "raw_reference_not_found");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_BACKEND_CLOSED.code(), "backend_closed");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_SHUTDOWN_TIMEOUT.code(), "shutdown_timeout");
|
||||
assert_eq!(ksp_store_lib::ERROR_CODE_WRONG_NETWORK.code(), "wrong_network");
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -66,3 +73,22 @@ fn pre_007_health_and_runtime_snapshot_types_are_portable_crate_root_contracts()
|
||||
let _runtime = std::mem::size_of::<std::option::Option<ksp_store_lib::StoreRuntimeSnapshot>>();
|
||||
return;
|
||||
}
|
||||
|
||||
fn assert_raw_transaction_capabilities<T>()
|
||||
where
|
||||
T: ksp_store_lib::RawTransactionRead
|
||||
+ ksp_store_lib::RawTransactionWrite
|
||||
+ ksp_store_lib::RawTransactionObservationRead
|
||||
+ ksp_store_lib::RawTransactionObservationWrite
|
||||
+ ksp_store_lib::RawTransactionRetentionRead
|
||||
+ ksp_store_lib::RawTransactionRetentionWrite,
|
||||
{
|
||||
let _marker = std::marker::PhantomData::<T>;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_store_facade_implements_all_six_raw_transaction_capabilities() {
|
||||
assert_raw_transaction_capabilities::<ksp_store_lib::Store>();
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/unit_tests/settings.rs
|
||||
// version: 3
|
||||
// version: 4
|
||||
|
||||
fn valid_network() -> crate::RawNetworkId {
|
||||
return match crate::RawNetworkId::new("devnet") {
|
||||
@@ -27,6 +27,12 @@ fn defaults_match_the_pre_001_runtime_bounds() {
|
||||
assert_eq!(pool.recycle_timeout(), std::time::Duration::from_millis(5_000));
|
||||
let bootstrap = crate::PostgresBootstrapSettings::default();
|
||||
assert!(bootstrap.auto_migrate());
|
||||
assert!(bootstrap.schema_autocreate());
|
||||
assert!(bootstrap.schema_autoupdate());
|
||||
let split_policy =
|
||||
crate::PostgresBootstrapSettings::with_schema_policy(false, true, std::time::Duration::from_millis(30_000), std::time::Duration::from_millis(10_000));
|
||||
assert!(!split_policy.schema_autocreate());
|
||||
assert!(split_policy.schema_autoupdate());
|
||||
assert_eq!(bootstrap.migration_timeout(), std::time::Duration::from_millis(30_000));
|
||||
assert_eq!(bootstrap.migration_lock_timeout(), std::time::Duration::from_millis(10_000));
|
||||
let store = crate::StoreSettings::with_default_shutdown(valid_network(), crate::StoreBackendSettings::Postgres(valid_postgres_settings()));
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-lib/unit_tests/store.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
fn poll_ready<T>(future: impl std::future::Future<Output = T>) -> T {
|
||||
let mut future = std::boxed::Box::pin(future);
|
||||
@@ -55,3 +55,53 @@ fn known_postgres_without_feature_is_rejected_before_io() {
|
||||
assert_eq!(error.map(|value| return value.code()), std::option::Option::Some(crate::ERROR_CODE_BACKEND_NOT_COMPILED));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_operation_network_guard_rejects_mismatch_without_echoing_requested_network() {
|
||||
let store_network = valid_network();
|
||||
let hostile = match crate::RawNetworkId::new("other-network") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid alternate network rejected: {error:?}"),
|
||||
};
|
||||
let result = super::validate_operation_network(&store_network, &hostile, crate::StoreBackendKind::Postgres);
|
||||
let error = match result {
|
||||
std::result::Result::Err(value) => value,
|
||||
std::result::Result::Ok(()) => panic!("wrong operation network unexpectedly accepted"),
|
||||
};
|
||||
assert_eq!(error.code(), crate::ERROR_CODE_WRONG_NETWORK);
|
||||
assert!(!std::format!("{error:?}").contains("other-network"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[cfg(feature = "postgres")]
|
||||
#[test]
|
||||
fn pre_010_postgres_error_code_mapping_covers_every_current_backend_kind() {
|
||||
let cases = [
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid, crate::ERROR_CODE_POSTGRES_CONFIG_INVALID),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed, crate::ERROR_CODE_POSTGRES_CONNECT_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout, crate::ERROR_CODE_POSTGRES_POOL_TIMEOUT),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed, crate::ERROR_CODE_POSTGRES_HEALTH_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict, ksp_store_api::ERROR_CODE_RAW_CONFLICT),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid, crate::ERROR_CODE_POSTGRES_DATA_INVALID),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed, crate::ERROR_CODE_POSTGRES_MIGRATION_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported, crate::ERROR_CODE_POSTGRES_PAGE_LIMIT_UNSUPPORTED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch, crate::ERROR_CODE_POSTGRES_MIGRATION_MISMATCH),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid, ksp_store_api::ERROR_CODE_RAW_QUERY_INVALID),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed, crate::ERROR_CODE_POSTGRES_READ_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound, crate::ERROR_CODE_RAW_REFERENCE_NOT_FOUND),
|
||||
(
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::RetentionCompactionUnsupported,
|
||||
crate::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED,
|
||||
),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer, crate::ERROR_CODE_POSTGRES_SCHEMA_NEWER),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout, crate::ERROR_CODE_SHUTDOWN_TIMEOUT),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed, crate::ERROR_CODE_POSTGRES_TLS_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::WriteFailed, crate::ERROR_CODE_POSTGRES_WRITE_FAILED),
|
||||
(ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork, crate::ERROR_CODE_WRONG_NETWORK),
|
||||
];
|
||||
assert_eq!(cases.len(), 18);
|
||||
for (kind, expected) in cases {
|
||||
assert_eq!(super::postgres_error_code(kind), expected);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
<!-- file: crates/ksp-store-postgres-lib/README.md -->
|
||||
<!-- version: 1 -->
|
||||
<!-- version: 10 -->
|
||||
|
||||
# ksp-store-postgres-lib
|
||||
|
||||
`ksp-store-postgres-lib` est le backend PostgreSQL physique officiel du Store KSP.
|
||||
|
||||
La crate implémente la fondation connexion/pool/TLS/migrations/health derrière `ksp-store-lib`. Elle dépend directement de `ksp-store-api` mais ne dépend jamais de la façade `ksp-store-lib`.
|
||||
La crate implémente connexion, pool, TLS, migrations, health et persistence RAW derrière `ksp-store-lib`. Elle dépend directement de `ksp-store-api` mais ne dépend jamais de la façade `ksp-store-lib`.
|
||||
|
||||
## Responsabilités
|
||||
|
||||
@@ -15,10 +15,10 @@ La crate possède seule pour PostgreSQL :
|
||||
- le pool borné `deadpool-postgres` ;
|
||||
- la policy TLS physique avec Rustls ;
|
||||
- les roots système et le provider cryptographique AWS-LC ;
|
||||
- le bootstrap/moteur de migrations privé KSP ;
|
||||
- le bootstrap et le moteur de migrations privé KSP ;
|
||||
- la table metadata `ksp_store_schema_migrations` ;
|
||||
- le sentinel `V000__bootstrap.sql` et son checksum SHA-256 ;
|
||||
- l'advisory transaction lock borné des migrations ;
|
||||
- le schéma physique et les statements privés `RawTransaction` ;
|
||||
- les snapshots runtime/health sûrs destinés au bridge de façade ;
|
||||
- la fermeture explicite du pool et son fallback `Drop` best-effort ;
|
||||
- la classification d'erreurs backend sans conserver le texte d'erreur PostgreSQL.
|
||||
@@ -31,7 +31,7 @@ Les applications, jobs et workers KSP ne dépendent normalement pas de cette cra
|
||||
consumer -> ksp-store-lib -> [feature postgres] ksp-store-postgres-lib
|
||||
```
|
||||
|
||||
La surface publique de cette crate existe pour le bridge inter-crates et les tests d'intégration backend. Elle ne constitue pas une seconde façade Store.
|
||||
La surface publique de cette crate existe pour le bridge inter-crates et les tests/intégrations backend. Elle ne constitue pas une seconde façade Store.
|
||||
|
||||
`ksp-store-postgres-lib` ne réexporte pas `tokio-postgres`, Deadpool ou Rustls.
|
||||
|
||||
@@ -57,33 +57,15 @@ Disabled
|
||||
VerifyFull
|
||||
```
|
||||
|
||||
`VerifyFull` exige :
|
||||
`VerifyFull` exige TLS, roots système, certificat valide et vérification de l'identité serveur. Une configuration ne permettant pas de vérifier cette identité, comme `hostaddr` seul, est rejetée.
|
||||
|
||||
- TLS ;
|
||||
- roots système ;
|
||||
- certificat valide ;
|
||||
- vérification de l'identité serveur ;
|
||||
- aucune dégradation automatique en plaintext.
|
||||
## Migrations et schéma
|
||||
|
||||
Les configurations ne permettant pas de vérifier une identité serveur, comme `hostaddr` seul, sont rejetées.
|
||||
Le moteur de migrations embarqué vérifie version logique, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime.
|
||||
|
||||
## Migrations
|
||||
Le bootstrap metadata est conservé comme migration V000. La migration logique V001 matérialise le schéma `RawTransaction` en ressources séparées `tables/`, `constraints/` et `indexes/` afin que le backend puisse vérifier leur compatibilité effective sans transformer les fichiers SQL en parser généraliste.
|
||||
|
||||
La fondation embarque uniquement :
|
||||
|
||||
```text
|
||||
migrations/V000__bootstrap.sql
|
||||
```
|
||||
|
||||
Elle crée la metadata privée :
|
||||
|
||||
```text
|
||||
ksp_store_schema_migrations
|
||||
```
|
||||
|
||||
Le moteur vérifie version, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime.
|
||||
|
||||
Aucune migration métier RAW n'appartient à cette fondation.
|
||||
La base est liée à un seul `RawNetworkId` via `ksp_store_identity`. Une migration enregistrée mais physiquement divergente est un mismatch ; les réparations additives sûres dépendent de `schema_autoupdate`.
|
||||
|
||||
## Health et erreurs
|
||||
|
||||
@@ -100,25 +82,69 @@ Le texte d'erreur PostgreSQL, l'URI, SQL et les valeurs bind ne traversent pas c
|
||||
|
||||
## Support PostgreSQL
|
||||
|
||||
La politique de support de `0.3.2` fixe PostgreSQL 15 comme major minimal. Le test live de fondation refuse explicitement un serveur plus ancien ; le backend ne fixe aucun plafond arbitraire de major PostgreSQL. La compatibilité de migration reste basée sur le schéma KSP.
|
||||
Le major minimal supporté est PostgreSQL 15. Le backend ne fixe aucun plafond arbitraire de major ; la compatibilité opérationnelle reste fondée sur le contrat de schéma KSP et l'introspection du catalogue.
|
||||
|
||||
La preuve opérateur réelle et le major effectivement exercé sont conservés dans la matrice de validation, pas dans cette documentation durable.
|
||||
## Lectures RAW transaction
|
||||
|
||||
## Hors périmètre actuel
|
||||
Le backend expose :
|
||||
|
||||
La crate ne contient encore :
|
||||
```text
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Le SQL et les rows restent privés. Le mapping PostgreSQL est fallible et couvre notamment `NUMERIC(20,0) -> u64`, `BIGINT -> u32/u64`, timestamps bornés, bytes de taille fixe et codes de provenance.
|
||||
|
||||
`Full` lit le payload chaud, `Archived` le reconstruit depuis la relation archive et `Purged` retourne `None`; le tombstone reste accessible séparément.
|
||||
|
||||
## Écritures RAW transaction
|
||||
|
||||
Le backend expose :
|
||||
|
||||
```text
|
||||
persist_raw_transaction_acquisition
|
||||
record_raw_transaction_observation
|
||||
```
|
||||
|
||||
L'acquisition canonique et son observation initiale sont commises dans une seule transaction PostgreSQL. Les clés uniques physiques fournissent l'admission idempotente ; après un conflit unique, le backend verrouille la ligne gagnante et compare le contenu réel avant de conclure `AlreadyPresent` ou `Conflict`.
|
||||
|
||||
Un tombstone `Purged` compatible produit `SkippedPurged/NotRecorded` en mode normal. `ForceRehydrate` restaure explicitement le payload `Full` et l'observation dans la même transaction.
|
||||
|
||||
## Pagination RAW transaction
|
||||
|
||||
`list_raw_transactions` parcourt les références canoniques récupérables avec un ordre total `(slot, signature)`. Les tombstones `Purged` sont exclus.
|
||||
|
||||
La continuation est une keyset stricte, jamais un `OFFSET`. Le cursor backend V1 est opaque et lié au réseau, à la direction et aux bornes de slots de la query. Store n'impose aucun plafond métier arbitraire à la taille de page ; seule la limitation physique du `LIMIT + 1` PostgreSQL est exposée.
|
||||
|
||||
## Rétention RAW transaction
|
||||
|
||||
`transition_raw_transaction_retention` applique les transitions physiques :
|
||||
|
||||
```text
|
||||
Full -> Archived -> Purged
|
||||
```
|
||||
|
||||
Le backend verrouille la ligne canonique avec `FOR UPDATE`, compare l'état courant à l'état attendu et applique la mutation atomiquement. L'archivage conserve le payload exact dans la relation archive ; la purge conserve seulement le tombstone minimal.
|
||||
|
||||
Toute transition impliquant `Compacted` est rejetée avec `RetentionCompactionUnsupported` tant qu'aucune représentation compactée réelle n'est implémentée.
|
||||
|
||||
## Hors périmètre
|
||||
|
||||
La crate ne contient :
|
||||
|
||||
- aucune implémentation PostgreSQL des capabilities `RawTransaction*` ;
|
||||
- aucune implémentation PostgreSQL des capabilities `RawAccount*` ;
|
||||
- aucun repository métier RAW ;
|
||||
- aucune table/index métier ;
|
||||
- aucune orchestration worker/job ;
|
||||
- aucun transport d'acquisition ou decoder Program.
|
||||
- aucun transport d'acquisition ou decoder Program ;
|
||||
- aucune policy autonome de batch, priorité ou rétention.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [`USAGE.md`](USAGE.md) — bridge physique et lifecycle ;
|
||||
- [`USAGE.md`](USAGE.md) — guide pratique du bridge physique et de ses capabilities ;
|
||||
- [`../ksp-store-lib/README.md`](../ksp-store-lib/README.md) — façade runtime destinée aux consumers ;
|
||||
- [`../../docs/architecture/008-DATA_MATERIALIZATION_AND_STORE.md`](../../docs/architecture/008-DATA_MATERIALIZATION_AND_STORE.md) — architecture Store ;
|
||||
- [`../../docs/plans/023-V0_3_2_STORE_POSTGRES_FOUNDATION_PLAN.md`](../../docs/plans/023-V0_3_2_STORE_POSTGRES_FOUNDATION_PLAN.md) — décisions pool/TLS/migrations ;
|
||||
- [`../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md`](../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md) — preuves déterministes et PostgreSQL réel.
|
||||
- [`../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md`](../../docs/validation/019-V0_3_2_STORE_POSTGRES_FOUNDATION.md) — validation de fondation ;
|
||||
- [`../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`](../../docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md) — design `RawTransaction` ;
|
||||
- [`../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`](../../docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md) — validation `RawTransaction`.
|
||||
|
||||
@@ -1,15 +1,13 @@
|
||||
<!-- file: crates/ksp-store-postgres-lib/USAGE.md -->
|
||||
<!-- version: 1 -->
|
||||
<!-- version: 10 -->
|
||||
|
||||
# Utilisation de ksp-store-postgres-lib
|
||||
|
||||
## 1. Quand utiliser cette crate directement
|
||||
## 1. Quand dépendre directement du backend
|
||||
|
||||
Le consumer applicatif normal utilise `ksp-store-lib`.
|
||||
|
||||
Une dépendance directe à `ksp-store-postgres-lib` est réservée aux composants qui implémentent ou testent le bridge physique PostgreSQL. La crate backend ne doit pas devenir une façade parallèle.
|
||||
|
||||
Un tel composant doit déclarer explicitement le backend et `ksp-store-api`, car `PostgresBackendSettings::new` reçoit le `RawNetworkId` backend-neutral sans le réexporter :
|
||||
Une dépendance directe à `ksp-store-postgres-lib` est réservée aux composants qui implémentent, intègrent ou testent le bridge physique PostgreSQL. Cette crate ne doit pas devenir une façade Store parallèle.
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
@@ -17,16 +15,18 @@ ksp-store-api = { path = "../ksp-store-api" }
|
||||
ksp-store-postgres-lib = { path = "../ksp-store-postgres-lib" }
|
||||
```
|
||||
|
||||
## 2. Construire le bridge physique
|
||||
Le backend reçoit les modèles et traits backend-neutral de `ksp-store-api`; il ne réexporte pas `tokio-postgres`, Deadpool ou Rustls.
|
||||
|
||||
`PostgresBackendSettings` reçoit des valeurs déjà possédées et validées par la couche appelante. L'URI est sensible et son `Debug` est redacted.
|
||||
## 2. Construire les settings physiques
|
||||
|
||||
Pour distinguer création initiale et mise à jour du schéma, utiliser `PostgresBackendSettings::with_schema_policy` :
|
||||
|
||||
```rust
|
||||
fn backend_settings(
|
||||
network: ksp_store_api::RawNetworkId,
|
||||
connection_uri: std::string::String,
|
||||
) -> ksp_store_postgres_lib::PostgresBackendSettings {
|
||||
return ksp_store_postgres_lib::PostgresBackendSettings::new(
|
||||
return ksp_store_postgres_lib::PostgresBackendSettings::with_schema_policy(
|
||||
network,
|
||||
connection_uri,
|
||||
8,
|
||||
@@ -36,15 +36,18 @@ fn backend_settings(
|
||||
std::time::Duration::from_secs(5),
|
||||
ksp_store_postgres_lib::PostgresBackendTlsMode::VerifyFull,
|
||||
true,
|
||||
true,
|
||||
std::time::Duration::from_secs(30),
|
||||
std::time::Duration::from_secs(10),
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Le backend reçoit un seul `RawNetworkId`. Une instance physique n'est pas un routeur multi-réseau.
|
||||
`schema_autocreate` autorise l'initialisation d'un Store vierge. `schema_autoupdate` autorise les migrations pending et les réparations additives sûres d'une migration déjà enregistrée. Le constructeur `new(..., auto_migrate, ...)` existe pour les callers utilisant encore un switch unique et applique cette valeur aux deux politiques.
|
||||
|
||||
## 3. Ouvrir, sonder et fermer
|
||||
L'URI est sensible : elle n'est jamais rendue par `Debug`.
|
||||
|
||||
## 3. Ouvrir, sonder et fermer le backend
|
||||
|
||||
```rust
|
||||
async fn use_backend(
|
||||
@@ -63,93 +66,176 @@ async fn use_backend(
|
||||
let _waiting = runtime.pool_waiting();
|
||||
|
||||
let health = backend.health().await;
|
||||
let _ready = health.ready();
|
||||
let _ready = health.is_ready();
|
||||
let _migration_version = health.migration_version();
|
||||
let _pending = health.pending_migration_count();
|
||||
let _safe_error_kind = health.last_error_kind();
|
||||
let _safe_error_kind = health.error_kind();
|
||||
|
||||
return backend.close(std::time::Duration::from_secs(5)).await;
|
||||
}
|
||||
```
|
||||
|
||||
`open` prouve la connexion et le bootstrap avant de retourner. `close` ferme le pool puis attend son drain dans la deadline fournie.
|
||||
`open` valide la configuration, construit le pool, prouve une connexion et vérifie/applique le bootstrap avant de retourner. `close` ferme le pool et attend son drain dans la deadline fournie.
|
||||
|
||||
Une instance physique est liée à un seul `RawNetworkId`.
|
||||
|
||||
## 4. Choisir le mode TLS
|
||||
|
||||
### `VerifyFull`
|
||||
|
||||
À utiliser pour les connexions PostgreSQL protégées :
|
||||
Pour une connexion PostgreSQL protégée :
|
||||
|
||||
```rust
|
||||
ksp_store_postgres_lib::PostgresBackendTlsMode::VerifyFull
|
||||
```
|
||||
|
||||
Le backend charge les roots système et vérifie certificat + identité serveur. Il rejette une configuration ne fournissant pas d'identité vérifiable.
|
||||
`VerifyFull` impose TLS, les roots système et la vérification de l'identité serveur. Une configuration ne fournissant pas d'identité vérifiable est rejetée.
|
||||
|
||||
### `Disabled`
|
||||
Pour une topologie explicitement non chiffrée :
|
||||
|
||||
```rust
|
||||
ksp_store_postgres_lib::PostgresBackendTlsMode::Disabled
|
||||
```
|
||||
|
||||
Ce mode désactive explicitement TLS. Il ne doit être utilisé que lorsque la topologie de déploiement justifie clairement une connexion non chiffrée.
|
||||
La policy typée choisie par KSP prime sur les paramètres SSL de l'URI.
|
||||
|
||||
La valeur typée choisie par KSP prime sur les paramètres SSL de l'URI.
|
||||
## 5. Lire une transaction et ses métadonnées
|
||||
|
||||
## 5. Bootstrap et migrations
|
||||
|
||||
Le backend embarque son propre moteur de migrations. Le seul artefact initial est :
|
||||
|
||||
```text
|
||||
migrations/V000__bootstrap.sql
|
||||
```
|
||||
|
||||
Le bootstrap maintient :
|
||||
|
||||
```text
|
||||
ksp_store_schema_migrations
|
||||
version
|
||||
name
|
||||
checksum SHA-256
|
||||
```
|
||||
|
||||
Le runner est transactionnel et sérialisé par advisory transaction lock. Une divergence de checksum/nom/version ou une history plus récente est terminale ; aucun down migration automatique n'est exécuté.
|
||||
|
||||
`auto_migrate = false` permet de vérifier l'état sans appliquer de migration pending.
|
||||
|
||||
## 6. Classifier les erreurs sans fuite
|
||||
Les méthodes backend retournent uniquement des modèles `ksp-store-api`.
|
||||
|
||||
```rust
|
||||
async fn read_transaction_state(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransaction>, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
let retention = backend.get_raw_transaction_retention_state(reference).await;
|
||||
if let std::result::Result::Err(error) = retention {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
|
||||
let tombstone = backend.get_raw_transaction_tombstone(reference).await;
|
||||
if let std::result::Result::Err(error) = tombstone {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
|
||||
return backend.get_raw_transaction(reference).await;
|
||||
}
|
||||
```
|
||||
|
||||
`Full` lit le payload chaud, `Archived` reconstruit le payload depuis l'archive et `Purged` retourne `None`. Un tombstone purgé reste lisible séparément.
|
||||
|
||||
Un réseau différent de celui du backend est rejeté avant acquisition d'un client du pool.
|
||||
|
||||
## 6. Lire une observation
|
||||
|
||||
```rust
|
||||
async fn read_observation(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
key: &ksp_store_api::RawObservationKey,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionObservation>, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
return backend.get_raw_transaction_observation(key).await;
|
||||
}
|
||||
```
|
||||
|
||||
Les rows PostgreSQL, SQLSTATE, statements et valeurs de bind ne traversent jamais cette API.
|
||||
|
||||
## 7. Persister une acquisition canonique
|
||||
|
||||
```rust
|
||||
async fn persist_acquisition(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
transaction: ksp_store_api::RawTransaction,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
) -> std::result::Result<ksp_store_api::RawAcquisitionWriteOutcome, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
return backend
|
||||
.persist_raw_transaction_acquisition(
|
||||
transaction,
|
||||
observation,
|
||||
ksp_store_api::RawTransactionAcquisitionMode::Normal,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
```
|
||||
|
||||
L'opération est atomique : le canonique et son observation initiale sont tous deux durables ou aucun ne l'est. Une identité déjà présente avec un contenu identique est idempotente ; un contenu divergent retourne `PostgresBackendErrorKind::Conflict` sans overwrite silencieux.
|
||||
|
||||
Pour un tombstone purgé compatible, le mode `Normal` ne restaure pas le payload. `ForceRehydrate` doit être demandé explicitement pour rétablir un payload `Full`.
|
||||
|
||||
## 8. Ajouter une observation à un canonique existant
|
||||
|
||||
```rust
|
||||
async fn record_observation(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
) -> std::result::Result<ksp_store_api::RawObservationWriteOutcome, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
return backend.record_raw_transaction_observation(observation).await;
|
||||
}
|
||||
```
|
||||
|
||||
Cette opération ne crée jamais la transaction canonique. Une référence absente retourne `ReferenceNotFound`; une transaction purgée produit l'outcome `NotRecorded` prévu par l'API.
|
||||
|
||||
## 9. Paginer les transactions
|
||||
|
||||
```rust
|
||||
async fn list_transactions(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
query: &ksp_store_api::RawTransactionQuery,
|
||||
) -> std::result::Result<ksp_store_api::RawPage<ksp_store_api::RawTransactionReference>, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
return backend.list_raw_transactions(query).await;
|
||||
}
|
||||
```
|
||||
|
||||
La navigation est keyset sur `(slot, signature)` et exclut les tombstones `Purged`. Le cursor retourné est opaque et lié au réseau, à la direction et aux bornes de slots de la query qui l'a produit.
|
||||
|
||||
Le backend n'utilise pas `OFFSET` et n'impose pas de plafond métier arbitraire. La seule borne exposée ici provient de la représentation physique de `LIMIT + 1` dans PostgreSQL.
|
||||
|
||||
## 10. Appliquer une transition de rétention
|
||||
|
||||
```rust
|
||||
async fn apply_retention(
|
||||
backend: &ksp_store_postgres_lib::PostgresBackend,
|
||||
transition: ksp_store_api::RawTransactionRetentionTransition,
|
||||
) -> std::result::Result<ksp_store_api::RawRetentionWriteOutcome, ksp_store_postgres_lib::PostgresBackendError> {
|
||||
return backend.transition_raw_transaction_retention(transition).await;
|
||||
}
|
||||
```
|
||||
|
||||
Le backend applique la transition choisie par le caller ; il ne décide pas de la policy d'éligibilité. Les transitions physiques prises en charge sont `Full -> Archived` puis `Archived -> Purged`.
|
||||
|
||||
Une transition impliquant `Compacted` est refusée avec `PostgresBackendErrorKind::RetentionCompactionUnsupported` tant qu'aucune représentation compactée réelle n'est disponible.
|
||||
|
||||
## 11. Classifier les erreurs sans fuite
|
||||
|
||||
```rust
|
||||
fn classify(error: &ksp_store_postgres_lib::PostgresBackendError) {
|
||||
match error.kind() {
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::RetentionCompactionUnsupported => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WriteFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork => {}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
let _safe_phase = error.phase();
|
||||
}
|
||||
```
|
||||
|
||||
Ne pas reconstruire un diagnostic utilisateur à partir de l'erreur brute PostgreSQL : cette erreur n'est volontairement pas conservée par le bridge.
|
||||
`PostgresBackendError` conserve uniquement une classification KSP et une phase statique. Ne pas reconstruire de diagnostic utilisateur à partir d'une erreur brute PostgreSQL.
|
||||
|
||||
## 7. Ce que cette crate ne permet pas encore
|
||||
## 12. Limites du backend direct
|
||||
|
||||
La fondation physique n'implémente pas les traits `RawTransaction*` ou `RawAccount*` de `ksp-store-api`.
|
||||
Le backend ne lit aucune variable d'environnement et ne possède aucune sélection de target Config. Les applications, jobs et workers doivent normalement passer par `ksp-store-lib`.
|
||||
|
||||
Un backend ouvert et healthy prouve uniquement :
|
||||
|
||||
```text
|
||||
connexion/pool
|
||||
TLS selon policy
|
||||
bootstrap/history
|
||||
health/readiness
|
||||
close borné
|
||||
```
|
||||
|
||||
Il ne prouve aucune persistence métier RAW.
|
||||
Les capabilities `RawAccount*` ne sont pas implémentées par ce backend. Les décisions de batch, priorité, backlog, scheduling et policy de rétention restent hors de sa responsabilité.
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'pk_ksp_store_identity'
|
||||
AND conrelid = to_regclass('ksp_store_identity')
|
||||
) THEN
|
||||
ALTER TABLE ksp_store_identity ADD CONSTRAINT pk_ksp_store_identity PRIMARY KEY (singleton);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_store_identity_singleton'
|
||||
AND conrelid = to_regclass('ksp_store_identity')
|
||||
) THEN
|
||||
ALTER TABLE ksp_store_identity ADD CONSTRAINT ck_ksp_store_identity_singleton CHECK (singleton = 1);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_store_identity_network'
|
||||
AND conrelid = to_regclass('ksp_store_identity')
|
||||
) THEN
|
||||
ALTER TABLE ksp_store_identity ADD CONSTRAINT ck_ksp_store_identity_network CHECK (
|
||||
octet_length(network) >= 1 AND octet_length(network) <= 128
|
||||
AND network ~ '^[A-Za-z0-9_.:-]+$'
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'pk_ksp_raw_transactions'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT pk_ksp_raw_transactions PRIMARY KEY (signature);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_signature'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_signature CHECK (octet_length(signature) = 64);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_slot'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_slot CHECK (slot >= 0 AND slot <= 18446744073709551615);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_block_time'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_block_time CHECK (
|
||||
block_time_unix_millis IS NULL
|
||||
OR block_time_unix_millis >= 0 AND block_time_unix_millis <= 253402300799999
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_format_id'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_format_id CHECK (
|
||||
octet_length(format_id) >= 1 AND octet_length(format_id) <= 128
|
||||
AND format_id ~ '^[A-Za-z0-9_.:-]+$'
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_format_version'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_format_version CHECK (format_version >= 1 AND format_version <= 4294967295);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_content_hash'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_content_hash CHECK (octet_length(content_hash) = 32);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_payload'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_payload CHECK (
|
||||
payload IS NULL
|
||||
OR octet_length(payload) >= 1 AND octet_length(payload) <= 16777216
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_retention_state'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_retention_state CHECK ((retention_state = 'full' OR retention_state = 'archived' OR retention_state = 'purged'));
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_payload_state'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_payload_state CHECK (
|
||||
(retention_state = 'full' AND payload IS NOT NULL)
|
||||
OR ((retention_state = 'archived' OR retention_state = 'purged') AND payload IS NULL)
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transactions_purged_block_time'
|
||||
AND conrelid = to_regclass('ksp_raw_transactions')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transactions ADD CONSTRAINT ck_ksp_raw_transactions_purged_block_time CHECK (
|
||||
retention_state <> 'purged'
|
||||
OR block_time_unix_millis IS NULL
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'pk_ksp_raw_transaction_observations'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT pk_ksp_raw_transaction_observations PRIMARY KEY (observation_key);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'fk_ksp_raw_transaction_observations_transaction'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT fk_ksp_raw_transaction_observations_transaction FOREIGN KEY (transaction_signature) REFERENCES ksp_raw_transactions(signature) ON DELETE RESTRICT;
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_key'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_key CHECK (octet_length(observation_key) = 32);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_signature'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_signature CHECK (octet_length(transaction_signature) = 64);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_provider'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_provider CHECK (
|
||||
octet_length(provider) >= 1 AND octet_length(provider) <= 128
|
||||
AND provider ~ '^[A-Za-z0-9_.:-]+$'
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_protocol'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_protocol CHECK (
|
||||
octet_length(protocol) >= 1 AND octet_length(protocol) <= 128
|
||||
AND protocol ~ '^[A-Za-z0-9_.:-]+$'
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_method'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_method CHECK (
|
||||
octet_length(acquisition_method) >= 1 AND octet_length(acquisition_method) <= 128
|
||||
AND acquisition_method ~ '^[A-Za-z0-9_.:-]+$'
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_origin'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_origin CHECK ((origin = 'backfill' OR origin = 'import' OR origin = 'live' OR origin = 'repair' OR origin = 'replay'));
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_received_at'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_received_at CHECK (received_at_unix_millis >= 0 AND received_at_unix_millis <= 253402300799999);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,18 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_capture_session'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_capture_session CHECK (
|
||||
capture_session_id IS NULL
|
||||
OR (
|
||||
octet_length(capture_session_id) >= 1 AND octet_length(capture_session_id) <= 128
|
||||
AND capture_session_id ~ '^[A-Za-z0-9_.:-]+$'
|
||||
)
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,18 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_commitment'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_commitment CHECK (
|
||||
commitment IS NULL
|
||||
OR (
|
||||
octet_length(commitment) >= 1 AND octet_length(commitment) <= 128
|
||||
AND commitment ~ '^[A-Za-z0-9_.:-]+$'
|
||||
)
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,18 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_endpoint'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_endpoint CHECK (
|
||||
endpoint_id IS NULL
|
||||
OR (
|
||||
octet_length(endpoint_id) >= 1 AND octet_length(endpoint_id) <= 128
|
||||
AND endpoint_id ~ '^[A-Za-z0-9_.:-]+$'
|
||||
)
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,18 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_filter'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_filter CHECK (
|
||||
filter_id IS NULL
|
||||
OR (
|
||||
octet_length(filter_id) >= 1 AND octet_length(filter_id) <= 128
|
||||
AND filter_id ~ '^[A-Za-z0-9_.:-]+$'
|
||||
)
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_observed_at'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_observed_at CHECK (
|
||||
observed_at_unix_millis IS NULL
|
||||
OR observed_at_unix_millis >= 0 AND observed_at_unix_millis <= 253402300799999
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_time_order'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_time_order CHECK (
|
||||
observed_at_unix_millis IS NULL
|
||||
OR observed_at_unix_millis <= received_at_unix_millis
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_source_hash'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_source_hash CHECK (
|
||||
source_payload_hash IS NULL
|
||||
OR octet_length(source_payload_hash) = 32
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,15 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_observations_source_size'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_observations')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD CONSTRAINT ck_ksp_raw_transaction_observations_source_size CHECK (
|
||||
source_payload_size_bytes IS NULL
|
||||
OR source_payload_size_bytes >= 0 AND source_payload_size_bytes <= 67108864
|
||||
);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'pk_ksp_raw_transaction_archive_payloads'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_archive_payloads')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD CONSTRAINT pk_ksp_raw_transaction_archive_payloads PRIMARY KEY (signature);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'fk_ksp_raw_transaction_archive_payloads_transaction'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_archive_payloads')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD CONSTRAINT fk_ksp_raw_transaction_archive_payloads_transaction FOREIGN KEY (signature) REFERENCES ksp_raw_transactions(signature) ON DELETE RESTRICT;
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_archive_payloads_signature'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_archive_payloads')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD CONSTRAINT ck_ksp_raw_transaction_archive_payloads_signature CHECK (octet_length(signature) = 64);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,12 @@
|
||||
DO $ksp$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'ck_ksp_raw_transaction_archive_payloads_payload'
|
||||
AND conrelid = to_regclass('ksp_raw_transaction_archive_payloads')
|
||||
) THEN
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD CONSTRAINT ck_ksp_raw_transaction_archive_payloads_payload CHECK (octet_length(payload) >= 1 AND octet_length(payload) <= 16777216);
|
||||
END IF;
|
||||
END
|
||||
$ksp$;
|
||||
@@ -0,0 +1,3 @@
|
||||
CREATE INDEX IF NOT EXISTS ix_ksp_raw_transactions_slot_signature
|
||||
ON ksp_raw_transactions (slot, signature)
|
||||
WHERE retention_state <> 'purged';
|
||||
@@ -0,0 +1,7 @@
|
||||
CREATE TABLE IF NOT EXISTS ksp_store_identity (
|
||||
singleton SMALLINT NOT NULL,
|
||||
network TEXT NOT NULL
|
||||
);
|
||||
|
||||
ALTER TABLE ksp_store_identity ADD COLUMN IF NOT EXISTS singleton SMALLINT NOT NULL;
|
||||
ALTER TABLE ksp_store_identity ADD COLUMN IF NOT EXISTS network TEXT NOT NULL;
|
||||
@@ -0,0 +1,19 @@
|
||||
CREATE TABLE IF NOT EXISTS ksp_raw_transactions (
|
||||
signature BYTEA NOT NULL,
|
||||
slot NUMERIC(20, 0) NOT NULL,
|
||||
block_time_unix_millis BIGINT NULL,
|
||||
format_id TEXT NOT NULL,
|
||||
format_version BIGINT NOT NULL,
|
||||
content_hash BYTEA NOT NULL,
|
||||
payload BYTEA NULL,
|
||||
retention_state TEXT NOT NULL
|
||||
);
|
||||
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS signature BYTEA NOT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS slot NUMERIC(20, 0) NOT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS block_time_unix_millis BIGINT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS format_id TEXT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS format_version BIGINT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS content_hash BYTEA NOT NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS payload BYTEA NULL;
|
||||
ALTER TABLE ksp_raw_transactions ADD COLUMN IF NOT EXISTS retention_state TEXT NOT NULL;
|
||||
@@ -0,0 +1,31 @@
|
||||
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_observations (
|
||||
observation_key BYTEA NOT NULL,
|
||||
transaction_signature BYTEA NOT NULL,
|
||||
provider TEXT NOT NULL,
|
||||
protocol TEXT NOT NULL,
|
||||
acquisition_method TEXT NOT NULL,
|
||||
origin TEXT NOT NULL,
|
||||
received_at_unix_millis BIGINT NOT NULL,
|
||||
capture_session_id TEXT NULL,
|
||||
commitment TEXT NULL,
|
||||
endpoint_id TEXT NULL,
|
||||
filter_id TEXT NULL,
|
||||
observed_at_unix_millis BIGINT NULL,
|
||||
source_payload_hash BYTEA NULL,
|
||||
source_payload_size_bytes BIGINT NULL
|
||||
);
|
||||
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS observation_key BYTEA NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS transaction_signature BYTEA NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS provider TEXT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS protocol TEXT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS acquisition_method TEXT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS origin TEXT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS received_at_unix_millis BIGINT NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS capture_session_id TEXT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS commitment TEXT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS endpoint_id TEXT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS filter_id TEXT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS observed_at_unix_millis BIGINT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS source_payload_hash BYTEA NULL;
|
||||
ALTER TABLE ksp_raw_transaction_observations ADD COLUMN IF NOT EXISTS source_payload_size_bytes BIGINT NULL;
|
||||
@@ -0,0 +1,7 @@
|
||||
CREATE TABLE IF NOT EXISTS ksp_raw_transaction_archive_payloads (
|
||||
signature BYTEA NOT NULL,
|
||||
payload BYTEA NOT NULL
|
||||
);
|
||||
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD COLUMN IF NOT EXISTS signature BYTEA NOT NULL;
|
||||
ALTER TABLE ksp_raw_transaction_archive_payloads ADD COLUMN IF NOT EXISTS payload BYTEA NOT NULL;
|
||||
@@ -1,5 +1,9 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/error.rs
|
||||
// version: 3
|
||||
// version: 8
|
||||
|
||||
/// Stable KSP error code reserved for PostgreSQL retention transitions that require unsupported physical compaction.
|
||||
pub const ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED: ksp_store_api::ErrorCode =
|
||||
ksp_store_api::ErrorCode::new("store", "postgres_retention_compaction_unsupported");
|
||||
|
||||
/// Safe backend-local classification used by the Store facade for stable error mapping.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
@@ -13,16 +17,34 @@ pub enum PostgresBackendErrorKind {
|
||||
PoolTimeout,
|
||||
/// A lightweight PostgreSQL health/readiness probe failed without exposing server text or SQL.
|
||||
HealthFailed,
|
||||
/// A canonical RAW identity or observation key already exists with divergent durable content.
|
||||
Conflict,
|
||||
/// PostgreSQL returned stored RAW data that cannot be represented by the stable Store API contract.
|
||||
DataInvalid,
|
||||
/// PostgreSQL migration/bootstrap execution failed without exposing server text or SQL.
|
||||
MigrationFailed,
|
||||
/// The requested RAW page size cannot be represented by PostgreSQL LIMIT plus the continuation probe row.
|
||||
PageLimitUnsupported,
|
||||
/// Applied PostgreSQL migration history diverges from the embedded immutable KSP history.
|
||||
MigrationMismatch,
|
||||
/// A bounded backend-private RAW query or opaque cursor is invalid for the requested navigation context.
|
||||
QueryInvalid,
|
||||
/// A PostgreSQL RAW read statement failed without exposing server text, SQL or bind values.
|
||||
ReadFailed,
|
||||
/// A RAW write requires an existing canonical reference that is not durable.
|
||||
ReferenceNotFound,
|
||||
/// The requested RAW retention transition requires a compacted representation unsupported by PostgreSQL.
|
||||
RetentionCompactionUnsupported,
|
||||
/// The database schema history contains a migration newer than this runtime understands.
|
||||
SchemaNewer,
|
||||
/// Explicit backend shutdown did not drain inside the supplied deadline.
|
||||
ShutdownTimeout,
|
||||
/// Verified TLS configuration or negotiation could not be established.
|
||||
TlsFailed,
|
||||
/// A PostgreSQL RAW write statement or transaction failed without exposing server text, SQL or bind values.
|
||||
WriteFailed,
|
||||
/// A network-scoped RAW operation targeted a network different from the backend binding.
|
||||
WrongNetwork,
|
||||
}
|
||||
|
||||
/// Redacted PostgreSQL backend error carrying only a safe classification and static lifecycle phase.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/lib.rs
|
||||
// version: 5
|
||||
// version: 14
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -7,10 +7,20 @@
|
||||
|
||||
//! Official PostgreSQL backend implementation for KSP Store.
|
||||
//!
|
||||
//! `0.3.2-pre.007` owns the physical `tokio-postgres` connection, bounded
|
||||
//! Deadpool pool, explicit Rustls TLS policy, private KSP migration/bootstrap
|
||||
//! engine and safe lightweight health/readiness probe. Business persistence
|
||||
//! remains absent from this foundation release.
|
||||
//! The backend owns the physical `tokio-postgres` connection, bounded Deadpool
|
||||
//! pool, explicit Rustls TLS policy, private KSP migration/bootstrap engine and
|
||||
//! safe lightweight health/readiness probe. `0.3.3-pre.003-fix.001` splits
|
||||
//! migrations into versioned physical resources and verifies the effective
|
||||
//! PostgreSQL schema contract before readiness. `0.3.3-pre.004` adds exact
|
||||
//! backend-private RAW transaction/observation/retention read mapping.
|
||||
//! `0.3.3-pre.005` adds atomic canonical/observation writes, real idempotence
|
||||
//! checks and safe conflict classification without exposing PostgreSQL rows or
|
||||
//! SQL through the public bridge. `0.3.3-pre.006` adds deterministic keyset
|
||||
//! pagination with a fixed opaque cursor bound to network, range and direction.
|
||||
//! `0.3.3-pre.007` adds atomic `Full -> Archived -> Purged` retention transitions
|
||||
//! with compare-and-transition outcomes and explicit rejection of `Compacted`.
|
||||
//! `0.3.3-pre.008` implements all six `RawTransaction*` capabilities directly on
|
||||
//! `PostgresBackend` while preserving the existing narrow backend bridge.
|
||||
//!
|
||||
//! This crate depends on `ksp-store-api` and never on `ksp-store-lib`. The
|
||||
//! common facade consumes only this crate's narrow backend bridge and never
|
||||
@@ -20,8 +30,12 @@ mod constants;
|
||||
mod error;
|
||||
mod health;
|
||||
mod migration;
|
||||
mod raw_transaction;
|
||||
mod runtime;
|
||||
mod schema;
|
||||
|
||||
/// Stable KSP error code for unsupported PostgreSQL retention compaction.
|
||||
pub use self::error::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED;
|
||||
/// Safe backend-local error returned to the common Store facade.
|
||||
pub use self::error::PostgresBackendError;
|
||||
/// Safe backend-local error classification used by the common Store facade.
|
||||
@@ -45,9 +59,45 @@ pub(crate) use self::health::probe_health;
|
||||
pub(crate) use self::migration::bootstrap;
|
||||
/// Current embedded migration version consumed by the private health probe.
|
||||
pub(crate) use self::migration::current_migration_version;
|
||||
/// Private RAW transaction cursor decoder consumed by the physical RAW module.
|
||||
pub(crate) use self::raw_transaction::cursor::decode_raw_transaction_cursor;
|
||||
/// Private RAW transaction cursor encoder consumed by the physical RAW module.
|
||||
pub(crate) use self::raw_transaction::cursor::encode_raw_transaction_cursor;
|
||||
/// Private physical page-limit converter consumed by the physical RAW module.
|
||||
pub(crate) use self::raw_transaction::cursor::raw_transaction_physical_page_limit;
|
||||
/// Private RAW transaction reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction;
|
||||
/// Private RAW transaction observation reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_observation;
|
||||
/// Private RAW transaction retention-state reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_retention_state;
|
||||
/// Private RAW transaction tombstone reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_tombstone;
|
||||
/// Private RAW transaction list reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::list_raw_transactions;
|
||||
/// Private atomic RAW transaction acquisition writer consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::persist_raw_transaction_acquisition;
|
||||
/// Private additional RAW transaction observation writer consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::record_raw_transaction_observation;
|
||||
/// Private RAW transaction retention transition writer consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::transition_raw_transaction_retention;
|
||||
/// Private Deadpool error mapper shared with the health probe.
|
||||
pub(crate) use self::runtime::map_pool_error;
|
||||
/// Private Deadpool status projector shared with the health probe.
|
||||
pub(crate) use self::runtime::runtime_snapshot_from_status;
|
||||
/// Private physical schema resource descriptor consumed by the migration engine.
|
||||
pub(crate) use self::schema::SchemaResource;
|
||||
/// Private physical schema resource compatibility state consumed by the migration engine.
|
||||
pub(crate) use self::schema::SchemaResourceState;
|
||||
/// Private V000 schema resource inventory consumed by the migration engine.
|
||||
pub(crate) use self::schema::V000_RESOURCES;
|
||||
/// Private V001 schema resource inventory consumed by the migration engine.
|
||||
pub(crate) use self::schema::V001_RESOURCES;
|
||||
/// Private physical schema resource inspector consumed by the migration engine.
|
||||
pub(crate) use self::schema::inspect_resource;
|
||||
/// Private V001 adoption probe consumed by the migration engine.
|
||||
pub(crate) use self::schema::managed_v001_objects_exist;
|
||||
/// Private external-schema compatibility gate consumed by the migration engine.
|
||||
pub(crate) use self::schema::verify_v001_external_compatibility;
|
||||
|
||||
const _: &str = crate::TRACING_TARGET;
|
||||
|
||||
@@ -1,15 +1,30 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/migration.rs
|
||||
// version: 2
|
||||
// version: 7
|
||||
|
||||
use sha2::Digest; // rust-rules: trait-import
|
||||
|
||||
const ADVISORY_LOCK_KEY: i64 = 0x4b53_5053_544f_5245;
|
||||
const BOOTSTRAP_MIGRATION_NAME: &str = "bootstrap";
|
||||
const BOOTSTRAP_MIGRATION_SQL: &str = include_str!("../migrations/V000__bootstrap.sql");
|
||||
const BOOTSTRAP_MIGRATION_VERSION: i64 = 0;
|
||||
const EMBEDDED_MIGRATIONS: &[EmbeddedMigration] = &[
|
||||
EmbeddedMigration {
|
||||
checksum: MigrationChecksum::LegacySql(include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql")),
|
||||
hook: MigrationHook::None,
|
||||
name: "bootstrap",
|
||||
resources: crate::V000_RESOURCES,
|
||||
version: 0,
|
||||
},
|
||||
EmbeddedMigration {
|
||||
checksum: MigrationChecksum::Resources,
|
||||
hook: MigrationHook::StoreIdentity,
|
||||
name: "raw_transaction",
|
||||
resources: crate::V001_RESOURCES,
|
||||
version: 1,
|
||||
},
|
||||
];
|
||||
const HEX_LOWER: &[u8; 16] = b"0123456789abcdef";
|
||||
const HISTORY_INSERT_SQL: &str = "INSERT INTO ksp_store_schema_migrations (version, name, checksum, applied_at) VALUES ($1, $2, $3, CURRENT_TIMESTAMP)";
|
||||
const HISTORY_LOAD_SQL: &str = "SELECT version, name, checksum FROM ksp_store_schema_migrations ORDER BY version";
|
||||
const IDENTITY_INSERT_SQL: &str = "INSERT INTO ksp_store_identity (singleton, network) VALUES (1, $1)";
|
||||
const IDENTITY_LOAD_SQL: &str = "SELECT singleton, network FROM ksp_store_identity ORDER BY singleton LIMIT 2";
|
||||
const LOCK_POLL_INTERVAL_MS: u64 = 25;
|
||||
const METADATA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||
SELECT 1 FROM information_schema.tables
|
||||
@@ -17,21 +32,6 @@ const METADATA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||
AND table_name = 'ksp_store_schema_migrations'
|
||||
AND table_type = 'BASE TABLE'
|
||||
)"#;
|
||||
const METADATA_PRIMARY_KEY_SQL: &str = r#"SELECT COUNT(*)::BIGINT,
|
||||
COUNT(*) FILTER (WHERE kcu.column_name = 'version')::BIGINT
|
||||
FROM information_schema.table_constraints tc
|
||||
JOIN information_schema.key_column_usage kcu
|
||||
ON tc.constraint_catalog = kcu.constraint_catalog
|
||||
AND tc.constraint_schema = kcu.constraint_schema
|
||||
AND tc.constraint_name = kcu.constraint_name
|
||||
WHERE tc.table_schema = current_schema()
|
||||
AND tc.table_name = 'ksp_store_schema_migrations'
|
||||
AND tc.constraint_type = 'PRIMARY KEY'"#;
|
||||
const METADATA_SHAPE_SQL: &str = r#"SELECT column_name, data_type, is_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = current_schema()
|
||||
AND table_name = 'ksp_store_schema_migrations'
|
||||
ORDER BY ordinal_position"#;
|
||||
const SET_STATEMENT_TIMEOUT_SQL: &str = "SELECT set_config('statement_timeout', $1, true)";
|
||||
|
||||
struct AppliedMigration {
|
||||
@@ -40,20 +40,63 @@ struct AppliedMigration {
|
||||
version: i64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
enum MigrationChecksum {
|
||||
LegacySql(&'static str),
|
||||
Resources,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
struct EmbeddedMigration {
|
||||
checksum: MigrationChecksum,
|
||||
hook: MigrationHook,
|
||||
name: &'static str,
|
||||
resources: &'static [crate::SchemaResource],
|
||||
version: i64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
enum MigrationHook {
|
||||
None,
|
||||
StoreIdentity,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
enum MigrationHookContext {
|
||||
AppliedNow,
|
||||
Existing,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
enum SchemaMutationMode {
|
||||
Create,
|
||||
Update,
|
||||
}
|
||||
|
||||
/// Returns the latest migration version embedded by this backend runtime.
|
||||
#[must_use]
|
||||
pub(crate) const fn current_migration_version() -> i64 {
|
||||
return BOOTSTRAP_MIGRATION_VERSION;
|
||||
return EMBEDDED_MIGRATIONS[EMBEDDED_MIGRATIONS.len() - 1].version;
|
||||
}
|
||||
|
||||
/// Runs the private bounded PostgreSQL schema bootstrap on one dedicated pooled client.
|
||||
pub(crate) async fn bootstrap(
|
||||
client: &mut deadpool_postgres::Client,
|
||||
auto_migrate: bool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
migration_timeout: std::time::Duration,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let bounded = tokio::time::timeout(migration_timeout, bootstrap_inner(client, auto_migrate, migration_timeout, migration_lock_timeout)).await;
|
||||
let registry_result = validate_embedded_registry(EMBEDDED_MIGRATIONS);
|
||||
if let std::result::Result::Err(error) = registry_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let bounded = tokio::time::timeout(
|
||||
migration_timeout,
|
||||
bootstrap_inner(client, network, schema_autocreate, schema_autoupdate, migration_timeout, migration_lock_timeout),
|
||||
)
|
||||
.await;
|
||||
return match bounded {
|
||||
std::result::Result::Ok(result) => result,
|
||||
std::result::Result::Err(_) => {
|
||||
@@ -64,7 +107,9 @@ pub(crate) async fn bootstrap(
|
||||
|
||||
async fn bootstrap_inner(
|
||||
client: &mut deadpool_postgres::Client,
|
||||
auto_migrate: bool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
migration_timeout: std::time::Duration,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
@@ -88,40 +133,58 @@ async fn bootstrap_inner(
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let checksum = bootstrap_checksum();
|
||||
if !metadata_exists {
|
||||
if !auto_migrate {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_pending"));
|
||||
}
|
||||
let create_result = transaction.batch_execute(BOOTSTRAP_MIGRATION_SQL).await;
|
||||
if create_result.is_err() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_create"));
|
||||
}
|
||||
let shape_result = verify_metadata_shape(&transaction).await;
|
||||
if let std::result::Result::Err(error) = shape_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let insert_result = transaction.execute(HISTORY_INSERT_SQL, &[&BOOTSTRAP_MIGRATION_VERSION, &BOOTSTRAP_MIGRATION_NAME, &checksum]).await;
|
||||
match insert_result {
|
||||
std::result::Result::Ok(1) => {},
|
||||
std::result::Result::Ok(_) | std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_insert"));
|
||||
let (next_index, mutation_mode) = if metadata_exists {
|
||||
let metadata_result = crate::inspect_resource(&transaction, &crate::V000_RESOURCES[0]).await;
|
||||
match metadata_result {
|
||||
std::result::Result::Ok(crate::SchemaResourceState::Compatible) => {},
|
||||
std::result::Result::Ok(crate::SchemaResourceState::Missing | crate::SchemaResourceState::Incompatible) => {
|
||||
log_schema_block("metadata_incompatible");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "metadata_incompatible"));
|
||||
},
|
||||
}
|
||||
} else {
|
||||
let shape_result = verify_metadata_shape(&transaction).await;
|
||||
if let std::result::Result::Err(error) = shape_result {
|
||||
return std::result::Result::Err(error);
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
}
|
||||
let history_result = load_history(&transaction).await;
|
||||
let history = match history_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let validation_result = validate_history(history.as_slice(), checksum.as_str());
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
let validation_result = validate_history(history.as_slice(), EMBEDDED_MIGRATIONS);
|
||||
let index = match validation_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
(index, SchemaMutationMode::Update)
|
||||
} else {
|
||||
if !schema_autocreate {
|
||||
log_schema_block("schema_autocreate_disabled");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "schema_autocreate_disabled"));
|
||||
}
|
||||
let managed_result = crate::managed_v001_objects_exist(&transaction).await;
|
||||
let managed_objects_exist = match managed_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
if managed_objects_exist && !schema_autoupdate {
|
||||
log_schema_block("schema_adoption_disabled");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "schema_adoption_disabled"));
|
||||
}
|
||||
(0, SchemaMutationMode::Create)
|
||||
};
|
||||
let existing_schema_result = verify_or_repair_applied_migrations(&transaction, next_index, schema_autoupdate).await;
|
||||
if let std::result::Result::Err(error) = existing_schema_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let existing_hook_result = run_applied_migration_hooks(&transaction, network, next_index).await;
|
||||
if let std::result::Result::Err(error) = existing_hook_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
if next_index < EMBEDDED_MIGRATIONS.len() && mutation_mode == SchemaMutationMode::Update && !schema_autoupdate {
|
||||
log_schema_block("migration_pending");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "migration_pending"));
|
||||
}
|
||||
let apply_result = apply_pending_migrations(&transaction, network, next_index, mutation_mode).await;
|
||||
if let std::result::Result::Err(error) = apply_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let commit_result = transaction.commit().await;
|
||||
return match commit_result {
|
||||
@@ -171,86 +234,104 @@ async fn acquire_advisory_lock(
|
||||
}
|
||||
}
|
||||
|
||||
async fn set_statement_timeout(
|
||||
async fn apply_migration(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
timeout: std::time::Duration,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
migration: &EmbeddedMigration,
|
||||
mutation_mode: SchemaMutationMode,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let timeout_value = format!("{}ms", timeout.as_millis());
|
||||
let result = transaction.query_one(SET_STATEMENT_TIMEOUT_SQL, &[&timeout_value]).await;
|
||||
return match result {
|
||||
std::result::Result::Ok(_) => std::result::Result::Ok(()),
|
||||
std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "statement_timeout"))
|
||||
for resource in migration.resources {
|
||||
let result = ensure_resource(transaction, resource, mutation_mode, false).await;
|
||||
if let std::result::Result::Err(error) = result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
}
|
||||
let contract_result = verify_migration_contract(transaction, migration.version).await;
|
||||
if let std::result::Result::Err(error) = contract_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let hook_result = run_migration_hook(transaction, network, migration.hook, MigrationHookContext::AppliedNow).await;
|
||||
if let std::result::Result::Err(error) = hook_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let checksum = migration_checksum(migration);
|
||||
let insert_result = transaction.execute(HISTORY_INSERT_SQL, &[&migration.version, &migration.name, &checksum]).await;
|
||||
return match insert_result {
|
||||
std::result::Result::Ok(1) => std::result::Result::Ok(()),
|
||||
std::result::Result::Ok(_) | std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "history_insert"))
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async fn metadata_exists(transaction: &deadpool_postgres::Transaction<'_>) -> std::result::Result<bool, crate::PostgresBackendError> {
|
||||
let result = transaction.query_one(METADATA_EXISTS_SQL, &[]).await;
|
||||
let row = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe"));
|
||||
},
|
||||
};
|
||||
return match row.try_get::<usize, bool>(0) {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe_decode"))
|
||||
},
|
||||
};
|
||||
async fn apply_pending_migrations(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
next_index: usize,
|
||||
mutation_mode: SchemaMutationMode,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let mut index = next_index;
|
||||
while index < EMBEDDED_MIGRATIONS.len() {
|
||||
let migration = &EMBEDDED_MIGRATIONS[index];
|
||||
let result = apply_migration(transaction, network, migration, mutation_mode).await;
|
||||
if let std::result::Result::Err(error) = result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
index += 1;
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn verify_metadata_shape(transaction: &deadpool_postgres::Transaction<'_>) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let result = transaction.query(METADATA_SHAPE_SQL, &[]).await;
|
||||
let rows = match result {
|
||||
async fn ensure_resource(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
resource: &crate::SchemaResource,
|
||||
mutation_mode: SchemaMutationMode,
|
||||
applied_history: bool,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let state_result = crate::inspect_resource(transaction, resource).await;
|
||||
let state = match state_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_shape"));
|
||||
},
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let expected = [("version", "bigint", "NO"), ("name", "text", "NO"), ("checksum", "text", "NO"), ("applied_at", "timestamp with time zone", "NO")];
|
||||
let mut found = [false; 4];
|
||||
for row in rows {
|
||||
let column_result = row.try_get::<usize, std::string::String>(0);
|
||||
let data_type_result = row.try_get::<usize, std::string::String>(1);
|
||||
let nullable_result = row.try_get::<usize, std::string::String>(2);
|
||||
let (column, data_type, nullable) = match (column_result, data_type_result, nullable_result) {
|
||||
(std::result::Result::Ok(column), std::result::Result::Ok(data_type), std::result::Result::Ok(nullable)) => (column, data_type, nullable),
|
||||
_ => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_shape_decode"));
|
||||
match state {
|
||||
crate::SchemaResourceState::Compatible => return std::result::Result::Ok(()),
|
||||
crate::SchemaResourceState::Incompatible => {
|
||||
log_schema_resource_block(resource.id, "incompatible");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||
"schema_resource_incompatible",
|
||||
));
|
||||
},
|
||||
};
|
||||
for (index, expected_row) in expected.iter().enumerate() {
|
||||
if column == expected_row.0 {
|
||||
if found[index] || data_type != expected_row.1 || nullable != expected_row.2 {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "metadata_shape"));
|
||||
crate::SchemaResourceState::Missing => {},
|
||||
}
|
||||
found[index] = true;
|
||||
break;
|
||||
if applied_history && !resource.repair_existing {
|
||||
log_schema_resource_block(resource.id, "repair_forbidden");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "schema_resource_missing"));
|
||||
}
|
||||
if applied_history && mutation_mode != SchemaMutationMode::Update {
|
||||
log_schema_resource_block(resource.id, "repair_mode_invalid");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "schema_resource_missing"));
|
||||
}
|
||||
if applied_history {
|
||||
ksp_logging_lib::warn!(
|
||||
target: crate::TRACING_TARGET,
|
||||
resource_id = resource.id,
|
||||
"repairing missing PostgreSQL Store schema resource under schema_autoupdate policy"
|
||||
);
|
||||
}
|
||||
for required in found {
|
||||
if !required {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "metadata_shape"));
|
||||
let execute_result = transaction.batch_execute(resource.sql).await;
|
||||
if execute_result.is_err() {
|
||||
log_schema_resource_block(resource.id, "apply_failed");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "schema_resource_apply"));
|
||||
}
|
||||
}
|
||||
let key_result = transaction.query_one(METADATA_PRIMARY_KEY_SQL, &[]).await;
|
||||
let key_row = match key_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_primary_key"));
|
||||
let verified_result = crate::inspect_resource(transaction, resource).await;
|
||||
return match verified_result {
|
||||
std::result::Result::Ok(crate::SchemaResourceState::Compatible) => std::result::Result::Ok(()),
|
||||
std::result::Result::Ok(crate::SchemaResourceState::Missing | crate::SchemaResourceState::Incompatible) => {
|
||||
log_schema_resource_block(resource.id, "post_apply_incompatible");
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, resource.id))
|
||||
},
|
||||
};
|
||||
let key_count = key_row.try_get::<usize, i64>(0);
|
||||
let version_count = key_row.try_get::<usize, i64>(1);
|
||||
return match (key_count, version_count) {
|
||||
(std::result::Result::Ok(1), std::result::Result::Ok(1)) => std::result::Result::Ok(()),
|
||||
(std::result::Result::Ok(_), std::result::Result::Ok(_)) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "metadata_primary_key"))
|
||||
},
|
||||
_ => std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_primary_key_decode")),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -279,31 +360,46 @@ async fn load_history(transaction: &deadpool_postgres::Transaction<'_>) -> std::
|
||||
return std::result::Result::Ok(history);
|
||||
}
|
||||
|
||||
fn validate_history(history: &[AppliedMigration], expected_checksum: &str) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let mut sentinel_found = false;
|
||||
for applied in history {
|
||||
if applied.version > BOOTSTRAP_MIGRATION_VERSION {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||
}
|
||||
if applied.version < BOOTSTRAP_MIGRATION_VERSION {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_unknown"));
|
||||
}
|
||||
if applied.name != BOOTSTRAP_MIGRATION_NAME || applied.checksum != expected_checksum {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_diverged"));
|
||||
}
|
||||
sentinel_found = true;
|
||||
}
|
||||
if !sentinel_found {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_missing"));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
async fn metadata_exists(transaction: &deadpool_postgres::Transaction<'_>) -> std::result::Result<bool, crate::PostgresBackendError> {
|
||||
let result = transaction.query_one(METADATA_EXISTS_SQL, &[]).await;
|
||||
let row = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe"));
|
||||
},
|
||||
};
|
||||
return match row.try_get::<usize, bool>(0) {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "metadata_probe_decode"))
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
fn bootstrap_checksum() -> std::string::String {
|
||||
fn migration_checksum(migration: &EmbeddedMigration) -> std::string::String {
|
||||
return match migration.checksum {
|
||||
MigrationChecksum::LegacySql(sql) => checksum_bytes(sql.as_bytes()),
|
||||
MigrationChecksum::Resources => {
|
||||
let mut hasher = sha2::Sha256::new();
|
||||
hasher.update(BOOTSTRAP_MIGRATION_SQL.as_bytes());
|
||||
let digest = hasher.finalize();
|
||||
let bytes = digest.as_slice();
|
||||
hasher.update(b"ksp-migration-resources-v1\0");
|
||||
for resource in migration.resources {
|
||||
hasher.update(resource.id.as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(resource.sql.as_bytes());
|
||||
hasher.update([0]);
|
||||
}
|
||||
return encode_digest(hasher.finalize().as_slice());
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
fn checksum_bytes(bytes: &[u8]) -> std::string::String {
|
||||
let mut hasher = sha2::Sha256::new();
|
||||
hasher.update(bytes);
|
||||
return encode_digest(hasher.finalize().as_slice());
|
||||
}
|
||||
|
||||
fn encode_digest(bytes: &[u8]) -> std::string::String {
|
||||
let mut encoded = std::string::String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
let value = *byte;
|
||||
@@ -313,6 +409,241 @@ fn bootstrap_checksum() -> std::string::String {
|
||||
return encoded;
|
||||
}
|
||||
|
||||
async fn run_applied_migration_hooks(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
applied_count: usize,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let mut index = 0_usize;
|
||||
while index < applied_count {
|
||||
let migration = &EMBEDDED_MIGRATIONS[index];
|
||||
let result = run_migration_hook(transaction, network, migration.hook, MigrationHookContext::Existing).await;
|
||||
if let std::result::Result::Err(error) = result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
index += 1;
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn run_migration_hook(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
hook: MigrationHook,
|
||||
context: MigrationHookContext,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
return match hook {
|
||||
MigrationHook::None => std::result::Result::Ok(()),
|
||||
MigrationHook::StoreIdentity => bind_store_identity(transaction, network, context).await,
|
||||
};
|
||||
}
|
||||
|
||||
async fn bind_store_identity(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
context: MigrationHookContext,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let first_read = load_store_identity(transaction).await;
|
||||
let first = match first_read {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
if first.is_empty() && context == MigrationHookContext::AppliedNow {
|
||||
let insert_result = transaction.execute(IDENTITY_INSERT_SQL, &[&network.as_str()]).await;
|
||||
match insert_result {
|
||||
std::result::Result::Ok(1) => {},
|
||||
std::result::Result::Ok(_) | std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "store_identity_insert"));
|
||||
},
|
||||
}
|
||||
let second_read = load_store_identity(transaction).await;
|
||||
let second = match second_read {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
return validate_store_identity(second.as_slice(), network);
|
||||
}
|
||||
return validate_store_identity(first.as_slice(), network);
|
||||
}
|
||||
|
||||
async fn load_store_identity(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
) -> std::result::Result<std::vec::Vec<tokio_postgres::Row>, crate::PostgresBackendError> {
|
||||
let rows_result = transaction.query(IDENTITY_LOAD_SQL, &[]).await;
|
||||
return match rows_result {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_read"))
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
fn validate_store_identity(rows: &[tokio_postgres::Row], network: &ksp_store_api::RawNetworkId) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
if rows.len() != 1 {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_count"));
|
||||
}
|
||||
let row = &rows[0];
|
||||
let singleton_result = row.try_get::<usize, i16>(0);
|
||||
let network_result = row.try_get::<usize, std::string::String>(1);
|
||||
let (singleton, stored_network) = match (singleton_result, network_result) {
|
||||
(std::result::Result::Ok(singleton), std::result::Result::Ok(stored_network)) => (singleton, stored_network),
|
||||
_ => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_decode"));
|
||||
},
|
||||
};
|
||||
if singleton != 1 {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_singleton"));
|
||||
}
|
||||
let stored_network = match ksp_store_api::RawNetworkId::new(stored_network) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_network"));
|
||||
},
|
||||
};
|
||||
if stored_network.as_str() != network.as_str() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "store_identity_network"));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn set_statement_timeout(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
timeout: std::time::Duration,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let timeout_value = format!("{}ms", timeout.as_millis());
|
||||
let result = transaction.query_one(SET_STATEMENT_TIMEOUT_SQL, &[&timeout_value]).await;
|
||||
return match result {
|
||||
std::result::Result::Ok(_) => std::result::Result::Ok(()),
|
||||
std::result::Result::Err(_) => {
|
||||
std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationFailed, "statement_timeout"))
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async fn verify_migration_contract(transaction: &deadpool_postgres::Transaction<'_>, version: i64) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
if version == 1 {
|
||||
let result = crate::verify_v001_external_compatibility(transaction).await;
|
||||
if let std::result::Result::Err(error) = result {
|
||||
log_schema_block(error.phase());
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn verify_or_repair_applied_migrations(
|
||||
transaction: &deadpool_postgres::Transaction<'_>,
|
||||
applied_count: usize,
|
||||
schema_autoupdate: bool,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
let mut index = 0_usize;
|
||||
while index < applied_count {
|
||||
let migration = &EMBEDDED_MIGRATIONS[index];
|
||||
for resource in migration.resources {
|
||||
let state_result = crate::inspect_resource(transaction, resource).await;
|
||||
let state = match state_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
match state {
|
||||
crate::SchemaResourceState::Compatible => {},
|
||||
crate::SchemaResourceState::Incompatible => {
|
||||
log_schema_resource_block(resource.id, "incompatible");
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(
|
||||
crate::PostgresBackendErrorKind::MigrationMismatch,
|
||||
"schema_resource_incompatible",
|
||||
));
|
||||
},
|
||||
crate::SchemaResourceState::Missing => {
|
||||
if !schema_autoupdate {
|
||||
log_schema_resource_block(resource.id, "schema_autoupdate_disabled");
|
||||
return std::result::Result::Err(schema_autoupdate_disabled_error());
|
||||
}
|
||||
let repair_result = ensure_resource(transaction, resource, SchemaMutationMode::Update, true).await;
|
||||
if let std::result::Result::Err(error) = repair_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
let contract_result = verify_migration_contract(transaction, migration.version).await;
|
||||
if let std::result::Result::Err(error) = contract_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
index += 1;
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn schema_autoupdate_disabled_error() -> crate::PostgresBackendError {
|
||||
return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "schema_autoupdate_disabled");
|
||||
}
|
||||
|
||||
fn validate_embedded_registry(migrations: &[EmbeddedMigration]) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
if migrations.is_empty() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "registry_empty"));
|
||||
}
|
||||
let mut expected_version = 0_i64;
|
||||
for migration in migrations {
|
||||
if migration.version != expected_version || migration.name.is_empty() || migration.resources.is_empty() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "registry_invalid"));
|
||||
}
|
||||
for (resource_index, resource) in migration.resources.iter().enumerate() {
|
||||
if resource.id.is_empty() || resource.sql.is_empty() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "registry_invalid"));
|
||||
}
|
||||
for previous in &migration.resources[..resource_index] {
|
||||
if previous.id == resource.id {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "registry_invalid"));
|
||||
}
|
||||
}
|
||||
}
|
||||
expected_version = match expected_version.checked_add(1) {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "registry_invalid"));
|
||||
},
|
||||
};
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_history(history: &[AppliedMigration], migrations: &[EmbeddedMigration]) -> std::result::Result<usize, crate::PostgresBackendError> {
|
||||
if history.is_empty() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_missing"));
|
||||
}
|
||||
let latest_version = migrations[migrations.len() - 1].version;
|
||||
let mut index = 0_usize;
|
||||
for applied in history {
|
||||
if applied.version > latest_version {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||
}
|
||||
if index >= migrations.len() {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::SchemaNewer, "history_newer"));
|
||||
}
|
||||
let expected = &migrations[index];
|
||||
let expected_checksum = migration_checksum(expected);
|
||||
if applied.version != expected.version || applied.name != expected.name || applied.checksum != expected_checksum {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::MigrationMismatch, "history_diverged"));
|
||||
}
|
||||
index += 1;
|
||||
}
|
||||
return std::result::Result::Ok(index);
|
||||
}
|
||||
|
||||
fn log_schema_block(phase: &'static str) {
|
||||
ksp_logging_lib::warn!(target: crate::TRACING_TARGET, phase, "PostgreSQL Store schema compatibility gate blocked automatic opening");
|
||||
}
|
||||
|
||||
fn log_schema_resource_block(resource_id: &'static str, reason: &'static str) {
|
||||
ksp_logging_lib::warn!(
|
||||
target: crate::TRACING_TARGET,
|
||||
resource_id,
|
||||
reason,
|
||||
"PostgreSQL Store schema resource requires manual reconciliation"
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[path = "../unit_tests/migration.rs"]
|
||||
mod tests;
|
||||
|
||||
1470
crates/ksp-store-postgres-lib/src/raw_transaction.rs
Normal file
1470
crates/ksp-store-postgres-lib/src/raw_transaction.rs
Normal file
File diff suppressed because it is too large
Load Diff
184
crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs
Normal file
184
crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs
Normal file
@@ -0,0 +1,184 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs
|
||||
// version: 1
|
||||
|
||||
use sha2::Digest; // rust-rules: trait-import
|
||||
|
||||
const CURSOR_BYTES: usize = 109;
|
||||
const CURSOR_DIGEST_OFFSET: usize = 77;
|
||||
const CURSOR_DOMAIN: &[u8] = b"KSP/raw-transaction-cursor/v1";
|
||||
const CURSOR_MAGIC: &[u8; 4] = b"KSPT";
|
||||
const CURSOR_VERSION: u8 = 1;
|
||||
const MAX_POSTGRES_PAGE_LIMIT: u64 = 9_223_372_036_854_775_806;
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
/// Decoded backend-private continuation key extracted from one validated RAW cursor.
|
||||
pub(crate) struct RawTransactionDecodedCursor {
|
||||
/// Last canonical transaction signature returned by the previous page.
|
||||
pub(crate) last_signature: [u8; 64],
|
||||
/// Last canonical transaction slot returned by the previous page.
|
||||
pub(crate) last_slot: u64,
|
||||
}
|
||||
|
||||
/// Decodes and validates one backend-private RAW transaction cursor against its query binding.
|
||||
pub(crate) fn decode_raw_transaction_cursor(
|
||||
query: &ksp_store_api::RawTransactionQuery,
|
||||
cursor: &ksp_store_api::RawPageCursor,
|
||||
) -> std::result::Result<RawTransactionDecodedCursor, crate::PostgresBackendError> {
|
||||
let bytes = cursor.as_bytes();
|
||||
if bytes.len() != CURSOR_BYTES {
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_size"));
|
||||
}
|
||||
if bytes.get(0..4) != std::option::Option::Some(CURSOR_MAGIC.as_ref()) {
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_magic"));
|
||||
}
|
||||
if bytes.get(4).copied() != std::option::Option::Some(CURSOR_VERSION) {
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_version"));
|
||||
}
|
||||
let slot_bytes = match bytes.get(5..13) {
|
||||
std::option::Option::Some(value) => match <[u8; 8]>::try_from(value) {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_slot")),
|
||||
},
|
||||
std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_slot")),
|
||||
};
|
||||
let last_slot = u64::from_be_bytes(slot_bytes);
|
||||
let last_signature = match bytes.get(13..CURSOR_DIGEST_OFFSET) {
|
||||
std::option::Option::Some(value) => match <[u8; 64]>::try_from(value) {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_signature")),
|
||||
},
|
||||
std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_signature")),
|
||||
};
|
||||
let range_result = validate_slot_in_range(query.slots(), last_slot);
|
||||
if let std::result::Result::Err(error) = range_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let expected_digest = match binding_digest(query, last_slot, &last_signature) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let stored_digest = match bytes.get(CURSOR_DIGEST_OFFSET..CURSOR_BYTES) {
|
||||
std::option::Option::Some(value) => match <[u8; 32]>::try_from(value) {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_digest")),
|
||||
},
|
||||
std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_digest")),
|
||||
};
|
||||
if stored_digest != expected_digest {
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_binding"));
|
||||
}
|
||||
return std::result::Result::Ok(RawTransactionDecodedCursor { last_signature, last_slot });
|
||||
}
|
||||
|
||||
/// Encodes one backend-private RAW transaction cursor bound to the supplied query context.
|
||||
pub(crate) fn encode_raw_transaction_cursor(
|
||||
query: &ksp_store_api::RawTransactionQuery,
|
||||
last_slot: u64,
|
||||
last_signature: &ksp_store_api::RawTransactionSignature,
|
||||
) -> std::result::Result<ksp_store_api::RawPageCursor, crate::PostgresBackendError> {
|
||||
let range_result = validate_slot_in_range(query.slots(), last_slot);
|
||||
if let std::result::Result::Err(error) = range_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let digest = match binding_digest(query, last_slot, last_signature.as_bytes()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let mut bytes = std::vec::Vec::with_capacity(CURSOR_BYTES);
|
||||
bytes.extend_from_slice(CURSOR_MAGIC);
|
||||
bytes.push(CURSOR_VERSION);
|
||||
bytes.extend_from_slice(&last_slot.to_be_bytes());
|
||||
bytes.extend_from_slice(last_signature.as_bytes());
|
||||
bytes.extend_from_slice(&digest);
|
||||
if bytes.len() != CURSOR_BYTES {
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_encode_size"));
|
||||
}
|
||||
return match ksp_store_api::RawPageCursor::try_new(bytes.into_boxed_slice()) {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(query_invalid("raw_cursor_encode")),
|
||||
};
|
||||
}
|
||||
|
||||
/// Converts one caller page size into the exact PostgreSQL LIMIT+1 representation.
|
||||
pub(crate) fn raw_transaction_physical_page_limit(requested: u64) -> std::result::Result<(usize, i64), crate::PostgresBackendError> {
|
||||
if requested > MAX_POSTGRES_PAGE_LIMIT {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit"));
|
||||
}
|
||||
let requested_usize = match usize::try_from(requested) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit"));
|
||||
},
|
||||
};
|
||||
let requested_plus_one = match requested.checked_add(1) {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit"));
|
||||
},
|
||||
};
|
||||
let sql_limit = match i64::try_from(requested_plus_one) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit"));
|
||||
},
|
||||
};
|
||||
return std::result::Result::Ok((requested_usize, sql_limit));
|
||||
}
|
||||
|
||||
fn binding_digest(
|
||||
query: &ksp_store_api::RawTransactionQuery,
|
||||
last_slot: u64,
|
||||
last_signature: &[u8; 64],
|
||||
) -> std::result::Result<[u8; 32], crate::PostgresBackendError> {
|
||||
let direction = match query.direction() {
|
||||
ksp_store_api::RawSortDirection::Ascending => 0_u8,
|
||||
ksp_store_api::RawSortDirection::Descending => 1_u8,
|
||||
_ => return std::result::Result::Err(query_invalid("raw_cursor_direction")),
|
||||
};
|
||||
let network_len = match u64::try_from(query.network().as_str().len()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_network")),
|
||||
};
|
||||
let mut hasher = sha2::Sha256::new();
|
||||
hasher.update(CURSOR_DOMAIN);
|
||||
hasher.update(network_len.to_be_bytes());
|
||||
hasher.update(query.network().as_str().as_bytes());
|
||||
hasher.update([direction]);
|
||||
hash_optional_slot(&mut hasher, query.slots().start_inclusive());
|
||||
hash_optional_slot(&mut hasher, query.slots().end_inclusive());
|
||||
hasher.update(last_slot.to_be_bytes());
|
||||
hasher.update(last_signature);
|
||||
let finalized = hasher.finalize();
|
||||
let mut digest = [0_u8; 32];
|
||||
digest.copy_from_slice(finalized.as_ref());
|
||||
return std::result::Result::Ok(digest);
|
||||
}
|
||||
|
||||
fn hash_optional_slot(hasher: &mut sha2::Sha256, value: std::option::Option<u64>) {
|
||||
match value {
|
||||
std::option::Option::Some(slot) => {
|
||||
hasher.update([1_u8]);
|
||||
hasher.update(slot.to_be_bytes());
|
||||
},
|
||||
std::option::Option::None => hasher.update([0_u8]),
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
fn validate_slot_in_range(range: ksp_store_api::RawSlotRange, slot: u64) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
if let std::option::Option::Some(start) = range.start_inclusive()
|
||||
&& slot < start
|
||||
{
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_slot_range"));
|
||||
}
|
||||
if let std::option::Option::Some(end) = range.end_inclusive()
|
||||
&& slot > end
|
||||
{
|
||||
return std::result::Result::Err(query_invalid("raw_cursor_slot_range"));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn query_invalid(phase: &'static str) -> crate::PostgresBackendError {
|
||||
return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::QueryInvalid, phase);
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
// version: 3
|
||||
// version: 10
|
||||
|
||||
const APPLICATION_NAME: &str = "ksp-store";
|
||||
const MAX_CONNECTION_URI_BYTES: usize = 4_096;
|
||||
@@ -143,18 +143,19 @@ pub struct PostgresBackendSettings {
|
||||
connect_timeout: std::time::Duration,
|
||||
connection_uri: std::string::String,
|
||||
create_timeout: std::time::Duration,
|
||||
auto_migrate: bool,
|
||||
max_connections: u32,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
migration_timeout: std::time::Duration,
|
||||
network: ksp_store_api::RawNetworkId,
|
||||
recycle_timeout: std::time::Duration,
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
tls_mode: PostgresBackendTlsMode,
|
||||
wait_timeout: std::time::Duration,
|
||||
}
|
||||
|
||||
impl PostgresBackendSettings {
|
||||
/// Creates the physical PostgreSQL settings bridge from already validated facade-owned values.
|
||||
/// Creates the physical PostgreSQL settings bridge using the legacy single migration switch.
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
network: ksp_store_api::RawNetworkId,
|
||||
@@ -169,8 +170,39 @@ impl PostgresBackendSettings {
|
||||
migration_timeout: std::time::Duration,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
) -> Self {
|
||||
return Self {
|
||||
return Self::with_schema_policy(
|
||||
network,
|
||||
connection_uri,
|
||||
max_connections,
|
||||
connect_timeout,
|
||||
wait_timeout,
|
||||
create_timeout,
|
||||
recycle_timeout,
|
||||
tls_mode,
|
||||
auto_migrate,
|
||||
auto_migrate,
|
||||
migration_timeout,
|
||||
migration_lock_timeout,
|
||||
);
|
||||
}
|
||||
|
||||
/// Creates the physical PostgreSQL settings bridge with independent schema creation and update policies.
|
||||
#[must_use]
|
||||
pub fn with_schema_policy(
|
||||
network: ksp_store_api::RawNetworkId,
|
||||
connection_uri: impl std::convert::Into<std::string::String>,
|
||||
max_connections: u32,
|
||||
connect_timeout: std::time::Duration,
|
||||
wait_timeout: std::time::Duration,
|
||||
create_timeout: std::time::Duration,
|
||||
recycle_timeout: std::time::Duration,
|
||||
tls_mode: PostgresBackendTlsMode,
|
||||
schema_autocreate: bool,
|
||||
schema_autoupdate: bool,
|
||||
migration_timeout: std::time::Duration,
|
||||
migration_lock_timeout: std::time::Duration,
|
||||
) -> Self {
|
||||
return Self {
|
||||
connect_timeout,
|
||||
connection_uri: connection_uri.into(),
|
||||
create_timeout,
|
||||
@@ -179,6 +211,8 @@ impl PostgresBackendSettings {
|
||||
migration_timeout,
|
||||
network,
|
||||
recycle_timeout,
|
||||
schema_autocreate,
|
||||
schema_autoupdate,
|
||||
tls_mode,
|
||||
wait_timeout,
|
||||
};
|
||||
@@ -203,7 +237,8 @@ impl std::fmt::Debug for PostgresBackendSettings {
|
||||
.debug_struct("PostgresBackendSettings")
|
||||
.field("network", &self.network)
|
||||
.field("connection_uri", &"<redacted>")
|
||||
.field("auto_migrate", &self.auto_migrate)
|
||||
.field("schema_autocreate", &self.schema_autocreate)
|
||||
.field("schema_autoupdate", &self.schema_autoupdate)
|
||||
.field("max_connections", &self.max_connections)
|
||||
.field("migration_timeout", &self.migration_timeout)
|
||||
.field("migration_lock_timeout", &self.migration_lock_timeout)
|
||||
@@ -263,7 +298,15 @@ impl PostgresBackend {
|
||||
tls_mode = settings.tls_mode().code(),
|
||||
"PostgreSQL Store backend established initial physical connection"
|
||||
);
|
||||
let bootstrap_result = crate::bootstrap(&mut client, settings.auto_migrate, settings.migration_timeout, settings.migration_lock_timeout).await;
|
||||
let bootstrap_result = crate::bootstrap(
|
||||
&mut client,
|
||||
settings.network(),
|
||||
settings.schema_autocreate,
|
||||
settings.schema_autoupdate,
|
||||
settings.migration_timeout,
|
||||
settings.migration_lock_timeout,
|
||||
)
|
||||
.await;
|
||||
if let std::result::Result::Err(error) = bootstrap_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
@@ -271,7 +314,8 @@ impl PostgresBackend {
|
||||
ksp_logging_lib::debug!(
|
||||
target: crate::TRACING_TARGET,
|
||||
network = settings.network().as_str(),
|
||||
auto_migrate = settings.auto_migrate,
|
||||
schema_autocreate = settings.schema_autocreate,
|
||||
schema_autoupdate = settings.schema_autoupdate,
|
||||
"PostgreSQL Store migration/bootstrap foundation verified"
|
||||
);
|
||||
return std::result::Result::Ok(Self { network: settings.network, pool });
|
||||
@@ -288,6 +332,72 @@ impl PostgresBackend {
|
||||
return crate::probe_health(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Reads one canonical RAW transaction without exposing physical PostgreSQL row types.
|
||||
pub async fn get_raw_transaction(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransaction>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Lists deterministic canonical RAW transaction references with a backend-owned opaque continuation cursor.
|
||||
pub async fn list_raw_transactions(
|
||||
&self,
|
||||
query: &ksp_store_api::RawTransactionQuery,
|
||||
) -> std::result::Result<ksp_store_api::RawPage<ksp_store_api::RawTransactionReference>, crate::PostgresBackendError> {
|
||||
return crate::list_raw_transactions(&self.pool, &self.network, query).await;
|
||||
}
|
||||
|
||||
/// Reads one persisted RAW transaction observation by producer-owned idempotence key.
|
||||
pub async fn get_raw_transaction_observation(
|
||||
&self,
|
||||
observation_key: &ksp_store_api::RawObservationKey,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionObservation>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_observation(&self.pool, &self.network, observation_key).await;
|
||||
}
|
||||
|
||||
/// Reads the retention state of one canonical RAW transaction identity.
|
||||
pub async fn get_raw_transaction_retention_state(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawRetentionState>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_retention_state(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Reads the minimal durable tombstone only when one RAW transaction is purged.
|
||||
pub async fn get_raw_transaction_tombstone(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_tombstone(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Persists one canonical RAW transaction and its acquisition observation atomically.
|
||||
pub async fn persist_raw_transaction_acquisition(
|
||||
&self,
|
||||
raw_transaction: ksp_store_api::RawTransaction,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
mode: ksp_store_api::RawTransactionAcquisitionMode,
|
||||
) -> std::result::Result<ksp_store_api::RawAcquisitionWriteOutcome, crate::PostgresBackendError> {
|
||||
return crate::persist_raw_transaction_acquisition(&self.pool, &self.network, raw_transaction, observation, mode).await;
|
||||
}
|
||||
|
||||
/// Persists one additional acquisition observation for an existing RAW transaction.
|
||||
pub async fn record_raw_transaction_observation(
|
||||
&self,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
) -> std::result::Result<ksp_store_api::RawObservationWriteOutcome, crate::PostgresBackendError> {
|
||||
return crate::record_raw_transaction_observation(&self.pool, &self.network, observation).await;
|
||||
}
|
||||
|
||||
/// Applies one policy-authorized atomic RAW transaction retention transition.
|
||||
pub async fn transition_raw_transaction_retention(
|
||||
&self,
|
||||
transition: ksp_store_api::RawTransactionRetentionTransition,
|
||||
) -> std::result::Result<ksp_store_api::RawRetentionWriteOutcome, crate::PostgresBackendError> {
|
||||
return crate::transition_raw_transaction_retention(&self.pool, &self.network, transition).await;
|
||||
}
|
||||
|
||||
/// Explicitly closes the pool and waits for all owned pooled objects to drain inside the supplied bound.
|
||||
pub async fn close(self, timeout: std::time::Duration) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
self.pool.close();
|
||||
@@ -318,6 +428,126 @@ impl std::fmt::Debug for PostgresBackend {
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRead for PostgresBackend {
|
||||
fn get_raw_transaction<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransaction>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::get_raw_transaction(self, reference).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
|
||||
fn list_raw_transactions<'a>(
|
||||
&'a self,
|
||||
query: &'a ksp_store_api::RawTransactionQuery,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawPage<ksp_store_api::RawTransactionReference>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::list_raw_transactions(self, query).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionWrite for PostgresBackend {
|
||||
fn persist_raw_transaction_acquisition<'a>(
|
||||
&'a self,
|
||||
transaction: ksp_store_api::RawTransaction,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
mode: ksp_store_api::RawTransactionAcquisitionMode,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawAcquisitionWriteOutcome>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::persist_raw_transaction_acquisition(self, transaction, observation, mode).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionObservationRead for PostgresBackend {
|
||||
fn get_raw_transaction_observation<'a>(
|
||||
&'a self,
|
||||
observation_key: &'a ksp_store_api::RawObservationKey,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransactionObservation>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::get_raw_transaction_observation(self, observation_key).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionObservationWrite for PostgresBackend {
|
||||
fn record_raw_transaction_observation<'a>(
|
||||
&'a self,
|
||||
observation: ksp_store_api::RawTransactionObservation,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawObservationWriteOutcome>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::record_raw_transaction_observation(self, observation).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRetentionRead for PostgresBackend {
|
||||
fn get_raw_transaction_retention_state<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawRetentionState>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::get_raw_transaction_retention_state(self, reference).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
|
||||
fn get_raw_transaction_tombstone<'a>(
|
||||
&'a self,
|
||||
reference: &'a ksp_store_api::RawTransactionReference,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::get_raw_transaction_tombstone(self, reference).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
impl ksp_store_api::RawTransactionRetentionWrite for PostgresBackend {
|
||||
fn transition_raw_transaction_retention<'a>(
|
||||
&'a self,
|
||||
transition: ksp_store_api::RawTransactionRetentionTransition,
|
||||
) -> ksp_store_api::StoreApiFuture<'a, ksp_store_api::Result<ksp_store_api::RawRetentionWriteOutcome>> {
|
||||
return std::boxed::Box::pin(async move {
|
||||
let result = PostgresBackend::transition_raw_transaction_retention(self, transition).await;
|
||||
return result.map_err(map_capability_error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn map_capability_error(error: crate::PostgresBackendError) -> ksp_store_api::Error {
|
||||
let code = match error.kind() {
|
||||
crate::PostgresBackendErrorKind::ConfigInvalid => ksp_store_api::ErrorCode::new("store", "postgres_config_invalid"),
|
||||
crate::PostgresBackendErrorKind::ConnectFailed => ksp_store_api::ErrorCode::new("store", "postgres_connect_failed"),
|
||||
crate::PostgresBackendErrorKind::Conflict => ksp_store_api::ERROR_CODE_RAW_CONFLICT,
|
||||
crate::PostgresBackendErrorKind::DataInvalid => ksp_store_api::ErrorCode::new("store", "postgres_data_invalid"),
|
||||
crate::PostgresBackendErrorKind::HealthFailed => ksp_store_api::ErrorCode::new("store", "postgres_health_failed"),
|
||||
crate::PostgresBackendErrorKind::MigrationFailed => ksp_store_api::ErrorCode::new("store", "postgres_migration_failed"),
|
||||
crate::PostgresBackendErrorKind::MigrationMismatch => ksp_store_api::ErrorCode::new("store", "postgres_migration_mismatch"),
|
||||
crate::PostgresBackendErrorKind::PageLimitUnsupported => ksp_store_api::ErrorCode::new("store", "postgres_page_limit_unsupported"),
|
||||
crate::PostgresBackendErrorKind::PoolTimeout => ksp_store_api::ErrorCode::new("store", "postgres_pool_timeout"),
|
||||
crate::PostgresBackendErrorKind::QueryInvalid => ksp_store_api::ERROR_CODE_RAW_QUERY_INVALID,
|
||||
crate::PostgresBackendErrorKind::ReadFailed => ksp_store_api::ErrorCode::new("store", "postgres_read_failed"),
|
||||
crate::PostgresBackendErrorKind::ReferenceNotFound => ksp_store_api::ErrorCode::new("store", "raw_reference_not_found"),
|
||||
crate::PostgresBackendErrorKind::RetentionCompactionUnsupported => crate::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED,
|
||||
crate::PostgresBackendErrorKind::SchemaNewer => ksp_store_api::ErrorCode::new("store", "postgres_schema_newer"),
|
||||
crate::PostgresBackendErrorKind::ShutdownTimeout => ksp_store_api::ErrorCode::new("store", "shutdown_timeout"),
|
||||
crate::PostgresBackendErrorKind::TlsFailed => ksp_store_api::ErrorCode::new("store", "postgres_tls_failed"),
|
||||
crate::PostgresBackendErrorKind::WriteFailed => ksp_store_api::ErrorCode::new("store", "postgres_write_failed"),
|
||||
crate::PostgresBackendErrorKind::WrongNetwork => ksp_store_api::ErrorCode::new("store", "wrong_network"),
|
||||
};
|
||||
return ksp_store_api::Error::new(code, "PostgreSQL Store capability operation failed")
|
||||
.with_context("backend", "postgres")
|
||||
.with_context("phase", error.phase());
|
||||
}
|
||||
|
||||
impl std::ops::Drop for PostgresBackend {
|
||||
fn drop(&mut self) {
|
||||
self.pool.close();
|
||||
|
||||
1217
crates/ksp-store-postgres-lib/src/schema.rs
Normal file
1217
crates/ksp-store-postgres-lib/src/schema.rs
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
// version: 5
|
||||
// version: 14
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -19,12 +19,15 @@ fn pre_005_backend_owns_exact_physical_runtime_dependencies_without_reverse_faca
|
||||
assert!(!manifest.contains(forbidden), "forbidden PostgreSQL backend dependency detected: {forbidden}");
|
||||
}
|
||||
let migration = include_str!("../src/migration.rs");
|
||||
let bootstrap_sql = include_str!("../migrations/V000__bootstrap.sql");
|
||||
assert!(migration.contains("include_str!(\"../migrations/V000__bootstrap.sql\")"));
|
||||
let schema = include_str!("../src/schema.rs");
|
||||
let bootstrap_sql = include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql");
|
||||
assert!(migration.contains("crate::V000_RESOURCES"));
|
||||
assert!(migration.contains("crate::V001_RESOURCES"));
|
||||
assert!(schema.contains("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql"));
|
||||
assert!(schema.contains("../migrations/v001_raw_transaction/tables/001_ksp_store_identity.sql"));
|
||||
assert!(bootstrap_sql.contains("ksp_store_schema_migrations"));
|
||||
for forbidden in ["RawTransaction", "RawAccountState", "raw_transaction", "raw_account", "CORE", "DECODE", "SPECIALIZED"] {
|
||||
assert!(!migration.contains(forbidden), "business migration implementation leaked into foundation: {forbidden}");
|
||||
assert!(!bootstrap_sql.contains(forbidden), "business schema leaked into foundation SQL: {forbidden}");
|
||||
assert!(!bootstrap_sql.contains(forbidden), "business schema leaked into immutable V000 SQL: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
@@ -35,7 +38,9 @@ fn pre_005_backend_keeps_environment_sql_migrations_and_physical_types_private()
|
||||
assert!(crate_root.contains("mod error;"));
|
||||
assert!(crate_root.contains("mod health;"));
|
||||
assert!(crate_root.contains("mod migration;"));
|
||||
assert!(crate_root.contains("mod raw_transaction;"));
|
||||
assert!(crate_root.contains("mod runtime;"));
|
||||
assert!(crate_root.contains("mod schema;"));
|
||||
assert!(crate_root.contains("const _: &str = crate::TRACING_TARGET;"));
|
||||
for forbidden in [
|
||||
"pub mod ",
|
||||
@@ -82,3 +87,188 @@ fn pre_007_health_probe_remains_foundation_only_and_private_sql() {
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_migration_engine_uses_split_schema_contract_and_binds_network_without_repository_scope() {
|
||||
let migration = include_str!("../src/migration.rs");
|
||||
let schema = include_str!("../src/schema.rs");
|
||||
let identity_table = include_str!("../migrations/v001_raw_transaction/tables/001_ksp_store_identity.sql");
|
||||
let raw_table = include_str!("../migrations/v001_raw_transaction/tables/002_ksp_raw_transactions.sql");
|
||||
let observation_table = include_str!("../migrations/v001_raw_transaction/tables/003_ksp_raw_transaction_observations.sql");
|
||||
let archive_table = include_str!("../migrations/v001_raw_transaction/tables/004_ksp_raw_transaction_archive_payloads.sql");
|
||||
let index = include_str!("../migrations/v001_raw_transaction/indexes/001_ix_ksp_raw_transactions_slot_signature.sql");
|
||||
assert!(migration.contains("const EMBEDDED_MIGRATIONS: &[EmbeddedMigration]"));
|
||||
assert!(migration.contains("MigrationHook::StoreIdentity"));
|
||||
assert!(migration.contains("MigrationHookContext::AppliedNow"));
|
||||
assert!(migration.contains("MigrationHookContext::Existing"));
|
||||
assert!(migration.contains("schema_autocreate"));
|
||||
assert!(migration.contains("schema_autoupdate"));
|
||||
assert!(migration.contains("INSERT INTO ksp_store_identity (singleton, network) VALUES (1, $1)"));
|
||||
assert!(migration.contains("SELECT singleton, network FROM ksp_store_identity ORDER BY singleton LIMIT 2"));
|
||||
assert!(migration.contains("ksp_store_api::RawNetworkId::new(stored_network)"));
|
||||
assert!(schema.contains("SchemaResourceState"));
|
||||
assert!(schema.contains("verify_v001_external_compatibility"));
|
||||
assert!(identity_table.contains("CREATE TABLE IF NOT EXISTS ksp_store_identity"));
|
||||
assert!(raw_table.contains("CREATE TABLE IF NOT EXISTS ksp_raw_transactions"));
|
||||
assert!(observation_table.contains("CREATE TABLE IF NOT EXISTS ksp_raw_transaction_observations"));
|
||||
assert!(archive_table.contains("CREATE TABLE IF NOT EXISTS ksp_raw_transaction_archive_payloads"));
|
||||
assert!(index.contains("CREATE INDEX IF NOT EXISTS ix_ksp_raw_transactions_slot_signature"));
|
||||
for forbidden in ["impl ksp_store_api::RawTransaction", "repository", "sqlx", "RawAccountState"] {
|
||||
assert!(!migration.contains(forbidden), "repository/cross-scope implementation leaked into migration engine: {forbidden}");
|
||||
assert!(!schema.contains(forbidden), "repository/cross-scope implementation leaked into schema contract: {forbidden}");
|
||||
}
|
||||
for removed in ["migrations/V000__bootstrap.sql", "migrations/V001__raw_transaction.sql"] {
|
||||
let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(removed);
|
||||
assert!(!path.exists(), "obsolete monolithic migration must be deleted by pre.003-fix.001: {removed}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_raw_read_sql_and_mapping_remain_backend_private() {
|
||||
let crate_root = include_str!("../src/lib.rs");
|
||||
let raw = include_str!("../src/raw_transaction.rs");
|
||||
assert!(crate_root.contains("mod raw_transaction;"));
|
||||
assert!(!crate_root.contains("pub mod raw_transaction"));
|
||||
for required in [
|
||||
"SELECT transaction_row.signature",
|
||||
"ksp_raw_transaction_observations",
|
||||
"slot::text AS slot_text",
|
||||
"RawPayload::try_new",
|
||||
"RawTransactionTombstone::try_new",
|
||||
"PostgresBackendErrorKind::DataInvalid",
|
||||
"PostgresBackendErrorKind::WrongNetwork",
|
||||
] {
|
||||
assert!(raw.contains(required), "missing private RAW read mapping contract: {required}");
|
||||
}
|
||||
for forbidden in ["std::env", "dotenv", "ksp_store_lib", "ksp_config_lib"] {
|
||||
assert!(!raw.contains(forbidden), "RAW module contains forbidden ownership material: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_raw_write_sql_is_atomic_idempotent_and_keeps_direct_trait_scope_closed() {
|
||||
let raw = include_str!("../src/raw_transaction.rs");
|
||||
for required in [
|
||||
"INSERT INTO ksp_raw_transactions",
|
||||
"ON CONFLICT (signature) DO NOTHING RETURNING signature",
|
||||
"FOR UPDATE",
|
||||
"INSERT INTO ksp_raw_transaction_observations",
|
||||
"ON CONFLICT (observation_key) DO NOTHING RETURNING observation_key",
|
||||
"REHYDRATE_TRANSACTION_SQL",
|
||||
"RawEntityWriteOutcome::SkippedPurged",
|
||||
"RawEntityWriteOutcome::Rehydrated",
|
||||
"RawObservationWriteOutcome::NotRecorded",
|
||||
"PostgresBackendErrorKind::Conflict",
|
||||
"PostgresBackendErrorKind::ReferenceNotFound",
|
||||
"PostgresBackendErrorKind::WriteFailed",
|
||||
] {
|
||||
assert!(raw.contains(required), "missing pre.005 RAW write contract: {required}");
|
||||
}
|
||||
for forbidden in ["impl ksp_store_api::RawTransactionWrite", "impl ksp_store_api::RawTransactionObservationWrite"] {
|
||||
assert!(!raw.contains(forbidden), "pre.005 opened direct Store trait scope prematurely: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_raw_pagination_is_keyset_cursor_bound_and_policy_free() {
|
||||
let raw = include_str!("../src/raw_transaction.rs");
|
||||
let cursor = include_str!("../src/raw_transaction/cursor.rs");
|
||||
let index = include_str!("../migrations/v001_raw_transaction/indexes/001_ix_ksp_raw_transactions_slot_signature.sql");
|
||||
for required in [
|
||||
"LIST_TRANSACTIONS_ASC_SQL",
|
||||
"LIST_TRANSACTIONS_DESC_SQL",
|
||||
"retention_state <> 'purged'",
|
||||
"(slot, signature) >",
|
||||
"(slot, signature) <",
|
||||
"ORDER BY slot ASC, signature ASC",
|
||||
"ORDER BY slot DESC, signature DESC",
|
||||
"LIMIT $5",
|
||||
"list_raw_transactions",
|
||||
] {
|
||||
assert!(raw.contains(required), "missing pre.006 keyset pagination contract: {required}");
|
||||
}
|
||||
for required in [
|
||||
"CURSOR_BYTES: usize = 109",
|
||||
"CURSOR_MAGIC",
|
||||
"b\"KSPT\"",
|
||||
"CURSOR_VERSION: u8 = 1",
|
||||
"KSP/raw-transaction-cursor/v1",
|
||||
"sha2::Sha256",
|
||||
"query.network().as_str()",
|
||||
"query.direction()",
|
||||
"query.slots().start_inclusive()",
|
||||
"query.slots().end_inclusive()",
|
||||
"last_slot",
|
||||
"last_signature",
|
||||
"PageLimitUnsupported",
|
||||
"9_223_372_036_854_775_806",
|
||||
] {
|
||||
assert!(cursor.contains(required), "missing pre.006 cursor/binding contract: {required}");
|
||||
}
|
||||
assert!(index.contains("ON ksp_raw_transactions (slot, signature)"));
|
||||
assert!(index.contains("WHERE retention_state <> 'purged'"));
|
||||
for forbidden in [" OFFSET ", "limit.min(", "clamp(", "500", "1000"] {
|
||||
assert!(!raw.contains(forbidden), "pre.006 contains forbidden pagination/policy/later-scope material: {forbidden}");
|
||||
assert!(!cursor.contains(forbidden), "pre.006 cursor contains forbidden pagination/policy/later-scope material: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_raw_retention_is_atomic_compare_and_transition_without_fake_compaction() {
|
||||
let raw = include_str!("../src/raw_transaction.rs");
|
||||
let runtime = include_str!("../src/runtime.rs");
|
||||
for required in [
|
||||
"LOCK_RETENTION_TRANSACTION_SQL",
|
||||
"FOR UPDATE",
|
||||
"INSERT_ARCHIVE_PAYLOAD_SQL",
|
||||
"INSERT INTO ksp_raw_transaction_archive_payloads (signature, payload)",
|
||||
"UPDATE_ARCHIVED_TRANSACTION_SQL",
|
||||
"SET payload = NULL, retention_state = 'archived'",
|
||||
"DELETE_ARCHIVE_PAYLOAD_SQL",
|
||||
"DELETE FROM ksp_raw_transaction_archive_payloads",
|
||||
"UPDATE_PURGED_TRANSACTION_SQL",
|
||||
"SET block_time_unix_millis = NULL, payload = NULL, retention_state = 'purged'",
|
||||
"if current == target",
|
||||
"if current != expected",
|
||||
"RawRetentionWriteOutcome::AlreadyAtTarget",
|
||||
"RawRetentionWriteOutcome::ExpectedStateMismatch",
|
||||
"RawRetentionWriteOutcome::Applied",
|
||||
"RetentionCompactionUnsupported",
|
||||
"transition_raw_transaction_retention",
|
||||
] {
|
||||
assert!(raw.contains(required), "missing pre.007 retention contract: {required}");
|
||||
}
|
||||
assert!(runtime.contains("pub async fn transition_raw_transaction_retention"));
|
||||
for forbidden in ["retention_state = 'compacted'", "impl ksp_store_api::RawTransactionRetentionWrite", "flate", "zstd", "lz4", "snappy"] {
|
||||
assert!(!raw.contains(forbidden), "pre.007 contains fake compaction/direct-trait scope: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_backend_trait_implementations_stay_in_runtime_bridge_and_raw_account_scope_stays_closed() {
|
||||
let runtime = include_str!("../src/runtime.rs");
|
||||
for required in [
|
||||
"impl ksp_store_api::RawTransactionRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionObservationRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionObservationWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionRetentionRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionRetentionWrite for PostgresBackend",
|
||||
] {
|
||||
assert!(runtime.contains(required), "missing pre.008 PostgreSQL capability implementation: {required}");
|
||||
}
|
||||
for forbidden in [
|
||||
"impl ksp_store_api::RawAccountStateRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountStateWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountObservationRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountObservationWrite for PostgresBackend",
|
||||
] {
|
||||
assert!(!runtime.contains(forbidden), "pre.008 opened RawAccount capability scope prematurely: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
// version: 1
|
||||
// version: 11
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -109,12 +109,13 @@ fn assert_pre_io_rejection(connection_uri: &str, tls_mode: ksp_store_postgres_li
|
||||
#[test]
|
||||
fn pre_009_backend_modules_exports_and_manifest_dependencies_are_exact() {
|
||||
let crate_root = include_str!("../src/lib.rs");
|
||||
for required in ["mod constants;", "mod error;", "mod health;", "mod migration;", "mod runtime;"] {
|
||||
for required in ["mod constants;", "mod error;", "mod health;", "mod migration;", "mod raw_transaction;", "mod runtime;", "mod schema;"] {
|
||||
assert!(crate_root.contains(required), "missing PostgreSQL backend module: {required}");
|
||||
}
|
||||
assert!(!crate_root.contains("pub mod "));
|
||||
let actual_exports = public_reexport_names(crate_root);
|
||||
let mut expected_exports = [
|
||||
"ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED",
|
||||
"PostgresBackend",
|
||||
"PostgresBackendError",
|
||||
"PostgresBackendErrorKind",
|
||||
@@ -125,7 +126,7 @@ fn pre_009_backend_modules_exports_and_manifest_dependencies_are_exact() {
|
||||
];
|
||||
expected_exports.sort_unstable();
|
||||
assert_eq!(actual_exports.as_slice(), expected_exports.as_slice());
|
||||
assert_eq!(actual_exports.len(), 7);
|
||||
assert_eq!(actual_exports.len(), 8);
|
||||
let manifest = include_str!("../Cargo.toml");
|
||||
let actual_dependencies = manifest_dependency_names(manifest);
|
||||
let expected_dependencies = [
|
||||
@@ -167,7 +168,13 @@ fn pre_009_backend_error_bridge_cannot_retain_external_error_or_secret_text() {
|
||||
let runtime = include_str!("../src/runtime.rs");
|
||||
assert!(runtime.contains("deadpool_postgres::PoolError::Backend(_)"));
|
||||
assert!(!runtime.contains("deadpool_postgres::PoolError::Backend(error)"));
|
||||
for source in [runtime, include_str!("../src/migration.rs"), include_str!("../src/health.rs")] {
|
||||
for source in [
|
||||
runtime,
|
||||
include_str!("../src/migration.rs"),
|
||||
include_str!("../src/health.rs"),
|
||||
include_str!("../src/raw_transaction.rs"),
|
||||
include_str!("../src/raw_transaction/cursor.rs"),
|
||||
] {
|
||||
for forbidden in ["format!(\"{error", "format!(\"{error:?", "error = ?", "error = %"] {
|
||||
assert!(!source.contains(forbidden), "backend source renders external error material: {forbidden}");
|
||||
}
|
||||
@@ -176,18 +183,24 @@ fn pre_009_backend_error_bridge_cannot_retain_external_error_or_secret_text() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_009_backend_has_no_env_bypass_or_business_persistence_capability() {
|
||||
fn pre_009_backend_has_no_env_bypass_reverse_facade_edge_or_raw_account_trait_implementation() {
|
||||
let production = std::format!(
|
||||
"{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}",
|
||||
include_str!("../src/error.rs"),
|
||||
include_str!("../src/health.rs"),
|
||||
include_str!("../src/lib.rs"),
|
||||
include_str!("../src/migration.rs"),
|
||||
include_str!("../src/runtime.rs")
|
||||
include_str!("../src/raw_transaction.rs"),
|
||||
include_str!("../src/raw_transaction/cursor.rs"),
|
||||
include_str!("../src/runtime.rs"),
|
||||
include_str!("../src/schema.rs")
|
||||
);
|
||||
for forbidden in [
|
||||
"std::env",
|
||||
@@ -206,15 +219,86 @@ fn pre_009_backend_has_no_env_bypass_or_business_persistence_capability() {
|
||||
"ksp_store_lib",
|
||||
"ksp_config_lib",
|
||||
"sqlx::",
|
||||
"impl ksp_store_api::RawTransaction",
|
||||
"impl ksp_store_api::RawAccount",
|
||||
] {
|
||||
assert!(!production.contains(forbidden), "forbidden backend ownership/capability material detected: {forbidden}");
|
||||
assert!(!production.contains(forbidden), "forbidden backend ownership/reverse-edge/RawAccount material detected: {forbidden}");
|
||||
}
|
||||
let bootstrap_sql = include_str!("../migrations/V000__bootstrap.sql");
|
||||
let bootstrap_sql = include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql");
|
||||
assert!(bootstrap_sql.contains("ksp_store_schema_migrations"));
|
||||
for forbidden in ["RawTransaction", "RawAccountState", "raw_transaction", "raw_account", "CORE", "DECODE", "SPECIALIZED"] {
|
||||
assert!(!bootstrap_sql.contains(forbidden), "business schema leaked into foundation migration: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_009_live_raw_transaction_proof_is_opt_in_isolated_and_secret_safe() {
|
||||
let live = include_str!("postgres_raw_transaction_live.rs");
|
||||
for required in [
|
||||
"#[ignore = \"opt-in real PostgreSQL RawTransaction proof; reads one dedicated URI from stdin\"]",
|
||||
"std::io::stdin().read_line",
|
||||
"managed_schema_preexisting_refusal",
|
||||
"prove_schema_update_policy",
|
||||
"prove_concurrent_identical_insert",
|
||||
"prove_concurrent_divergent_insert",
|
||||
"prove_pagination",
|
||||
"prove_retention_and_rehydrate",
|
||||
"prove_retention_races",
|
||||
"prove_cancellation_rollback",
|
||||
"task.abort()",
|
||||
"cleanup_verification",
|
||||
] {
|
||||
assert!(live.contains(required), "missing pre.009 live proof guard/scenario: {required}");
|
||||
}
|
||||
for forbidden in ["std::env", "KSP_SECRET_", "PGPASSWORD", "connection_uri = %", "connection_uri = ?", "println!(uri", "eprintln!(uri"] {
|
||||
assert!(!live.contains(forbidden), "pre.009 live proof contains forbidden secret/environment material: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_010_raw_transaction_capability_implementation_inventory_is_exact_and_raw_account_scope_stays_closed() {
|
||||
let runtime = include_str!("../src/runtime.rs");
|
||||
let capability_impls = [
|
||||
"impl ksp_store_api::RawTransactionRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionObservationRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionObservationWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionRetentionRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawTransactionRetentionWrite for PostgresBackend",
|
||||
];
|
||||
for implementation in capability_impls {
|
||||
assert_eq!(runtime.matches(implementation).count(), 1, "unexpected PostgreSQL capability implementation inventory: {implementation}");
|
||||
}
|
||||
for forbidden in [
|
||||
"impl ksp_store_api::RawAccountStateRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountStateWrite for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountObservationRead for PostgresBackend",
|
||||
"impl ksp_store_api::RawAccountObservationWrite for PostgresBackend",
|
||||
] {
|
||||
assert!(!runtime.contains(forbidden), "RawAccountState scope opened during RawTransaction hardening: {forbidden}");
|
||||
}
|
||||
let migration = include_str!("../src/migration.rs");
|
||||
assert!(!migration.contains("ksp_raw_account"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_010_raw_transaction_private_sql_keeps_keyset_navigation_and_bounded_statement_surface() {
|
||||
let source = include_str!("../src/raw_transaction.rs");
|
||||
for required in [
|
||||
"ORDER BY slot ASC, signature ASC",
|
||||
"ORDER BY slot DESC, signature DESC",
|
||||
"LIMIT $5",
|
||||
"FOR UPDATE",
|
||||
"ON CONFLICT (signature) DO NOTHING",
|
||||
"ON CONFLICT (observation_key) DO NOTHING",
|
||||
"ksp_raw_transaction_archive_payloads",
|
||||
] {
|
||||
assert!(source.contains(required), "required hardened RawTransaction SQL contract missing: {required}");
|
||||
}
|
||||
for forbidden in [" OFFSET ", "SELECT *", "ON CONFLICT DO UPDATE", "processing_state", "batch_size", "priority"] {
|
||||
assert!(!source.contains(forbidden), "forbidden RawTransaction scope/policy SQL detected: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/postgres_foundation_live.rs
|
||||
// version: 1
|
||||
// version: 3
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -8,12 +8,30 @@
|
||||
//! Opt-in real PostgreSQL proof for the Store foundation runtime.
|
||||
//!
|
||||
//! The test reads one dedicated PostgreSQL URI from stdin, refuses to start
|
||||
//! when the KSP migration metadata table already exists, never prints the URI,
|
||||
//! creates no business table and cleans up only metadata it proved it created.
|
||||
//! when any KSP Store table managed by V000/V001 already exists, never prints
|
||||
//! the URI, and cleans up only the isolated schema it proved absent first. It
|
||||
//! validates migration/bootstrap behavior, not RawTransaction capabilities.
|
||||
|
||||
const LIVE_BOOTSTRAP_SQL: &str = include_str!("../migrations/V000__bootstrap.sql");
|
||||
const LIVE_BOOTSTRAP_SQL: &str = include_str!("../migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql");
|
||||
const LIVE_BROKEN_CHECKSUM_A: &str = "0000000000000000000000000000000000000000000000000000000000000000";
|
||||
const LIVE_BROKEN_CHECKSUM_B: &str = "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff";
|
||||
const LIVE_MANAGED_SCHEMA_DROP_SQL: &str = r#"DROP TABLE IF EXISTS ksp_raw_transaction_observations;
|
||||
DROP TABLE IF EXISTS ksp_raw_transaction_archive_payloads;
|
||||
DROP TABLE IF EXISTS ksp_raw_transactions;
|
||||
DROP TABLE IF EXISTS ksp_store_identity;
|
||||
DROP TABLE IF EXISTS ksp_store_schema_migrations;"#;
|
||||
const LIVE_MANAGED_SCHEMA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||
SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema()
|
||||
AND table_name IN (
|
||||
'ksp_store_schema_migrations',
|
||||
'ksp_store_identity',
|
||||
'ksp_raw_transactions',
|
||||
'ksp_raw_transaction_observations',
|
||||
'ksp_raw_transaction_archive_payloads'
|
||||
)
|
||||
AND table_type = 'BASE TABLE'
|
||||
)"#;
|
||||
const LIVE_MAX_URI_BYTES: usize = 4_096;
|
||||
const LIVE_METADATA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||
SELECT 1 FROM information_schema.tables
|
||||
@@ -21,7 +39,6 @@ const LIVE_METADATA_EXISTS_SQL: &str = r#"SELECT EXISTS (
|
||||
AND table_name = 'ksp_store_schema_migrations'
|
||||
AND table_type = 'BASE TABLE'
|
||||
)"#;
|
||||
const LIVE_METADATA_DROP_SQL: &str = "DROP TABLE IF EXISTS ksp_store_schema_migrations";
|
||||
const LIVE_SENTINEL_CHECKSUM_SQL: &str = "SELECT checksum FROM ksp_store_schema_migrations WHERE version = 0";
|
||||
const LIVE_SENTINEL_INSERT_SQL: &str =
|
||||
"INSERT INTO ksp_store_schema_migrations (version, name, checksum, applied_at) VALUES (0, 'bootstrap', 'pre008_rollback_injected', CURRENT_TIMESTAMP)";
|
||||
@@ -83,13 +100,13 @@ async fn run_live_test(uri: &str) -> std::result::Result<(), LiveFailure> {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let preexisting_result = metadata_exists(&admin).await;
|
||||
let preexisting_result = managed_schema_exists(&admin).await;
|
||||
let preexisting = match preexisting_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
if preexisting {
|
||||
return std::result::Result::Err(LiveFailure::new("metadata_preexisting_refusal"));
|
||||
return std::result::Result::Err(LiveFailure::new("managed_schema_preexisting_refusal"));
|
||||
}
|
||||
let major_result = postgres_major(&admin).await;
|
||||
let major = match major_result {
|
||||
@@ -100,16 +117,16 @@ async fn run_live_test(uri: &str) -> std::result::Result<(), LiveFailure> {
|
||||
return std::result::Result::Err(LiveFailure::new("postgres_major_unsupported"));
|
||||
}
|
||||
eprintln!("KSP Store PostgreSQL live proof: server major {major}");
|
||||
let mut owns_metadata = false;
|
||||
let scenario = run_foundation_scenario(&mut admin, uri, &mut owns_metadata).await;
|
||||
let cleanup = if owns_metadata { drop_metadata(&admin).await } else { std::result::Result::Ok(()) };
|
||||
let mut owns_schema = false;
|
||||
let scenario = run_foundation_scenario(&mut admin, uri, &mut owns_schema).await;
|
||||
let cleanup = if owns_schema { drop_managed_schema(&admin).await } else { std::result::Result::Ok(()) };
|
||||
if let std::result::Result::Err(error) = cleanup {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
if let std::result::Result::Err(error) = scenario {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let remains_result = metadata_exists(&admin).await;
|
||||
let remains_result = managed_schema_exists(&admin).await;
|
||||
let remains = match remains_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
@@ -120,7 +137,7 @@ async fn run_live_test(uri: &str) -> std::result::Result<(), LiveFailure> {
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str, owns_metadata: &mut bool) -> std::result::Result<(), LiveFailure> {
|
||||
async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str, owns_schema: &mut bool) -> std::result::Result<(), LiveFailure> {
|
||||
let initial_result = open_backend(uri).await;
|
||||
let initial = match initial_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
@@ -134,9 +151,9 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
||||
if !created {
|
||||
return std::result::Result::Err(LiveFailure::new("initial_bootstrap_metadata"));
|
||||
}
|
||||
*owns_metadata = true;
|
||||
*owns_schema = true;
|
||||
let initial_health = initial.health().await;
|
||||
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(0) || initial_health.pending_migration_count() != 0 {
|
||||
if !initial_health.is_ready() || initial_health.migration_version() != std::option::Option::Some(1) || initial_health.pending_migration_count() != 0 {
|
||||
return std::result::Result::Err(LiveFailure::new("initial_health"));
|
||||
}
|
||||
let initial_close = close_backend(initial).await;
|
||||
@@ -150,7 +167,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
||||
};
|
||||
let idempotent_health = idempotent.health().await;
|
||||
if !idempotent_health.is_ready()
|
||||
|| idempotent_health.migration_version() != std::option::Option::Some(0)
|
||||
|| idempotent_health.migration_version() != std::option::Option::Some(1)
|
||||
|| idempotent_health.pending_migration_count() != 0
|
||||
{
|
||||
return std::result::Result::Err(LiveFailure::new("idempotent_health"));
|
||||
@@ -159,7 +176,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
||||
if let std::result::Result::Err(error) = idempotent_close {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let reset_result = drop_metadata(admin).await;
|
||||
let reset_result = drop_managed_schema(admin).await;
|
||||
if let std::result::Result::Err(error) = reset_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
@@ -213,7 +230,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
||||
if let std::result::Result::Err(error) = recovered_close {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let rollback_reset = drop_metadata(admin).await;
|
||||
let rollback_reset = drop_managed_schema(admin).await;
|
||||
if let std::result::Result::Err(error) = rollback_reset {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
@@ -233,7 +250,7 @@ async fn run_foundation_scenario(admin: &mut tokio_postgres::Client, uri: &str,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let final_health = final_backend.health().await;
|
||||
if !final_health.is_ready() || final_health.migration_version() != std::option::Option::Some(0) || final_health.pending_migration_count() != 0 {
|
||||
if !final_health.is_ready() || final_health.migration_version() != std::option::Option::Some(1) || final_health.pending_migration_count() != 0 {
|
||||
return std::result::Result::Err(LiveFailure::new("final_health"));
|
||||
}
|
||||
return close_backend(final_backend).await;
|
||||
@@ -354,6 +371,18 @@ async fn postgres_major(client: &tokio_postgres::Client) -> std::result::Result<
|
||||
return std::result::Result::Ok(version_num / 10_000);
|
||||
}
|
||||
|
||||
async fn managed_schema_exists(client: &tokio_postgres::Client) -> std::result::Result<bool, LiveFailure> {
|
||||
let row_result = client.query_one(LIVE_MANAGED_SCHEMA_EXISTS_SQL, &[]).await;
|
||||
let row = match row_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(LiveFailure::new("managed_schema_probe")),
|
||||
};
|
||||
return match row.try_get::<usize, bool>(0) {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("managed_schema_probe_decode")),
|
||||
};
|
||||
}
|
||||
|
||||
async fn metadata_exists(client: &tokio_postgres::Client) -> std::result::Result<bool, LiveFailure> {
|
||||
let row_result = client.query_one(LIVE_METADATA_EXISTS_SQL, &[]).await;
|
||||
let row = match row_result {
|
||||
@@ -366,10 +395,10 @@ async fn metadata_exists(client: &tokio_postgres::Client) -> std::result::Result
|
||||
};
|
||||
}
|
||||
|
||||
async fn drop_metadata(client: &tokio_postgres::Client) -> std::result::Result<(), LiveFailure> {
|
||||
return match client.batch_execute(LIVE_METADATA_DROP_SQL).await {
|
||||
async fn drop_managed_schema(client: &tokio_postgres::Client) -> std::result::Result<(), LiveFailure> {
|
||||
return match client.batch_execute(LIVE_MANAGED_SCHEMA_DROP_SQL).await {
|
||||
std::result::Result::Ok(()) => std::result::Result::Ok(()),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("metadata_cleanup")),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(LiveFailure::new("managed_schema_cleanup")),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
1176
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
Normal file
1176
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
// version: 3
|
||||
// version: 9
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -38,15 +38,24 @@ fn pre_005_backend_error_projection_is_safe_and_static() {
|
||||
let kinds = [
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::Conflict,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::RetentionCompactionUnsupported,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WriteFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork,
|
||||
];
|
||||
assert_eq!(kinds.len(), 9);
|
||||
assert_eq!(kinds.len(), 18);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -58,3 +67,57 @@ fn pre_007_backend_health_bridge_exposes_only_safe_snapshot_types() {
|
||||
let _health_probe = ksp_store_postgres_lib::PostgresBackend::health;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_retention_compaction_error_code_matches_store_contract_value() {
|
||||
assert_eq!(ksp_store_postgres_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED.domain(), "store");
|
||||
assert_eq!(ksp_store_postgres_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED.code(), "postgres_retention_compaction_unsupported",);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_raw_read_bridge_uses_only_backend_independent_models() {
|
||||
let _get = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction;
|
||||
let _observation = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_observation;
|
||||
let _retention = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_retention_state;
|
||||
let _tombstone = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_tombstone;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_raw_write_bridge_uses_only_backend_independent_models_and_outcomes() {
|
||||
let _acquisition = ksp_store_postgres_lib::PostgresBackend::persist_raw_transaction_acquisition;
|
||||
let _observation = ksp_store_postgres_lib::PostgresBackend::record_raw_transaction_observation;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_raw_list_bridge_uses_backend_independent_query_page_and_reference_models() {
|
||||
let _list = ksp_store_postgres_lib::PostgresBackend::list_raw_transactions;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_raw_retention_write_bridge_uses_backend_independent_transition_and_outcome_models() {
|
||||
let _transition = ksp_store_postgres_lib::PostgresBackend::transition_raw_transaction_retention;
|
||||
return;
|
||||
}
|
||||
|
||||
fn assert_raw_transaction_capabilities<T>()
|
||||
where
|
||||
T: ksp_store_api::RawTransactionRead
|
||||
+ ksp_store_api::RawTransactionWrite
|
||||
+ ksp_store_api::RawTransactionObservationRead
|
||||
+ ksp_store_api::RawTransactionObservationWrite
|
||||
+ ksp_store_api::RawTransactionRetentionRead
|
||||
+ ksp_store_api::RawTransactionRetentionWrite,
|
||||
{
|
||||
let _marker = std::marker::PhantomData::<T>;
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_postgres_backend_implements_all_six_raw_transaction_capabilities() {
|
||||
assert_raw_transaction_capabilities::<ksp_store_postgres_lib::PostgresBackend>();
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,50 +1,142 @@
|
||||
// file: crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
||||
// version: 1
|
||||
// version: 5
|
||||
|
||||
fn applied(version: i64, name: &str, checksum: &str) -> super::AppliedMigration {
|
||||
return super::AppliedMigration { checksum: checksum.to_owned(), name: name.to_owned(), version };
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_migration_is_static_metadata_only_and_checksum_is_stable_sha256() {
|
||||
assert_eq!(super::BOOTSTRAP_MIGRATION_VERSION, 0);
|
||||
assert_eq!(super::BOOTSTRAP_MIGRATION_NAME, "bootstrap");
|
||||
assert!(super::BOOTSTRAP_MIGRATION_SQL.contains("CREATE TABLE ksp_store_schema_migrations"));
|
||||
for forbidden in ["RawTransaction", "RawAccountState", "raw_transaction", "raw_account", "CORE", "DECODE", "SPECIALIZED"] {
|
||||
assert!(!super::BOOTSTRAP_MIGRATION_SQL.contains(forbidden), "business schema leaked into bootstrap SQL: {forbidden}");
|
||||
fn embedded(version: i64, name: &'static str, resources: &'static [crate::SchemaResource]) -> super::EmbeddedMigration {
|
||||
return super::EmbeddedMigration {
|
||||
checksum: super::MigrationChecksum::Resources,
|
||||
hook: super::MigrationHook::None,
|
||||
name,
|
||||
resources,
|
||||
version,
|
||||
};
|
||||
}
|
||||
let checksum = super::bootstrap_checksum();
|
||||
assert_eq!(checksum.len(), 64);
|
||||
assert_eq!(checksum, "d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450");
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_embedded_registry_keeps_v000_checksum_and_uses_resource_owned_v001() {
|
||||
assert_eq!(super::EMBEDDED_MIGRATIONS.len(), 2);
|
||||
let v000 = &super::EMBEDDED_MIGRATIONS[0];
|
||||
assert_eq!(v000.version, 0);
|
||||
assert_eq!(v000.name, "bootstrap");
|
||||
assert_eq!(v000.hook, super::MigrationHook::None);
|
||||
assert_eq!(v000.resources.len(), 1);
|
||||
assert!(v000.resources[0].sql.contains("CREATE TABLE ksp_store_schema_migrations"));
|
||||
assert_eq!(super::migration_checksum(v000), "d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450");
|
||||
let v001 = &super::EMBEDDED_MIGRATIONS[1];
|
||||
assert_eq!(v001.version, 1);
|
||||
assert_eq!(v001.name, "raw_transaction");
|
||||
assert_eq!(v001.hook, super::MigrationHook::StoreIdentity);
|
||||
assert_eq!(v001.resources.len(), 40);
|
||||
assert_eq!(super::migration_checksum(v001), "31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51");
|
||||
assert!(super::validate_embedded_registry(super::EMBEDDED_MIGRATIONS).is_ok());
|
||||
assert_eq!(crate::current_migration_version(), 1);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn matching_sentinel_history_is_accepted() {
|
||||
let checksum = super::bootstrap_checksum();
|
||||
let history = [applied(0, "bootstrap", checksum.as_str())];
|
||||
assert!(super::validate_history(&history, checksum.as_str()).is_ok());
|
||||
fn pre_003_fix_001_v001_resource_order_and_api_bounds_are_exact() {
|
||||
let v001 = &super::EMBEDDED_MIGRATIONS[1];
|
||||
let ids = v001.resources.iter().map(|resource| return resource.id).collect::<std::vec::Vec<_>>();
|
||||
assert_eq!(ids.iter().filter(|id| return id.starts_with("tables/")).count(), 4);
|
||||
assert_eq!(ids.iter().filter(|id| return id.starts_with("constraints/")).count(), 35);
|
||||
assert_eq!(ids.iter().filter(|id| return id.starts_with("indexes/")).count(), 1);
|
||||
assert!(ids[..4].iter().all(|id| return id.starts_with("tables/")));
|
||||
assert!(ids[4..39].iter().all(|id| return id.starts_with("constraints/")));
|
||||
assert!(ids[39..].iter().all(|id| return id.starts_with("indexes/")));
|
||||
let mut unique = std::collections::BTreeSet::<&str>::new();
|
||||
for id in &ids {
|
||||
assert!(unique.insert(id), "duplicate embedded migration resource: {id}");
|
||||
}
|
||||
let sql = v001.resources.iter().map(|resource| return resource.sql).collect::<std::vec::Vec<_>>().concat();
|
||||
for required in [
|
||||
"CREATE TABLE IF NOT EXISTS ksp_store_identity",
|
||||
"CREATE TABLE IF NOT EXISTS ksp_raw_transactions",
|
||||
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_observations",
|
||||
"CREATE TABLE IF NOT EXISTS ksp_raw_transaction_archive_payloads",
|
||||
"CREATE INDEX IF NOT EXISTS ix_ksp_raw_transactions_slot_signature",
|
||||
"WHERE retention_state <> 'purged'",
|
||||
"octet_length(signature) = 64",
|
||||
"slot >= 0 AND slot <= 18446744073709551615",
|
||||
"block_time_unix_millis >= 0 AND block_time_unix_millis <= 253402300799999",
|
||||
"octet_length(content_hash) = 32",
|
||||
"octet_length(payload) >= 1 AND octet_length(payload) <= 16777216",
|
||||
"format_version >= 1 AND format_version <= 4294967295",
|
||||
"received_at_unix_millis >= 0 AND received_at_unix_millis <= 253402300799999",
|
||||
"source_payload_size_bytes >= 0 AND source_payload_size_bytes <= 67108864",
|
||||
"origin = 'backfill' OR origin = 'import' OR origin = 'live' OR origin = 'repair' OR origin = 'replay'",
|
||||
"retention_state = 'full' OR retention_state = 'archived' OR retention_state = 'purged'",
|
||||
] {
|
||||
assert!(sql.contains(required), "V001 physical contract is missing: {required}");
|
||||
}
|
||||
assert!(!sql.contains("compacted"));
|
||||
assert!(!sql.contains("BIGSERIAL"));
|
||||
assert!(!sql.contains("slot BIGINT"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn divergent_or_missing_sentinel_is_terminal_mismatch() {
|
||||
let checksum = super::bootstrap_checksum();
|
||||
let wrong_name = [applied(0, "changed", checksum.as_str())];
|
||||
let wrong_checksum = [applied(0, "bootstrap", "00")];
|
||||
let missing: [super::AppliedMigration; 0] = [];
|
||||
for history in [&wrong_name[..], &wrong_checksum[..], &missing[..]] {
|
||||
let result = super::validate_history(history, checksum.as_str());
|
||||
fn pre_003_fix_001_ordered_registry_accepts_v000_prefix_and_full_v001_history() {
|
||||
let v000 = super::EMBEDDED_MIGRATIONS[0];
|
||||
let v001 = super::EMBEDDED_MIGRATIONS[1];
|
||||
let v000_checksum = super::migration_checksum(&v000);
|
||||
let prefix = [applied(0, v000.name, v000_checksum.as_str())];
|
||||
assert_eq!(super::validate_history(&prefix, super::EMBEDDED_MIGRATIONS).ok(), std::option::Option::Some(1));
|
||||
let v001_checksum = super::migration_checksum(&v001);
|
||||
let full = [applied(0, v000.name, v000_checksum.as_str()), applied(1, v001.name, v001_checksum.as_str())];
|
||||
assert_eq!(super::validate_history(&full, super::EMBEDDED_MIGRATIONS).ok(), std::option::Option::Some(2));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_registry_rejects_empty_nonzero_gap_empty_metadata_and_empty_resources() {
|
||||
let empty: [super::EmbeddedMigration; 0] = [];
|
||||
let starts_at_one = [embedded(1, "future", crate::V000_RESOURCES)];
|
||||
let gap = [embedded(0, "bootstrap", crate::V000_RESOURCES), embedded(2, "future", crate::V000_RESOURCES)];
|
||||
let empty_name = [embedded(0, "", crate::V000_RESOURCES)];
|
||||
let empty_resources = [embedded(0, "bootstrap", &[])];
|
||||
for registry in [&empty[..], &starts_at_one[..], &gap[..], &empty_name[..], &empty_resources[..]] {
|
||||
let result = super::validate_embedded_registry(registry);
|
||||
assert_eq!(result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::MigrationMismatch));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn newer_history_is_rejected_without_down_migration() {
|
||||
let checksum = super::bootstrap_checksum();
|
||||
let history = [applied(0, "bootstrap", checksum.as_str()), applied(1, "future", "future-checksum")];
|
||||
let result = super::validate_history(&history, checksum.as_str());
|
||||
fn pre_003_fix_001_divergent_missing_or_gapped_history_is_terminal_mismatch() {
|
||||
let v000 = super::EMBEDDED_MIGRATIONS[0];
|
||||
let v001 = super::EMBEDDED_MIGRATIONS[1];
|
||||
let v000_checksum = super::migration_checksum(&v000);
|
||||
let v001_checksum = super::migration_checksum(&v001);
|
||||
let wrong_name = [applied(0, "changed", v000_checksum.as_str())];
|
||||
let wrong_checksum = [applied(0, v000.name, "00")];
|
||||
let missing: [super::AppliedMigration; 0] = [];
|
||||
let missing_v000 = [applied(1, v001.name, v001_checksum.as_str())];
|
||||
for history in [&wrong_name[..], &wrong_checksum[..], &missing[..], &missing_v000[..]] {
|
||||
let result = super::validate_history(history, super::EMBEDDED_MIGRATIONS);
|
||||
assert_eq!(result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::MigrationMismatch));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_newer_history_is_rejected_without_down_migration() {
|
||||
let v000 = super::EMBEDDED_MIGRATIONS[0];
|
||||
let v001 = super::EMBEDDED_MIGRATIONS[1];
|
||||
let v000_checksum = super::migration_checksum(&v000);
|
||||
let v001_checksum = super::migration_checksum(&v001);
|
||||
let history = [applied(0, v000.name, v000_checksum.as_str()), applied(1, v001.name, v001_checksum.as_str()), applied(2, "future", "future-checksum")];
|
||||
let result = super::validate_history(&history, super::EMBEDDED_MIGRATIONS);
|
||||
assert_eq!(result.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::SchemaNewer));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_009_fix_006_missing_applied_resource_with_autoupdate_disabled_is_migration_mismatch() {
|
||||
let error = super::schema_autoupdate_disabled_error();
|
||||
assert_eq!(error.kind(), crate::PostgresBackendErrorKind::MigrationMismatch);
|
||||
assert_eq!(error.phase(), "schema_autoupdate_disabled");
|
||||
return;
|
||||
}
|
||||
|
||||
571
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
Normal file
571
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
Normal file
@@ -0,0 +1,571 @@
|
||||
// file: crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
// version: 4
|
||||
|
||||
fn network() -> ksp_store_api::RawNetworkId {
|
||||
return match ksp_store_api::RawNetworkId::new("devnet") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test network rejected: {error:?}"),
|
||||
};
|
||||
}
|
||||
|
||||
fn transaction_row(state: &str) -> super::RawTransactionDbRow {
|
||||
return super::RawTransactionDbRow {
|
||||
archive_payload: std::option::Option::None,
|
||||
block_time_unix_millis: std::option::Option::Some(1_700_000_000_000),
|
||||
content_hash: vec![7; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: i64::from(u32::MAX),
|
||||
payload: std::option::Option::Some(vec![1, 2, 3, 4]),
|
||||
retention_state: state.to_owned(),
|
||||
signature: vec![9; 64],
|
||||
slot_text: u64::MAX.to_string(),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_full_and_archived_rows_round_trip_without_integer_narrowing() {
|
||||
let network = network();
|
||||
let full = super::decode_raw_transaction_row(&network, transaction_row("full"));
|
||||
let full = match full {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid full row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(full.slot(), u64::MAX);
|
||||
assert_eq!(full.block_time().map(|value| return value.unix_millis()), std::option::Option::Some(1_700_000_000_000));
|
||||
assert_eq!(full.payload().format_version(), u32::MAX);
|
||||
assert_eq!(full.payload().bytes(), &[1, 2, 3, 4]);
|
||||
assert_eq!(full.reference().network().as_str(), "devnet");
|
||||
assert_eq!(full.reference().signature().as_bytes(), &[9; 64]);
|
||||
let mut archived_row = transaction_row("archived");
|
||||
archived_row.payload = std::option::Option::None;
|
||||
archived_row.archive_payload = std::option::Option::Some(vec![5, 6, 7]);
|
||||
let archived = super::decode_raw_transaction_row(&network, archived_row);
|
||||
let archived = match archived {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid archived row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(archived.slot(), u64::MAX);
|
||||
assert_eq!(archived.payload().bytes(), &[5, 6, 7]);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_purged_get_returns_none_and_rejects_payload_or_block_time_residue() {
|
||||
let network = network();
|
||||
let mut purged_row = transaction_row("purged");
|
||||
purged_row.payload = std::option::Option::None;
|
||||
purged_row.archive_payload = std::option::Option::None;
|
||||
purged_row.block_time_unix_millis = std::option::Option::None;
|
||||
let purged = super::decode_raw_transaction_row(&network, purged_row);
|
||||
assert!(matches!(purged, std::result::Result::Ok(std::option::Option::None)));
|
||||
let mut malformed = transaction_row("purged");
|
||||
malformed.archive_payload = std::option::Option::None;
|
||||
malformed.block_time_unix_millis = std::option::Option::None;
|
||||
let malformed = super::decode_raw_transaction_row(&network, malformed);
|
||||
assert_eq!(malformed.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_observation_row_reconstructs_complete_safe_provenance() {
|
||||
let network = network();
|
||||
let row = super::RawObservationDbRow {
|
||||
acquisition_method: "transactionSubscribe".to_owned(),
|
||||
capture_session_id: std::option::Option::Some("capture-1".to_owned()),
|
||||
commitment: std::option::Option::Some("confirmed".to_owned()),
|
||||
endpoint_id: std::option::Option::Some("publicnode-devnet".to_owned()),
|
||||
filter_id: std::option::Option::Some("filter-1".to_owned()),
|
||||
observation_key: vec![3; 32],
|
||||
observed_at_unix_millis: std::option::Option::Some(1_699_999_999_000),
|
||||
origin: "live".to_owned(),
|
||||
protocol: "solana-ws".to_owned(),
|
||||
provider: "publicnode".to_owned(),
|
||||
received_at_unix_millis: 1_700_000_000_000,
|
||||
source_payload_hash: std::option::Option::Some(vec![4; 32]),
|
||||
source_payload_size_bytes: std::option::Option::Some(64 * 1024 * 1024),
|
||||
transaction_signature: vec![8; 64],
|
||||
};
|
||||
let decoded = super::decode_raw_observation_row(&network, row);
|
||||
let decoded = match decoded {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid observation row rejected: {error:?}"),
|
||||
};
|
||||
assert_eq!(decoded.observation_key().as_bytes(), &[3; 32]);
|
||||
assert_eq!(decoded.transaction().network().as_str(), "devnet");
|
||||
assert_eq!(decoded.transaction().signature().as_bytes(), &[8; 64]);
|
||||
assert_eq!(decoded.provenance().provider().as_str(), "publicnode");
|
||||
assert_eq!(decoded.provenance().protocol().as_str(), "solana-ws");
|
||||
assert_eq!(decoded.provenance().acquisition_method().as_str(), "transactionSubscribe");
|
||||
assert_eq!(decoded.provenance().capture_session_id().map(|value| return value.as_str()), std::option::Option::Some("capture-1"));
|
||||
assert_eq!(decoded.provenance().commitment().map(|value| return value.as_str()), std::option::Option::Some("confirmed"));
|
||||
assert_eq!(decoded.provenance().endpoint_id().map(|value| return value.as_str()), std::option::Option::Some("publicnode-devnet"));
|
||||
assert_eq!(decoded.provenance().filter_id().map(|value| return value.as_str()), std::option::Option::Some("filter-1"));
|
||||
assert_eq!(decoded.provenance().observed_at().map(|value| return value.unix_millis()), std::option::Option::Some(1_699_999_999_000));
|
||||
assert_eq!(decoded.provenance().received_at().unix_millis(), 1_700_000_000_000);
|
||||
assert_eq!(decoded.provenance().source_payload_hash().map(|value| return *value.as_bytes()), std::option::Option::Some([4; 32]));
|
||||
assert_eq!(decoded.provenance().source_payload_size_bytes(), std::option::Option::Some(64 * 1024 * 1024));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_hostile_rows_map_to_static_data_invalid_without_echoing_values() {
|
||||
let network = network();
|
||||
let mut oversized_slot = transaction_row("full");
|
||||
oversized_slot.slot_text = "18446744073709551616".to_owned();
|
||||
let slot_error = super::decode_raw_transaction_row(&network, oversized_slot).err();
|
||||
assert_eq!(slot_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert_eq!(slot_error.as_ref().map(|value| return value.phase()), std::option::Option::Some("raw_transaction_slot"));
|
||||
let mut bad_hash = transaction_row("full");
|
||||
bad_hash.content_hash = vec![1; 31];
|
||||
let hash_error = super::decode_raw_transaction_row(&network, bad_hash).err();
|
||||
assert_eq!(hash_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert!(!format!("{hash_error:?}").contains("18446744073709551616"));
|
||||
let bad_origin = super::decode_origin("https://hostile.invalid/secret");
|
||||
let origin_error = bad_origin.err();
|
||||
assert_eq!(origin_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert!(!format!("{origin_error:?}").contains("hostile.invalid"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_retention_and_tombstone_decoding_is_exact() {
|
||||
assert!(matches!(super::decode_retention_state("full"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Full)));
|
||||
assert!(matches!(super::decode_retention_state("archived"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Archived)));
|
||||
assert!(matches!(super::decode_retention_state("purged"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Purged)));
|
||||
assert_eq!(
|
||||
super::decode_retention_state("compacted").err().map(|value| return value.kind()),
|
||||
std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid),
|
||||
);
|
||||
let network = network();
|
||||
let row = super::RawTombstoneDbRow {
|
||||
block_time_unix_millis: std::option::Option::None,
|
||||
content_hash: vec![6; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: i64::from(u32::MAX),
|
||||
retention_state: "purged".to_owned(),
|
||||
signature: vec![2; 64],
|
||||
slot_text: u64::MAX.to_string(),
|
||||
};
|
||||
let tombstone = super::decode_raw_tombstone_row(&network, row);
|
||||
let tombstone = match tombstone {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid tombstone row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(tombstone.slot(), u64::MAX);
|
||||
assert_eq!(tombstone.format_version(), u32::MAX);
|
||||
assert_eq!(tombstone.content_hash().as_bytes(), &[6; 32]);
|
||||
assert_eq!(tombstone.reference().signature().as_bytes(), &[2; 64]);
|
||||
let malformed = super::RawTombstoneDbRow {
|
||||
block_time_unix_millis: std::option::Option::Some(1),
|
||||
content_hash: vec![6; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: 1,
|
||||
retention_state: "purged".to_owned(),
|
||||
signature: vec![2; 64],
|
||||
slot_text: "1".to_owned(),
|
||||
};
|
||||
let malformed = super::decode_raw_tombstone_row(&network, malformed);
|
||||
assert_eq!(malformed.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_wrong_network_is_rejected_by_the_private_pre_io_guard() {
|
||||
let backend_network = network();
|
||||
let other_network = match ksp_store_api::RawNetworkId::new("mainnet-beta") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid alternate network rejected: {error:?}"),
|
||||
};
|
||||
let reference = ksp_store_api::RawTransactionReference::new(other_network, ksp_store_api::RawTransactionSignature::new([1; 64]));
|
||||
let rejected = super::ensure_network(&backend_network, &reference, "test_network");
|
||||
assert_eq!(rejected.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::WrongNetwork));
|
||||
return;
|
||||
}
|
||||
|
||||
fn raw_transaction(signature_byte: u8, payload_bytes: &[u8], content_hash_byte: u8) -> ksp_store_api::RawTransaction {
|
||||
let reference = ksp_store_api::RawTransactionReference::new(network(), ksp_store_api::RawTransactionSignature::new([signature_byte; 64]));
|
||||
let format_id = match ksp_store_api::RawFormatId::new("ksp.raw.transaction") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test format rejected: {error:?}"),
|
||||
};
|
||||
let payload = match ksp_store_api::RawPayload::try_new(
|
||||
format_id,
|
||||
1,
|
||||
payload_bytes.to_vec().into_boxed_slice(),
|
||||
ksp_store_api::RawContentHash::new([content_hash_byte; 32]),
|
||||
) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test payload rejected: {error:?}"),
|
||||
};
|
||||
return ksp_store_api::RawTransaction::new(reference, 42, std::option::Option::None, payload);
|
||||
}
|
||||
|
||||
fn observation(reference: ksp_store_api::RawTransactionReference, key_byte: u8, provider: &str) -> ksp_store_api::RawTransactionObservation {
|
||||
let provider = match ksp_store_api::RawProvenanceCode::new(provider) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test provider rejected: {error:?}"),
|
||||
};
|
||||
let protocol = match ksp_store_api::RawProvenanceCode::new("solana-ws") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test protocol rejected: {error:?}"),
|
||||
};
|
||||
let method = match ksp_store_api::RawProvenanceCode::new("transactionSubscribe") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test method rejected: {error:?}"),
|
||||
};
|
||||
let received_at = match ksp_store_api::RawTimestamp::from_unix_millis(1_700_000_000_000) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test timestamp rejected: {error:?}"),
|
||||
};
|
||||
let provenance = ksp_store_api::RawAcquisitionProvenance::new(provider, protocol, method, ksp_store_api::RawAcquisitionOrigin::Live, received_at);
|
||||
return ksp_store_api::RawTransactionObservation::new(ksp_store_api::RawObservationKey::new([key_byte; 32]), reference, provenance);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_atomic_acquisition_pre_io_guard_requires_backend_network_and_exact_reference() {
|
||||
let backend_network = network();
|
||||
let raw_transaction = raw_transaction(1, &[1, 2, 3], 7);
|
||||
let matching = observation(raw_transaction.reference().clone(), 3, "publicnode");
|
||||
assert!(super::ensure_acquisition_inputs(&backend_network, &raw_transaction, &matching).is_ok());
|
||||
let other_reference = ksp_store_api::RawTransactionReference::new(backend_network.clone(), ksp_store_api::RawTransactionSignature::new([2; 64]));
|
||||
let mismatched = observation(other_reference, 4, "publicnode");
|
||||
let mismatch = super::ensure_acquisition_inputs(&backend_network, &raw_transaction, &mismatched);
|
||||
assert_eq!(mismatch.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::Conflict));
|
||||
let other_network = match ksp_store_api::RawNetworkId::new("mainnet-beta") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid alternate network rejected: {error:?}"),
|
||||
};
|
||||
let foreign_reference = ksp_store_api::RawTransactionReference::new(other_network, ksp_store_api::RawTransactionSignature::new([1; 64]));
|
||||
let foreign = observation(foreign_reference, 5, "publicnode");
|
||||
let wrong_network = super::ensure_acquisition_inputs(&backend_network, &raw_transaction, &foreign);
|
||||
assert_eq!(wrong_network.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::WrongNetwork));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_existing_full_content_requires_exact_payload_equality_not_hash_only() {
|
||||
let network = network();
|
||||
let incoming = raw_transaction(9, &[1, 2, 3, 4], 7);
|
||||
let matching = super::compare_existing_transaction(&network, transaction_row("full"), &incoming);
|
||||
assert!(matching.is_err(), "fixture intentionally differs in slot/content and must conflict");
|
||||
let mut exact_row = transaction_row("full");
|
||||
exact_row.signature = vec![9; 64];
|
||||
exact_row.slot_text = "42".to_owned();
|
||||
exact_row.block_time_unix_millis = std::option::Option::None;
|
||||
exact_row.format_version = 1;
|
||||
exact_row.payload = std::option::Option::Some(vec![1, 2, 3, 4]);
|
||||
let exact = super::compare_existing_transaction(&network, exact_row, &incoming);
|
||||
assert!(matches!(exact, std::result::Result::Ok(super::ExistingTransactionMatch::Active)));
|
||||
let mut same_hash_different_bytes = transaction_row("full");
|
||||
same_hash_different_bytes.signature = vec![9; 64];
|
||||
same_hash_different_bytes.slot_text = "42".to_owned();
|
||||
same_hash_different_bytes.block_time_unix_millis = std::option::Option::None;
|
||||
same_hash_different_bytes.format_version = 1;
|
||||
same_hash_different_bytes.payload = std::option::Option::Some(vec![9, 9, 9, 9]);
|
||||
let conflict = super::compare_existing_transaction(&network, same_hash_different_bytes, &incoming);
|
||||
assert_eq!(conflict.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::Conflict));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_purged_tombstone_matches_only_retained_identity_metadata() {
|
||||
let network = network();
|
||||
let incoming = raw_transaction(9, &[1, 2, 3, 4], 7);
|
||||
let mut purged = transaction_row("purged");
|
||||
purged.signature = vec![9; 64];
|
||||
purged.slot_text = "42".to_owned();
|
||||
purged.block_time_unix_millis = std::option::Option::None;
|
||||
purged.format_version = 1;
|
||||
purged.payload = std::option::Option::None;
|
||||
purged.archive_payload = std::option::Option::None;
|
||||
let compatible = super::compare_existing_transaction(&network, purged, &incoming);
|
||||
assert!(matches!(compatible, std::result::Result::Ok(super::ExistingTransactionMatch::Purged)));
|
||||
let mut divergent = transaction_row("purged");
|
||||
divergent.signature = vec![9; 64];
|
||||
divergent.slot_text = "42".to_owned();
|
||||
divergent.block_time_unix_millis = std::option::Option::None;
|
||||
divergent.format_version = 1;
|
||||
divergent.content_hash = vec![8; 32];
|
||||
divergent.payload = std::option::Option::None;
|
||||
divergent.archive_payload = std::option::Option::None;
|
||||
let conflict = super::compare_existing_transaction(&network, divergent, &incoming);
|
||||
assert_eq!(conflict.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::Conflict));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_observation_idempotence_compares_reference_and_complete_provenance() {
|
||||
let incoming_transaction = raw_transaction(8, &[1, 2, 3], 4);
|
||||
let incoming = observation(incoming_transaction.reference().clone(), 3, "publicnode");
|
||||
let matching_row = super::RawObservationDbRow {
|
||||
acquisition_method: "transactionSubscribe".to_owned(),
|
||||
capture_session_id: std::option::Option::None,
|
||||
commitment: std::option::Option::None,
|
||||
endpoint_id: std::option::Option::None,
|
||||
filter_id: std::option::Option::None,
|
||||
observation_key: vec![3; 32],
|
||||
observed_at_unix_millis: std::option::Option::None,
|
||||
origin: "live".to_owned(),
|
||||
protocol: "solana-ws".to_owned(),
|
||||
provider: "publicnode".to_owned(),
|
||||
received_at_unix_millis: 1_700_000_000_000,
|
||||
source_payload_hash: std::option::Option::None,
|
||||
source_payload_size_bytes: std::option::Option::None,
|
||||
transaction_signature: vec![8; 64],
|
||||
};
|
||||
let matching = super::decode_raw_observation_row(&network(), matching_row);
|
||||
let matching = match matching {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid matching observation rejected: {error:?}"),
|
||||
};
|
||||
assert!(matching.eq(&incoming));
|
||||
let divergent_row = super::RawObservationDbRow {
|
||||
acquisition_method: "transactionSubscribe".to_owned(),
|
||||
capture_session_id: std::option::Option::None,
|
||||
commitment: std::option::Option::None,
|
||||
endpoint_id: std::option::Option::None,
|
||||
filter_id: std::option::Option::None,
|
||||
observation_key: vec![3; 32],
|
||||
observed_at_unix_millis: std::option::Option::None,
|
||||
origin: "live".to_owned(),
|
||||
protocol: "solana-ws".to_owned(),
|
||||
provider: "another-provider".to_owned(),
|
||||
received_at_unix_millis: 1_700_000_000_000,
|
||||
source_payload_hash: std::option::Option::None,
|
||||
source_payload_size_bytes: std::option::Option::None,
|
||||
transaction_signature: vec![8; 64],
|
||||
};
|
||||
let divergent = super::decode_raw_observation_row(&network(), divergent_row);
|
||||
let divergent = match divergent {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid divergent observation rejected: {error:?}"),
|
||||
};
|
||||
assert!(!divergent.eq(&incoming));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_005_observation_origin_encoding_is_exact_and_static() {
|
||||
assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Backfill), std::result::Result::Ok("backfill"));
|
||||
assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Import), std::result::Result::Ok("import"));
|
||||
assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Live), std::result::Result::Ok("live"));
|
||||
assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Repair), std::result::Result::Ok("repair"));
|
||||
assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Replay), std::result::Result::Ok("replay"));
|
||||
return;
|
||||
}
|
||||
|
||||
fn page_query(
|
||||
network_id: &str,
|
||||
start: std::option::Option<u64>,
|
||||
end: std::option::Option<u64>,
|
||||
direction: ksp_store_api::RawSortDirection,
|
||||
cursor: std::option::Option<ksp_store_api::RawPageCursor>,
|
||||
) -> ksp_store_api::RawTransactionQuery {
|
||||
let network = match ksp_store_api::RawNetworkId::new(network_id) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid page-query network rejected: {error:?}"),
|
||||
};
|
||||
let range = match ksp_store_api::RawSlotRange::new(start, end) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid page-query range rejected: {error:?}"),
|
||||
};
|
||||
let limit = match ksp_store_api::RawPageLimit::new(2) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid page limit rejected: {error:?}"),
|
||||
};
|
||||
let page = match cursor {
|
||||
std::option::Option::Some(value) => ksp_store_api::RawPageRequest::after(limit, value),
|
||||
std::option::Option::None => ksp_store_api::RawPageRequest::first(limit),
|
||||
};
|
||||
return ksp_store_api::RawTransactionQuery::new(network, range, direction, page);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_cursor_v1_round_trip_is_fixed_109_bytes_and_exact() {
|
||||
let query = page_query("devnet", std::option::Option::Some(10), std::option::Option::Some(20), ksp_store_api::RawSortDirection::Ascending, None);
|
||||
let signature = ksp_store_api::RawTransactionSignature::new([7_u8; 64]);
|
||||
let cursor = match crate::encode_raw_transaction_cursor(&query, 15, &signature) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"),
|
||||
};
|
||||
assert_eq!(cursor.as_bytes().len(), 109);
|
||||
assert_eq!(cursor.as_bytes().get(0..4), std::option::Option::Some(b"KSPT".as_ref()));
|
||||
assert_eq!(cursor.as_bytes().get(4).copied(), std::option::Option::Some(1));
|
||||
let decoded = match crate::decode_raw_transaction_cursor(&query, &cursor) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid cursor decoding rejected: {error:?}"),
|
||||
};
|
||||
assert_eq!(decoded.last_slot, 15);
|
||||
assert_eq!(decoded.last_signature, [7_u8; 64]);
|
||||
assert_eq!(std::format!("{cursor:?}"), "RawPageCursor { len: 109 }");
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_cursor_v1_rejects_replay_across_network_direction_and_range() {
|
||||
let query = page_query("devnet", std::option::Option::Some(10), std::option::Option::Some(20), ksp_store_api::RawSortDirection::Ascending, None);
|
||||
let signature = ksp_store_api::RawTransactionSignature::new([9_u8; 64]);
|
||||
let cursor = match crate::encode_raw_transaction_cursor(&query, 15, &signature) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"),
|
||||
};
|
||||
let other_network = page_query("mainnet-beta", Some(10), Some(20), ksp_store_api::RawSortDirection::Ascending, None);
|
||||
let other_direction = page_query("devnet", Some(10), Some(20), ksp_store_api::RawSortDirection::Descending, None);
|
||||
let other_range = page_query("devnet", Some(11), Some(20), ksp_store_api::RawSortDirection::Ascending, None);
|
||||
for candidate in [&other_network, &other_direction, &other_range] {
|
||||
let error = crate::decode_raw_transaction_cursor(candidate, &cursor).err();
|
||||
assert_eq!(error.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::QueryInvalid));
|
||||
}
|
||||
let outside_range = page_query("devnet", Some(16), Some(20), ksp_store_api::RawSortDirection::Ascending, None);
|
||||
let error = crate::decode_raw_transaction_cursor(&outside_range, &cursor).err();
|
||||
assert_eq!(error.map(|value| return value.phase()), Some("raw_cursor_slot_range"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_cursor_v1_hostile_size_magic_version_and_digest_are_rejected() {
|
||||
let query = page_query("devnet", None, None, ksp_store_api::RawSortDirection::Ascending, None);
|
||||
for size in [1_usize, 108, 109, 110, 4_096] {
|
||||
let raw = match ksp_store_api::RawPageCursor::try_new(vec![0_u8; size].into_boxed_slice()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("bounded hostile cursor rejected by API before backend test: {error:?}"),
|
||||
};
|
||||
assert!(crate::decode_raw_transaction_cursor(&query, &raw).is_err());
|
||||
}
|
||||
let signature = ksp_store_api::RawTransactionSignature::new([3_u8; 64]);
|
||||
let valid = match crate::encode_raw_transaction_cursor(&query, 42, &signature) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"),
|
||||
};
|
||||
for index in [0_usize, 4, 108] {
|
||||
let mut bytes = valid.as_bytes().to_vec();
|
||||
bytes[index] ^= 0xff;
|
||||
let hostile = match ksp_store_api::RawPageCursor::try_new(bytes.into_boxed_slice()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("fixed-size hostile cursor rejected by API before backend test: {error:?}"),
|
||||
};
|
||||
let error = crate::decode_raw_transaction_cursor(&query, &hostile).err();
|
||||
assert_eq!(error.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::QueryInvalid));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_006_page_limit_exposes_only_the_real_postgres_limit_plus_one_boundary() {
|
||||
let maximum = crate::raw_transaction_physical_page_limit(9_223_372_036_854_775_806);
|
||||
assert_eq!(maximum, Ok((9_223_372_036_854_775_806_usize, i64::MAX)));
|
||||
let rejected = crate::raw_transaction_physical_page_limit(9_223_372_036_854_775_807).err();
|
||||
assert_eq!(rejected.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::PageLimitUnsupported));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_retention_transition_decision_matches_compare_and_transition_contract() {
|
||||
assert_eq!(
|
||||
super::retention_transition_decision(
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
),
|
||||
std::result::Result::Ok(super::RetentionTransitionDecision::Archive),
|
||||
);
|
||||
assert_eq!(
|
||||
super::retention_transition_decision(
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
),
|
||||
std::result::Result::Ok(super::RetentionTransitionDecision::AlreadyAtTarget),
|
||||
);
|
||||
assert_eq!(
|
||||
super::retention_transition_decision(
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
ksp_store_api::RawRetentionState::Purged,
|
||||
),
|
||||
std::result::Result::Ok(super::RetentionTransitionDecision::Purge),
|
||||
);
|
||||
assert_eq!(
|
||||
super::retention_transition_decision(
|
||||
ksp_store_api::RawRetentionState::Purged,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
ksp_store_api::RawRetentionState::Purged,
|
||||
),
|
||||
std::result::Result::Ok(super::RetentionTransitionDecision::AlreadyAtTarget),
|
||||
);
|
||||
assert_eq!(
|
||||
super::retention_transition_decision(
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
ksp_store_api::RawRetentionState::Purged,
|
||||
),
|
||||
std::result::Result::Ok(super::RetentionTransitionDecision::ExpectedStateMismatch),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_compacted_transitions_are_rejected_by_the_pre_io_guard() {
|
||||
let backend_network = network();
|
||||
let reference = ksp_store_api::RawTransactionReference::new(backend_network.clone(), ksp_store_api::RawTransactionSignature::new([5; 64]));
|
||||
let compact = match ksp_store_api::RawTransactionRetentionTransition::try_new(
|
||||
reference.clone(),
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Compacted,
|
||||
) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid logical compact transition rejected by API: {error:?}"),
|
||||
};
|
||||
let compact_error = super::ensure_retention_transition_inputs(&backend_network, &compact).err();
|
||||
assert_eq!(
|
||||
compact_error.as_ref().map(|value| return value.kind()),
|
||||
std::option::Option::Some(crate::PostgresBackendErrorKind::RetentionCompactionUnsupported),
|
||||
);
|
||||
assert_eq!(compact_error.as_ref().map(|value| return value.phase()), std::option::Option::Some("raw_retention_compaction"));
|
||||
let compact_to_archive = match ksp_store_api::RawTransactionRetentionTransition::try_new(
|
||||
reference,
|
||||
ksp_store_api::RawRetentionState::Compacted,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid logical compacted archive transition rejected by API: {error:?}"),
|
||||
};
|
||||
let compact_to_archive_error = super::ensure_retention_transition_inputs(&backend_network, &compact_to_archive).err();
|
||||
assert_eq!(
|
||||
compact_to_archive_error.as_ref().map(|value| return value.kind()),
|
||||
std::option::Option::Some(crate::PostgresBackendErrorKind::RetentionCompactionUnsupported),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_retention_transition_pre_io_guard_rejects_wrong_network() {
|
||||
let backend_network = network();
|
||||
let other_network = match ksp_store_api::RawNetworkId::new("mainnet-beta") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid alternate network rejected: {error:?}"),
|
||||
};
|
||||
let reference = ksp_store_api::RawTransactionReference::new(other_network, ksp_store_api::RawTransactionSignature::new([6; 64]));
|
||||
let transition = match ksp_store_api::RawTransactionRetentionTransition::try_new(
|
||||
reference,
|
||||
ksp_store_api::RawRetentionState::Full,
|
||||
ksp_store_api::RawRetentionState::Archived,
|
||||
) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid retention transition rejected by API: {error:?}"),
|
||||
};
|
||||
let rejected = super::ensure_retention_transition_inputs(&backend_network, &transition);
|
||||
assert_eq!(rejected.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::WrongNetwork));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_007_retained_payload_shape_rejects_empty_and_oversized_bytes() {
|
||||
assert!(super::validate_retained_payload_bytes(&[1], "test_retention_payload").is_ok());
|
||||
let empty = super::validate_retained_payload_bytes(&[], "test_retention_payload");
|
||||
assert_eq!(empty.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
let oversized = vec![0; ksp_store_api::MAX_RAW_PAYLOAD_BYTES + 1];
|
||||
let oversized = super::validate_retained_payload_bytes(oversized.as_slice(), "test_retention_payload");
|
||||
assert_eq!(oversized.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
return;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/unit_tests/runtime.rs
|
||||
// version: 3
|
||||
// version: 5
|
||||
|
||||
fn network() -> ksp_store_api::RawNetworkId {
|
||||
return match ksp_store_api::RawNetworkId::new("devnet") {
|
||||
@@ -100,3 +100,62 @@ fn libpq_server_options_are_rejected_in_foundation_runtime() {
|
||||
assert_eq!(error.map(|value| return value.phase()), std::option::Option::Some("server_options"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_008_capability_error_mapping_uses_stable_store_and_store_api_codes() {
|
||||
let cases = [
|
||||
(crate::PostgresBackendErrorKind::Conflict, ksp_store_api::ERROR_CODE_RAW_CONFLICT),
|
||||
(crate::PostgresBackendErrorKind::DataInvalid, ksp_store_api::ErrorCode::new("store", "postgres_data_invalid")),
|
||||
(crate::PostgresBackendErrorKind::PageLimitUnsupported, ksp_store_api::ErrorCode::new("store", "postgres_page_limit_unsupported")),
|
||||
(crate::PostgresBackendErrorKind::QueryInvalid, ksp_store_api::ERROR_CODE_RAW_QUERY_INVALID),
|
||||
(crate::PostgresBackendErrorKind::ReadFailed, ksp_store_api::ErrorCode::new("store", "postgres_read_failed")),
|
||||
(crate::PostgresBackendErrorKind::ReferenceNotFound, ksp_store_api::ErrorCode::new("store", "raw_reference_not_found")),
|
||||
(crate::PostgresBackendErrorKind::RetentionCompactionUnsupported, crate::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED),
|
||||
(crate::PostgresBackendErrorKind::WriteFailed, ksp_store_api::ErrorCode::new("store", "postgres_write_failed")),
|
||||
(crate::PostgresBackendErrorKind::WrongNetwork, ksp_store_api::ErrorCode::new("store", "wrong_network")),
|
||||
];
|
||||
for (kind, expected) in cases {
|
||||
let backend = crate::PostgresBackendError::new(kind, "pre_008_canary");
|
||||
let mapped = super::map_capability_error(backend);
|
||||
assert_eq!(mapped.code(), expected);
|
||||
let rendered = std::format!("{mapped:?}");
|
||||
assert!(!rendered.contains("postgresql://"));
|
||||
assert!(!rendered.contains("SELECT "));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_010_capability_error_mapping_covers_every_current_backend_error_kind() {
|
||||
let cases = [
|
||||
(crate::PostgresBackendErrorKind::ConfigInvalid, ksp_store_api::ErrorCode::new("store", "postgres_config_invalid")),
|
||||
(crate::PostgresBackendErrorKind::ConnectFailed, ksp_store_api::ErrorCode::new("store", "postgres_connect_failed")),
|
||||
(crate::PostgresBackendErrorKind::PoolTimeout, ksp_store_api::ErrorCode::new("store", "postgres_pool_timeout")),
|
||||
(crate::PostgresBackendErrorKind::HealthFailed, ksp_store_api::ErrorCode::new("store", "postgres_health_failed")),
|
||||
(crate::PostgresBackendErrorKind::Conflict, ksp_store_api::ERROR_CODE_RAW_CONFLICT),
|
||||
(crate::PostgresBackendErrorKind::DataInvalid, ksp_store_api::ErrorCode::new("store", "postgres_data_invalid")),
|
||||
(crate::PostgresBackendErrorKind::MigrationFailed, ksp_store_api::ErrorCode::new("store", "postgres_migration_failed")),
|
||||
(crate::PostgresBackendErrorKind::PageLimitUnsupported, ksp_store_api::ErrorCode::new("store", "postgres_page_limit_unsupported")),
|
||||
(crate::PostgresBackendErrorKind::MigrationMismatch, ksp_store_api::ErrorCode::new("store", "postgres_migration_mismatch")),
|
||||
(crate::PostgresBackendErrorKind::QueryInvalid, ksp_store_api::ERROR_CODE_RAW_QUERY_INVALID),
|
||||
(crate::PostgresBackendErrorKind::ReadFailed, ksp_store_api::ErrorCode::new("store", "postgres_read_failed")),
|
||||
(crate::PostgresBackendErrorKind::ReferenceNotFound, ksp_store_api::ErrorCode::new("store", "raw_reference_not_found")),
|
||||
(crate::PostgresBackendErrorKind::RetentionCompactionUnsupported, crate::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED),
|
||||
(crate::PostgresBackendErrorKind::SchemaNewer, ksp_store_api::ErrorCode::new("store", "postgres_schema_newer")),
|
||||
(crate::PostgresBackendErrorKind::ShutdownTimeout, ksp_store_api::ErrorCode::new("store", "shutdown_timeout")),
|
||||
(crate::PostgresBackendErrorKind::TlsFailed, ksp_store_api::ErrorCode::new("store", "postgres_tls_failed")),
|
||||
(crate::PostgresBackendErrorKind::WriteFailed, ksp_store_api::ErrorCode::new("store", "postgres_write_failed")),
|
||||
(crate::PostgresBackendErrorKind::WrongNetwork, ksp_store_api::ErrorCode::new("store", "wrong_network")),
|
||||
];
|
||||
assert_eq!(cases.len(), 18);
|
||||
for (kind, expected) in cases {
|
||||
let backend = crate::PostgresBackendError::new(kind, "pre_010_safe_phase");
|
||||
let mapped = super::map_capability_error(backend);
|
||||
assert_eq!(mapped.code(), expected);
|
||||
let rendered = std::format!("{mapped:?}");
|
||||
assert!(rendered.contains("pre_010_safe_phase"));
|
||||
assert!(!rendered.contains("postgresql://"));
|
||||
assert!(!rendered.contains("SELECT "));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
129
crates/ksp-store-postgres-lib/unit_tests/schema.rs
Normal file
129
crates/ksp-store-postgres-lib/unit_tests/schema.rs
Normal file
@@ -0,0 +1,129 @@
|
||||
// file: crates/ksp-store-postgres-lib/unit_tests/schema.rs
|
||||
// version: 4
|
||||
|
||||
fn actual_column(name: &str, udt_name: &str, nullable: bool) -> super::ActualColumn {
|
||||
return super::ActualColumn {
|
||||
default: std::option::Option::None,
|
||||
generated: "NEVER".to_owned(),
|
||||
identity: "NO".to_owned(),
|
||||
name: name.to_owned(),
|
||||
nullable,
|
||||
numeric_precision: std::option::Option::None,
|
||||
numeric_scale: std::option::Option::None,
|
||||
udt_name: udt_name.to_owned(),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_v000_resource_is_relocated_without_changing_legacy_sql() {
|
||||
assert_eq!(crate::V000_RESOURCES.len(), 1);
|
||||
let resource = crate::V000_RESOURCES[0];
|
||||
assert_eq!(resource.id, "tables/001_ksp_store_schema_migrations.sql");
|
||||
assert_eq!(
|
||||
resource.sql,
|
||||
"CREATE TABLE ksp_store_schema_migrations (\n version BIGINT PRIMARY KEY,\n name TEXT NOT NULL,\n checksum TEXT NOT NULL,\n applied_at TIMESTAMPTZ NOT NULL\n);\n",
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_v001_resources_are_split_and_idempotent_by_object_family() {
|
||||
assert_eq!(crate::V001_RESOURCES.len(), 40);
|
||||
let table_resources = crate::V001_RESOURCES.iter().filter(|resource| return resource.id.starts_with("tables/")).collect::<std::vec::Vec<_>>();
|
||||
let constraint_resources = crate::V001_RESOURCES.iter().filter(|resource| return resource.id.starts_with("constraints/")).collect::<std::vec::Vec<_>>();
|
||||
let index_resources = crate::V001_RESOURCES.iter().filter(|resource| return resource.id.starts_with("indexes/")).collect::<std::vec::Vec<_>>();
|
||||
assert_eq!(table_resources.len(), 4);
|
||||
assert_eq!(constraint_resources.len(), 35);
|
||||
assert_eq!(index_resources.len(), 1);
|
||||
for resource in table_resources {
|
||||
assert!(resource.sql.contains("CREATE TABLE IF NOT EXISTS"));
|
||||
assert!(resource.sql.contains("ADD COLUMN IF NOT EXISTS"));
|
||||
}
|
||||
for resource in constraint_resources {
|
||||
assert!(resource.sql.contains("IF NOT EXISTS"));
|
||||
}
|
||||
for resource in index_resources {
|
||||
assert!(resource.sql.contains("CREATE INDEX IF NOT EXISTS"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_external_extra_columns_are_accepted_only_when_they_cannot_block_ksp_inserts() {
|
||||
let nullable = actual_column("external_nullable", "text", true);
|
||||
assert!(nullable.is_non_blocking_extra());
|
||||
let mut with_default = actual_column("external_default", "text", true);
|
||||
with_default.default = std::option::Option::Some("'x'::text".to_owned());
|
||||
assert!(!with_default.is_non_blocking_extra());
|
||||
let mut identity = actual_column("external_identity", "int8", true);
|
||||
identity.identity = "YES".to_owned();
|
||||
assert!(!identity.is_non_blocking_extra());
|
||||
let mut generated = actual_column("external_generated", "text", true);
|
||||
generated.generated = "ALWAYS".to_owned();
|
||||
assert!(!generated.is_non_blocking_extra());
|
||||
let blocking = actual_column("external_required", "text", false);
|
||||
assert!(!blocking.is_non_blocking_extra());
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_required_column_matching_ignores_integer_precision_but_requires_numeric_20_0() {
|
||||
let expected_int = super::ColumnContract {
|
||||
name: "version",
|
||||
nullable: false,
|
||||
numeric_precision: std::option::Option::None,
|
||||
numeric_scale: std::option::Option::None,
|
||||
udt_name: "int8",
|
||||
};
|
||||
let mut actual_int = actual_column("version", "int8", false);
|
||||
actual_int.numeric_precision = std::option::Option::Some(64);
|
||||
actual_int.numeric_scale = std::option::Option::Some(0);
|
||||
assert!(actual_int.matches(&expected_int));
|
||||
let expected_numeric = super::ColumnContract {
|
||||
name: "slot",
|
||||
nullable: false,
|
||||
numeric_precision: std::option::Option::Some(20),
|
||||
numeric_scale: std::option::Option::Some(0),
|
||||
udt_name: "numeric",
|
||||
};
|
||||
let mut actual_numeric = actual_column("slot", "numeric", false);
|
||||
actual_numeric.numeric_precision = std::option::Option::Some(20);
|
||||
actual_numeric.numeric_scale = std::option::Option::Some(0);
|
||||
assert!(actual_numeric.matches(&expected_numeric));
|
||||
actual_numeric.numeric_precision = std::option::Option::Some(19);
|
||||
assert!(!actual_numeric.matches(&expected_numeric));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_003_fix_001_catalog_normalization_and_resource_owned_constraint_definition_are_deterministic() {
|
||||
let normalized = super::normalize_catalog_sql("CHECK ((slot >= (0)::numeric) AND (slot <= (18446744073709551615)::numeric))");
|
||||
assert_eq!(normalized, "checkslot>=0andslot<=18446744073709551615");
|
||||
let postgres_17_normalized = super::normalize_catalog_sql("CHECK ((slot >= '0'::numeric) AND (slot <= '18446744073709551615'::numeric))");
|
||||
assert_eq!(postgres_17_normalized, normalized);
|
||||
let bigint_normalized = super::normalize_catalog_sql(
|
||||
"CHECK ((block_time_unix_millis IS NULL) OR ((block_time_unix_millis >= '0'::bigint) AND (block_time_unix_millis <= '253402300799999'::bigint)))",
|
||||
);
|
||||
let block_time_resource = crate::V001_RESOURCES.iter().find(|resource| return resource.id == "constraints/007_ck_ksp_raw_transactions_block_time.sql");
|
||||
assert!(block_time_resource.is_some(), "V001 block-time constraint resource must remain embedded");
|
||||
let block_time_resource = match block_time_resource {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return,
|
||||
};
|
||||
let block_time_expected = super::expected_constraint_definition(block_time_resource.sql, "ck_ksp_raw_transactions_block_time");
|
||||
assert_eq!(block_time_expected.as_deref(), std::option::Option::Some(bigint_normalized.as_str()));
|
||||
let integer_alias = super::normalize_catalog_sql("CHECK (format_version <= '4294967295'::int8)");
|
||||
assert_eq!(integer_alias, "checkformat_version<=4294967295");
|
||||
let text_literal = super::normalize_catalog_sql("CHECK (retention_state = 'full'::text)");
|
||||
assert_eq!(text_literal, "checkretention_state='full'");
|
||||
let resource = crate::V001_RESOURCES.iter().find(|resource| return resource.id == "constraints/006_ck_ksp_raw_transactions_slot.sql");
|
||||
assert!(resource.is_some(), "V001 slot constraint resource must remain embedded");
|
||||
let resource = match resource {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return,
|
||||
};
|
||||
let expected = super::expected_constraint_definition(resource.sql, "ck_ksp_raw_transactions_slot");
|
||||
assert_eq!(expected.as_deref(), std::option::Option::Some(normalized.as_str()));
|
||||
assert_ne!(expected.as_deref(), std::option::Option::Some("checkslot>=0andslot<=10"));
|
||||
return;
|
||||
}
|
||||
283
deltas/0.3.3/pre.001.md
Normal file
283
deltas/0.3.3/pre.001.md
Normal file
@@ -0,0 +1,283 @@
|
||||
<!-- file: deltas/0.3.3/pre.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.001` — audit et physical design RawTransaction PostgreSQL
|
||||
|
||||
## 1. Objet
|
||||
|
||||
Ouverture de :
|
||||
|
||||
```text
|
||||
0.3.3 — Store/PostgreSQL RawTransaction vertical slice
|
||||
```
|
||||
|
||||
Cette tranche reste volontairement documentaire/conception : elle fixe l'audit, le threat model, le design physique V001 et le séquencement avant toute migration ou persistence métier.
|
||||
|
||||
## 2. Base
|
||||
|
||||
Base canonique :
|
||||
|
||||
```text
|
||||
v0.3.2
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `0.3.2` est propre : audits Rust/Markdown, workspace check, Clippy, tests Store/API/PostgreSQL foundation/Config et `ksp-store-lib --no-default-features` passent ; les tests live restent opt-in/ignored conformément à la fondation.
|
||||
|
||||
## 3. Version
|
||||
|
||||
Version workspace après overlay :
|
||||
|
||||
```text
|
||||
0.3.3-pre.1
|
||||
```
|
||||
|
||||
Label de livraison :
|
||||
|
||||
```text
|
||||
0.3.3-pre.001
|
||||
```
|
||||
|
||||
## 4. Fichiers ajoutés
|
||||
|
||||
```text
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.001.md
|
||||
```
|
||||
|
||||
## 5. Fichier modifié
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
```
|
||||
|
||||
Modifications :
|
||||
|
||||
```text
|
||||
workspace.package.version : 0.3.2 -> 0.3.3-pre.1
|
||||
file header version : 347 -> 348
|
||||
```
|
||||
|
||||
## 6. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 7. Audit effectué
|
||||
|
||||
Ont été relus avant design :
|
||||
|
||||
- règles générales/KSP/Rust/dépendances/docs/file contracts/version workflow/prompt structure ;
|
||||
- architectures Layers/Dependencies/Store/API/Config concernées ;
|
||||
- plan/validation `0.3.1` ;
|
||||
- source/tests complets `ksp-store-api` ;
|
||||
- plan/validation `0.3.2` ;
|
||||
- source/tests complets `ksp-store-lib` et `ksp-store-postgres-lib` ;
|
||||
- surface Config `std.store` ;
|
||||
- archive historique kbot3 ciblée demandée par le prompt ;
|
||||
- documentation PostgreSQL/tokio-postgres nécessaire aux choix numériques et de concurrence.
|
||||
|
||||
## 8. Décisions majeures
|
||||
|
||||
### 8.1 `slot: u64`
|
||||
|
||||
`BIGINT` est rejeté pour `slot` car il narrowe le domaine API. V001 utilisera :
|
||||
|
||||
```text
|
||||
NUMERIC(20,0)
|
||||
```
|
||||
|
||||
avec conversion décimale fallible et tests jusqu'à `u64::MAX`.
|
||||
|
||||
### 8.2 Réseau physique
|
||||
|
||||
V001 introduira :
|
||||
|
||||
```text
|
||||
ksp_store_identity
|
||||
```
|
||||
|
||||
Une base V001 est mono-réseau. Le binding est créé/validé sous le bootstrap lock afin qu'une URI pointant vers un autre réseau échoue avant exposition d'un backend prêt.
|
||||
|
||||
### 8.3 Schéma RAW minimal
|
||||
|
||||
Tables décidées :
|
||||
|
||||
```text
|
||||
ksp_store_identity
|
||||
ksp_raw_transactions
|
||||
ksp_raw_transaction_observations
|
||||
ksp_raw_transaction_archive_payloads
|
||||
```
|
||||
|
||||
Index métier additionnel unique :
|
||||
|
||||
```text
|
||||
(slot, signature) WHERE retention_state <> 'purged'
|
||||
```
|
||||
|
||||
Aucun index provider/status/created_at/hash n'est ajouté sans requête correspondante.
|
||||
|
||||
### 8.4 Atomicité/idempotence
|
||||
|
||||
Le repository n'utilisera aucun `has_*` préalable. La stratégie est :
|
||||
|
||||
```text
|
||||
INSERT ... ON CONFLICT DO NOTHING RETURNING
|
||||
-> si conflit : SELECT ... FOR UPDATE
|
||||
-> comparaison exacte du contenu
|
||||
-> AlreadyPresent ou ERROR_CODE_RAW_CONFLICT
|
||||
```
|
||||
|
||||
Le canonique et son observation d'acquisition sont dans la même transaction PostgreSQL.
|
||||
|
||||
### 8.5 Pagination
|
||||
|
||||
Ordre total :
|
||||
|
||||
```text
|
||||
(slot, signature)
|
||||
```
|
||||
|
||||
Cursor backend-private V1 fixe 109 octets, lié par SHA-256 au réseau, à la direction, au slot range et à la dernière clé. Aucun cap métier arbitraire n'est ajouté.
|
||||
|
||||
### 8.6 Rétention
|
||||
|
||||
`Full`, `Archived` et `Purged` sont représentables honnêtement par le backend `0.3.3` avec une relation archive séparée du hot path.
|
||||
|
||||
`Compacted` n'est pas assimilé au TOAST PostgreSQL et n'est pas simulé. Le plan `0.3.1` le qualifiait déjà d'optionnel ; aucun changement `ksp-store-api` n'est donc justifié. Les transitions qui l'impliquent seront rejetées avant I/O par `store.postgres_retention_compaction_unsupported`.
|
||||
|
||||
## 9. Audit kbot3
|
||||
|
||||
Classification effectuée :
|
||||
|
||||
- **REPRENDRE** : unicités, FK, keyset/index slot conceptuels ;
|
||||
- **REDESSINER** : fixed bytes, payload opaque, u64 exact, atomicité, conflit réel, réseau, retention compare-and-transition ;
|
||||
- **REPORTER** : indexes provider/status/time, processing/replay, batches, apps, AccountState ;
|
||||
- **REJETER** : sqlx, `has_*`, JSONB canonique, narrowing i64/i32, FK `SET NULL`, error text SQL, caps 500/1000, processing_state RAW.
|
||||
|
||||
Aucun code historique n'est copié.
|
||||
|
||||
## 10. Threat model couvert
|
||||
|
||||
Le plan couvre explicitement :
|
||||
|
||||
- mauvaise URI/réseau ;
|
||||
- DB bytes malformés ;
|
||||
- overflow u64 ;
|
||||
- observation orpheline ;
|
||||
- inserts identiques/divergents concurrents ;
|
||||
- collision observation ;
|
||||
- cursor hostile/replay ;
|
||||
- ambiguïté pagination ;
|
||||
- retention race ;
|
||||
- purge/read/write race ;
|
||||
- ForceRehydrate race ;
|
||||
- fuite SQL/server text ;
|
||||
- cancellation/rollback ;
|
||||
- migration mismatch.
|
||||
|
||||
## 11. Séquencement recalibré
|
||||
|
||||
Le forecast initial est étendu à `pre.013` afin de séparer proprement :
|
||||
|
||||
1. généralisation du moteur de migration ;
|
||||
2. V001/binding réseau ;
|
||||
3. mapping/read ;
|
||||
4. write/observation atomiques ;
|
||||
5. pagination ;
|
||||
6. rétention ;
|
||||
7. façade ;
|
||||
8. live PostgreSQL ;
|
||||
9. hardening ;
|
||||
10. gate ;
|
||||
11. docs ;
|
||||
12. publication.
|
||||
|
||||
## 12. Hors scope conservé
|
||||
|
||||
Aucun changement fonctionnel n'est apporté à :
|
||||
|
||||
```text
|
||||
RawAccountState
|
||||
N2
|
||||
workers/jobs/backfills
|
||||
apps
|
||||
transport
|
||||
codecs wire
|
||||
event bus
|
||||
compression/archive worker global
|
||||
autres backends
|
||||
```
|
||||
|
||||
## 13. Validations exécutées
|
||||
|
||||
Audit statique de la base et de l'overlay :
|
||||
|
||||
```text
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
General Rust rule audit: clean
|
||||
Rust export completeness audit: 0 candidate(s)
|
||||
KSP workspace Rust rule audit: clean
|
||||
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
Markdown table audit: clean (214 table(s), 131 file(s))
|
||||
|
||||
assertions structurelles locales
|
||||
workspace.package.version = 0.3.3-pre.1
|
||||
headers/version des trois nouveaux Markdown corrects
|
||||
aucune référence résiduelle au faux code API RAW_RETENTION_UNSUPPORTED
|
||||
```
|
||||
|
||||
L'audit de base couvre aussi manifests/features, modules/exports, V000 et checksum, Config `std.store`, plans/validations `0.3.1`/`0.3.2`, tests hardening/completeness et preuve live documentée.
|
||||
|
||||
## 14. Validations non exécutées
|
||||
|
||||
Le conteneur d'assemblage ne fournit pas l'exécutable `cargo`. Les commandes suivantes sont donc **NON EXÉCUTÉES**, jamais déclarées PASS :
|
||||
|
||||
```text
|
||||
cargo fmt --all
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Les tests PostgreSQL live ne sont pas exécutés non plus. `pre.001` ne modifie aucun SQL/runtime métier, la preuve foundation réelle `0.3.2` est déjà acquise et le futur test RawTransaction n'existe pas encore.
|
||||
|
||||
## 15. Questions ouvertes
|
||||
|
||||
Aucune question de design bloquante n'est laissée ouverte par `pre.001`.
|
||||
|
||||
Les valeurs runtime retenues sont déjà fixées :
|
||||
|
||||
```text
|
||||
store.postgres_retention_compaction_unsupported
|
||||
store.wrong_network
|
||||
store.raw_reference_not_found
|
||||
store.postgres_read_failed
|
||||
store.postgres_write_failed
|
||||
store.postgres_data_invalid
|
||||
store.postgres_page_limit_unsupported
|
||||
```
|
||||
|
||||
Le checksum V001 n'est pas une question de design : il sera calculé sur les bytes exacts lorsque `V001__raw_transaction.sql` sera créé en `pre.003`.
|
||||
|
||||
## 16. Gate demandé
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Aucun test PostgreSQL live n'est requis dans cette tranche documentaire.
|
||||
286
deltas/0.3.3/pre.002.md
Normal file
286
deltas/0.3.3/pre.002.md
Normal file
@@ -0,0 +1,286 @@
|
||||
<!-- file: deltas/0.3.3/pre.002.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.002` — moteur de migrations PostgreSQL multi-version
|
||||
|
||||
## 1. Base requise
|
||||
|
||||
```text
|
||||
0.3.3-pre.001
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `pre.001` est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
audit Rust général / exports / workspace PASS
|
||||
audit Markdown PASS — 214 tables / 131 files
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS
|
||||
cargo test -p ksp-config-lib PASS
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
Les tests live `#[ignore]` restent volontairement hors de ce gate ; `pre.001` était documentaire et `pre.002` ne crée encore aucun SQL métier V001.
|
||||
|
||||
## 2. Objectif
|
||||
|
||||
Généraliser le bootstrap PostgreSQL V000 acquis en `0.3.2` vers un moteur embedded multi-version avant d'introduire la première migration métier.
|
||||
|
||||
La tranche doit préserver :
|
||||
|
||||
```text
|
||||
checksum immuable
|
||||
mismatch terminal
|
||||
schema newer terminal
|
||||
aucun down automatique
|
||||
advisory transaction lock
|
||||
migration timeout
|
||||
statement timeout
|
||||
transaction unique
|
||||
metadata shape validation
|
||||
erreurs externes non rendues
|
||||
```
|
||||
|
||||
Elle prépare également la frontière transactionnelle du futur binding `RawNetworkId`, sans créer V001 ni `ksp_store_identity`.
|
||||
|
||||
## 3. Version
|
||||
|
||||
Le workspace passe à :
|
||||
|
||||
```text
|
||||
0.3.3-pre.2
|
||||
```
|
||||
|
||||
## 4. Registre embedded
|
||||
|
||||
Le moteur ne possède plus une série de constantes spécialisées V000. Il utilise désormais :
|
||||
|
||||
```text
|
||||
EMBEDDED_MIGRATIONS: &[EmbeddedMigration]
|
||||
```
|
||||
|
||||
Le registre réel de cette tranche contient exactement :
|
||||
|
||||
```text
|
||||
V000 bootstrap
|
||||
```
|
||||
|
||||
Aucune V001 n'est anticipée dans les migrations physiques.
|
||||
|
||||
Chaque entrée possède :
|
||||
|
||||
```text
|
||||
version
|
||||
name
|
||||
sql embedded
|
||||
migration hook
|
||||
```
|
||||
|
||||
Le registre est validé avant I/O :
|
||||
|
||||
```text
|
||||
non vide
|
||||
commence à 0
|
||||
versions contiguës strictement ordonnées
|
||||
name non vide
|
||||
SQL non vide
|
||||
```
|
||||
|
||||
La version courante est dérivée du dernier élément du registre.
|
||||
|
||||
## 5. Validation d'historique multi-version
|
||||
|
||||
L'historique PostgreSQL est maintenant validé comme **préfixe exact** du registre embedded.
|
||||
|
||||
Cas :
|
||||
|
||||
```text
|
||||
historique exact complet -> prêt
|
||||
préfixe exact -> migrations pending
|
||||
nom divergent -> MigrationMismatch
|
||||
checksum divergent -> MigrationMismatch
|
||||
version/trou inattendu connu -> MigrationMismatch
|
||||
version > runtime courant -> SchemaNewer
|
||||
metadata existante sans V000 -> MigrationMismatch
|
||||
```
|
||||
|
||||
Un préfixe exact n'est donc plus confondu avec une divergence : il représente précisément le cas normal d'upgrade V000 -> V001 attendu en `pre.003`.
|
||||
|
||||
## 6. Application des migrations pending
|
||||
|
||||
Sous la transaction PostgreSQL et l'advisory lock déjà acquis :
|
||||
|
||||
```text
|
||||
load/validate history
|
||||
-> déterminer next_index
|
||||
-> refuser si pending && auto_migrate=false
|
||||
-> appliquer chaque migration dans l'ordre
|
||||
-> vérifier metadata shape
|
||||
-> exécuter migration hook
|
||||
-> insérer checksum/name/version dans history
|
||||
-> migration suivante
|
||||
-> commit unique
|
||||
```
|
||||
|
||||
Une erreur à n'importe quelle étape provoque le rollback transactionnel normal ; aucune migration partiellement enregistrée n'est considérée appliquée.
|
||||
|
||||
## 7. Hook atomique de binding réseau
|
||||
|
||||
`bootstrap` reçoit désormais le `RawNetworkId` du backend et le transmet jusqu'à chaque migration.
|
||||
|
||||
Le moteur distingue deux contextes privés :
|
||||
|
||||
```text
|
||||
Existing migration déjà présente dans history à la réouverture
|
||||
AppliedNow migration exécutée pendant le bootstrap courant
|
||||
```
|
||||
|
||||
Pour les migrations déjà appliquées, le hook est rejoué sous la transaction/advisory lock avant que le backend puisse être déclaré prêt. Pour une migration nouvellement appliquée, l'ordre est :
|
||||
|
||||
```text
|
||||
batch_execute(migration.sql)
|
||||
-> metadata shape check
|
||||
-> migration hook AppliedNow
|
||||
-> INSERT history
|
||||
-> commit final
|
||||
```
|
||||
|
||||
V000 utilise :
|
||||
|
||||
```text
|
||||
MigrationHook::None
|
||||
```
|
||||
|
||||
`pre.003` pourra donc ajouter le hook V001 `ksp_store_identity` avec deux comportements sûrs : création/validation en `AppliedNow`, validation stricte sans recréation en `Existing`. La frontière reste transactionnelle et ne crée aucune seconde phase de bootstrap race-prone.
|
||||
|
||||
Aucune identité réseau n'est écrite dans cette tranche.
|
||||
|
||||
## 8. Compatibilité V000
|
||||
|
||||
Le fichier reste inchangé :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql
|
||||
```
|
||||
|
||||
Checksum conservé :
|
||||
|
||||
```text
|
||||
d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
```
|
||||
|
||||
La metadata historique reste :
|
||||
|
||||
```text
|
||||
ksp_store_schema_migrations
|
||||
version BIGINT PRIMARY KEY
|
||||
name TEXT NOT NULL
|
||||
checksum TEXT NOT NULL
|
||||
applied_at TIMESTAMPTZ NOT NULL
|
||||
```
|
||||
|
||||
## 9. Tests du moteur
|
||||
|
||||
Les unit tests `migration.rs` couvrent maintenant :
|
||||
|
||||
```text
|
||||
registre réel V000 immutable
|
||||
checksum V000 stable
|
||||
current version dérivée du registre
|
||||
registre synthétique V000/V001 ordonné
|
||||
historique V000 comme préfixe pending valide
|
||||
historique synthétique V000/V001 complet
|
||||
registre vide/non-zero/gap/name vide/SQL vide rejeté
|
||||
nom/checksum/history missing/gap rejetés
|
||||
version future -> SchemaNewer
|
||||
```
|
||||
|
||||
Un canari d'intégration source vérifie aussi :
|
||||
|
||||
```text
|
||||
registre embedded présent
|
||||
hook de migration + contextes `AppliedNow`/`Existing` présents
|
||||
RawNetworkId transmis au hook
|
||||
validation préfixe utilisée
|
||||
application pending utilisée
|
||||
aucun V001__raw_transaction.sql
|
||||
aucun ksp_store_identity
|
||||
```
|
||||
|
||||
## 10. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/migration.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
```
|
||||
|
||||
## 11. Fichier ajouté
|
||||
|
||||
```text
|
||||
deltas/0.3.3/pre.002.md
|
||||
```
|
||||
|
||||
## 12. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 13. Hors scope confirmé
|
||||
|
||||
Aucun changement n'est apporté à :
|
||||
|
||||
```text
|
||||
V001 physique
|
||||
ksp_store_identity
|
||||
tables/indexes RawTransaction
|
||||
mapping Row/SQL métier
|
||||
six capabilities RawTransaction
|
||||
ksp-store-lib dispatch
|
||||
RawAccountState
|
||||
Config std.store
|
||||
workers/jobs/apps
|
||||
codecs wire
|
||||
```
|
||||
|
||||
## 14. Validations exécutées dans l'environnement de génération
|
||||
|
||||
```text
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
General Rust rule audit: clean
|
||||
Rust export completeness audit: 0 candidate(s)
|
||||
KSP workspace Rust rule audit: clean
|
||||
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
Markdown table audit: clean
|
||||
```
|
||||
|
||||
`cargo`, `rustc` et `rustfmt` ne sont pas disponibles dans l'environnement d'assemblage. Les validations Cargo ci-dessous restent donc opérateur et ne sont jamais déclarées PASS ici.
|
||||
|
||||
## 15. Gate opérateur demandé
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Aucun test PostgreSQL live n'est requis : le SQL physique reste strictement V000 et aucun schéma métier n'est introduit.
|
||||
|
||||
## 16. Suite
|
||||
|
||||
Après gate vert, `0.3.3-pre.003` créera la vraie migration V001, les tables/constraints/indexes décidés en `pre.001` et le binding atomique `ksp_store_identity` via le hook préparé ici.
|
||||
273
deltas/0.3.3/pre.003-fix.001.md
Normal file
273
deltas/0.3.3/pre.003-fix.001.md
Normal file
@@ -0,0 +1,273 @@
|
||||
<!-- file: deltas/0.3.3/pre.003-fix.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.003-fix.001` — Store/PostgreSQL schema compatibility hardening
|
||||
|
||||
## 1. Base et objectif
|
||||
|
||||
Ce correctif s'applique **par-dessus le commit `0.3.3-pre.003` déjà réalisé**. Il ne réécrit ni ne remplace ce commit dans l'historique.
|
||||
|
||||
Le gate opérateur de `pre.002` est vert. Le gate opérateur de `pre.003` n'a pas été fourni avant l'ouverture de ce fix ; aucune commande Cargo du fix n'est donc déclarée PASS dans ce document.
|
||||
|
||||
Objectif du fix : conserver les bonnes garanties fonctionnelles historiques d'inspection/réconciliation du schéma PostgreSQL sans reprendre l'architecture kbot3, et corriger V001 avant l'ouverture du repository `RawTransaction` de `pre.004`.
|
||||
|
||||
Version workspace après application :
|
||||
|
||||
```text
|
||||
0.3.3-pre.3.fix.1
|
||||
```
|
||||
|
||||
Identifiant de livraison :
|
||||
|
||||
```text
|
||||
0.3.3-pre.003-fix.001
|
||||
```
|
||||
|
||||
## 2. Décisions corrigées
|
||||
|
||||
### 2.1 Migration logique vs ressources physiques
|
||||
|
||||
V000 et V001 restent des **migrations logiques** versionnées. Les objets physiques ne deviennent pas chacun une version de migration.
|
||||
|
||||
Arborescence retenue :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/
|
||||
v000_bootstrap/
|
||||
tables/
|
||||
001_ksp_store_schema_migrations.sql
|
||||
v001_raw_transaction/
|
||||
tables/
|
||||
001_ksp_store_identity.sql
|
||||
002_ksp_raw_transactions.sql
|
||||
003_ksp_raw_transaction_observations.sql
|
||||
004_ksp_raw_transaction_archive_payloads.sql
|
||||
constraints/
|
||||
001_...sql -> 035_...sql
|
||||
indexes/
|
||||
001_ix_ksp_raw_transactions_slot_signature.sql
|
||||
```
|
||||
|
||||
V001 contient exactement :
|
||||
|
||||
```text
|
||||
4 tables
|
||||
35 contraintes
|
||||
1 index
|
||||
40 ressources embedded
|
||||
```
|
||||
|
||||
V000 est seulement déplacée : ses bytes restent strictement identiques.
|
||||
|
||||
### 2.2 Checksums
|
||||
|
||||
V000 historique :
|
||||
|
||||
```text
|
||||
d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
```
|
||||
|
||||
V001 multi-ressources :
|
||||
|
||||
```text
|
||||
31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
Le checksum V001 est calculé sur le préfixe de format du registre puis, dans l'ordre embedded, sur l'identifiant relatif et les bytes exacts de chaque ressource. Le checksum monolithique `pre.003` n'est pas réutilisé.
|
||||
|
||||
Une base ayant réellement enregistré l'ancien checksum V001 du commit `pre.003` n'est jamais réécrite silencieusement : elle doit être réinitialisée si elle est jetable ou réconciliée manuellement.
|
||||
|
||||
### 2.3 `IF NOT EXISTS` n'est pas une preuve de compatibilité
|
||||
|
||||
Les ressources utilisent `CREATE ... IF NOT EXISTS`, `ADD COLUMN IF NOT EXISTS` ou des guards catalogue lorsque cela permet une création additive idempotente.
|
||||
|
||||
Après et avant mutation, le backend introspecte néanmoins le catalogue PostgreSQL. Chaque ressource est classée :
|
||||
|
||||
```text
|
||||
Compatible
|
||||
Missing
|
||||
Incompatible
|
||||
```
|
||||
|
||||
Un objet préexistant du bon nom mais de mauvaise définition est donc refusé.
|
||||
|
||||
### 2.4 Contrat physique vérifié
|
||||
|
||||
Le module backend-private `schema` contrôle notamment :
|
||||
|
||||
- tables gérées et nature `BASE TABLE` ;
|
||||
- colonnes obligatoires, type, nullabilité et absence d'identity/generated sur le contrat KSP ;
|
||||
- précision/scale `NUMERIC(20,0)` du slot ;
|
||||
- PK, FK et CHECK ;
|
||||
- définition canonique des contraintes à partir de leur propre ressource SQL ;
|
||||
- index partiel `(slot, signature)` hors `purged` ;
|
||||
- RLS ;
|
||||
- triggers utilisateur actifs ;
|
||||
- rewrite rules ;
|
||||
- contraintes/indexes uniques externes susceptibles de modifier la sémantique de write.
|
||||
|
||||
Les marqueurs COMMENT ne servent pas de preuve de compatibilité et ne sont pas requis.
|
||||
|
||||
### 2.5 Extensions externes compatibles
|
||||
|
||||
KSP ne requiert pas une égalité byte-for-byte du schéma complet avec son propre DDL. Les extensions externes qui sont prouvées non bloquantes restent admises.
|
||||
|
||||
Le correctif tolère notamment :
|
||||
|
||||
- une colonne externe nullable, sans default, identity ni generated expression ;
|
||||
- un index externe non unique ;
|
||||
- une contrainte attendue physiquement équivalente même si son nom diffère.
|
||||
|
||||
Le correctif bloque conservativement les extensions susceptibles de modifier/contraindre les writes KSP lorsqu'il ne peut pas en prouver l'innocuité : mauvais type/nullabilité sur une colonne KSP, colonne externe write-blocking, contrainte non équivalente, unique index autonome, trigger actif, rule ou RLS.
|
||||
|
||||
### 2.6 Politiques `schema_autocreate` / `schema_autoupdate`
|
||||
|
||||
`std.store` passe en format V2 et sépare :
|
||||
|
||||
```text
|
||||
schema_autocreate
|
||||
schema_autoupdate
|
||||
```
|
||||
|
||||
`schema_autocreate` autorise l'initialisation ou l'adoption contrôlée d'une base sans metadata KSP.
|
||||
|
||||
`schema_autoupdate` autorise les migrations pending et les réparations additives sûres de ressources manquantes sur un schéma KSP existant.
|
||||
|
||||
Une incompatibilité exigeant une mutation destructive, ambiguë ou non prouvée sûre reste bloquante même avec `schema_autoupdate=true`; le backend émet uniquement une phase/resource sûre pour permettre l'intervention manuelle.
|
||||
|
||||
Le format V1 reste lisible : `auto_migrate` est mappé vers les deux politiques. Les constructeurs Rust historiques restent également disponibles et mappent leur booléen vers les deux politiques.
|
||||
|
||||
### 2.7 Identité réseau
|
||||
|
||||
Le binding V001 reste sous la même transaction et le même advisory lock.
|
||||
|
||||
Une `ksp_store_identity` absente alors que V001 est déjà enregistrée n'est jamais reconstruite automatiquement, même avec `schema_autoupdate=true`, afin d'interdire tout rebind réseau silencieux.
|
||||
|
||||
## 3. Suppressions explicites après application de l'overlay
|
||||
|
||||
L'extraction d'un zip n'efface pas les anciens fichiers. Après extraction du delta à la racine du dépôt, supprimer explicitement :
|
||||
|
||||
```bash
|
||||
rm crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql
|
||||
rm crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
```
|
||||
|
||||
Fichiers supprimés :
|
||||
|
||||
- `crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql`
|
||||
|
||||
## 4. Fichiers modifiés
|
||||
|
||||
- `Cargo.toml`
|
||||
- `config/examples/std.store.example.json`
|
||||
- `config/schemas/std.store.schema.json`
|
||||
- `config/std.store.json`
|
||||
- `crates/ksp-config-lib/src/store.rs`
|
||||
- `crates/ksp-config-lib/unit_tests/fixtures/std.store.json`
|
||||
- `crates/ksp-config-lib/unit_tests/store.rs`
|
||||
- `crates/ksp-store-lib/src/settings.rs`
|
||||
- `crates/ksp-store-lib/src/store.rs`
|
||||
- `crates/ksp-store-lib/unit_tests/settings.rs`
|
||||
- `crates/ksp-store-postgres-lib/src/lib.rs`
|
||||
- `crates/ksp-store-postgres-lib/src/migration.rs`
|
||||
- `crates/ksp-store-postgres-lib/src/runtime.rs`
|
||||
- `crates/ksp-store-postgres-lib/tests/dependency_boundary.rs`
|
||||
- `crates/ksp-store-postgres-lib/tests/hardening_completeness.rs`
|
||||
- `crates/ksp-store-postgres-lib/unit_tests/migration.rs`
|
||||
- `docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md`
|
||||
- `docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`
|
||||
|
||||
## 5. Fichiers ajoutés
|
||||
|
||||
- `crates/ksp-store-postgres-lib/migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/001_pk_ksp_store_identity.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/002_ck_ksp_store_identity_singleton.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/003_ck_ksp_store_identity_network.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/004_pk_ksp_raw_transactions.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/005_ck_ksp_raw_transactions_signature.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/006_ck_ksp_raw_transactions_slot.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/007_ck_ksp_raw_transactions_block_time.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/008_ck_ksp_raw_transactions_format_id.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/009_ck_ksp_raw_transactions_format_version.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/010_ck_ksp_raw_transactions_content_hash.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/011_ck_ksp_raw_transactions_payload.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/012_ck_ksp_raw_transactions_retention_state.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/013_ck_ksp_raw_transactions_payload_state.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/014_ck_ksp_raw_transactions_purged_block_time.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/015_pk_ksp_raw_transaction_observations.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/016_fk_ksp_raw_transaction_observations_transaction.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/017_ck_ksp_raw_transaction_observations_key.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/018_ck_ksp_raw_transaction_observations_signature.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/019_ck_ksp_raw_transaction_observations_provider.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/020_ck_ksp_raw_transaction_observations_protocol.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/021_ck_ksp_raw_transaction_observations_method.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/022_ck_ksp_raw_transaction_observations_origin.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/023_ck_ksp_raw_transaction_observations_received_at.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/024_ck_ksp_raw_transaction_observations_capture_session.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/025_ck_ksp_raw_transaction_observations_commitment.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/026_ck_ksp_raw_transaction_observations_endpoint.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/027_ck_ksp_raw_transaction_observations_filter.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/028_ck_ksp_raw_transaction_observations_observed_at.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/029_ck_ksp_raw_transaction_observations_time_order.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/030_ck_ksp_raw_transaction_observations_source_hash.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/031_ck_ksp_raw_transaction_observations_source_size.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/032_pk_ksp_raw_transaction_archive_payloads.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/033_fk_ksp_raw_transaction_archive_payloads_transaction.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/034_ck_ksp_raw_transaction_archive_payloads_signature.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/constraints/035_ck_ksp_raw_transaction_archive_payloads_payload.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/indexes/001_ix_ksp_raw_transactions_slot_signature.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/tables/001_ksp_store_identity.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/tables/002_ksp_raw_transactions.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/tables/003_ksp_raw_transaction_observations.sql`
|
||||
- `crates/ksp-store-postgres-lib/migrations/v001_raw_transaction/tables/004_ksp_raw_transaction_archive_payloads.sql`
|
||||
- `crates/ksp-store-postgres-lib/src/schema.rs`
|
||||
- `crates/ksp-store-postgres-lib/unit_tests/schema.rs`
|
||||
- `deltas/0.3.3/pre.003-fix.001.md`
|
||||
|
||||
## 6. Validation de génération
|
||||
|
||||
Les validations réalisables dans l'environnement de génération doivent être propres avant publication de l'archive :
|
||||
|
||||
```text
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
validation JSON/JSON Schema std.store V1+V2
|
||||
inventaire/checksum migration V000/V001
|
||||
```
|
||||
|
||||
L'environnement de génération ne possède pas `cargo`/`rustfmt`. Les gates Cargo ci-dessous restent donc **NON EXÉCUTÉS** jusqu'au passage opérateur.
|
||||
|
||||
## 7. Gate opérateur demandé
|
||||
|
||||
Après extraction de l'overlay, suppression explicite des deux fichiers monolithiques puis application du formatage :
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Canaris de suppression :
|
||||
|
||||
```bash
|
||||
test ! -e crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql
|
||||
test ! -e crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
```
|
||||
|
||||
Le vrai PostgreSQL live du contrat V001, des dérives et des réparations reste volontairement réservé au gate live prévu plus tard dans `0.3.3`; ce fix ne le présente pas comme PASS.
|
||||
|
||||
## 8. Suite
|
||||
|
||||
Après gate opérateur vert de `pre.003-fix.001`, reprendre le séquencement avec :
|
||||
|
||||
```text
|
||||
0.3.3-pre.004 — mapping SQL privé + lectures RawTransaction/observation/rétention/tombstone
|
||||
```
|
||||
99
deltas/0.3.3/pre.003-fix.002.md
Normal file
99
deltas/0.3.3/pre.003-fix.002.md
Normal file
@@ -0,0 +1,99 @@
|
||||
<!-- file: deltas/0.3.3/pre.003-fix.002.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.003-fix.002` — correctif compilation et ownership du contrat PostgreSQL
|
||||
|
||||
## 1. Base et objectif
|
||||
|
||||
Ce correctif s’applique **par-dessus `0.3.3-pre.003-fix.001`**. Le gate opérateur de `fix.001` a confirmé que les audits Rust/Markdown étaient propres, mais a échoué sur deux défauts localisés :
|
||||
|
||||
1. `schema.rs` transmettait un `&str` directement dans deux slices de paramètres `tokio-postgres`, alors que le cast vers `&(dyn ToSql + Sync)` nécessite ici un `&&str` ;
|
||||
2. cinq constantes internes du contrat de colonnes commençaient par `KSP_`, namespace que les canaris Config réservent aux variables d’environnement runtime, ce qui provoquait à la fois un refus d’ownership Store et une fausse exigence `.env.example`.
|
||||
|
||||
Version workspace après application :
|
||||
|
||||
```text
|
||||
0.3.3-pre.3.fix.2
|
||||
```
|
||||
|
||||
Identifiant de livraison :
|
||||
|
||||
```text
|
||||
0.3.3-pre.003-fix.002
|
||||
```
|
||||
|
||||
## 2. Correctifs
|
||||
|
||||
### 2.1 Paramètres catalogue `tokio-postgres`
|
||||
|
||||
Les deux appels concernés deviennent :
|
||||
|
||||
```text
|
||||
transaction.query(UNEXPECTED_CONSTRAINTS_SQL, &[&table])
|
||||
transaction.query(UNIQUE_INDEXES_SQL, &[&table])
|
||||
```
|
||||
|
||||
Aucune requête SQL ni logique d’introspection n’est modifiée.
|
||||
|
||||
### 2.2 Namespace `KSP_` réservé à Config
|
||||
|
||||
Les constantes privées suivantes sont renommées sans changer leur contenu :
|
||||
|
||||
```text
|
||||
KSP_STORE_SCHEMA_MIGRATIONS_COLUMNS -> STORE_SCHEMA_MIGRATIONS_COLUMNS
|
||||
KSP_STORE_IDENTITY_COLUMNS -> STORE_IDENTITY_COLUMNS
|
||||
KSP_RAW_TRANSACTIONS_COLUMNS -> RAW_TRANSACTIONS_COLUMNS
|
||||
KSP_RAW_TRANSACTION_OBSERVATIONS_COLUMNS -> RAW_TRANSACTION_OBSERVATIONS_COLUMNS
|
||||
KSP_RAW_TRANSACTION_ARCHIVE_PAYLOADS_COLUMNS -> RAW_TRANSACTION_ARCHIVE_PAYLOADS_COLUMNS
|
||||
```
|
||||
|
||||
Le test ownership n’est pas assoupli et `.env.example` n’est pas pollué par de faux noms runtime.
|
||||
|
||||
## 3. Invariants inchangés
|
||||
|
||||
Ce fix ne modifie aucune ressource SQL. L’arborescence V000/V001, l’ordre embedded, le binding réseau et le contrat `Compatible/Missing/Incompatible` restent inchangés.
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 4. Fichiers modifiés
|
||||
|
||||
- `Cargo.toml`
|
||||
- `crates/ksp-store-postgres-lib/src/schema.rs`
|
||||
- `docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`
|
||||
|
||||
## 5. Fichier ajouté
|
||||
|
||||
- `deltas/0.3.3/pre.003-fix.002.md`
|
||||
|
||||
## 6. Suppressions
|
||||
|
||||
Aucune suppression supplémentaire dans `fix.002`. Les deux suppressions demandées par `fix.001` restent acquises :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql
|
||||
crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
```
|
||||
|
||||
## 7. Gate opérateur
|
||||
|
||||
Après extraction du delta par-dessus `fix.001` :
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le correctif ne doit être considéré validé qu’après gate Cargo opérateur vert.
|
||||
107
deltas/0.3.3/pre.003-fix.003.md
Normal file
107
deltas/0.3.3/pre.003-fix.003.md
Normal file
@@ -0,0 +1,107 @@
|
||||
<!-- file: deltas/0.3.3/pre.003-fix.003.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.003-fix.003` — clôture Clippy et relocalisation du canari live V000
|
||||
|
||||
## 1. Base et objectif
|
||||
|
||||
Ce correctif s’applique **par-dessus `0.3.3-pre.003-fix.002`**. Son gate opérateur confirme que les audits Rust/Markdown, `cargo check` jusqu’au backend et les tests Config/ownership sont désormais propres, mais révèle trois défauts résiduels strictement locaux :
|
||||
|
||||
1. une variable `name` décodée mais non utilisée dans l’inventaire des indexes uniques ;
|
||||
2. cinq usages de l’opérateur `?` dans `expected_constraint_definition`, interdits par la policy workspace `clippy::question_mark_used = deny` ;
|
||||
3. le test opt-in `postgres_foundation_live.rs` référence encore l’ancien fichier monolithique V000 supprimé par `fix.001`.
|
||||
|
||||
Un warning Clippy de test (`assert!(false, ...)`) est également supprimé afin de garder le gate sans bruit.
|
||||
|
||||
Version workspace après application :
|
||||
|
||||
```text
|
||||
0.3.3-pre.3.fix.3
|
||||
```
|
||||
|
||||
Identifiant de livraison :
|
||||
|
||||
```text
|
||||
0.3.3-pre.003-fix.003
|
||||
```
|
||||
|
||||
## 2. Correctifs
|
||||
|
||||
### 2.1 Introspection catalogue
|
||||
|
||||
Le nom d’index décodé mais inutilisé est explicitement marqué `_name`. Le comportement reste inchangé : tout index unique externe non adossé à une contrainte reste incompatible.
|
||||
|
||||
### 2.2 Policy Clippy `question_mark_used`
|
||||
|
||||
`expected_constraint_definition` conserve exactement sa sémantique `Option<String>`, mais remplace les cinq propagations `?` par des `match` explicites retournant `None` lorsque la ressource SQL ne peut pas être découpée selon le contrat attendu.
|
||||
|
||||
### 2.3 Canari unitaire
|
||||
|
||||
L’assertion constante `assert!(false, ...)` est remplacée par une assertion sur `resource.is_some()`, puis un `match` non paniquant pour satisfaire simultanément le canari et la policy KSP.
|
||||
|
||||
### 2.4 Test PostgreSQL live V000
|
||||
|
||||
Le `include_str!` du test opt-in suit maintenant le déplacement effectué dans `fix.001` :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql
|
||||
```
|
||||
|
||||
Aucun comportement live supplémentaire n’est ajouté.
|
||||
|
||||
### 2.5 Documentation backend
|
||||
|
||||
Le README et l’USAGE du backend sont réconciliés avec l’arborescence versionnée introduite par `fix.001`; ils ne présentent plus les anciens fichiers monolithiques supprimés comme artefacts actifs.
|
||||
|
||||
## 3. Invariants inchangés
|
||||
|
||||
Aucune ressource SQL n’est modifiée. Les checksums restent :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
Le contrat `schema_autocreate` / `schema_autoupdate`, l’introspection physique, le binding réseau et les 40 ressources V001 ne changent pas.
|
||||
|
||||
## 4. Fichiers modifiés
|
||||
|
||||
- `Cargo.toml`
|
||||
- `crates/ksp-store-postgres-lib/README.md`
|
||||
- `crates/ksp-store-postgres-lib/USAGE.md`
|
||||
- `crates/ksp-store-postgres-lib/src/schema.rs`
|
||||
- `crates/ksp-store-postgres-lib/tests/postgres_foundation_live.rs`
|
||||
- `crates/ksp-store-postgres-lib/unit_tests/schema.rs`
|
||||
- `docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md`
|
||||
|
||||
## 5. Fichier ajouté
|
||||
|
||||
- `deltas/0.3.3/pre.003-fix.003.md`
|
||||
|
||||
## 6. Suppressions
|
||||
|
||||
Aucune suppression supplémentaire. Les suppressions de `fix.001` restent acquises :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/V000__bootstrap.sql
|
||||
crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
```
|
||||
|
||||
## 7. Gate opérateur
|
||||
|
||||
Après extraction par-dessus `fix.002` :
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le correctif n’est validé qu’après gate opérateur entièrement vert.
|
||||
353
deltas/0.3.3/pre.003.md
Normal file
353
deltas/0.3.3/pre.003.md
Normal file
@@ -0,0 +1,353 @@
|
||||
<!-- file: deltas/0.3.3/pre.003.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.003` — V001 RawTransaction physique et binding réseau
|
||||
|
||||
## 1. Base requise
|
||||
|
||||
```text
|
||||
0.3.3-pre.002
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `pre.002` est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
audit Rust général / exports / workspace PASS
|
||||
audit Markdown PASS — 214 tables / 132 files
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS
|
||||
cargo test -p ksp-config-lib PASS
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
Le test PostgreSQL foundation reste `#[ignore]` dans ce gate ordinaire. La preuve live complète de la vertical slice `0.3.3` reste réservée à `pre.009`.
|
||||
|
||||
## 2. Objectif
|
||||
|
||||
Matérialiser le design physique décidé en `pre.001` sur le moteur multi-version acquis en `pre.002`, sans commencer encore les repositories/capabilities métier.
|
||||
|
||||
La tranche livre :
|
||||
|
||||
```text
|
||||
V001__raw_transaction.sql
|
||||
registre embedded réel V000 + V001
|
||||
binding mono-réseau atomique
|
||||
contraintes/indexes RAW exacts
|
||||
checksum/inventory/bounds statiques
|
||||
code stable PostgreSQL Compacted unsupported
|
||||
```
|
||||
|
||||
## 3. Version
|
||||
|
||||
Le workspace passe à :
|
||||
|
||||
```text
|
||||
0.3.3-pre.3
|
||||
```
|
||||
|
||||
## 4. Migration V001
|
||||
|
||||
La migration ajoutée est :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
```
|
||||
|
||||
Le registre embedded contient désormais exactement :
|
||||
|
||||
```text
|
||||
V000 bootstrap
|
||||
V001 raw_transaction
|
||||
```
|
||||
|
||||
La version courante dérivée du registre vaut donc :
|
||||
|
||||
```text
|
||||
1
|
||||
```
|
||||
|
||||
V000 reste byte-identique avec son checksum historique :
|
||||
|
||||
```text
|
||||
d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
```
|
||||
|
||||
Checksum SHA-256 V001 :
|
||||
|
||||
```text
|
||||
6fe57ed0313d2ed295280dd6e49f6d86695d4e4effee2724a25e36db1ea17761
|
||||
```
|
||||
|
||||
## 5. Inventaire physique V001
|
||||
|
||||
V001 crée exactement les tables métier/identité suivantes :
|
||||
|
||||
```text
|
||||
ksp_store_identity
|
||||
ksp_raw_transactions
|
||||
ksp_raw_transaction_observations
|
||||
ksp_raw_transaction_archive_payloads
|
||||
```
|
||||
|
||||
et l'index secondaire suivant :
|
||||
|
||||
```text
|
||||
ix_ksp_raw_transactions_slot_signature
|
||||
ON ksp_raw_transactions(slot, signature)
|
||||
WHERE retention_state <> 'purged'
|
||||
```
|
||||
|
||||
Les clés physiques restent :
|
||||
|
||||
```text
|
||||
ksp_store_identity(singleton)
|
||||
ksp_raw_transactions(signature)
|
||||
ksp_raw_transaction_observations(observation_key)
|
||||
ksp_raw_transaction_archive_payloads(signature)
|
||||
```
|
||||
|
||||
Aucun ID SQL auxiliaire n'est ajouté.
|
||||
|
||||
## 6. Bornes et contraintes
|
||||
|
||||
Les contraintes SQL matérialisent les invariants backend-agnostiques déjà acquis :
|
||||
|
||||
```text
|
||||
network/code UTF-8 ASCII sûr 1..=128 octets
|
||||
signature transaction 64 octets
|
||||
observation key 32 octets
|
||||
content/source hash 32 octets
|
||||
slot NUMERIC(20,0), 0..=u64::MAX
|
||||
format_version BIGINT, 1..=u32::MAX
|
||||
RawTimestamp BIGINT, 0..=253402300799999
|
||||
payload canonical/archive 1..=16 MiB lorsqu'il existe
|
||||
source payload size 0..=64 MiB
|
||||
origin backfill/import/live/repair/replay
|
||||
retention physique full/archived/purged
|
||||
```
|
||||
|
||||
`slot` n'est jamais réduit en `BIGINT` signé.
|
||||
|
||||
La cohérence de rétention physique impose :
|
||||
|
||||
```text
|
||||
full -> payload hot présent
|
||||
archived -> payload hot absent
|
||||
purged -> payload hot absent + block_time absent
|
||||
```
|
||||
|
||||
La relation archive reste séparée du hot path. V001 ne prétend jamais représenter `Compacted`.
|
||||
|
||||
## 7. Binding mono-réseau atomique
|
||||
|
||||
V001 déclare le hook privé :
|
||||
|
||||
```text
|
||||
MigrationHook::StoreIdentity
|
||||
```
|
||||
|
||||
Le comportement est distinct selon le contexte déjà acquis en `pre.002`.
|
||||
|
||||
### `AppliedNow`
|
||||
|
||||
Sous la même transaction et le même advisory lock :
|
||||
|
||||
```text
|
||||
batch_execute(V001)
|
||||
-> INSERT singleton=1 + network runtime
|
||||
-> relire exactement l'identité
|
||||
-> valider singleton + RawNetworkId + égalité réseau
|
||||
-> INSERT history V001
|
||||
-> commit final
|
||||
```
|
||||
|
||||
L'insertion n'utilise aucun upsert : une divergence physique ne peut pas être absorbée silencieusement.
|
||||
|
||||
### `Existing`
|
||||
|
||||
Une réouverture V001 effectue uniquement :
|
||||
|
||||
```text
|
||||
SELECT singleton, network
|
||||
-> exactement une ligne
|
||||
-> singleton = 1
|
||||
-> network valide RawNetworkId
|
||||
-> network == backend.network
|
||||
```
|
||||
|
||||
Une identité absente, multiple, illisible, malformée ou différente devient `MigrationMismatch` avec une phase statique. La valeur réseau persistée et les erreurs PostgreSQL ne sont jamais retenues/rendues.
|
||||
|
||||
Une base V001 dont l'identité a disparu n'est donc jamais réclamée/rebindée automatiquement.
|
||||
|
||||
## 8. `Compacted` PostgreSQL non supporté
|
||||
|
||||
Le code stable suivant est désormais matérialisé par valeur dans le backend et la façade :
|
||||
|
||||
```text
|
||||
store.postgres_retention_compaction_unsupported
|
||||
```
|
||||
|
||||
Exports :
|
||||
|
||||
```text
|
||||
ksp_store_postgres_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED
|
||||
ksp_store_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED
|
||||
```
|
||||
|
||||
Les deux crates construisent le même `ErrorCode` KSP sans créer de dépendance inverse `ksp-store-postgres-lib -> ksp-store-lib`.
|
||||
|
||||
L'utilisation opérationnelle de ce code par `RawTransactionRetentionWrite` reste réservée à `pre.007`.
|
||||
|
||||
## 9. Tests statiques ajoutés/actualisés
|
||||
|
||||
Les tests de migration couvrent désormais :
|
||||
|
||||
```text
|
||||
registre réel [V000,V001]
|
||||
V000 checksum immuable
|
||||
V001 checksum exact
|
||||
version courante = 1
|
||||
V000 seul = préfixe valide / V001 pending
|
||||
historique complet V000+V001 valide
|
||||
mismatch/newer toujours terminaux
|
||||
inventaire exact des quatre tables
|
||||
index partiel exact
|
||||
slot u64 max sans narrowing
|
||||
bornes u32/timestamp/payload/source payload
|
||||
retention physique sans compacted
|
||||
hook StoreIdentity réel
|
||||
contextes AppliedNow / Existing
|
||||
absence de repository/capability dans la tranche
|
||||
```
|
||||
|
||||
Les canaris de surface publique sont actualisés :
|
||||
|
||||
```text
|
||||
ksp-store-postgres-lib root pub use : 7 -> 8
|
||||
ksp-store-lib root pub use : 84 -> 85
|
||||
```
|
||||
|
||||
La seule nouvelle surface est le code d'erreur PostgreSQL `Compacted` explicitement planifié.
|
||||
|
||||
Le test live foundation hérité de `0.3.2` est maintenu cohérent avec V001 sans devenir la preuve métier de cette release :
|
||||
|
||||
```text
|
||||
version health attendue 0 -> 1
|
||||
refus de départ toute table Store V000/V001 déjà présente
|
||||
cleanup ensemble des tables Store V000/V001 possédées
|
||||
preuve de rollback historique V000 inchangée
|
||||
statut dans le gate ordinaire toujours #[ignore]
|
||||
```
|
||||
|
||||
Il n'est pas exécuté dans cette tranche ; les scénarios V001 spécifiques restent réservés à `pre.009`.
|
||||
|
||||
## 10. Preuves volontairement différées
|
||||
|
||||
Cette tranche ne prétend pas avoir exécuté PostgreSQL réel pour V001.
|
||||
|
||||
Restent au gate live `pre.009` :
|
||||
|
||||
```text
|
||||
upgrade réel V000 -> V001
|
||||
rollback réel si hook échoue
|
||||
réouverture même réseau
|
||||
réouverture réseau différent
|
||||
identity supprimée/malformée sur DB réelle
|
||||
checksum V001 divergent sur DB réelle
|
||||
concurrence réelle du bootstrap V001
|
||||
```
|
||||
|
||||
Ces points ne sont donc pas marqués PASS dans la validation `pre.003`.
|
||||
|
||||
## 11. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-lib/src/error.rs
|
||||
crates/ksp-store-lib/src/lib.rs
|
||||
crates/ksp-store-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/src/error.rs
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/migration.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/postgres_foundation_live.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
```
|
||||
|
||||
## 12. Fichiers ajoutés
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/migrations/V001__raw_transaction.sql
|
||||
deltas/0.3.3/pre.003.md
|
||||
```
|
||||
|
||||
## 13. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 14. Hors scope confirmé
|
||||
|
||||
Aucun changement n'implémente encore :
|
||||
|
||||
```text
|
||||
RawTransactionRead
|
||||
RawTransactionWrite
|
||||
RawTransactionObservationRead
|
||||
RawTransactionObservationWrite
|
||||
RawTransactionRetentionRead
|
||||
RawTransactionRetentionWrite
|
||||
row codecs SQL métier
|
||||
get/list/cursor
|
||||
canonical+observation writes
|
||||
archive/purge/rehydrate behavior
|
||||
ksp-store-lib capability dispatch
|
||||
RawAccountState PostgreSQL
|
||||
Config std.store
|
||||
workers/jobs/apps
|
||||
```
|
||||
|
||||
## 15. Validations exécutées dans l'environnement de génération
|
||||
|
||||
```text
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
General Rust rule audit: clean
|
||||
Rust export completeness audit: 0 candidate(s)
|
||||
KSP workspace Rust rule audit: clean
|
||||
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
Markdown table audit: clean (214 table(s), 133 file(s))
|
||||
```
|
||||
|
||||
Les commandes Cargo ne sont pas disponibles dans l'environnement de génération. Elles restent **NON EXÉCUTÉES** ici et doivent être exécutées côté opérateur.
|
||||
|
||||
## 16. Gate opérateur demandé
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le test PostgreSQL live demeure hors gate ordinaire jusqu'à la tranche dédiée `pre.009`.
|
||||
|
||||
## 17. Suite prévue après gate vert
|
||||
|
||||
```text
|
||||
0.3.3-pre.004 — mapping SQL privé + lectures RawTransaction/observation/rétention/tombstone
|
||||
```
|
||||
323
deltas/0.3.3/pre.004.md
Normal file
323
deltas/0.3.3/pre.004.md
Normal file
@@ -0,0 +1,323 @@
|
||||
<!-- file: deltas/0.3.3/pre.004.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.004` — mapping PostgreSQL privé et lectures RawTransaction
|
||||
|
||||
## 1. Base requise
|
||||
|
||||
```text
|
||||
0.3.3-pre.3.fix.3
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `pre.003-fix.003` est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
audit Rust général / exports / workspace PASS
|
||||
audit Markdown PASS — 214 tables / 136 files
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS — 19 unit tests + canaris, live ignored
|
||||
cargo test -p ksp-config-lib PASS — 128 unit tests + ownership/public API
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
La V001 split, son contrat catalogue et les politiques `schema_autocreate/schema_autoupdate` sont donc considérés acquis.
|
||||
|
||||
## 2. Objectif
|
||||
|
||||
Implémenter la tranche lecture de la vertical slice PostgreSQL `RawTransaction` sans ouvrir encore les écritures ni la pagination :
|
||||
|
||||
```text
|
||||
mapping SQL privé
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Les méthodes du backend retournent uniquement des modèles `ksp-store-api`. Aucun `tokio_postgres::Row`, SQL, SQLSTATE ou bind ne traverse le bridge public.
|
||||
|
||||
## 3. Version
|
||||
|
||||
Le workspace passe à :
|
||||
|
||||
```text
|
||||
0.3.3-pre.4
|
||||
```
|
||||
|
||||
## 4. Module physique privé
|
||||
|
||||
Nouveau module :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
```
|
||||
|
||||
Il possède les SELECT et les codecs physiques de cette tranche. `runtime.rs` ne contient aucun SQL métier et délègue les quatre lectures au module privé.
|
||||
|
||||
La tranche reste strictement read-only :
|
||||
|
||||
```text
|
||||
INSERT INTO absent
|
||||
UPDATE absent
|
||||
DELETE FROM absent
|
||||
```
|
||||
|
||||
Les écritures canonique/observation restent `pre.005`; `list_raw_transactions` et le cursor restent `pre.006`.
|
||||
|
||||
## 5. Conversion physique exacte
|
||||
|
||||
Le mapping respecte les contrats `ksp-store-api` sans narrowing :
|
||||
|
||||
```text
|
||||
slot NUMERIC(20,0) -> PostgreSQL ::text -> parse u64
|
||||
format_version BIGINT -> u32::try_from
|
||||
block/observed/received BIGINT -> u64::try_from -> RawTimestamp
|
||||
source payload size BIGINT -> u64::try_from + borne API
|
||||
signature -> BYTEA exactement 64 bytes
|
||||
hash/key -> BYTEA exactement 32 bytes
|
||||
format/provenance codes -> constructeurs API fallibles
|
||||
```
|
||||
|
||||
Le chemin `slot` couvre explicitement :
|
||||
|
||||
```text
|
||||
0
|
||||
i64::MAX
|
||||
i64::MAX + 1
|
||||
u64::MAX
|
||||
```
|
||||
|
||||
Aucune dépendance décimale supplémentaire n'est introduite.
|
||||
|
||||
## 6. `get_raw_transaction`
|
||||
|
||||
La requête lit la ligne canonique avec `LEFT JOIN` du payload archive.
|
||||
|
||||
Décodage :
|
||||
|
||||
```text
|
||||
Full
|
||||
payload chaud obligatoire
|
||||
payload archive interdit
|
||||
-> Some(RawTransaction)
|
||||
|
||||
Archived
|
||||
payload chaud absent
|
||||
payload archive obligatoire
|
||||
-> Some(RawTransaction)
|
||||
|
||||
Purged
|
||||
payload chaud absent
|
||||
payload archive absent
|
||||
block_time absent
|
||||
-> None
|
||||
```
|
||||
|
||||
Toute incohérence de forme, entier hors domaine, code invalide, signature/hash mal dimensionné ou payload incompatible produit `DataInvalid` sans conserver la valeur hostile.
|
||||
|
||||
## 7. Observation
|
||||
|
||||
`get_raw_transaction_observation` reconstruit :
|
||||
|
||||
```text
|
||||
observation_key
|
||||
transaction signature
|
||||
provider
|
||||
protocol
|
||||
acquisition_method
|
||||
origin
|
||||
received_at
|
||||
capture_session_id optionnel
|
||||
commitment optionnel
|
||||
endpoint_id optionnel
|
||||
filter_id optionnel
|
||||
observed_at optionnel
|
||||
source_payload_hash optionnel
|
||||
source_payload_size_bytes optionnel
|
||||
```
|
||||
|
||||
`RawObservationKey` ne porte pas de réseau ; le `RawTransactionReference` reconstruit utilise donc exclusivement le réseau mono-backend déjà vérifié à l'ouverture.
|
||||
|
||||
L'ordre temporel `observed_at <= received_at` est revalidé par le constructeur API.
|
||||
|
||||
## 8. Rétention et tombstone
|
||||
|
||||
`get_raw_transaction_retention_state` mappe uniquement les états physiques supportés par V001 :
|
||||
|
||||
```text
|
||||
full
|
||||
archived
|
||||
purged
|
||||
```
|
||||
|
||||
Une valeur inconnue ou `compacted` stockée physiquement est une corruption `DataInvalid`; `Compacted` n'a toujours aucune représentation PostgreSQL prétendue.
|
||||
|
||||
`get_raw_transaction_tombstone` retourne `Some` uniquement pour `Purged`, vérifie également que `block_time` est absent, puis reconstruit exactement :
|
||||
|
||||
```text
|
||||
network + signature
|
||||
slot
|
||||
format_id
|
||||
format_version
|
||||
content_hash
|
||||
```
|
||||
|
||||
Pour `Full` ou `Archived`, le résultat est `None`.
|
||||
|
||||
## 9. Réseau pré-I/O
|
||||
|
||||
Les trois lectures portant un `RawTransactionReference` passent par un garde privé commun avant `pool.get()` :
|
||||
|
||||
```text
|
||||
reference.network == backend.network -> continuer
|
||||
sinon -> WrongNetwork
|
||||
```
|
||||
|
||||
Le mauvais réseau ne consomme donc aucune connexion PostgreSQL et ne dépend d'aucun SQL.
|
||||
|
||||
## 10. Erreurs backend
|
||||
|
||||
`PostgresBackendErrorKind` ajoute :
|
||||
|
||||
```text
|
||||
DataInvalid
|
||||
ReadFailed
|
||||
WrongNetwork
|
||||
```
|
||||
|
||||
Classification :
|
||||
|
||||
```text
|
||||
pool wait/connect -> kinds existants PoolTimeout/ConnectFailed
|
||||
SELECT/driver read -> ReadFailed
|
||||
row/cardinality/model -> DataInvalid
|
||||
reference mauvais réseau -> WrongNetwork
|
||||
```
|
||||
|
||||
`PostgresBackendError` reste composé uniquement de :
|
||||
|
||||
```text
|
||||
kind
|
||||
phase &'static str
|
||||
```
|
||||
|
||||
Aucune erreur externe n'est retenue ni rendue.
|
||||
|
||||
## 11. Surface backend
|
||||
|
||||
`PostgresBackend` expose désormais :
|
||||
|
||||
```text
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Ces méthodes ne constituent pas encore les implémentations finales des traits `RawTransaction*` : `RawTransactionRead` exige également `list_raw_transactions`, réservé à `pre.006`. Les six traits seront complets avant leur dispatch par `ksp-store-lib` en `pre.008`.
|
||||
|
||||
## 12. Tests déterministes
|
||||
|
||||
Le nouveau miroir `unit_tests/raw_transaction.rs` couvre notamment :
|
||||
|
||||
```text
|
||||
Full u64::MAX
|
||||
Archived u64::MAX
|
||||
Purged -> None
|
||||
résidu payload/block_time sur Purged -> DataInvalid
|
||||
provenance complète/optionnelle
|
||||
source payload max
|
||||
slot > u64::MAX -> DataInvalid
|
||||
hash mal dimensionné -> DataInvalid
|
||||
origin hostile -> DataInvalid sans écho
|
||||
états Full/Archived/Purged
|
||||
Compacted physique -> DataInvalid
|
||||
tombstone u64::MAX/u32::MAX
|
||||
wrong-network pré-I/O
|
||||
```
|
||||
|
||||
Les canaris d'intégration figent également :
|
||||
|
||||
```text
|
||||
module SQL privé
|
||||
aucun write SQL en pre.004
|
||||
aucune dépendance nouvelle
|
||||
surface publique sans types PostgreSQL
|
||||
nouveaux error kinds sûrs
|
||||
```
|
||||
|
||||
## 13. Migrations
|
||||
|
||||
Aucune ressource de migration n'est modifiée.
|
||||
|
||||
Les checksums restent :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 14. Fichiers ajoutés
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
deltas/0.3.3/pre.004.md
|
||||
```
|
||||
|
||||
## 15. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/src/error.rs
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
```
|
||||
|
||||
## 16. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 17. Hors scope confirmé
|
||||
|
||||
Aucun changement n'est apporté à :
|
||||
|
||||
```text
|
||||
ksp-store-api contrats
|
||||
ksp-store-lib dispatch métier
|
||||
écriture RawTransaction
|
||||
écriture observation
|
||||
pagination/cursor
|
||||
transition rétention
|
||||
RawAccountState
|
||||
worker/job/app
|
||||
migration SQL V000/V001
|
||||
```
|
||||
|
||||
## 18. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
La tranche n'est validée qu'après gate opérateur entièrement vert.
|
||||
324
deltas/0.3.3/pre.005.md
Normal file
324
deltas/0.3.3/pre.005.md
Normal file
@@ -0,0 +1,324 @@
|
||||
<!-- file: deltas/0.3.3/pre.005.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.005` — écriture atomique RawTransaction + observation
|
||||
|
||||
## 1. Base requise
|
||||
|
||||
```text
|
||||
0.3.3-pre.4
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `pre.004` est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
audit Rust général / exports / workspace PASS
|
||||
audit Markdown PASS — 214 tables / 137 files
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS — 25 unit tests + canaris, live ignored
|
||||
cargo test -p ksp-config-lib PASS — 128 unit tests + ownership/public API
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
Les lectures physiques `RawTransaction` de `pre.004` sont donc acquises.
|
||||
|
||||
## 2. Objectif
|
||||
|
||||
Implémenter la tranche écriture de la vertical slice PostgreSQL sans ouvrir encore pagination ni transitions mutantes de rétention :
|
||||
|
||||
```text
|
||||
persist_raw_transaction_acquisition
|
||||
record_raw_transaction_observation
|
||||
```
|
||||
|
||||
La première opération doit rendre le canonical et son observation atomiques. Les deux opérations doivent être réellement idempotentes et ne jamais confondre une collision de clé avec un contenu identique.
|
||||
|
||||
## 3. Version
|
||||
|
||||
Le workspace passe à :
|
||||
|
||||
```text
|
||||
0.3.3-pre.5
|
||||
```
|
||||
|
||||
## 4. Pré-I/O
|
||||
|
||||
Avant `pool.get()` :
|
||||
|
||||
```text
|
||||
transaction.reference.network == backend.network
|
||||
observation.transaction.network == backend.network
|
||||
observation.transaction == transaction.reference
|
||||
```
|
||||
|
||||
Un mauvais réseau retourne `WrongNetwork`. Une acquisition composée de deux références logiques différentes retourne `Conflict` sans toucher PostgreSQL.
|
||||
|
||||
`record_raw_transaction_observation` valide également son réseau avant toute acquisition du pool.
|
||||
|
||||
## 5. Acquisition atomique canonique + observation
|
||||
|
||||
L'algorithme est :
|
||||
|
||||
```text
|
||||
BEGIN
|
||||
INSERT canonical
|
||||
ON CONFLICT (signature) DO NOTHING
|
||||
RETURNING signature
|
||||
|
||||
si inséré
|
||||
-> candidate Inserted
|
||||
|
||||
sinon
|
||||
SELECT canonical FOR UPDATE
|
||||
SELECT archive payload éventuel
|
||||
comparer le contenu réel
|
||||
|
||||
écrire/idempotenter observation
|
||||
|
||||
COMMIT
|
||||
```
|
||||
|
||||
Toute erreur après l'insert canonique, notamment une collision d'observation divergente, fait sortir sans commit : PostgreSQL rollbacke donc l'ensemble de l'acquisition.
|
||||
|
||||
Aucun `has_*` ni SELECT préventif n'est introduit.
|
||||
|
||||
## 6. Idempotence canonique
|
||||
|
||||
Pour `Full` et `Archived`, l'égalité exige exactement :
|
||||
|
||||
```text
|
||||
reference
|
||||
slot
|
||||
block_time
|
||||
format_id
|
||||
format_version
|
||||
content_hash
|
||||
payload bytes
|
||||
```
|
||||
|
||||
Le hash n'est jamais utilisé seul lorsque les octets restent disponibles.
|
||||
|
||||
Résultats :
|
||||
|
||||
```text
|
||||
première insertion -> Inserted
|
||||
même identité + contenu identique -> AlreadyPresent
|
||||
même identité + contenu divergent -> Conflict
|
||||
```
|
||||
|
||||
Le conflit backend sera projeté vers `ERROR_CODE_RAW_CONFLICT` par la façade dans `pre.008`.
|
||||
|
||||
## 7. Tombstone et ForceRehydrate
|
||||
|
||||
Lorsqu'une signature existe en `Purged`, le backend exige que la forme physique soit réellement minimale :
|
||||
|
||||
```text
|
||||
payload hot absent
|
||||
payload archive absent
|
||||
block_time absent
|
||||
```
|
||||
|
||||
Il compare ensuite uniquement les métadonnées conservées par le tombstone :
|
||||
|
||||
```text
|
||||
reference
|
||||
slot
|
||||
format_id
|
||||
format_version
|
||||
content_hash
|
||||
```
|
||||
|
||||
Une divergence produit `Conflict`.
|
||||
|
||||
Pour un tombstone compatible :
|
||||
|
||||
```text
|
||||
Normal
|
||||
-> SkippedPurged / NotRecorded
|
||||
|
||||
ForceRehydrate
|
||||
-> UPDATE payload + block_time + state=full
|
||||
-> observation dans la même transaction
|
||||
-> Rehydrated
|
||||
```
|
||||
|
||||
Le `ForceRehydrate` ne masque donc jamais une divergence détectable.
|
||||
|
||||
## 8. Observation idempotente
|
||||
|
||||
L'observation utilise :
|
||||
|
||||
```text
|
||||
INSERT ...
|
||||
ON CONFLICT (observation_key) DO NOTHING
|
||||
RETURNING observation_key
|
||||
```
|
||||
|
||||
Après collision :
|
||||
|
||||
```text
|
||||
SELECT observation ... FOR UPDATE
|
||||
```
|
||||
|
||||
Puis le modèle reconstruit est comparé intégralement à l'observation entrante : référence transaction + provenance complète.
|
||||
|
||||
Résultats :
|
||||
|
||||
```text
|
||||
key absente + insert réussi -> Inserted
|
||||
key présente + contenu identique -> AlreadyPresent
|
||||
key présente + contenu divergent -> Conflict
|
||||
```
|
||||
|
||||
## 9. Observation write séparée
|
||||
|
||||
`record_raw_transaction_observation` ne crée jamais un canonical implicite.
|
||||
|
||||
Sous transaction :
|
||||
|
||||
```text
|
||||
SELECT retention_state FROM canonical FOR UPDATE
|
||||
```
|
||||
|
||||
Puis :
|
||||
|
||||
```text
|
||||
canonical absent -> ReferenceNotFound
|
||||
canonical Purged -> NotRecorded
|
||||
Full / Archived -> insertion/idempotence observation
|
||||
```
|
||||
|
||||
Le verrou canonical prépare aussi la sérialisation future avec purge/rehydrate de `pre.007`.
|
||||
|
||||
## 10. Erreurs backend
|
||||
|
||||
`PostgresBackendErrorKind` ajoute :
|
||||
|
||||
```text
|
||||
Conflict
|
||||
ReferenceNotFound
|
||||
WriteFailed
|
||||
```
|
||||
|
||||
Ces erreurs restent composées exclusivement de :
|
||||
|
||||
```text
|
||||
kind
|
||||
phase &'static str
|
||||
```
|
||||
|
||||
Aucun texte PostgreSQL, SQLSTATE, query, bind, URI, signature, hash ou payload n'est retenu.
|
||||
|
||||
## 11. Surface backend
|
||||
|
||||
`PostgresBackend` expose désormais en plus :
|
||||
|
||||
```text
|
||||
persist_raw_transaction_acquisition
|
||||
record_raw_transaction_observation
|
||||
```
|
||||
|
||||
Les traits `RawTransactionWrite` / `RawTransactionObservationWrite` ne sont pas encore implémentés directement : la tranche finale des six traits attend `list_raw_transactions` en `pre.006`, puis la rétention mutante en `pre.007` et le dispatch façade en `pre.008`.
|
||||
|
||||
## 12. Tests déterministes
|
||||
|
||||
Le miroir `unit_tests/raw_transaction.rs` couvre en plus :
|
||||
|
||||
```text
|
||||
pré-I/O réseau + référence acquisition exacte
|
||||
égalité canonique stricte
|
||||
même hash mais payload différent -> Conflict
|
||||
tombstone compatible -> Purged match
|
||||
tombstone divergent -> Conflict
|
||||
mapping exact des origins d'observation
|
||||
```
|
||||
|
||||
Les canaris d'intégration figent :
|
||||
|
||||
```text
|
||||
INSERT unique direct sans has_*
|
||||
FOR UPDATE après collision
|
||||
observation unique idempotente
|
||||
ForceRehydrate limité à UPDATE
|
||||
aucun DELETE
|
||||
aucune pagination/cursor
|
||||
aucune transition retention mutante
|
||||
aucune impl complète de trait prématurée
|
||||
```
|
||||
|
||||
La concurrence PostgreSQL réelle et les preuves de rollback restent réservées au test live `pre.009`.
|
||||
|
||||
## 13. Migrations
|
||||
|
||||
Aucune ressource de migration n'est modifiée.
|
||||
|
||||
Checksums inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 14. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/src/error.rs
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
```
|
||||
|
||||
## 15. Fichiers ajoutés
|
||||
|
||||
```text
|
||||
deltas/0.3.3/pre.005.md
|
||||
```
|
||||
|
||||
## 16. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 17. Hors scope confirmé
|
||||
|
||||
Aucun changement n'est apporté à :
|
||||
|
||||
```text
|
||||
ksp-store-api contrats
|
||||
ksp-store-lib dispatch métier
|
||||
list_raw_transactions / cursor
|
||||
transition Full -> Archived -> Purged
|
||||
Compacted physique
|
||||
RawAccountState
|
||||
worker/job/app
|
||||
migration SQL V000/V001
|
||||
```
|
||||
|
||||
## 18. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le test PostgreSQL live de fondation reste `#[ignore]`. Aucune URI réelle n'est requise pour cette tranche.
|
||||
92
deltas/0.3.3/pre.006-fix.001.md
Normal file
92
deltas/0.3.3/pre.006-fix.001.md
Normal file
@@ -0,0 +1,92 @@
|
||||
<!-- file: deltas/0.3.3/pre.006-fix.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.006-fix.001` — correction gate pagination
|
||||
|
||||
## 1. Base
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.6
|
||||
```
|
||||
|
||||
Le gate opérateur de `pre.006` a confirmé les audits Rust/Markdown et la compilation de la majorité du workspace, puis a détecté deux défauts locaux dans `ksp-store-postgres-lib` :
|
||||
|
||||
```text
|
||||
unused import: RawTransactionDecodedCursor
|
||||
hardening_completeness.rs: argument never used
|
||||
```
|
||||
|
||||
Aucun défaut fonctionnel de pagination, cursor ou migration n'a été signalé avant cet arrêt.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.6.fix.1
|
||||
```
|
||||
|
||||
## 3. Corrections
|
||||
|
||||
### 3.1 Re-export interne inutilisé
|
||||
|
||||
Le re-export crate-root suivant est supprimé :
|
||||
|
||||
```text
|
||||
RawTransactionDecodedCursor
|
||||
```
|
||||
|
||||
Le type reste privé dans `raw_transaction::cursor` et continue d'être utilisé directement par le codec qui le produit. Aucun contrat public ou comportement cursor n'est modifié.
|
||||
|
||||
### 3.2 Canari de hardening
|
||||
|
||||
`pre_009_backend_has_no_env_bypass_or_direct_store_trait_implementation` concatène huit sources backend, dont `schema.rs`, mais son format string ne contenait que sept placeholders. Le format string est corrigé à huit placeholders ; le canari conserve exactement le même périmètre d'inspection.
|
||||
|
||||
## 4. Invariants
|
||||
|
||||
Ce fix ne modifie pas :
|
||||
|
||||
```text
|
||||
raw_transaction.rs
|
||||
raw_transaction/cursor.rs
|
||||
SQL de lecture/écriture/pagination
|
||||
migrations V000/V001
|
||||
format cursor V1 109 octets
|
||||
digest/binding cursor
|
||||
keyset ASC/DESC
|
||||
limite physique requested <= i64::MAX - 1
|
||||
```
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 5. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.006-fix.001.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 6. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
311
deltas/0.3.3/pre.006.md
Normal file
311
deltas/0.3.3/pre.006.md
Normal file
@@ -0,0 +1,311 @@
|
||||
<!-- file: deltas/0.3.3/pre.006.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.006` — pagination keyset et cursor V1 RawTransaction
|
||||
|
||||
## 1. Base et gate d'entrée
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.5
|
||||
```
|
||||
|
||||
Le gate opérateur fourni le 2026-08-30 est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
scripts/audit_rust_workspace_rules.py PASS
|
||||
scripts/audit_markdown_tables.py PASS
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS (30 unit backend, live foundation ignoré)
|
||||
cargo test -p ksp-config-lib PASS (128 unit + ownership/public API)
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
La tranche peut donc ouvrir la pagination sans rouvrir l'écriture atomique acquise en `pre.005`.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.6
|
||||
```
|
||||
|
||||
## 3. Scope exact
|
||||
|
||||
Cette tranche ajoute uniquement la navigation backend PostgreSQL des transactions RAW :
|
||||
|
||||
```text
|
||||
PostgresBackend::list_raw_transactions
|
||||
RawTransactionQuery
|
||||
RawPage<RawTransactionReference>
|
||||
RawPageCursor backend-private V1
|
||||
```
|
||||
|
||||
Ne sont pas ouverts :
|
||||
|
||||
```text
|
||||
transitions Full -> Archived -> Purged
|
||||
compare-and-transition de rétention
|
||||
implémentations complètes des six traits RawTransaction*
|
||||
dispatch ksp-store-lib
|
||||
RawAccountState
|
||||
worker/job policy
|
||||
```
|
||||
|
||||
## 4. Ordre physique et keyset
|
||||
|
||||
L'ordre total reste celui figé en `pre.001` :
|
||||
|
||||
```text
|
||||
Ascending = (slot ASC, signature ASC)
|
||||
Descending = (slot DESC, signature DESC)
|
||||
```
|
||||
|
||||
Les deux requêtes privées utilisent :
|
||||
|
||||
```text
|
||||
retention_state <> 'purged'
|
||||
slot >= start_inclusive si présent
|
||||
slot <= end_inclusive si présent
|
||||
(slot, signature) > cursor pour Ascending
|
||||
(slot, signature) < cursor pour Descending
|
||||
LIMIT requested + 1
|
||||
```
|
||||
|
||||
Aucun `OFFSET` n'est utilisé. La signature 64 octets constitue le tie-breaker déterministe pour plusieurs transactions au même slot.
|
||||
|
||||
Le prédicat et l'ordre correspondent à l'index V001 existant :
|
||||
|
||||
```text
|
||||
ix_ksp_raw_transactions_slot_signature
|
||||
ON ksp_raw_transactions (slot, signature)
|
||||
WHERE retention_state <> 'purged'
|
||||
```
|
||||
|
||||
Aucune migration n'est modifiée.
|
||||
|
||||
## 5. Cursor V1
|
||||
|
||||
Le cursor est implémenté dans un sous-module privé au domaine `raw_transaction`.
|
||||
|
||||
Format fixe :
|
||||
|
||||
```text
|
||||
magic 4 bytes = KSPT
|
||||
version 1 byte = 1
|
||||
last_slot 8 bytes = u64 big-endian
|
||||
last_signature 64 bytes
|
||||
binding_digest 32 bytes = SHA-256
|
||||
--------------------------------------
|
||||
total 109 bytes
|
||||
```
|
||||
|
||||
Le digest utilise le domaine séparé :
|
||||
|
||||
```text
|
||||
KSP/raw-transaction-cursor/v1
|
||||
```
|
||||
|
||||
Il lie de façon non ambiguë :
|
||||
|
||||
```text
|
||||
longueur du network + network
|
||||
direction
|
||||
flag start + start slot si présent
|
||||
flag end + end slot si présent
|
||||
last_slot
|
||||
last_signature
|
||||
```
|
||||
|
||||
Le digest n'est ni un secret ni un mécanisme d'autorisation. Il sert uniquement à détecter corruption et replay sous un autre contexte de navigation.
|
||||
|
||||
Le decode rejette avant `pool.get()` :
|
||||
|
||||
```text
|
||||
taille != 109
|
||||
magic incorrect
|
||||
version inconnue
|
||||
digest divergent
|
||||
network différent
|
||||
range différente
|
||||
direction différente
|
||||
last_slot hors range
|
||||
bytes hostiles bornés par RawPageCursor
|
||||
```
|
||||
|
||||
Ces cas sont classés `PostgresBackendErrorKind::QueryInvalid` avec phase statique uniquement.
|
||||
|
||||
## 6. Limite physique réelle
|
||||
|
||||
`RawPageLimit` reste sans cap métier KSP.
|
||||
|
||||
Le backend doit obtenir une row supplémentaire pour déterminer la présence d'une continuation :
|
||||
|
||||
```text
|
||||
LIMIT requested + 1
|
||||
```
|
||||
|
||||
La borne physique exacte retenue sur PostgreSQL est donc :
|
||||
|
||||
```text
|
||||
requested <= i64::MAX - 1
|
||||
```
|
||||
|
||||
Une valeur supérieure produit :
|
||||
|
||||
```text
|
||||
PostgresBackendErrorKind::PageLimitUnsupported
|
||||
```
|
||||
|
||||
Aucun clamp vers `100`, `500`, `1000` ou une autre taille de worker/job n'est introduit.
|
||||
|
||||
## 7. Décodage des pages
|
||||
|
||||
Toutes les rows retournées par le statement, y compris la row de probe `limit+1`, passent par le mapping fallible :
|
||||
|
||||
```text
|
||||
signature BYTEA -> [u8; 64]
|
||||
slot NUMERIC(20,0) -> texte décimal -> u64
|
||||
```
|
||||
|
||||
Une corruption physique n'est donc pas masquée simplement parce qu'elle se trouve sur la row supplémentaire.
|
||||
|
||||
Si plus de `requested` rows sont valides :
|
||||
|
||||
1. la row de probe est retirée du résultat ;
|
||||
2. le cursor est construit depuis la dernière row réellement retournée ;
|
||||
3. la page contient exactement au plus `requested` références.
|
||||
|
||||
Les tombstones `Purged` ne sont jamais listés.
|
||||
|
||||
## 8. Concurrence et snapshot
|
||||
|
||||
Chaque page est cohérente au niveau de son statement PostgreSQL, mais la navigation ne prétend pas fournir un snapshot inter-pages.
|
||||
|
||||
Une mutation concurrente qui insère une clé ordonnée avant le cursor déjà consommé peut ne pas être vue par cette navigation. Cette propriété est désormais explicitement documentée et n'est pas confondue avec une garantie de replay transactionnel.
|
||||
|
||||
## 9. Erreurs sûres
|
||||
|
||||
Deux kinds backend supplémentaires sont matérialisés :
|
||||
|
||||
```text
|
||||
PageLimitUnsupported
|
||||
QueryInvalid
|
||||
```
|
||||
|
||||
Comme les autres erreurs physiques, ils ne conservent que :
|
||||
|
||||
```text
|
||||
kind
|
||||
phase &'static str
|
||||
```
|
||||
|
||||
Aucun cursor, network hostile, signature, query SQL, bind, SQLSTATE ou texte PostgreSQL n'est rendu.
|
||||
|
||||
## 10. Tests et canaris
|
||||
|
||||
Les tests unitaires ajoutés couvrent :
|
||||
|
||||
```text
|
||||
round-trip cursor V1 exact
|
||||
109 bytes exacts
|
||||
magic/version exacts
|
||||
replay autre network
|
||||
replay autre direction
|
||||
replay autre range
|
||||
last_slot hors range
|
||||
taille hostile 1/108/109/110/4096
|
||||
mutation magic/version/digest
|
||||
borne page i64::MAX - 1
|
||||
rejet i64::MAX
|
||||
```
|
||||
|
||||
Les canaris d'intégration figent :
|
||||
|
||||
```text
|
||||
keyset > / <
|
||||
ASC/DESC total
|
||||
pas d'OFFSET
|
||||
exclusion Purged
|
||||
index V001 compatible
|
||||
pas de cap 500/1000
|
||||
pas de DELETE
|
||||
pas de transition de rétention
|
||||
pas d'implémentation complète de trait prématurée
|
||||
```
|
||||
|
||||
La pagination PostgreSQL réelle multi-page avec ties/ranges reste réservée au gate live opt-in `pre.009`.
|
||||
|
||||
## 11. Migrations
|
||||
|
||||
Les migrations sont byte-identiques à `pre.005` :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
V001 conserve exactement :
|
||||
|
||||
```text
|
||||
4 tables
|
||||
35 contraintes
|
||||
1 index
|
||||
40 ressources
|
||||
```
|
||||
|
||||
## 12. Fichiers ajoutés/modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/src/error.rs
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.006.md
|
||||
```
|
||||
|
||||
Aucun fichier n'est supprimé.
|
||||
|
||||
## 13. Audits exécutables dans l'environnement de génération
|
||||
|
||||
```text
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
```
|
||||
|
||||
Les commandes Cargo/rustfmt ne sont pas disponibles dans l'environnement de génération. Elles doivent rester `NON EXÉCUTÉES` jusqu'au gate opérateur ; elles ne doivent jamais être présentées comme PASS sans sortie réelle.
|
||||
|
||||
## 14. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
## 15. Suite après gate vert
|
||||
|
||||
```text
|
||||
0.3.3-pre.007 — archive/purge/tombstone/compare-and-transition + Compacted unsupported stable
|
||||
```
|
||||
106
deltas/0.3.3/pre.007-fix.001.md
Normal file
106
deltas/0.3.3/pre.007-fix.001.md
Normal file
@@ -0,0 +1,106 @@
|
||||
<!-- file: deltas/0.3.3/pre.007-fix.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.007-fix.001` — correction du canari de scope rétention
|
||||
|
||||
## 1. Base
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.7
|
||||
```
|
||||
|
||||
Le gate opérateur du 2026-08-30 est vert sur les audits, `cargo check`, Clippy, Store API, Store façade et Config. Il échoue uniquement dans :
|
||||
|
||||
```text
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
```
|
||||
|
||||
sur le canari historique :
|
||||
|
||||
```text
|
||||
pre_005_raw_write_sql_is_atomic_idempotent_and_keeps_later_scope_closed
|
||||
```
|
||||
|
||||
Ce canari interdisait globalement `DELETE FROM` dans `raw_transaction.rs`. Cette interdiction était correcte en `pre.005`, mais elle devient obsolète en `pre.007` puisque `Archived -> Purged` doit supprimer atomiquement le payload archive.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.7.fix.1
|
||||
```
|
||||
|
||||
## 3. Correction
|
||||
|
||||
Le canari `pre.005` est renommé pour refléter le contrat qui reste valide :
|
||||
|
||||
```text
|
||||
pre_005_raw_write_sql_is_atomic_idempotent_and_keeps_direct_trait_scope_closed
|
||||
```
|
||||
|
||||
Il conserve toutes les preuves `pre.005` d'insertion/idempotence/rehydration et continue d'interdire :
|
||||
|
||||
```text
|
||||
impl ksp_store_api::RawTransactionWrite
|
||||
impl ksp_store_api::RawTransactionObservationWrite
|
||||
```
|
||||
|
||||
Il n'interdit plus globalement `DELETE FROM`, car ce SQL appartient désormais légitimement à la tranche `pre.007`.
|
||||
|
||||
Le canari `pre.007_raw_retention_is_atomic_compare_and_transition_without_fake_compaction` reste inchangé et exige explicitement :
|
||||
|
||||
```text
|
||||
DELETE_ARCHIVE_PAYLOAD_SQL
|
||||
DELETE FROM ksp_raw_transaction_archive_payloads
|
||||
UPDATE_PURGED_TRANSACTION_SQL
|
||||
```
|
||||
|
||||
Aucun garde-fou fonctionnel n'est donc supprimé : il est simplement rattaché à la tranche qui possède désormais cette responsabilité.
|
||||
|
||||
## 4. Invariants
|
||||
|
||||
Ce fix ne modifie pas :
|
||||
|
||||
```text
|
||||
src/raw_transaction.rs
|
||||
src/raw_transaction/cursor.rs
|
||||
runtime/error mapping
|
||||
SQL lecture/écriture/pagination/rétention
|
||||
migrations V000/V001
|
||||
format cursor V1
|
||||
surface publique backend
|
||||
```
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 5. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.007-fix.001.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 6. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
158
deltas/0.3.3/pre.007.md
Normal file
158
deltas/0.3.3/pre.007.md
Normal file
@@ -0,0 +1,158 @@
|
||||
<!-- file: deltas/0.3.3/pre.007.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.007` — rétention atomique RawTransaction
|
||||
|
||||
## 1. Base et gate d'entrée
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.6.fix.1
|
||||
```
|
||||
|
||||
Le gate opérateur fourni le 2026-08-30 est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
scripts/audit_rust_workspace_rules.py PASS
|
||||
scripts/audit_markdown_tables.py PASS
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS (34 unit backend, live foundation ignoré)
|
||||
cargo test -p ksp-config-lib PASS (128 unit + ownership/public API)
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
La pagination/cursor `pre.006` est donc acquise et la tranche peut ouvrir la rétention mutante.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.7
|
||||
```
|
||||
|
||||
## 3. Scope exact
|
||||
|
||||
Cette tranche ajoute uniquement le bridge backend PostgreSQL de transition :
|
||||
|
||||
```text
|
||||
PostgresBackend::transition_raw_transaction_retention
|
||||
Full -> Archived
|
||||
Archived -> Purged
|
||||
compare-and-transition
|
||||
Compacted unsupported
|
||||
```
|
||||
|
||||
Ne sont pas encore ouverts :
|
||||
|
||||
```text
|
||||
implémentations complètes des six traits RawTransaction*
|
||||
dispatch ksp-store-lib
|
||||
RawAccountState
|
||||
worker/job policy
|
||||
```
|
||||
|
||||
## 4. Pré-I/O
|
||||
|
||||
Avant `pool.get()` :
|
||||
|
||||
- le réseau de `transition.reference()` doit égaler le binding backend ;
|
||||
- si `expected` ou `target` vaut `Compacted`, le backend retourne `RetentionCompactionUnsupported` ;
|
||||
- le code KSP stable reste `store.postgres_retention_compaction_unsupported`.
|
||||
|
||||
Aucune ligne `retention_state = 'compacted'` n'est créée et aucune dépendance de compression n'est ajoutée.
|
||||
|
||||
## 5. Compare-and-transition
|
||||
|
||||
Sous transaction PostgreSQL, le canonical est lu avec `FOR UPDATE`. Sa forme physique est validée avant de retourner un outcome ou de muter :
|
||||
|
||||
```text
|
||||
current == target -> AlreadyAtTarget
|
||||
current != expected -> ExpectedStateMismatch
|
||||
Full + expected Full + target Archived -> archive
|
||||
Archived + expected Archived + target Purged -> purge
|
||||
```
|
||||
|
||||
Une référence absente est `ReferenceNotFound`. Aucun succès n'est rendu avant commit.
|
||||
|
||||
## 6. `Full -> Archived`
|
||||
|
||||
La transition :
|
||||
|
||||
1. exige un payload hot non vide et borné ;
|
||||
2. exige l'absence d'une archive résiduelle ;
|
||||
3. insère les octets exacts dans `ksp_raw_transaction_archive_payloads` ;
|
||||
4. met `payload = NULL` et `retention_state = 'archived'` sur le canonical ;
|
||||
5. commit les deux opérations ensemble.
|
||||
|
||||
Aucun état `Archived` sans payload archive n'est committé par KSP.
|
||||
|
||||
## 7. `Archived -> Purged`
|
||||
|
||||
La transition :
|
||||
|
||||
1. exige l'absence de payload hot et la présence d'un payload archive valide ;
|
||||
2. supprime la ligne archive ;
|
||||
3. met `block_time_unix_millis = NULL`, `payload = NULL` et `retention_state = 'purged'` ;
|
||||
4. conserve signature, slot, format id/version et content hash ;
|
||||
5. commit atomiquement.
|
||||
|
||||
Le tombstone reste donc limité aux cinq champs du contrat API.
|
||||
|
||||
## 8. Races croisées
|
||||
|
||||
Les acquisitions/idempotence/ForceRehydrate de `pre.005` et les transitions de cette tranche verrouillent la même ligne canonical avant mutation. Les races archive/purge/rehydrate sont donc sérialisées ; la preuve concurrente PostgreSQL réelle reste réservée à `pre.009`.
|
||||
|
||||
## 9. Erreurs
|
||||
|
||||
La classification backend ajoute :
|
||||
|
||||
```text
|
||||
RetentionCompactionUnsupported
|
||||
```
|
||||
|
||||
Les autres classifications réutilisées sont :
|
||||
|
||||
```text
|
||||
WrongNetwork
|
||||
ReferenceNotFound
|
||||
DataInvalid
|
||||
WriteFailed
|
||||
```
|
||||
|
||||
Aucun texte serveur, SQLSTATE, query ou bind n'est retenu.
|
||||
|
||||
## 10. Migrations
|
||||
|
||||
Aucune ressource V000/V001 n'est modifiée. Les checksums attendus restent :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 11. Gate opérateur demandé
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le test PostgreSQL live reste `#[ignore]`; la preuve métier rétention/concurrence est réservée à `pre.009`.
|
||||
|
||||
## 12. Suite si gate vert
|
||||
|
||||
```text
|
||||
0.3.3-pre.008 — six implémentations de capabilities + dispatch ksp-store-lib
|
||||
```
|
||||
106
deltas/0.3.3/pre.008-fix.001.md
Normal file
106
deltas/0.3.3/pre.008-fix.001.md
Normal file
@@ -0,0 +1,106 @@
|
||||
<!-- file: deltas/0.3.3/pre.008-fix.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.008-fix.001` — canaris de capabilities warning-clean
|
||||
|
||||
## 1. Base
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.8
|
||||
```
|
||||
|
||||
Le gate opérateur du 2026-08-30 passe les audits, `cargo check`, les tests Store API/Store/PostgreSQL/Config et `--no-default-features`. Clippy ne bloque pas le build, mais signale deux warnings :
|
||||
|
||||
```text
|
||||
clippy::extra_unused_type_parameters
|
||||
```
|
||||
|
||||
Ils concernent uniquement les helpers génériques de preuve de traits dans :
|
||||
|
||||
```text
|
||||
crates/ksp-store-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
```
|
||||
|
||||
Le paramètre `T` est utilisé uniquement dans les bounds et pas dans le corps des fonctions, ce que Clippy classe comme paramètre générique inutilisé.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.8.fix.1
|
||||
```
|
||||
|
||||
## 3. Correction
|
||||
|
||||
Les deux helpers conservent exactement les six bounds `RawTransaction*` et matérialisent désormais le paramètre de type dans leur corps avec :
|
||||
|
||||
```rust
|
||||
let _marker = std::marker::PhantomData::<T>;
|
||||
```
|
||||
|
||||
Cette forme :
|
||||
|
||||
- conserve une preuve purement compile-time ;
|
||||
- ne construit aucune valeur backend ;
|
||||
- ne fait aucun I/O ;
|
||||
- n'affaiblit aucun bound ;
|
||||
- n'ajoute aucun `#[allow]` ;
|
||||
- ne modifie aucune surface publique.
|
||||
|
||||
## 4. Invariants
|
||||
|
||||
Ce fix ne modifie pas :
|
||||
|
||||
```text
|
||||
code de production ksp-store-lib
|
||||
code de production ksp-store-postgres-lib
|
||||
six implémentations RawTransaction* backend
|
||||
six dispatches RawTransaction* façade
|
||||
mapping d'erreurs
|
||||
pagination/cursor
|
||||
rétention/ForceRehydrate
|
||||
Config
|
||||
migrations V000/V001
|
||||
```
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 5. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.008-fix.001.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 6. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
## 7. Suite si gate warning-clean
|
||||
|
||||
```text
|
||||
0.3.3-pre.009 — preuve PostgreSQL live complète RawTransaction
|
||||
```
|
||||
135
deltas/0.3.3/pre.008.md
Normal file
135
deltas/0.3.3/pre.008.md
Normal file
@@ -0,0 +1,135 @@
|
||||
<!-- file: deltas/0.3.3/pre.008.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.008` — conformance capabilities et dispatch Store
|
||||
|
||||
## 1. Base et gate d'entrée
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.7.fix.1
|
||||
```
|
||||
|
||||
Le gate opérateur fourni le 2026-08-30 est entièrement vert : audits Rust/Markdown, workspace check, Clippy, tests Store API/façade/PostgreSQL/Config et `ksp-store-lib --no-default-features` passent. `pre.007` est donc acquise.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.8
|
||||
```
|
||||
|
||||
## 3. Six capabilities sur `PostgresBackend`
|
||||
|
||||
`PostgresBackend` implémente désormais directement :
|
||||
|
||||
```text
|
||||
RawTransactionRead
|
||||
RawTransactionWrite
|
||||
RawTransactionObservationRead
|
||||
RawTransactionObservationWrite
|
||||
RawTransactionRetentionRead
|
||||
RawTransactionRetentionWrite
|
||||
```
|
||||
|
||||
Les implémentations adaptent les méthodes physiques déjà acquises en `pre.004` à `pre.007` vers `StoreApiFuture` et `ksp_store_api::Result`. Aucun SQL n'est déplacé hors du backend et aucune capability `RawAccount*` n'est ouverte.
|
||||
|
||||
## 4. Six dispatches sur `Store`
|
||||
|
||||
`Store` implémente les mêmes six traits et dispatch vers `StoreRuntime::Postgres` lorsque la feature `postgres` est compilée. Aucun type `PostgresBackend`, pool, client, row, statement ou SQL n'entre dans la surface publique de `ksp-store-lib`.
|
||||
|
||||
Les consumers ordinaires continuent donc à dépendre uniquement de `ksp-store-lib`.
|
||||
|
||||
## 5. Validation réseau pré-I/O
|
||||
|
||||
La façade valide avant dispatch toutes les opérations dont l'input porte un réseau :
|
||||
|
||||
```text
|
||||
get/list transaction
|
||||
persist acquisition
|
||||
record observation
|
||||
retention read
|
||||
retention transition
|
||||
```
|
||||
|
||||
Une divergence retourne `store.wrong_network` sans rendre le réseau hostile. `RawTransactionObservationRead` reçoit uniquement une `RawObservationKey`; son scope réseau est celui de l'instance Store mono-réseau.
|
||||
|
||||
## 6. Taxonomie d'erreurs stabilisée
|
||||
|
||||
```text
|
||||
Conflict -> store_api.raw_conflict
|
||||
QueryInvalid -> store_api.raw_query_invalid
|
||||
WrongNetwork -> store.wrong_network
|
||||
ReferenceNotFound -> store.raw_reference_not_found
|
||||
ReadFailed -> store.postgres_read_failed
|
||||
WriteFailed -> store.postgres_write_failed
|
||||
DataInvalid -> store.postgres_data_invalid
|
||||
PageLimitUnsupported -> store.postgres_page_limit_unsupported
|
||||
RetentionCompactionUnsupported -> store.postgres_retention_compaction_unsupported
|
||||
```
|
||||
|
||||
Le mapping conserve seulement backend, réseau propre de l'instance et phase statique. Aucun SQLSTATE, SQL, bind ou texte serveur n'est exposé.
|
||||
|
||||
## 7. Feature mismatch
|
||||
|
||||
Le contrat `default = ["postgres"]` reste inchangé. Sans default feature, `Store::open` conserve le rejet pré-I/O `store.backend_not_compiled`; le gate `cargo check -p ksp-store-lib --no-default-features` reste obligatoire.
|
||||
|
||||
## 8. Migrations
|
||||
|
||||
Aucune ressource V000/V001 n'est modifiée. Checksums attendus :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 9. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-lib/README.md
|
||||
crates/ksp-store-lib/USAGE.md
|
||||
crates/ksp-store-lib/src/error.rs
|
||||
crates/ksp-store-lib/src/lib.rs
|
||||
crates/ksp-store-lib/src/store.rs
|
||||
crates/ksp-store-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-lib/tests/public_api.rs
|
||||
crates/ksp-store-lib/unit_tests/store.rs
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/runtime.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.008.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 10. Gate opérateur demandé
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Le test PostgreSQL live reste ignoré dans ce gate ; la preuve métier/concurrence complète appartient à `pre.009`.
|
||||
|
||||
## 11. Suite si gate vert
|
||||
|
||||
```text
|
||||
0.3.3-pre.009 — preuve PostgreSQL live complète RawTransaction
|
||||
```
|
||||
100
deltas/0.3.3/pre.009-fix.001.md
Normal file
100
deltas/0.3.3/pre.009-fix.001.md
Normal file
@@ -0,0 +1,100 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.001.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.001` — correction borrow du live cancellation
|
||||
|
||||
## 1. Base et constat
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9
|
||||
```
|
||||
|
||||
Le gate standard fourni le 2026-08-30 est propre sur les audits Rust/Markdown, `ksp-store-api`, `ksp-store-lib`, `ksp-config-lib` et `ksp-store-lib --no-default-features`, mais `ksp-store-postgres-lib` ne compile pas son nouveau test live :
|
||||
|
||||
```text
|
||||
E0502: cannot borrow `*admin` as immutable because it is also borrowed as mutable
|
||||
```
|
||||
|
||||
Le scénario cancellation stockait le résultat de `admin.transaction().await` dans `lock_transaction_result`. Ce `Result<Transaction<'_>>` pouvait conserver l'emprunt mutable de `admin` jusqu'à sa destruction en fin de fonction, alors que le probe final `transaction_exists(admin, ...)` demande ensuite un emprunt immuable.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.1
|
||||
```
|
||||
|
||||
## 3. Correction
|
||||
|
||||
Le test live matche désormais directement :
|
||||
|
||||
```rust
|
||||
let lock_transaction = match admin.transaction().await {
|
||||
...
|
||||
};
|
||||
```
|
||||
|
||||
Le binding intermédiaire `lock_transaction_result` est supprimé. Après `lock_transaction.rollback().await`, la transaction est consommée et l'emprunt mutable de `admin` peut se terminer avant le probe final.
|
||||
|
||||
Aucune logique de preuve n'est modifiée : le test conserve le verrou observation réel, le blocage de l'acquisition concurrente, `JoinHandle::abort()`, le rollback du lock administrateur puis la vérification d'absence du canonical annulé.
|
||||
|
||||
## 4. Scope strict
|
||||
|
||||
Aucun changement dans :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/src/**
|
||||
crates/ksp-store-lib/src/**
|
||||
crates/ksp-store-api/src/**
|
||||
crates/ksp-store-postgres-lib/migrations/**
|
||||
```
|
||||
|
||||
Aucun SQL, capability, cursor, contrat de rétention, mapping d'erreur, Config ou comportement runtime n'est modifié.
|
||||
|
||||
## 5. Migrations
|
||||
|
||||
Checksums inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 6. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.001.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 7. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Après gate standard vert, exécuter le live opt-in sur une base PostgreSQL dédiée et vide de toute table KSP gérée :
|
||||
|
||||
```bash
|
||||
printf '%s\n' '<URI_POSTGRES_DEDIEE>' | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture
|
||||
```
|
||||
|
||||
## 8. Suite si les deux gates sont verts
|
||||
|
||||
```text
|
||||
0.3.3-pre.010 — hardening/completeness
|
||||
```
|
||||
101
deltas/0.3.3/pre.009-fix.002.md
Normal file
101
deltas/0.3.3/pre.009-fix.002.md
Normal file
@@ -0,0 +1,101 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.002.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.002` — Clippy live + phase sûre d’ouverture
|
||||
|
||||
## 1. Base et constat
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9.fix.1
|
||||
```
|
||||
|
||||
Le gate standard du 2026-08-30 confirme que le borrow `E0502` de `pre.009-fix.001` est corrigé : `cargo check --workspace`, les tests `ksp-store-api`, `ksp-store-lib`, `ksp-store-postgres-lib`, `ksp-config-lib` et `cargo check -p ksp-store-lib --no-default-features` passent.
|
||||
|
||||
Clippy bloque encore le nouveau test live sur trois closures soumises à `clippy::implicit_return = deny` et remonte deux `redundant_guards` sur les tableaux de pagination.
|
||||
|
||||
Le premier essai live avec le placeholder littéral `<URI_POSTGRES_DEDIEE>` échoue logiquement à `admin_config` et n'est pas une preuve. Le second essai avec une URI dédiée réelle atteint PostgreSQL 17 puis échoue à `backend_open`. Cette étiquette était trop agrégée : elle supprimait le `phase()` statique, pourtant secret-safe, déjà porté par `PostgresBackendError`.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.2
|
||||
```
|
||||
|
||||
## 3. Corrections Clippy
|
||||
|
||||
Les trois filtres utilisent désormais un `return` explicite dans leur closure, conformément à la policy KSP :
|
||||
|
||||
```rust
|
||||
.filter(|value| return matches_inserted(value))
|
||||
.filter(|value| return matches_already_present(value))
|
||||
```
|
||||
|
||||
Les deux validations de pagination utilisent directement le pattern tableau :
|
||||
|
||||
```rust
|
||||
Result::Ok([50, 51, 52, 53, 54])
|
||||
Result::Ok([54, 53, 52, 51, 50])
|
||||
```
|
||||
|
||||
Aucune exemption Clippy n'est ajoutée.
|
||||
|
||||
## 4. Diagnostic live secret-safe
|
||||
|
||||
`open_backend` distingue désormais :
|
||||
|
||||
- succès backend ;
|
||||
- erreur backend : `LiveFailure::new(error.phase())` ;
|
||||
- erreur locale de préparation : propagation du `LiveFailure` existant.
|
||||
|
||||
`PostgresBackendError::phase()` est une chaîne statique KSP. Elle ne contient ni URI, ni texte serveur PostgreSQL, ni SQL, ni SQLSTATE, ni bind. Le live peut donc identifier une étape telle que `schema_resource_post_apply`, `store_identity_*`, `history_insert`, etc. sans affaiblir la redaction.
|
||||
|
||||
Ce correctif ne modifie volontairement aucune logique de migration tant que la phase réelle n'est pas connue.
|
||||
|
||||
## 5. Hors scope / invariants
|
||||
|
||||
Aucun changement de :
|
||||
|
||||
- production `ksp-store-postgres-lib` ;
|
||||
- `ksp-store-lib` / `ksp-store-api` / Config ;
|
||||
- SQL RawTransaction ;
|
||||
- cursor ;
|
||||
- rétention ;
|
||||
- migrations V000/V001 ;
|
||||
- checksums de migration ;
|
||||
- scope `RawAccountState`.
|
||||
|
||||
## 6. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.002.md
|
||||
```
|
||||
|
||||
Aucune suppression.
|
||||
|
||||
## 7. Gate opérateur attendu
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Puis, avec une base PostgreSQL dédiée vide de toute table KSP :
|
||||
|
||||
```bash
|
||||
read -rsp "Dedicated PostgreSQL URI: " KSP_PG_TEST_URI; echo
|
||||
printf '%s\n' "$KSP_PG_TEST_URI" | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture --test-threads=1
|
||||
unset KSP_PG_TEST_URI
|
||||
```
|
||||
105
deltas/0.3.3/pre.009-fix.003.md
Normal file
105
deltas/0.3.3/pre.009-fix.003.md
Normal file
@@ -0,0 +1,105 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.003.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.003` — ressource V001 post-apply identifiable sans fuite
|
||||
|
||||
## 1. Base et constat
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9.fix.2
|
||||
```
|
||||
|
||||
Le gate standard fourni le 2026-08-30 est entièrement propre : audits Rust/Markdown, `cargo check --workspace`, Clippy all-targets, tests Store/API/backend/Config et `--no-default-features` passent.
|
||||
|
||||
Le live PostgreSQL réel atteint PostgreSQL 17 puis échoue pendant `PostgresBackend::open` avec la phase :
|
||||
|
||||
```text
|
||||
schema_resource_post_apply
|
||||
```
|
||||
|
||||
Cette phase prouve qu'une ressource V001 a bien été exécutée mais que l'introspection immédiate la classe ensuite `Missing` ou `Incompatible`. Elle ne permet toutefois pas encore de savoir quelle ressource parmi les 40 est concernée.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.3
|
||||
```
|
||||
|
||||
## 3. Diagnostic post-apply exact et secret-safe
|
||||
|
||||
Dans `ensure_resource`, le seul chemin `post_apply_incompatible` conserve `MigrationMismatch` mais utilise désormais :
|
||||
|
||||
```rust
|
||||
resource.id
|
||||
```
|
||||
|
||||
comme `PostgresBackendError::phase()`.
|
||||
|
||||
`SchemaResource::id` est une chaîne `&'static str` embarquée dans le binaire et contrôlée par KSP, par exemple :
|
||||
|
||||
```text
|
||||
tables/002_ksp_raw_transactions.sql
|
||||
constraints/006_ck_ksp_raw_transactions_slot.sql
|
||||
indexes/001_ix_ksp_raw_transactions_slot_signature.sql
|
||||
```
|
||||
|
||||
Elle ne contient aucune URI, valeur de configuration, donnée utilisateur, SQLSTATE, bind, texte serveur ou payload. Le prochain live peut donc identifier exactement la ressource fautive sans affaiblir la redaction.
|
||||
|
||||
Les autres phases de migration restent inchangées.
|
||||
|
||||
## 4. Pourquoi aucune correction SQL n'est appliquée ici
|
||||
|
||||
Le live ne fournit encore que la classification `schema_resource_post_apply`. V000 est déjà acquis sur PostgreSQL 17 et aucune incompatibilité évidente ne justifie de modifier à l'aveugle une ressource V001 ou son comparateur de catalogue.
|
||||
|
||||
`fix.003` est donc volontairement diagnostique : la prochaine exécution doit donner l'identifiant exact, puis le correctif suivant pourra cibler le contrat d'introspection ou la ressource concernée avec preuve.
|
||||
|
||||
## 5. Invariants
|
||||
|
||||
Aucun changement de :
|
||||
|
||||
- fichiers SQL V000/V001 ;
|
||||
- ordre ou nombre des 40 ressources V001 ;
|
||||
- checksums migrations ;
|
||||
- Store API / façade ;
|
||||
- RawTransaction reads/writes/pagination/rétention ;
|
||||
- cursor ;
|
||||
- Config ;
|
||||
- scope `RawAccountState`.
|
||||
|
||||
## 6. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/migration.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.003.md
|
||||
```
|
||||
|
||||
Aucune suppression.
|
||||
|
||||
## 7. Gate opérateur attendu
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Puis :
|
||||
|
||||
```bash
|
||||
read -rsp "Dedicated PostgreSQL URI: " KSP_PG_TEST_URI; echo
|
||||
printf '%s\n' "$KSP_PG_TEST_URI" | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture --test-threads=1
|
||||
unset KSP_PG_TEST_URI
|
||||
```
|
||||
|
||||
Si le post-apply échoue encore, le message doit maintenant exposer uniquement l'identifiant statique de la ressource V001 concernée.
|
||||
117
deltas/0.3.3/pre.009-fix.004.md
Normal file
117
deltas/0.3.3/pre.009-fix.004.md
Normal file
@@ -0,0 +1,117 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.004.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.004` — normalisation du CHECK NUMERIC déparsée par PostgreSQL 17
|
||||
|
||||
## 1. Base et diagnostic réel
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9.fix.3
|
||||
```
|
||||
|
||||
Le gate standard fourni le 2026-08-30 est entièrement propre. Le live PostgreSQL réel atteint PostgreSQL 17 puis identifie précisément la ressource V001 rejetée après application :
|
||||
|
||||
```text
|
||||
constraints/006_ck_ksp_raw_transactions_slot.sql
|
||||
```
|
||||
|
||||
La ressource crée :
|
||||
|
||||
```sql
|
||||
CHECK (slot >= 0 AND slot <= 18446744073709551615)
|
||||
```
|
||||
|
||||
La colonne `slot` est `NUMERIC(20,0)`. PostgreSQL peut déparser un littéral `NUMERIC` hors plage bigint sous forme quotée avec cast explicite, notamment :
|
||||
|
||||
```text
|
||||
'18446744073709551615'::numeric
|
||||
```
|
||||
|
||||
L'ancien `normalize_catalog_sql` supprimait `::numeric` mais conservait les quotes simples. Le contrat embarqué devenait donc `18446744073709551615` tandis que le catalogue pouvait devenir `'18446744073709551615'`, ce qui produisait un faux `Incompatible` immédiatement après création.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.4
|
||||
```
|
||||
|
||||
## 3. Correction d'introspection
|
||||
|
||||
`schema.rs` ajoute une normalisation ciblée avant la normalisation générale :
|
||||
|
||||
- seules les chaînes composées d'un signe ASCII optionnel et de chiffres ASCII ;
|
||||
- uniquement lorsqu'elles sont immédiatement suivies de `::numeric` ;
|
||||
- voient leurs quotes simples et leur cast `::numeric` retirés dans cette étape ciblée ;
|
||||
- les autres casts continuent d’être traités par la normalisation existante.
|
||||
|
||||
Cette règle ne retire donc pas les quotes des littéraux métier/textuels tels que :
|
||||
|
||||
```text
|
||||
'full'::text
|
||||
'archived'::text
|
||||
'purged'::text
|
||||
```
|
||||
|
||||
Le canari unitaire existant du CHECK `slot` couvre désormais explicitement la forme PostgreSQL 17 quotée et vérifie également qu'un littéral texte conserve ses quotes.
|
||||
|
||||
## 4. Pourquoi la migration reste inchangée
|
||||
|
||||
Le SQL V001 est sémantiquement correct et PostgreSQL l'applique avant que KSP ne le rejette. Le défaut est donc dans l'équivalence du catalogue, pas dans la contrainte elle-même. Modifier la ressource SQL ferait inutilement varier le checksum V001 et réécrirait le contrat de migration pour un problème de déparsing.
|
||||
|
||||
`fix.004` ne modifie donc aucune ressource V000/V001.
|
||||
|
||||
## 5. Invariants
|
||||
|
||||
Aucun changement de :
|
||||
|
||||
- tables, contraintes ou indexes V001 ;
|
||||
- ordre ou nombre des 40 ressources V001 ;
|
||||
- Store API / façade ;
|
||||
- RawTransaction read/write/pagination/rétention ;
|
||||
- cursor ;
|
||||
- Config ;
|
||||
- scope `RawAccountState`.
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 6. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/schema.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/schema.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.004.md
|
||||
```
|
||||
|
||||
Aucune suppression.
|
||||
|
||||
## 7. Gate opérateur attendu
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Puis :
|
||||
|
||||
```bash
|
||||
read -rsp "Dedicated PostgreSQL URI: " KSP_PG_TEST_URI; echo
|
||||
printf '%s\n' "$KSP_PG_TEST_URI" | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture --test-threads=1
|
||||
unset KSP_PG_TEST_URI
|
||||
```
|
||||
128
deltas/0.3.3/pre.009-fix.005.md
Normal file
128
deltas/0.3.3/pre.009-fix.005.md
Normal file
@@ -0,0 +1,128 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.005.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.005` — canonicalisation des littéraux numériques/intégraux PostgreSQL
|
||||
|
||||
## 1. Base et diagnostic réel
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9.fix.4
|
||||
```
|
||||
|
||||
Le gate standard `pre.009-fix.004` fourni le 2026-08-30 est entièrement propre. Le live PostgreSQL 17 confirme que la correction du CHECK `slot` est effective puis avance jusqu'à la ressource suivante :
|
||||
|
||||
```text
|
||||
constraints/007_ck_ksp_raw_transactions_block_time.sql
|
||||
```
|
||||
|
||||
La ressource crée :
|
||||
|
||||
```sql
|
||||
CHECK (
|
||||
block_time_unix_millis IS NULL
|
||||
OR block_time_unix_millis >= 0 AND block_time_unix_millis <= 253402300799999
|
||||
)
|
||||
```
|
||||
|
||||
`block_time_unix_millis` est `BIGINT`. `pg_get_constraintdef` reconstruit une expression depuis la représentation catalogue au lieu de restituer le texte SQL original ; une constante entière typée peut donc apparaître avec un cast explicite. La normalisation de `fix.004` savait retirer les quotes uniquement pour `::numeric` et pouvait encore considérer comme différente une forme telle que :
|
||||
|
||||
```text
|
||||
'253402300799999'::bigint
|
||||
```
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.5
|
||||
```
|
||||
|
||||
## 3. Correction d'introspection
|
||||
|
||||
`schema.rs` généralise la canonicalisation ciblée des littéraux entiers quotés. Une chaîne n'est déquotée que si :
|
||||
|
||||
- elle est constituée d'un signe ASCII optionnel suivi uniquement de chiffres ASCII ;
|
||||
- elle est immédiatement suivie d'un cast numérique/intégral PostgreSQL retenu ;
|
||||
- le cast appartient à `numeric`, `bigint`, `integer`, `smallint`, `int8`, `int4` ou `int2`.
|
||||
|
||||
Les casts équivalents non quotés sont également supprimés par la normalisation générale.
|
||||
|
||||
Cette règle reste volontairement distincte des littéraux métier/textuels :
|
||||
|
||||
```text
|
||||
'full'::text
|
||||
'archived'::text
|
||||
'purged'::text
|
||||
```
|
||||
|
||||
conservent leurs quotes dans la forme normalisée.
|
||||
|
||||
## 4. Canaris
|
||||
|
||||
Le test d'équivalence catalogue couvre désormais :
|
||||
|
||||
- le CHECK `slot` avec `'18446744073709551615'::numeric` ;
|
||||
- le CHECK `block_time` avec `'0'::bigint` et `'253402300799999'::bigint` ;
|
||||
- l'alias intégral `::int8` sur la borne `format_version` ;
|
||||
- la préservation du littéral texte `'full'::text`.
|
||||
|
||||
Le but est de normaliser des représentations typées équivalentes, jamais de rendre un CHECK divergent compatible.
|
||||
|
||||
## 5. Pourquoi V001 reste inchangée
|
||||
|
||||
Comme pour `slot`, PostgreSQL applique la contrainte avant que KSP ne la rejette lors de la relecture post-apply. Le défaut reste donc dans l'équivalence de catalogue. Modifier le SQL V001 créerait une migration logique différente sans corriger la cause.
|
||||
|
||||
Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 6. Scope
|
||||
|
||||
Aucun changement de :
|
||||
|
||||
- ressource SQL V000/V001 ;
|
||||
- ordre/nombre des 40 ressources V001 ;
|
||||
- runtime RawTransaction ;
|
||||
- pagination/cursor/rétention ;
|
||||
- Store API/façade ;
|
||||
- Config ;
|
||||
- scope `RawAccountState`.
|
||||
|
||||
## 7. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/schema.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/schema.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.005.md
|
||||
```
|
||||
|
||||
Aucune suppression.
|
||||
|
||||
## 8. Gate opérateur attendu
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Puis :
|
||||
|
||||
```bash
|
||||
read -rsp "Dedicated PostgreSQL URI: " KSP_PG_TEST_URI; echo
|
||||
printf '%s\n' "$KSP_PG_TEST_URI" | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture --test-threads=1
|
||||
unset KSP_PG_TEST_URI
|
||||
```
|
||||
104
deltas/0.3.3/pre.009-fix.006.md
Normal file
104
deltas/0.3.3/pre.009-fix.006.md
Normal file
@@ -0,0 +1,104 @@
|
||||
<!-- file: deltas/0.3.3/pre.009-fix.006.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009-fix.006` — restauration `schema.rs` et classification du drift appliqué
|
||||
|
||||
## 1. Base et diagnostic
|
||||
|
||||
Base opérateur :
|
||||
|
||||
```text
|
||||
0.3.3-pre.9.fix.5
|
||||
```
|
||||
|
||||
Deux faits distincts sont corrigés.
|
||||
|
||||
Premièrement, l'overlay `pre.009-fix.005` a accidentellement tronqué la fin de `crates/ksp-store-postgres-lib/src/schema.rs` lors de la généralisation de la normalisation PostgreSQL. Les éléments privés suivants avaient disparu de l'archive alors qu'ils restent requis :
|
||||
|
||||
```rust
|
||||
fn schema_incompatible<T>(phase: &'static str) -> std::result::Result<T, crate::PostgresBackendError>
|
||||
fn schema_query_error<T>(phase: &'static str) -> std::result::Result<T, crate::PostgresBackendError>
|
||||
|
||||
#[cfg(test)]
|
||||
#[path = "../unit_tests/schema.rs"]
|
||||
mod tests;
|
||||
```
|
||||
|
||||
L'opérateur les a restaurés et le gate standard est redevenu entièrement propre. `fix.006` réembarque explicitement le fichier complet afin que l'historique d'overlays soit autonome.
|
||||
|
||||
Deuxièmement, le live PostgreSQL 17 progresse maintenant jusqu'au scénario qui supprime volontairement l'index géré puis rouvre le backend avec `schema_autoupdate=false`. Le backend renvoyait `MigrationFailed`, alors que le contrat live attend `MigrationMismatch`.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9.fix.6
|
||||
```
|
||||
|
||||
## 3. Classification corrigée
|
||||
|
||||
Dans `verify_or_repair_applied_migrations`, une ressource `Missing` d'une migration déjà présente dans l'historique signifie que l'état physique ne correspond plus au schéma déclaré. Lorsque `schema_autoupdate=false`, KSP ne tente aucune réparation et retourne désormais :
|
||||
|
||||
```text
|
||||
PostgresBackendErrorKind::MigrationMismatch
|
||||
phase = schema_autoupdate_disabled
|
||||
```
|
||||
|
||||
Cette correction est volontairement limitée à ce chemin. Les cas où aucune divergence appliquée n'existe mais où une mutation est interdite par policy, notamment `migration_pending` ou `schema_autocreate_disabled`, restent classés `MigrationFailed`.
|
||||
|
||||
## 4. Canari
|
||||
|
||||
Un helper privé `schema_autoupdate_disabled_error()` centralise la classification et le test unitaire vérifie exactement :
|
||||
|
||||
```text
|
||||
kind = MigrationMismatch
|
||||
phase = schema_autoupdate_disabled
|
||||
```
|
||||
|
||||
## 5. Migrations inchangées
|
||||
|
||||
Aucun SQL V000/V001 n'est modifié. Checksums attendus inchangés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 6. Scope
|
||||
|
||||
Aucun changement de runtime RawTransaction, pagination/cursor, rétention, Store API, façade, Config ou `RawAccountState`.
|
||||
|
||||
## 7. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/src/migration.rs
|
||||
crates/ksp-store-postgres-lib/src/schema.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/migration.rs
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009-fix.006.md
|
||||
```
|
||||
|
||||
Aucune suppression.
|
||||
|
||||
## 8. Gate opérateur attendu
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
Puis :
|
||||
|
||||
```bash
|
||||
read -rsp "Dedicated PostgreSQL URI: " KSP_PG_TEST_URI; echo
|
||||
printf '%s\n' "$KSP_PG_TEST_URI" | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture --test-threads=1
|
||||
unset KSP_PG_TEST_URI
|
||||
```
|
||||
184
deltas/0.3.3/pre.009.md
Normal file
184
deltas/0.3.3/pre.009.md
Normal file
@@ -0,0 +1,184 @@
|
||||
<!-- file: deltas/0.3.3/pre.009.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.009` — preuve PostgreSQL live complète RawTransaction
|
||||
|
||||
## 1. Base et gate d'entrée
|
||||
|
||||
Base opérateur obligatoire :
|
||||
|
||||
```text
|
||||
0.3.3-pre.8.fix.1
|
||||
```
|
||||
|
||||
Le gate opérateur fourni le 2026-08-30 est entièrement propre : audits Rust/Markdown, `cargo check --workspace`, Clippy all-targets sans warning, tests Store API/façade/PostgreSQL/Config et `ksp-store-lib --no-default-features` passent. `pre.008-fix.001` est donc acquise.
|
||||
|
||||
## 2. Version
|
||||
|
||||
```text
|
||||
workspace.package.version = 0.3.3-pre.9
|
||||
```
|
||||
|
||||
## 3. Nouveau test PostgreSQL réel
|
||||
|
||||
La tranche ajoute :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||
```
|
||||
|
||||
Le test est `#[ignore]`, lit une URI PostgreSQL dédiée uniquement sur `stdin`, ne l'affiche jamais et refuse de démarrer lorsqu'une table KSP V000/V001 existe déjà. Il ne nettoie que le schéma qu'il a prouvé absent avant son propre bootstrap.
|
||||
|
||||
Il refuse PostgreSQL < 15 comme la preuve fondation existante.
|
||||
|
||||
## 4. Migration, identité réseau et schema update
|
||||
|
||||
La preuve couvre :
|
||||
|
||||
- bootstrap V000 + V001 sur base vide ;
|
||||
- health Ready avec migration 1 ;
|
||||
- refus d'une réouverture sous un autre `RawNetworkId` ;
|
||||
- réouverture idempotente ;
|
||||
- suppression contrôlée de `ix_ksp_raw_transactions_slot_signature` ;
|
||||
- `schema_autoupdate=false` qui bloque le drift ;
|
||||
- `schema_autoupdate=true` qui recrée et revalide l'index manquant.
|
||||
|
||||
Aucune ressource de migration n'est modifiée et aucun historique n'est réécrit.
|
||||
|
||||
## 5. Écriture, idempotence, concurrence et rollback
|
||||
|
||||
Le live couvre :
|
||||
|
||||
- canonical + observation atomiques ;
|
||||
- get canonical exact ;
|
||||
- observation round-trip exact avec provenance complète ;
|
||||
- deux acquisitions identiques en vraies tâches concurrentes : un `Inserted`, un `AlreadyPresent` ;
|
||||
- deux acquisitions divergentes sous la même signature : un gagnant et un `Conflict` ;
|
||||
- vérification qu'une seule observation du duel divergent est durable ;
|
||||
- observation supplémentaire `Inserted` puis `AlreadyPresent` ;
|
||||
- même observation key divergente -> `Conflict` ;
|
||||
- collision observation pendant une nouvelle acquisition -> rollback du canonical nouvellement tenté.
|
||||
|
||||
## 6. Pagination
|
||||
|
||||
Le test insère cinq signatures sur trois slots, dont deux paires ex æquo, et prouve :
|
||||
|
||||
```text
|
||||
Ascending 50, 51, 52, 53, 54
|
||||
Descending 54, 53, 52, 51, 50
|
||||
page size 2
|
||||
```
|
||||
|
||||
Il rejoue ensuite le cursor V1 sous :
|
||||
|
||||
- direction différente ;
|
||||
- range différente.
|
||||
|
||||
Les deux cas doivent être `QueryInvalid`.
|
||||
|
||||
## 7. Rétention et rehydrate
|
||||
|
||||
La preuve couvre :
|
||||
|
||||
- compare mismatch observable ;
|
||||
- `Full -> Archived` et lecture exacte depuis archive ;
|
||||
- idempotence `AlreadyAtTarget` ;
|
||||
- `Archived -> Purged` ;
|
||||
- `get -> None` et tombstone minimal exact ;
|
||||
- acquisition normale sur tombstone -> `SkippedPurged/NotRecorded` ;
|
||||
- ForceRehydrate divergent -> `Conflict` sans mutation ;
|
||||
- ForceRehydrate compatible -> `Rehydrated/Inserted` ;
|
||||
- `Full -> Compacted` rejeté par `RetentionCompactionUnsupported` sans changement d'état ;
|
||||
- deux archives concurrentes puis deux purges concurrentes : un `Applied` et un `AlreadyAtTarget` par race ;
|
||||
- rehydrate après la race purge.
|
||||
|
||||
## 8. Cancellation réelle
|
||||
|
||||
Le test crée une observation durable, la verrouille via une transaction PostgreSQL administrateur, puis démarre une nouvelle acquisition qui :
|
||||
|
||||
1. insère son canonical dans sa transaction non committée ;
|
||||
2. bloque sur la collision de l'observation verrouillée ;
|
||||
3. reste bloquée au-delà du délai canari ;
|
||||
4. est annulée via `JoinHandle::abort()` ;
|
||||
5. libère ensuite le verrou administrateur ;
|
||||
6. prouve que le canonical de l'acquisition annulée n'est pas durable.
|
||||
|
||||
Cette preuve exerce un rollback par cancellation réelle, pas une simulation séquentielle.
|
||||
|
||||
## 9. Réouverture finale
|
||||
|
||||
Après toutes les opérations, le backend est rouvert sur le même réseau. Le test exige :
|
||||
|
||||
- health Ready ;
|
||||
- migration 1 ;
|
||||
- lecture exacte d'un canonical durable antérieur.
|
||||
|
||||
Le cleanup final supprime les cinq tables KSP uniquement parce que leur absence initiale a été prouvée.
|
||||
|
||||
## 10. Migrations
|
||||
|
||||
Aucune ressource V000/V001 n'est modifiée.
|
||||
|
||||
Checksums attendus et recalculés :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
V001 reste exactement à 40 ressources.
|
||||
|
||||
## 11. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/postgres_raw_transaction_live.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
deltas/0.3.3/pre.009.md
|
||||
```
|
||||
|
||||
Aucune suppression de fichier.
|
||||
|
||||
## 12. Validation exécutée dans l'environnement de génération
|
||||
|
||||
```text
|
||||
General Rust rule audit: clean
|
||||
Rust export completeness audit: 0 candidate(s)
|
||||
KSP workspace Rust rule audit: clean
|
||||
Markdown table audit: clean
|
||||
```
|
||||
|
||||
Cargo, rustfmt et PostgreSQL réel ne sont pas disponibles dans l'environnement de génération. Le gate opérateur et le live restent donc à exécuter localement.
|
||||
|
||||
## 13. Gate opérateur standard
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
## 14. Gate PostgreSQL live opt-in
|
||||
|
||||
Sur une base PostgreSQL dédiée et vide de toute table KSP gérée :
|
||||
|
||||
```bash
|
||||
printf '%s\n' '<URI_POSTGRES_DEDIEE>' | cargo test -p ksp-store-postgres-lib --test postgres_raw_transaction_live -- --ignored --nocapture
|
||||
```
|
||||
|
||||
## 15. Suite si les deux gates sont verts
|
||||
|
||||
```text
|
||||
0.3.3-pre.010 — hardening/completeness
|
||||
```
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user