diff --git a/Cargo.toml b/Cargo.toml index 5bb71eb..2a9ed33 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,12 +1,12 @@ # file: Cargo.toml -# version: 352 +# version: 353 [workspace] resolver = "3" members = ["crates/ksp-app-config-desk", "crates/ksp-app-solprices-desk", "crates/ksp-app-wallet-desk", "crates/ksp-config-lib", "crates/ksp-core-lib", "crates/ksp-interface-lib", "crates/ksp-logging-lib", "crates/ksp-offchain-transport-lib", "crates/ksp-onchain-transport-lib", "crates/ksp-program-api", "crates/ksp-store-api", "crates/ksp-store-lib", "crates/ksp-store-postgres-lib", "crates/ksp-wallet-lib"] [workspace.package] -version = "0.3.3-pre.5" +version = "0.3.3-pre.6" edition = "2024" license = "MIT" repository = "https://git.sasedev.com/Sasedev/khadhroony-solana-project" diff --git a/crates/ksp-store-postgres-lib/README.md b/crates/ksp-store-postgres-lib/README.md index afd9d00..1b586d8 100644 --- a/crates/ksp-store-postgres-lib/README.md +++ b/crates/ksp-store-postgres-lib/README.md @@ -1,5 +1,5 @@ - + # ksp-store-postgres-lib @@ -83,7 +83,7 @@ ksp_store_schema_migrations Le moteur vérifie version, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime. -V001 possède désormais le schéma physique `RawTransaction` et son contrat de compatibilité. Les lectures exactes sont acquises depuis `pre.004`; `pre.005` ajoute les écritures atomiques transaction + observation, l'idempotence réelle et la classification de conflit. +V001 possède désormais le schéma physique `RawTransaction` et son contrat de compatibilité. Les lectures exactes sont acquises depuis `pre.004`; `pre.005` ajoute les écritures atomiques transaction + observation, l'idempotence réelle et la classification de conflit. `pre.006` ajoute la navigation keyset déterministe sur l'index `(slot, signature)` et son cursor opaque lié à la requête. ## Health et erreurs @@ -134,6 +134,19 @@ Un tombstone `Purged` compatible produit `SkippedPurged/NotRecorded` en mode nor Les erreurs physiques d'écriture sont réduites à `WriteFailed`; aucune erreur serveur, SQLSTATE, query ou valeur de bind n'est conservée. +## Pagination RAW `0.3.3-pre.006` + +`list_raw_transactions` parcourt uniquement les références canoniques récupérables, en excluant les tombstones `Purged`. L'ordre total est : + +```text +Ascending = (slot ASC, signature ASC) +Descending = (slot DESC, signature DESC) +``` + +La continuation est une keyset stricte `>` / `<`, jamais un `OFFSET`. Le cursor backend V1 fait exactement 109 octets et transporte `KSPT`, version 1, le dernier slot, la dernière signature et un digest SHA-256 lié au réseau, à la direction et aux bornes de slots. Un cursor rejoué sous une autre query est `QueryInvalid`. + +`RawPageLimit` n'est pas réduit par une policy KSP : PostgreSQL utilise `LIMIT requested + 1`, avec comme seule borne physique `requested <= i64::MAX - 1`. Une valeur supérieure produit `PageLimitUnsupported` sans clamp. La pagination ne promet aucun snapshot inter-pages. + ## Hors périmètre actuel La crate ne contient encore : diff --git a/crates/ksp-store-postgres-lib/USAGE.md b/crates/ksp-store-postgres-lib/USAGE.md index 4cbed4e..64a0199 100644 --- a/crates/ksp-store-postgres-lib/USAGE.md +++ b/crates/ksp-store-postgres-lib/USAGE.md @@ -1,5 +1,5 @@ - + # Utilisation de ksp-store-postgres-lib @@ -129,6 +129,8 @@ match error.kind() { ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed => {} ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed => {} ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch => {} + ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported => {} + ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid => {} ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed => {} ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound => {} ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer => {} @@ -188,12 +190,25 @@ Pour une transaction purgée, le mode normal retourne `SkippedPurged/NotRecorded Les références réseau-scopées sont toujours validées avant `pool.get()`. -## 9. Ce que cette crate ne permet pas encore +## 9. Paginer les transactions RAW + +Depuis `0.3.3-pre.006`, le backend fournit une navigation keyset déterministe : + +```rust +let page = backend.list_raw_transactions(&query).await; +``` + +La query utilise les bornes inclusives de `RawSlotRange`, la direction demandée et le `RawPageLimit` exact. Les lignes `Purged` sont exclues. L'ordre est `(slot, signature)` dans les deux directions et la continuation utilise un cursor opaque de 109 octets lié au réseau, à la direction et aux bornes de la query. + +Le cursor peut être transmis uniquement à une query ayant le même binding. Un changement de réseau/range/direction ou des bytes hostiles produit `QueryInvalid` avant acquisition du pool. Le backend n'utilise aucun `OFFSET` et ne fournit pas de snapshot transactionnel entre deux pages. + +La seule limite physique est liée au `LIMIT + 1` PostgreSQL : `requested <= i64::MAX - 1`. Au-delà, `PageLimitUnsupported` est retourné ; la demande n'est jamais ramenée à 100, 500, 1000 ou une autre policy de worker. + +## 10. Ce que cette crate ne permet pas encore La tranche ne fournit pas encore : ```text -list_raw_transactions / cursor transitions de rétention implémentations complètes des six traits RawTransaction* capabilities RawAccount* diff --git a/crates/ksp-store-postgres-lib/src/error.rs b/crates/ksp-store-postgres-lib/src/error.rs index a558df2..c273092 100644 --- a/crates/ksp-store-postgres-lib/src/error.rs +++ b/crates/ksp-store-postgres-lib/src/error.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/src/error.rs -// version: 6 +// version: 7 /// Stable KSP error code reserved for PostgreSQL retention transitions that require unsupported physical compaction. pub const ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED: ksp_store_api::ErrorCode = @@ -23,8 +23,12 @@ pub enum PostgresBackendErrorKind { DataInvalid, /// PostgreSQL migration/bootstrap execution failed without exposing server text or SQL. MigrationFailed, + /// The requested RAW page size cannot be represented by PostgreSQL LIMIT plus the continuation probe row. + PageLimitUnsupported, /// Applied PostgreSQL migration history diverges from the embedded immutable KSP history. MigrationMismatch, + /// A bounded backend-private RAW query or opaque cursor is invalid for the requested navigation context. + QueryInvalid, /// A PostgreSQL RAW read statement failed without exposing server text, SQL or bind values. ReadFailed, /// A RAW write requires an existing canonical reference that is not durable. diff --git a/crates/ksp-store-postgres-lib/src/lib.rs b/crates/ksp-store-postgres-lib/src/lib.rs index 117573f..73fe233 100644 --- a/crates/ksp-store-postgres-lib/src/lib.rs +++ b/crates/ksp-store-postgres-lib/src/lib.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/src/lib.rs -// version: 10 +// version: 11 #![warn(missing_docs)] #![deny(unreachable_pub)] @@ -15,7 +15,8 @@ //! backend-private RAW transaction/observation/retention read mapping. //! `0.3.3-pre.005` adds atomic canonical/observation writes, real idempotence //! checks and safe conflict classification without exposing PostgreSQL rows or -//! SQL through the public bridge. +//! SQL through the public bridge. `0.3.3-pre.006` adds deterministic keyset +//! pagination with a fixed opaque cursor bound to network, range and direction. //! //! This crate depends on `ksp-store-api` and never on `ksp-store-lib`. The //! common facade consumes only this crate's narrow backend bridge and never @@ -54,6 +55,14 @@ pub(crate) use self::health::probe_health; pub(crate) use self::migration::bootstrap; /// Current embedded migration version consumed by the private health probe. pub(crate) use self::migration::current_migration_version; +/// Private decoded RAW cursor representation consumed by tests and the physical RAW module. +pub(crate) use self::raw_transaction::cursor::RawTransactionDecodedCursor; +/// Private RAW transaction cursor decoder consumed by the physical RAW module. +pub(crate) use self::raw_transaction::cursor::decode_raw_transaction_cursor; +/// Private RAW transaction cursor encoder consumed by the physical RAW module. +pub(crate) use self::raw_transaction::cursor::encode_raw_transaction_cursor; +/// Private physical page-limit converter consumed by the physical RAW module. +pub(crate) use self::raw_transaction::cursor::raw_transaction_physical_page_limit; /// Private RAW transaction reader consumed by the physical backend runtime. pub(crate) use self::raw_transaction::get_raw_transaction; /// Private RAW transaction observation reader consumed by the physical backend runtime. @@ -62,6 +71,8 @@ pub(crate) use self::raw_transaction::get_raw_transaction_observation; pub(crate) use self::raw_transaction::get_raw_transaction_retention_state; /// Private RAW transaction tombstone reader consumed by the physical backend runtime. pub(crate) use self::raw_transaction::get_raw_transaction_tombstone; +/// Private RAW transaction list reader consumed by the physical backend runtime. +pub(crate) use self::raw_transaction::list_raw_transactions; /// Private atomic RAW transaction acquisition writer consumed by the physical backend runtime. pub(crate) use self::raw_transaction::persist_raw_transaction_acquisition; /// Private additional RAW transaction observation writer consumed by the physical backend runtime. diff --git a/crates/ksp-store-postgres-lib/src/raw_transaction.rs b/crates/ksp-store-postgres-lib/src/raw_transaction.rs index 0813a6a..e0ca9da 100644 --- a/crates/ksp-store-postgres-lib/src/raw_transaction.rs +++ b/crates/ksp-store-postgres-lib/src/raw_transaction.rs @@ -1,5 +1,7 @@ // file: crates/ksp-store-postgres-lib/src/raw_transaction.rs -// version: 2 +// version: 3 + +pub(crate) mod cursor; const GET_ARCHIVE_PAYLOAD_SQL: &str = "SELECT payload FROM ksp_raw_transaction_archive_payloads WHERE signature = $1"; const GET_OBSERVATION_SQL: &str = "SELECT observation_key, transaction_signature, provider, protocol, acquisition_method, origin, received_at_unix_millis, capture_session_id, commitment, endpoint_id, filter_id, observed_at_unix_millis, source_payload_hash, source_payload_size_bytes FROM ksp_raw_transaction_observations WHERE observation_key = $1"; @@ -8,12 +10,19 @@ const GET_TOMBSTONE_SQL: &str = "SELECT signature, slot::text AS slot_text, bloc const GET_TRANSACTION_SQL: &str = "SELECT transaction_row.signature, transaction_row.slot::text AS slot_text, transaction_row.block_time_unix_millis, transaction_row.format_id, transaction_row.format_version, transaction_row.content_hash, transaction_row.payload, transaction_row.retention_state, archive_row.payload AS archive_payload FROM ksp_raw_transactions AS transaction_row LEFT JOIN ksp_raw_transaction_archive_payloads AS archive_row ON archive_row.signature = transaction_row.signature WHERE transaction_row.signature = $1"; const INSERT_OBSERVATION_SQL: &str = "INSERT INTO ksp_raw_transaction_observations (observation_key, transaction_signature, provider, protocol, acquisition_method, origin, received_at_unix_millis, capture_session_id, commitment, endpoint_id, filter_id, observed_at_unix_millis, source_payload_hash, source_payload_size_bytes) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) ON CONFLICT (observation_key) DO NOTHING RETURNING observation_key"; const INSERT_TRANSACTION_SQL: &str = "INSERT INTO ksp_raw_transactions (signature, slot, block_time_unix_millis, format_id, format_version, content_hash, payload, retention_state) VALUES ($1, $2::TEXT::NUMERIC, $3, $4, $5, $6, $7, 'full') ON CONFLICT (signature) DO NOTHING RETURNING signature"; +const LIST_TRANSACTIONS_ASC_SQL: &str = "SELECT signature, slot::text AS slot_text FROM ksp_raw_transactions WHERE retention_state <> 'purged' AND ($1::TEXT IS NULL OR slot >= $1::TEXT::NUMERIC) AND ($2::TEXT IS NULL OR slot <= $2::TEXT::NUMERIC) AND ($3::TEXT IS NULL OR (slot, signature) > ($3::TEXT::NUMERIC, $4::BYTEA)) ORDER BY slot ASC, signature ASC LIMIT $5"; +const LIST_TRANSACTIONS_DESC_SQL: &str = "SELECT signature, slot::text AS slot_text FROM ksp_raw_transactions WHERE retention_state <> 'purged' AND ($1::TEXT IS NULL OR slot >= $1::TEXT::NUMERIC) AND ($2::TEXT IS NULL OR slot <= $2::TEXT::NUMERIC) AND ($3::TEXT IS NULL OR (slot, signature) < ($3::TEXT::NUMERIC, $4::BYTEA)) ORDER BY slot DESC, signature DESC LIMIT $5"; const LOCK_OBSERVATION_SQL: &str = "SELECT observation_key, transaction_signature, provider, protocol, acquisition_method, origin, received_at_unix_millis, capture_session_id, commitment, endpoint_id, filter_id, observed_at_unix_millis, source_payload_hash, source_payload_size_bytes FROM ksp_raw_transaction_observations WHERE observation_key = $1 FOR UPDATE"; const LOCK_TRANSACTION_SQL: &str = "SELECT signature, slot::text AS slot_text, block_time_unix_millis, format_id, format_version, content_hash, payload, retention_state, NULL::BYTEA AS archive_payload FROM ksp_raw_transactions WHERE signature = $1 FOR UPDATE"; const LOCK_TRANSACTION_STATE_SQL: &str = "SELECT retention_state FROM ksp_raw_transactions WHERE signature = $1 FOR UPDATE"; const REHYDRATE_TRANSACTION_SQL: &str = "UPDATE ksp_raw_transactions SET block_time_unix_millis = $2, payload = $3, retention_state = 'full' WHERE signature = $1"; +struct RawListDbRow { + signature: std::vec::Vec, + slot_text: std::string::String, +} + struct RawObservationDbRow { acquisition_method: std::string::String, capture_session_id: std::option::Option, @@ -94,6 +103,82 @@ pub(crate) async fn get_raw_transaction( return decode_raw_transaction_row(network, physical); } +/// Lists deterministic canonical RAW transaction references using PostgreSQL keyset pagination. +pub(crate) async fn list_raw_transactions( + pool: &deadpool_postgres::Pool, + network: &ksp_store_api::RawNetworkId, + query: &ksp_store_api::RawTransactionQuery, +) -> std::result::Result, crate::PostgresBackendError> { + if query.network() != network { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::WrongNetwork, "raw_transaction_list_network")); + } + let (requested_usize, sql_limit) = match crate::raw_transaction_physical_page_limit(query.page().limit().get()) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let decoded_cursor = match query.page().cursor() { + std::option::Option::Some(value) => match crate::decode_raw_transaction_cursor(query, value) { + std::result::Result::Ok(decoded) => std::option::Option::Some(decoded), + std::result::Result::Err(error) => return std::result::Result::Err(error), + }, + std::option::Option::None => std::option::Option::None, + }; + let client_result = pool.get().await; + let client = match client_result { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(crate::map_pool_error(error)), + }; + let slots = query.slots(); + let start_text = slots.start_inclusive().map(|value| return value.to_string()); + let end_text = slots.end_inclusive().map(|value| return value.to_string()); + let cursor_slot_text = decoded_cursor.as_ref().map(|value| return value.last_slot.to_string()); + let cursor_signature = decoded_cursor.as_ref().map(|value| return value.last_signature.to_vec()); + let sql = match query.direction() { + ksp_store_api::RawSortDirection::Ascending => LIST_TRANSACTIONS_ASC_SQL, + ksp_store_api::RawSortDirection::Descending => LIST_TRANSACTIONS_DESC_SQL, + _ => { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::QueryInvalid, "raw_transaction_list_direction")); + }, + }; + let rows_result = client.query(sql, &[&start_text, &end_text, &cursor_slot_text, &cursor_signature, &sql_limit]).await; + let rows = match rows_result { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::ReadFailed, "raw_transaction_list_query")); + }, + }; + let mut decoded = std::vec::Vec::with_capacity(rows.len()); + for row in rows { + let physical = match raw_list_db_row(&row) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let item = match decode_raw_list_row(network, physical) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + decoded.push(item); + } + let has_more = decoded.len() > requested_usize; + if has_more { + decoded.truncate(requested_usize); + } + let next_cursor = if has_more { + let last = match decoded.last() { + std::option::Option::Some(value) => value, + std::option::Option::None => return std::result::Result::Err(data_invalid("raw_transaction_list_page")), + }; + match crate::encode_raw_transaction_cursor(query, last.0, &last.1.signature()) { + std::result::Result::Ok(value) => std::option::Option::Some(value), + std::result::Result::Err(error) => return std::result::Result::Err(error), + } + } else { + std::option::Option::None + }; + let items = decoded.into_iter().map(|value| return value.1).collect(); + return std::result::Result::Ok(ksp_store_api::RawPage::new(items, next_cursor)); +} + /// Reads one RAW transaction observation from the physical PostgreSQL backend. pub(crate) async fn get_raw_transaction_observation( pool: &deadpool_postgres::Pool, @@ -655,6 +740,18 @@ fn write_failed(phase: &'static str) -> crate::PostgresBackendError { return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::WriteFailed, phase); } +fn raw_list_db_row(row: &tokio_postgres::Row) -> std::result::Result { + let signature = match row.try_get::<_, std::vec::Vec>("signature") { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_list_decode")), + }; + let slot_text = match row.try_get::<_, std::string::String>("slot_text") { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_list_decode")), + }; + return std::result::Result::Ok(RawListDbRow { signature, slot_text }); +} + fn raw_transaction_db_row(row: &tokio_postgres::Row) -> std::result::Result { let signature = match row.try_get::<_, std::vec::Vec>("signature") { std::result::Result::Ok(value) => value, @@ -772,6 +869,22 @@ fn raw_tombstone_db_row(row: &tokio_postgres::Row) -> std::result::Result std::result::Result<(u64, ksp_store_api::RawTransactionReference), crate::PostgresBackendError> { + let signature = match fixed_bytes::<64>(row.signature) { + std::result::Result::Ok(value) => ksp_store_api::RawTransactionSignature::new(value), + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let slot = match decode_u64_decimal(row.slot_text.as_str(), "raw_transaction_list_slot") { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let reference = ksp_store_api::RawTransactionReference::new(network.clone(), signature); + return std::result::Result::Ok((slot, reference)); +} + fn decode_raw_transaction_row( network: &ksp_store_api::RawNetworkId, row: RawTransactionDbRow, diff --git a/crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs b/crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs new file mode 100644 index 0000000..e8ed910 --- /dev/null +++ b/crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs @@ -0,0 +1,184 @@ +// file: crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs +// version: 1 + +use sha2::Digest; // rust-rules: trait-import + +const CURSOR_BYTES: usize = 109; +const CURSOR_DIGEST_OFFSET: usize = 77; +const CURSOR_DOMAIN: &[u8] = b"KSP/raw-transaction-cursor/v1"; +const CURSOR_MAGIC: &[u8; 4] = b"KSPT"; +const CURSOR_VERSION: u8 = 1; +const MAX_POSTGRES_PAGE_LIMIT: u64 = 9_223_372_036_854_775_806; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +/// Decoded backend-private continuation key extracted from one validated RAW cursor. +pub(crate) struct RawTransactionDecodedCursor { + /// Last canonical transaction signature returned by the previous page. + pub(crate) last_signature: [u8; 64], + /// Last canonical transaction slot returned by the previous page. + pub(crate) last_slot: u64, +} + +/// Decodes and validates one backend-private RAW transaction cursor against its query binding. +pub(crate) fn decode_raw_transaction_cursor( + query: &ksp_store_api::RawTransactionQuery, + cursor: &ksp_store_api::RawPageCursor, +) -> std::result::Result { + let bytes = cursor.as_bytes(); + if bytes.len() != CURSOR_BYTES { + return std::result::Result::Err(query_invalid("raw_cursor_size")); + } + if bytes.get(0..4) != std::option::Option::Some(CURSOR_MAGIC.as_ref()) { + return std::result::Result::Err(query_invalid("raw_cursor_magic")); + } + if bytes.get(4).copied() != std::option::Option::Some(CURSOR_VERSION) { + return std::result::Result::Err(query_invalid("raw_cursor_version")); + } + let slot_bytes = match bytes.get(5..13) { + std::option::Option::Some(value) => match <[u8; 8]>::try_from(value) { + std::result::Result::Ok(decoded) => decoded, + std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_slot")), + }, + std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_slot")), + }; + let last_slot = u64::from_be_bytes(slot_bytes); + let last_signature = match bytes.get(13..CURSOR_DIGEST_OFFSET) { + std::option::Option::Some(value) => match <[u8; 64]>::try_from(value) { + std::result::Result::Ok(decoded) => decoded, + std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_signature")), + }, + std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_signature")), + }; + let range_result = validate_slot_in_range(query.slots(), last_slot); + if let std::result::Result::Err(error) = range_result { + return std::result::Result::Err(error); + } + let expected_digest = match binding_digest(query, last_slot, &last_signature) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let stored_digest = match bytes.get(CURSOR_DIGEST_OFFSET..CURSOR_BYTES) { + std::option::Option::Some(value) => match <[u8; 32]>::try_from(value) { + std::result::Result::Ok(decoded) => decoded, + std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_digest")), + }, + std::option::Option::None => return std::result::Result::Err(query_invalid("raw_cursor_digest")), + }; + if stored_digest != expected_digest { + return std::result::Result::Err(query_invalid("raw_cursor_binding")); + } + return std::result::Result::Ok(RawTransactionDecodedCursor { last_signature, last_slot }); +} + +/// Encodes one backend-private RAW transaction cursor bound to the supplied query context. +pub(crate) fn encode_raw_transaction_cursor( + query: &ksp_store_api::RawTransactionQuery, + last_slot: u64, + last_signature: &ksp_store_api::RawTransactionSignature, +) -> std::result::Result { + let range_result = validate_slot_in_range(query.slots(), last_slot); + if let std::result::Result::Err(error) = range_result { + return std::result::Result::Err(error); + } + let digest = match binding_digest(query, last_slot, last_signature.as_bytes()) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => return std::result::Result::Err(error), + }; + let mut bytes = std::vec::Vec::with_capacity(CURSOR_BYTES); + bytes.extend_from_slice(CURSOR_MAGIC); + bytes.push(CURSOR_VERSION); + bytes.extend_from_slice(&last_slot.to_be_bytes()); + bytes.extend_from_slice(last_signature.as_bytes()); + bytes.extend_from_slice(&digest); + if bytes.len() != CURSOR_BYTES { + return std::result::Result::Err(query_invalid("raw_cursor_encode_size")); + } + return match ksp_store_api::RawPageCursor::try_new(bytes.into_boxed_slice()) { + std::result::Result::Ok(value) => std::result::Result::Ok(value), + std::result::Result::Err(_) => std::result::Result::Err(query_invalid("raw_cursor_encode")), + }; +} + +/// Converts one caller page size into the exact PostgreSQL LIMIT+1 representation. +pub(crate) fn raw_transaction_physical_page_limit(requested: u64) -> std::result::Result<(usize, i64), crate::PostgresBackendError> { + if requested > MAX_POSTGRES_PAGE_LIMIT { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit")); + } + let requested_usize = match usize::try_from(requested) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit")); + }, + }; + let requested_plus_one = match requested.checked_add(1) { + std::option::Option::Some(value) => value, + std::option::Option::None => { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit")); + }, + }; + let sql_limit = match i64::try_from(requested_plus_one) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => { + return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::PageLimitUnsupported, "raw_page_limit")); + }, + }; + return std::result::Result::Ok((requested_usize, sql_limit)); +} + +fn binding_digest( + query: &ksp_store_api::RawTransactionQuery, + last_slot: u64, + last_signature: &[u8; 64], +) -> std::result::Result<[u8; 32], crate::PostgresBackendError> { + let direction = match query.direction() { + ksp_store_api::RawSortDirection::Ascending => 0_u8, + ksp_store_api::RawSortDirection::Descending => 1_u8, + _ => return std::result::Result::Err(query_invalid("raw_cursor_direction")), + }; + let network_len = match u64::try_from(query.network().as_str().len()) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(_) => return std::result::Result::Err(query_invalid("raw_cursor_network")), + }; + let mut hasher = sha2::Sha256::new(); + hasher.update(CURSOR_DOMAIN); + hasher.update(network_len.to_be_bytes()); + hasher.update(query.network().as_str().as_bytes()); + hasher.update([direction]); + hash_optional_slot(&mut hasher, query.slots().start_inclusive()); + hash_optional_slot(&mut hasher, query.slots().end_inclusive()); + hasher.update(last_slot.to_be_bytes()); + hasher.update(last_signature); + let finalized = hasher.finalize(); + let mut digest = [0_u8; 32]; + digest.copy_from_slice(finalized.as_ref()); + return std::result::Result::Ok(digest); +} + +fn hash_optional_slot(hasher: &mut sha2::Sha256, value: std::option::Option) { + match value { + std::option::Option::Some(slot) => { + hasher.update([1_u8]); + hasher.update(slot.to_be_bytes()); + }, + std::option::Option::None => hasher.update([0_u8]), + } + return; +} + +fn validate_slot_in_range(range: ksp_store_api::RawSlotRange, slot: u64) -> std::result::Result<(), crate::PostgresBackendError> { + if let std::option::Option::Some(start) = range.start_inclusive() + && slot < start + { + return std::result::Result::Err(query_invalid("raw_cursor_slot_range")); + } + if let std::option::Option::Some(end) = range.end_inclusive() + && slot > end + { + return std::result::Result::Err(query_invalid("raw_cursor_slot_range")); + } + return std::result::Result::Ok(()); +} + +fn query_invalid(phase: &'static str) -> crate::PostgresBackendError { + return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::QueryInvalid, phase); +} diff --git a/crates/ksp-store-postgres-lib/src/runtime.rs b/crates/ksp-store-postgres-lib/src/runtime.rs index dc141cd..972a0d9 100644 --- a/crates/ksp-store-postgres-lib/src/runtime.rs +++ b/crates/ksp-store-postgres-lib/src/runtime.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/src/runtime.rs -// version: 7 +// version: 8 const APPLICATION_NAME: &str = "ksp-store"; const MAX_CONNECTION_URI_BYTES: usize = 4_096; @@ -340,6 +340,14 @@ impl PostgresBackend { return crate::get_raw_transaction(&self.pool, &self.network, reference).await; } + /// Lists deterministic canonical RAW transaction references with a backend-owned opaque continuation cursor. + pub async fn list_raw_transactions( + &self, + query: &ksp_store_api::RawTransactionQuery, + ) -> std::result::Result, crate::PostgresBackendError> { + return crate::list_raw_transactions(&self.pool, &self.network, query).await; + } + /// Reads one persisted RAW transaction observation by producer-owned idempotence key. pub async fn get_raw_transaction_observation( &self, diff --git a/crates/ksp-store-postgres-lib/tests/dependency_boundary.rs b/crates/ksp-store-postgres-lib/tests/dependency_boundary.rs index cbe6c85..b93936d 100644 --- a/crates/ksp-store-postgres-lib/tests/dependency_boundary.rs +++ b/crates/ksp-store-postgres-lib/tests/dependency_boundary.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/tests/dependency_boundary.rs -// version: 10 +// version: 11 #![warn(missing_docs)] #![deny(unreachable_pub)] @@ -168,9 +168,6 @@ fn pre_005_raw_write_sql_is_atomic_idempotent_and_keeps_later_scope_closed() { } for forbidden in [ "DELETE FROM", - "list_raw_transactions", - "RawPage", - "RawCursor", "transition_raw_transaction_retention", "impl ksp_store_api::RawTransactionWrite", "impl ksp_store_api::RawTransactionObservationWrite", @@ -179,3 +176,48 @@ fn pre_005_raw_write_sql_is_atomic_idempotent_and_keeps_later_scope_closed() { } return; } + +#[test] +fn pre_006_raw_pagination_is_keyset_cursor_bound_and_policy_free() { + let raw = include_str!("../src/raw_transaction.rs"); + let cursor = include_str!("../src/raw_transaction/cursor.rs"); + let index = include_str!("../migrations/v001_raw_transaction/indexes/001_ix_ksp_raw_transactions_slot_signature.sql"); + for required in [ + "LIST_TRANSACTIONS_ASC_SQL", + "LIST_TRANSACTIONS_DESC_SQL", + "retention_state <> 'purged'", + "(slot, signature) >", + "(slot, signature) <", + "ORDER BY slot ASC, signature ASC", + "ORDER BY slot DESC, signature DESC", + "LIMIT $5", + "list_raw_transactions", + ] { + assert!(raw.contains(required), "missing pre.006 keyset pagination contract: {required}"); + } + for required in [ + "CURSOR_BYTES: usize = 109", + "CURSOR_MAGIC", + "b\"KSPT\"", + "CURSOR_VERSION: u8 = 1", + "KSP/raw-transaction-cursor/v1", + "sha2::Sha256", + "query.network().as_str()", + "query.direction()", + "query.slots().start_inclusive()", + "query.slots().end_inclusive()", + "last_slot", + "last_signature", + "PageLimitUnsupported", + "9_223_372_036_854_775_806", + ] { + assert!(cursor.contains(required), "missing pre.006 cursor/binding contract: {required}"); + } + assert!(index.contains("ON ksp_raw_transactions (slot, signature)")); + assert!(index.contains("WHERE retention_state <> 'purged'")); + for forbidden in [" OFFSET ", "limit.min(", "clamp(", "500", "1000", "DELETE FROM", "transition_raw_transaction_retention"] { + assert!(!raw.contains(forbidden), "pre.006 contains forbidden pagination/policy/later-scope material: {forbidden}"); + assert!(!cursor.contains(forbidden), "pre.006 cursor contains forbidden pagination/policy/later-scope material: {forbidden}"); + } + return; +} diff --git a/crates/ksp-store-postgres-lib/tests/hardening_completeness.rs b/crates/ksp-store-postgres-lib/tests/hardening_completeness.rs index 90be626..3607c0f 100644 --- a/crates/ksp-store-postgres-lib/tests/hardening_completeness.rs +++ b/crates/ksp-store-postgres-lib/tests/hardening_completeness.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/tests/hardening_completeness.rs -// version: 5 +// version: 6 #![warn(missing_docs)] #![deny(unreachable_pub)] @@ -168,7 +168,13 @@ fn pre_009_backend_error_bridge_cannot_retain_external_error_or_secret_text() { let runtime = include_str!("../src/runtime.rs"); assert!(runtime.contains("deadpool_postgres::PoolError::Backend(_)")); assert!(!runtime.contains("deadpool_postgres::PoolError::Backend(error)")); - for source in [runtime, include_str!("../src/migration.rs"), include_str!("../src/health.rs"), include_str!("../src/raw_transaction.rs")] { + for source in [ + runtime, + include_str!("../src/migration.rs"), + include_str!("../src/health.rs"), + include_str!("../src/raw_transaction.rs"), + include_str!("../src/raw_transaction/cursor.rs"), + ] { for forbidden in ["format!(\"{error", "format!(\"{error:?", "error = ?", "error = %"] { assert!(!source.contains(forbidden), "backend source renders external error material: {forbidden}"); } @@ -191,6 +197,7 @@ fn pre_009_backend_has_no_env_bypass_or_direct_store_trait_implementation() { include_str!("../src/lib.rs"), include_str!("../src/migration.rs"), include_str!("../src/raw_transaction.rs"), + include_str!("../src/raw_transaction/cursor.rs"), include_str!("../src/runtime.rs"), include_str!("../src/schema.rs") ); diff --git a/crates/ksp-store-postgres-lib/tests/public_api.rs b/crates/ksp-store-postgres-lib/tests/public_api.rs index 2e8a6bf..8c76adf 100644 --- a/crates/ksp-store-postgres-lib/tests/public_api.rs +++ b/crates/ksp-store-postgres-lib/tests/public_api.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/tests/public_api.rs -// version: 6 +// version: 7 #![warn(missing_docs)] #![deny(unreachable_pub)] @@ -44,6 +44,8 @@ fn pre_005_backend_error_projection_is_safe_and_static() { ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed, ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed, ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch, + ksp_store_postgres_lib::PostgresBackendErrorKind::PageLimitUnsupported, + ksp_store_postgres_lib::PostgresBackendErrorKind::QueryInvalid, ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed, ksp_store_postgres_lib::PostgresBackendErrorKind::ReferenceNotFound, ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer, @@ -52,7 +54,7 @@ fn pre_005_backend_error_projection_is_safe_and_static() { ksp_store_postgres_lib::PostgresBackendErrorKind::WriteFailed, ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork, ]; - assert_eq!(kinds.len(), 15); + assert_eq!(kinds.len(), 17); return; } @@ -87,3 +89,9 @@ fn pre_005_raw_write_bridge_uses_only_backend_independent_models_and_outcomes() let _observation = ksp_store_postgres_lib::PostgresBackend::record_raw_transaction_observation; return; } + +#[test] +fn pre_006_raw_list_bridge_uses_backend_independent_query_page_and_reference_models() { + let _list = ksp_store_postgres_lib::PostgresBackend::list_raw_transactions; + return; +} diff --git a/crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs b/crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs index b698ccc..2d0f358 100644 --- a/crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs +++ b/crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs @@ -1,5 +1,5 @@ // file: crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs -// version: 2 +// version: 3 fn network() -> ksp_store_api::RawNetworkId { return match ksp_store_api::RawNetworkId::new("devnet") { @@ -353,3 +353,108 @@ fn pre_005_observation_origin_encoding_is_exact_and_static() { assert_eq!(super::encode_origin(ksp_store_api::RawAcquisitionOrigin::Replay), std::result::Result::Ok("replay")); return; } + +fn page_query( + network_id: &str, + start: std::option::Option, + end: std::option::Option, + direction: ksp_store_api::RawSortDirection, + cursor: std::option::Option, +) -> ksp_store_api::RawTransactionQuery { + let network = match ksp_store_api::RawNetworkId::new(network_id) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid page-query network rejected: {error:?}"), + }; + let range = match ksp_store_api::RawSlotRange::new(start, end) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid page-query range rejected: {error:?}"), + }; + let limit = match ksp_store_api::RawPageLimit::new(2) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid page limit rejected: {error:?}"), + }; + let page = match cursor { + std::option::Option::Some(value) => ksp_store_api::RawPageRequest::after(limit, value), + std::option::Option::None => ksp_store_api::RawPageRequest::first(limit), + }; + return ksp_store_api::RawTransactionQuery::new(network, range, direction, page); +} + +#[test] +fn pre_006_cursor_v1_round_trip_is_fixed_109_bytes_and_exact() { + let query = page_query("devnet", std::option::Option::Some(10), std::option::Option::Some(20), ksp_store_api::RawSortDirection::Ascending, None); + let signature = ksp_store_api::RawTransactionSignature::new([7_u8; 64]); + let cursor = match crate::encode_raw_transaction_cursor(&query, 15, &signature) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"), + }; + assert_eq!(cursor.as_bytes().len(), 109); + assert_eq!(cursor.as_bytes().get(0..4), std::option::Option::Some(b"KSPT".as_ref())); + assert_eq!(cursor.as_bytes().get(4).copied(), std::option::Option::Some(1)); + let decoded = match crate::decode_raw_transaction_cursor(&query, &cursor) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid cursor decoding rejected: {error:?}"), + }; + assert_eq!(decoded.last_slot, 15); + assert_eq!(decoded.last_signature, [7_u8; 64]); + assert_eq!(std::format!("{cursor:?}"), "RawPageCursor { len: 109 }"); + return; +} + +#[test] +fn pre_006_cursor_v1_rejects_replay_across_network_direction_and_range() { + let query = page_query("devnet", std::option::Option::Some(10), std::option::Option::Some(20), ksp_store_api::RawSortDirection::Ascending, None); + let signature = ksp_store_api::RawTransactionSignature::new([9_u8; 64]); + let cursor = match crate::encode_raw_transaction_cursor(&query, 15, &signature) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"), + }; + let other_network = page_query("mainnet-beta", Some(10), Some(20), ksp_store_api::RawSortDirection::Ascending, None); + let other_direction = page_query("devnet", Some(10), Some(20), ksp_store_api::RawSortDirection::Descending, None); + let other_range = page_query("devnet", Some(11), Some(20), ksp_store_api::RawSortDirection::Ascending, None); + for candidate in [&other_network, &other_direction, &other_range] { + let error = crate::decode_raw_transaction_cursor(candidate, &cursor).err(); + assert_eq!(error.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::QueryInvalid)); + } + let outside_range = page_query("devnet", Some(16), Some(20), ksp_store_api::RawSortDirection::Ascending, None); + let error = crate::decode_raw_transaction_cursor(&outside_range, &cursor).err(); + assert_eq!(error.map(|value| return value.phase()), Some("raw_cursor_slot_range")); + return; +} + +#[test] +fn pre_006_cursor_v1_hostile_size_magic_version_and_digest_are_rejected() { + let query = page_query("devnet", None, None, ksp_store_api::RawSortDirection::Ascending, None); + for size in [1_usize, 108, 109, 110, 4_096] { + let raw = match ksp_store_api::RawPageCursor::try_new(vec![0_u8; size].into_boxed_slice()) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("bounded hostile cursor rejected by API before backend test: {error:?}"), + }; + assert!(crate::decode_raw_transaction_cursor(&query, &raw).is_err()); + } + let signature = ksp_store_api::RawTransactionSignature::new([3_u8; 64]); + let valid = match crate::encode_raw_transaction_cursor(&query, 42, &signature) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("valid cursor encoding rejected: {error:?}"), + }; + for index in [0_usize, 4, 108] { + let mut bytes = valid.as_bytes().to_vec(); + bytes[index] ^= 0xff; + let hostile = match ksp_store_api::RawPageCursor::try_new(bytes.into_boxed_slice()) { + std::result::Result::Ok(value) => value, + std::result::Result::Err(error) => panic!("fixed-size hostile cursor rejected by API before backend test: {error:?}"), + }; + let error = crate::decode_raw_transaction_cursor(&query, &hostile).err(); + assert_eq!(error.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::QueryInvalid)); + } + return; +} + +#[test] +fn pre_006_page_limit_exposes_only_the_real_postgres_limit_plus_one_boundary() { + let maximum = crate::raw_transaction_physical_page_limit(9_223_372_036_854_775_806); + assert_eq!(maximum, Ok((9_223_372_036_854_775_806_usize, i64::MAX))); + let rejected = crate::raw_transaction_physical_page_limit(9_223_372_036_854_775_807).err(); + assert_eq!(rejected.map(|value| return value.kind()), Some(crate::PostgresBackendErrorKind::PageLimitUnsupported)); + return; +} diff --git a/deltas/0.3.3/pre.006.md b/deltas/0.3.3/pre.006.md new file mode 100644 index 0000000..2b1e4d9 --- /dev/null +++ b/deltas/0.3.3/pre.006.md @@ -0,0 +1,311 @@ + + + +# Delta `0.3.3-pre.006` — pagination keyset et cursor V1 RawTransaction + +## 1. Base et gate d'entrée + +Base opérateur obligatoire : + +```text +0.3.3-pre.5 +``` + +Le gate opérateur fourni le 2026-08-30 est entièrement vert : + +```text +cargo fmt --all PASS +scripts/audit_rust_workspace_rules.py PASS +scripts/audit_markdown_tables.py PASS +cargo check --workspace PASS +cargo clippy --workspace --all-targets PASS +cargo test -p ksp-store-api PASS +cargo test -p ksp-store-lib PASS +cargo test -p ksp-store-postgres-lib PASS (30 unit backend, live foundation ignoré) +cargo test -p ksp-config-lib PASS (128 unit + ownership/public API) +cargo check -p ksp-store-lib --no-default-features PASS +``` + +La tranche peut donc ouvrir la pagination sans rouvrir l'écriture atomique acquise en `pre.005`. + +## 2. Version + +```text +workspace.package.version = 0.3.3-pre.6 +``` + +## 3. Scope exact + +Cette tranche ajoute uniquement la navigation backend PostgreSQL des transactions RAW : + +```text +PostgresBackend::list_raw_transactions +RawTransactionQuery +RawPage +RawPageCursor backend-private V1 +``` + +Ne sont pas ouverts : + +```text +transitions Full -> Archived -> Purged +compare-and-transition de rétention +implémentations complètes des six traits RawTransaction* +dispatch ksp-store-lib +RawAccountState +worker/job policy +``` + +## 4. Ordre physique et keyset + +L'ordre total reste celui figé en `pre.001` : + +```text +Ascending = (slot ASC, signature ASC) +Descending = (slot DESC, signature DESC) +``` + +Les deux requêtes privées utilisent : + +```text +retention_state <> 'purged' +slot >= start_inclusive si présent +slot <= end_inclusive si présent +(slot, signature) > cursor pour Ascending +(slot, signature) < cursor pour Descending +LIMIT requested + 1 +``` + +Aucun `OFFSET` n'est utilisé. La signature 64 octets constitue le tie-breaker déterministe pour plusieurs transactions au même slot. + +Le prédicat et l'ordre correspondent à l'index V001 existant : + +```text +ix_ksp_raw_transactions_slot_signature +ON ksp_raw_transactions (slot, signature) +WHERE retention_state <> 'purged' +``` + +Aucune migration n'est modifiée. + +## 5. Cursor V1 + +Le cursor est implémenté dans un sous-module privé au domaine `raw_transaction`. + +Format fixe : + +```text +magic 4 bytes = KSPT +version 1 byte = 1 +last_slot 8 bytes = u64 big-endian +last_signature 64 bytes +binding_digest 32 bytes = SHA-256 +-------------------------------------- +total 109 bytes +``` + +Le digest utilise le domaine séparé : + +```text +KSP/raw-transaction-cursor/v1 +``` + +Il lie de façon non ambiguë : + +```text +longueur du network + network +direction +flag start + start slot si présent +flag end + end slot si présent +last_slot +last_signature +``` + +Le digest n'est ni un secret ni un mécanisme d'autorisation. Il sert uniquement à détecter corruption et replay sous un autre contexte de navigation. + +Le decode rejette avant `pool.get()` : + +```text +taille != 109 +magic incorrect +version inconnue +digest divergent +network différent +range différente +direction différente +last_slot hors range +bytes hostiles bornés par RawPageCursor +``` + +Ces cas sont classés `PostgresBackendErrorKind::QueryInvalid` avec phase statique uniquement. + +## 6. Limite physique réelle + +`RawPageLimit` reste sans cap métier KSP. + +Le backend doit obtenir une row supplémentaire pour déterminer la présence d'une continuation : + +```text +LIMIT requested + 1 +``` + +La borne physique exacte retenue sur PostgreSQL est donc : + +```text +requested <= i64::MAX - 1 +``` + +Une valeur supérieure produit : + +```text +PostgresBackendErrorKind::PageLimitUnsupported +``` + +Aucun clamp vers `100`, `500`, `1000` ou une autre taille de worker/job n'est introduit. + +## 7. Décodage des pages + +Toutes les rows retournées par le statement, y compris la row de probe `limit+1`, passent par le mapping fallible : + +```text +signature BYTEA -> [u8; 64] +slot NUMERIC(20,0) -> texte décimal -> u64 +``` + +Une corruption physique n'est donc pas masquée simplement parce qu'elle se trouve sur la row supplémentaire. + +Si plus de `requested` rows sont valides : + +1. la row de probe est retirée du résultat ; +2. le cursor est construit depuis la dernière row réellement retournée ; +3. la page contient exactement au plus `requested` références. + +Les tombstones `Purged` ne sont jamais listés. + +## 8. Concurrence et snapshot + +Chaque page est cohérente au niveau de son statement PostgreSQL, mais la navigation ne prétend pas fournir un snapshot inter-pages. + +Une mutation concurrente qui insère une clé ordonnée avant le cursor déjà consommé peut ne pas être vue par cette navigation. Cette propriété est désormais explicitement documentée et n'est pas confondue avec une garantie de replay transactionnel. + +## 9. Erreurs sûres + +Deux kinds backend supplémentaires sont matérialisés : + +```text +PageLimitUnsupported +QueryInvalid +``` + +Comme les autres erreurs physiques, ils ne conservent que : + +```text +kind +phase &'static str +``` + +Aucun cursor, network hostile, signature, query SQL, bind, SQLSTATE ou texte PostgreSQL n'est rendu. + +## 10. Tests et canaris + +Les tests unitaires ajoutés couvrent : + +```text +round-trip cursor V1 exact +109 bytes exacts +magic/version exacts +replay autre network +replay autre direction +replay autre range +last_slot hors range +taille hostile 1/108/109/110/4096 +mutation magic/version/digest +borne page i64::MAX - 1 +rejet i64::MAX +``` + +Les canaris d'intégration figent : + +```text +keyset > / < +ASC/DESC total +pas d'OFFSET +exclusion Purged +index V001 compatible +pas de cap 500/1000 +pas de DELETE +pas de transition de rétention +pas d'implémentation complète de trait prématurée +``` + +La pagination PostgreSQL réelle multi-page avec ties/ranges reste réservée au gate live opt-in `pre.009`. + +## 11. Migrations + +Les migrations sont byte-identiques à `pre.005` : + +```text +V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450 +V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51 +``` + +V001 conserve exactement : + +```text +4 tables +35 contraintes +1 index +40 ressources +``` + +## 12. Fichiers ajoutés/modifiés + +```text +Cargo.toml +crates/ksp-store-postgres-lib/README.md +crates/ksp-store-postgres-lib/USAGE.md +crates/ksp-store-postgres-lib/src/error.rs +crates/ksp-store-postgres-lib/src/lib.rs +crates/ksp-store-postgres-lib/src/raw_transaction.rs +crates/ksp-store-postgres-lib/src/raw_transaction/cursor.rs +crates/ksp-store-postgres-lib/src/runtime.rs +crates/ksp-store-postgres-lib/tests/dependency_boundary.rs +crates/ksp-store-postgres-lib/tests/hardening_completeness.rs +crates/ksp-store-postgres-lib/tests/public_api.rs +crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs +docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md +docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md +deltas/0.3.3/pre.006.md +``` + +Aucun fichier n'est supprimé. + +## 13. Audits exécutables dans l'environnement de génération + +```text +python3 scripts/audit_rust_workspace_rules.py +python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3 +``` + +Les commandes Cargo/rustfmt ne sont pas disponibles dans l'environnement de génération. Elles doivent rester `NON EXÉCUTÉES` jusqu'au gate opérateur ; elles ne doivent jamais être présentées comme PASS sans sortie réelle. + +## 14. Gate opérateur + +```bash +cargo fmt --all +python3 scripts/audit_rust_workspace_rules.py +python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3 +cargo check --workspace +cargo clippy --workspace --all-targets +cargo test -p ksp-store-api +cargo test -p ksp-store-lib +cargo test -p ksp-store-postgres-lib +cargo test -p ksp-config-lib +cargo check -p ksp-store-lib --no-default-features +``` + +## 15. Suite après gate vert + +```text +0.3.3-pre.007 — archive/purge/tombstone/compare-and-transition + Compacted unsupported stable +``` diff --git a/docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md b/docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md index bef221f..324dda0 100644 --- a/docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md +++ b/docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md @@ -1,5 +1,5 @@ - + # Plan `0.3.3` — Store/PostgreSQL RawTransaction vertical slice @@ -28,15 +28,18 @@ Tranches techniques validées : ```text 0.3.3-pre.001 — audit, threat model, physical design, sizing et planning 0.3.3-pre.002 — moteur de migrations multi-version +0.3.3-pre.003 + fix.001/fix.002/fix.003 — V001 physique, compatibilité de schéma et binding réseau +0.3.3-pre.004 — mapping et lectures RAW +0.3.3-pre.005 — écriture atomique et observations ``` Tranche technique courante : ```text -0.3.3-pre.003 — V001 physique + binding réseau +0.3.3-pre.006 — pagination keyset + cursor V1 ``` -Les gates opérateur de `pre.001` et `pre.002` sont verts. `pre.003` matérialise le schéma physique V001 et le binding mono-réseau atomique, mais n'ouvre encore ni repository PostgreSQL métier, ni capability RAW, ni dispatch dans `ksp-store-lib`. +Les gates opérateur jusqu'à `pre.005` sont verts. `pre.006` complète la surface backend-specific de lecture `RawTransaction` avec `list_raw_transactions`, sans ouvrir encore la rétention mutante, les implémentations complètes des six traits ni le dispatch dans `ksp-store-lib`. ## 2. Sources et autorité @@ -1108,12 +1111,23 @@ Tranche matérialisée : ### `0.3.3-pre.006` — pagination/cursor -- ordre total ; -- index utilisé ; -- cursor V1 opaque ; -- binding de query ; -- limites physiques explicites ; -- matrice hostile. +Tranche matérialisée : + +- `PostgresBackend::list_raw_transactions` retourne uniquement `RawPage` ; +- mauvais réseau rejeté avant acquisition d'un client du pool ; +- lignes `Purged` exclues de la liste ; +- bornes `RawSlotRange` appliquées de façon inclusive ; +- ordre total `Ascending = (slot ASC, signature ASC)` et `Descending = (slot DESC, signature DESC)` ; +- continuation keyset stricte `(slot, signature) > cursor` ou `< cursor`, sans `OFFSET` ; +- requêtes alignées avec l'index partiel V001 `(slot, signature) WHERE retention_state <> 'purged'` ; +- cursor V1 backend-private de 109 octets : magic `KSPT`, version 1, `last_slot`, `last_signature`, digest SHA-256 ; +- digest lié avec domaine séparé au network, à la direction, aux flags/valeurs des bornes et à la dernière clé ; +- replay sous autre network/range/direction, taille/magic/version/digest hostiles et `last_slot` hors range -> `QueryInvalid` avant pool I/O ; +- `RawPageLimit` sans cap métier : `LIMIT requested + 1`, borne PostgreSQL exacte `requested <= i64::MAX - 1`, sinon `PageLimitUnsupported` sans clamp ; +- la row `limit+1` est décodée elle aussi afin de ne pas masquer une corruption physique ; +- aucune promesse de snapshot inter-pages ; une mutation concurrente placée avant le cursor peut ne pas être observée ; +- aucune transition de rétention mutante et aucune implémentation complète de trait dans cette tranche ; +- V000/V001 restent byte-identiques. ### `0.3.3-pre.007` — archive, purge, tombstone, ForceRehydrate diff --git a/docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md b/docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md index 6333779..7f0ff30 100644 --- a/docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md +++ b/docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md @@ -1,5 +1,5 @@ - + # Validation `0.3.3` — Store/PostgreSQL RawTransaction vertical slice @@ -580,12 +580,21 @@ cap 500/1000 dans Store pagination - observation write séparée, key identique/divergente, absent/purged : PASS statique/unit design ; - `Conflict` / `ReferenceNotFound` / `WriteFailed` sans texte serveur : PASS statique ; - preuve PostgreSQL concurrente/rollback réelle : différée à `pre.009` ; -- gate Cargo opérateur : À EXÉCUTER. +- gate opérateur complet du 2026-08-30 : PASS (`check`, Clippy, Store/API/PostgreSQL/Config, no-default-features ; 30 tests unit backend). ### `pre.006` -- list/cursor/order/ranges ; -- hostile cursor matrix. +- `list_raw_transactions` backend-specific et mauvais réseau pré-I/O : PASS statique/unit design ; +- ordre total ASC/DESC `(slot, signature)` et keyset stricte sans OFFSET : PASS statique ; +- bornes inclusives et exclusion `Purged` : PASS statique ; +- cursor V1 fixe 109 octets, magic/version/last key/digest : PASS unit design ; +- binding digest network/direction/range + replay hostile rejeté : PASS unit design ; +- taille/magic/version/digest/random bytes hostiles : PASS unit design ; +- `last_slot` hors range rejeté : PASS unit design ; +- `LIMIT requested + 1`, limite `i64::MAX - 1`, aucun cap worker : PASS unit/statique ; +- index V001 `(slot, signature)` partiel réutilisé, migration inchangée : PASS statique ; +- snapshot inter-pages explicitement non garanti : PASS doc ; +- gate Cargo opérateur : À EXÉCUTER. ### `pre.007` @@ -622,7 +631,7 @@ cap 500/1000 dans Store pagination - publication stable. -## 22. Gate courant `pre.005` +## 22. Gate courant `pre.006` ```bash cargo fmt --all @@ -687,4 +696,15 @@ cargo check -p ksp-store-lib --no-default-features - [PASS] référence canonique absente classée `ReferenceNotFound` ; - [PASS] erreurs physiques d'écriture classées `WriteFailed` sans conserver le texte PostgreSQL ; - [PASS] aucune migration SQL modifiée et aucun scope pagination/rétention mutante ouvert ; -- [À FAIRE] gate Cargo opérateur complet de `pre.005`. +- [PASS] gate Cargo opérateur complet de `pre.005` fourni le 2026-08-30. +### `pre.006` — état de la tranche + +- [PASS] bridge `list_raw_transactions` ajouté sans type PostgreSQL public ; +- [PASS] keyset stricte ASC/DESC sur `(slot, signature)`, sans `OFFSET` ; +- [PASS] bornes de slots inclusives et tombstones `Purged` exclus ; +- [PASS] cursor V1 fixe 109 octets et digest lié à la query ; +- [PASS] replay network/direction/range et cursor hostile classés `QueryInvalid` ; +- [PASS] limite physique `requested <= i64::MAX - 1`, au-delà `PageLimitUnsupported` sans clamp ; +- [PASS] aucune migration SQL modifiée, aucun scope rétention mutante/façade ouvert ; +- [À FAIRE] gate Cargo opérateur complet de `pre.006`. +