v0.3.3-pre.004
This commit is contained in:
@@ -1,12 +1,12 @@
|
||||
# file: Cargo.toml
|
||||
# version: 350
|
||||
# version: 351
|
||||
|
||||
[workspace]
|
||||
resolver = "3"
|
||||
members = ["crates/ksp-app-config-desk", "crates/ksp-app-solprices-desk", "crates/ksp-app-wallet-desk", "crates/ksp-config-lib", "crates/ksp-core-lib", "crates/ksp-interface-lib", "crates/ksp-logging-lib", "crates/ksp-offchain-transport-lib", "crates/ksp-onchain-transport-lib", "crates/ksp-program-api", "crates/ksp-store-api", "crates/ksp-store-lib", "crates/ksp-store-postgres-lib", "crates/ksp-wallet-lib"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.3.3-pre.3.fix.3"
|
||||
version = "0.3.3-pre.4"
|
||||
edition = "2024"
|
||||
license = "MIT"
|
||||
repository = "https://git.sasedev.com/Sasedev/khadhroony-solana-project"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: crates/ksp-store-postgres-lib/README.md -->
|
||||
<!-- version: 2 -->
|
||||
<!-- version: 3 -->
|
||||
|
||||
# ksp-store-postgres-lib
|
||||
|
||||
@@ -83,7 +83,7 @@ ksp_store_schema_migrations
|
||||
|
||||
Le moteur vérifie version, nom et checksum SHA-256, sérialise les runners par advisory transaction lock et refuse une history divergente ou plus récente que le runtime.
|
||||
|
||||
Aucune migration métier RAW n'appartient à cette fondation.
|
||||
V001 possède désormais le schéma physique `RawTransaction` et son contrat de compatibilité ; les opérations métier restent introduites par tranches afin de préserver des gates courts et vérifiables.
|
||||
|
||||
## Health et erreurs
|
||||
|
||||
@@ -104,14 +104,29 @@ La politique de support de `0.3.2` fixe PostgreSQL 15 comme major minimal. Le te
|
||||
|
||||
La preuve opérateur réelle et le major effectivement exercé sont conservés dans la matrice de validation, pas dans cette documentation durable.
|
||||
|
||||
## Lectures RAW `0.3.3-pre.004`
|
||||
|
||||
Le backend expose désormais quatre lectures étroites qui retournent uniquement des modèles `ksp-store-api` :
|
||||
|
||||
```text
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Le SQL et les rows restent privés au backend. Le mapping PostgreSQL est fallible et couvre notamment `NUMERIC(20,0) -> u64`, `BIGINT -> u32/u64`, timestamps bornés, bytes de taille fixe et codes de provenance. Une ligne stockée incompatible produit uniquement `PostgresBackendErrorKind::DataInvalid`; un échec de SELECT produit `ReadFailed`. Les lectures portant un `RawTransactionReference` rejettent un réseau différent avant toute acquisition de pool.
|
||||
|
||||
`Full` lit le payload chaud, `Archived` le reconstruit depuis la relation archive et `Purged` retourne `None`; le tombstone reste accessible séparément pour `Purged`.
|
||||
|
||||
## Hors périmètre actuel
|
||||
|
||||
La crate ne contient encore :
|
||||
|
||||
- aucune implémentation PostgreSQL des capabilities `RawTransaction*` ;
|
||||
- aucune écriture PostgreSQL `RawTransaction*` ;
|
||||
- aucune pagination/listing `RawTransaction` ;
|
||||
- aucune implémentation complète des traits `RawTransaction*` de `ksp-store-api` tant que `list_raw_transactions` manque ;
|
||||
- aucune implémentation PostgreSQL des capabilities `RawAccount*` ;
|
||||
- aucun repository métier RAW ;
|
||||
- aucune table/index métier ;
|
||||
- aucune orchestration worker/job ;
|
||||
- aucun transport d'acquisition ou decoder Program.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: crates/ksp-store-postgres-lib/USAGE.md -->
|
||||
<!-- version: 2 -->
|
||||
<!-- version: 3 -->
|
||||
|
||||
# Utilisation de ksp-store-postgres-lib
|
||||
|
||||
@@ -115,7 +115,7 @@ checksum SHA-256
|
||||
|
||||
Le runner est transactionnel et sérialisé par advisory transaction lock. Une divergence de checksum/nom/version ou une history plus récente est terminale ; aucun down migration automatique n'est exécuté.
|
||||
|
||||
`auto_migrate = false` permet de vérifier l'état sans appliquer de migration pending.
|
||||
`schema_autocreate` contrôle l'initialisation/adoption du schéma et `schema_autoupdate` les migrations pending ainsi que les réparations additives sûres. Le constructeur legacy `auto_migrate` mappe encore les deux politiques pour compatibilité source.
|
||||
|
||||
## 6. Classifier les erreurs sans fuite
|
||||
|
||||
@@ -123,13 +123,16 @@ Le runner est transactionnel et sérialisé par advisory transaction lock. Une d
|
||||
match error.kind() {
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed => {}
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork => {}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
@@ -138,18 +141,40 @@ let _safe_phase = error.phase();
|
||||
|
||||
Ne pas reconstruire un diagnostic utilisateur à partir de l'erreur brute PostgreSQL : cette erreur n'est volontairement pas conservée par le bridge.
|
||||
|
||||
## 7. Ce que cette crate ne permet pas encore
|
||||
## 7. Lectures RAW transaction
|
||||
|
||||
La fondation physique n'implémente pas les traits `RawTransaction*` ou `RawAccount*` de `ksp-store-api`.
|
||||
Depuis `0.3.3-pre.004`, un backend ouvert expose quatre lectures backend-specific retournant exclusivement les modèles communs :
|
||||
|
||||
Un backend ouvert et healthy prouve uniquement :
|
||||
|
||||
```text
|
||||
connexion/pool
|
||||
TLS selon policy
|
||||
bootstrap/history
|
||||
health/readiness
|
||||
close borné
|
||||
```rust
|
||||
let transaction = backend.get_raw_transaction(&reference).await;
|
||||
let observation = backend.get_raw_transaction_observation(&observation_key).await;
|
||||
let retention = backend.get_raw_transaction_retention_state(&reference).await;
|
||||
let tombstone = backend.get_raw_transaction_tombstone(&reference).await;
|
||||
```
|
||||
|
||||
Il ne prouve aucune persistence métier RAW.
|
||||
Le backend ne rend jamais `tokio_postgres::Row`, SQL, SQLSTATE ou valeur de bind. Pour les références réseau-scopées, un mauvais réseau est rejeté avant acquisition d'un client du pool. Une corruption de row est projetée vers `DataInvalid`, un échec physique de lecture vers `ReadFailed`, et les erreurs de pool conservent leur classification bornée existante.
|
||||
|
||||
`get_raw_transaction` retourne :
|
||||
|
||||
```text
|
||||
Full -> payload chaud
|
||||
Archived -> payload archive reconstruit
|
||||
Purged -> None
|
||||
absent -> None
|
||||
```
|
||||
|
||||
Le tombstone `Purged` reste lisible séparément.
|
||||
|
||||
## 8. Ce que cette crate ne permet pas encore
|
||||
|
||||
La tranche ne fournit pas encore :
|
||||
|
||||
```text
|
||||
list_raw_transactions / cursor
|
||||
écritures canonique + observation
|
||||
transitions de rétention
|
||||
implémentations complètes des six traits RawTransaction*
|
||||
capabilities RawAccount*
|
||||
```
|
||||
|
||||
Ces surfaces sont ajoutées dans les prereleases suivantes avant le dispatch `ksp-store-lib`.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/error.rs
|
||||
// version: 4
|
||||
// version: 5
|
||||
|
||||
/// Stable KSP error code reserved for PostgreSQL retention transitions that require unsupported physical compaction.
|
||||
pub const ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED: ksp_store_api::ErrorCode =
|
||||
@@ -17,16 +17,22 @@ pub enum PostgresBackendErrorKind {
|
||||
PoolTimeout,
|
||||
/// A lightweight PostgreSQL health/readiness probe failed without exposing server text or SQL.
|
||||
HealthFailed,
|
||||
/// PostgreSQL returned stored RAW data that cannot be represented by the stable Store API contract.
|
||||
DataInvalid,
|
||||
/// PostgreSQL migration/bootstrap execution failed without exposing server text or SQL.
|
||||
MigrationFailed,
|
||||
/// Applied PostgreSQL migration history diverges from the embedded immutable KSP history.
|
||||
MigrationMismatch,
|
||||
/// A PostgreSQL RAW read statement failed without exposing server text, SQL or bind values.
|
||||
ReadFailed,
|
||||
/// The database schema history contains a migration newer than this runtime understands.
|
||||
SchemaNewer,
|
||||
/// Explicit backend shutdown did not drain inside the supplied deadline.
|
||||
ShutdownTimeout,
|
||||
/// Verified TLS configuration or negotiation could not be established.
|
||||
TlsFailed,
|
||||
/// A network-scoped RAW operation targeted a network different from the backend binding.
|
||||
WrongNetwork,
|
||||
}
|
||||
|
||||
/// Redacted PostgreSQL backend error carrying only a safe classification and static lifecycle phase.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/lib.rs
|
||||
// version: 8
|
||||
// version: 9
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -11,8 +11,9 @@
|
||||
//! pool, explicit Rustls TLS policy, private KSP migration/bootstrap engine and
|
||||
//! safe lightweight health/readiness probe. `0.3.3-pre.003-fix.001` splits
|
||||
//! migrations into versioned physical resources and verifies the effective
|
||||
//! PostgreSQL schema contract before readiness; business capability
|
||||
//! implementations remain deferred to later prereleases.
|
||||
//! PostgreSQL schema contract before readiness. `0.3.3-pre.004` adds exact
|
||||
//! backend-private RAW transaction/observation/retention read mapping without
|
||||
//! exposing PostgreSQL rows or SQL through the public bridge.
|
||||
//!
|
||||
//! This crate depends on `ksp-store-api` and never on `ksp-store-lib`. The
|
||||
//! common facade consumes only this crate's narrow backend bridge and never
|
||||
@@ -22,6 +23,7 @@ mod constants;
|
||||
mod error;
|
||||
mod health;
|
||||
mod migration;
|
||||
mod raw_transaction;
|
||||
mod runtime;
|
||||
mod schema;
|
||||
|
||||
@@ -50,6 +52,14 @@ pub(crate) use self::health::probe_health;
|
||||
pub(crate) use self::migration::bootstrap;
|
||||
/// Current embedded migration version consumed by the private health probe.
|
||||
pub(crate) use self::migration::current_migration_version;
|
||||
/// Private RAW transaction reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction;
|
||||
/// Private RAW transaction observation reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_observation;
|
||||
/// Private RAW transaction retention-state reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_retention_state;
|
||||
/// Private RAW transaction tombstone reader consumed by the physical backend runtime.
|
||||
pub(crate) use self::raw_transaction::get_raw_transaction_tombstone;
|
||||
/// Private Deadpool error mapper shared with the health probe.
|
||||
pub(crate) use self::runtime::map_pool_error;
|
||||
/// Private Deadpool status projector shared with the health probe.
|
||||
|
||||
660
crates/ksp-store-postgres-lib/src/raw_transaction.rs
Normal file
660
crates/ksp-store-postgres-lib/src/raw_transaction.rs
Normal file
@@ -0,0 +1,660 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
// version: 1
|
||||
|
||||
const GET_OBSERVATION_SQL: &str = "SELECT observation_key, transaction_signature, provider, protocol, acquisition_method, origin, received_at_unix_millis, capture_session_id, commitment, endpoint_id, filter_id, observed_at_unix_millis, source_payload_hash, source_payload_size_bytes FROM ksp_raw_transaction_observations WHERE observation_key = $1";
|
||||
const GET_RETENTION_SQL: &str = "SELECT retention_state FROM ksp_raw_transactions WHERE signature = $1";
|
||||
const GET_TOMBSTONE_SQL: &str = "SELECT signature, slot::text AS slot_text, block_time_unix_millis, format_id, format_version, content_hash, retention_state FROM ksp_raw_transactions WHERE signature = $1";
|
||||
const GET_TRANSACTION_SQL: &str = "SELECT transaction_row.signature, transaction_row.slot::text AS slot_text, transaction_row.block_time_unix_millis, transaction_row.format_id, transaction_row.format_version, transaction_row.content_hash, transaction_row.payload, transaction_row.retention_state, archive_row.payload AS archive_payload FROM ksp_raw_transactions AS transaction_row LEFT JOIN ksp_raw_transaction_archive_payloads AS archive_row ON archive_row.signature = transaction_row.signature WHERE transaction_row.signature = $1";
|
||||
|
||||
struct RawObservationDbRow {
|
||||
acquisition_method: std::string::String,
|
||||
capture_session_id: std::option::Option<std::string::String>,
|
||||
commitment: std::option::Option<std::string::String>,
|
||||
endpoint_id: std::option::Option<std::string::String>,
|
||||
filter_id: std::option::Option<std::string::String>,
|
||||
observation_key: std::vec::Vec<u8>,
|
||||
observed_at_unix_millis: std::option::Option<i64>,
|
||||
origin: std::string::String,
|
||||
protocol: std::string::String,
|
||||
provider: std::string::String,
|
||||
received_at_unix_millis: i64,
|
||||
source_payload_hash: std::option::Option<std::vec::Vec<u8>>,
|
||||
source_payload_size_bytes: std::option::Option<i64>,
|
||||
transaction_signature: std::vec::Vec<u8>,
|
||||
}
|
||||
|
||||
struct RawTransactionDbRow {
|
||||
archive_payload: std::option::Option<std::vec::Vec<u8>>,
|
||||
block_time_unix_millis: std::option::Option<i64>,
|
||||
content_hash: std::vec::Vec<u8>,
|
||||
format_id: std::string::String,
|
||||
format_version: i64,
|
||||
payload: std::option::Option<std::vec::Vec<u8>>,
|
||||
retention_state: std::string::String,
|
||||
signature: std::vec::Vec<u8>,
|
||||
slot_text: std::string::String,
|
||||
}
|
||||
|
||||
struct RawTombstoneDbRow {
|
||||
block_time_unix_millis: std::option::Option<i64>,
|
||||
content_hash: std::vec::Vec<u8>,
|
||||
format_id: std::string::String,
|
||||
format_version: i64,
|
||||
retention_state: std::string::String,
|
||||
signature: std::vec::Vec<u8>,
|
||||
slot_text: std::string::String,
|
||||
}
|
||||
|
||||
/// Reads one canonical RAW transaction from the physical PostgreSQL backend.
|
||||
pub(crate) async fn get_raw_transaction(
|
||||
pool: &deadpool_postgres::Pool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransaction>, crate::PostgresBackendError> {
|
||||
let network_result = ensure_network(network, reference, "raw_transaction_network");
|
||||
if let std::result::Result::Err(error) = network_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let client_result = pool.get().await;
|
||||
let client = match client_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(crate::map_pool_error(error)),
|
||||
};
|
||||
let signature = reference.signature();
|
||||
let signature_bytes: &[u8] = signature.as_bytes();
|
||||
let rows_result = client.query(GET_TRANSACTION_SQL, &[&signature_bytes]).await;
|
||||
let rows = match rows_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::ReadFailed, "raw_transaction_query"));
|
||||
},
|
||||
};
|
||||
if rows.is_empty() {
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
}
|
||||
if rows.len() != 1 {
|
||||
return std::result::Result::Err(data_invalid("raw_transaction_cardinality"));
|
||||
}
|
||||
let row = match rows.first() {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_transaction_cardinality")),
|
||||
};
|
||||
let physical = match raw_transaction_db_row(row) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
return decode_raw_transaction_row(network, physical);
|
||||
}
|
||||
|
||||
/// Reads one RAW transaction observation from the physical PostgreSQL backend.
|
||||
pub(crate) async fn get_raw_transaction_observation(
|
||||
pool: &deadpool_postgres::Pool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
observation_key: &ksp_store_api::RawObservationKey,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionObservation>, crate::PostgresBackendError> {
|
||||
let client_result = pool.get().await;
|
||||
let client = match client_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(crate::map_pool_error(error)),
|
||||
};
|
||||
let observation_key_bytes: &[u8] = observation_key.as_bytes();
|
||||
let rows_result = client.query(GET_OBSERVATION_SQL, &[&observation_key_bytes]).await;
|
||||
let rows = match rows_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::ReadFailed, "raw_observation_query"));
|
||||
},
|
||||
};
|
||||
if rows.is_empty() {
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
}
|
||||
if rows.len() != 1 {
|
||||
return std::result::Result::Err(data_invalid("raw_observation_cardinality"));
|
||||
}
|
||||
let row = match rows.first() {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_observation_cardinality")),
|
||||
};
|
||||
let physical = match raw_observation_db_row(row) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let decoded = match decode_raw_observation_row(network, physical) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
if decoded.observation_key() != *observation_key {
|
||||
return std::result::Result::Err(data_invalid("raw_observation_identity"));
|
||||
}
|
||||
return std::result::Result::Ok(std::option::Option::Some(decoded));
|
||||
}
|
||||
|
||||
/// Reads one RAW transaction retention state from the physical PostgreSQL backend.
|
||||
pub(crate) async fn get_raw_transaction_retention_state(
|
||||
pool: &deadpool_postgres::Pool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawRetentionState>, crate::PostgresBackendError> {
|
||||
let network_result = ensure_network(network, reference, "raw_retention_network");
|
||||
if let std::result::Result::Err(error) = network_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let client_result = pool.get().await;
|
||||
let client = match client_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(crate::map_pool_error(error)),
|
||||
};
|
||||
let signature = reference.signature();
|
||||
let signature_bytes: &[u8] = signature.as_bytes();
|
||||
let rows_result = client.query(GET_RETENTION_SQL, &[&signature_bytes]).await;
|
||||
let rows = match rows_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::ReadFailed, "raw_retention_query"));
|
||||
},
|
||||
};
|
||||
if rows.is_empty() {
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
}
|
||||
if rows.len() != 1 {
|
||||
return std::result::Result::Err(data_invalid("raw_retention_cardinality"));
|
||||
}
|
||||
let row = match rows.first() {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_retention_cardinality")),
|
||||
};
|
||||
let state_result = row.try_get::<_, std::string::String>("retention_state");
|
||||
let state = match state_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_retention_decode")),
|
||||
};
|
||||
let decoded = match decode_retention_state(state.as_str()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
return std::result::Result::Ok(std::option::Option::Some(decoded));
|
||||
}
|
||||
|
||||
/// Reads one minimal RAW transaction tombstone from the physical PostgreSQL backend.
|
||||
pub(crate) async fn get_raw_transaction_tombstone(
|
||||
pool: &deadpool_postgres::Pool,
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>, crate::PostgresBackendError> {
|
||||
let network_result = ensure_network(network, reference, "raw_tombstone_network");
|
||||
if let std::result::Result::Err(error) = network_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let client_result = pool.get().await;
|
||||
let client = match client_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(crate::map_pool_error(error)),
|
||||
};
|
||||
let signature = reference.signature();
|
||||
let signature_bytes: &[u8] = signature.as_bytes();
|
||||
let rows_result = client.query(GET_TOMBSTONE_SQL, &[&signature_bytes]).await;
|
||||
let rows = match rows_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::ReadFailed, "raw_tombstone_query"));
|
||||
},
|
||||
};
|
||||
if rows.is_empty() {
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
}
|
||||
if rows.len() != 1 {
|
||||
return std::result::Result::Err(data_invalid("raw_tombstone_cardinality"));
|
||||
}
|
||||
let row = match rows.first() {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_tombstone_cardinality")),
|
||||
};
|
||||
let physical = match raw_tombstone_db_row(row) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let decoded = match decode_raw_tombstone_row(network, physical) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let tombstone = match decoded {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Ok(std::option::Option::None),
|
||||
};
|
||||
if tombstone.reference() != reference {
|
||||
return std::result::Result::Err(data_invalid("raw_tombstone_identity"));
|
||||
}
|
||||
return std::result::Result::Ok(std::option::Option::Some(tombstone));
|
||||
}
|
||||
|
||||
fn raw_transaction_db_row(row: &tokio_postgres::Row) -> std::result::Result<RawTransactionDbRow, crate::PostgresBackendError> {
|
||||
let signature = match row.try_get::<_, std::vec::Vec<u8>>("signature") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let slot_text = match row.try_get::<_, std::string::String>("slot_text") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let block_time_unix_millis = match row.try_get::<_, std::option::Option<i64>>("block_time_unix_millis") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let format_id = match row.try_get::<_, std::string::String>("format_id") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let format_version = match row.try_get::<_, i64>("format_version") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let content_hash = match row.try_get::<_, std::vec::Vec<u8>>("content_hash") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let payload = match row.try_get::<_, std::option::Option<std::vec::Vec<u8>>>("payload") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let retention_state = match row.try_get::<_, std::string::String>("retention_state") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
let archive_payload = match row.try_get::<_, std::option::Option<std::vec::Vec<u8>>>("archive_payload") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_decode")),
|
||||
};
|
||||
return std::result::Result::Ok(RawTransactionDbRow {
|
||||
archive_payload,
|
||||
block_time_unix_millis,
|
||||
content_hash,
|
||||
format_id,
|
||||
format_version,
|
||||
payload,
|
||||
retention_state,
|
||||
signature,
|
||||
slot_text,
|
||||
});
|
||||
}
|
||||
|
||||
fn raw_observation_db_row(row: &tokio_postgres::Row) -> std::result::Result<RawObservationDbRow, crate::PostgresBackendError> {
|
||||
macro_rules! required {
|
||||
($name:literal, $ty:ty) => {
|
||||
match row.try_get::<_, $ty>($name) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_observation_decode")),
|
||||
}
|
||||
};
|
||||
}
|
||||
return std::result::Result::Ok(RawObservationDbRow {
|
||||
acquisition_method: required!("acquisition_method", std::string::String),
|
||||
capture_session_id: required!("capture_session_id", std::option::Option<std::string::String>),
|
||||
commitment: required!("commitment", std::option::Option<std::string::String>),
|
||||
endpoint_id: required!("endpoint_id", std::option::Option<std::string::String>),
|
||||
filter_id: required!("filter_id", std::option::Option<std::string::String>),
|
||||
observation_key: required!("observation_key", std::vec::Vec<u8>),
|
||||
observed_at_unix_millis: required!("observed_at_unix_millis", std::option::Option<i64>),
|
||||
origin: required!("origin", std::string::String),
|
||||
protocol: required!("protocol", std::string::String),
|
||||
provider: required!("provider", std::string::String),
|
||||
received_at_unix_millis: required!("received_at_unix_millis", i64),
|
||||
source_payload_hash: required!("source_payload_hash", std::option::Option<std::vec::Vec<u8>>),
|
||||
source_payload_size_bytes: required!("source_payload_size_bytes", std::option::Option<i64>),
|
||||
transaction_signature: required!("transaction_signature", std::vec::Vec<u8>),
|
||||
});
|
||||
}
|
||||
|
||||
fn raw_tombstone_db_row(row: &tokio_postgres::Row) -> std::result::Result<RawTombstoneDbRow, crate::PostgresBackendError> {
|
||||
let signature = match row.try_get::<_, std::vec::Vec<u8>>("signature") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let slot_text = match row.try_get::<_, std::string::String>("slot_text") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let block_time_unix_millis = match row.try_get::<_, std::option::Option<i64>>("block_time_unix_millis") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let format_id = match row.try_get::<_, std::string::String>("format_id") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let format_version = match row.try_get::<_, i64>("format_version") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let content_hash = match row.try_get::<_, std::vec::Vec<u8>>("content_hash") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
let retention_state = match row.try_get::<_, std::string::String>("retention_state") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_decode")),
|
||||
};
|
||||
return std::result::Result::Ok(RawTombstoneDbRow {
|
||||
block_time_unix_millis,
|
||||
content_hash,
|
||||
format_id,
|
||||
format_version,
|
||||
retention_state,
|
||||
signature,
|
||||
slot_text,
|
||||
});
|
||||
}
|
||||
|
||||
fn decode_raw_transaction_row(
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
row: RawTransactionDbRow,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransaction>, crate::PostgresBackendError> {
|
||||
let signature = match fixed_bytes::<64>(row.signature) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawTransactionSignature::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let slot = match decode_u64_decimal(row.slot_text.as_str(), "raw_transaction_slot") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let block_time = match decode_optional_timestamp(row.block_time_unix_millis, "raw_transaction_block_time") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let format_id = match decode_format_id(row.format_id) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let format_version = match decode_u32_i64(row.format_version, "raw_transaction_format_version") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let content_hash = match fixed_bytes::<32>(row.content_hash) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawContentHash::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let retention_state = match decode_retention_state(row.retention_state.as_str()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let payload_bytes = match retention_state {
|
||||
ksp_store_api::RawRetentionState::Full => {
|
||||
if row.archive_payload.is_some() {
|
||||
return std::result::Result::Err(data_invalid("raw_transaction_full_archive"));
|
||||
}
|
||||
match row.payload {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_transaction_full_payload")),
|
||||
}
|
||||
},
|
||||
ksp_store_api::RawRetentionState::Archived => {
|
||||
if row.payload.is_some() {
|
||||
return std::result::Result::Err(data_invalid("raw_transaction_archived_hot_payload"));
|
||||
}
|
||||
match row.archive_payload {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return std::result::Result::Err(data_invalid("raw_transaction_archive_payload")),
|
||||
}
|
||||
},
|
||||
ksp_store_api::RawRetentionState::Purged => {
|
||||
if row.payload.is_some() || row.archive_payload.is_some() || block_time.is_some() {
|
||||
return std::result::Result::Err(data_invalid("raw_transaction_purged_shape"));
|
||||
}
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
},
|
||||
_ => return std::result::Result::Err(data_invalid("raw_transaction_retention_state")),
|
||||
};
|
||||
let payload = match ksp_store_api::RawPayload::try_new(format_id, format_version, payload_bytes.into_boxed_slice(), content_hash) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_transaction_payload")),
|
||||
};
|
||||
let reference = ksp_store_api::RawTransactionReference::new(network.clone(), signature);
|
||||
return std::result::Result::Ok(std::option::Option::Some(ksp_store_api::RawTransaction::new(reference, slot, block_time, payload)));
|
||||
}
|
||||
|
||||
fn decode_raw_observation_row(
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
row: RawObservationDbRow,
|
||||
) -> std::result::Result<ksp_store_api::RawTransactionObservation, crate::PostgresBackendError> {
|
||||
let observation_key = match fixed_bytes::<32>(row.observation_key) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawObservationKey::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let signature = match fixed_bytes::<64>(row.transaction_signature) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawTransactionSignature::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let provider = match decode_provenance_code(row.provider) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let protocol = match decode_provenance_code(row.protocol) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let acquisition_method = match decode_provenance_code(row.acquisition_method) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let origin = match decode_origin(row.origin.as_str()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let received_at = match decode_timestamp_i64(row.received_at_unix_millis, "raw_observation_received_at") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let mut provenance = ksp_store_api::RawAcquisitionProvenance::new(provider, protocol, acquisition_method, origin, received_at);
|
||||
provenance = match row.capture_session_id {
|
||||
std::option::Option::Some(value) => match decode_provenance_code(value) {
|
||||
std::result::Result::Ok(code) => provenance.with_capture_session_id(code),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.commitment {
|
||||
std::option::Option::Some(value) => match decode_provenance_code(value) {
|
||||
std::result::Result::Ok(code) => provenance.with_commitment(code),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.endpoint_id {
|
||||
std::option::Option::Some(value) => match decode_provenance_code(value) {
|
||||
std::result::Result::Ok(code) => provenance.with_endpoint_id(code),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.filter_id {
|
||||
std::option::Option::Some(value) => match decode_provenance_code(value) {
|
||||
std::result::Result::Ok(code) => provenance.with_filter_id(code),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.observed_at_unix_millis {
|
||||
std::option::Option::Some(value) => {
|
||||
let timestamp = match decode_timestamp_i64(value, "raw_observation_observed_at") {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
match provenance.try_with_observed_at(timestamp) {
|
||||
std::result::Result::Ok(updated) => updated,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_observation_time_order")),
|
||||
}
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.source_payload_hash {
|
||||
std::option::Option::Some(value) => {
|
||||
let hash = match fixed_bytes::<32>(value) {
|
||||
std::result::Result::Ok(decoded) => ksp_store_api::RawContentHash::new(decoded),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
provenance.with_source_payload_hash(hash)
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
provenance = match row.source_payload_size_bytes {
|
||||
std::option::Option::Some(value) => {
|
||||
let size = match decode_u64_i64(value, "raw_observation_source_size") {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
match provenance.try_with_source_payload_size_bytes(size) {
|
||||
std::result::Result::Ok(updated) => updated,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_observation_source_size")),
|
||||
}
|
||||
},
|
||||
std::option::Option::None => provenance,
|
||||
};
|
||||
let transaction = ksp_store_api::RawTransactionReference::new(network.clone(), signature);
|
||||
return std::result::Result::Ok(ksp_store_api::RawTransactionObservation::new(observation_key, transaction, provenance));
|
||||
}
|
||||
|
||||
fn decode_raw_tombstone_row(
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
row: RawTombstoneDbRow,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>, crate::PostgresBackendError> {
|
||||
let state = match decode_retention_state(row.retention_state.as_str()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
if state != ksp_store_api::RawRetentionState::Purged {
|
||||
return std::result::Result::Ok(std::option::Option::None);
|
||||
}
|
||||
if row.block_time_unix_millis.is_some() {
|
||||
return std::result::Result::Err(data_invalid("raw_tombstone_block_time"));
|
||||
}
|
||||
let signature = match fixed_bytes::<64>(row.signature) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawTransactionSignature::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let slot = match decode_u64_decimal(row.slot_text.as_str(), "raw_tombstone_slot") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let format_id = match decode_format_id(row.format_id) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let format_version = match decode_u32_i64(row.format_version, "raw_tombstone_format_version") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let content_hash = match fixed_bytes::<32>(row.content_hash) {
|
||||
std::result::Result::Ok(value) => ksp_store_api::RawContentHash::new(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let reference = ksp_store_api::RawTransactionReference::new(network.clone(), signature);
|
||||
let tombstone = match ksp_store_api::RawTransactionTombstone::try_new(reference, slot, format_id, format_version, content_hash) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid("raw_tombstone_model")),
|
||||
};
|
||||
return std::result::Result::Ok(std::option::Option::Some(tombstone));
|
||||
}
|
||||
|
||||
fn decode_format_id(value: std::string::String) -> std::result::Result<ksp_store_api::RawFormatId, crate::PostgresBackendError> {
|
||||
return match ksp_store_api::RawFormatId::new(value) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid("raw_format_id")),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_origin(value: &str) -> std::result::Result<ksp_store_api::RawAcquisitionOrigin, crate::PostgresBackendError> {
|
||||
return match value {
|
||||
"backfill" => std::result::Result::Ok(ksp_store_api::RawAcquisitionOrigin::Backfill),
|
||||
"import" => std::result::Result::Ok(ksp_store_api::RawAcquisitionOrigin::Import),
|
||||
"live" => std::result::Result::Ok(ksp_store_api::RawAcquisitionOrigin::Live),
|
||||
"repair" => std::result::Result::Ok(ksp_store_api::RawAcquisitionOrigin::Repair),
|
||||
"replay" => std::result::Result::Ok(ksp_store_api::RawAcquisitionOrigin::Replay),
|
||||
_ => std::result::Result::Err(data_invalid("raw_observation_origin")),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_provenance_code(value: std::string::String) -> std::result::Result<ksp_store_api::RawProvenanceCode, crate::PostgresBackendError> {
|
||||
return match ksp_store_api::RawProvenanceCode::new(value) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid("raw_provenance_code")),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_retention_state(value: &str) -> std::result::Result<ksp_store_api::RawRetentionState, crate::PostgresBackendError> {
|
||||
return match value {
|
||||
"full" => std::result::Result::Ok(ksp_store_api::RawRetentionState::Full),
|
||||
"archived" => std::result::Result::Ok(ksp_store_api::RawRetentionState::Archived),
|
||||
"purged" => std::result::Result::Ok(ksp_store_api::RawRetentionState::Purged),
|
||||
_ => std::result::Result::Err(data_invalid("raw_retention_state")),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_optional_timestamp(
|
||||
value: std::option::Option<i64>,
|
||||
phase: &'static str,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTimestamp>, crate::PostgresBackendError> {
|
||||
return match value {
|
||||
std::option::Option::Some(inner) => match decode_timestamp_i64(inner, phase) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(std::option::Option::Some(decoded)),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
},
|
||||
std::option::Option::None => std::result::Result::Ok(std::option::Option::None),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_timestamp_i64(value: i64, phase: &'static str) -> std::result::Result<ksp_store_api::RawTimestamp, crate::PostgresBackendError> {
|
||||
let unsigned = match u64::try_from(value) {
|
||||
std::result::Result::Ok(decoded) => decoded,
|
||||
std::result::Result::Err(_) => return std::result::Result::Err(data_invalid(phase)),
|
||||
};
|
||||
return match ksp_store_api::RawTimestamp::from_unix_millis(unsigned) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid(phase)),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_u32_i64(value: i64, phase: &'static str) -> std::result::Result<u32, crate::PostgresBackendError> {
|
||||
return match u32::try_from(value) {
|
||||
std::result::Result::Ok(decoded) if decoded > 0 => std::result::Result::Ok(decoded),
|
||||
_ => std::result::Result::Err(data_invalid(phase)),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_u64_decimal(value: &str, phase: &'static str) -> std::result::Result<u64, crate::PostgresBackendError> {
|
||||
return match value.parse::<u64>() {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid(phase)),
|
||||
};
|
||||
}
|
||||
|
||||
fn decode_u64_i64(value: i64, phase: &'static str) -> std::result::Result<u64, crate::PostgresBackendError> {
|
||||
return match u64::try_from(value) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid(phase)),
|
||||
};
|
||||
}
|
||||
|
||||
fn fixed_bytes<const N: usize>(value: std::vec::Vec<u8>) -> std::result::Result<[u8; N], crate::PostgresBackendError> {
|
||||
return match <[u8; N]>::try_from(value.as_slice()) {
|
||||
std::result::Result::Ok(decoded) => std::result::Result::Ok(decoded),
|
||||
std::result::Result::Err(_) => std::result::Result::Err(data_invalid("raw_fixed_bytes")),
|
||||
};
|
||||
}
|
||||
|
||||
fn ensure_network(
|
||||
network: &ksp_store_api::RawNetworkId,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
phase: &'static str,
|
||||
) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
if reference.network() != network {
|
||||
return std::result::Result::Err(crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::WrongNetwork, phase));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn data_invalid(phase: &'static str) -> crate::PostgresBackendError {
|
||||
return crate::PostgresBackendError::new(crate::PostgresBackendErrorKind::DataInvalid, phase);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[path = "../unit_tests/raw_transaction.rs"]
|
||||
mod tests;
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
// version: 5
|
||||
// version: 6
|
||||
|
||||
const APPLICATION_NAME: &str = "ksp-store";
|
||||
const MAX_CONNECTION_URI_BYTES: usize = 4_096;
|
||||
@@ -332,6 +332,38 @@ impl PostgresBackend {
|
||||
return crate::probe_health(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Reads one canonical RAW transaction without exposing physical PostgreSQL row types.
|
||||
pub async fn get_raw_transaction(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransaction>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Reads one persisted RAW transaction observation by producer-owned idempotence key.
|
||||
pub async fn get_raw_transaction_observation(
|
||||
&self,
|
||||
observation_key: &ksp_store_api::RawObservationKey,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionObservation>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_observation(&self.pool, &self.network, observation_key).await;
|
||||
}
|
||||
|
||||
/// Reads the retention state of one canonical RAW transaction identity.
|
||||
pub async fn get_raw_transaction_retention_state(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawRetentionState>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_retention_state(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Reads the minimal durable tombstone only when one RAW transaction is purged.
|
||||
pub async fn get_raw_transaction_tombstone(
|
||||
&self,
|
||||
reference: &ksp_store_api::RawTransactionReference,
|
||||
) -> std::result::Result<std::option::Option<ksp_store_api::RawTransactionTombstone>, crate::PostgresBackendError> {
|
||||
return crate::get_raw_transaction_tombstone(&self.pool, &self.network, reference).await;
|
||||
}
|
||||
|
||||
/// Explicitly closes the pool and waits for all owned pooled objects to drain inside the supplied bound.
|
||||
pub async fn close(self, timeout: std::time::Duration) -> std::result::Result<(), crate::PostgresBackendError> {
|
||||
self.pool.close();
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
// version: 8
|
||||
// version: 9
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -38,6 +38,7 @@ fn pre_005_backend_keeps_environment_sql_migrations_and_physical_types_private()
|
||||
assert!(crate_root.contains("mod error;"));
|
||||
assert!(crate_root.contains("mod health;"));
|
||||
assert!(crate_root.contains("mod migration;"));
|
||||
assert!(crate_root.contains("mod raw_transaction;"));
|
||||
assert!(crate_root.contains("mod runtime;"));
|
||||
assert!(crate_root.contains("mod schema;"));
|
||||
assert!(crate_root.contains("const _: &str = crate::TRACING_TARGET;"));
|
||||
@@ -122,3 +123,26 @@ fn pre_003_fix_001_migration_engine_uses_split_schema_contract_and_binds_network
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_raw_read_sql_and_mapping_remain_backend_private_and_read_only() {
|
||||
let crate_root = include_str!("../src/lib.rs");
|
||||
let raw = include_str!("../src/raw_transaction.rs");
|
||||
assert!(crate_root.contains("mod raw_transaction;"));
|
||||
assert!(!crate_root.contains("pub mod raw_transaction"));
|
||||
for required in [
|
||||
"SELECT transaction_row.signature",
|
||||
"ksp_raw_transaction_observations",
|
||||
"slot::text AS slot_text",
|
||||
"RawPayload::try_new",
|
||||
"RawTransactionTombstone::try_new",
|
||||
"PostgresBackendErrorKind::DataInvalid",
|
||||
"PostgresBackendErrorKind::WrongNetwork",
|
||||
] {
|
||||
assert!(raw.contains(required), "missing private RAW read mapping contract: {required}");
|
||||
}
|
||||
for forbidden in ["INSERT INTO", "UPDATE ", "DELETE FROM", "std::env", "dotenv", "ksp_store_lib", "ksp_config_lib"] {
|
||||
assert!(!raw.contains(forbidden), "pre.004 RAW read module contains forbidden ownership/write material: {forbidden}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
// version: 3
|
||||
// version: 4
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -109,7 +109,7 @@ fn assert_pre_io_rejection(connection_uri: &str, tls_mode: ksp_store_postgres_li
|
||||
#[test]
|
||||
fn pre_009_backend_modules_exports_and_manifest_dependencies_are_exact() {
|
||||
let crate_root = include_str!("../src/lib.rs");
|
||||
for required in ["mod constants;", "mod error;", "mod health;", "mod migration;", "mod runtime;", "mod schema;"] {
|
||||
for required in ["mod constants;", "mod error;", "mod health;", "mod migration;", "mod raw_transaction;", "mod runtime;", "mod schema;"] {
|
||||
assert!(crate_root.contains(required), "missing PostgreSQL backend module: {required}");
|
||||
}
|
||||
assert!(!crate_root.contains("pub mod "));
|
||||
@@ -168,7 +168,7 @@ fn pre_009_backend_error_bridge_cannot_retain_external_error_or_secret_text() {
|
||||
let runtime = include_str!("../src/runtime.rs");
|
||||
assert!(runtime.contains("deadpool_postgres::PoolError::Backend(_)"));
|
||||
assert!(!runtime.contains("deadpool_postgres::PoolError::Backend(error)"));
|
||||
for source in [runtime, include_str!("../src/migration.rs"), include_str!("../src/health.rs")] {
|
||||
for source in [runtime, include_str!("../src/migration.rs"), include_str!("../src/health.rs"), include_str!("../src/raw_transaction.rs")] {
|
||||
for forbidden in ["format!(\"{error", "format!(\"{error:?", "error = ?", "error = %"] {
|
||||
assert!(!source.contains(forbidden), "backend source renders external error material: {forbidden}");
|
||||
}
|
||||
@@ -177,18 +177,20 @@ fn pre_009_backend_error_bridge_cannot_retain_external_error_or_secret_text() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_009_backend_has_no_env_bypass_or_business_persistence_capability() {
|
||||
fn pre_009_backend_has_no_env_bypass_or_business_write_capability() {
|
||||
let production = std::format!(
|
||||
"{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}
|
||||
{}",
|
||||
include_str!("../src/error.rs"),
|
||||
include_str!("../src/health.rs"),
|
||||
include_str!("../src/lib.rs"),
|
||||
include_str!("../src/migration.rs"),
|
||||
include_str!("../src/raw_transaction.rs"),
|
||||
include_str!("../src/runtime.rs"),
|
||||
include_str!("../src/schema.rs")
|
||||
);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
// version: 4
|
||||
// version: 5
|
||||
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
@@ -38,15 +38,18 @@ fn pre_005_backend_error_projection_is_safe_and_static() {
|
||||
let kinds = [
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConfigInvalid,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ConnectFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::DataInvalid,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::PoolTimeout,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::HealthFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::MigrationMismatch,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ReadFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::SchemaNewer,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::ShutdownTimeout,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::TlsFailed,
|
||||
ksp_store_postgres_lib::PostgresBackendErrorKind::WrongNetwork,
|
||||
];
|
||||
assert_eq!(kinds.len(), 9);
|
||||
assert_eq!(kinds.len(), 12);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -65,3 +68,12 @@ fn pre_003_retention_compaction_error_code_matches_store_contract_value() {
|
||||
assert_eq!(ksp_store_postgres_lib::ERROR_CODE_POSTGRES_RETENTION_COMPACTION_UNSUPPORTED.code(), "postgres_retention_compaction_unsupported",);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_raw_read_bridge_uses_only_backend_independent_models() {
|
||||
let _get = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction;
|
||||
let _observation = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_observation;
|
||||
let _retention = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_retention_state;
|
||||
let _tombstone = ksp_store_postgres_lib::PostgresBackend::get_raw_transaction_tombstone;
|
||||
return;
|
||||
}
|
||||
|
||||
183
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
Normal file
183
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
Normal file
@@ -0,0 +1,183 @@
|
||||
// file: crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
// version: 1
|
||||
|
||||
fn network() -> ksp_store_api::RawNetworkId {
|
||||
return match ksp_store_api::RawNetworkId::new("devnet") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid test network rejected: {error:?}"),
|
||||
};
|
||||
}
|
||||
|
||||
fn transaction_row(state: &str) -> super::RawTransactionDbRow {
|
||||
return super::RawTransactionDbRow {
|
||||
archive_payload: std::option::Option::None,
|
||||
block_time_unix_millis: std::option::Option::Some(1_700_000_000_000),
|
||||
content_hash: vec![7; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: i64::from(u32::MAX),
|
||||
payload: std::option::Option::Some(vec![1, 2, 3, 4]),
|
||||
retention_state: state.to_owned(),
|
||||
signature: vec![9; 64],
|
||||
slot_text: u64::MAX.to_string(),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_full_and_archived_rows_round_trip_without_integer_narrowing() {
|
||||
let network = network();
|
||||
let full = super::decode_raw_transaction_row(&network, transaction_row("full"));
|
||||
let full = match full {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid full row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(full.slot(), u64::MAX);
|
||||
assert_eq!(full.block_time().map(|value| return value.unix_millis()), std::option::Option::Some(1_700_000_000_000));
|
||||
assert_eq!(full.payload().format_version(), u32::MAX);
|
||||
assert_eq!(full.payload().bytes(), &[1, 2, 3, 4]);
|
||||
assert_eq!(full.reference().network().as_str(), "devnet");
|
||||
assert_eq!(full.reference().signature().as_bytes(), &[9; 64]);
|
||||
let mut archived_row = transaction_row("archived");
|
||||
archived_row.payload = std::option::Option::None;
|
||||
archived_row.archive_payload = std::option::Option::Some(vec![5, 6, 7]);
|
||||
let archived = super::decode_raw_transaction_row(&network, archived_row);
|
||||
let archived = match archived {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid archived row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(archived.slot(), u64::MAX);
|
||||
assert_eq!(archived.payload().bytes(), &[5, 6, 7]);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_purged_get_returns_none_and_rejects_payload_or_block_time_residue() {
|
||||
let network = network();
|
||||
let mut purged_row = transaction_row("purged");
|
||||
purged_row.payload = std::option::Option::None;
|
||||
purged_row.archive_payload = std::option::Option::None;
|
||||
purged_row.block_time_unix_millis = std::option::Option::None;
|
||||
let purged = super::decode_raw_transaction_row(&network, purged_row);
|
||||
assert!(matches!(purged, std::result::Result::Ok(std::option::Option::None)));
|
||||
let mut malformed = transaction_row("purged");
|
||||
malformed.archive_payload = std::option::Option::None;
|
||||
malformed.block_time_unix_millis = std::option::Option::None;
|
||||
let malformed = super::decode_raw_transaction_row(&network, malformed);
|
||||
assert_eq!(malformed.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_observation_row_reconstructs_complete_safe_provenance() {
|
||||
let network = network();
|
||||
let row = super::RawObservationDbRow {
|
||||
acquisition_method: "transactionSubscribe".to_owned(),
|
||||
capture_session_id: std::option::Option::Some("capture-1".to_owned()),
|
||||
commitment: std::option::Option::Some("confirmed".to_owned()),
|
||||
endpoint_id: std::option::Option::Some("publicnode-devnet".to_owned()),
|
||||
filter_id: std::option::Option::Some("filter-1".to_owned()),
|
||||
observation_key: vec![3; 32],
|
||||
observed_at_unix_millis: std::option::Option::Some(1_699_999_999_000),
|
||||
origin: "live".to_owned(),
|
||||
protocol: "solana-ws".to_owned(),
|
||||
provider: "publicnode".to_owned(),
|
||||
received_at_unix_millis: 1_700_000_000_000,
|
||||
source_payload_hash: std::option::Option::Some(vec![4; 32]),
|
||||
source_payload_size_bytes: std::option::Option::Some(64 * 1024 * 1024),
|
||||
transaction_signature: vec![8; 64],
|
||||
};
|
||||
let decoded = super::decode_raw_observation_row(&network, row);
|
||||
let decoded = match decoded {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid observation row rejected: {error:?}"),
|
||||
};
|
||||
assert_eq!(decoded.observation_key().as_bytes(), &[3; 32]);
|
||||
assert_eq!(decoded.transaction().network().as_str(), "devnet");
|
||||
assert_eq!(decoded.transaction().signature().as_bytes(), &[8; 64]);
|
||||
assert_eq!(decoded.provenance().provider().as_str(), "publicnode");
|
||||
assert_eq!(decoded.provenance().protocol().as_str(), "solana-ws");
|
||||
assert_eq!(decoded.provenance().acquisition_method().as_str(), "transactionSubscribe");
|
||||
assert_eq!(decoded.provenance().capture_session_id().map(|value| return value.as_str()), std::option::Option::Some("capture-1"));
|
||||
assert_eq!(decoded.provenance().commitment().map(|value| return value.as_str()), std::option::Option::Some("confirmed"));
|
||||
assert_eq!(decoded.provenance().endpoint_id().map(|value| return value.as_str()), std::option::Option::Some("publicnode-devnet"));
|
||||
assert_eq!(decoded.provenance().filter_id().map(|value| return value.as_str()), std::option::Option::Some("filter-1"));
|
||||
assert_eq!(decoded.provenance().observed_at().map(|value| return value.unix_millis()), std::option::Option::Some(1_699_999_999_000));
|
||||
assert_eq!(decoded.provenance().received_at().unix_millis(), 1_700_000_000_000);
|
||||
assert_eq!(decoded.provenance().source_payload_hash().map(|value| return *value.as_bytes()), std::option::Option::Some([4; 32]));
|
||||
assert_eq!(decoded.provenance().source_payload_size_bytes(), std::option::Option::Some(64 * 1024 * 1024));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_hostile_rows_map_to_static_data_invalid_without_echoing_values() {
|
||||
let network = network();
|
||||
let mut oversized_slot = transaction_row("full");
|
||||
oversized_slot.slot_text = "18446744073709551616".to_owned();
|
||||
let slot_error = super::decode_raw_transaction_row(&network, oversized_slot).err();
|
||||
assert_eq!(slot_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert_eq!(slot_error.as_ref().map(|value| return value.phase()), std::option::Option::Some("raw_transaction_slot"));
|
||||
let mut bad_hash = transaction_row("full");
|
||||
bad_hash.content_hash = vec![1; 31];
|
||||
let hash_error = super::decode_raw_transaction_row(&network, bad_hash).err();
|
||||
assert_eq!(hash_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert!(!format!("{hash_error:?}").contains("18446744073709551616"));
|
||||
let bad_origin = super::decode_origin("https://hostile.invalid/secret");
|
||||
let origin_error = bad_origin.err();
|
||||
assert_eq!(origin_error.as_ref().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
assert!(!format!("{origin_error:?}").contains("hostile.invalid"));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_retention_and_tombstone_decoding_is_exact() {
|
||||
assert!(matches!(super::decode_retention_state("full"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Full)));
|
||||
assert!(matches!(super::decode_retention_state("archived"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Archived)));
|
||||
assert!(matches!(super::decode_retention_state("purged"), std::result::Result::Ok(ksp_store_api::RawRetentionState::Purged)));
|
||||
assert_eq!(
|
||||
super::decode_retention_state("compacted").err().map(|value| return value.kind()),
|
||||
std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid),
|
||||
);
|
||||
let network = network();
|
||||
let row = super::RawTombstoneDbRow {
|
||||
block_time_unix_millis: std::option::Option::None,
|
||||
content_hash: vec![6; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: i64::from(u32::MAX),
|
||||
retention_state: "purged".to_owned(),
|
||||
signature: vec![2; 64],
|
||||
slot_text: u64::MAX.to_string(),
|
||||
};
|
||||
let tombstone = super::decode_raw_tombstone_row(&network, row);
|
||||
let tombstone = match tombstone {
|
||||
std::result::Result::Ok(std::option::Option::Some(value)) => value,
|
||||
other => panic!("valid tombstone row rejected: {other:?}"),
|
||||
};
|
||||
assert_eq!(tombstone.slot(), u64::MAX);
|
||||
assert_eq!(tombstone.format_version(), u32::MAX);
|
||||
assert_eq!(tombstone.content_hash().as_bytes(), &[6; 32]);
|
||||
assert_eq!(tombstone.reference().signature().as_bytes(), &[2; 64]);
|
||||
let malformed = super::RawTombstoneDbRow {
|
||||
block_time_unix_millis: std::option::Option::Some(1),
|
||||
content_hash: vec![6; 32],
|
||||
format_id: "ksp.raw.transaction".to_owned(),
|
||||
format_version: 1,
|
||||
retention_state: "purged".to_owned(),
|
||||
signature: vec![2; 64],
|
||||
slot_text: "1".to_owned(),
|
||||
};
|
||||
let malformed = super::decode_raw_tombstone_row(&network, malformed);
|
||||
assert_eq!(malformed.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::DataInvalid));
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pre_004_wrong_network_is_rejected_by_the_private_pre_io_guard() {
|
||||
let backend_network = network();
|
||||
let other_network = match ksp_store_api::RawNetworkId::new("mainnet-beta") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("valid alternate network rejected: {error:?}"),
|
||||
};
|
||||
let reference = ksp_store_api::RawTransactionReference::new(other_network, ksp_store_api::RawTransactionSignature::new([1; 64]));
|
||||
let rejected = super::ensure_network(&backend_network, &reference, "test_network");
|
||||
assert_eq!(rejected.err().map(|value| return value.kind()), std::option::Option::Some(crate::PostgresBackendErrorKind::WrongNetwork));
|
||||
return;
|
||||
}
|
||||
323
deltas/0.3.3/pre.004.md
Normal file
323
deltas/0.3.3/pre.004.md
Normal file
@@ -0,0 +1,323 @@
|
||||
<!-- file: deltas/0.3.3/pre.004.md -->
|
||||
<!-- version: 1 -->
|
||||
|
||||
# Delta `0.3.3-pre.004` — mapping PostgreSQL privé et lectures RawTransaction
|
||||
|
||||
## 1. Base requise
|
||||
|
||||
```text
|
||||
0.3.3-pre.3.fix.3
|
||||
```
|
||||
|
||||
Le gate opérateur fourni pour `pre.003-fix.003` est entièrement vert :
|
||||
|
||||
```text
|
||||
cargo fmt --all PASS
|
||||
audit Rust général / exports / workspace PASS
|
||||
audit Markdown PASS — 214 tables / 136 files
|
||||
cargo check --workspace PASS
|
||||
cargo clippy --workspace --all-targets PASS
|
||||
cargo test -p ksp-store-api PASS
|
||||
cargo test -p ksp-store-lib PASS
|
||||
cargo test -p ksp-store-postgres-lib PASS — 19 unit tests + canaris, live ignored
|
||||
cargo test -p ksp-config-lib PASS — 128 unit tests + ownership/public API
|
||||
cargo check -p ksp-store-lib --no-default-features PASS
|
||||
```
|
||||
|
||||
La V001 split, son contrat catalogue et les politiques `schema_autocreate/schema_autoupdate` sont donc considérés acquis.
|
||||
|
||||
## 2. Objectif
|
||||
|
||||
Implémenter la tranche lecture de la vertical slice PostgreSQL `RawTransaction` sans ouvrir encore les écritures ni la pagination :
|
||||
|
||||
```text
|
||||
mapping SQL privé
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Les méthodes du backend retournent uniquement des modèles `ksp-store-api`. Aucun `tokio_postgres::Row`, SQL, SQLSTATE ou bind ne traverse le bridge public.
|
||||
|
||||
## 3. Version
|
||||
|
||||
Le workspace passe à :
|
||||
|
||||
```text
|
||||
0.3.3-pre.4
|
||||
```
|
||||
|
||||
## 4. Module physique privé
|
||||
|
||||
Nouveau module :
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
```
|
||||
|
||||
Il possède les SELECT et les codecs physiques de cette tranche. `runtime.rs` ne contient aucun SQL métier et délègue les quatre lectures au module privé.
|
||||
|
||||
La tranche reste strictement read-only :
|
||||
|
||||
```text
|
||||
INSERT INTO absent
|
||||
UPDATE absent
|
||||
DELETE FROM absent
|
||||
```
|
||||
|
||||
Les écritures canonique/observation restent `pre.005`; `list_raw_transactions` et le cursor restent `pre.006`.
|
||||
|
||||
## 5. Conversion physique exacte
|
||||
|
||||
Le mapping respecte les contrats `ksp-store-api` sans narrowing :
|
||||
|
||||
```text
|
||||
slot NUMERIC(20,0) -> PostgreSQL ::text -> parse u64
|
||||
format_version BIGINT -> u32::try_from
|
||||
block/observed/received BIGINT -> u64::try_from -> RawTimestamp
|
||||
source payload size BIGINT -> u64::try_from + borne API
|
||||
signature -> BYTEA exactement 64 bytes
|
||||
hash/key -> BYTEA exactement 32 bytes
|
||||
format/provenance codes -> constructeurs API fallibles
|
||||
```
|
||||
|
||||
Le chemin `slot` couvre explicitement :
|
||||
|
||||
```text
|
||||
0
|
||||
i64::MAX
|
||||
i64::MAX + 1
|
||||
u64::MAX
|
||||
```
|
||||
|
||||
Aucune dépendance décimale supplémentaire n'est introduite.
|
||||
|
||||
## 6. `get_raw_transaction`
|
||||
|
||||
La requête lit la ligne canonique avec `LEFT JOIN` du payload archive.
|
||||
|
||||
Décodage :
|
||||
|
||||
```text
|
||||
Full
|
||||
payload chaud obligatoire
|
||||
payload archive interdit
|
||||
-> Some(RawTransaction)
|
||||
|
||||
Archived
|
||||
payload chaud absent
|
||||
payload archive obligatoire
|
||||
-> Some(RawTransaction)
|
||||
|
||||
Purged
|
||||
payload chaud absent
|
||||
payload archive absent
|
||||
block_time absent
|
||||
-> None
|
||||
```
|
||||
|
||||
Toute incohérence de forme, entier hors domaine, code invalide, signature/hash mal dimensionné ou payload incompatible produit `DataInvalid` sans conserver la valeur hostile.
|
||||
|
||||
## 7. Observation
|
||||
|
||||
`get_raw_transaction_observation` reconstruit :
|
||||
|
||||
```text
|
||||
observation_key
|
||||
transaction signature
|
||||
provider
|
||||
protocol
|
||||
acquisition_method
|
||||
origin
|
||||
received_at
|
||||
capture_session_id optionnel
|
||||
commitment optionnel
|
||||
endpoint_id optionnel
|
||||
filter_id optionnel
|
||||
observed_at optionnel
|
||||
source_payload_hash optionnel
|
||||
source_payload_size_bytes optionnel
|
||||
```
|
||||
|
||||
`RawObservationKey` ne porte pas de réseau ; le `RawTransactionReference` reconstruit utilise donc exclusivement le réseau mono-backend déjà vérifié à l'ouverture.
|
||||
|
||||
L'ordre temporel `observed_at <= received_at` est revalidé par le constructeur API.
|
||||
|
||||
## 8. Rétention et tombstone
|
||||
|
||||
`get_raw_transaction_retention_state` mappe uniquement les états physiques supportés par V001 :
|
||||
|
||||
```text
|
||||
full
|
||||
archived
|
||||
purged
|
||||
```
|
||||
|
||||
Une valeur inconnue ou `compacted` stockée physiquement est une corruption `DataInvalid`; `Compacted` n'a toujours aucune représentation PostgreSQL prétendue.
|
||||
|
||||
`get_raw_transaction_tombstone` retourne `Some` uniquement pour `Purged`, vérifie également que `block_time` est absent, puis reconstruit exactement :
|
||||
|
||||
```text
|
||||
network + signature
|
||||
slot
|
||||
format_id
|
||||
format_version
|
||||
content_hash
|
||||
```
|
||||
|
||||
Pour `Full` ou `Archived`, le résultat est `None`.
|
||||
|
||||
## 9. Réseau pré-I/O
|
||||
|
||||
Les trois lectures portant un `RawTransactionReference` passent par un garde privé commun avant `pool.get()` :
|
||||
|
||||
```text
|
||||
reference.network == backend.network -> continuer
|
||||
sinon -> WrongNetwork
|
||||
```
|
||||
|
||||
Le mauvais réseau ne consomme donc aucune connexion PostgreSQL et ne dépend d'aucun SQL.
|
||||
|
||||
## 10. Erreurs backend
|
||||
|
||||
`PostgresBackendErrorKind` ajoute :
|
||||
|
||||
```text
|
||||
DataInvalid
|
||||
ReadFailed
|
||||
WrongNetwork
|
||||
```
|
||||
|
||||
Classification :
|
||||
|
||||
```text
|
||||
pool wait/connect -> kinds existants PoolTimeout/ConnectFailed
|
||||
SELECT/driver read -> ReadFailed
|
||||
row/cardinality/model -> DataInvalid
|
||||
reference mauvais réseau -> WrongNetwork
|
||||
```
|
||||
|
||||
`PostgresBackendError` reste composé uniquement de :
|
||||
|
||||
```text
|
||||
kind
|
||||
phase &'static str
|
||||
```
|
||||
|
||||
Aucune erreur externe n'est retenue ni rendue.
|
||||
|
||||
## 11. Surface backend
|
||||
|
||||
`PostgresBackend` expose désormais :
|
||||
|
||||
```text
|
||||
get_raw_transaction
|
||||
get_raw_transaction_observation
|
||||
get_raw_transaction_retention_state
|
||||
get_raw_transaction_tombstone
|
||||
```
|
||||
|
||||
Ces méthodes ne constituent pas encore les implémentations finales des traits `RawTransaction*` : `RawTransactionRead` exige également `list_raw_transactions`, réservé à `pre.006`. Les six traits seront complets avant leur dispatch par `ksp-store-lib` en `pre.008`.
|
||||
|
||||
## 12. Tests déterministes
|
||||
|
||||
Le nouveau miroir `unit_tests/raw_transaction.rs` couvre notamment :
|
||||
|
||||
```text
|
||||
Full u64::MAX
|
||||
Archived u64::MAX
|
||||
Purged -> None
|
||||
résidu payload/block_time sur Purged -> DataInvalid
|
||||
provenance complète/optionnelle
|
||||
source payload max
|
||||
slot > u64::MAX -> DataInvalid
|
||||
hash mal dimensionné -> DataInvalid
|
||||
origin hostile -> DataInvalid sans écho
|
||||
états Full/Archived/Purged
|
||||
Compacted physique -> DataInvalid
|
||||
tombstone u64::MAX/u32::MAX
|
||||
wrong-network pré-I/O
|
||||
```
|
||||
|
||||
Les canaris d'intégration figent également :
|
||||
|
||||
```text
|
||||
module SQL privé
|
||||
aucun write SQL en pre.004
|
||||
aucune dépendance nouvelle
|
||||
surface publique sans types PostgreSQL
|
||||
nouveaux error kinds sûrs
|
||||
```
|
||||
|
||||
## 13. Migrations
|
||||
|
||||
Aucune ressource de migration n'est modifiée.
|
||||
|
||||
Les checksums restent :
|
||||
|
||||
```text
|
||||
V000 d29068b8c13b9dc0cc9ef6aaadd0fa12d41e0fe4c56541a1118c4bfc846a1450
|
||||
V001 31488cda2f08f3f46c4cdbdbb6c18c243662fada02eac4487040c8735d72cc51
|
||||
```
|
||||
|
||||
## 14. Fichiers ajoutés
|
||||
|
||||
```text
|
||||
crates/ksp-store-postgres-lib/src/raw_transaction.rs
|
||||
crates/ksp-store-postgres-lib/unit_tests/raw_transaction.rs
|
||||
deltas/0.3.3/pre.004.md
|
||||
```
|
||||
|
||||
## 15. Fichiers modifiés
|
||||
|
||||
```text
|
||||
Cargo.toml
|
||||
crates/ksp-store-postgres-lib/README.md
|
||||
crates/ksp-store-postgres-lib/USAGE.md
|
||||
crates/ksp-store-postgres-lib/src/error.rs
|
||||
crates/ksp-store-postgres-lib/src/lib.rs
|
||||
crates/ksp-store-postgres-lib/src/runtime.rs
|
||||
crates/ksp-store-postgres-lib/tests/dependency_boundary.rs
|
||||
crates/ksp-store-postgres-lib/tests/hardening_completeness.rs
|
||||
crates/ksp-store-postgres-lib/tests/public_api.rs
|
||||
docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md
|
||||
docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md
|
||||
```
|
||||
|
||||
## 16. Fichiers supprimés
|
||||
|
||||
Aucun.
|
||||
|
||||
## 17. Hors scope confirmé
|
||||
|
||||
Aucun changement n'est apporté à :
|
||||
|
||||
```text
|
||||
ksp-store-api contrats
|
||||
ksp-store-lib dispatch métier
|
||||
écriture RawTransaction
|
||||
écriture observation
|
||||
pagination/cursor
|
||||
transition rétention
|
||||
RawAccountState
|
||||
worker/job/app
|
||||
migration SQL V000/V001
|
||||
```
|
||||
|
||||
## 18. Gate opérateur
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
python3 scripts/audit_rust_workspace_rules.py
|
||||
python3 scripts/audit_markdown_tables.py README.md RULES.md ROADMAP.md CHANGELOG.md docs prompts crates deltas/0.3.3
|
||||
cargo check --workspace
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo test -p ksp-store-api
|
||||
cargo test -p ksp-store-lib
|
||||
cargo test -p ksp-store-postgres-lib
|
||||
cargo test -p ksp-config-lib
|
||||
cargo check -p ksp-store-lib --no-default-features
|
||||
```
|
||||
|
||||
La tranche n'est validée qu'après gate opérateur entièrement vert.
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: docs/plans/024-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION_PLAN.md -->
|
||||
<!-- version: 4 -->
|
||||
<!-- version: 5 -->
|
||||
|
||||
# Plan `0.3.3` — Store/PostgreSQL RawTransaction vertical slice
|
||||
|
||||
@@ -1074,12 +1074,19 @@ Une base PostgreSQL sur laquelle le commit `pre.003` aurait déjà enregistré l
|
||||
|
||||
### `0.3.3-pre.004` — mapping et lectures
|
||||
|
||||
- codecs SQL privés ;
|
||||
- conversions exactes `u64/u32/timestamps/bytes/codes` ;
|
||||
- `get_raw_transaction` ;
|
||||
- `get_raw_transaction_observation` ;
|
||||
- lectures rétention/tombstone ;
|
||||
- corruption DB -> erreurs sûres.
|
||||
Tranche matérialisée :
|
||||
|
||||
- module SQL/mapping `raw_transaction` privé au backend ;
|
||||
- aucune dépendance physique nouvelle ;
|
||||
- `slot NUMERIC(20,0)` lu en décimal texte puis converti falliblement vers `u64`, couvrant `u64::MAX` sans narrowing ;
|
||||
- `format_version BIGINT -> u32`, timestamps/source-size `BIGINT -> u64`, bytes fixes et codes tous décodés falliblement ;
|
||||
- `get_raw_transaction` : `Full` depuis payload chaud, `Archived` via `LEFT JOIN` archive, `Purged -> None`, incohérence hot/archive -> `DataInvalid` ;
|
||||
- `get_raw_transaction_observation` : reconstruction exacte de la provenance complète/optionnelle et réseau issu du backend mono-réseau ;
|
||||
- `get_raw_transaction_retention_state` et `get_raw_transaction_tombstone`, avec validation de l'absence de `block_time` sur un tombstone `Purged` ;
|
||||
- références réseau-scopées rejetées via garde `WrongNetwork` avant acquisition d'un client du pool ;
|
||||
- nouveaux kinds backend sûrs `ReadFailed`, `DataInvalid`, `WrongNetwork`, sans texte serveur/SQL/bind ;
|
||||
- tests déterministes des bornes, rows hostiles, `Full/Archived/Purged`, tombstone et provenance ;
|
||||
- aucune écriture, aucun cursor/list et aucune implémentation complète de trait avant les tranches dédiées.
|
||||
|
||||
### `0.3.3-pre.005` — écriture atomique et observations
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<!-- file: docs/validation/020-V0_3_3_STORE_POSTGRES_RAW_TRANSACTION.md -->
|
||||
<!-- version: 6 -->
|
||||
<!-- version: 7 -->
|
||||
|
||||
# Validation `0.3.3` — Store/PostgreSQL RawTransaction vertical slice
|
||||
|
||||
@@ -551,12 +551,24 @@ cap 500/1000 dans Store pagination
|
||||
- binding des paramètres catalogue `&str` vers `ToSql` corrigé avec `&&str` : PASS statique ;
|
||||
- constantes de contrat renommées sans préfixe `KSP_` réservé au namespace d’environnement Config : PASS statique ;
|
||||
- aucune ressource SQL, aucun ordre de migration et aucun checksum V000/V001 modifié ;
|
||||
- gate Cargo opérateur : À EXÉCUTER.
|
||||
- gate opérateur : FAIL Clippy (`question_mark_used` + warning) et ancien `include_str!` V000 du test live.
|
||||
|
||||
### `pre.003-fix.003`
|
||||
|
||||
- warning `unused variable`, opérateurs `?` et canari test Clippy corrigés : PASS ;
|
||||
- `postgres_foundation_live` relocalisé vers V000 versionnée : PASS ;
|
||||
- README/USAGE backend réconciliés avec l'arborescence split : PASS ;
|
||||
- gate opérateur complet du 2026-08-30 : PASS (`check`, Clippy, Store/API/PostgreSQL/Config, no-default-features).
|
||||
|
||||
### `pre.004`
|
||||
|
||||
- row codecs et lectures ;
|
||||
- malformed DB matrix.
|
||||
- quatre lectures RAW backend-specific sans fuite de row/SQL : PASS statique ;
|
||||
- conversion `NUMERIC(20,0) -> u64` jusqu'à `u64::MAX` et conversions entières/timestamps fallibles : PASS unit design ;
|
||||
- `Full/Archived/Purged`, observation complète, rétention/tombstone : PASS unit design ;
|
||||
- mauvais réseau avant pool I/O : PASS unit design ;
|
||||
- malformed DB -> `DataInvalid`, SELECT -> `ReadFailed`, aucune valeur hostile retenue : PASS statique/unit design ;
|
||||
- SQL strictement read-only dans cette tranche : PASS canari source ;
|
||||
- gate Cargo opérateur : À EXÉCUTER.
|
||||
|
||||
### `pre.005`
|
||||
|
||||
@@ -603,7 +615,7 @@ cap 500/1000 dans Store pagination
|
||||
|
||||
- publication stable.
|
||||
|
||||
## 22. Gate courant `pre.002`
|
||||
## 22. Gate courant `pre.004`
|
||||
|
||||
```bash
|
||||
cargo fmt --all
|
||||
@@ -643,4 +655,16 @@ cargo check -p ksp-store-lib --no-default-features
|
||||
- [PASS] canari unitaire sans assertion constante ;
|
||||
- [PASS] test live V000 relocalisé vers `migrations/v000_bootstrap/tables/001_ksp_store_schema_migrations.sql` ;
|
||||
- [PASS] aucune ressource SQL V000/V001 modifiée ; checksums inchangés ;
|
||||
- [À FAIRE] gate Cargo opérateur complet de `fix.003`.
|
||||
- [PASS] gate opérateur complet de `fix.003` fourni le 2026-08-30.
|
||||
|
||||
### `pre.004` — état de la tranche
|
||||
|
||||
- [PASS] module privé `raw_transaction` avec SELECT uniquement ;
|
||||
- [PASS] `get_raw_transaction` Full/Archived/Purged ;
|
||||
- [PASS] `get_raw_transaction_observation` et provenance complète ;
|
||||
- [PASS] lectures rétention/tombstone ;
|
||||
- [PASS] conversions fallibles `NUMERIC(20,0)/BIGINT/BYTEA/TEXT` sans narrowing ;
|
||||
- [PASS] wrong-network gardé avant pool I/O ;
|
||||
- [PASS] `ReadFailed` / `DataInvalid` / `WrongNetwork` sans texte externe ;
|
||||
- [PASS] tests déterministes de row mapping et hostile data ajoutés ;
|
||||
- [À FAIRE] gate Cargo opérateur complet de `pre.004`.
|
||||
|
||||
Reference in New Issue
Block a user