Sasedev/khadhroony-solana-project
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.3.13-pre.010

Browse Source
This commit is contained in:
SinuS von SifriduS
2026-09-10 21:02:29 +02:00
parent 0f8f38ff52
commit f636169783
16 changed files with 476 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
crates/ksp-worker-raw-transaction-ingest-lib/README.md
View File

@@ -1,5 +1,5 @@
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/README.md -->
<!-- version: 9 -->
<!-- version: 10 -->
# ksp-worker-raw-transaction-ingest-lib
@@ -213,15 +213,23 @@ Un `BlockMeta` ou `Slot` continuity-only est settled localement sans produire de
Le reconnect/replay Yellowstone appartient à Transport. Le Worker n'écrit pas `from_slot` et n'interprète pas directement `SubscribeReplayInfo`.
Le snapshot Worker projette uniquement des informations sûres :
Le snapshot Worker projette uniquement des informations sûres et source-neutral :
```text
source_total
source_active
source_reconnecting
source_failed
source_state
source_failure_total
backpressure_wait_total
source_reconnect_total
source_replay_attempt_total
source_continuity_gap_total
```
Les quatre comptes de sources sont des gauges latest-value et ne contiennent aucune identité de source. `source_total` reste le nombre de sources logiques configurées pour le run. Tant que toutes les sources attendues ne sont pas `Active`, la health publique reste conservative : un reconnect transitoire projette `Degraded`, une source `Failed` projette `Unhealthy`, et le retour de toutes les sources attendues à `Active` permet de revenir à `Healthy`.
`RawTransactionIngestSourceState` distingue :
```text

12
crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md
View File

@@ -1,5 +1,5 @@
<!-- file: crates/ksp-worker-raw-transaction-ingest-lib/USAGE.md -->
<!-- version: 9 -->
<!-- version: 10 -->
# Utilisation de ksp-worker-raw-transaction-ingest-lib
@@ -281,6 +281,10 @@ let in_flight = current.in_flight_persistence();
let hydration_pending = current.hydration_pending();
let frontier = current.processing_frontier_slot();
let oldest_pending = current.oldest_pending_slot();
let source_total = current.source_total();
let source_active = current.source_active();
let source_reconnecting = current.source_reconnecting();
let source_failed = current.source_failed();
let source_state = current.source_state();
```
@@ -329,7 +333,7 @@ let replay_attempts = snapshot.source_replay_attempt_total();
let proven_gaps = snapshot.source_continuity_gap_total();
```
`admission_queue_depth()`, `in_flight_persistence()` et `hydration_pending()` sont des gauges latest-value. Les compteurs cumulés ne wrapent jamais silencieusement ; l'épuisement est terminal avec `worker_raw_transaction_ingest.counter_exhausted`.
`admission_queue_depth()`, `in_flight_persistence()`, `hydration_pending()`, `source_total()`, `source_active()`, `source_reconnecting()` et `source_failed()` sont des gauges latest-value. Les compteurs cumulés ne wrapent jamais silencieusement ; l'épuisement est terminal avec `worker_raw_transaction_ingest.counter_exhausted`.
## Interpréter la processing frontier
@@ -348,7 +352,9 @@ Un `Missing` HTTP règle le signal du point de vue source-processing sans créer
## Interpréter reconnect et replay
`source_state()` peut retourner `Active`, `Reconnecting`, `Closing`, `Closed` ou `Failed` après démarrage de la source productive.
`source_state()` peut retourner `Active`, `Reconnecting`, `Closing`, `Closed` ou `Failed` comme état agrégé source-neutral. Les gauges `source_total()`, `source_active()`, `source_reconnecting()` et `source_failed()` permettent d'interpréter une composition multi-source sans exposer provider, endpoint, filtre ou `source_key`.
La health commune est conservative : pendant `Running`, au moins une source `Failed` donne `Unhealthy`; au moins une source `Reconnecting`, ou une composition dont toutes les sources attendues ne sont pas encore `Active`, donne `Degraded`; toutes les sources attendues `Active` permettent `Healthy`. Une transition Worker `Faulted` reste `Unhealthy`.
Les compteurs ont des sémantiques distinctes :

11
crates/ksp-worker-raw-transaction-ingest-lib/src/lib.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/lib.rs
// version: 24
// version: 25
#![warn(missing_docs)]
#![deny(unreachable_pub)]
@@ -10,11 +10,10 @@
//! This tranche owns the concrete Worker family identity, validated technical settings
//! and the caller-runtime-owned lifecycle with private child-task supervision. This tranche also
//! owns bounded source-neutral admission, common RAW canonicalization/assembly and backend-neutral
//! Store persistence in normal mode plus concrete latest-value snapshots projected onto Worker API. `pre.006` keeps the bounded 1..32 caller-composed
//! aggregate and adds productive HTTP live block polling beside Yellowstone, Standard Logs, Standard Block and Helius Transaction while
//! simultaneous multi-source activation remains gated until the dedicated supervisor tranche. Helius Full notifications, Standard Logs and Yellowstone
//! reference paths converge into one source-neutral hydration coordinator contract; qualified Standard Block Legacy/V0/V1 transactions enter the existing
//! central admission path directly. Yellowstone
//! Store persistence in normal mode plus concrete latest-value snapshots projected onto Worker API. The bounded 1..32 caller-composed aggregate starts
//! all validated Yellowstone, Standard Logs, Standard Block, Helius Transaction and HTTP live block polling sources under one private supervisor. Reference-bearing
//! sources share bounded cross-source hydration and fairness budgets; direct-qualified Standard Block and HTTP polling transactions enter the existing central
//! admission path directly. Public snapshots expose only source-neutral aggregate counts/state and conservative Worker health. Yellowstone
//! Transaction/TransactionStatus/Block and standard logs notifications become signature/slot references; BlockMeta/Slot remain continuity-only signals.
//! Hydration is coalesced by network/signature/commitment under bounded in-flight and pending budgets. A bounded run-local processing frontier projects
//! hydration pending, oldest pending slot and highest unblocked actually observed slot. The productive source also projects safe Transport reconnect/replay

9
crates/ksp-worker-raw-transaction-ingest-lib/src/runtime.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/runtime.rs
// version: 15
// version: 16
type PersistencePort = std::sync::Arc<dyn crate::RawTransactionIngestPersistencePort + 'static>;
type PersistenceTasks = tokio::task::JoinSet<ksp_core_lib::Result<crate::RawTransactionIngestPersistenceOutcome>>;
@@ -117,6 +117,7 @@ impl crate::RawTransactionIngestWorker {
if let std::result::Result::Err(error) = runtime_resources.validate_network(settings.network()) {
return std::result::Result::Err(error);
}
let source_total = runtime_resources.source_count();
let source_settings = settings.clone();
let (processing_frontier_sender, processing_frontier_receiver) =
tokio::sync::watch::channel(crate::RawTransactionIngestProcessingFrontierProjection::empty());
@@ -126,6 +127,7 @@ impl crate::RawTransactionIngestWorker {
runtime,
std::option::Option::Some(port),
std::option::Option::Some(processing_frontier_receiver),
source_total,
move |children, stop_receiver, admission_sender| {
let _abort_handle = children.spawn(async move {
return runtime_resources.run_live_sources(source_settings, stop_receiver, admission_sender, processing_frontier_sender).await;
@@ -458,7 +460,7 @@ where
Spawner:
FnOnce(&mut SourceTasks, tokio::sync::watch::Receiver<bool>, tokio::sync::mpsc::Sender<crate::RawTransactionIngress>) + std::marker::Send + 'static,
{
return start_foundation_with_port_source_spawner_and_frontier(settings, runtime, port, std::option::Option::None, source_spawner);
return start_foundation_with_port_source_spawner_and_frontier(settings, runtime, port, std::option::Option::None, 0, source_spawner);
}
fn start_foundation_with_port_source_spawner_and_frontier<Spawner>(
@@ -466,6 +468,7 @@ fn start_foundation_with_port_source_spawner_and_frontier<Spawner>(
runtime: tokio::runtime::Handle,
port: std::option::Option<PersistencePort>,
processing_frontier_receiver: std::option::Option<ProcessingFrontierReceiver>,
source_total: usize,
source_spawner: Spawner,
) -> ksp_core_lib::Result<crate::RawTransactionIngestHandle>
where
@@ -482,7 +485,7 @@ where
}
let stop_token = ksp_worker_api::WorkerStopToken::new();
let (stop_sender, stop_receiver) = tokio::sync::watch::channel(false);
let (snapshots, snapshot_source) = crate::RawTransactionIngestSnapshotPublisher::new(&settings, &lifecycle);
let (snapshots, snapshot_source) = crate::RawTransactionIngestSnapshotPublisher::new(&settings, &lifecycle, source_total);
let handle = crate::RawTransactionIngestHandle { snapshots: snapshot_source, stop_sender, stop_token };
std::mem::drop(runtime.spawn(run_supervisor(settings, lifecycle, port, stop_receiver, snapshots, processing_frontier_receiver, source_spawner)));
return std::result::Result::Ok(handle);

12
crates/ksp-worker-raw-transaction-ingest-lib/src/runtime_resources.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/runtime_resources.rs
// version: 24
// version: 25
use sha2::Digest; // rust-rules: trait-import
@@ -179,6 +179,10 @@ impl RawTransactionIngestSourceInventory {
let mut source_continuity_gap_total = 0_u64;
let mut source_reconnect_total = 0_u64;
let mut source_replay_attempt_total = 0_u64;
let source_total = self.source_projections.len();
let mut source_active = 0_usize;
let mut source_reconnecting = 0_usize;
let mut source_failed = 0_usize;
let mut any_active = false;
let mut any_closing = false;
let mut any_failed = false;
@@ -200,6 +204,7 @@ impl RawTransactionIngestSourceInventory {
source_replay_attempt_total = source_replay_attempt_total.saturating_add(projection.source_replay_attempt_total());
match projection.source_state() {
std::option::Option::Some(crate::RawTransactionIngestSourceState::Active) => {
source_active += 1;
any_active = true;
all_closed = false;
},
@@ -209,10 +214,12 @@ impl RawTransactionIngestSourceInventory {
},
std::option::Option::Some(crate::RawTransactionIngestSourceState::Closed) => {},
std::option::Option::Some(crate::RawTransactionIngestSourceState::Failed) => {
source_failed += 1;
any_failed = true;
all_closed = false;
},
std::option::Option::Some(crate::RawTransactionIngestSourceState::Reconnecting) => {
source_reconnecting += 1;
any_reconnecting = true;
all_closed = false;
},
@@ -238,7 +245,8 @@ impl RawTransactionIngestSourceInventory {
std::option::Option::None
};
return crate::RawTransactionIngestProcessingFrontierProjection::new(hydration_pending, processing_frontier_slot, oldest_pending_slot)
.with_source_continuity(source_state, source_reconnect_total, source_replay_attempt_total, source_continuity_gap_total);
.with_source_continuity(source_state, source_reconnect_total, source_replay_attempt_total, source_continuity_gap_total)
.with_source_counts(source_total, source_active, source_reconnecting, source_failed);
}
fn new(source_keys: std::vec::Vec<[u8; 32]>) -> Self {

116
crates/ksp-worker-raw-transaction-ingest-lib/src/snapshot.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/src/snapshot.rs
// version: 4
// version: 5
/// Runtime-neutral boxed future resolving to one newer concrete RAW transaction ingest Worker snapshot.
pub type RawTransactionIngestSnapshotFuture<'a> =
@@ -27,6 +27,10 @@ pub(crate) struct RawTransactionIngestProcessingFrontierProjection {
processing_frontier_slot: std::option::Option<u64>,
oldest_pending_slot: std::option::Option<u64>,
source_state: std::option::Option<crate::RawTransactionIngestSourceState>,
source_total: usize,
source_active: usize,
source_reconnecting: usize,
source_failed: usize,
source_reconnect_total: u64,
source_replay_attempt_total: u64,
source_continuity_gap_total: u64,
@@ -40,6 +44,10 @@ impl crate::RawTransactionIngestProcessingFrontierProjection {
processing_frontier_slot: std::option::Option::None,
oldest_pending_slot: std::option::Option::None,
source_state: std::option::Option::None,
source_total: 0,
source_active: 0,
source_reconnecting: 0,
source_failed: 0,
source_reconnect_total: 0,
source_replay_attempt_total: 0,
source_continuity_gap_total: 0,
@@ -57,6 +65,10 @@ impl crate::RawTransactionIngestProcessingFrontierProjection {
processing_frontier_slot,
oldest_pending_slot,
source_state: std::option::Option::None,
source_total: 0,
source_active: 0,
source_reconnecting: 0,
source_failed: 0,
source_reconnect_total: 0,
source_replay_attempt_total: 0,
source_continuity_gap_total: 0,
@@ -93,6 +105,35 @@ impl crate::RawTransactionIngestProcessingFrontierProjection {
return self;
}
/// Returns a copy carrying source-neutral multi-source lifecycle counts.
pub(crate) const fn with_source_counts(mut self, source_total: usize, source_active: usize, source_reconnecting: usize, source_failed: usize) -> Self {
self.source_total = source_total;
self.source_active = source_active;
self.source_reconnecting = source_reconnecting;
self.source_failed = source_failed;
return self;
}
/// Returns the configured logical source count carried by this private projection.
pub(crate) const fn source_total(&self) -> usize {
return self.source_total;
}
/// Returns the number of sources currently projected Active.
pub(crate) const fn source_active(&self) -> usize {
return self.source_active;
}
/// Returns the number of sources currently projected Reconnecting.
pub(crate) const fn source_reconnecting(&self) -> usize {
return self.source_reconnecting;
}
/// Returns the number of sources currently projected Failed.
pub(crate) const fn source_failed(&self) -> usize {
return self.source_failed;
}
/// Returns the latest source-neutral lifecycle state carried by this private projection.
pub(crate) const fn source_state(&self) -> std::option::Option<crate::RawTransactionIngestSourceState> {
return self.source_state;
@@ -138,6 +179,10 @@ pub struct RawTransactionIngestSnapshot {
processing_frontier_slot: std::option::Option<u64>,
oldest_pending_slot: std::option::Option<u64>,
source_state: std::option::Option<crate::RawTransactionIngestSourceState>,
source_total: usize,
source_active: usize,
source_reconnecting: usize,
source_failed: usize,
source_reconnect_total: u64,
source_replay_attempt_total: u64,
source_continuity_gap_total: u64,
@@ -264,6 +309,30 @@ impl crate::RawTransactionIngestSnapshot {
return self.oldest_pending_slot;
}
/// Returns the configured number of logical live sources for this Worker run.
#[must_use]
pub const fn source_total(&self) -> usize {
return self.source_total;
}
/// Returns the latest number of sources projected Active.
#[must_use]
pub const fn source_active(&self) -> usize {
return self.source_active;
}
/// Returns the latest number of sources projected Reconnecting.
#[must_use]
pub const fn source_reconnecting(&self) -> usize {
return self.source_reconnecting;
}
/// Returns the latest number of sources projected Failed.
#[must_use]
pub const fn source_failed(&self) -> usize {
return self.source_failed;
}
/// Returns the latest source-neutral lifecycle state when the productive source has started.
#[must_use]
pub const fn source_state(&self) -> std::option::Option<crate::RawTransactionIngestSourceState> {
@@ -314,6 +383,10 @@ impl std::fmt::Debug for crate::RawTransactionIngestSnapshot {
.field("processing_frontier_slot", &self.processing_frontier_slot)
.field("oldest_pending_slot", &self.oldest_pending_slot)
.field("source_state", &self.source_state)
.field("source_total", &self.source_total)
.field("source_active", &self.source_active)
.field("source_reconnecting", &self.source_reconnecting)
.field("source_failed", &self.source_failed)
.field("source_reconnect_total", &self.source_reconnect_total)
.field("source_replay_attempt_total", &self.source_replay_attempt_total)
.field("source_continuity_gap_total", &self.source_continuity_gap_total)
@@ -403,6 +476,7 @@ impl crate::RawTransactionIngestSnapshotPublisher {
pub(crate) fn new(
settings: &crate::RawTransactionIngestSettings,
lifecycle: &ksp_worker_api::WorkerLifecycle,
source_total: usize,
) -> (crate::RawTransactionIngestSnapshotPublisher, crate::RawTransactionIngestSnapshotSource) {
let worker = ksp_worker_api::WorkerSnapshot::new(
lifecycle.id().clone(),
@@ -434,6 +508,10 @@ impl crate::RawTransactionIngestSnapshotPublisher {
processing_frontier_slot: std::option::Option::None,
oldest_pending_slot: std::option::Option::None,
source_state: std::option::Option::None,
source_total,
source_active: 0,
source_reconnecting: 0,
source_failed: 0,
source_reconnect_total: 0,
source_replay_attempt_total: 0,
source_continuity_gap_total: 0,
@@ -449,7 +527,14 @@ impl crate::RawTransactionIngestSnapshotPublisher {
self.snapshot.worker.kind().clone(),
self.snapshot.worker.sequence(),
state,
health_for_state(state, self.snapshot.worker.health()),
health_for_state(
state,
self.snapshot.worker.health(),
self.snapshot.source_total,
self.snapshot.source_active,
self.snapshot.source_reconnecting,
self.snapshot.source_failed,
),
ksp_worker_api::WorkerActivity::Idle,
);
self.snapshot.worker = worker;
@@ -530,6 +615,12 @@ impl crate::RawTransactionIngestSnapshotPublisher {
self.snapshot.processing_frontier_slot = projection.processing_frontier_slot();
self.snapshot.oldest_pending_slot = projection.oldest_pending_slot();
self.snapshot.source_state = projection.source_state();
if projection.source_total() > 0 {
self.snapshot.source_total = projection.source_total();
self.snapshot.source_active = projection.source_active();
self.snapshot.source_reconnecting = projection.source_reconnecting();
self.snapshot.source_failed = projection.source_failed();
}
self.snapshot.source_reconnect_total = projection.source_reconnect_total();
self.snapshot.source_replay_attempt_total = projection.source_replay_attempt_total();
self.snapshot.source_continuity_gap_total = projection.source_continuity_gap_total();
@@ -646,7 +737,14 @@ impl crate::RawTransactionIngestSnapshotPublisher {
self.snapshot.worker.kind().clone(),
sequence,
state,
health_for_state(state, self.snapshot.worker.health()),
health_for_state(
state,
self.snapshot.worker.health(),
self.snapshot.source_total,
self.snapshot.source_active,
self.snapshot.source_reconnecting,
self.snapshot.source_failed,
),
activity_for_state(state, admission_queue_depth, in_flight_persistence, self.snapshot.hydration_pending, self.snapshot.source_state),
);
self.snapshot.worker = worker;
@@ -698,8 +796,18 @@ fn checked_optional_counter(current: u64, increment: bool, field: &'static str)
return checked_counter(current, field);
}
fn health_for_state(state: ksp_worker_api::WorkerState, previous: ksp_worker_api::WorkerHealth) -> ksp_worker_api::WorkerHealth {
fn health_for_state(
state: ksp_worker_api::WorkerState,
previous: ksp_worker_api::WorkerHealth,
source_total: usize,
source_active: usize,
source_reconnecting: usize,
source_failed: usize,
) -> ksp_worker_api::WorkerHealth {
return match state {
ksp_worker_api::WorkerState::Running if source_failed > 0 => ksp_worker_api::WorkerHealth::Unhealthy,
ksp_worker_api::WorkerState::Running if source_reconnecting > 0 => ksp_worker_api::WorkerHealth::Degraded,
ksp_worker_api::WorkerState::Running if source_total > 0 && source_active < source_total => ksp_worker_api::WorkerHealth::Degraded,
ksp_worker_api::WorkerState::Running => ksp_worker_api::WorkerHealth::Healthy,
ksp_worker_api::WorkerState::Stopping | ksp_worker_api::WorkerState::Stopped => previous,
ksp_worker_api::WorkerState::Faulted(_) => ksp_worker_api::WorkerHealth::Unhealthy,

15
crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/dependency_boundary.rs
// version: 26
// version: 27
//! Dependency firewall canaries for the RAW transaction ingest Worker foundation.
@@ -650,3 +650,16 @@ fn v0_3_13_pre_009_global_bounds_and_fairness_stay_inside_worker_facades() {
}
return;
}
#[test]
fn v0_3_13_pre_010_multi_source_snapshot_health_remains_source_neutral_and_backend_free() {
let snapshot = include_str!("../src/snapshot.rs");
let resources = include_str!("../src/runtime_resources.rs");
for required in ["source_total", "source_active", "source_reconnecting", "source_failed", "WorkerHealth::Degraded", "with_source_counts"] {
assert!(snapshot.contains(required) || resources.contains(required), "required pre.010 source-neutral snapshot contract missing: {required}");
}
for forbidden in ["ksp_store_postgres_lib::", "reqwest::", "tonic::", "yellowstone_grpc_proto::", "SourceSnapshotByKey", "provider_health"] {
assert!(!snapshot.contains(forbidden), "pre.010 snapshot crossed source-neutral/backend boundary: {forbidden}");
}
return;
}

28
crates/ksp-worker-raw-transaction-ingest-lib/tests/hardening.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/hardening.rs
// version: 21
// version: 22
//! External public, security, redaction and release-boundary hardening canaries for `pre.010`.
@@ -828,3 +828,29 @@ fn v0_3_13_pre_009_duplicate_storm_disagreement_and_starvation_guards_are_explic
}
return;
}
#[test]
fn v0_3_13_pre_010_multi_source_health_is_conservative_counted_and_redacted() {
let snapshot = include_str!("../src/snapshot.rs");
let resources = include_str!("../src/runtime_resources.rs");
for required in [
"source_total: usize",
"source_active: usize",
"source_reconnecting: usize",
"source_failed: usize",
"source_reconnecting > 0",
"source_failed > 0",
"source_active < source_total",
"WorkerHealth::Degraded",
"WorkerHealth::Unhealthy",
] {
assert!(snapshot.contains(required), "required pre.010 health guard missing: {required}");
}
for required in ["source_total = self.source_projections.len()", "with_source_counts(source_total, source_active, source_reconnecting, source_failed)"] {
assert!(resources.contains(required), "required pre.010 inventory count guard missing: {required}");
}
for forbidden in ["source_key: [u8; 32]", "provider_url", "endpoint_url", "filter_fingerprint", "credential", "api_key", "token_header"] {
assert!(!snapshot.contains(forbidden), "pre.010 snapshot leaks source/provider material: {forbidden}");
}
return;
}

19
crates/ksp-worker-raw-transaction-ingest-lib/tests/public_api.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/public_api.rs
// version: 19
// version: 20
//! External public-surface proofs for the RAW transaction ingest Worker foundation.
@@ -368,3 +368,20 @@ fn v0_3_13_pre_009_fairness_budgets_semaphore_and_canonical_state_remain_private
}
return;
}
#[test]
fn v0_3_13_pre_010_multi_source_snapshot_counts_are_public_and_source_neutral() {
let _source_total: fn(&ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot) -> usize =
ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot::source_total;
let _source_active: fn(&ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot) -> usize =
ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot::source_active;
let _source_reconnecting: fn(&ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot) -> usize =
ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot::source_reconnecting;
let _source_failed: fn(&ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot) -> usize =
ksp_worker_raw_transaction_ingest_lib::RawTransactionIngestSnapshot::source_failed;
let root = include_str!("../src/lib.rs");
for forbidden in ["SourceHealthByKey", "SourceSnapshotByKey", "provider_health", "endpoint_health"] {
assert!(!root.contains(forbidden), "pre.010 public root leaked source-specific health material: {forbidden}");
}
return;
}

5
crates/ksp-worker-raw-transaction-ingest-lib/tests/release_completeness.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/tests/release_completeness.rs
// version: 17
// version: 18
//! Release-completeness canaries through the `pre.010` public/release/security hardening tranche.
@@ -130,6 +130,7 @@ fn pre_010_external_hardening_suite_is_present_and_scoped() {
"v0_3_13_pre_007_multi_source_supervisor_is_fail_closed_joined_and_does_not_publish_source_identity",
"v0_3_13_pre_008_cross_source_convergence_is_bounded_conflict_checked_and_private",
"v0_3_13_pre_009_duplicate_storm_disagreement_and_starvation_guards_are_explicit",
"v0_3_13_pre_010_multi_source_health_is_conservative_counted_and_redacted",
] {
assert!(hardening.contains(required), "required pre.010 hardening canary missing: {required}");
}
@@ -150,6 +151,7 @@ fn pre_010_external_hardening_suite_is_present_and_scoped() {
assert!(dependency_boundary.contains("v0_3_13_pre_007_multi_source_supervisor_and_inventory_remain_private_bounded_and_source_neutral"));
assert!(dependency_boundary.contains("v0_3_13_pre_008_cross_source_convergence_reuses_store_and_transport_facades_only"));
assert!(dependency_boundary.contains("v0_3_13_pre_009_global_bounds_and_fairness_stay_inside_worker_facades"));
assert!(dependency_boundary.contains("v0_3_13_pre_010_multi_source_snapshot_health_remains_source_neutral_and_backend_free"));
let public_api = include_str!("public_api.rs");
assert!(public_api.contains("pre_003_kind_code_and_settings_are_consumable_from_crate_root"));
assert!(public_api.contains("pre_004_start_handle_and_terminal_future_are_consumable_without_public_join_handle"));
@@ -163,6 +165,7 @@ fn pre_010_external_hardening_suite_is_present_and_scoped() {
assert!(public_api.contains("v0_3_13_pre_007_source_inventory_and_logical_keys_remain_private"));
assert!(public_api.contains("v0_3_13_pre_008_convergence_cache_registry_and_source_keys_remain_private"));
assert!(public_api.contains("v0_3_13_pre_009_fairness_budgets_semaphore_and_canonical_state_remain_private"));
assert!(public_api.contains("v0_3_13_pre_010_multi_source_snapshot_counts_are_public_and_source_neutral"));
assert!(public_api.contains("v0_3_12_pre_007_processing_frontier_snapshot_getters_are_public_and_processing_only"));
assert!(public_api.contains("pre_008_snapshot_surface_and_common_projection_are_public_and_stable"));
return;

10
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime_resources.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/runtime_resources.rs
// version: 21
// version: 22
fn grpc_endpoint(cluster: &str) -> std::option::Option<ksp_onchain_transport_lib::YellowstoneGrpcEndpointSettings> {
return grpc_endpoint_with_identity(cluster, "yellowstone-fixture", "fixture-provider");
@@ -984,6 +984,10 @@ fn v0_3_13_pre_007_source_inventory_aggregates_frontier_and_lifecycle_conservati
assert_eq!(first_aggregate.processing_frontier_slot(), std::option::Option::None);
assert_eq!(first_aggregate.oldest_pending_slot(), std::option::Option::Some(45));
assert_eq!(first_aggregate.source_state(), std::option::Option::Some(crate::RawTransactionIngestSourceState::Active));
assert_eq!(first_aggregate.source_total(), 2);
assert_eq!(first_aggregate.source_active(), 1);
assert_eq!(first_aggregate.source_reconnecting(), 0);
assert_eq!(first_aggregate.source_failed(), 0);
second.publish(
crate::RawTransactionIngestProcessingFrontierProjection::new(2, std::option::Option::Some(42), std::option::Option::Some(40)).with_source_continuity(
std::option::Option::Some(crate::RawTransactionIngestSourceState::Reconnecting),
@@ -997,6 +1001,10 @@ fn v0_3_13_pre_007_source_inventory_aggregates_frontier_and_lifecycle_conservati
assert_eq!(aggregate.processing_frontier_slot(), std::option::Option::Some(42));
assert_eq!(aggregate.oldest_pending_slot(), std::option::Option::Some(40));
assert_eq!(aggregate.source_state(), std::option::Option::Some(crate::RawTransactionIngestSourceState::Reconnecting));
assert_eq!(aggregate.source_total(), 2);
assert_eq!(aggregate.source_active(), 1);
assert_eq!(aggregate.source_reconnecting(), 1);
assert_eq!(aggregate.source_failed(), 0);
assert_eq!(aggregate.source_reconnect_total(), 4);
assert_eq!(aggregate.source_replay_attempt_total(), 6);
assert_eq!(aggregate.source_continuity_gap_total(), 1);

56
crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/snapshot.rs
View File

@@ -1,7 +1,9 @@
// file: crates/ksp-worker-raw-transaction-ingest-lib/unit_tests/snapshot.rs
// version: 4
// version: 5
fn snapshot_foundation() -> std::option::Option<(crate::RawTransactionIngestSnapshotPublisher, crate::RawTransactionIngestSnapshotSource)> {
fn snapshot_foundation_with_source_total(
source_total: usize,
) -> std::option::Option<(crate::RawTransactionIngestSnapshotPublisher, crate::RawTransactionIngestSnapshotSource)> {
let network = match ksp_store_lib::RawNetworkId::new("mainnet") {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_) => return std::option::Option::None,
@@ -19,7 +21,11 @@ fn snapshot_foundation() -> std::option::Option<(crate::RawTransactionIngestSnap
if lifecycle.start().is_err() {
return std::option::Option::None;
}
return std::option::Option::Some(crate::RawTransactionIngestSnapshotPublisher::new(&settings, &lifecycle));
return std::option::Option::Some(crate::RawTransactionIngestSnapshotPublisher::new(&settings, &lifecycle, source_total));
}
fn snapshot_foundation() -> std::option::Option<(crate::RawTransactionIngestSnapshotPublisher, crate::RawTransactionIngestSnapshotSource)> {
return snapshot_foundation_with_source_total(0);
}
#[test]
@@ -54,6 +60,10 @@ fn pre_008_initial_snapshot_and_common_projection_are_exact() {
assert_eq!(concrete.processing_frontier_slot(), std::option::Option::None);
assert_eq!(concrete.oldest_pending_slot(), std::option::Option::None);
assert_eq!(concrete.source_state(), std::option::Option::None);
assert_eq!(concrete.source_total(), 0);
assert_eq!(concrete.source_active(), 0);
assert_eq!(concrete.source_reconnecting(), 0);
assert_eq!(concrete.source_failed(), 0);
assert_eq!(concrete.source_reconnect_total(), 0);
assert_eq!(concrete.source_replay_attempt_total(), 0);
assert_eq!(concrete.source_continuity_gap_total(), 0);
@@ -199,3 +209,43 @@ fn pre_008_source_continuity_projection_preserves_processing_frontier_and_distin
assert_eq!(resumed.worker_snapshot().activity(), ksp_worker_api::WorkerActivity::Idle);
return;
}
#[test]
fn v0_3_13_pre_010_multi_source_counts_and_health_are_conservative_and_source_neutral() {
let (mut publisher, source) = match snapshot_foundation_with_source_total(3) {
std::option::Option::Some(value) => value,
std::option::Option::None => return,
};
let initial = source.current();
assert_eq!(initial.source_total(), 3);
assert_eq!(initial.source_active(), 0);
assert_eq!(initial.source_reconnecting(), 0);
assert_eq!(initial.source_failed(), 0);
assert_eq!(initial.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Unknown);
let reconnecting = crate::RawTransactionIngestProcessingFrontierProjection::new(1, std::option::Option::Some(50), std::option::Option::Some(50))
.with_source_continuity(std::option::Option::Some(crate::RawTransactionIngestSourceState::Reconnecting), 0, 1, 0)
.with_source_counts(3, 2, 1, 0);
assert!(publisher.record_processing_frontier(ksp_worker_api::WorkerState::Running, 0, 0, reconnecting).is_ok());
let degraded = source.current();
assert_eq!(degraded.source_total(), 3);
assert_eq!(degraded.source_active(), 2);
assert_eq!(degraded.source_reconnecting(), 1);
assert_eq!(degraded.source_failed(), 0);
assert_eq!(degraded.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Degraded);
assert_eq!(degraded.worker_snapshot().activity(), ksp_worker_api::WorkerActivity::Active);
let active = crate::RawTransactionIngestProcessingFrontierProjection::new(0, std::option::Option::Some(51), std::option::Option::None)
.with_source_continuity(std::option::Option::Some(crate::RawTransactionIngestSourceState::Active), 1, 1, 0)
.with_source_counts(3, 3, 0, 0);
assert!(publisher.record_processing_frontier(ksp_worker_api::WorkerState::Running, 0, 0, active).is_ok());
let healthy = source.current();
assert_eq!(healthy.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Healthy);
assert_eq!(healthy.worker_snapshot().activity(), ksp_worker_api::WorkerActivity::Idle);
let failed = crate::RawTransactionIngestProcessingFrontierProjection::new(0, std::option::Option::Some(51), std::option::Option::None)
.with_source_continuity(std::option::Option::Some(crate::RawTransactionIngestSourceState::Failed), 1, 1, 0)
.with_source_counts(3, 2, 0, 1);
assert!(publisher.record_processing_frontier(ksp_worker_api::WorkerState::Running, 0, 0, failed).is_ok());
let unhealthy = source.current();
assert_eq!(unhealthy.source_failed(), 1);
assert_eq!(unhealthy.worker_snapshot().health(), ksp_worker_api::WorkerHealth::Unhealthy);
return;
}