Sasedev/khadhroony-solana-project
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

0.3.15-pre.015

Browse Source
This commit is contained in:
SinuS von SifriduS
2026-09-18 21:06:58 +02:00
parent b266019ea8
commit 582a286baa
9 changed files with 323 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
crates/ksp-onchain-transport-lib/README.md
View File

@@ -1,5 +1,5 @@
<!-- file: crates/ksp-onchain-transport-lib/README.md -->
<!-- version: 24 -->
<!-- version: 25 -->
# `ksp-onchain-transport-lib`
@@ -239,7 +239,9 @@ YellowstoneGrpcSubscribeSnapshot
Les sept unary standards retenus sont `SubscribeReplayInfo`, `Ping`, `GetLatestBlockhash`, `GetBlockHeight`, `GetSlot`, `IsBlockhashValid` et `GetVersion`. `Subscribe` couvre accounts, slots, transactions, transaction status, blocks, block metadata et entries, avec `commitment`, `accounts_data_slice`, `ping` et `from_slot`. `SubscribeDeshred` reste hors scope de la foundation standard.
Le stream bidirectionnel est borné : request/update queues, tailles inbound/outbound, half-close, close timeout et reconnect budget sont explicites. Après reconnect, KSP rejoue la dernière requête complète acceptée et avance prudemment `from_slot` selon le dernier slot observé et `SubscribeReplayInfo.first_available`. Les compteurs de gap et duplicate sont de l’observabilité ; ils ne constituent jamais une garantie exactly-once ou lossless.
Le stream bidirectionnel est borné : request/update queues, tailles inbound/outbound, half-close, close timeout et reconnect budget sont explicites. La queue d'updates Yellowstone reste bornée mais sa saturation locale n'est plus un fault : l'acteur attend asynchronement qu'une place soit rendue et cesse donc de poller le stream Tonic, ce qui propage la backpressure via gRPC/HTTP/2 sans drop silencieux ni queue non bornée. Le shutdown préempte cette attente. La queue de mutations de requête reste au contraire synchrone/fail-fast : un caller qui la sature reçoit toujours `grpc_backpressure_overflow`.
Après reconnect, KSP rejoue la dernière requête complète acceptée et avance prudemment `from_slot` selon le dernier slot observé et `SubscribeReplayInfo.first_available`. Les compteurs de gap et duplicate sont de l’observabilité ; ils ne constituent jamais une garantie exactly-once ou lossless.
Config Transport V3 peut mapper des `grpc_endpoints` vers ces settings sans inverser la dépendance. `protocol = solana_yellowstone` décrit le wire standard, tandis que `provider` reste un descripteur distinct. Les profils committés `publicnode_mainnet` et `publicnode_testnet` utilisent le standard sans façade PublicNode spécifique. Leur personal token est injecté comme metadata secrète `x-token` depuis Config ; Transport ne lit jamais les variables KSP.

46
crates/ksp-onchain-transport-lib/src/grpc_stream.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-onchain-transport-lib/src/grpc_stream.rs
// version: 5
// version: 6
use tonic_prost::prost::Message; // rust-rules: trait-import
@@ -296,8 +296,9 @@ impl SolanaYellowstoneGrpcSubscribeSession {
/// Receives the next decoded standard Yellowstone update.
///
/// Transient remote stream loss is hidden while KSP performs a bounded reconnect. Terminal remote statuses after the reconnect budget, malformed updates
/// and overflow surface a safe KSP error. A normal close returns `Ok(None)`.
/// Transient remote stream loss is hidden while KSP performs a bounded reconnect. A full bounded update queue applies asynchronous backpressure to the
/// gRPC stream instead of dropping an update or terminating locally; terminal remote statuses after the reconnect budget and malformed updates still
/// surface a safe KSP error. A normal close returns `Ok(None)`.
pub async fn next_update(&mut self) -> ksp_core_lib::Result<std::option::Option<crate::YellowstoneSubscribeUpdate>> {
return match self.update_rx.recv().await {
std::option::Option::Some(std::result::Result::Ok(update)) => std::result::Result::Ok(std::option::Option::Some(update)),
@@ -635,23 +636,30 @@ async fn run_subscribe_actor(
}
tracker.observe(&update, &mut snapshot);
snapshot_tx.send_replace(snapshot);
match update_tx.try_send(std::result::Result::Ok(update)) {
std::result::Result::Ok(()) => {},
std::result::Result::Err(tokio::sync::mpsc::error::TrySendError::Full(_)) => {
ksp_logging_lib::warn!(
target: crate::TRACING_TARGET,
endpoint_name = endpoint_name.as_str(),
provider = provider.as_str(),
cluster = cluster.as_str(),
"Yellowstone subscribe update queue overflowed"
);
fail_actor(crate::ERROR_CODE_GRPC_BACKPRESSURE_OVERFLOW, &request_state, &mut snapshot, &snapshot_tx);
let delivery = tokio::select! {
biased;
shutdown_changed = shutdown_rx.changed() => {
let deadline = shutdown_deadline(&shutdown_rx, shutdown_changed);
snapshot.state = crate::YellowstoneGrpcSubscribeState::Closing;
snapshot_tx.send_replace(snapshot);
clear_request_sender(&request_state);
finish_client_half_close(
&endpoint_name,
&provider,
&cluster,
&mut incoming,
deadline,
&mut snapshot,
&snapshot_tx,
)
.await;
return;
},
std::result::Result::Err(tokio::sync::mpsc::error::TrySendError::Closed(_)) => {
close_actor(&request_state, &mut snapshot, &snapshot_tx);
return;
},
}
result = update_tx.send(std::result::Result::Ok(update)) => result,
};
if delivery.is_err() {
close_actor(&request_state, &mut snapshot, &snapshot_tx);
return;
}
},
std::result::Result::Ok(std::option::Option::None) => {

24
crates/ksp-onchain-transport-lib/tests/release_completeness.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-onchain-transport-lib/tests/release_completeness.rs
// version: 47
// version: 48
//! Release-level completeness canaries for staged HTTP and WebSocket Transport coverage.
@@ -1412,3 +1412,25 @@ fn release_v0_3_15_pre_002_websocket_capabilities_are_endpoint_declared_typed_an
assert!(!settings_source.contains("provider().as_str() == \"helius\""));
assert!(!settings_source.contains("provider().as_str() == \"publicnode\""));
}
#[test]
fn release_v0_3_15_pre_015_yellowstone_update_delivery_backpressures_without_local_overflow() {
let stream_source = include_str!("../src/grpc_stream.rs");
let stream_tests = include_str!("../unit_tests/grpc_stream.rs");
let actor_start = stream_source.find("async fn run_subscribe_actor(").expect("Subscribe actor must remain present");
let actor_tail = &stream_source[actor_start..];
let actor_end = actor_tail
.find(
"
enum ReconnectOutcome",
)
.expect("Subscribe actor boundary must remain present");
let actor = &actor_tail[..actor_end];
assert!(actor.contains("result = update_tx.send(std::result::Result::Ok(update)) => result"));
assert!(actor.contains("shutdown_changed = shutdown_rx.changed()"));
assert!(actor.contains("biased;"));
assert!(!actor.contains("Yellowstone subscribe update queue overflowed"));
assert!(stream_source.contains("ERROR_CODE_GRPC_BACKPRESSURE_OVERFLOW"));
assert!(!stream_source.contains("unbounded_channel"));
assert!(stream_tests.contains("yellowstone_slow_receiver_applies_bounded_backpressure_without_terminal_overflow"));
}

31
crates/ksp-onchain-transport-lib/unit_tests/grpc_stream.rs
View File

@@ -1,5 +1,5 @@
// file: crates/ksp-onchain-transport-lib/unit_tests/grpc_stream.rs
// version: 6
// version: 7
#[derive(Clone, Copy)]
enum FixtureMode {
@@ -501,21 +501,32 @@ async fn yellowstone_session_drop_best_effort_half_closes_request_stream() {
}
#[tokio::test(flavor = "current_thread")]
async fn yellowstone_slow_receiver_overflow_is_terminal_and_observable() {
async fn yellowstone_slow_receiver_applies_bounded_backpressure_without_terminal_overflow() {
let server = FixtureServer::start(FixtureMode::Flood).await;
let defaults = crate::YellowstoneGrpcSessionSettings::default();
let settings = fixture_settings(server.endpoint_url.as_str(), 8, 2, defaults.max_inbound_message_size_bytes(), defaults.max_outbound_message_size_bytes());
let channel = crate::YellowstoneGrpcChannel::connect(&settings).await.expect("fixture channel must connect");
let mut session = channel.open_standard_subscribe(initial_request()).await.expect("fixture Subscribe stream must open");
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
assert!(session.next_update().await.expect("first queued update must remain readable").is_some());
assert!(session.next_update().await.expect("second queued update must remain readable").is_some());
let error = session.next_update().await.expect_err("overflow must become terminal after bounded queued updates drain");
assert_eq!(error.code(), crate::ERROR_CODE_GRPC_BACKPRESSURE_OVERFLOW);
let rendered = format!("{error:?} {session:?}");
assert!(!rendered.contains("GRPC-STREAM-SECRET-CANARY"));
assert!(!rendered.contains(server.endpoint_url.as_str()));
let _ = session.close().await;
assert_eq!(session.snapshot().state(), crate::YellowstoneGrpcSubscribeState::Active);
assert_eq!(session.snapshot().terminal_error_code(), std::option::Option::None);
for expected_slot in 1_000_u64..1_003_u64 {
let update = tokio::time::timeout(std::time::Duration::from_secs(1), session.next_update())
.await
.expect("bounded backpressure must resume after consumer capacity returns")
.expect("slow consumer must not receive a local overflow error")
.expect("flood fixture must continue delivering queued updates");
match update {
crate::YellowstoneSubscribeUpdate::Slot(value) => assert_eq!(value.slot(), expected_slot),
_ => panic!("flood fixture must deliver slot updates"),
}
}
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
assert_eq!(session.snapshot().terminal_error_code(), std::option::Option::None);
tokio::time::timeout(std::time::Duration::from_secs(1), session.close())
.await
.expect("shutdown must preempt a blocked bounded update delivery")
.expect("backpressured session must close cleanly");
server.stop().await;
}