AllCoreDevs - Testing #089
Transcript
- Justin Traglia
Gm
- Mario Vega
gm
- Mario Vega
- Parithosh Jayanthi
Tysm scenarios are going on, right now epbs mutator flips bits in the bids and propagates them on the network
- Stefan Starflinger
- Parithosh Jayanthi
Next one will perform targeted PTC vote drops
- Stefan Starflinger
https://discord.com/channels/595666850260713488/892088344438255616/1529440070908510308
- Barnabas
we might have some bid propagation issues
- Barnabas
we are investigating
- Nico Flaig
that seems hard to know without more info, we will take that bid unless the circuit breaker is active
- Barnabas
you take the bid even if local is higher?
- Barnabas
do you do any basic comparison in the value?
- Nico Flaig
no, but Stefan said the bid is really high value
- Parithosh Jayanthi
With circuit breaker then you’d local build anyway right? Even if the bid is high
- potuz
Is the gas computation correct?
- terence
Is the bid building on the right head?
- terence
If the head changes last sec then
- potuz
That’s the main suspect to not taking it
- terence
Anyway prysm logs should be clear why we ignore such bid
- Nico Flaig
Replying to "With circuit breaker..." yes with circuit breaker active we only use local payloads
- Barnabas
Reacted to "Anyway prysm logs should be clear why we ignore such bid" with 👍
- Nico Flaig
lodestar should alos have a debug log for this
- Parithosh Jayanthi
Yeah please let us know what the log line looks like, just something with “circuit breaker live” in it?
- Nico Flaig
Replying to "Yeah please let us k..." there is a log for that too
- Parithosh Jayanthi
Reacted to "there is a log for that too" with ❤️
- Iván | ethrex
We will add optimizations to devnet-7 today
- Daniel Lehrner (Besu)
I need to check with the other people on the team working on performance
- Nico Flaig
Replying to "Yeah please let us k..." it's a warning log as following: "Builder circuit breaker activated, ignoring builder bids"
- Dmitrii Shmatko
(Teku) We are merging a lot of bid reprocessing improvements today/tomorrow, it will also add debug logging with reasons
- Parithosh Jayanthi
Reacted to "it's a warning log as following: "Builder circuit breaker activated, ignoring builder bids"" with ❤️
- Dmitrii Shmatko
Replying to "Yeah please let us k..." (Teku) LOG.debug( "Builder circuit breaker engaged: slot:
- Stefan Starflinger
https://notes.ethereum.org/@ethpandaops/glamsterdam-devnet-8
- Mario Vega
EELS: https://github.com/ethereum/execution-specs/issues/3167
- Parithosh Jayanthi
Could we get an update on the builder API?
- Parithosh Jayanthi
That was a big change afaik
- Justin Traglia
Reacted to "That was a big chang..." with 👍
- potuz
Reacted to "That was a big change afaik" with 👍
- Ahmad
Reacted to "That was a big cha..." with 👍
- Justin Traglia
Reacted to "Could we get an upda..." with ➕
- Mario Vega
Reacted to "That was a big change afaik" with 👍
- Daniel Lehrner (Besu)
Regarding the Besu performance improvements: The changes we tested today and needed to roll back should be the last ones. We just need to fix the bugs those improvements have introduced and then merge them again into our devnet branch
- Stefan Starflinger
Reacted to "That was a big chang..." with 👍
- Guru
Reacted to "EELS: https://github..." with 👍
- Nico Flaig
0 consensus changes and also optional
- Maria Silva
Reacted to "Regarding the Besu p..." with 👍
- Justin Traglia
Reacted to "0 consensus changes ..." with 👍
- Justin Traglia
Replying to "0 consensus changes ..." If it were critical (to the core protocol) it would be included in consensus-specs 🙂
- Nico Flaig
Reacted to "If it were critical ..." with 👍
- Justin Traglia
- spencer
Do we need repricing numbers for devnet 8?
- spencer
Updates^^
- Justin Traglia
The CL specs team will make an alpha.13 release for devnet-8 by the end of this week.
- Justin Traglia
Replying to "The CL specs team wi..." Hopefully mid-week.
- Stefan Starflinger
Do we need benchmarks for final numbers on devnet-8? Is devnet-7 enough?
- jochem-brouwer
We also need the optimizations
- Stefan Starflinger
lets revisit devnet-8 launch date on ACDT next week?
- Mario Vega
https://github.com/ethereum/pm/issues/2170#issuecomment-5091899084
- Justin Traglia
That’s Bosul from the networking team 🙂
- Mario Vega
Reacted to "That’s Bosul from the networking team 🙂" with 🙏🏼
- Will Corcoran
Reacted to "That’s Bosul from the networking team 🙂" with ❤️
- Parithosh Jayanthi
I think if we intend to ship this for Glamsterdam, we should have it in a devnet
- spencer
Replying to "I think if we inte..." Big agree!
- Mario Vega
Reacted to "I think if we intend to ship this for Glamsterdam, we should have it in a devnet" with 👍🏼
- Mario Vega
Reacted to "Big agree!" with 👍🏼
- Guru
Reacted to "I think if we intend..." with 👍🏼
- Stefan Starflinger
Reacted to "I think if we intend..." with 👍🏼
- Barnabas
imo this is a non priority: https://dune.com/queries/3757544/6319515 Would propose delaying it to h*. Most CL clients don’t have the partial cells implementation complete yet, so this is a major lift for them.
- Parithosh Jayanthi
What timelines are we looking at with this?
- Barnabas
whats the point of the engine api if the feature is not there?
- Daniel Knopik
Replying to "whats the point of..." do you mean partial data columns or sparse blobpool?
- Mario Vega
Replying to "What timelines are w..." Launch alongside devnet-8 is what I understand
- Barnabas
Replying to "whats the point of t..." isn’t partial data columns a prerequisite for sparse blobpool?
- Daniel Knopik
Replying to "whats the point of..." no, both have benefits by themselves afaik
- Fabio Di Fabio
for Besu should be fine to implement only Engine API part
- Marius Van Der Wijden (M)
Reacted to no, both have benefi... with "👍"
- Bosul Mun
Reacted to "no, both have benefits by themselves afaik" with 👍
- Parithosh Jayanthi
Maybe worth going client to client?
- Mario Vega
Reacted to "Maybe worth going client to client?" with 👍🏼
- Barnabas
Replying to "whats the point of t..." okay so one could have sparse blobpool implementation but have 0 partial data column impl?
- Iván | ethrex
No problem on our side
- Bosul Mun
Replying to "whats the point of t..." ^ yes it is possible
- Daniel Knopik
IMO, for getBlobsV4 it is fine to not support it as it is gated by capability. I am just worried about only some clients understanding the custody_columns parameter in fcu
- spencer
We have engine api tests in EELS for this - thanks to Louis (that we can run in hive), Geth is passing all of them (with small change), have vibed it in Nethermind too (all passing)!
- Ben Adams
Nethermind is good; have PR for it
- spencer
Reacted to "Nethermind is good..." with 🔥
- Stefan Starflinger
Reacted to "Nethermind is good; ..." with 🔥
- draganrakita
Yeah, reth already have it merged on main
- spencer
Reacted to "Yeah, reth already..." with 🔥
- Justin Traglia
I’m not sure. I’d need to look into it.
- Daniel Knopik
LH has PRs ready to go
- Mario Vega
Reacted to "LH has PRs ready to go" with 🙏🏼
- Barnabas
where is the execution api pr for this btw?
- spencer
Are we using fcuV3 atm on the devnets? Or fcuV4 without the custody field
- Barnabas
Replying to "where is the executi..." https://github.com/ethereum/execution-apis/pull/774
- Stefan Starflinger
lighthouse still uses fcuV3 and only fcuV4 when they are proposers
- potuz
Ohh man I should’ve followed this discussion, it doesn’t make any sense to send this on FCU
- Mario Vega
Reacted to "https://github.com/ethereum/execution-apis/pull/774" with 🙏🏼
- Ben Adams
Nethermind already running sparse blob pool in devnet-7
- Marius Van Der Wijden (M)
Yep, its optional
- Daniel Knopik
personally, i also would prefer a separate call, but do not feel tooo strongly about it
- Daniel Knopik
(especally because we are quite ahead)
- Daniel Knopik
call as in RPC method
- spencer
Reacted to "call as in RPC met..." with 📞
- Parithosh Jayanthi
Reacted to "call as in RPC method" with 📞
- Justin Traglia
https://ethereumfoundation.zoom.us/j/89441658268?pwd=yjqf4TheEb1R7ZSuesCdOAy5tdyVk7.1
- Will Corcoran
Reacted to "https://ethereumfoundation.zoom.us/j/89441658268?pwd=yjqf4TheEb1R7ZSuesCdOAy5tdyVk7.1" with 🫡
- wolovim
Reacted to "https://ethereumfoundation.zoom.us/j/89441658268?pwd=yjqf4TheEb1R7ZSuesCdOAy5tdyVk7.1" with 🫡
- Nico Flaig
Reacted to "https://ethereumfoun..." with 🫡
- Mario Vega
- Mario Vega
@spencer do you know if there’s a specs PR for this?
- spencer
Replying to "@spencer do you kn..." Guru's PR: https://github.com/ethereum/execution-specs/pull/3214
- Mario Vega
- spencer
Replying to "https://github.com..." EELS PR: https://github.com/ethereum/execution-specs/pull/3238
- Luis Pinto | Besu
I also don’t want to hang on in naming issues 😅 so fine for me, just a nit
- draganrakita
Reacted to "I also don’t want to hang on in naming issues 😅 so fine for me, just a nit" with 😂
- draganrakita
It is hard to name it as it covers both execution and block bytes/size. Execution gas sgtm, it is nicer.
- jochem-brouwer
Antwoord verzenden naar "I also don’t want..."
- a software point it is just a name, for the community it is easier
)
- jochem-brouwer
Heeft gereageerd op "It is hard to name..." met ❤️
- spencer
Reacted to "from a software po..." with 💯
- Mario Vega
Reacted to "It is hard to name it as it covers both execution and block bytes/size. Execution gas sgtm, it is nicer." with ❤️
- Luis Pinto | Besu
Reacted to "It is hard to name it as it covers both execution and block bytes/size. Execution gas sgtm, it is nicer." with 👍
- Ahmad
Reacted to "It is hard to name..." with ❤️
- Mario Vega
- Mario Vega
- Kevaundray Wedderburn
Sorry didn’t read it before the call
- Kevaundray Wedderburn
I don’t think this affects zkVM
- jochem-brouwer
The EIP now says; If the chain is already running with gas parameters that no longer allow deployment via the keyless transaction, the only way to add the contract at the expected address is a coordinated irregular state transition, which is outside the scope of this EIP. Client software MUST NOT check for the existence of the contract at the fork boundary. I think this is fine and we should not change
- wolovim
pari has advocated for chaos/adversarial testing on a devnet 9. not sure the impact on testnet scheduling
Call summary
Targets
- •CL specs alpha.13 release for devnet-8 by end of this week (mid-week target) - 00:22:23
- •Glamsterdam devnet-8 launch: early August (exact date TBD at next ACDT) - 00:21:50
Decisions
- •EIP-8070 Engine API changes (GetBlobsV3/V4, FCU custody_columns) are mandatory for devnet-8; cell-level DAS deferred - 00:34:35
- •EIP-2780 log cost simplification included in devnet-8 with no objections - 00:44:05
- •EIP-8037 rename 'regular gas' to 'execution gas' included in devnet-8 with no objections - 00:49:49
- •EIP-7997: close both open PRs; existing spec text is correct — no BAL read or contract check at fork boundary - 01:00:10
- •devnet-8 launch date deferred; to be revisited at next ACDT once repricing numbers are available - 00:24:08
Highlights
- Testing Progress:
- ·Glamsterdam devnet-7 held up over weekend; ramping up load/chaos scenarios now - 00:06:50
- ·Geth and Nethermind builders not winning bids on devnet-7; under investigation - 00:07:34
- ·Teku favors local blocks over external bids ~90% of the time; logging improvements incoming - 00:11:42
- ·Glamsterdam devnet-8 spec sheet shared; key focus: discv5-only (no V4 fallback), SSZ fork transition, increased chaos - 00:17:07
- Devnet 8 Planning:
- ·devnet-8 launch tentatively early August; next Wednesday proposed but likely too tight pending repricing numbers - 00:21:50
- ·Repricing analysis started Friday; final numbers not expected before next week — devnet-8 launch date to be revisited at next ACDT - 00:22:09
- ·Sparse blob pool (EIP-8070) Engine API changes (GetBlobsV3/V4 + FCU custody_columns field) proposed mandatory for devnet-8 - 00:24:50
- ·EIP-8070 Engine API endpoints agreed for devnet-8; Besu, Geth, Nethermind, Reth, ethrex all confirmed; Lighthouse PRs ready - 00:34:35
- El Breakout Eip Changes:
- ·EIP-2780 (log cost simplification for repricing): merged PR, no objections; going into devnet-8 spec release - 00:42:40
- ·EIP-8037 (rename 'regular gas' to 'execution gas'): pure rename, no behavior change; no objections, going into devnet-8 - 00:45:25
- ·EIP-7997 fork-boundary behavior: spec already correct — client software must NOT check contract existence at fork block; two open PRs to be closed - 00:51:32
- ·Geth uniquely reads/deploys EIP-7997 contract at fork boundary; safe for mainnet (already deployed), but diverges on chains without factory pre-deployed - 01:00:17
Action Items
- •CL teams - CL clients: review bid-selection logic and add logging for accepted/rejected bids and circuit breaker activation - 00:10:37
- •All EL/CL teams - Raise any Engine API implementation concerns for EIP-8070 promptly; revisit at next ACDT - 00:40:58
- •Mario Vega / Spencer / Guru - Merge EIP-2780 and EIP-8037 into EL specs and include in devnet-8 spec release - 00:50:29
Key decisions
EIP-8070 Engine API changes (GetBlobsV3/V4, FCU custody_columns) are mandatory for devnet-8; cell-level DAS deferred
devnet-8EIP-2780 log cost simplification included in devnet-8 with no objections
devnet-8EIP-8037 rename 'regular gas' to 'execution gas' included in devnet-8 with no objections
devnet-8EIP-7997: close both open PRs; existing spec text is correct — no BAL read or contract check at fork boundary
devnet-8 launch date deferred; to be revisited at next ACDT once repricing numbers are available
Unknown