AllCoreDevs — Testing #073
Transcript
Ask about this call
Answered from this call's summary and transcript only.
AI-generated — verify against the recording.
Call summary
Targets
- •BAL DevNet3 launch: targeting March 11, pending client readiness — 00:19:12
Decisions
- •SSZ engine API will be optional via content-type headers; deferred to ACD for final approval — 00:09:02
- •EL must support building on older heads for ePBS; spec will be updated — 00:32:46
- •ACDT cadence will be evaluated weekly; cede to breakouts when no cross-cutting topics — 01:02:30
Highlights
- Epbs Devnet 0:
- ·ePBS forces EL to trigger block production on older heads; spec currently forbids — 00:26:30
- ·Marius confirms horrible UX but accepts reorgs will happen; can prepare branch — 00:32:29
- ·Marius pushed fix allowing reorging to parents on epbs-devnet-0 branch — 00:39:26
- ·PTC lookahead issue: attestations valid in epoch N invalid in N+1 — 00:39:57
- ·Proposed fix: cache 256KB of PTC committees in beacon state — 00:41:12
- ·Decision on PTC fix needed this week before testing attestations — 00:43:05
- Organizational:
- ·ACDT cadence discussion: proposal to rotate topics or convert to breakouts — 00:58:14
- ·Proposal: Reserve ACDT slot for focused technical breakouts on-demand — 01:01:42
- Fusaka Blob Devnet 0:
- ·Engine API serialization bottleneck identified; SSZ transport proposal under review — 00:06:24
- ·SSZ can be implemented optionally via content-type header routing — 00:07:51
- ·Giulio implemented SSZ proof-of-concept for Prism/Lodester/Lighthouse/Teku/Aragon/Nethermind/Geth — 00:08:28
- Networking Proposals:
- ·snap/2 draft: BAL-based state healing using block access lists — 00:50:00
- ·eth/71 response size cap clarified: 2MB soft cap, not 10MB — 00:51:03
- Testing Infrastructure:
- ·Execution specs v5.3.0 released: expanded EIP-8037 coverage plus all Osaka tests — 00:16:07
- ·Perf DevNet3: 50GB ERC20 complete, EOA bloating in progress — 00:52:27
- ·Geth synced on perf-devnet-3; Reth/Nethermind/Besu sync issues being resolved — 00:54:30
- ·5 Gigagas testing revealing out-of-memory issues in some clients — 00:56:14
- Glamsterdam Bal Devnet 2:
- ·BAL DevNet2 running with prefetch/optimization benchmarking nodes added — 00:11:27
- ·Aragon switching focus to DevNet3; Besu tracing issue under investigation — 00:12:47
- Glamsterdam Bal Devnet 3:
- ·EIP-8037 cross-client bug hunt ongoing; multiple consensus issues found — 00:13:23
- ·Besu: 7 failing tests, 2 false positives; fixes in progress — 00:14:28
- ·Marius: EIP-8037 implementation deeply complex; AI-generated code causing issues — 00:19:16
- ·Ben/Nethermind: 8037 implementation has tendrils affecting everything; disliked from complexity standpoint — 00:21:38
- ·Nethermind/Reth: parallel I/O not showing major benefits yet — 00:22:43
Action Items
- •CL teams / Justin Traglia — Make decision on PTC committee caching approach this week — 00:43:01
- •Parithosh / DevOps — Test withdrawals, deposits, blobs on ePBS DevNet0 locally before live testing — 00:46:06
Meeting chat151
- felix (eest)
gmonday
- danceratopz
gmonday!
- felipe
gm
- danceratopz
- Parithosh Jayanthi
- Giulio
- Stefan Starflinger
Reacted to "https://ethresear.ch..." with 👍
- Łukasz Rozmej
not basically done -> in Nethermind it needs to be redone from scratch properly.
- Łukasz Rozmej
Have we experimented with also using compression between EL<->CL?
- FLCL
compression makes it slower
- Parithosh Jayanthi
Also amount of data isn’t the issue right? Its just serializing/deserializing overhead
- Giulio
yeah the best case is literally passing pointer to the data struct
- Łukasz Rozmej
you mean shared memory or something?
- Giulio
yeah
- Giulio
but that's for the future, shared memory beetwen processes is hacky
- Giulio
possible ofc
- Łukasz Rozmej
it's quite simple, I've coded those in the past
- Potuz
The best is not to pass anything
- Giulio
the best is to pray
- Potuz
not even a pointer, just a root to sign
- Łukasz Rozmej
you can have simple messages for communication and shared memory for the payload
- Łukasz Rozmej
makes it simple
- Ben Adams
Replying to "The best is not to p..." vibe validation
- Giulio
some machine may run CL and EL on different machines
- Potuz
Replying to "The best is not to p..." no, we don't need anything, and on notify_newPayload, we should pass the snappy we get
- Giulio
*some operators
- Łukasz Rozmej
yeah we need a fallback then
- Potuz
Replying to "The best is not to p..." and you deal with it directly
- Giulio
the best case is to pass pointer :)
- Łukasz Rozmej
so it is only optional optimization
- Maria Silva
Sorry if I missed it, but what is the status on BAL optimizations for the various clients?
- Giulio
yeah like SSZ, we can do that in the future
- Potuz
Replying to "The best is not to p..." requests are essentially the only thing we need
- Potuz
Replying to "The best is not to p..." withdrawals could be just the hash
- Ben Adams
Reacted to "and you deal with it..." with 👍
- Giulio
I recall there was some feature for fast jsonrpc through shared memory but forgot about it
- Łukasz Rozmej
Replying to "Sorry if I missed it..." still in progress on Nethermind side, sorry
- Maria Silva
Reacted to "still in progress on..." with 👍
- Toni Wahrstätter
no updates,but Geth and Besu have been ready and devnet-2 is running with the optimizations enabled/disabled on different nodes
- Stefan Starflinger
Replying to "Sorry if I missed it..." Ah yes I forgot to ask, reth has some flags updates otherewise nethermind and etherex we don't have any flags yet. Thanks for the update Lukasz
- Łukasz Rozmej
just FYI the more we implemented 8037 the more we disliked it :(
- Stefan Starflinger
Replying to "Sorry if I missed it..." https://notes.ethereum.org/@ethpandaops/bal-devnet-3 flags documented here
- Dragan Rakita
Does it include refill of reservoir?
- Łukasz Rozmej
Reacted to "Does it include refi..." with 👍
- Łukasz Rozmej
Replying to "Does it include refi..." @Ben Adams?
- Stefan Starflinger
- Dragan Rakita
Replying to "Does it include refi..." On revert/halt return child spent state_gas to reservoir.
- Dragan Rakita
Replying to "Does it include refi..." Let me ping you in discord
- Maria Silva
Replying to "just FYI the more we..." What are the messy parts? Anything we should take into the design and improve on?
- Dragan Rakita
Replying to "Does it include refi..." https://discord.com/channels/595666850260713488/1476205965895536782/1476208158421549096
- spencer
Replying to "Does it include re..." Yeah it does! It is up to date with EIPs repo now. This commit: https://github.com/ethereum/EIPs/pull/11328/changes/ec612217fd2fda7409369a98d160a929829cb66f
- Ben Adams
Replying to "just FYI the more we..." Is a very deep change that effects everything
- Maria Silva
Reacted to "Is a very deep chang..." with 👍
- Mario Vega
Reacted to "just FYI the more we..." with 👍🏼
- Mario Vega
Reacted to "Is a very deep chang..." with 👍
- Dragan Rakita
Reacted to "Yeah it does! It is ..." with ❤️
- Stefan Starflinger
AI slob testing :D
- spencer
Reacted to "AI slob testing :D" with 🚀
- Ben Adams
Replying to "just FYI the more we..." @Marius van der Wijden would be interested in what you'd change
- felix (eest)
Replying to "AI slob testing :D" 0-90% feels so nice with AI, but the last 10%..
- spencer
Replying to "AI slob testing :D" Can we create an agent called slob?
- felipe
I have some more BAL benchmark tests lined up that we should sync on @Toni Wahrstätter
- Potuz
I'd like to discuss the two topics I added in the end to the agenda
- Parithosh Jayanthi
- Louis
Replying to "I have some more ben..." @Toni Wahrstätter @felipe https://github.com/jochem-brouwer/execution-specs/pull/1 This is for BAL benchmark by Felipe
- Stefan Starflinger
Reacted to "https://www.potuz.ne..." with 🔥
- Louis
Replying to "I have some more ben..." Sorry I’ve not yet reviewed it
- felipe
Reacted to "@Toni Wahrstätter..." with 💯
- Stefan Starflinger
Replying to "AI slob testing :D" Its very good when there are 1 or 2 things wrong, but when there are like 3+ things wrong its a real mess
- Toni Wahrstätter
Reacted to "@Toni Wahrstätter @f..." with 💯
- felipe
Reacted to "Sorry I’ve not ye..." with 👍
- Toni Wahrstätter
Replying to "I have some more ben..." Yeah def! That's the one focusing on parallel IO I meant: https://github.com/ethereum/execution-specs/pull/2033
- felix (eest)
Reacted to "https://www.potuz...." with 🔥
- Stefan Starflinger
Reacted to "@Toni Wahrstätter @f..." with 💯
- Potuz
The spec does allow it
- Potuz
it just says MAY
- Łukasz Rozmej
is Marius breaking up?
- Mario Vega
Reacted to "is Marius breaking u..." with 👍🏼
- NC
Replying to "is Marius breaking u..." yes
- Eitan Seri-Levi
Reacted to "is Marius breaking u…" with 👍🏼
- felix (eest)
german internet back at it again
- Francesco
Reacted to "is Marius breaking u..." with 👍🏼
- Justin Traglia
Reacted to "german internet back..." with 👍
- Justin Traglia
Reacted to "is Marius breaking u..." with 👍🏼
- spencer
Reacted to "german internet ba..." with 👍
- nixo
Reacted to "german internet back..." with 👍
- Dragan Rakita
Reacted to "german internet back..." with 👍
- felix (eest)
Reacted to "is Marius breaking..." with 👍🏼
- felipe
Reacted to "german internet ba..." with 👍
- nflaig
Reacted to "german internet back..." with 👍
- felipe
Reacted to "Yeah def! That's t..." with 💯
- Parithosh Jayanthi
Reacted to "german internet back..." with 😂
- Toni Wahrstätter
Replying to "I have some more BAL..." @felipe lmk how you want to proceed. My PR is still a draft but happy to convert it :)
- Potuz
Thanks :heart:
- Marius van der Wijden
Pushed a fix to our epbs-devnet-0 branch that allows for reorging to parents
- Francesco
The current PR seems fine if clients are fine with the occasional last minute change for the first slot of the epoch
- Łukasz Rozmej
Reacted to "german internet back..." with 👍
- Justin Traglia
- Ben Adams
Will we need the re-org behaviour spec'd somewhere? Since isn't allowed atm
- Marius van der Wijden
Replying to "Will we need the re-..." Yes we should. I remember there were engine api tests that demanded this behavior
- NC
We are adding 256kb to the state, but we really only 4kb (1 slot of PTC) of it. Is that right?
- nflaig
Replying to "The current PR seems..." I think this is fine, we fetch duties at the start of the epoch and ptc duty is quite late in the slot, imo we don't really need 2 epoch lookahead
- Francesco
Reacted to "I think this is fine..." with 👍
- Potuz
I love you @Marius van der Wijden
- Mario Vega
Reacted to "I love you @Marius v..." with ❤️
- Justin Traglia
Reacted to "I love you @Marius v..." with ❤️
- felix (eest)
Reacted to "I love you @Marius..." with ❤️
- raxhvl
Reacted to I love you @Marius v... with "❤️"
- jochem-brouwer
Heeft gereageerd op "I love you @Marius..." met ❤️
- nflaig
Reacted to "I love you @Marius v..." with ❤️
- Marius van der Wijden
Reacted to I love you @Marius v... with "❤️"
- Eitan Seri-Levi
I think LH might be able to handle them
- Łukasz Rozmej
Reacted to "Will we need the re-..." with 👍
- felipe
Reacted to "I love you @Marius..." with ❤️
- Potuz
@Parithosh Jayanthi that's mostly a call for you, I suspect that deposits should not affect neither Teku's or Nimbus ability to catch up
- Parithosh Jayanthi
Replying to "@Parithosh Jayanthi ..." Yeah I agree, I think its worth already breaking/finding out what’s broken in existing clients
- Justin Traglia
Reacted to "Yeah I agree, I thin..." with 👍
- Toni Wahrstätter
snap/2: https://github.com/nerolation/EIPs/blob/toni/snap-2/EIPS/eip-9999.md
- jochem-brouwer
Does this remove the same msgs from eth/71?
- jochem-brouwer
Antwoord verzenden naar "Does this remove t..." I will discuss this async
- Potuz
@stokes want to discuss the honest reorg feature here in light of FCU changes?
- Toni Wahrstätter
That's the PR for the eth/71 soft cap: https://github.com/ethereum/EIPs/pull/11387
- Ameziane Hamlat
So the idea is to test the BAL optimisations on top of perf-devnet-3 (Bloat network) ?
- Maria Silva
Replying to "So the idea is to te..." Yes, because we need bloatnet for the stageful repricings tests
- Maria Silva
Replying to "So the idea is to te..." *stateful
- Toni Wahrstätter
Replying to "So the idea is to te..." The pure exec optimizations (no state) can already be tested on devnet-2. For state related things we need to use a mainnet-state fork or bloatnet
- Ansgar Dietrichs
I heard 5 Gigagas on mainnet? brb notifying stakers
- jochem-brouwer
Heeft gereageerd op "I heard 5 Gigagas ..." met 😂
- Parithosh Jayanthi
Reacted to "I heard 5 Gigagas on..." with 😂
- Ameziane Hamlat
Reacted to "Yes, because we need..." with 👍
- Ameziane Hamlat
Reacted to "The pure exec optimi..." with 👍
- nixo
Reacted to "I heard 5 Gigagas on..." with 😰
- Mario Vega
Reacted to "I heard 5 Gigagas on..." with 😰
- Mario Vega
Reacted to "I heard 5 Gigagas on..." with 😂
- Ameziane Hamlat
Replying to "So the idea is to te..." So we want to do the tests on one snapshot instead of having a snapshot of mainnet and another one of the bloat network
- Ameziane Hamlat
Replying to "So the idea is to te..." I’m saying this because I don’t see the link between BAL and Bloat network
- nixo
isn’t that… acdt
- jochem-brouwer
Antwoord verzenden naar "isn’t that… acdt" acd-topic call? :)
- Louis
Reacted to "I heard 5 Gigagas on..." with 😂
- Dragan Rakita
Can we do async reporting?
- felipe
acd-TBD
- Louis
Reacted to "acd-TBD" with 😭
- Potuz
yeah TBD is a good topic :)
- felix (eest)
Reacted to "acd-TBD" with 😂
- danceratopz
Reacted to "acd-TBD" with 😆
- Justin Traglia
I can help here too
- Mario Vega
Reacted to "I can help here too" with ❤️
- Parithosh Jayanthi
Reacted to "I can help here too" with ❤️
- nixo
Reacted to "I can help here too" with ❤️
- felix (eest)
Reacted to "I can help here too" with ❤️
- felipe
Reacted to "I can help here too" with ❤️
Key decisions
SSZ engine API will be optional via content-type headers; deferred to ACD for final approval
PFIDeferred to ACD for final approvalEL must support building on older heads for ePBS; spec will be updated
To support ePBSACDT cadence will be evaluated weekly; cede to breakouts when no cross-cutting topics
To be evaluated weekly