AllCoreDevs - Testing #091
Transcript
- jochem-brouwer
gm gm!!
- Justin Traglia
Gm!
- Daniel Knopik
gm
- Mehdi Aouadi
gm gn
- Justin Traglia
- draganrakita
Reacted to "Agenda: https://github.com/ethereum/pm/issues/2181" with ❤️
- Marcos A. Maceo
Reacted to "Agenda: https://github.com/ethereum/pm/issues/2181" with ❤️
- Mario Vega
Reacted to "Agenda: https://github.com/ethereum/pm/issues/2181" with ❤️
- terence
fixed
- Parithosh Jayanthi
Should be fixed
- Mario Vega
Reacted to "fixed" with 🚀
- Justin Traglia
Reacted to "fixed" with 🚀
- Parithosh Jayanthi
(Tested fix on like 4 nodes today when swapping back from tysm)
- Barnabas
lighthouse-nimbusel seems to have struggled from toni’s attack
- spencer
Reacted to "fixed" with 🚀
- terence
What’s Toni’s attacK
- Iván | ethrex
Reacted to "fixed" with 🚀
- Iván | ethrex
Reacted to "What’s Toni’s attacK" with 🥷
- spencer
Reacted to "What’s Toni’s at..." with 🥷
- Justin Traglia
Reacted to "What’s Toni’s attacK" with 🥷
- draganrakita
Reacted to "What’s Toni’s attacK" with 🥷
- jochem-brouwer
Great attack Toni 😃👍
- jochem-brouwer
Heeft gereageerd op "What’s Toni’s at..." met 🥷
- Rimeeeeee
Reacted to "What’s Toni’s at..." with 🥷
- Edgar | ethrex
Se ha reaccionado a "What’s Toni’s at..." con 🥷
- Mario Vega
Reacted to "What’s Toni’s attacK" with 🥷
- Parithosh Jayanthi
Reacted to "What’s Toni’s attacK" with 🥷
- jochem-brouwer
Is there a way how we can add this to benchmarks? I think I understand that this is a CL+EL benchmark, so not a pure EL one (cannot add this to execution-specs benchmarks / benchmarkoor)?
- Dustin
CLs don't know this really
- Stefan Starflinger
how does the block hash stay the same when the payload changes?
- Stefan Starflinger
Reacted to "What’s Toni’s attacK" with 🥷
- Kevaundray Wedderburn
Is there a writeup of this issue?
- Kevaundray Wedderburn
Replying to "Is there a writeup o..." Was having audio issues, so didn’t hear it entirely
- Mehdi Aouadi
Reacted to "Is there a writeup of this issue?" with ➕
- Mario Vega
Reacted to "Is there a writeup of this issue?" with ➕
- jochem-brouwer
We should add this to some kind of CL benchmark infrastructure / test infrastructure if possible
- Nico Flaig
why doesn't `[IGNORE] The node has not seen another valid `SignedExecutionPayloadEnvelope` for this block root from this builder` cover the re-propagation?
- Mehdi Aouadi
Reacted to "why doesn't `[IGNORE] The node has not seen another valid `SignedExecutionPayloadEnvelope` for this block root from this builder` cover the re-propagation?" with ➕
- jochem-brouwer
Antwoord verzenden naar "We should add this..." I think we can add this to a hive test?
- terence
Replying to "why doesn't `[IGNORE..." Is there a bug with re propagation here?
- Nico Flaig
Replying to "why doesn't `[IGNORE..." lodestar only propagates the first signed envelope from a builder for block root
- terence
How does a large payload affect BLS?
- Mehdi Aouadi
Replying to "why doesn't `[IGNORE..." Teku does the same
- Eitan Seri-Levi
Replying to "why doesn't `[IGNORE] The node has not seen another valid `SignedExecutionPayloadEnvelope` for this block root from this builder` cover the re-propagation?" We are repropogating envelopes we’ve already seen. We are fixing this in the coming days it will be ready for devnet 8
- Toni Wahrstätter
yeah not having the IGNORE might be the bug here
- Justin Traglia
Reacted to "yeah not having the IGNORE might be the bug here" with 👍
- Mehdi Aouadi
Reacted to "yeah not having the IGNORE might be the bug here" with 👍
- Mario Vega
Reacted to "yeah not having the IGNORE might be the bug here" with 👍
- Justin Traglia
Replying to "yeah not having the ..." To be clear, it is in the spec.
- Stefan Starflinger
Reacted to "To be clear, it is i..." with 👍
- Mehdi Aouadi
Reacted to "To be clear, it is in the spec." with 👍
- Nico Flaig
Reacted to "To be clear, it is i..." with 👍
- jochem-brouwer
Hive test?
- Justin Traglia
Replying to "yeah not having the ..." And in the next release, there will be reference tests for Gloas gossip validation functions. Here is the test for that check.
- Barnabas
geth reth still failing 50ish tests
- Barnabas
others failing under 10
- Eitan Seri-Levi
Reacted to "yeah not having the IGNORE might be the bug here" with 👍
- Dustin
public devnet or public testnet?
- Stefan Starflinger
devnet-8 is currently planned to be public
- Dustin
i.e. is there going to be a "devnet 9" of any form at all
- Barnabas
public testnet
- Mario Vega
Replying to "public devnet or pub..." public devnet, before public testnets iiuc
- Justin Traglia
Replying to "devnet-8 is currentl..." Aren’t they all technically public 😅
- Barnabas
devnet 8 == public testnet
- Stefan Starflinger
Replying to "devnet-8 is currentl..." yes
- Justin Traglia
Reacted to "yes" with 😄
- Barnabas
devnet 8 timeline is whenever you guys are done
- FLCL
might be cool to have a copy paste to share on twitter about the most affected parties, who use this or that
- Nico Flaig
Reacted to "yeah not having the ..." with 👍
- Chris Berry
Is Platåberget just another name for devnet 8?
- Barnabas
Reacted to "Is Platåberget just another name for devnet 8?" with 👍
- nixo
Replying to "Is Platåberget just ..." yes
- Mario Vega
Reacted to "Is Platåberget just another name for devnet 8?" with 👍
- Stefan Starflinger
Reacted to "Is Platåberget just ..." with 👍
- Barnabas
Replying to "Is Platåberget just ..." devnet 8 is just easier to spell lol
- Stefan Starflinger
Reacted to "might be cool to hav..." with 👍
- Chris Berry
Replying to "Is Platåberget just ..." I’ve been referring to it as PB as it’s unpronounceable
- nixo
Reacted to "I’ve been referring to it as PB as it’s unpronounceable" with 😂
- Justin Traglia
Reacted to "devnet 8 is just easier to spell lol" with 😄
- Marius Van Der Wijden (M)
The numbers haven't changed drastically between 7 and 8 though
- nixo
Replying to "Is Platåberget just ..." the peanut butter devnet
- Chris Berry
Replying to "Is Platåberget just ..." I support this name change
- Miguel Tenorio
Reacted to "the peanut butter devnet" with 😂
- Chris Berry
Replying to "Is Platåberget just ..." Will supply jelly.
- Miguel Tenorio
Reacted to "I support this name change" with ➕
- Parithosh Jayanthi
Reacted to "the peanut butter devnet" with 😂
- Marius Van Der Wijden (M)
So I wouldn't expect large changes going forward
- Justin Traglia
Reacted to "Will supply jelly." with 🪼
- FLCL
Reacted to "I’ve been referring to it as PB as it’s unpronounceable" with 😂
- FLCL
Reacted to "the peanut butter devnet" with 😂
- nixo
Reacted to "Will supply jelly." with 🪼
- Justin Traglia
Reacted to "the peanut butter devnet" with 😂
- Dustin
what's the scope of remaining changes in terms of, is there convergence to prices and it's figuring out +/- the last few %?
- cayman
Reacted to "the peanut butter de..." with 😂
- Parithosh Jayanthi
Replying to "Is Platåberget just ..." Someone redo this song with “peanut butter devnet time” pls https://www.youtube.com/watch?v=eRBOgtp0Hac
- Dustin
or is it possible some are still 50% off
- Chris Berry
Reacted to "the peanut butter de..." with 😂
- spencer
Reacted to "the peanut butter ..." with 😂
- Barnabas
does security team would have an update on the audit progress of the builder system contracts?
- jochem-brouwer
Heeft gereageerd op "does security team..." met ➕
- Marius Van Der Wijden (M)
Delaying 8, yes
- Parithosh Jayanthi
Reacted to "does security team would have an update on the audit progress of the builder system contracts?" with ➕
- Justin Traglia
Replying to "does security team w..." I believe Felix is the one handling this. Not the security team.
- Justin Traglia
Replying to "does security team w..." Like the one coordinating with the auditors.
- jochem-brouwer
Heeft gereageerd op "Delaying 8, yes" met ❤️
- Stefan Starflinger
Reacted to "Delaying 8, yes" with ❤️
- spencer
Reacted to "Delaying 8, yes" with ❤️
- terence
For prysm, I don’t think we put genesis state in binary
- Stefan Starflinger
https://github.com/ethereum/pm/issues/2181#issuecomment-5241502214
- cayman
17MB is fine, we don't put any state in the binary
- Parithosh Jayanthi
70mb
- Eitan Seri-Levi
yeah same with us
- terence
Fetch it from GitHub previous, we just provide a example cmd
- Dustin
for the devnets yeah we use the ethclients repo version of these. we do by default include it in the binaries, would have to see what makes the most sense here
- Dustin
70MB would be large for that
- Parithosh Jayanthi
Is this via git Lfs? (Ill have to make sure our billing limits are appropriate if so)
- Parithosh Jayanthi
S3 = cheap, git ifs == cheap-ish
- jochem-brouwer
For EIP editors in the call, please approve https://github.com/ethereum/ERCs/pull/1917 and https://github.com/ethereum/EIPs/pull/12021 due to approval of https://github.com/ethcatherders/EIPIP/issues/411 It blocks moving an EIP to Review for Glamsterdam, thanks 😃
- Nico Flaig
Reacted to "the peanut butter de..." with 😂
- Barnabas
hoodi genesis.ssz is 145MB
- Stefan Starflinger
Sounds to me generally it should be fine and I would prefer not to forgo this test vector
- Justin Traglia
Can CL’s “schedule” a default gas target value? I wouldn’t want 200m enabled whenever glamsterdam releases are made.
- Nico Flaig
we can just do another release and bump the default incrementally
- draganrakita
Reacted to "we can just do another release and bump the default incrementally" with 👍
- terence
Couldn’t you set it in the EL ?
- Dustin
CLs do drive it via builder API
- Toni Wahrstätter
just let the gas limit naturally ramp up
- Justin Traglia
Dragan’s suggestion is my preference. I would prefer to bump the gas target in CL clients in the release after the upgrade.
- Toni Wahrstätter
for a few blocks, a few transactions that are heavy on state creation might fail and work again a bit later
- Nico Flaig
Reacted to "Dragan’s suggestion ..." with 👍
- Justin Traglia
Reacted to "we can just do another release and bump the default incrementally" with 👍
- Toni Wahrstätter
Why doing it in a separate release though?
- Toni Wahrstätter
This wastes having more granular controll over it at the hardfork. It makes things messier, requiring coordination, pushing validators to upgrade
- draganrakita
Only downside is that blocks would gradually increase over time, and would hit 200M only after few months when all validators update to newest client version.
- terence
Why can’t client set it with 2 gas limits. One pre gloss and one post gloas
- Justin Traglia
What percentage of the network needs to bump their gas targets in order for the network to reach 200m? 51%?
- Nico Flaig
you can't do that Toni
- Nico Flaig
would need to be epoch/slot aware
- Barnabas
can’t we have a gas_limit -> fulu_gas_limit, and a new flag like gloas_gas_limit, that will be the new gas_limit post gloas ?
- draganrakita
Replying to "What percentage of t..." With 51% it would levitate over ~130M, half would decrease it and other half would increase it by 12%
- Justin Traglia
Reacted to "With 51% it would levitate over ~130M, half would decrease it and other half would increase it by 12%" with 🙏
- Chase Wright
Call it “next fork” then it doesn’t have to disappear
- Daniel Knopik
Reacted to "Call it “next fork..." with 👍
- Barnabas
Reacted to "Call it “next fork” then it doesn’t have to disappear" with 👍
- Barnabas
Replying to "Call it “next fork” ..." or that
- Ben Adams
gloas calls a different engine api that has the gaslimit; can't use same switch?
- Nico Flaig
the gas limit it supposed to be controlled by operators, not client devs, so this feature goes into the other direction
- jochem-brouwer
Can someone EIP this idea?
- Barnabas
Replying to "gloas calls a differ..." cl teams could introduce a new flag for this.
- Mario Vega
Reacted to "Can someone EIP this idea?" with 👍🏼
- Barnabas
Replying to "gloas calls a differ..." but they wanted to override the existing flag
- Dustin
"deprecated" flags in production never really disappear
- jochem-brouwer
Antwoord verzenden naar "Can someone EIP th..." This is non-Core as it should not require (?) any protocol changes. So we can "just" add this to Hegota for instance
- Dustin
it's quite bad
- spencer
why do we need 200M at the fork boundary?
- terence
One flag for pre gloss becomes no op after gloas One flag for post gloss
- Barnabas
Replying to "why do we need 200M ..." we don’t, but we need to start signaling towards 200M
- Mario Vega
Does it have to be a flag? Is there any other way to bake a default for a specific fork?
- draganrakita
Replying to "why do we need 200M ..." We don’t need it
- Simon (@siladu) | Besu
Replying to ""deprecated" flags i..." Can’t the existing override flag just override both defaults?
- Nico Flaig
Replying to "why do we need 200M ..." I don't think we should have 200M at the fork, seems risky
- spencer
does it take a while to get to 200M? ahh its more a people wont update it either
- terence
To Barnabas point we can always pressure the professional node operator set
- spencer
Reacted to "To Barnabas point ..." with 🙏
- terence
Replying to "To Barnabas point we..." We just need a few of them, but it will be slow
- terence
Replying to "To Barnabas point we..." Because of the ramp up
- spencer
Replying to "To Barnabas point ..." "we promise no issuance changes in the future"
- Chase Wright
Reacted to ""we promise no issuance changes in the future"" with 😂
- Chris Berry
Reacted to ""we promise no issua..." with 😂
- Ben Adams
Issue is state gas is getting much higher so needs higher gaslimit; or txs pre fork will fail post fork
- Edgar | ethrex
Se ha reaccionado a ""we promise no iss..." con 😂
- Barnabas
Reacted to "Issue is state gas is getting much higher so needs higher gaslimit" with 💯
- Barnabas
yes mandatory, we delaying the launch anyway, so get to it
- Mario Vega
- FLCL
btw does it make sense to have both sparse + non sparse nodes?
- draganrakita
Reacted to "Issue is state gas is getting much higher so needs higher gaslimit" with 💯
- spencer
Reacted to "Issue is state gas..." with 💯
- spencer
Reacted to "yes mandatory, we ..." with ❤️
- FLCL
i mean do you guys want a flag?
- Barnabas
Replying to "btw does it make sen..." if its mandatory, then it should all be sparse
- FLCL
Replying to "btw does it make sense to have both sparse + non sparse nodes?" some time network will contain both
- FLCL
Replying to "btw does it make sense to have both sparse + non sparse nodes?" i mean mainnet and so on
- Chris Berry
Reacted to "Someone redo this so..." with 😂
- Barnabas
https://eips.ethereum.org/EIPS/eip-8136 https://eips.ethereum.org/EIPS/eip-8189
- jochem-brouwer
Heeft gereageerd op "https://eips.ether..." met 🚀
- Justin Traglia
Reacted to "https://eips.ethereum.org/EIPS/eip-8136 https://eips.ethereum.org/EIPS/eip-8189" with 🚀
- Mario Vega
Reacted to "https://eips.ethereum.org/EIPS/eip-8136 https://eips.ethereum.org/EIPS/eip-8189" with 🚀
- Barnabas
potuz’s post: https://www.potuz.net/posts/gloas-circuit-breaker/
- terence
Our first part (just to deal with builder) has been approved, we haven’t merged it. We can cherry pick it to devnet7 if you guys want to test
- wolovim
Unrelated psa: will be scheduling a bi-weekly frame tx breakout call, lead by lightclient, starting tuesday 18th @ 2pm utc (right after FOCIL breakout). Will also announce on discord.
- Barnabas
could we trigger it ?
- jochem-brouwer
Heeft gereageerd op "Unrelated psa: wil..." met ❤️
- Justin Traglia
Reacted to "Unrelated psa: will be scheduling a bi-weekly frame tx breakout call, lead by lightclient, starting tuesday 18th @ 2pm utc (right after FOCIL breakout). Will also announce on discord." with ❤️
- Barnabas
Replying to "Our first part (just..." if its gnna be in devnet 8 branch then its fine?
- jochem-brouwer
@lightclient could you approve https://github.com/ethereum/EIPs/pull/12021 as EIP editor? This unblocks updates to EIP-8282 (https://github.com/ethereum/EIPs/pull/11869 ), thanks 😃
- Mehdi Aouadi
Replying to "could we trigger it ..." Teku CB uses the same existing (blocks) circuit breaker options (the default values are too permissive for Gloas) so we should reduce them to trigger it. I’ll share stricter values to configure
- cayman
Reacted to "@lightclient could y..." with 😍
- Nico Flaig
Reacted to "@lightclient could y..." with 😍
Call summary
Targets
- •ACDE (this week): confirm or revise Glamsterdam repricing numbers - 00:25:03
Decisions
- •Payload-flooding retest deferred to DevNet 8 (not DevNet 7); spec fix for duplicate envelope IGNORE rule to be implemented by affected clients - 00:21:22
- •Delay DevNet 8 launch until ACDE; repricing numbers must be confirmed final first - 00:31:18
- •EIP-8070 mandatory for DevNet 8; no objections raised - 00:58:30
Highlights
- Eip 8070:
- ·EIP-8070 last-call for objections; none raised; confirmed mandatory for DevNet 8 - 00:56:07
- Gas Limit Bump:
- ·Proposal: bump default gas limit to 200M at Glamsterdam fork to prevent pre-fork transactions failing post-repricing - 00:42:39
- ·CL clients pushed back: enabling 200M in Glamsterdam release risks triggering pre-fork; requires effectively two gas limit flags - 00:46:14
- ·No resolution; Barnabas to draft a proposal that avoids requiring two client releases without dual-flag complexity - 00:55:37
- Devnet 7 Status:
- ·DevNet 7 stable; no finality stalls recently; Prysm sync stall being investigated/fixed - 00:08:43
- ·Toni's attack: flooding network with thousands of 9.5 MB signed payloads causes missed slots; Lighthouse repropagating envelopes it already saw (fix incoming for DevNet 8) - 00:09:36
- ·Root cause: missing IGNORE rule for duplicate SignedExecutionPayloadEnvelope per builder; in spec but not implemented by all clients - 00:20:07
- Devnet 8 Planning:
- ·Repricing numbers for DevNet 8 not yet confirmed final; running benchmarks on mainnet-representative state to verify; decision expected by ACDE - 00:23:44
- ·DevNet 8 (Platåberget) is public-facing testnet; repricing must be final before launch to avoid second repricing on public network - 00:31:48
- ·Stefan proposes ~70 MB genesis SSZ state in DevNet 8 to retest SSZ encoding issues from DevNet 7; general consensus to proceed - 00:33:01
- ·Barnabas requests optional SNAP2 and stateful deltas testing on DevNet 8; EIP-8136 and EIP-8189 shared in chat - 00:59:07
- Circuit Breaker Updates:
- ·Teku circuit breaker implemented and in DevNet 8 branch; enabled by default but values too permissive; Mehdi to share tighter config flags - 01:00:20
- ·Nimbus circuit breaker: porting existing Fusaka design to payloads; no Lighthouse rep present to give update - 01:02:09
Action Items
- •Toni Wahrstätter / all clients - Toni to write up payload-flooding attack and retest on DevNet 8; notify all clients when starting so they can capture live profiles - 00:17:52
- •lightclient (EIP editor) - Approve EIPs/pull/12021 to unblock EIP-8282 updates for Glamsterdam - 01:03:29
Key decisions
Payload-flooding retest deferred to DevNet 8 (not DevNet 7); spec fix for duplicate envelope IGNORE rule to be implemented by affected clients
Payload-flooding retest and spec fix for duplicate envelope IGNORE ruleDelay DevNet 8 launch until ACDE; repricing numbers must be confirmed final first
DevNet 8Repricing numbers must be confirmed final before DevNet 8 launch

