AllCoreDevs - Consensus #180

gm

Good evening from Australia

9pm in Melbourne

It’s early

9pm > midnight

🦘🦘🦘🦘🦘🦘🦘

Aussie ACDC

Replying to "9pm > midnight" agree do disagree

Reacted to "Aussie ACDC" with ❤️

Reacted to "Aussie ACDC" with ❤️

Wait IF? You mean it isnt?

lol

Lets just all move to Svalbard

Also have less problems

Reacted to "Wait IF? You mean it..." with 🤣

Potuz is one of these ppl that think Earth is round…

Oh wow, what an incredible energy in this call today

Can’t afford beer there

Reacted to "Can’t afford beer th..." with 🔥

Reacted to "Potuz is one of these ppl that think Earth is round…" with 😂

Replying to "Can’t afford beer th..." he said: less problems ;)

Reacted to "he said: less problems ;)" with 😂

Replying to "Can’t afford beer th..." If we play poker every night at least Barnabas can afford the beers

Reacted to "If we play poker eve..." with 😂

Reacted to "Potuz is one of these ppl that think Earth is round…" with 😂

Reacted to "If we play poker every night at least Barnabas can afford the beers" with 😂

Reacted to "If we play poker every night at least Barnabas can afford the beers" with 😂

Reacted to "If we play poker every night at least Barnabas can afford the beers" with 😂

Reacted to "Potuz is one of these ppl that think Earth is round…" with 😂

https://dora.glamsterdam-devnet-5.ethpandaops.io/

Our bug was in sending invalid PTC attestations, that caused peering issues

If we catch up to head 🙂

Corroborated by Nico and his bot, we really do live in the future now

Reacted to "Corroborated by Nico..." with 🤖

Reacted to "Corroborated by Nico..." with 🤖

Reacted to "Corroborated by Nico..." with 🤖

Reacted to "Corroborated by Nico..." with 🤖

Replying to "Corroborated by Nico..." Trying to artificially increase the participant count on the call lol

ptc duties per branch sounds painful

Btw everyone seems to be struggling to range sync during non finality

Lodestar struggling Lighthouse struggling Prysm struggling

Replying to "Corroborated by Nico..." It took me 2 seconds to corroborate this when Eitan said the block was canonical in their branch

Yeh branch duties would be .. new

Yea seems gnarly

Its rolled

Add as trusted peers?

we just synced a lodestar node to head in our infra

Reacted to "we just synced a lodestar node to head in our infra" with 🥳

Enrico, I think we should do one thing at once though, otherwise it's even harder to debug what's going on

https://github.com/ethereum/beacon-APIs/pull/608

Reacted to "Enrico, I think we s..." with 👍

Replying to "Enrico, I think we s..." There are two different flags for blocks and payload

Replying to "Enrico, I think we s..." I can also add column delay so we can even do DA timeliness

Replying to "Enrico, I think we s..." nice, yeah I have a long todo list for deathstar to add stuff like that and more

Reacted to "nice, yeah I have a ..." with 👍

James is the maintainer

https://github.com/ethereum/consensus-specs/pull/5330

Reacted to "James is the maintai..." with 😂

Replying to "James is the maintai..." ‘A’ maintainer

Reacted to "James is the maintainer" with 😂

Replying to "James is the maintai..." I need approval from other maintainers

Lh making it default: https://github.com/sigp/lighthouse/pull/9365

Replying to "James is the maintai..." ok will try to get this reviewed more soon

Replying to "Lh making it default..." This is removing molex, not making quic default

Reacted to "ok will try to get t..." with 👍

Reacted to "This is removing molex, not making quic default" with 👍

LOL that’s deprecation no?

If you disable it by default

Reacted to "LOL that’s deprecation no?" with 💯

Replying to "If you disable it by..." we will run our nodes with it enabled for a time

Imo its very risky to have no molex running without flag is probably putting us in a difficult position, have discussed with lighthouse on this

Could we enable by default in gloas?

Reacted to "LOL that’s deprecation no?" with ❤️

do you guys have any concerns around quic? is it perfectly smooth?

Replying to "do you guys have any..." For teku, its minimally tested and this is a risk.

Replying to "If you disable it by..." Would be good to be able to fall back even without the flag for now

It will end up creating clusters then because those nodes that can’t connect to LH will connect with other nodes that support them, making LH less connected to those nodes

Reacted to "For teku, its minima..." with 👍

Which in turn makes LH more isolated

I like the rationale though, if we do it without gloas as well, there’s less chance of breaking things

Seems a bit risky to put this under a flag, I’d put it under a flag that is enabled by default and then flip it in the next release

Why remove mplex?

Btw Teku was able to sync up from the last finalized checkpoint - lighthouse, Prysm and lodestar struggling

Replying to "Btw Teku was able to..." Yeah you just do this to have Lucas bragging

Reacted to "Yeah you just do this to have Lucas bragging" with 😂

Reacted to "Yeah you just do this to have Lucas bragging" with 😂

Reacted to "Yeah you just do thi..." with 😂

I sympathise, but its actually in the spec also

We cannot have that consensus here cause we don’t know the other side of the story AFAICT, and we have had serious bugs on mplex that had to fix when we wanted to get rid of this for years now. But you should really weigh in the topology change that will surely happen with this change

Always having a fallback sounds necessary

I’d like to propose to drop mplex on gloas + 2month maybe?

Reacted to "I’d like to propose ..." with 👍

Reacted to "I’d like to propose ..." with 👍

Reacted to "I’d like to propose to drop mplex on gloas + 2month maybe?" with 👍

We just need 2 releases from Teku

We have mplex enabled on the devnet rn

Would need a cli arg - easy enough?

Reacted to "Would need a cli arg - easy enough?" with 👍

ethereum/beacon-APIs#618

https://www.migalabs.io/blog/validator-states

Doesn’t this risk breaking existing tooling? Should we get people that build those tools to weigh in?

Yeh would need to be v2

Reacted to "Yeh would need to be v2" with 👍

Reacted to "Yeh would need to be..." with 👍

extend /eth/v1/node/peers with peer scoring and disconnect reasons https://github.com/ethereum/beacon-APIs/pull/606

I love this idea, not sure if we can actually fill those fields accurately though

But we definitely should ship this

Replying to "extend /eth/v1/node/..." might also wanna do a v2 here but curious if people are happy with adding fields, it's only a json api anyways, so might be fine

Yep sounds fine

extra_data field is a good idea

Lgtm

@Barnabas Prysm-Nethermind is fine, Prysm-Ethrex is synced on one node and almost there on the second one. Only Prysm-Erigon is having trouble. So you posted above only to make Lucas happy

Reacted to "@Barnabas Prysm-Nethermind is fine, Prysm-Ethrex is synced on one node and almost there on the second one. Only Prysm-Erigon is having trouble. So you posted above only to make Lucas happy" with 😂

Builder execution requests: https://github.com/ethereum/EIPs/pull/11760

Justin's list https://github.com/jtraglia/kurtosis-devnets/tree/master/gloas-deposits

I think this is the right solution, the tradeoff here is that this will considerably delay the fork and it’s purely a decision between these two priorities IMO

It would revert when it reaches the max on the EL?

Last time we copy / pasted we broke holesky

Reacted to "Last time we copy / pasted we broke holesky" with 🤳

Reacted to "Last time we copy / pasted we broke holesky" with 🤳

Why would a seperate builder deposit contract solve the max deposit issue?

I honestly think we are spending too much time on this problem that cannot really happen

Reacted to "I honestly think w..." with 👍

Reacted to "I honestly think we are spending too much time on this problem that cannot really happen" with 👍

Reacted to "I honestly think we are spending too much time on this problem that cannot really happen" with 👍

Reacted to "I honestly think we ..." with 👍

Replying to "I honestly think w..." then the scenarios can be ignored? if not it's not too much time

Replying to "I honestly think w..." either they're irrelevant or they have to be addressed

Replying to "Why would a seperate..." the separate contract has a smaller maximum

Replying to "Why would a seperate..." also has an actual queue like the consolidation contract

all of these attacks go away when you use this new contract ;)

There's definitely an appeal

I think the barrier for this level of a change is far higher than back in dec

Replying to "all of these attacks..." and all of the annoying caches (error prone, forever testing surface)

Replying to "all of these attacks..." Agreed

Replying to "all of these attacks..." The right solution is the new contract

Replying to "all of these attacks..." But all of this is also overblown for no reason

Replying to "all of these attacks..." well its enough of a reason to need to test and maintain the code :(

Replying to "all of these attacks..." preference for never writing it in the first place

I think this is really overly exaggerated, there are much cheaper attacks on liveness on Ethereum

Final call on Staked Builder API for Glamsterdam builder-specs#138, I ll be merging it. We can update it if needed by raising PRs.

It would be great to determine if its a worthwhile attack that are likely to happen or not

Replying to "I think this is real..." This exactly

Replying to "I think this is re..." I continue to maintain the emphasis on the caches belies this

Reacted to "It would be great to determine if its a worthwhile attack that are likely to happen or not" with 👍

Replying to "It would be great to..." It is a worthwhile attack if you want to prevent liveness and clients can’t handle it. BUT if you want to do that, you can save money doing it differently

Replying to "Final call on Staked..." sendit

Replying to "Final call on Staked..." Pr is already too long to review lol

if clients pass all s1-s9 (soon up to s10) scenarios we are fine

Replying to "Final call on Staked..." Had to scroll 20 mins to get to the bottom of it

remove submitBeaconBlock API builder-specs#162

Replying to "I think this is real..." Tts a testing / maintenance burden. Getting it right is not exactly simple. We already thought we got it right, and yet were able to cause a chain stall in testing yesterday. The tradeoff is submitting a new contract that is 99% copy of an existing audited contract. Imo not even close.

Replying to "I think this is re..." Yeah I'm not sure I buy this narrative of delaying the fork into 2027

Replying to "I think this is real..." Not only that, you need to change specification and then audit the new specification. The number of bugs that were introduced in the spec when we moved from builders being validators to not was huge. This change is along the same lines. I completely agree that this is the right solution, but it’s not a simple change by no means

Replying to "I think this is real..." the spec change is small. it actually is a simple change.

Stateful is a concern I think for several ppl

Replying to "I think this is real..." Not, really, you need to deal with different engine API, you need to add new requests, you need to remove any optimization that dealt with not having the same pub key on different slices, etc..

Replying to "Stateful is a conc..." Yes, I disagree with adding stateful things to the APIs

Replying to "I think this is real..." @Dustin wdym? Even with 0 changes 2026 seems tight. How would we achieve a fork this year with more changes? That being said, is your claim that we shouldn’t care when the fork happens? If that’s the viewpoint then we should really split the fork and ship BAL/repricings, we cannot afford to hold off those changes for months when they are mostly done.

Replying to "I think this is real..." I am in no rush to ship the fork, so I personally do not mind a delay, it would be useful to do other maintenance… but this will delay it considerably

Replying to "Stateful is a conc..." They work poorly/not at all across multi-BN or multi-EL setups (depending on where in the stack they are)

Beacon-api 580 😄 jk

https://github.com/ethereum/execution-apis/pull/793

Replying to "I think this is real..." well we already have a non-zero testing timeline as it is. (The S1-S9 attacks that are currently breaking clients). We can spend time on patching attack scenarios, or we can spend time doing some minor spec massage.

Replying to "Stateful is a concer..." it isn't really that stateful though, you can always default to publishing to p2p if you have a cache miss

neth has it done, but we are not sure about some spec details, which is in a draft

Didn’t you have Barnabas’ version?

Replying to "Stateful is a conc..." "They work poorly"

EIP-8243: # Batching Attestations at Source (github.com/ethereum/EIPs/pull/11589

Reacted to "well we already have a non-zero testing timeline as it is. (The S1-S9 attacks that are currently breaking clients). We can spend time on patching attack scenarios, or we can spend time doing some minor spec massage." with 👍

Andrew B Coathup: Mascot needed for Glamsterdam upgrade 🐜🦫🐝🦩🐹🐙🐻‍❄️🐩 Vote for your favorite on Eth Magicians https://ethereum-magicians.org/t/mascot-needed-for-glamsterdam-upgrade/26008

Replying to "I think this is real..." Yes that is fair, if we do go down this path we should consider splitting the fork + shipping the new contract + ssz as a CL fork after. BAL + repricing can be done earlier.

Reacted to "Mascot needed for Glamsterdam upgrade 🐜🦫🐝🦩🐹🐙🐻‍❄️🐩 Vote for your favorite on Eth Magicians https://ethereum-magicians.org/t/mascot-needed-for-glamsterdam-upgrade/26008" with ❤️

Reacted to "well we already have a non-zero testing timeline as it is. (The S1-S9 attacks that are currently breaking clients). We can spend time on patching attack scenarios, or we can spend time doing some minor spec massage." with 👍

Replying to "neth has it done, bu..." let's know if you need an EL image support any kind of engine via SSZ spec

Replying to "I think this is real..." Nah that doesn’t make sense, going with the contract needs to be done before Gloas of course

Reacted to "well we already have..." with 👍

The 19% or whatever doesn't matter, it's the # vals/node

BitMine and BlackRock are missing in that list

Which is orders, orders of magnitude smaller

Reacted to "BitMine and BlackR..." with 👍

Lido there seems incorrect, I hardly see them being able to aggregate at source

RPL etc can't either

it's just an irrelevant set of numbers

Reacted to "The 19% or whateve..." with 👍

Reacted to "The 19% or whatever doesn't matter, it's the # vals/node" with 👍

Feels too much work for too little benefit? Can’t we just fix this on ethp2p?

We already need to identify stronger aggregates...

Yeah, that code is already there anyway

In all clients

2 more mins raul, would like some time for q&a

Replying to "Lido there seems i..." Point is, with current status (0x01) it saves a factor of more than 20, but after consolidations, it will not

Replying to "2 more mins raul, wo..." You’ll have to say that out loud I think

Reacted to "You’ll have to say that out loud I think" with 👍

It both hurts/harms attester privacy

Replying to "Lido there seems i..." Permissioned Operators run avg 750 vals per node, meaning 750/32 = 23 per slot.

can, but it shows affiliation. it's a measure of trust

Replying to "Lido there seems i..." But with 0x02 you divide this by 64, so no effect essentially..

Replying to "Lido there seems i..." CSM is almost 1 val per NO, so no impact at all

good point

So just one big subnet then?

Yeah I think with 16 that number is going to be very low I assume

Reacted to "Yeah I think with 16..." with 👍

Most Pro Operators run 1000+ validators per node

Yeah, the latency thing is a concern

I’d say, if you have too many validator problem, consolidate.

Is it usually 1000+ validators on a 1BN<>1VC setup? Or do you mean 1BN<>multiple VCs?

I am on the other side of Paul, I think this is a great idea, I’d like to get rid of the batcher and the seal cause that is a deeper change on the validator client flow and it looks to me that it saves us very little

Reacted to "So just one big su..." with 😁

How much value does the cross-node delegation add, in general? There's the batching part and the delegation part, and they're sort of separate

Yes that may become more interesting

@Parithosh Jayanthi will we try the 11am UTC timeslot again for ACDC?

Replying to "@Parithosh Jayanthi ..." I think not, we will default to the old time

Replying to "@Parithosh Jayanthi ..." And discuss asynchronous how useful the time was

Reacted to "I think not, we will..." with 👍

Replying to "@Parithosh Jayanthi ..." Why?

Mascot needed for Glamsterdam upgrade 🐜🦫🐝🦩🐹🐙🐻‍❄️🐩 Vote for your favorite on Eth Magicians https://ethereum-magicians.org/t/mascot-needed-for-glamsterdam-upgrade/26008

Replying to "@Parithosh Jayanthi ..." I thought this was permanent?

Replying to "Mascot needed for Gl..." Giphy [ID:alRXefLYnqElvEgsqY] [Full message cannot be displayed on this version]

Reacted to "I think not, we will..." with 😅

Replying to "@Parithosh Jayanthi ..." It was a trial