This week in Bitcoin Core
Bitcoin Core contributor kevkevin covers some of the latest updates...
Hello 👋 folks, I’m kevkevin. I’m an open-source developer and reporter for Insider Edition. Last week, I reviewed several pull requests from the Bitcoin Core repo. Here’s what I found notable.
Merged PR’s
Every week, several changes are officially added to Bitcoin Core. This week, 19 changes were merged. Here are some I thought were interesting from this week.
Add util::Expected (std::expected) by maflcko
This change adds a new util named Expected. The Expected class provides a standard way for low-level functions to return either error values or result values. It is basically the same as std::expected but missing some features, but they can be added in the future if wanted.
This makes it easier to return an error or a result value from a function. Right now we are using std::optional for this but it does not make sense to add an error value to this.precalculate SipHash constant salt XORs by l0rinc
This is part of the IBD speed up initiative. Basically the in-memory UTXO set SipHash to avoid collision attacks. The issue with that is that the hashing is done quite frequently throughout the codebase. This can be quite inefficent. What l0rinc did in this PR is introduced a PresaltedSipHasher that caches the initial SipHash state that reduced a redundant constant computation.
We can see these results compared to master:create_set - 7,386,349.49 / 7,312,133.16 - 1.0% faster
hash - 42,254,558.08 / 41,978,882.62 - 0.6% faster
match - 16,962,220.14 / 16,549,695.42 - 2.4% faster
mismatch - 12,991,047.69 / 12,713,595.35 - 2.1% faster
validation: Improve warnings in case of chain corruption by mzumsande
The idea with this change is that in the case of corruption that leads to a block being marked as invalid when the rest of the network sees it as valid the user does not get any good error messages. But they will still be stuck in a forever header-sync loop which is not ideal.
This change adds visibility if this issue were to happen by adding logs that the user can see. If this happens to a user in the future then it will be easier to debug by deleting that block or marking it as valid.scripted-diff: Use LogInfo over LogPrintf by maflcko
Last week we actually saw a change that was a precourser to this. You can checkout last weeks edition to read more on it. This change is a straight forwared one hence the scripted-diff. It replaces where LogPrintf is used with LogInfo. This just makes the logging in the project more uniform.
There are always changes being updated and reviewed in real-time. Here are some notable PR’s that are still up and looking for reviews.
Cuts out some wasted time in net socket handling. First, only calculates the current time once every 50ms, rather than once for each peer, which given we only care about second-level precision seems more than adequate. Second, caches the value of the
-capturemessagessetting inCConnmanrather than re-evaluating it every time we invokePushMessaage.
IRC meeting notes
Every week on Thursday, there is an IRC meeting. Here are some short notes from that meeting.
Fuzzing WG Update (dergoegge)
dergoegge: no update, but i’ll commit to an update next week
Kernel WG Update (sedited)
stickies-v: not sure if sedited is here, so i’l do somehting impromptu
stickies-v: stringintech has been doing a lot of work on extending the language bindings test suite, latest PR in https://github.com/stringintech/kernel-bindings-tests/pull/11
stickies-v: i’ve been doing a lot of work on separating node and kernel logging, will open an RFC issue by tomorrow
Benchmarking WG Update (l0rinc, andrewtoth)
l0rinc: #30442 and #33602 were just merged - thanks for the reviews!
l0rinc: andrewtoth rebased #31132 after the above changes and added new test cases. We also had our first Umbrel measurement: a full sync finished in 6.5 hours (31% faster than master) B-)
l0rinc: We are also making good progress reviewing #33657. Roman is very responsive, and once the remaining Windows test inconsistencies are fixed, I expect it to be ready for review (and merge) soon.
dergoegge: what are the specs on the umbrel machine?
l0rinc: dergoegge `reindex-chainstate | 923319 blocks | dbcache 450 | umbrel | x86_64 | Intel(R) N150 | 4 cores | 15Gi RAM | ext4 | SSD`
l0rinc: Rob pushed a pruned prototype of SwiftSync in #34004. My understanding is that it is meant as a first iteration, not a final implementation; high-level reviews and benchmarks would be interesting.
l0rinc: Now that we have stable access to a few benchmarking servers, we noticed some memory anomalies: a -reindex-chainstate with the default 450 MB -dbcache was running 4x slower on an rpi5 with 8 GB of memory than on an rpi5 with 16 GB of memory
l0rinc: We also observed that on the 16 GB system, runs with -dbcache values of 4 GB and higher were a lot slower than with -dbcache of 3 GB, and that an rpi5 with 16 GB of memory ran out of memory with -dbcache of 10 GB`.
l0rinc: We are still investigating these, but so far it seems: heavy memory fragmentation overhead may play a role in the premature OOMs. We are benchmarking lowering `M_ARENA_MAX` on 64-bit systems (similarly to the existing 32-bit architectures) to see if that helps in lower-memory environments (for example, an rpi4 with 2 GB of memory, which currently IBDs in about 3 weeks) a high -dbcache value may crowd out the OS page cache (blocks + UTXO set), especially when the UTXO set size approaches the memory limit, which could be the cause of the reported sudden slowdown around block 800k (more reads from disk when the files cannot be cached in memory) SSDs (and HDDs with certain filesystems) can experience severe performance degradation when they are nearly full, which, paired with more frequent disk access, dramatically slows down validation for existing 1 TB drives
There was a bit more conversation here, get the full logs in the meeting in minutes
Silent Payments WG Update (Novo)
Novo: No changes from last week. I plan to do some tests with other silent payment wallets. I will give more updates by next week
Cluster Mempool WG Update (sdaftuar, sipa)
sipa: I have split off #34023 (which contains just optimizations and follow-ups) from #32545 (which switches cluster linearization to the SFL algorithm). Reviews on the PR have focused mainly on the big algorithm description comment added in the beginning, but I hope to see code review soon. There is also the much simpler #33335 still open
Stratum v2 WG Update (sjors)
no update
Net Split WG Update (cfields)
cfields: Been cleaning up my chacha20 simd impl this week (PR incoming), so no net split update.
dergoegge: any conclusions from the discussion with aj?
cfields: Not yet. I want to get a little further along and get some other opinions. I think we just pretty fundamentally disagree about shared memory and we’ll have to pick one direction or the other...
l0rinc: cfields: were you able to generalize the simd instructions (i.e. convince the compilers via hints)?
cfields: But that decision doesn’t have to be made yet, it’s a good bit down the road.
There is more convo on this in the meeting in minutes
libevent organization members team needs update (https://github.com/libevent/libevent/issues/1812) (pinheadmz)
parked for next week
Update the release lifecycle page to reflect current practices (https://github.com/bitcoin-core/bitcoincore.org/pull/1200) (darosior)
darosior: I’d like to know if anyone else has an opinion about dropping the EOM status from our releases. We do not observe it, and as far as i can tell we never really have. This PR also updates a bunch of details about the release lifecycle and in general what users can expect from releases, to reflect current practices.
darosior: https://github.com/bitcoin-core/bitcoincore.org/pull/1200
sipa: concept ack on simiplifying things, but i’ve refrained from reviewing it because thinking about all the off-by-ones involved about these things hurts my brain
more convo in the meeting in minutes
For the full log, check the meeting in minutes
Releases
Thank you for reading. Be sure to tune in again next week for your updates on Bitcoin Core!
If there are any comments, suggestions, or errors, do not hesitate to reach out or comment
 | A guest post by
|