GNUnet++ Jenuary update - multi-threading primitives and toy projects
GNUnet++
It's mostly bugfixes and extending GNUnet++ to handel multithreaded enviroments. Last month I started working on peerinfo support. But ended up focusing on other stuff.
Multi-threading
GNUnet in of itself is single threaded and runs on it's own event loop. But to support more complex applications, GNUnet++ needs to support multi-threading. To be particular, GNUnet++ needs to allow other threads to queue tasks onto the GNUnet thread. Unfortunately, calling GNUNET_SCHEDULER_add_now
from a different thread don't just work. The reason is that the scheduler calls poll()
to wait for events. But adding a task does not trigger poll()
to wake up. This isn't a problem under the single threaded enviroment, because only thread that adds task is the same thread that calls poll()
. Thus it only needs to check if there's more task to run before calling poll
.
The solution is also quite simple. While initializing, GNUnet++ will create a pipe. The read end of the pipe will be added to the scheduler. Whenever a task is added, GNUnet++ will write a byte to the pipe. This will trigger poll()
to wake up and check if there's more task to run. I blieve this is the same technique libgnunet-workers
uses.
Ideally we would move GNUnet to use epoll
instead of poll
. But that's a much bigger change. And since GNUnet is a microservice architecture (connections gets amortized by design), it probably won't be as big as a problem as it is for web servers.
Unit tests
I finally got around to writing unit tests for GNUnet++. Mostly out of horror of discovering the piles of bugs I have during play. I'm using Drogon's testing framework for the job as it's the only one I know of supports testing async C++ (and I wrote it). It's mostly testing core functionality like crypto and scheduler. I'll add more as I go.
One fun thing to know. C++ concepts are quite more vercitile then I thought! For example, I want to test if a type supports the UFO operator <=>
. This would be annoying to do in C++17 using TMP. But in C++20, I can just use concepts.
template <typename T>
concept SupportsUFO = requires(T v) { a <=> b };
static_assert(SupportsUFO<MyType>);
To check if a type supports all comparsion operators, I can just spam concepts together.
template <typename T>
concept SupportsAllComparsion = requires(T v) {
{ a <=> b };
{ a < b } -> std::same_as<bool>;
{ a > b } -> std::same_as<bool>;
{ a <= b } -> std::same_as<bool>;
{ a >= b } -> std::same_as<bool>;
{ a == b } -> std::same_as<bool>;
{ a != b } -> std::same_as<bool>;
};
static_assert(SupportsAllComparsion<MyType>);
Which would have been a pain to do in older C++.
Misc
Started getting Peerinfo and Peerstore support. But it's in a very early stage. I also started looking at wrapping signing and encryption API. But that part is complicated and needs more thought.
Kanine - expose TCP services over GNUnet
Haven't upload this yet. Kanine (pernounce: Can-9) is a fun project I work on in my spare time. It's a bridge from bare TCP to CADET. In the process I also learned some limitations of CADET. Namely
- CADET does not distubgush between port not open/host not exist
- Unlike TCP, closing a CADET connection stops all pending sends
- CADET does not support half-closed connections
Which is mostly fine, just more states I need to handle in Kanine to properly simulate TCP. But note worthy.

Martin Chang
Systems software, HPC, GPGPU and AI. I mostly write stupid C++ code. Sometimes does AI research. Chronic VRChat addict
I run TLGS, a major search engine on Gemini. Used by Buran by default.
- marty1885 \at protonmail.com
- Matrix: @clehaxze:matrix.clehaxze.tw
- Jami: a72b62ac04a958ca57739247aa1ed4fe0d11d2df