GNUnet++ Jenuary update - multi-threading primitives and toy projects

GNUnet++

It's mostly bugfixes and extending GNUnet++ to handel multithreaded enviroments. Last month I started working on peerinfo support. But ended up focusing on other stuff.

Multi-threading

GNUnet in of itself is single threaded and runs on it's own event loop. But to support more complex applications, GNUnet++ needs to support multi-threading. To be particular, GNUnet++ needs to allow other threads to queue tasks onto the GNUnet thread. Unfortunately, calling GNUNET_SCHEDULER_add_now from a different thread don't just work. The reason is that the scheduler calls poll() to wait for events. But adding a task does not trigger poll() to wake up. This isn't a problem under the single threaded enviroment, because only thread that adds task is the same thread that calls poll(). Thus it only needs to check if there's more task to run before calling poll.

The solution is also quite simple. While initializing, GNUnet++ will create a pipe. The read end of the pipe will be added to the scheduler. Whenever a task is added, GNUnet++ will write a byte to the pipe. This will trigger poll() to wake up and check if there's more task to run. I blieve this is the same technique libgnunet-workers uses.

Ideally we would move GNUnet to use epoll instead of poll. But that's a much bigger change. And since GNUnet is a microservice architecture (connections gets amortized by design), it probably won't be as big as a problem as it is for web servers.

Unit tests

I finally got around to writing unit tests for GNUnet++. Mostly out of horror of discovering the piles of bugs I have during play. I'm using Drogon's testing framework for the job as it's the only one I know of supports testing async C++ (and I wrote it). It's mostly testing core functionality like crypto and scheduler. I'll add more as I go.

One fun thing to know. C++ concepts are quite more vercitile then I thought! For example, I want to test if a type supports the UFO operator <=>. This would be annoying to do in C++17 using TMP. But in C++20, I can just use concepts.

template <typename T>
concept SupportsUFO = requires(T v) { a <=> b };

static_assert(SupportsUFO<MyType>);

To check if a type supports all comparsion operators, I can just spam concepts together.

template <typename T>
concept SupportsAllComparsion = requires(T v) {
    { a <=> b };
    { a < b } -> std::same_as<bool>;
    { a > b } -> std::same_as<bool>;
    { a <= b } -> std::same_as<bool>;
    { a >= b } -> std::same_as<bool>;
    { a == b } -> std::same_as<bool>;
    { a != b } -> std::same_as<bool>;
};

static_assert(SupportsAllComparsion<MyType>);

Which would have been a pain to do in older C++.

Misc

Started getting Peerinfo and Peerstore support. But it's in a very early stage. I also started looking at wrapping signing and encryption API. But that part is complicated and needs more thought.

Kanine - expose TCP services over GNUnet

Haven't upload this yet. Kanine (pernounce: Can-9) is a fun project I work on in my spare time. It's a bridge from bare TCP to CADET. In the process I also learned some limitations of CADET. Namely

• CADET does not distubgush between port not open/host not exist
• Unlike TCP, closing a CADET connection stops all pending sends
• CADET does not support half-closed connections

Which is mostly fine, just more states I need to handle in Kanine to properly simulate TCP. But note worthy.