Unexpected complexity writing an async Spartan protocol server

Spartan[1] is a cool smol internet protocol. It's even simpler than Gemini. There's no TLS, no need for URL parsing and even less status codes! I just wrote my own Spartan client and server - Spartoi[2] in an afternoon, for fun. What I'm not expecting is some parts of it being (slightly) more complicated than Gemini. Maybe my server deisgn was not what the spartan creator expected.

Also, I'm not saying Spartan is a bad protocol. It does it's job of being really simple. Just I'm not expecting the server being more complicated than Gemini.

Spartan requests are composed of a host, path, content size and content itself. A trivial request would look like the following. Like HTTP (not HTTPS) the host is directly transmitted in the TCP stream. The path is a string of characters. Then the content size is a number. 0 means no content.

example.com /file.gmi 0

And a request with content would look like this:

example.com /forum/post_new 12
Hello World!

This enables Spartan to support uploading files. (Unlike Gemini, which files has to be base64 encoded and squeezed into the URL. Which has a 1024 byte limit.)

example.com /upload/ubuntu.iso <file_size>
<file_content>

Like my Gemini library, dremini[3]. Spartoi is based on trantor[4]. Which forces everything to be written in an asynchronous fashion. In Gemini, the entire request is just a URL followed by CRLF. I could just scan for CRLF in the TCP stream receive callback. And the entire parser would be stateless.

server_.setRecvMessageCallback([&](const TcpConnectionPtr& conn, MsgBuffer* buf) {
    auto crlf = buf->findCRLF();
    if(crlf != nullptr)
        process_request(conn, buf->peek(), crlf);
});

Legit that's it. But with Spartan. I need to store the expected body size and and a half-generated request in the per-connection context. And handle the edgecase of everything being sent in one TCP packet.

server_.setRecvMessageCallback([&](const TcpConnectionPtr& conn, MsgBuffer* buf) {
    auto state = conn->getContext<SpartanState>();
    if(state.request != nullptr &&
        buf->readableBytes() >= state.request->content_size) {
        ... // Handle the request
        process_request(conn, req);
    }


    auto crlf = buf->findCRLF();
    buf->retrieve(std::distance(buf->peek(), crlf));
    // since there's a chace we see a small MTU, thus the request
    //  line may be split into multiple packets
    if(crlf == nullptr)
        return;

    auto [req, content_length] = parseSpartanRequest(buf->peek(), crlf);
    ...

    conn->setContext(SpartanState{req, content_length});

    // Everything is transmitted in a single packet, we won't receive any
    // more data. We are forced to handle the request now.
    if(buf->readableBytes() >= content_length)
        process_request(conn, req);
});

Spartan's design makes perfect sense in a blocking environment. But when your framework forces you to be asynchronous. Spartan can get more complicated. Funny a smol net protocol reminded me how writing asynchronous code can make your problem at hand get exponentially harder.

Author's profile
Martin Chang
Systems software, HPC, GPGPU and AI. I mostly write stupid C++ code. Sometimes does AI research. Chronic VRChat addict

I run TLGS, a major search engine on Gemini. Used by Buran by default.


  • marty1885 \at protonmail.com
  • GPG: 76D1 193D 93E9 6444
  • Jami: a72b62ac04a958ca57739247aa1ed4fe0d11d2df