GNUNet File Sharing Tutorial and an Alternative to IPFS

With Web3 on the headlines all the day. I want to shift my view onto more traditional technologies. Blockchain is a great. Really, especially for transactions. Once I had to wire some money to an overseas bank. It literally took a week from sending to confirmed receiving. The fees are high too. With XMR or Eitherium. I can send Crypto to the destination in like 15 minutes. With sane fees. But I think that's where blockchain's advitange ends that, I don't think there's applications that blockchain can do better than traditional client server architectures.

Back to Web3. A decentralized web is very attractive. We can solve the most annoying part of internet in one go - dead links. As long as enough machines are hosting the content. A decentralized web will always serve up the content even if the original hoster vanished into the ether. IPFS is probably the 1st thing people think of for this. However, there's a more old-fashioned system doing the same thing. With much less Cryptocurrency and tech fluff. GNUNet[1]

GNUNet is GNU's version of overlay network. Just like Tor and I2P. Yet, unlike Tor and I2P. GNUNet rethinks and reimplements the entire 7 OSI layers. Instead of TCP/IP they have CADET, instead of OSI transport layer they have gnunet-core. etc.. Despite that, GNUNet can run over TCP/IP. However, GNUNet has a very small userbase right now. It's a large project involving may aspects. From my recent experiments, The major GNUNet functionalities, GNUNet FS and GNS (GNU Name Service) works as advertised. This post will be focusing on the former.

gnunet-fs is GNUNet's filesharing subsystem. Like IPFS, one can publish files onto the GNUNet network and someone else can retrieve them. Caching and storing the files in the process, thus in the future, once the publisher unpublishes the file, or their machine went down. The file will still be available. First, we want to install GNUNet itself. This step depends on the distro. I only have access to Ubuntu and Arch so the only commands I can share are for them:

• Ubuntu: apt install gnunet
• Arch (AUR): yay -S gnunet-git

After installing. Start GNUNet services using the gnunet-arm command. A -s flag tells it to launch subsystems. Running GNUNet is like running an IPFS node that it joins file index and assists sharnig files to other nodes. Under the hood GNUNet does a lot more encryption and ananonymization compared to IPFS. Unrelated subsystem is also started.

❯ gnunet-arm -s

Searching for files

You are ready to go if no error shows up. Let's search for LICENSE (project license) files that people may have published as tests with the gnunet-search command. The -t parameter denotes how long the file search goes on. By default it'll run forever, until you Ctrl-C out of it.

❯ gnunet-search LICENSE -t 4s
#1:
gnunet-download -o "LICENSE" gnunet://fs/chk/759AD2T0092YBD5AFXJGEY08JC1TH4HKGGFR1MTH2Q7CMWPAPEATK0JYMXDAZQWNB7RFG33ZHWE6N3ZE6JR0E55MW8CHXZ7S2E338D8.0RP3SPX1SV66X6MV2WYJF70F8K20RK9PZ5PB78ZH9FR2WPWPQVD4B8B5FFBB2S8NDW7R5YQETNP03WNCR6W6BF002DGTBBCAXZBYHX0.1952

#2:
gnunet-download -o "LICENSE" gnunet://fs/chk/EKDWDB2K3F5CB8M6JVBAKXTN1AS369KKXVEVR2650CKJQWQ3ECJRTQPNGC8EA1A2DS4XJCWE08RCS7QBTCGTW7FP3DAXWE1CW1WPNM8.ZSMGMEY8FP8MHS62DC1TH3TQ0Y55YP2SVR9PZVB6EMDG0HDASE2BHXRZXRT973X9ZCBBHW76VVJ41BMR968WC0F3XQJ1DA2GM1BBFBG.1063

You see several download commands on the console. Executing the command downloads the file from peers. If the download hangs. It's likely no peers currently have the file (GNUNet is quite small right now). In my case, the download is successful and it turns out to be the license file for OpenJS

❯ gnunet-download -o "LICENSE" gnunet://fs/chk/759AD2T0092YBD5AFXJGEY08JC1TH4HKGGFR1MTH2Q7CMWPAPEATK0JYMXDAZQWNB7RFG33ZHWE6N3ZE6JR0E55MW8CHXZ7S2E338D8.0RP3SPX1SV66X6MV2WYJF70F8K20RK9PZ5PB78ZH9FR2WPWPQVD4B8B5FFBB2S8NDW7R5YQETNP03WNCR6W6BF002DGTBBCAXZBYHX0.1952

100% [============================================================]
Downloading `LICENSE' done (1490 b/s).

❯ head LICENSE -n 3
Copyright OpenJS Foundation and other contributors <https://openjsf.org/>

Based on Underscore.js, copyright Jeremy Ashkenas,

You can also download my slides from COSCUP2022 about hacking Arch Linux onto ARM systems. I'm not sure if the download will be successful as I might be the only node with this file, thus prone to file missing.

❯ gnunet-search -t "4 s" coscup
#1:
gnunet-download -o "coscup2022-slides.pdf" gnunet://fs/chk/DPT7S0RHZ4Q0SCTCJTNB43NFEAWQNC806ZE81BZ0QN43M63VDGJRT2ESV731KNCHNY7DH39BN2S74QMSG0H5CDCJW4980Y6E7GFV8J8.QMJ145WZ01FKG6NXZFRK5ATTQ3FBR3AN5JRWJ4BD91CW9JKW18KGYQK3R6XB1HD00JHKMT41H5NHM684H82GSW097A5JZCMY891KGS0.2051732

❯ gnunet-download -o "coscup2022-slides.pdf" gnunet://fs/chk/DPT7S0RHZ4Q0SCTCJTNB43NFEAWQNC806ZE81BZ0QN43M63VDGJRT2ESV731KNCHNY7DH39BN2S74QMSG0H5CDCJW4980Y6E7GFV8J8.QMJ145WZ01FKG6NXZFRK5ATTQ3FBR3AN5JRWJ4BD91CW9JKW18KGYQK3R6XB1HD00JHKMT41H5NHM684H82GSW097A5JZCMY891KGS0.2051732
100% [============================================================]
Downloading `coscup2022-slides.pdf' done (19 KiB/s).

Publishing files

The gnunet-publish command allows users to upload files onto GNUNet. Like IPFS, shared files are chunked, encrypted, signed, and be stored on peers when applicable. GNUNet also comes with it's own DHT - R5N[2] (Which makes it funny now GNU has 2 projects with their own DHT. GNUNet's R5N and Jami's OpenDHT) to share data across nodes.

Let's say we want to upload the a Hello World text file onto GNUNet. We use the command gnunet-publish hello.txt.

❯ echo "Hello World" > hello.txt
❯ gnunet-publish hello.txt
Publishing `/home/marty/hello.txt' done.
URI is `gnunet://fs/chk/W70H5ZWGHZNW7ECB2T9TDK9NCKNFHSF6S9H9T22DKW7BN694FJPDTWQ3D7ZRJG9SFGM0EG4ZYSNYCJZ913D1FBBVH94T58KC1T08DAG.MFFDY2YMMS07Y32T2AAQHHRCAJGA8PGQCAP0EGN4PGV4AKAFQJEAP8WQ58MWFRKARB2QEYTGJ41BVK5616KMCJMAJHTC7CYPRW0XK2R.12'.

Now searching for hello.txt will find this file

❯ gnunet-search -t 4s hello.txt
#1:
gnunet-download -o "hello.txt" gnunet://fs/chk/W70H5ZWGHZNW7ECB2T9TDK9NCKNFHSF6S9H9T22DKW7BN694FJPDTWQ3D7ZRJG9SFGM0EG4ZYSNYCJZ913D1FBBVH94T58KC1T08DAG.MFFDY2YMMS07Y32T2AAQHHRCAJGA8PGQCAP0EGN4PGV4AKAFQJEAP8WQ58MWFRKARB2QEYTGJ41BVK5616KMCJMAJHTC7CYPRW0XK2R.12

But wait. We can append more flags to gnunet-publish to add more about our file. Like gnunet-publish hello.txt -k hello -n. The -k flag stands for keyword. People can search for "hello" for this file to show up. And -n makes GNUNet make a local copy of the file. So you can remove the original file once published. Let's try nother example. This time with the famous Lenna image.

❯ gnunet-publish lena_color.gif -n -k lenna -k "computer graphics"
Publishing `/home/marty/lena_color.gif' done.
URI is `gnunet://fs/chk/R8QTF12V2TYKVMESVPMP5D5GFX3HBSXRT6FNVZXDPCFN7AJG57BY45QCB318D3KVC49MTPMSDNJXE6S7Q8EQTPX3RM4AQPYH960G7E0.5XFQ8YVJ917W6GPDARJBZ2N7DN7E568YBV5JMBZ5Z3CRH5SHV51M4RG830W1CJS2GWHZT3PBNE7RTXPF5P4FW5RZT6BC77AFSGGB0VG.227335'.

❯ gnunet-search -t 4s "computer graphics"
#1:
gnunet-download -o "lena_color.gif" gnunet://fs/chk/R8QTF12V2TYKVMESVPMP5D5GFX3HBSXRT6FNVZXDPCFN7AJG57BY45QCB318D3KVC49MTPMSDNJXE6S7Q8EQTPX3RM4AQPYH960G7E0.5XFQ8YVJ917W6GPDARJBZ2N7DN7E568YBV5JMBZ5Z3CRH5SHV51M4RG830W1CJS2GWHZT3PBNE7RTXPF5P4FW5RZT6BC77AFSGGB0VG.227335

See now we can search for the keyword "computer graphics" and our image shows up.

Shutting down GNUNet

After done with GNUNet. You can (although I'd just leave it running) shutdown your GNUNet node with gnunet-arm -e. This ends all GNUNet services on you machine. You can't publish or download file now. Nor your computer will be assisting in file sharing.

IMO, GNUNet is as easy if not easier than IPFS. It's easy to setup nodes. Easy to publish and download files. I hope one day GNUNet can grow to have more users and be a viable alternative to IPFS.