Understanding Torrents with Web Seeds

A web seed is a URL that is located inside the torrent file that points to a web server hosting the file(s).

A web seed helps kick-start the seeding process when the torrent is young. When only a few seeds (and even none) are available, the torrent client will connect to those web seeds and download the file using the http protocol.

At first it can be taxing on the web servers as many request will go directly to them, but that the stress on the server would be the same than a traditional browser/wget/curl download.

As more people download (and seed) the torrent's content, less an impact it will make on the web server(s). At one point, there will be enough seeders and more traffic will go to the traditional BitTorrent/P2P protocol.

Not all torrent clients support web seeds. This is the case for rtorrent a popular command line torrent client (there might be others, rtorrent is just an example. I'm not throwing rock at the team, it's my default torrent client when I need to download something on a remote computer).

It requires the use of an unsecured http connection (plain HTTP). Although this is what you would have anyway by using traditional torrents.

It can be quite taxing on the servers as more bandwidth will be taken from them.

A torrent with multiple files will most likely not work. (See next section)

To better explain this problem, let's suppose that we have two files called PRODUCT and CHECKSUM. PRODUCT is a 1.2 GB file and CHECKSUM is a small .sha256 file containing ... the sha256 checksum of PRODUCT.

As mentioned in the introduction to torrents (coming Soon), the files are divided into small chunk that are sent by different peers. Now lets suppose that PRODUCT is filling all its chunk except the last one where a remaining of 200kb. When using web seed, you can't combine the CHECKSUM into that chunk as it will produce a PRODUCT-CHECKSUM file (which doesn't exist on the web servers).

To resolve, this issue, a "padding" (pad) file is created. This pad file only contains 0 (zeros) thus creating an empty file. In our example, the pad file would be 200kb. CHECKSUM can now fit in its own chunk(s) and since no file comes after CHECKSUM, no pad file is necessary.

And here is the problem! Padding files.

When using web seeds, the padding files are not present on the web servers, so they can't be downloaded. Resulting in having to use the traditional BitTorrent protocol making web seeds useless.

As of December 14th, 2020, I'm not sure how to resolve this issue, there seems to be a fix implemented inside libtorrent version 2.0.1 but I was not able to make it work on Manjaro nor Solus.

If you have any feedback or ideas how I could fix this issue, give me a shout on Twitter .

Torrents with web seeds, when used with a single file, seems to have great benefits for the users. The users are guaranteed to get the content if it still exist (even without any torrent peers). The content can be downloaded even faster than the traditional browser download (web servers + torrent peers). The security is the same than a traditional torrent.

There doesn't seem to have any major down side for the user. For the projects, this means their servers will be more solicited, but it would be the same impact than having a direct download.