Member of The Internet Defense League Últimos cambios
Últimos Cambios
Blog personal: El hilo del laberinto Geocaching

[BitTorrent] About multiple torrents in a single listening port

Última Actualización: 08 de Julio de 2004 - Jueves

The standard and official BitTorrent client is pretty "lean" in features. In particular, every torrent you share will use its own TCP port.

As I said in the BitTorrent mailing list in 26 Jan 2004:

Hi, Bram.

Do you have any idea/project to allow listening to several incomming connections/torrents through a single listening TCP port?.

The current status, a individual port for each listening torrent, is a nightmare in NAT environments, bandwidth monitoring/management, firewalls...

I'd say this should be a development priority. I can volunteer for the job, if you wish and nobody is working already on the issue.

I'm looking for your comments. Thanks.

My email got (almost) no useful answers, so I tried again in 13 Apr 2004:

Bram Cohen wrote:
> Single port for multiple torrents is supported in the protocol but not
> yet implemented in clients. That will probably be in the next release.

I'm doing a patch for 3.4.2 just now. Would you be interested?. Any time estimation for your version?.

My idea is the following:

  1. A separate and unique thread waiting for incomming connections.

  2. When a connection arrives, parse the X first bytes, trying to determine the download_id.

  3. If the download_id corresponds to a valid id (we are publishing the torrent), put the new socket in the queue private for the appropiate downloading thread (each download has its own thread) and awake the thread via a private "signaling" socket.

  4. When a download thread get traffic in the "signaling" socket (the thread "polls" it like any other socket), takes the socket from the private queue and make the usual things: check the peer id, protocol, download id, etc. Proceed as usual.

  5. When a new thread starts a download, instead of creating a listening socket and binding it to a free port, it create a listening socket request (ip,port), create a private "signaling socket", and put them in the private "listen thread" queue. It put also the peer_id, the download_id and a private queue.

  6. The "listen thread" awakes every second and check if there any data in its private queue. If it sees data, take it. If there exists already a listening socket for the requested (ip, port), add the "signaling socket", peer_id, download_id to the "subscription" for that socket. If the listening socket doesn't exists (first download thread, for example), create it.

The one second awaking is cheap since the CPU eater would be "poll" and this particular socket ONLY "poll" a socket per listening port. Tipically, ONE.

Problems in my scheme:

  • I'm trying to do minimal changes to current BT 3.4.2, in order to ease port to future BT releases, so the code is ugly. Refactoring the bittorrent core to better support "library" behaviour would be more profitable (and recommendable). But this is an issue that only Bram or other "cvs commit" developer can acomplish easily. I'd be glad to help. In fact I have a refactored 3.3 tracker very modular, "clean" and easily extensible, along the lines proposed in http://groups.yahoo.com/group/BitTorrent/message/4361. I'd be very happy if these changes could be merged with the official BT. Any coder interested?

  • Currently, the listening socket would block when a new connection arrives, waiting for the BT header. So you could trivially DoS the system. It can be easily resolved, but my main priority now is having the system up and running. The issue is in my "todo".

In my environment, I have three thousand torrents in the system, but only a handful of them could be active in a given time. This proposal eats the very same CPU and memory (and more or less the same number of sockets) that standard BT, but listen in a SINGLE socket.

Supporting so many threads can be a bit stressfull to the OS, although most of them are idle. But the problem is just the same than current BT. A better approach could be to rewrite BT to support multiple downloads using a single thread.

As I already said, the code is not nice. I'd rather prefer to refactor the BT code to better support the "core library" concept. This way extending the tracker or improving the clients should be trivial. I firmy believe BT would be a far better program and, more important, a better "building block" for reuse.

If a "CVS commiter" could peer with me, I'd be glad to help :-)

Bram comments that he finds thread programming confusing and error prone (in particular, deadlock risk). But my patch is deadlock free by design.

My final patch is not implemented as described, but the approach is fairly similar. I've been using this modified client since April 2004 without any problem at all.

This patch is distributed AS IS (if it erases your harddisk is your fault, not mine ;-) ) using the original BitTorrent MIT License. I haven't removed the original code, so you can patch your client and enable/disable the new code at will.

Currently the patch implements the single listening port in "btlaunchmany.py" and "btlaunchmanycurses.py". You can add or remove torrents "on the fly", like the original client. If you wish to use this improvement in your our code/client, just do the following in the beginning of your source: 

from BitTorrent import RawServer
RawServer.listen_thread=RawServer._listen_thread()

This tiny change will activate my code.

Historia

  • 08/Oct/04: Documento bugs y enlace la página con la solución.

  • 08/Jul/04: Primera versión de esta página.

Python Zope ©2004 jcea@jcea.es

Más información sobre los OpenBadges

Donación BitCoin: 19niBN42ac2pqDQFx6GJZxry2JQSFvwAfS