DHT questions

RetroShare Development discussion

DHT questions

Postby RedCraig » Sat Feb 08, 2014 10:31 pm

Hi there,

UPDATE: I've actually been in touch with drbob who very helpfully answered my questions. I'll share the information here later today (when I get home from work :) ).

I have some questions which I was hoping someone could help me with.

I'm looking at adding some functionality to Retroshare which involves using the DHT to store information other than peer ip:ports in the value part of the key:value in the DHT. If that's not possible, then I would look at creating my own DHT which I can write the data to (if that's possible!).

I've read all of the reference material listed here:
http://bitdht.sourceforge.net/References.html
and on the retroshare wiki.
I've been looking through bitdht code, but there are some aspects of DHTs that I still don't understand.

1. Is it possible to write data to the DHT that retroshare is using, which is not peer info ip:port?

2. Retroshare leverages bittorrents mainline DHT to bootstrap itself. When connecting to peers the connecting peer gets information on all other peers in the bucket, and also gets a copy of all of the key:values in the DHT. Bittorrent DHT stores fileInfoHash:<peer info1, peer info 2...>. What does retroshare store instead of the fileInfoHash?

3. If using the bittorrent mainline DHT and Retroshare writes data to it, how can other connecting retroshare peers find the retroshare data in the DHT amongst all of the other bittorrent data?

I feel like I'm missing a few key pieces of information in how bittorrent and retroshare use DHTs, and how DHTs work in general. Any help would greatly appreciated.

Thanks,
-Craig
RedCraig
 
Posts: 2
Joined: Sat Feb 08, 2014 10:18 pm

Re: DHT questions

Postby RedCraig » Tue Feb 11, 2014 10:49 pm

Here is my current understanding. Thanks again to drbob for the patience and time answering my questions.

2. Retroshare leverages bittorrents mainline DHT to bootstrap itself. When connecting to peers the connecting peer gets information on all other peers in the bucket, and also gets a copy of all of the key:values in the DHT. Bittorrent DHT stores fileInfoHash:<peer info1, peer info 2...>. What does retroshare store instead of the fileInfoHash?

Retroshare peers generate a hash that is related to their RS ID.
e.g RsDhtHash = sha1( Rs.SSLID + RS.PGPID + "Some Random Text")
This is a one way function. So that if you know a peers details you can look them up in the DHT.
The purpose of this is for retroshare users to find other retroshare users. Given a username, you can construct the RsDhtHash of it and look them up in bittorrents DHT.
Normally bittorrent stores fileInfoHash:[(ip1:port1), (ip2:port2)]
with multiple ip:ports being the different peers which are downloading that torrent.

Retroshare stores the RsDhtHash and a single ip:port, which is that users.


1. Is it possible to write data to the DHT that retroshare is using, which is not peer info ip:port?

Not with the current implementation.
First of all, retroshare is using bittorrents DHT. It's storing an RsDhtHash with a single ip:port instead of a fileInfoHash and multiple ip ports.
Retroshares DHT and bittorrents DHT are one and the same.
Bittorrent only supports the following calls[1]:
ping
find_node
get_peers
announce_peer

To write to the value of a key:value in bittorrents DHT:
- you find the node your fileInfoHash is on
- you call get_peers(fileInfoHash) on that node
- that node records your IP, and returns a token to you along with the get_peers result
- you then call announce_peer(fileInfoHash, token) to say that you're downloading the torrent, and that the node should add you to the peers list. The node remembers the token and your IP from your get_peers request and appends your ip:port to the value part of the key:value.

The node doesn't write any specific value it's told, it will only write your ip:port.

Creating a new key:value entry in bittorrents DHT:
When you find the node that should be handling the fileInfoHash and you call get_peers(fileInfoHash), if the node has no entry for that fileInfoHash then you are the first to request that torrent.
When you follow up with announce_peer(fileInfoHash, token) the node will create a new entry fileInfoHash:[(ip:port)].


3. If using the bittorrent mainline DHT and Retroshare writes data to it, how can other connecting retroshare peers find the retroshare data in the DHT amongst all of the other bittorrent data?

By using the RsDhtHash I mentioned for point 2.
You call find_node(RsDhtHash) until you find the node which is handling that RsDhtHash.One of the system properties of a DHT[2] is distance based hash lookups. If you have a hash, the nearest node to you can direct you to a node that it knows is 'closer' to the node you want. After a few hops you will find the node you want.

Bittorrent nodes think the RsDhtHash is a fileInfoHash, they don't mind. If anything, this is giving the bittorrent DHT more nodes which will keep a copy of the routing table for the keyspace of the DHT bucket they're in. More nodes means less volatile DHT, which is a good thing.

[1] http://www.bittorrent.org/beps/bep_0005.html
[2] http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf
^ The references are from the bitdht reference list: http://bitdht.sourceforge.net/References.html
RedCraig
 
Posts: 2
Joined: Sat Feb 08, 2014 10:18 pm


Return to Developers Corner

Who is online

Users browsing this forum: No registered users and 1 guest

cron