Memory leak ?

Old SourceForge Discussion forum that is now archived. Please use one of the other forums.

Memory leak ?

Postby CsoL » Mon Oct 20, 2008 11:55 am

Hi,

Since version 753 (the one with the DHT working) I've got a memory leak in RS which touches both the GUI and no-GUI versions: RS takes more and more memory until it crashes within 1 min.

I remember having this same bug already 3 months ago, then it disappeared. Does anybody remember how to correct this ?

Using valgrind I don't see much leakage (11MB at most). So maybe this is a thread problem, which means that RS spews a lot of threads which do not terminate.

This happens on linux 64 bits (ubuntu hardy) only, not on 32 bits.

Thanks for any help
Cyril
CsoL
 

RE: Memory leak ?

Postby Dr Bob » Mon Oct 20, 2008 7:49 pm

Hi Cyril,

Its likely to be a thread issue: the DHT launches a thread per request.
If these aren't cleaned up then there is memory bloat, etc.

The code is in: libretroshare/src/dht/opendhtmgr.cc

You'll notice the pthreads_detach() line I added for 32bit linux thread cleanup.
You might need to do something fancier for 64bit version - but I'd be surprised..

DrBob.
Dr Bob
 

RE: Memory leak ?

Postby mahendra » Mon Oct 20, 2008 9:09 pm

Hello DrBob,

While compiling following code in retromessenger.

p3Peers* m_rsPeers;
m_rsPeers->getFriendList( peers );

we are also getting segmentation fault(and probable reason is memory leak, i m using ubuntu on 64 bit machine.

and i went through mentioned added code for 32 bit in at The code is in: libretroshare/src/dht/opendhtmgr.cc

bool OpenDHTMgr::publishDHT(std::string key, std::string value, uint32_t ttl)

May be a silly guess....if integer type defined uint32_t ttl is the required to be change to 64 bit?....not sure...may be some other reason and modification required?

Any input or suggestion about it?

Thanks & Regards,
Mahendra

may be some other reasons are therneed to cross check with it
mahendra
 

RE: Memory leak ?

Postby CsoL » Tue Oct 21, 2008 3:09 pm

Hi,

I passed some hours looking for this memory leak.

First I tried to enable/disable some function calls in the dht server as you indicated me, but I got really incoherent results like no leaks when suppressing the call to resultDHT, but leaks when commenting everything inside resultDHT (yeah, I know, this is freaky). I finally got this from valgrind:

==9008== 524,423,288 bytes in 1,887,694 blocks are possibly lost in loss record 130 of 130
==9008== at 0x4C23809: operator new(unsigned long) (vg_replace_malloc.c:230)
==9008== by 0x54FF2E0: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/libstdc++.so.6.0.9)
==9008== by 0x55000F4: (within /usr/lib/libstdc++.so.6.0.9)
==9008== by 0x5500271: std::string::string(char const*, std::allocator const&) (in /usr/lib/libstdc++.so.6.0.9)
==9008== by 0x4FB73F: __static_initialization_and_destruction_0(int, int) (rstypes.h:162)
==9008== by 0x4FB8A2: _GLOBAL__I__ZN14FileHashSearch15searchLocalHashESsRSsRm (hashsearch.cc:50)
==9008== by 0x595D39: (within /media/disc2/csoler/RetroShare/retroshare-dev/retroshare-svn/libretroshare/src/rsiface/retroshare-nogui)
==9008== by 0x405A0A: (within /media/disc2/csoler/RetroShare/retroshare-dev/retroshare-svn/libretroshare/src/rsiface/retroshare-nogui)
==9008==

Here's another one:

==14841== 68,098,244 bytes in 263,700 blocks are possibly lost in loss record 129 of 129
==14841== at 0x4C23809: operator new(unsigned long) (vg_replace_malloc.c:230)
==14841== by 0x54FF2E0: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/libstdc++.so.6.0.9)
==14841== by 0x55000F4: (within /usr/lib/libstdc++.so.6.0.9)
==14841== by 0x5500271: std::string::string(char const*, std::allocator const&) (in /usr/lib/libstdc++.so.6.0.9)
==14841== by 0x4F5D3F: __static_initialization_and_destruction_0(int, int) (opendht.cc:37)
==14841== by 0x4F5DF0: _GLOBAL__I__ZN13OpenDHTClient15checkServerFileESs (opendht.cc:808)
==14841== by 0x593C79: (within /media/disc2/csoler/RetroShare/retroshare-dev/retroshare-svn/libretroshare/src/rsiface/retroshare-nogui)
==14841== by 0x405A0A: (within /media/disc2/csoler/RetroShare/retroshare-dev/retroshare-svn/libretroshare/src/rsiface/retroshare-nogui)

after running for a few minutes.

Why is the static initialization called so many times ? My theory is the following:

Each time a thread is spawn, static intializations are called for global variables of the thread. The problem is that in std::string, there is a leak and the memory is not completely released. As you spawn many threads, the leaking memory accumulates. Normaly, the pthread_detached should do the trick, but it seems it does not.

- should we switch to some other string implementation ? That's a huge modification
- perhaps passing strings as & more often would help a lot.
- most static string initializations (such as opendht.cc line 37) can be replaced by defines with quotes. I think this would be a workaround.

What do you think DrBob ?
CsoL
 

RE: Memory leak ?

Postby Dr Bob » Tue Oct 21, 2008 9:12 pm

The Code works fine on. Mac OSX, Win XP, Linux 32bit... so something 64 bit specific is happening.

Lets not jump to wanting to switch away from strings. They are just an indicator of the problem - but unlikely to be the root cause. You're going to have to continue digging.

DrBob.
Dr Bob
 

RE: Memory leak ?

Postby Dr Bob » Tue Oct 21, 2008 9:23 pm

It shouldn't be too hard to write a single threaded DHT client....
All you have to do is replace the multiple threads with a mutex protected list,
and service each request by turn.

It might be a good stop-gap soln - until we find out why the threads don't die nicely.
Alternatively you can switch DHT off in the control panel.

DrBob.
Dr Bob
 

RE: Memory leak ?

Postby CsoL » Wed Oct 22, 2008 7:52 am

I definitely think this is a ubuntu 64 implementation problem of the std. Because, changing

const std::string foo("bar") ;

into

#define foo "bar"

suppresses the corresponding leak. Others however do appear at similar places.

I do think that switching from std::string to something else *is not* a good idea.

Alternatively, there may be a way to force static declarations to be shared with the parent thread (which makes sense!) and it is possible this is not the default on ubuntu 64. I'm looking into this now.

CsoL
CsoL
 


Return to Developers Corner - Archive

Who is online

Users browsing this forum: No registered users and 1 guest

cron