pthreads leaks handles and threads when threads use sockets

[Reini Urban]

Mark Pizzolato schrieb:
>> Reini Urban wrote:
>> Mark Pizzolato schrieb:
>> > I've been using clamav's clamd under cygwin and noticed that over time
>> > the handle count as viewed with TaskManager seems to grow to arbitrary
>> > values. I used clamd's option IdleTimeout set to 600 seconds which
>> > dramatically reduced the growth rate of the Handle Count. Of course
>> > clamd has many things going on that could contribute to handle leakags,
>> > so I tried to write a simple program to demonstrate the problem.
>>
>> Thanks a lot! Maybe we should restart the two daemons daily or weekly?
>>
>> I will change the default IdleTimeout to 600 secs with the upcoming
>> clamav-0.81 release. Which fixes the freshclam proxy problem and some
>> OLE issues.
> 
> Merely setting IdleTimeout to 600 is currently insufficient due to a bug 
> which only uses the IdleTimeout parameter for the Initial value used.  
> After the AV Database is reloaded, the idle timeout is hardset to 30 
> seconds.  The attached patch (to 0.80 or 0.81) fixes this issue.  This 
> patch has been submitted on the clamav-devel list.

Yes, I saw it and already patched my 0.81.
But I still have to find some time reproduce your problem.

> The right choice for the IdleTimeout is a value which is larger than the 
> largest gap between messages that arrive on your system.  This is 
> somewhat complicated by multiple connections arriving concurrently which 
> is handled by MaxThreads.  MaxThreads has a default value of 10.  This 
> would be fine for most systems, however libclamav uses tmpfile() 
> internally which is NOT threadsafe (using newlib's tmpfile()) for any 
> system which returns the same value for getpid() for each thread in a 
> process (i.e. it works fine on Linux since getpid() on Linux returns a 
> unique value for all threads on the system).  I've submitted changes 
> which address this to the clamav folks(avoiding tempfile()), but they 
> have not been accepted as yet.  To avoid this issue (and avoid clamd 
> producing "ERROR: ScanStream: Can't create temporary file." messages), 
> setting MaxThreads to 1 should work, but will probably affect the 
> behavior of client software that talks to it (possibly causing deadlocks).
> 
> Do you have any insight to help address the underlying socket issues in 
> threaded programs would clearly help with clamd and every other multi 
> threaded program which may not even know it has these issues.

Unfortunately not, but Corinna had.
clamav might not accept the tempfile() workaround when it's clearly 
cygwin's/newlib fault.

But they still have to fix (think about) their SESSION/SCAN socket 
protocol as you wrote there. This was really a great catch. CRLF would 
be easy to add.
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/