This is the mail archive of the
ecos-discuss@sources.redhat.com
mailing list for the eCos project.
Re: Problem with IP Stack ...
- From: Thomas BINDER <Thomas dot Binder at frequentis dot com>
- To: Andrew Lunn <andrew dot lunn at ascom dot ch>
- Cc: ecos-discuss at sources dot redhat dot com, Geoff Patch <grp at cea dot com dot au>
- Date: Mon, 09 Sep 2002 12:59:26 +0200
- Subject: Re: [ECOS] Problem with IP Stack ...
- Organization: Frequentis
- References: <3D788653.B1C6FCC0@frequentis.com> <20020906112124.GO8516@biferten.ma.tech.ascom.ch>
Hi Andrew!
Andrew Lunn wrote:
>
> On Fri, Sep 06, 2002 at 12:41:23PM +0200, Thomas BINDER wrote:
>
> Quite old. There was a problem in the transmit direction fixed
> recently. Could you try this again with an up-to-date version.
I had applied the scatter-gatter patches in my version, it just didn't change
anything. I also tried the latest snapshot (this morning) of ECOS, still no
success.
There is one interesting thing, however, I discovered just this morning. I
redirected the diag_printf messages to a different (debug) stream (not IP based)
and ECOS does not crash any longer! I now can read several "out of MBUFs"
messages (as opposed to reading only zero or one, right before the crash). It
seems to me that the crash is somehow related to debugging messages that are
sent over a network channel. Disabling the printing of the "out of MBUFs" error
message (CYGPKG_IO_ETH_DRIVERS_WARN_NO_MBUFS) does not help in that context.
With the diag_printf redirection in place I also see that the IP stack does not
recover MBUFs (at least not within a few minutes) although it is still answering
pings. It seems that MBUFs are never recovered not even in regular operation. At
least that is what the statistics tell. Could the failed recovery be the real
cause of the crash?
Attached you'll find a log file that illustrate the error messages and also the
IP-stack statistics. Note that some of the log messages are missing/crippled
because our debug stream is quite slow.
>
> Also, please enable asserts in the infra package. Its quite likely an
> assert is happening. What/where will help track down the problem.
Didn't see any assertions.
>
> The other thing that should happen is ICMP_SQUELCH messages should be
> sent from eCos back to your linux host. The linux host should then
> slow down the TX of UDP packets. Is this happening? Run tcpdump and
> filter for ICMP.
No such messages either.
Tom
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 0, clusters 2, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 65280, blocksize 128
Clust pool: total 131072, free 124928, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61872, max free block 61252
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 43, free clusters 7
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 40960, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 45, free clusters 9
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 36864, blocksize 2048
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 10 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_rUFs
warning: eth_recv out of MBUFs
out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of
Failed to get 0 times
Waited to get 0 times
Drained queues to get 100 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 55808, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
: eth_recv out of MBUFs
wafe sign timeout expired
BUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_rearning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
warning: eth_recv out of MBUFs
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Falocksize 128
Clust pool: total 131072, free 0, blocksize 2048
max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
*ELAN BUFFER OVERFLOW*
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
Network stack mbuf stats:
mbufs 72, clusters 63, free clusters 27
Failed to get 0 times
Waited to get 0 times
Drained queues to get 139 times
Misc mpool: total 65520, free 61648, max free block 61028
Mbufs pool: total 65408, free 56064, blocksize 128
Clust pool: total 131072, free 0, blocksize 2048
--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss