This is the mail archive of the
cygwin
mailing list for the Cygwin project.
Re: Strange errors running gcc tests on Cygwin
First of all, thank you for your response!
On 03/08/2017 02:21 AM, Brian Inglis wrote:
After any Windows Update, or a lot of package installs, you may want
look at running
rebase-trigger full[rebase]
before rebooting to remove all Cygwin and Windows processes, then
(with no Cygwin services or processes running) run Cygwin
setup-x86{,_64} to rebase everything and maximize memory available
to Cygwin processes.
So does rebase organize the Cygwin dlls and reassign them all new base
loading addresses? So far I've only done the 64-bit tests using Cygwin.
I haven't even gotten to 64-bit mingw or 32 bit. Also, I have *not* done
a Windows Update since I have installed. Could this be an OS issue that
was addressed in some patch? I'm running Windows 7 Ultimate SP1.
This could be critical if you are doing any builds under Cygwin 32
and have a lot of packages and/or large exes/dlls installed.
If you are running a lot of Cygwin services, cron or Scheduled Tasks,
and/or background processes, you may want to look at running cygserver
to cache process info and common system info (including SAM/AD).
I'm only running sshd -- no cron or "at" jobs (except whatever Windows
installs by its self). However, gcc's make check spawns a LOT of processes.
Setup cygserver initially by running cygserver-config with
elevated/admin rights, and start it at system startup by running
cygrunsrv with elevated/admin rights.
If you ever expect to be running more than 62 simultaneous Cygwin
processes on a system, bump kern.srv.process_cache_size in
/etc/cygserver.conf created by cygserver-config to one of the higher
values recommended in the man page.
I found this seemed to reduce process startup overhead and eliminated
random broken pipes, hung, and failed processes due to a lot of shell
forking.
I didn't know about cygserver, so I tried it out and set the
kern.srv.process_cache_size to 256. I ran a make -kj4 check and had
broken pipes within a few short minutes. I tried it again running make
-k check, but on two directories at once and got broken pipes somewhere
along the way (before they completed). So I should probably try it with
only a single make running at once.
Even with just the two instances of make (with one job each), there are
many processes of bash, make, expect, etc. That's just one of the
challenges of a Cygwin venture -- Windows was never designed for light
process creation (or thread creation for that matter). A lot of the
*nix software concepts are hinged upon that lightweight, copy-on-write
fork, whereas Windows-based concepts seem to be centered more around
pre-spawning and maintaining thread pools and task-specific services.
Preallocate contiguous paging space at double RAM to reduce the chance
that any set of Windows processes will reduce Windows free memory too
low for an instant, and cause something to hang or fail, by giving
Windows somewhere to page out a lot of LRU pages from inactive
processes.
I'm not sure how to go about this. Should I bring the VM down, mount
the image with ntfs-g3 and dd if=/dev/zero it or is there another way?
Also, I don't know what Windows expects to see in a pagefile. Or maybe
defrag. I'll look at it.
I found that if Windows (at least up to W7) free memory ever got too
low, especially when all processors are pegged, it seemed unable to
dynamically allocate and use more paging space to alleviate the
crunch.
From watching the system monitor, I can see that I never get above 35%
memory usage. I've assigned 4GiB of ram to the VM and Windows never
seems to use more than 280MiB for page cache (I guess that's due to the
license-specific limitation) -- at least there is abundant caching
happening at the host level (3-ish GiB).
The situation may have improved on Windows 10, but I've already
allocated the paging space, and uninstalled some (probably buggy)
hogs that seemed to only ever allocate memory and never free any.
YMMV
As far as the dll search path issue, I *think* I have a woorking
patchset in gcc and DejaGnu to resolve the dll load problems. (I
haven't yet done the on-off switch yet, so it's basically hard-coded. I
found a Tcl "is_Cygwin" procedure in git-gui that I'm going to borrow
for now.) So while my make check is still failing with broken pipes, I
appear to have resolved the issue with the dll search path.
It would probably help if I can figure out where the broken pipe
actually occurs and try to get that piece to log more verbosly. I'm
guessing it's from expect. This is the exact error message:
FAIL: gcc.c-torture/execute/20000822-1.c -O2 -flto -fuse-linker-plugin
-fno-fat-lto-objects execution test
parent: sync byte write: broken pipe^M
make[3]: Leaving directory
'/d/builds/head-test-moutline-x86_64-pc-cygwin/gcc'
Don't worry about the FAILed test. The main thing is that after each
broken pipe message, I see make "leaving" that directory, so it doesn't
finish the rest of the tests in that directory.
Daniel
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple