GNU make losing jobserver tokens
Magnus Ihse Bursie
Mon Mar 21 14:28:17 GMT 2022
I'm working for Oracle on the OpenJDK build team. We're using GNU make
to build the JDK on all supported platforms. For Windows, we use Cygwin
as our build environment, including the Cygwin version of GNU make.
We have had a long-standing issue with make losing jobserver tokens.
("long-standing" here means for years, and years, at least since GNU
make 4.0, up to and including the current latest version in Cygwin.)
Most runs end with something like:
make: INTERNAL: Exiting with 11 jobserver tokens available; should be
Since the build still succeeds, and it just affects performance (and
typically not that much), we have not spend too much time getting to the
bottom of this.
Now, however, I've come across a machine where this happens repeatedly,
and on a much worse scale:
make: INTERNAL: Exiting with 1 jobserver tokens available; should be 24!
This effectively turns the highly parallelized builds into
single-threaded builds, and is absolutely detrimental for performance.
On the flip side, this also makes for the perfect testing environment to
really get to the bottom of this issue.
I started out by sending a question to email@example.com. The folks over
there reported that this was not a known problem with GNU make on
Windows in general, and that as far as they knew, the mingw port did not
suffer from this problem.
Instead, they suggested that it was a Cygwin-specific problem, possibly
related to issues with emulating Posix pipes and/or signals in Cygwin.
So, my first question is: Is this a known problem in Cygwin GNU make?
Are there any workarounds/fixes to get around it?
Otherwise: Any suggestions on how to go on and debug this? I am willing
to build and test an instrumented debug build of make, but I will need
assistance to find my way around the source and spot likely candidates
for the source of the problem.
More information about the Cygwin