Calling GetConsoleProcessList in tight loop allocates new buffer via condrv ioctl results in excessive page-faults with Windows Terminal

[Steven Buehler]

On 12/3/2024 3:42 AM, Takashi Yano wrote:
> On Mon, 2 Dec 2024 19:57:25 +0000
> Steven Buehler wrote:
>> I am experiencing an abnormal number of page-faults per second (averaging 800 to over 2000) when using Cygwin within the Windows Terminal app. This produces visible stutters and cursor movement during terminal screen redraws.
>>
>> I have opened an issue on the Windows Terminal GitHub issues page. The initial investigation by one of the Windows Terminal developers has determined that "Cygwin is calling console APIs in its steady state. It looks like it's calling GetConsoleProcessList in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface. I don't think there's anything we can do about that, other than stopping them from doing so."
>>
>> Following this response, I am attempting to bring this issue to the Cygwin developer team's awareness for a possible resolution. Please see https://github.com/microsoft/terminal/issues/18264 for a detailed discussion and accompanying video demonstration of the page-fault counter.
> 
> Thanks for the report.
> However, I cannot reproduce your problem.
> Which cygwin version do you use?
> 

I am using Cygwin version 3.5.4.

FWIW, the GitHub issue at 
https://github.com/microsoft/terminal/issues/18264 has just been updated 
with the following comment by lhecker:

I figured out why this happens: When we read from the console pipe, we 
currently need to provide a buffer where the message is copied into, 
similar to ReadFile. In order to slightly amortize the cost of 
allocating a buffer we reuse the previous allocation unless the previous 
one was >16KiB and the next one is less than half the size. This avoids 
issues where a huge 100MB buffer never gets freed.
This however does not work well for this usage of the API, because the 
GetConsoleProcessList call needs 128KiB and the PeekConsoleInput call 
only needs 20 (?) bytes. This causes the buffer to be repeatedly freed 
and reallocated.

IMO there's 3 things we should do:

    * cygwin should avoid making "large" data requests if there's no 
reason for that. That DWORD buf[65536]; buffer is 128KiB large and in 
practice maybe 16 byte of that will ever be used. It could improve the 
code by only increasing the request size if the return value is equal to 
the buffer size.

    * We should increase the cutoff size from 16KiB to 128KiB, because 
that's the size of the cat chunk size anyway.

    * We should switch to a cheap allocator for processing console 
messages in the future, because malloc is a poor fit for it. But this 
doesn't fix the issue for large allocations, since we still need to be 
conservative about our resident set size. This means that even with a 
custom allocator, cygwin's usage pattern may still be suboptimal.