Self-compiled xorg-server 1.20.0 crashes at startup, buffer overflow

[Jon Turney]

On 30/07/2018 16:33, tumtum00 wrote:
> after building xorg-server 1.20.0 without errors, the XWin binary crashes at startup due to a buffer overflow. XWin 1.20.0 from Cygwin repos works as expected.
> 
> Setup:
> - Freshly installed Win7 x86_64 VM
> - Cygwin x86_64
> - Dependencies installed as described in https://x.cygwin.com/docs/cg/prog-build-prerequisites.html, plus xcb-composite-devel

Thanks for pointing out this oversight.  I will amend the documentation.

> - Applied the following patches to my Cygwin environment, otherwise the build would fail:
>    https://github.com/cygwinports/cygport/commit/1072fd872757bd7aa97fa8daf95ed13f70f56291
>    https://github.com/cygwinports/cygport/commit/c4e0f0ec5f9a5a691fd19efe2bda3cdf278baef5
> - Downloaded patched xorg-server 1.20.0-2 sources as described in https://x.cygwin.com/docs/cg/prog-obtaining-source.html, "Obtaining via Cygwin setup"
> 
> Build:
>    $ cygport --64 xorg-server.cygport prep
>    $ cygport --64 xorg-server.cygport compile
> 
> XWin binary
>    /usr/src/xorg-server-1.20.0-2.src/xorg-server-1.20.0-2.x86_64/src/xserver-cygwin-1.20.0-2/x86_64-pc-cygwin/hw/xwin/XWin
> is then executed.
> 
> The `--buildtype=debug` option was added to `cygmeson` in `xorg-server.cygport` to generate the attached reports.
> 
> I'm out of ideas on how to continue debugging this. Help would be much appreciated.

I tried to reproduce this problem by following these steps, but the 
produced executable works ok for me.

> #7  0x000000018017e7e9 in __chk_fail () at /usr/src/debug/cygwin-2.10.0-1/newlib/libc/ssp/chk_fail.c:11
>         msg = "*** buffer overflow detected ***: terminated\n"
> #8  0x000000018018c1c3 in __memcpy_chk (dst=0x60008e570, src=0x45, len=0, slen=0) at /usr/src/debug/cygwin-2.10.0-1/newlib/libc/ssp/memcpy_chk.c:48

This is a wrapper around memcpy, which is being used as we are building 
with _FORTIFY_SOURCE

I'd suggest setting a breakpoint on this line (after installing the 
cygwin-debuginfo package for relevant debug info) and trying to 
understand why it's not succeeding.

> No locals.
> #9  0x00000001004b80d5 in RRConfigureOutputProperty (output=output@entry=0x60008e570, property=<optimized out>, pending=pending@entry=0, range=range@entry=0,
>     immutable=immutable@entry=0, num_values=num_values@entry=2, values=values@entry=0x10059f4b0 <values>) at ../randr/rrproperty.c:385
>         prop = 0x60008cff0
>         add = <optimized out>
>         new_values = 0x60008e650
> #10 0x00000001004b682e in RROutputCreate (pScreen=pScreen@entry=0x60008bff0, name=name@entry=0x100586942 <__func__.9648+722> "default", nameLength=nameLength@entry=7,
>     devPrivate=devPrivate@entry=0x0) at ../randr/rroutput.c:120
>         values = {0, 1}
>         output = 0x60008e570
>         outputs = <optimized out>
>         nonDesktopAtom = <optimized out>
> #11 0x000000010040c11b in winRandRInit (pScreen=pScreen@entry=0x60008bff0) at ../hw/xwin/winrandr.c:256
>         crtc = 0x60008e240
>         output = <optimized out>


-- 
Jon Turney
Volunteer Cygwin/X X Server maintainer

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple