popup consoles on Windows 7

[Andy Koppe <andy dot koppe at gmail dot com>]

Thanks, Chuck and Dave for pointing me towards 'objdump -p'. I'd got
as far as trying objdump, but hadn't spotted the -p option.

MinTTY is a -mwindows app, which so far seemed to work well on XP and
Vista, because no console is popped up, not even momentarily. Not so
on Windows 7, unfortunately. forkpty() and execve() are used to create
the child process and invoke the command, whereby it appears to be the
exec call that causes a console to pop up.

I guess the fact that rxvt is a CUI app explains why its trick for
creating a hidden console didn't work in MinTTY: rxvt opens that
console in the main process, which in a GUI app isn't passed down to a
child process.

So here's what I've done instead in 0.4-beta1: call AttachConsole(-1)
to attach to any parent console from the main mintty process, and call
it again from the child process to pass it down to that. If there is
no console, create one using AllocConsole (in the child process), and
hide it using ShowWindowAsync.

Unfortunately two problems have been reported with this. The console
isn't always hidden, and a really weird one on a 64-bit Windows 7:
when invoking mintty from a bash running in a console, that console
window locks up. I'm rather stuck on what to do about those, so
suggestions would be welcome.

Also, a couple of questions.

When mintty is invoked from a shortcut or the Run dialog, the main
process actually does get a console, i.e. GetConsoleWindow() returns a
handle, but it doesn't appear on the screen, and the child process
can't attach to it using AttachConsole(). When invoked from a console
or rxvt, GetConsoleWindow() returns null. What's the reason for that?

Furthermore, I'd previously seen the popup console problem on an old
Cygwin install on XP, but updating cured it. So has this been
addressed in the Cygwin DLL in the past, and is it likely that it can
be solved for Windows 7 too? Any idea what might have changed in
Windows that causes this issue?

Btw, XWin is affected by this too. Starting it from a shortcut on
Vista is fine, but on 7 multiple consoles pop up briefly, and one
stays up.

Andy

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/