This is the mail archive of the
mailing list for the Cygwin project.
RE: PATHEXT is fundamental to Windows and Should be recognised by CYGWIN
- From: "Michel LaBarre" <michel dot labarre at rogers dot com>
- To: "'Kaz Kylheku'" <kaz at kylheku dot com>
- Cc: <cygwin at cygwin dot com>
- Date: Thu, 4 Aug 2016 00:30:46 -0400
- Subject: RE: PATHEXT is fundamental to Windows and Should be recognised by CYGWIN
- Authentication-results: sourceware.org; auth=none
- References: <email@example.com> <firstname.lastname@example.org>
Thanks for the reply Kaz. I did not embed any comments in your reply for the sake of brevity.
I had seen the reference to a patch to support PATHEXT; it was dismissed pretty lightly.
PATHEXT has been part of Windows for as long as I can remember - back to the mid-80's - used by shells that run under windows (e.g. CMD, PowerShell, VBS, etc.). Windows inherently uses file suffixes to associate programs to data files, hence CMD files to CMD.exe, VBS to VB, etc.. All its shells refer to PATHEXT and/or file associations in the registry.
I have worked with Unix for a long time (1974) and Windows for almost as long (1986?). Because some notion does not exist in Unix is no reason for discrediting it within Windows esp in the context of a tool framework specifically catering to Windows according to its defining mission. If you have been web-searching for references to my query, surely you have found many items referring to the problem in practice. I used to move a whack of scripts between Solaris and Windows - same scripts supporting an enterprise backup product with HQ hosted on Solaris and 150 sites worldwide running Windows only. My automated port process stripped or added suffixes as necessary going between Solaris and Windows. However, once in Windows, a tool could be called from a shell script or a CMD script without regard of the suffix - the (MKS) shell worked with the suffix or not while CMD (and vbs) relied upon PATHEXT.
One has to look at practical applications when deciding what is warranted. I cannot help feeling that CYGWIN's proponents give lip service to interoperability with Windows - that they would rather be running Linux in a VM in which case why bother with a Windows foundation in the first place? Other than Microsoft fearing competition, the second biggest drawback to Windows developers adopting Unix tools has been due to Unix developers wishing that Windows would go away.
MKS has done a great job of working with Windows. You may recall Interix which tried to launch a product that ran isolated within the POSIX subsystem - it went nowhere - Microsoft bought the dregs of the company for no rational reason I can discern. They released a Unix environment for NT and it tanked. They are trying again with Ubuntu under Windows 10 but that will fail as well because it will be completely isolated from the Win32 environment.
Windows developers need to get to Windows tools and resources - CYGWIN could be a very smart supplement to that requirement. Windows 7 with the MKS Toolkit is in fact one of the most productive environments I have used. MKS's value lays in its integration with Windows. CYGWIN's developers recognising that Windows is not a passing fad might push CYGWIN into the mainstream of Windows development. As it stands, if getting into it is a challenge for people with plenty of Unix and Windows background, it will not make much headway with pure Windows programmers.
Your comment regarding " What does not work from CMD is Microsoft's problem, not Cygwin's." ignores that Cygwin is supposed to work within Windows. You penalise CYGWIN users - not Microsoft. Frankly, that comment strikes me as somewhat cavalier.
"Cygwin provides a POSIX environment on Windows. Users of a POSIX environment don't expect a command "foo" to resolve to a file "foo.bat". If they want to run "foo.bat" they use "foo.bat"."
I disagree completely. If you are in an interactive bash, running on a Windows computer, you sure as hell expect to be able to run "foo.bat" by typing "foo". And if you are in a script you expect to release to a large community, If "foo" comes from some 3rd party package released independently, then you don't want to be wiring into your script that "foo" is a bat, exe, vbs, cmd, or whatever in case the 3rd party package ever changes its implementation (e.g. converting a CMD to an EXE for performance reasons). I agree fully that PATHEXT is not a great mechanism - it just happens to be wired into Windows.
It is a myth that Unix-y programmers work in a vacuum within Windows. Any serious software will have to exploit, hence interact with, Windows native and add-on tools.
As for CR/LF, I admit that I have not mastered all the opportunities fstab might present under CYGWIN but I don't think they will apply if the bash script is invoked from a Windows environment rather than from a bash script would they.
Thanks for your thoughts Kaz. I realise you are a volunteer and appreciate your efforts. Having cut my teeth on Unix, I appreciate its value however having been forced to work with Windows for so many decades, I have come to appreciate a few aspects. I just miss all the tools for day-to-day use I took for granted under Unix. If CYGWIN could mitigate some of the recurring impediments new users trip over, (as evidenced by the many web-references to both my problems) it would facilitate its adoption by both Unix and non-Unix types.
From: email@example.com [mailto:firstname.lastname@example.org] On Behalf Of Kaz Kylheku
Sent: August-03-16 10:55 PM
To: Michel LaBarre
Subject: Re: PATHEXT is fundamental to Windows and Should be recognised by CYGWIN
On 03.08.2016 18:43, Michel LaBarre wrote:
> Problem 1: Cygwin does not support PATHEXT and really should.
A casual websearch shows that this topic has come up before.
For instance, someone posted, some decade ago, to the Cygwin mailing list, a patch against GNU Bash to make its command search algorithm take PATHEXT into account.
> Fundamental reason: from the Cygwin FAQ - What is it? "Cygwin is a
> distribution of popular GNU and other Open Source tools running on
> Microsoft Windows."
> PATHEXT is as fundamental component of Windows program execution as
I can't find any reference anywhere to PATHEXT being used outside of the CMD.EXE interpreter, which rather tends to make it substantially less than fundamental to Windows, though noteworthy.
Certainly, CreateProcess does not use PATHEXT.
I can't find any documentation which says that ShellExecuteEx uses PATHEXT, either. (Can anyone confirm?)
Everything points to it being a CMD.EXE "gizmo". If you want to execute a command such that PATHEXT is taken into account, you have to spawn CMD.EXE /C <yourcommand>.
Cygwin provides a POSIX environment on Windows. Users of a POSIX environment don't expect a command "foo" to resolve to a file "foo.bat". If they want to run "foo.bat" they use "foo.bat".
> Without using extensions, bash can use execution privileges to
> determine if a file is executable. However, that does not work when
> invoking a command from CMD.
What does not work from CMD is Microsoft's problem, not Cygwin's.
Yes, even though you have a "myscript.sh" and you do "chmod a+x myscript.sh"
inside Cygwin, the outside Windows world doesn't agree that myscript.sh is now executable, and that it uses /bin/sh by default as its interpreter, if a #! line is absent, otherwise the interpreter nominated by the #!
Adding ".sh" to PATHEXT might work in causing CMD.EXE to resolve "myscript" to "myscript.sh"; however, it will not then successfully execute "myscript.sh", because the underlying CreateProcess API will not find it executable.
CMD.EXE will probably *try* to execute it, and fail.
> This means that when invoked from BASH, you name a command "ZOT"
> but "ZOT.sh" (or similar) if invoked from CMD.
From CMD, you have do something explicit like this:
That is, you have to run Bash, and pass it the script as an argument.
Windows and CMD.EXE will not associate .sh with Bash and certainly won't look at any #! line you may have in the script.
Not supporting arbitrary interpreters for scripts is a Windows problem/limitation.
Cygwin overcomes that limitation within its "garden".
If you solve the entry point problem of how to invoke Cygwin code from the outside, once you are in Cygwin land, you have no further problems. Scripts marked executable and containing #! use their respective interpreters and so on.
> The published solutions in
> the various FAQs are not satisfactory. Creating links between ZOT.sh
> and ZOT creates substantial overhead.
I don't think that will work, unless by "creating a link" you mean that you create a ZOT.BAT file shim which invokes the neighboring ZOT.sh by passing its full path to bash.exe.
> If CYGWIN really intends to support Windows it should support its
> fundamental execution framework. It should be equally easy to invoke
> a bash script from a bash script or a CMD script.
What you need on Windows is a script-to-EXE application deployment tool, which takes your "script.sh" and combines it with a copy of a special binary executable, and writes out the combination to "script.exe".
Then even a raw CreateProcess Win32 API call can invoke "script.exe".
> Problem 2: Cygwin does not support CR-LF delimiters. For the same
> reason, it is unfortunate that CYGWIN/bash does not cope with both
> types of line delimiters transparently.
The way Cygwin deals with CR/LF has evolved over time, and there are various ways to deal with this, depending on the specific situation.
Firstly, the binary mode default treatment for files comes from the mount table:
C:/Cygwin/bin on /usr/bin type ntfs (binary,auto)
C:/Cygwin/lib on /usr/lib type ntfs (binary,auto)
C:/Cygwin on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
See how that /cygdrive/c is mounted "binary" (as is everything else?) That's what causes text files to be treated as LF rather than CR/LF.
That can be altered; you can mount some Windows path into Cygwin's POSIX file system view as text, and then CR/LF conversion is done.
Then, secondly, there are approaches for individual C programs.
If you're porting something written in C, the C library in Cygwin supports the "t" flag in fopen and related functions. This is similar to the Microsoft extension, found in the Visual C run-time library.
In ISO C, if you omit the "b" mode, then a file is open in text mode, but on POSIX that is the same as binary mode. In Cygwin, if you specify "t", you get the Windows text mode, CR/LF. Without "t"
or "b", I think it defaults to the mode from the mount table. So if you're working on porting a C program, that's one tool in your arsenal.
As an alternative to introducing "t" into C programs, there is a link-time method. Cygwin provides several special .o object files that can be specified when linking a C program, which change its treatment of files in this regard.
Note that some standard utilities for processing text, though written in C, do not use the C stdio library. So they don't respond to these mechanisms. For instance GNU Awk does its own I/O, and that is LF terminated.
Problem reports: http://cygwin.com/problems.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple