Sparse file support for SMB by default? Re: Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks?

[Cedric Blancher]

On Wed, 6 Dec 2023 at 10:39, Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Dec  6 09:49, Cedric Blancher via Cygwin wrote:
> > On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin
> > VMware emulates NVME SSD these days, so this should work, yes?
>
> It doesn't matter, see below.
>
> > Primary concern is how to detect whether sparse file support is ON in
> > Cygwin for a specific filesystem? Maybe /sbin/mount without options
> > should have a sparse/nosparse mount flag to reflect what is used?
>
> What concern?  I explained exactly what happens, and it all revolves
> around the fact that a filesystem sets the FILE_SUPPORTS_SPARSE_FILES
> flag or not.  This is the only interesting fact.  Nothing else
> matters.
>
> If you want to see if a filesystem supports that flag, just use the
> /usr/lib/csih/getVolInfo tool:
>
>   $ /usr/lib/csih/getVolInfo .
>   Device Type        : 7
>   Characteristics    : 20020
>   Volume Name        : <root>
>   Serial Number      : 2292001085
>   Max Filenamelength : 255
>   Filesystemname     : <NTFS>
>   Flags              : 3e72eff
>     FILE_CASE_SENSITIVE_SEARCH        : TRUE
>     FILE_CASE_PRESERVED_NAMES         : TRUE
>     FILE_UNICODE_ON_DISK              : TRUE
>     FILE_PERSISTENT_ACLS              : TRUE
>     FILE_FILE_COMPRESSION             : TRUE
>     FILE_VOLUME_QUOTAS                : TRUE
>     FILE_SUPPORTS_SPARSE_FILES        : TRUE
>     FILE_SUPPORTS_REPARSE_POINTS      : TRUE
>     FILE_SUPPORTS_REMOTE_STORAGE      : FALSE
>     FILE_RETURNS_CLEANUP_RESULT_INFO  : TRUE
>     FILE_SUPPORTS_POSIX_UNLINK_RENAME : TRUE
>     FILE_VOLUME_IS_COMPRESSED         : FALSE
>     FILE_SUPPORTS_OBJECT_IDS          : TRUE
>     FILE_SUPPORTS_ENCRYPTION          : TRUE
>     FILE_NAMED_STREAMS                : TRUE
>     FILE_READ_ONLY_VOLUME             : FALSE
>     FILE_SEQUENTIAL_WRITE_ONCE        : FALSE
>     FILE_SUPPORTS_TRANSACTIONS        : TRUE
>     FILE_SUPPORTS_HARD_LINKS          : TRUE
>     FILE_SUPPORTS_EXTENDED_ATTRIBUTES : TRUE
>     FILE_SUPPORTS_OPEN_BY_FILE_ID     : TRUE
>     FILE_SUPPORTS_USN_JOURNAL         : TRUE
>     FILE_SUPPORTS_INTEGRITY_STREAMS   : FALSE
>     FILE_SUPPORTS_BLOCK_REFCOUNTING   : FALSE
>     FILE_SUPPORTS_SPARSE_VDL          : FALSE
>     FILE_DAX_VOLUME                   : FALSE
>     FILE_SUPPORTS_GHOSTING            : FALSE
>
> > >   What Cygwin gets to see is that you want to access a file on some
> > >   disk.  It opens the file and fetches volume information.  So it finds:
> > >
> > >   - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES
> > >
> > >     So we can create sparse files.
> > >
> > >   - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag.
> > >
> > >     So we're on a drive having no seek penalty.  This is *probably*
> > >     an SSD, but may be some other type of drive.
> >
> > What about SMB? SMB supports sparse files, and should certainly use
> > this by DEFAULT, as it makes many HPC apps operate much faster.
> > BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a
> > drive, and not for network filesystem.
> >
> > Same applies to the NFSv4.1 filesystem driver, I'm going to pester
> > Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue
> > as SMB here.
>
> Again, the filesystem doesn't matter.  It either sets the
> FILE_SUPPORTS_SPARSE_FILES flag or not, as simple as that.
>
> If it does, you can create sparse files with chattr +S, or you can rely
> on the lseek/ftruncate/posix_fallocate automatism, or you stamp a hole
> into the file with fallocate(FALLOC_FL_PUNCH_HOLE).
>
> The *only* difference is if you have to use the "sparse" mount option or
> not.
>
> Basically, with 3.4, you always have to set the "sparse" mount option,
> with 3.5, on local SSDs you don't.  I don't see a problem here.

How can I remount the existing C: mount with the "sparse" option in
Cygwin 3.4? I cannot get it working, or I am too stupid.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur