This is the mail archive of the
mailing list for the Cygwin project.
Re: SPARSE files considered harmful - please revert
- From: "John Vincent" <jpv50 at hotmail dot com>
- To: cygwin at cygwin dot com
- Date: Mon, 19 May 2003 18:44:23 +0000
- Subject: Re: SPARSE files considered harmful - please revert
I looked up sparse files on MSDN and found the following link:
The most interesting thing is that a sparse file is only sparse if the zeros
in the file are written with a special operation. I strongly suspect that
the patch to support sparse files introduced in cygwin is incorrect (or at
I've quoted the contents of the entry below, I hope this is helpful:
Sparse File Operations
To determine whether a file system supports sparse files, call the
GetVolumeInformation function and examine the FILE_SUPPORTS_SPARSE_FILES bit
Most applications are not aware of sparse files and will not create sparse
files. The fact that an application is reading a sparse file is transparent
to the application. An application that is aware of sparse-files should
determine whether its data set is suitable to be kept in a sparse file.
After that determination is made, the application must explicitly declare a
file as sparse, using the FSCTL_SET_SPARSE control code.
After an application has set a file to be sparse, the application can use
the FSCTL_SET_ZERO_DATA control code to set a region of the file to zero. In
addition, the application can use the FSCTL_QUERY_ALLOCATED_RANGES control
code to speed searches for nonzero data in the sparse file.
When you perform a write operation (with a function or operation other than
FSCTL_SET_ZERO_DATA) whose data consists of nothing but zeros, zeros will be
written to the disk for the entire length of the write. To zero out a range
of the file and maintain sparseness, use FSCTL_SET_ZERO_DATA.
A sparseness-aware application may also set an existing file to be sparse.
If an application sets an existing file to be sparse, it should then scan
the file for regions which contain zeros, and use FSCTL_SET_ZERO_DATA to
reset those regions, thereby possibly deallocating some physical disk
storage. An application upgraded to sparse file awareness should perform
When you perform a read operation from a zeroed-out portion of a sparse
file, the operating system may not read from the hard drive. Instead, the
system recognizes that the portion of the file to be read contains zeros,
and it returns a buffer full of zeros without actually reading from the
As with any other file, the system can write data to or read data from any
position in a sparse file. Nonzero data being written to a previously zeroed
portion of the file may result in allocation of disk space. Zeros being
written over nonzero data (only with FSCTL_SET_ZERO_DATA) may result in a
deallocation of disk space.
Note It is up to the application to maintain sparseness by writing zeros
Defragmenting tools that handle compressed files on NTFS file systems will
correctly handle sparse files on NTFS volumes.
Stay in touch with absent friends - get MSN Messenger
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html