This is the mail archive of the
mailing list for the Cygwin project.
Re: SPARSE files considered harmful - please revert
- From: "Max Bowsher" <maxb at ukf dot net>
- To: <martin at xemacs dot org>,<cygwin at cygwin dot com>
- Date: Sun, 18 May 2003 23:25:48 +0100
- Subject: Re: SPARSE files considered harmful - please revert
- References: <email@example.com.HOWL>
Martin Buchholz wrote:
> This patch is a bad idea.
> 2003-02-18 Vaclav Haisman <V.Haisman@sh.cvut.cz>
> * fhandler_disk_file.cc: Include winioctl.h for DeviceIoControl.
> (fhandler_disk_file::open): Set newly created and truncated files as
> sparse on platforms that support it.
> As someone on the mailing list asked, "If making every file sparse is
> such a good idea, why isn't it the default?".
Me, I think.
I agree that making *all* files sparse is a bad idea, but let me just say
that I consider the usage of the words "considered harmful" harmful to
effective discussion. It sets the scene for antagonistic flamewars. Ditto
Actual test data, as you give below is *good*, though:
> My experience has been that for me, sparse files take up much more
> disk space than non-sparse files, and are also signicantly slower.
> I build software. My build trees have 50000 files, average size 8k.
> When I copied build trees to a Win2000 NTFS disk using Cygwin tools
> (either cp or tar or rsync) the actual space used on the disk (as
> reported by df, not du) quintupled.
> Here's what I think is happening. Sparse files are implemented like
> compressed files, using 16 clusters. See this web page:
> As a result, a non-empty but small sparse file takes up a minimum of
> 16*clustersize bytes on the disk. My measurements suggest an overhead
> of 32kb per file with a cluster size of 4kb.
> Here are some experiments to support my results:
> MKS's commands creates files 5 times smaller than Cygwin commands.
> I'm sure if you do the experiments yourself, you will see this for
> yourself. To reproduce this problem, you need NTFS 5.0 on Windows
> 2000. Sparse files are a recent NTFS feature.
> The patch is obvious, but I'll send it to cygwin-patches anyways.
> Without this patch, Cygwin is unusable for me.
May I suggest a middle road? Why not let sparse files be configurable as a
$CYGWIN option? This would allow those users who actually want them to
enable them with minimal effort, but keep them off for most users.
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html