locate and updatedb

[Byron Boulton]

On 2/17/2016 11:00 AM, Buchbinder, Barry (NIH/NIAID) [E] wrote:
> Byron Boulton sent the following at Wednesday, February 17, 2016 8:43 AM
>> On 2/16/2016 5:55 PM, Buchbinder, Barry (NIH/NIAID) [E] wrote:
>>>
>>> This is technically OT since this involved a non-cygwin tool.
>>>
>>> find is slow compared with a non-Cygwin tool, specifically dir (cmd.exe).
>>>
>>> Compare find with cmd.exe's dir.  Note that even with the benefit of
>>> caching (compare the 1st and 3rd times), find takes twice as long as dir.
>>> Comparing cached times (2nd vs 3rd), dir is 3X faster.
>>>
>>> $ time cmd /c dir /s /b 'C:\usr' > /dev/null ; \ time find /c/usr >
>>> /dev/null ; \ time cmd /c dir /s /b 'C:\usr' > /dev/null
>>>
>>> real    0m1.326s
>>> user    0m0.000s
>>> sys     0m0.047s
>>>
>>> real    0m2.465s
>>> user    0m0.280s
>>> sys     0m2.184s
>>>
>>> real    0m0.874s
>>> user    0m0.000s
>>> sys     0m0.031s
>>>
>>> (Note: c:\usr has nothing to do with /usr.)
>>>
>>> Here's how I use dir *in the abstract* for drives C: and D:.  (Note:
>>> the
>>> /a: option of dir lists all files, including hidden ones; /o:n sorts
>>> by
>>> name.)
>>>
>>> for D in /c /d
>>> do
>>>       "$(cygpath "${COMSPEC}")" /c dir /s /b /a: /o:n "$(cygpath -w "$D")"
>>> done | \
>>> tr -s '\r\n' '\n' | \
>>> cygpath -u -f - | \
>>> sed -e '/^$/d' -e 's,/\+,/,g' \
>>> sort -u \
>>> /usr/libexec/frcode > /tmp/updatedb.tmp chmod --reference
>>> /var/locatedb /tmp/updatedb.tmp mv /tmp/updatedb.tmp /var/locatedb
>>>
>>> What I actually do (attached) is more complicated.  My script chooses
>>> which directories are scanned, does them in parallel, and prints
>>> pretty messages.  I get error messages for very long paths (> ~250
>>> bytes).  It works well enough for me; YMMV.
>>
>> Are you using dir in some sort of custom way to build the database
>> used by locate? Or are you saying that rather than ever using the find
>> command to find files, you use a custom script which uses dir?
>
> I use dir only to generate the locate database, because scanning the
> better part of several disks takes so long.  I do not substitute dir for
> find for other purposes.  One could, but usually locate does what I need,
> and when it doesn't, I use find.
>
> Best wishes,
>
> - Barry
>    Disclaimer: Statements made herein are not made on behalf of NIAID.
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
locate understands how to read this custom database? If I read you 
updatedb.sh script properly, it produces a file which is just a sorted 
text file with one line per file found by updatedb.sh.

Byron


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple