[vox-tech] solved puzzle: free du and df disagree about free disk space

[Ken Bloom]

On Tue, 21 Sep 2004 22:02:41 -0400
David Hummel <dhml at comcast.net> wrote:

> On Tue, Sep 21, 2004 at 06:29:31PM -0700, Karsten M. Self wrote:
> > 
> > on Tue, Sep 21, 2004 at 08:25:20AM -0700, Henry House
> > (hajhouse at houseag.com) wrote:
> > > 
> > > As you can see from these transcripts, 'du' told me that I had
> > > used less than 1 GB on this machine's /var, but according to 'df'
> > > there was still no free space.
> > 
> > Moral Jr:  'du' and 'df' report different statistics.
> > 
> 
> Here's a quick and dirty Perl script I wrote a while back to report
> the accurate size of files and directories.  It uses stat()
> recursively. I'm attaching it in case someone might find it useful. 
> Just call it with your file/dir arguments (or none for the current
> directory).  It seems to work pretty well (I'll rely on the Perl
> hackers in the group to verify).

Define "accurate". Each of the three utilities reports different sizes
that can be used for different purposes.

> > 'df' reports the free space on disk.

This is used to determine how much data you can store in new files.
Files which are held open are 

> > 'du' reports the utilization
> > (in blocks) of detectectable files (by definition:  files which
> > aren't, say, deleted but with handles held open by running
> > processes).  Adding a third mix:  

This is used to determine how much space can be added to df's free space
statistic when the given files are deleted and closed. (Detectable means
that these files still have filenames and paths)

This is also used to determine which files to delete and close(see
below) to bring yourself under quota.

> >'ls' reports not the size-on-disk
> > (used blocks) but actual data size within the file.

This reports the number of bytes of data in the file. If you were to
transfer the file over the network, this value is the one that is most
directly related to how long the transfer the file.

du -b reports file sizes the same way as ls. du (without the -b) is
different from ls.

[bloom at kabloom ~]$ du -b *.jpg
461573  phto0001.jpg
478825  phto0002.jpg
[bloom at kabloom ~]$ du --si *.jpg  #--si uses powers of 1000, not 1024
467k    phto0001.jpg
484k    phto0002.jpg

I don't know how your perl script works, and what sizes it reports, but
I'm thinking either du or du -b does what you want.

There's another command that's fits into this universe of file size
commands, and that's the"quota" command. Quota reports your disk quota
and usage in blocks. Blocks * block size = the values handled by du and
df.

To demonstrate that closing the files is necessary when deleting them to
come under quota, you can try the following experiment. Use 2 xterms on
a system with quotas. (I used CSIF)

In the first xterm:
 $ quota
     Filesystem  blocks   quota   limit   grace   files   quota   limit 
csifhome:/csifhome
                  15412   25600   25600            1911  1546240 1546240
 $ dd if=/dev/zero of=file bs=1024 count=1024

In the second xterm
 $ less file
now wait until less computes the number of lines in the file
this way, you know that less doesn't close the file after it finishes
reading it

In the first xterm
 $ fuser file
file:                 3676
(this demonstrates that less is keeping the 
 file open the whole time it runs)
 $ quota
Disk quotas for user bloom (uid 9740): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit 
csifhome:/csifhome
                  16440   25600   25600            1912  1546240 1546240
 $ rm file
 $ quota
Disk quotas for user bloom (uid 9740): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit 
csifhome:/csifhome
                  16440   25600   25600            1912  1546240 1546240

In the second xterm, quit less

In the first xterm
 $ quota
Disk quotas for user bloom (uid 9740): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit 
csifhome:/csifhome
                  15412   25600   25600            1911  1546240 1546240
       

See that? The quota usage dropped only when you quit less.

--Ken Bloom

-- 
I usually have a GPG digital signature included as an attachment.
See http://www.gnupg.org/ for info about these digital signatures.

G'mar Chatima Tova
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://ns1.livepenguin.com/pipermail/vox-tech/attachments/20040921/9ca9d78e/attachment-0001.bin