Main Area and Open Discussion > Living Room

File Size vs. Size on Disk: Why such a difference?

<< < (4/5) > >>

Deozaan:
32kb is a pretty large cluster size - but I guess it might have been set to the flash erase-block size?
-f0dder (April 08, 2010, 10:16 AM)
--- End quote ---

Yeah, you're right. Somehow I had been thinking that the 4096 bytes I've seen for NTFS was actually 4 MiB, so I was thinking that 32 KiB was relatively small.

I may have used some Panasonic SD card formatting software on the thing as I'd heard that using Windows formatter could cause problems with microSD cards.

I honestly don't know much about microSD and flash memory. Do you think it would be wise to reformat the card to FAT32 with a 4096 byte cluster size? Or could that cause problems?

Deozaan:
Realize that each file will fill all clusters it uses except the last cluster (which might be full, but generally won't be).  If we assume that each file will use on average only half of the last cluster, we come up with the following guesstimate for 'wasted space'

    32768/2 * 16047 = 262,914,048 wasted bytes

Still not accounting for all the waste that you're apparently seeing, but I'd suspect that the true numbers skew toward more files that use far less than a cluster (I think of the files that use only a single cluster, there are far more that use less than half than those that use more than half).

Also, this doesn't account for the waste used by directories (which if I recall, are allocated clusters similarly to files except for the root directory), which would bring us up to close to about 300MB of wasted space.

This analysis hasn't account for all the waste you're seeing, but we're less than a factor of 2 away...
-mwb1100 (April 08, 2010, 12:39 AM)
--- End quote ---

I had taken this into consideration with my initial calculations (though my math may have been wrong), but failed to explain it properly. I've reworded it in the post to hopefully make that more clear.

Basically with the average file size being about 39,356 bytes, that means that each file will fill one cluster and the remaining 6,587 bytes "spill over" into the next 32 KiB cluster.

Actually, calculating it again as 16,047 * (1024 * 32) - (16,047 * 6,587) gives me 420,126,507 wasted bytes, which is 400.66 MiB, which is pretty close to the 391 MB wasted.

EDIT: It's only by chance that those calculations came out so close, since assuming every file size is 39,356 bytes would by nature not calculate exactly. The real (estimated) math goes something like this:

Since every file size (assuming every file is of the average size) takes up 2 clusters, the total space on disk should be displaying as 1,051,656,192 bytes. We know that at least half of that is actually being used since each file is more than 32 KiB, so cut it in half to get 525,828,096 bytes. Of that, 105,701,589 is also actually used, which means that the remaining 420,126,507 bytes are "wasted." But even so, if this were an accurate calculation then the size on disk would display as approximately 1 GB rather than "only" approx. 900 MB.

Deozaan:
32kb is a pretty large cluster size - but I guess it might have been set to the flash erase-block size?
-f0dder (April 08, 2010, 10:16 AM)
--- End quote ---

I decided to format it and when I went to do that I found out that the default cluster size for FAT is 32 KiB. So I decided to format it to NTFS with 4096 byte cluster size and now there is a significant decrease in wasted bytes.

It went from just under 400 MB of wasted space to just under 40 MB.

Thanks for the info and help, everyone! :Thmbsup:

Stoic Joker:
I decided to format it and when I went to do that I found out that the default cluster size for FAT is 32 KiB.-Deozaan (April 09, 2010, 01:41 AM)
--- End quote ---
...Which is why I mentioned the MS Oformat utility earlier - That's what it's for - creating low cluster size FAT/FAT32 partitions that can easily/safely be converted to NTFS (with Win2k era convert.exe). It was a required procedure back then if you wanted/needed the NTFS compression to work...and were going to be starting with a FAT drive (for whatever reason).

app103:
I love when I go googling for info and one of the first results is the answer, right here on this forum.  :-*

Navigation

[0] Message Index

[#] Next page

[*] Previous page