"File Size" Not Increasing As It Should Be

[nkormanik]

A lot of people are complaining about this out there, I see.  Just wondering what you folks have to say on the issue.

Given:  I have a 'process' executing.  And an Explorer window open, to monitor a folder that is supposed to be 'populated' with the results of 'process.'  A file exists there which is supposed to grow in size.

Existing file: XXYY.xxyy
Existing file size: 0 bytes

I'm expecting said file to increase.  12kb.  130kb.  256kb.  etc.

After an hour the file still remains at 0 bytes.

That is, in Explorer.  The little Explorer window I have open to watch it.

Yet, the actual fact of the matter is, the file HAS been growing.  Is now growing.  Is around 5 megabytes in size.  And growing still.

Yet Explorer continues to show the file as 0 bytes in size.

So..., out there in cyberland, various solutions to this issue have been proposed.

I'm wondering what your take is.

Thanks!

Nicholas Kormanik

[skwire]

Nick, since you put this in the Coding Snack section, was this question in relation to a past Coding Snack?  If not, which application are you referencing?  Also, if not related to a Coding Snack, do you mind if we move this post to a more relevant section?

[Target]

so are you refreshing that explorer window while your watching it?  Those windows are static and won't show any changes until refreshed (F5)

[nkormanik]

Perhaps best to move to more relevant section, Skwire.

Solution to issue, my hunch is, a small alternative to Explorer, that can dig in to the actual size, and has a refresh button.  Periodically press refresh to see updated sizes of files in particular folder.

Not sure that such program exists.

[mouser]

I actually noticed this happening very recently, where even refreshing the current file explorer panel was still showing 0 as the file was being incrementally written over a long time.  This is not how I remember things behaving, so perhaps it is a recent issue on windows..

[4wd]

This has been normal for me ever since ... well ... a very long time, across different file managers, eg. Explorer, DOpus, etc.

Normally I'd have to do a Properties on the relevant file and then refresh the display to get the current size.

Oddly enough it seems to be random sometimes, eg. writing an MP4 video file will:

• sometimes show a steady increase,
• sometimes just sit at 0 until it's closed,
• or sometimes show a steady increase but only after forcing the system to interrogate the file, (eg. Properties).

A F5 refresh alone doesn't seem to initiate a return to increasing file size.

Maybe it depends on the way the file is being written and/or the filesystem of the destination?
eg. One long continuously open write as opposed to the file being closed between successive appends.

I just ignore it now until I see the process is finished.

[Target]

maybe this - https://www.nirsoft....er_changes_view.html

[wraith808]

maybe this - https://www.nirsoft....er_changes_view.html

-Target (December 02, 2018, 09:46 PM)

There are a few out there that do this.  A lot depends on the methodology even with these, as I've noticed that even doing it programmatically using the Windows API, it stops working after a while.  I've narrowed down this particular issue in several cases to be that the file isn't flushed, but kept locked, so that the operation isn't evident on the drive.  Is it possible that the data hasn't been flushed to the file?

[Target]

There are a few out there that do this.  A lot depends on the methodology even with these, as I've noticed that even doing it programmatically using the Windows API, it stops working after a while.  I've narrowed down this particular issue in several cases to be that the file isn't flushed, but kept locked, so that the operation isn't evident on the drive.  Is it possible that the data hasn't been flushed to the file?

-wraith808 (December 02, 2018, 11:14 PM)

yep, more than a few tools around to do stuff like this. 

Lots of gaps in the OP so the nirsoft stuff is probably a good starting point

[nkormanik]

As 4wd points out, yes, context menu, Properties, does seem to do the trick of updating the filesize.

Other files in the folder don't get updated, I don't think.  Therefore, one would have to right-click on each, and do the Properties route.

[KodeZwerg]

TotalCommander watches in realtime all changes.
The problem is how windows works or better how you application that write something work.

The filesystem will get a notification message to update explorer (and others) just after a file is written / the application that do writing have to close the filehandle. Some applications send several messages to explorer after they wrote a block of data.

The standard would be:
When an application create a file to write something, and pumps in data, all is done in your windows and hdd cache.
If you want to see every little change in realtime you should disable all caches.
You will notice that your hdd activity will increase by a lot of more disk reading and writing.
All become slower due to this but you will be able to see in realtime how a file grow.

[x16wda]

stat.exe %s filename will show the correct size if it is available, sometimes it isn't. I don't see it included in coreutils, but I thought it came with unxutils Windows port. Can't find it with a quickie search though.

[nkormanik]

Nirsoft FolderChangesView seems the best option so far.

However, even this doesn't show the actual file size.

Here is an email I just wrote to Nirsoft:

Enhancement Request -- FolderChangesView

Please incorporate a ‘button’ or ‘menu offering’ to really refresh all the present files shown.

For some reason the existing refresh does not really update file sizes, at least not when files are still being written to.

A process I have running keeps adding to a particular file.  Only way to see the increased real file size is to bring up ‘File Properties,’ one individual file at a time.

Far, far better to have a means to refresh all files shown, really refreshing them, showing true file sizes.

Thanks!

Nicholas Kormanik

[x16wda]

So I verified stat.exe is actually in coreutils 5.3 at least even though it isn't mentioned in the list: gnu coreutils

Try using this periodically during your process and see if it doesn't give you a more up to date value:  stat.exe -c %s full-filename.here

Or you can just do 'stat filename' and you'll see the full display.

[KodeZwerg]

Aslong a file aint written you do not see much changes. It is all in cache.
There are possibilies for programmers to update explorer.exe, if they dont use them you just have to wait...
In some of my own copy-routines for example you will instant see final filesize, due how i deal with it.

Why not just look your copy-progress to see how much data you received?

To explain a bit more why you never be able to have such thing under your control...
Imagine this situation, it is a practical way to quick save multithreaded:
 - Copy-Program begin
 - progress is more than one thread
 - progress create first of all the final filesize
 - prograss create individual thread that write to different offsets a specific amount of data
 - now tell me how would you ever be able to see how many bytes are written with explorer.exe or stat.exe or whateverelse.exe
 - solution: watch programs progress and forget about windows

next sample:
 - Copy-Program begin
 - progress is one thread
 - progress read file a) into memory
 - progress write memory to file b)
 - aslong routine is writing, no external can access it to get proper information
 - you can try readout by overriding, this may end in a corrupted file due that action
 - solution: watch programs progress and forget about windows

good programmed routines work like this:
 - Copy-Program begin
 - progress read chunk(s)
 - progress write chunk(s)
 - progress update explorer.exe that it has written some more chunks to file
 - only in this situation you are able to get proper information
 - to force this situation, you need to disable all caches

[nkormanik]

Thank you all for sharing your wise, experienced insights.  Terrific resource here.

Nirsoft's FolderChangesView seems the best one can do for now, all things considered.

In that program getting 'File Properties' on parent folder additionally updates the current file sizes for all child files.  That eliminates the need to get individual file properties.

The whole purpose here was/is to merely make sure the program process is indeed chugging along as it should be, as the program itself doesn't let one know directly.