DonationCoder.com Forum
Main Area and Open Discussion => Living Room => Topic started by: superboyac on November 19, 2012, 12:36 PM
-
All USB sticks come formatted as FAT. But FAT can't store large multi-gigabyte files. Does anyone see any problems with formatting a disk in NTFS? I can't come up with a reason not to...
-
I have no idea/opinion myself
but there's this:
http://serverfault.com/questions/41674/flash-drives-should-be-formatted-ntfs
-
Nothing wrong with it- however there are some things that should be considered:
1. You have to make sure that you eject safely. The OS holds on to NTFS resources longer, and if you just remove it, the chances are high that you will corrupt data (write-caching).
2. There's also a lot more writing to the drive, which will reduce the life of the drive, and minimally reduce performance.
3. NTFS allows encryption- but be aware that if you do, you will not be able to open the files anywhere else.
4. There's also a space overhead for NTFS. How much varies by where you look for the information, but it seems to be about 4%.
No breaking changes- but they are some things to be aware of.
-
All USB sticks come formatted as FAT. But FAT can't store large multi-gigabyte files. Does anyone see any problems with formatting a disk in NTFS? I can't come up with a reason not to...
-superboyac
you can always use an archiver like winrar that can split big files into chunks of 4GB max. And keep the FAT32 format.
-
All USB sticks come formatted as FAT. But FAT can't store large multi-gigabyte files. Does anyone see any problems with formatting a disk in NTFS? I can't come up with a reason not to...
-superboyac
you can always use an archiver like winrar that can split big files into chunks of 4GB max. And keep the FAT32 format.
-MerleOne
(Obviously) not if you want to use the files directly, though - like large .WIM files for Windows installs, or HD .MKVs for playback on a media thingamajig :)
I've used NTFS-formatted pendrives for a while, and they work fine (as fine as those unrealiable POSs go, anyway). Only trouble I've run into is that NTFS is read-only on OSX, so you can't use it to get data from loose-wristed coworkers :-)
-
All USB sticks come formatted as FAT. But FAT can't store large multi-gigabyte files. Does anyone see any problems with formatting a disk in NTFS? I can't come up with a reason not to...
-superboyac
you can always use an archiver like winrar that can split big files into chunks of 4GB max. And keep the FAT32 format.
-MerleOne
(Obviously) not if you want to use the files directly, though - like large .WIM files for Windows installs, or HD .MKVs for playback on a media thingamajig :)
I've used NTFS-formatted pendrives for a while, and they work fine (as fine as those unrealiable POSs go, anyway). Only trouble I've run into is that NTFS is read-only on OSX, so you can't use it to get data from loose-wristed coworkers :-)
-f0dder
Sure, but when clicking on the .rar file it will uncompress it to a temp folder and run it from there. Less efficient but it would still work - you still need winrar installed I think.
-
1. You have to make sure that you eject safely. The OS holds on to NTFS resources longer, and if you just remove it, the chances are high that you will corrupt data (write-caching).
-wraith808
Once you formatted the drive to use NTFS as file-system, you can revert to usual settings for flash drives and turn off write caching. That will minimize data corruption risks.
-
If the device comes formatted as FAT32 my advice is to do a slow format when formatting to NTFS. I don't remember seeing it stipulated anywhere that it's required. But I've had USB that I changed from FAT32 and once it got past the point where I had written the drive in the past, in terms of disk space used, I started getting flaky behavior. They performed as expected after a slow format as NTFS.
Also to cut down on disk accesses with NTFS you can disable last access. I do that on all my systems. I don't use databases or backup robots that need that setting.
Just google "fsutil disable last access" for the command line.
-
If the device comes formatted as FAT32 my advice is to do a slow format when formatting to NTFS. I don't remember seeing it stipulated anywhere that it's required. But I've had USB that I changed from FAT32 and once it got past the point where I had used the drive in the past, I started getting flaky behavior. They performed as expected after a slow format.-MilesAhead
Quick and normal formats do exactly the same amount of formatting - which is simply to initialize a relatively modest amount of sectors with a blank filesystem. The "slow" additionally does a read across the entire partition to check if it's readable, but it doesn't do any kind of write, repair or sector-reallocation triggering (at least for harddrives, sector reallocation only happens on write)... so it sounds weird that a "slow" format helped you. I'd attribute it to general pendrive flakyness :)
-
I had a feeling you'd comment. All I can tell you is I've had problems with quick format both for bare drives and USB keys. Long format fixed it up. I don't really care why.
-
Please don't subscribe to Cargo Cults :)
Doing a full format on new pendrives is a good idea anyway, as there's been more than one example of cheap Chinese brands that were labelled (and controller-configured) with a higher capacity than they actually have - which ends up giving you data corruption if you fill the drive beyond it's real capacity. (If controller simply does offset%realsize, a full format will not be enough to detect this malpractice, though - so it'd be even better to do a quickformat, fill the drive with data, and verify the data was written correctly).
-
It's probably the ghost in the machine. This isn't just cheapie pen drives but WD Caviar Black Sata III bare drives in USB 3.0 docks. None of 'em like the quick format. Maybe it's just a conspiracy by Windows Seven to waste 2 of my hours. Or perhaps the drives weren't aware of your theoretical objections. In any case, just my experience. I realize the BMW convertible crowd may be able to do things the easy way. That's how it goes.
-
lol ;D thanks guys, very helpful. I'll just leave it be for now.
-
Did a little investigation, seems things have changed (http://support.microsoft.com/kb/941961) since does-format-write-zeroes was last brought up (https://www.donationcoder.com/forum/index.php?topic=24578.msg223831#msg223831). So it actually does make sense that a non-quick format has helped you :)
XP and below, "full format" actually only does quick-format + sector verification... Vista and onwards, you get zero-overwrite (and reallocation triggering). Yay research ^_^
-
Did a little investigation, seems things have changed (http://support.microsoft.com/kb/941961) since does-format-write-zeroes was last brought up (https://www.donationcoder.com/forum/index.php?topic=24578.msg223831#msg223831). So it actually does make sense that a non-quick format has helped you :)
XP and below, "full format" actually only does quick-format + sector verification... Vista and onwards, you get zero-overwrite (and reallocation triggering). Yay research ^_^
-f0dder
Glad you found that out. I had the "feeling" it was writing something all that time. But I wasn't certain. Probably with format to the same file type the quickie is fine. But changing to another file type it may be worth it to spend the time for the full shootin' match. :)
-
I work in an environment that supports Macs as well as PCs. The Macs can read NTFS, but can't write to it (natively - you can install drivers or whatnot), but they *can* read and write Fat32.
So, all of my work USB sticks are formatted Fat32.
-
All of my USB memory devices are formatted NTFS except where other devices like cameras do not support NTFS formatted memory.
The thing to do is go into the options for the device under device manager and disable caching. Although doing this causes a performance penalty, disabling caching makes it so that removing the device without first doing so in software is less likely to corrupt data.
-
Sometimes Eject or Safely Remove does not want to function. Even though the stick will be shown as a removable drive, if you explicitly list the letter, Sync (http://technet.microsoft.com/en-us/sysinternals/bb897438.aspx) from SystemInternals should flush the data to disk.
I have USB 3.0 docks with caching enabled. I have sync.exe hooked to F10 function key. Before I'm going to shut off a dock, I hit F10 a few times to make sure the data is written. Sync works by simply locking the volume. This triggers Windows to write all data in file buffers to the drive. Couldn't be much safer. I've been using Sync.exe through many flavors of Windows.
By default just running sync.exe at a command prompt flushes all fixed disks. To use with a USB key that has write caching disabled, just specify the drive letter, like
sync F
without the ':'
The drives in my docks show as fixed disks. The fastest way to see if you have to specify the drive letter is just run sync.exe. It will report the letters of the drives flushed. If it shows your thumb, then no need to specify it.
It couldn't hurt. :)
-
^ I use Zentimo (http://zentimo.com/) (used to be usb safely remove) for that.
-
^ I use Zentimo (http://zentimo.com/) (used to be usb safely remove) for that.
-wraith808
I got a free copy of Safely Remove. It was totally inconsistent on my machines. Basically a waste of time hitting the button.
Plus you don't want to "safely remove" hardware that is seen as a fixed drive, like the bare drives in my docks. Flush buffer kill power does it. Sync.exe works on everything, even Win9x. I've been using it since 9x days.
-
^ I use Zentimo (http://zentimo.com/) (used to be usb safely remove) for that.
-wraith808
I got a free copy of Safely Remove. It was totally inconsistent on my machines. Basically a waste of time hitting the button.
Plus you don't want to "safely remove" hardware that is seen as a fixed drive, like the bare drives in my docks. Flush buffer kill power does it. Sync.exe works on everything, even Win9x. I've been using it since 9x days.
-MilesAhead
Really? What was inconsistent? I've been using it for a couple of years and have had no problems, but maybe I'm doing something dangerous? I've used it to eject usb drives and esata drives in docks.
-
Inconsistent meaning I invoked it again and again and it almost never told me the drive could now be removed. I liked the eye candy. I wouldn't have stopped using it if it worked for me.
Like many things if it works for you it's great. If not it's a lemon.
-
I found USB Safely Remove to be much better (stable) than Zentimo on Vista Business 32-bit, Zentimo hangs with errors , but USBSR worked smoothly since last 2 years, on same flash drives/external drives(USB 2.0/3.0) .
Hard drives (2.5 or 3.5") in USB enclosures are far more reliable than flash/pen drives/"sticks", esp. if the former (external hard drives) have their own power supply(ac adapter etc)
My guess: All pendrives (even 16/32/64/128/256GB versions) are not so reliable as they rely on USB socket(2.0/3.0) for power,so a (heat) glitch in motherboard (whether desktop or laptop) could affect your data. So one is forced to have multiple devices for backup, of which I recommend at least one external powered drive.
exFAT file system(Microsoft): http://support.microsoft.com/kb/955704
-Supports volumes larger than 32 GB, Theoretical max volume size 64 ZB,
recommended max volume size is 512 TB.
-Supports files larger than 4 GB, Theoretical max file size 64 ZB, recommended max file size 512 TB.
some more links on exFAT:
http://www.lifehacker.com.au/2012/07/use-the-exfat-file-system-and-never-format-your-external-drive-again/
http://www.mactalk.com.au/11/98585-exfat-not-compatible-between-windows-7-mac.html
http://www.ehow.com/how_5878184_format-exfat-formatting-windows-7.html
http://en.wikipedia.org/wiki/ExFAT
http://www.sevenforums.com/installation-setup/7627-exfat-format.html
http://www.sevenforums.com/hardware-devices/241494-problem-exfat-usb-flash-drive.html
rgds
Ska
-
The drives in my docks show as fixed disks.-MilesAhead
For an USB dock? What?!
I've seen that happen with eSata, but only if the BIOS hasn't configured the particular port used for hotswap... while you can do some hotswapping in that scenario, it's kinda flaky and dangerous. But if you're stuck with an old BIOS, you might not be able to toggle the port to hotswap mode :(
As for exFAT, I wouldn't use it unless for a specific device... devices that only support FAT don't support exFAT, so in that case you might as well use the (much) more robust NTFS.
-
But if you're stuck with an old BIOS, you might not be able to toggle the port to hotswap mode
Write caching may be disabled. But since I'm running Sata III bare drives why would I want to take a performance hit? I have battery backup on the docks.
-
But if you're stuck with an old BIOS, you might not be able to toggle the port to hotswap mode
Write caching may be disabled. But since I'm running Sata III bare drives why would I want to take a performance hit? I have battery backup on the docks.-MilesAhead
I don't think (esATA) hotswap reduces performance? - I'll have to investigate a bit, though :) Disabling write caching is a definite performance hit, though.
Is your dock USB3 or eSATA? I'd be surprised if an USB connector didn't show up as a removable device.
-
I was talking about sync.exe picking them up as fixed disks. Running it with no params flushes all fixed disks. That's all. If it does not report flushing the drive letter for the usb when run with no params, you have to specify the drive letter to force it as in
sync F
as I stated earlier
The docks are SIIG USB 3.0. WD Sata III drives freshly formatted show benchmarks at full speed. I think the fastest rated WD I have is the discontinued 640 GB 64 MB cache model. Sustained transfer is listed at 126 MB/s. Sequential read/write freshly formatted is 130/138 MB/s Crystal Disk Mark. But CDM considers 1 million bytes a MB for some reason I can't fathom. After all, they have access to a computer to do the math. :) But they seem to prefer orders of magnitude. In any case that comes close to what I'd expect.
I just got a cheap SSD on sale. I'm using it in the dock for video mux. Supposedly in a direct Sata III connection it should do 500 MB/s sequential. In the dock I'm getting 170/145 read/write. But of course the beauty is in the random access. I can use it for both video source and destination with no thrashing penalty. It's a Kingston. I think they phased it out.. reason for the sale. But I got a good price. $70 for 128 GB. This machine has a plastic cage devised by Satan for the system HD. I saw the video of the tech doing R&R of the drive. Forget it!! Besides, fast boot isn't the only thing in life. I get good use of the SSD muxing in the dock. The SSD has the same controller as the other higher priced models. I think the memory is not rated to last as long. Eight years typical use as system disk.
I haven't tried it on W7 yet with the Trim notifications. I'm using it on a Vista machine. But the garbage collector works well enough for my purposes. I did a mux with audio only encode to another video container and I kept killing the job because I changed a setting in the middle and I thought it was picking up the temp file and resuming it was so fast!! I just watch the progress bar go across.
-
In fact, now that I remember... sync.exe isn't picking up on the write caching. I had the caching disabled for a time and it still flushed them by default. It was last updated in 2006. It probably uses disk capacity to make assumptions. But since it only uses volume lock API call on NT and later I have no qualms using it still. The SysInternals stuff always seemed pretty solid to me. :)
-
I came across this article tonight, perhaps it will shed a little more light on the subject:
What's the best file system for an external drive?
Cyberknight asked the Hard Drives, NAS Drives, Storage forum if a new flash drive should be formatted in FAT32 or NTFS.
http://www.pcworld.com/article/2013758/whats-the-best-file-system-for-an-external-drive.html
-
There was a PCWorld Answer Line forum in recent days that talked about the drive writing problem with NTFS file systems on flash drives. Because of the excessive writing NTFS drives do, I reformatted my 16 Gb drives to FAT32, with 512 Kb allocation unit sizing. That takes care of the problem.
-
Does anybody have some hard numbers on how much extra write NTFS adds, and how much it actually matters - or is this the usual wishy-washy uninformed "oh, it has a journal, so it must be bad!" FUD? :)
The NTFS journalling is only done on filesystem metadata, not file data itself - if you're creating and deleting a zillion small files (which is where journal overhead might matter), you're already stressing the drive. And on the other hand, NTFS has optimizations like storing the content of very small files directly in the filesystem structures, where FAT needs to allocate separate blocks...
Flash drives tend to flake out far earlier than you hit the NAND cell max erase cycles anyway, so it's not something I'd personally worry about :-)
-
:( Pardon my screwup! The correct answer is format the drive in exFAT, then you can use 8 x 32Gb flash drives for ReadyBoost (the max allowed), or have the drive ready for very large files (the limit is 256Tb in size)! For drives 7Mb to 256Mb cluster size is 4K, for drives 256Mb to 32Gb its 32Kb and for drives 32 Gb to 256Tb its 128Kb.
-
:( Pardon my screwup! The correct answer is format the drive in exFAT, then you can use 8 x 32Gb flash drives for ReadyBoost (the max allowed), or have the drive ready for very large files (the limit is 256Tb in size)! For drives 7Mb to 256Mb cluster size is 4K, for drives 256Mb to 32Gb its 32Kb and for drives 32 Gb to 256Tb its 128Kb.
-BearPup7
Thanks!
-
https://www.donationcoder.com/forum/index.php?topic=32917.msg307456#msg307456