topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • November 16, 2018, 06:42 AM
  • Proudly celebrating 13 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: IDEA: Any program required to move files to a NAS.  (Read 3807 times)

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
IDEA: Any program required to move files to a NAS.
« on: June 21, 2018, 02:17 PM »
As title states, I want a file to move a bunch of files from G:\Source on my windows 10 computer
to the following 12 hard drives on my NAS depending on the first 2-3 characters of their filename as indicated by the destination locations.
Basically I want to alphabetically sort all the files moved from G:\Source into the proper folders on the NAS.

I am sooo tired of sorting files one at a time.

If the filename already exists on the NAS I want a standard windows prompt to ask me what to do for duplicate files.
Windows 10 also prompts me Is it ok to move files without properties each time as I am moving to a Linux box.
It would be nice to either auto allow this or stop windows from asking in the first place as I want to move all files in G:\Source on my Windows 10 box.

Below is a list of my 12 NAS Drives & Destination folders: aka the first drive on the NAS is named 09 and has 2 folders named 0-3 and 4-9
All files starting with any special characters, 0,1,2 0r 3 go into the first folder on the "09" drive.
All files starting with a 4,5,6,7,8 or 9 go into the second folder on the "09" drive.

\\192.168.1.203\09\0-3
\\192.168.1.203\09\4-9

\\192.168.1.203\a\A#-AF
\\192.168.1.203\a\AG-AM
\\192.168.1.203\a\AN-AR
\\192.168.1.203\a\AS-AZ

\\192.168.1.203\b\B#-BD
\\192.168.1.203\b\BE-BH
\\192.168.1.203\b\BI-BL
\\192.168.1.203\b\BM-BZ

\\192.168.1.203\cd\C#-CAO
\\192.168.1.203\cd\CAP-CAR
\\192.168.1.203\cd\CAS-CG
\\192.168.1.203\cd\CH
\\192.168.1.203\cd\CI-CL
\\192.168.1.203\cd\CM-CO
\\192.168.1.203\cd\CP-CZ
\\192.168.1.203\cd\D#-DA
\\192.168.1.203\cd\DB-DH
\\192.168.1.203\cd\DI-DZ

\\192.168.1.203\ef\E#-EN
\\192.168.1.203\ef\EM-EZ
\\192.168.1.203\ef\F#-FI
\\192.168.1.203\ef\FJ-FZ

\\192.168.1.203\ghi\G#-GN
\\192.168.1.203\ghi\GO-GZ
\\192.168.1.203\ghi\H#-HI
\\192.168.1.203\ghi\HJ-HZ
\\192.168.1.203\ghi\I#-IM
\\192.168.1.203\ghi\IN-IZ

\\192.168.1.203\jkl\J#-JE
\\192.168.1.203\jkl\JF-JZ
\\192.168.1.203\jkl\K#-KI
\\192.168.1.203\jkl\KJ-KZ
\\192.168.1.203\jkl\L#-LA
\\192.168.1.203\jkl\LB-LE
\\192.168.1.203\jkl\LF-LZ

\\192.168.1.203\mn\M#-MD
\\192.168.1.203\mn\ME
\\192.168.1.203\mn\MF-MN
\\192.168.1.203\mn\MO
\\192.168.1.203\mn\MP-MZ
\\192.168.1.203\mn\N#-NI
\\192.168.1.203\mn\NJ-NZ

\\192.168.1.203\opqr\O#-OO
\\192.168.1.203\opqr\OP-OZ
\\192.168.1.203\opqr\P#-PG
\\192.168.1.203\opqr\PH-PL
\\192.168.1.203\opqr\PM-PZ
\\192.168.1.203\opqr\Q
\\192.168.1.203\opqr\R#-RA
\\192.168.1.203\opqr\RE
\\192.168.1.203\opqr\RF-RZ

\\192.168.1.203\s\S#-SC
\\192.168.1.203\s\SE
\\192.168.1.203\s\SF-SJ
\\192.168.1.203\s\SK-SO
\\192.168.1.203\s\SP-SS
\\192.168.1.203\s\ST
\\192.168.1.203\s\SU-SZ

\\192.168.1.203\t\T#-TH
\\192.168.1.203\t\TI-TQ
\\192.168.1.203\t\TR
\\192.168.1.203\t\TS-TZ

\\192.168.1.203\uvwxyz\U
\\192.168.1.203\uvwxyz\V
\\192.168.1.203\uvwxyz\W#-WA
\\192.168.1.203\uvwxyz\WB-WH
\\192.168.1.203\uvwxyz\WI
\\192.168.1.203\uvwxyz\WJ-WZ
\\192.168.1.203\uvwxyz\X
\\192.168.1.203\uvwxyz\y
\\192.168.1.203\uvwxyz\Z
« Last Edit: September 08, 2018, 04:22 PM by Craig381 »

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Will a Windows console or Gui application also fit?
Can you give an example from your \Source\ folder for a file that belongs to \\192.168.1.203\ghi\HJ-HZ
Somehow i cannot follow your destination paths without information of Sourcenames.
If files actually named "HJanything.123" - "HZanything.123" ignore that question.
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
KodeZwerg,

You are correct,
Files in the folder \\192.168.1.203\ghi\HJ-HZ COULD start with HJ (but none presently do)
I had to split up the files "somewhere" in the middle so the previous folder has all files starting with H to HI

examples per hard drive
\\192.168.1.203\ghi\HA-HI\H.zappy.avi
\\192.168.1.203\ghi\HA-HI\H Happy.avi

\\192.168.1.203\ghi\HJ-HZ\Hj.avi
\\192.168.1.203\ghi\HJ-HZ\Hope.avi
\\192.168.1.203\ghi\HJ-HZ\Hz Happey.avi

\\192.168.1.203\ghi\HA-HI  contains any file starting with Hnumber, Hspace, Ha, Hb, Hc, Hd, He, Hf, Hg, Hianything.anything
\\192.168.1.203\ghi\HJ-HZ contains ALL OTHER REMAINING files starting with H although most files in here start with Ho as O is the next vowel but theoretically there could be a file named Hjanything.anything in there.

The drives are literally divided by the first 2 characters of each filename with the exception of the C's \\192.168.1.203\cd\CA-CAO
\\192.168.1.203\cd\CAP-CAR and \\192.168.1.203\cd\CAS-CG as there were too many files starting with CA for one folder.

and also yes, I will accept any program that will run on windows 10 Home but I prefer a batch or .vbs script as it is the simplest thing that I know can do the job. I have not used this autohotkey thingy but anything that will move my files will do.
« Last Edit: June 21, 2018, 08:01 PM by Craig381 »

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Craig381,
i will start saturday (i am just hobbycoder), if no one has a batch/script or whatever solution you get a Gui App from me with some Progressbar and Cancel ability.
I guess a Commandline switch could be usefull for you aswell if you want to shedule that operation.
As ive read you wish a MOVE and not a COPY, just to be clear.

BTW now i understand your filesystem.
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
yes move, with probably the exception of the filenames which already exist on the NAS as a windows prompt should appear and I might select the tick box to leave the file on the NAS as it is and not overwrite it.

I could write 200-300 separate lines of code in a batch file to cover all variations of filenames and all 12 drives but before doing that I thought I would ask at this web site for a more elegant solution. I have tried wrapping my head around regex comparisons in a vbs script but it quickly became ugly with this many drives and folders.

If it helps, first working solution gets a coder donation from me!  lol

Craig

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Helllo Craig!

I am about to start this Project now, i have some questions.

Should the procedure include sub-folders in G:\SOURCE\ ? If yes, should my App delete empty folders?

Should targetfiles have original fileattributes or do you wish for example remove/set Read-Only/Hidden attributes to a specific attribute?

Must targetfiles have same Datestamp? (ATM target would get a fresh/new timestamp)

What i've already prepared in beta phase:
- Collect all Sourcefilename with Progressbar
- Initiate Copy procedure with Progressbar
- Delete all successful copied files with Progressbar

Whats missing:
- Dialog for Existing files. I could skip this by implenting a "is Source newer than Target" or simply adding a number in filename if you like.
- Bundle everything together (in beta i just test with one file)
- Remove unused debug & testing stuff and clean up source
- your Filesystem ;-)
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 38,960
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
A suggestion:
Many of us coders get very nervous writing code that deletes files -- it's the one place where a mistake can cause real harm.  It might be a good idea to make the deletion of files an option, and turn it off by default.  Or not enable it in your initial version, and instead write some messages about what files would be deleted.  Until you are really sure there are no mistakes regarding deleting files.  This goes doubly for deleting entire folders..

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Wouldnt be a problem to include from my side since i do safeway first custom-COPY then Api-DELETE instead of Api-MOVE.
(i raise a buffered copy thread with cancel ability, in my test it copy faster than api-COPY/MOVE)

edit
added "Auto-Delete" checkbox
added "Delete finished" button
added Delete-Type switch (Recycle-Bin <> instant kill)
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 12:25 AM by KodeZwerg »

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Screenshot - 23_06.jpgIDEA: Any program required to move files to a NAS. work in progress Preview
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 38,960
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Keeping in line with my suggest before, I suggest you not have an option to bypass the recycle bin -- at least until the app is well tested (and certainly don't make it bypass recycle bin by default!).  I suggest you change the option to say "Bypass recycle bin (dangerous)."

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Keeping in line with my suggest before, I suggest you not have an option to bypass the recycle bin -- at least until the app is well tested (and certainly don't make it bypass recycle bin by default!).  I suggest you change the option to say "Bypass recycle bin (dangerous)."
Thank you for your attention on that!  :Thmbsup: Screenshot - 23_06 003.jpgIDEA: Any program required to move files to a NAS. Gui Redesign, when done, i remove "craig" option and add to NANY  :tellme:

edit
another redesign done, "Recyle-Bin" option is disabled by default aslong not "Move" is checked.
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 02:58 AM by KodeZwerg »

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,288
    • View Profile
    • Donate to Member
Do you have an screenshot of the part of the config screen to set up file filters & destinations? At least, I hope that part is configurable... :huh:

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Do you have an screenshot of the part of the config screen to set up file filters & destinations? At least, I hope that part is configurable... :huh:
I might need to add such possibilties when i pump it to NANY
For now its written for Craig with "*.*" as Sourcefolder Filemask and Destinations you can read in Post #1 (still not implemented, working on it)

edit
after thinking about, i will not hardcode Craigs filesystem instead i will add a rule-set editor for both, Source and Destination.
For Source:
- edit filemasks - limited to extensions (eg: ".hlp;.chm"), for now i do not plan to add "?" / "*" support (those are placeholders for any content) except "*.*"
- "include/exclude" by "datetime/attribute/name"
For Destination:
- "if filename begin/contain" then "copy to specific folder" (with folder-selector)
(Craigs Filesystem will be included as example)

If i missed other useful rules feel free to post wishes!
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 04:20 AM by KodeZwerg »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 38,960
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Another security thing you might add is, because you are copying to a network drive, not just check that the copy operation succeeded, but actually check that the destination file exists (and maybe is the same size)?  I'm not trying to make more work for you, I'm just being paranoid when it comes to anything that involves deleting files.  The good part is this is a custom app for one person so there's not too much room to do harm.

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Good point and sorry i had edited my last post wich neglate "The good part is this is a custom app for one person so there's not too much room to do harm.".
Now it will be an Application for everyone.

It makes no trouble to do "fileexists()" checks, but since its a custom-Copy (ie: i write data by myself) i know at any moment whats good and whats not.
For paranoids i can add Filehash checks (CRC16/CRC32/CRC64/MD4/MD5 shouldnt slow down progress much, depends on drivetempo/connection speed)

Heres a Preview of current main Gui Screenshot - 23_06 003.jpgIDEA: Any program required to move files to a NAS. now working on Rule-Set design

Short update: added UAC controlled "Begin" button with shield symbol. If running without admin-rights my program will ask to elevate. Screenshot - 23_06 005.jpg

More News: Due to UAC integration (not with Manifest, at runtime!) i have to redesign the whole copy process. Now main gui is shrinked and a new Window appear for file copy progress. Alot hidden stuff must be ported now to work with UAC correct, i am sorry!
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 02:10 PM by KodeZwerg »

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
OMG Im so excited.... is there a working version yet?

To answer your questions:

I am about to start this Project now, i have some questions.

Should the procedure include sub-folders in G:\SOURCE\ ? If yes, should my App delete empty folders?
If there were to be folders in G:\Source the folder would become the "sort" criteria for moving the entire folder.
aka the first 2-3 letters of the folder name. soooo   "Eat me" folder and its entire contents regardless of the names of the folder contents would
presently be moved to \\192.168.1.203\ef\EA-EN  leaving nothing back in G:\Source

Should targetfiles have original fileattributes or do you wish for example remove/set Read-Only/Hidden attributes to a specific attribute?
attributes do not matter as they are dictated by the NAS OS based on which user is connected.
The only problem/annoyance I have when moving files now is windows nags about attributes every time I move files to the Linux NAS.

Must targetfiles have same Datestamp? (ATM target would get a fresh/new timestamp)
that's fine for datestamp

Oh and concerning this issue:
- Dialog for Existing files. I could skip this by implenting a "is Source newer than Target" or simply adding a number in filename if you like.
A tick box to "overwrite destination file if source file is larger" would be nice otherwise presently I let the windows let me decide thingy come up and I decide on each file sometimes keeping a different file extension but most times replacing the destination file if the new one is larger.

The ability to change destinations is awesome as I might add another hard drive or split up one of the folders in the future.
What I do now on my NAS is split a folder when the number of files in each folder gets too large for viewing on a couple screens.
aka: when it gets over 90
Plus every time a hard drive gets full I add another drive and move some of the folders from the above and below drives to the new empty drive.
(NAS currently has 12 drives can physically hold 16 max.)
« Last Edit: June 23, 2018, 03:04 PM by Craig381 »

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
ASAP i upload my initial "one file copy demonstration" for testing purposes, i wait for a reply from administrator. (i have upload problems, Website timeout issue)

Right now i must start from scratch the copy process to a new UAC friendly version, about 70% done.
Then the main part starts, the Rule-Editor.... i have many design ideas, i must be sure that everything works well together, needs alot of checking, very critical part this is!

The final product should be a complete copy-station, with or without rules.
Thank you for answering but as you could see on snapshot i include every critical part per checkbox so user always have a bit control over what should be done in wich way.

If you miss more options, just tell and i try to add if possible.

edit
Fileexists Dialog will contain:
- Source Timestamp & Size
- Target Timestamp & Size
Combobox Options with
- Overwrite
- Overwrite older/newer
- Overwrite smaller/bigger
Checkbox
- Apply to all
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 03:34 PM by KodeZwerg »

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
Awesome like what I see so far!!

The basic rule of the first 2-3 characters of each file/folder name dictate exactly where the file/folder is to be moved on the NAS never changes.
Only the actual name of each folder and the number and size of drives on the NAS can change.
This is to allow for future expansion as each folder/drive becomes too full.

which brings up another point...  if the actual destination folder names do NOT agree with the present rule set there should be a warning or a way to verify/check for ruleset NAS folder agreement.

DANG!!!   even better have a button which READS the NAS and makes up a rule set depending on the actual shares/folders on the destination itself???
or maybe the program automatically does this anyways and the folders/drives on the NAS can change at will and still sort/move the files correctly?
« Last Edit: June 23, 2018, 03:52 PM by Craig381 »

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
Awesome like what I see so far!!
Nice that you like it. Makes me happy!
This is to allow for future expansion as each folder/drive becomes too full.
The idea with my rule-set would be:
You set-up rules one time (as promised i include your set for example on how to)
The variable in this case is destination, for example:
Destination: \\192.168.1.203\
a file named "EEanything.123" is current task,
rule-set will identify that it belongs to "ef\EA-EN" folder on \\192.168.1.203\
if \\192.168.1.203\ changes to \\192.168.1.204\ it will use "ef\EA-EN" on it.
or any other sub-folder you enter at line "Destination" like "\\192.168.1.203\somewhere else\"
than file "EEanything.123" would land in "\\192.168.1.203\somewhere else\ef\EA-EN"

Source Folder-Names are not part of rule-set. Only Filenames. I might extend when filenames are done.
For now if "G:\Source\Eat me\a file.ext" would be found, rule-set should give "\\192.168.1.203\a\AA-AF" as destination path.

DANG!!!   even better have a button which READS the NAS and makes up a rule set depending on the actual shares/folders on the destination itself???
I have to think about this and its possibility to include such.

>>"which brings up another point...  if the actual destination folder names do NOT agree with the present rule set there should be a warning or a way to verify/check for ruleset NAS folder agreement."
ATM all what doesnt match with rule-set land in Destinations root or i skip them.... unsure but planned is that:
I will add later another Dialog showing all unmatched files to let user add either a rule or let you choose what to do. I havent really thought much yet about this because many main parts arent finished yet.
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 23, 2018, 04:11 PM by KodeZwerg »

Craig381

  • Supporting Member
  • Joined in 2018
  • **
  • default avatar
  • Posts: 13
    • View Profile
    • Donate to Member
Excellent, subfolders within each current folder I previously listed are presently far and few between on the NAS anyways and I can always move them manually. (Not an initial requirement I just got greedy!!)

As far as the ruleset goes, I only change the folder names and add drives once every 3-4 months anyways so its all good!

thank you as this gui version will be way better/versatile than a hard coded/hard to change batch file that I was working on.

PS: doesnt match with rule-set files could have a tick box to skip them OR move to root of closest destination.
either way is fine by me.

Craig
« Last Edit: June 23, 2018, 05:02 PM by Craig381 »

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,288
    • View Profile
    • Donate to Member
Should targetfiles have original fileattributes or do you wish for example remove/set Read-Only/Hidden attributes to a specific attribute?
If there are options to change attributes after copying, then those should apply, but otherwise, not preserving attributes (except the Archive bit) I'd probably report as a bug...

Must targetfiles have same Datestamp? (ATM target would get a fresh/new timestamp)
As this is supposed to be a copy/move tool, IMHO, it should be compatible with Windows, and keep the original timestamps like a normal copy would. Again, I'd report it as a bug if it didn't... despite the limited value of file timestamps.

Different but related topic:
Have you thought about the Alternate Data Streams (ADS) that can be stored on the NTFS filesystem? (Available since Windows 2000) They should be copied/moved when the destination filesystem is also NTFS, and probably/optionally give a warning or not delete the source when moving, when ADS's are found. A simple "dir /r" in a cmd prompt will show any ADS's for files that have them (try that in your Downloads folder..., NirSoft's AlternateStreamViewer can show you their content)

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,288
    • View Profile
    • Donate to Member
I want a file to move a bunch of files from G:\Source on my windows 10 computer
to the following 12 hard drives on my NAS
A question that I should have formulated earlier:
Why haven't you configured your NAS to present all those harddrives as a single logical volume? Optionally adding (checksum) redundancy by configuring it to use RAID? Several RAID configurations can usually be selected in most NAS systems, even if only running from software instead of a hardware RAID controller. If you want to maximize storage availability, JBOD instead of RAID could have solved this too.

You might want to share your NAS make and model so help on this subject can be provided on request.

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
If there are options to change attributes after copying, then those should apply, but otherwise, not preserving attributes (except the Archive bit) I'd probably report as a bug...
As this is supposed to be a copy/move tool, IMHO, it should be compatible with Windows, and keep the original timestamps like a normal copy would. Again, I'd report it as a bug if it didn't... despite the limited value of file timestamps.
Hello Ath, thank you for telling, i would like to explain how i do:
I do copy only files content (ie: i read in a stream and write that to target, it is a 1:1 copy of original content, realized with Delphi Filestream)
I will take your bugs and squish them by implementing more Copy Methods (Winapi: CopyFile; MoveFile; SHFileOperation), would that satisfy?
Since my method use Streams to read/write, original attributes and timestamp are lost with my own method, implemented checkboxes to set them back to original what cost performance (i need to touch everything twice), so i let user define. (by default all checkboxes are set to give most compatible target)

If you wonder why i use Streams for read/write, simple answer:
1. copy progress is faster than Winapi
2. i can react on bad-sectors, Winapi would fail, my method can atleast skip bad-parts (while let user decide what to do on read errors)
to clarify:
first i try using normal stream to read/write, on read errors i switch to hardware mode wich is like:
read file direct from partition, byte by byte and skipping bad. This feature is alpha phase, fragmentation is my enemy :-)
Different but related topic:
Have you thought about the Alternate Data Streams (ADS) that can be stored on the NTFS filesystem? (Available since Windows 2000) They should be copied/moved when the destination filesystem is also NTFS, and probably/optionally give a warning or not delete the source when moving, when ADS's are found. A simple "dir /r" in a cmd prompt will show any ADS's for files that have them (try that in your Downloads folder..., NirSoft's AlternateStreamViewer can show you their content)
My only experience with ADS is to remove them. Eg: Download a .exe file with InternetExplorer, when you try to run .exe your Windows is telling that file might be unsafe because its downloaded, my ADS-Remover would fix that situation, no more "file might unsafe" warnings, more experience i havent made yet! I will investigate your linked resource and see what i can do. Thank you!

My App is not intended to be a "Cloner" yet (additional NTFS Security Attributes, Windows GUIDs, other OS-Specific Information are not handled yet!)
For now it realy just do 1:1 content copies. My App handle "1:1 content, Standard-file attributes, Datetimestamp"
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 24, 2018, 05:16 AM by KodeZwerg »

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,288
    • View Profile
    • Donate to Member
If you wonder why i use Streams for read/write
Well, though I appreciate your effort, I'm on the same page as @mouser: I'd like to copy/move the file as safe as possible, so using Operating System native tools has my preference. Even if that means I have to use other tools for copying files from a damaged source. In that case there are wholly different problems to solve :tellme:
Offering an option to use SHFileOperation calls will be my standard setting, might I need/use this tool.

Transferring attributes and timestamps when moving/copying files is still a priority (in my case: requirement, with exceptions as stated in a previous post), independent of the way the file is copied. The few milliseconds of setting them is almost neglectable compared to the time transferring the data, or updating the UI (appending a line into a text control?) for that matter.

KodeZwerg

  • Honorary Member
  • Joined in 2018
  • **
  • Posts: 317
    • View Profile
    • Donate to Member
I will respect any comment and wishes! Main Gui is now destroyed, i create a complete new one wich has
a) better structure
b) include a real configuration dialog where you setup your personal defaults

I might skip for now bad-sector reading to have a working sample finally ready and then re-work on everything.
Configuration for several copy methods (including my own) aint a challenge and as default it will run with native OS Apis, no problem.
I need to test a bit with native Api how good it can handle (super) hidden files.

Second thing what i cant tell yet if i am able to use Progressbar with Api.... need to figure such things out.
For "Full Progress" it isnt a problem since it just counts files down.

Middle next week a first public-working-demo-version should be avail. What i have tested so far work well. (each function tested solo in own apps)
Rule-Set is now a simple String-Grid with Add/Remove/Edit abilities, still not satisfy myself, i hate to design such things but i think after preview version is out i can redesign things that bother myself and respect every other following comment for it.

Later on my Todo list will follow "Profiles" to setup several jobs. Maby add bass.dll interface to let users play background music, but thats just gimmicks for very much later if at all.

Not a promise but at end, when everything works like it should be, i planned to transform everything to non-Vcl pure Api designed (.exe size will shrink down to 50-100kb)
PS: Vcl stands for Visual Component Library, its a part of Delphi. All visible content belongs to it (Controls)
sorry bad english and Delphi are my hobby ;)
politeness is not one of my strengths in writing, just because it sounds rough doesn't mean that I mean it rough.
« Last Edit: June 24, 2018, 01:41 PM by KodeZwerg »