Move/Organize Files Into Folders

[nkormanik]

I seem to recall a program for what I'm asking.  If you know of one, please mention it.  Any and all thoughts appreciated.

I have 600,000 midi files, in a single folder.  I'd like to move/organize into folders.  100 midi files per new folder.  Sorted by size of files.

So, the smallest 100 midi files will go into, say, folder 0001.

Second 100 smallest midi files will go into folder 0002.

Etc.

Will need to create all the required folders.

Assume same drive for all actions.

Prefer to leave original filenames, though adding a prefix (suffix?) would be okay.  That's actually how I'll handle this little chore if I have to on my own.

Solution should be able to handle any sort of file, so others out there can find this topic and use solution for their own interests.

Thanks much!

Nicholas Kormanik

[MilesAhead]

I would take a look at this free file manager, MultiCommander.

It has a built in scripting mechanism.  So I am sure it could do things like move files to folder by age/size as you require.  I have not used it other than to look around it and the documentation a bit since I have so much stuff that launches Explorer windows and moves them.  But it seems quite well done.  It has both native 32 and 64 bit builds.

Please do not ask me how to use the scripting.  I have been trying to figure that out in the documentation.  But it has online forums for guidance.  It is just that I haven't spent much time with it, being sort of locked into Explorer.

[skwire]

I can do this one a little later today.  Stay tuned.

[nkormanik]

Thanks to both of you!  I'll look into the program mentioned.  Always good to have another tool handy.  Skwire, if you do have the chance and inclination, go for it.

A friend of mind wrote some C-sharp code that does the job, too, more or less as requested.  But one must edit the code a tiny bit, and then compile.  Not sure if I should somehow post his code, or what.

[MilesAhead]

Thanks to both of you!  I'll look into the program mentioned.  Always good to have another tool handy.  Skwire, if you do have the chance and inclination, go for it.

A friend of mind wrote some C-sharp code that does the job, too, more or less as requested.  But one must edit the code a tiny bit, and then compile.  Not sure if I should somehow post his code, or what.

-nkormanik (March 21, 2016, 05:09 PM)

Unless your friend posted the code somewhere publicly I would check with him/her before publishing it yourself.  If there are directories "hard wired" into the source code perhaps you can request the author get the directory paths from an .ini file.  That would remove the necessity to recompile.  I believe C# has Properties so it should be pretty easy to set default directories and other settings and make a small input box so that the user can change them while the program is running, and save for next time or whatnot.

[skwire]

Apologies, Nick, I got completely sidetracked.  Did you still need this?

[nkormanik]

Skwire, if you want to make it.  Surprisingly dividing a sorted collection into sets, say, by size, is lacking presently.  Go figure.  I'd think there would be a desire to have such functionality.

There IS the ability to separate based on other criteria, such as extension.  I think you yourself made such a program already.

[MilesAhead]

I notice MultiCommander has updated the online documentation.  There are step by step instructions for creating user defined commands etc..  One of these days I have to play with it in depth as it is supposed to have 64 bit builds with true multi-threading.

http://multicommander.com/docs

[Lintalist]

I don't know if you are familiar with AutoHotkey, a useful script language, you can find it here http://autohotkey.com/

This script should work (be sure to make a backup and be careful which folders you use)

The DIR command (MS Dos) has a useful SORT by size option so we make use of that to create a list of files sorted by size first, then read the list and parse it and create the folders when we need them to move the files into it. Again use at your own risk and be careful which folders you try it out on. You can compile this script into a exe and also have it read some settings from an INI file if that is useful by using the AutoHotkey IniRead command. You can also dress it up with a GUI and progress bars and what not, this is just a bare bones script.

Code: Autohotkey [Select]

1. #NoEnv
2. #SingleInstance, force
3. SetBatchLines, -1
4. SetWorkingDir, %A_ScriptDir%
5.  
6. Counter:=0
7. FilesPerFolder:=100
8. Folder=c:\test\ ; set folder here or uncomment the lines below by removing the ; so you can select a folder
9.  
10. ;FileSelectFolder, Folder, , 0, Select folder
11. ;If (Folder = "")
12. ;       ExitApp
13.  
14. Folder:=RegExReplace(Folder, "\\$")
15.  
16. ; make a list sorted by size, type DIR /? at a command prompt for more options.
17. RunWait, %ComSpec% /c dir %Folder% /b /OS > $$filelistsortedbysize.tmp, , Hide ; use /O-S if you want to sort from large to small
18. FileRead, FileList, $$filelistsortedbysize.tmp
19. Sleep 100
20.  
21. TargetFolder:=SubStr("00000" ++Counter, -4)
22. FileCreateDir, %Folder%\%TargetFolder%
23.  
24. Loop, Parse, FileList, `n, `r
25.         {
26.          ; MsgBox %Folder%\%A_LoopField%`n%Folder%\%TargetFolder% ; debug msgbox so we can see what is going on
27.          FileMove, %Folder%\%A_LoopField%, %Folder%\%TargetFolder%
28.          If (Mod(A_Index,FilesPerFolder) = 0)
29.                 {
30.                  TargetFolder:=SubStr("00000" ++Counter, -4)
31.                  FileCreateDir, %Folder%\%TargetFolder%
32.                 }
33.         }
34.  
35. FileDelete, $$filelistsortedbysize.tmp
36.  
37. MsgBox Done!
38.  
39. ; Esc::ExitApp ; if you want to be able to stop the script at any time by pressing Esc remove the ;

[nkormanik]

Thanks Lintalist, a lot of work you put in there.  Seems to do the job as advertised.

Skwire, unless you or someone else wants to add anything, this one looks solved.

[AbteriX]

Skwire, unless you or someone else wants to add anything, this one looks solved.

-nkormanik (March 29, 2016, 12:53 AM)

Anyway  , here a script, done with PowerShell, just for fun and because PoSh is there already.

(Tested with PoSh v4, but forced to behave like v2. Write $PSVersionTable to get your version)

I have 600,000 midi files, in a single folder. 
I'd like to move/organize into folders.  100 midi files per new folder.  Sorted by size of files.
So, the smallest 100 midi files will go into, say, folder 0001.
Second 100 smallest midi files will go into folder 0002.
Etc.
Will need to create all the required folders.

-nkormanik (March 13, 2016, 07:58 PM)

With PowerShell:


Open a PowerShell console in your "single folder" with the  "600,000 midi files".


---

If wanted, Create some test files (with random names and different size) to test my script:

Code: PowerShell [Select]

1. 1..200|%{ fsutil file createnew ([System.IO.Path]::GetRandomFileName()) $_ | Out-Null}

---

Move every ten files (sorted by size) to a new created sub folder:
(Change "-lt 10" to the wanted amount, like "-lt 100")

Code: PowerShell [Select]

1. ls|?{!$_.PSIsContainer}|sort length|%{$C=0;$I=1}{$C++;if($C-lt10){mkdir $I -ea 0|Out-Null;Move $_ $I}else{Move $_ $I;$C=0;$I++}}

---

The same Move-script, but as long version, with official command names and such:

Code: PowerShell [Select]

1. # DIR folder:
2. Get-ChildItem |
3. # Files only, no directories:
4. Where-Object{ -not $_.PSIsContainer} |
5. # Sort by file size:
6. Sort-Object -Property length |
7. # For Each File from DIR Do:
8. ForEach-Object
9.     # One time, common for all files, at the beginning, set counter:
10.     -Begin{ $FileCount=0; $LotFolder=1 }
11.     # For each individual file:
12.     -Process{ $FileCount++;
13.             # break/reset at every max count, here 10:
14.             if($FileCount -lt 10){
15.                 # Create sub folder, suppress error message on existing folder (ErrorAction), suppress success message too (>NUL):
16.                 New-Item -path $LotFolder -ItemType directory -ErrorAction SilentlyContinue|Out-Null;
17.                 # Move current file to sub folder:
18.                 Move $_ $LotFolder}
19.             else{
20.                 # On last file in set, Move current file to sub folder and adjust counter:
21.                 Move $_ $LotFolder; $FileCount=0; $LotFolder++}
22.             }
23.     # One time, common for all files, at the very end:
24.     -End {"Done!"}

.

[nkormanik]

AbteriX, I had no idea PowerShell could be so powerful.  Thanks for a masterful demonstration!  And another solution.

[nkormanik]

Here, by the way, is the C# code someone else did to solve the issue.  Author gave permission to post.:

Code: C# [Select]

1. using System;
2. using System.IO;
3. using System.Linq;
4.  
5. namespace FileOrganizer
6. {
7.     class Program
8.     {
9.         static void Main(string[] args)
10.         {
11.             // Options
12.             var filePattern = "*.url";
13.             var filesPerFolder = 10;
14.  
15.  
16.  
17.  
18.  
19.             var currentDir = new DirectoryInfo(Environment.CurrentDirectory);
20.  
21.             var files = currentDir.GetFiles(filePattern, SearchOption.AllDirectories);
22.  
23.             var sortedFiles = files.OrderBy(x => x.Length).ToList();
24.  
25.             for (var i=0; i < sortedFiles.Count; i++)
26.             {
27.                 var sourceFilePath = sortedFiles[i].FullName;
28.                 var targetFolderName = (i / filesPerFolder).ToString("X8");
29.                 var targetFolderPath = Path.Combine(Environment.CurrentDirectory, targetFolderName);
30.                 var targetFilePath = Path.Combine(targetFolderPath, sortedFiles[i].Name);
31.  
32.                 if (i % filesPerFolder == 0)
33.                 {
34.                     Directory.CreateDirectory(targetFolderPath);
35.                 }
36.  
37.                 File.Move(sourceFilePath, targetFilePath);
38.             }
39.         }
40.     }
41. }