IDEA: Command-line tool to move dir contents to existing subdirs

[Cavalcader]

This is one I've been needing for years, and I bet others could use it too (maybe with slight variations). I've got a big directory (folder) containing a hodgepodge of downloads. There're already subdirectories in the same dir, holding grouped files with a common theme. What I'd like to do is move files with a specific part of their name matching an existing subdir to those subdirs. This would ideally support delimiters, or some way to tell the program what part of the filename to use.

A mass-move of zip or EXE files isn't what I'm thinking of here -- more like "* saver*.*" being moved to a dir named "*saver*" (which would break out to savers, screensavers, or something similar -- although that particular wildcard would also include VersaVerter if it exists, which is the name of a cool measurement converter).

If there're multiple "hits", it would it would ideally return a numbered list of candidates, where you could just type the number and hit enter, but if that's a deal-breaker then it could just be whichever the program finds first.

Whatcha think? It sounds simple, but maybe there're coding issues I haven't thought of.

[brotherS]

Whatcha think? It sounds simple, but maybe there're coding issues I haven't thought of.

Hmm... are you really downloading dozens of installers etc. that could be auto-categorized each day?

If you read http://www.donationcoder.com/forum/index.php?topic=1167.0 you know that I love to automate stuff, but I just add the version number to installers that I download and move the stuff around manually.

[mouser]

my memory is this came up in another discussion (in a renamer discussion?) and someone pointed us to a nice app for doing this.

[brotherS]

my memory is this came up in another discussion (in a renamer discussion?) and someone pointed us to a nice app for doing this.

I remember that too, but if I recall correctly it was a tool that created subfolders based on file names and moved the files in those folders.

Not sure... 

[skrommel]

I think CacheSort at http://www.donationcoder.com/forum/index.php?topic=1592.0 could be changed into what you want. Do you need a complete RegEx, or are wildcards sufficient?

Skrommel

[Cavalcader]

Hmm... are you really downloading dozens of installers etc. that could be auto-categorized each day?

No -- just trying to figure out a way to clear out a directory with files that are orphaned. Currently: 15,497 files, 1255 folders.

my memory is this came up in another discussion (in a renamer discussion?) and someone pointed us to a nice app for doing this.

That was me.   But BrotherS is right; it makes subfolders based on file names and moved the files into those folders. The result is that it leaves orphans when at least one of two things happen: 1. you get new files, or 2. you have subtle variations on the filename.

I keep the threshold set for two or more files to move, because if I let it act on just one then I end up with a bunch of dirs with a single file in each. The developer of the app I mentioned ( Directify!) has been swamped and so hasn't updated the tool from its 0.2 version in about a year and a half. 

I thinkCacheSort at http://www.donationcoder.com/forum/index.php?topic=1592.0 could be changed into what you want. Do you need a complete RegEx, or are wildcards sufficient?

That's a great idea! Wildcards would be a great improvement over what I have now, heh! But if even a primitive RegEx is possible, that would allow much more control. Whatever you have time to mess with would be gratefully received.   Even better is if there's a way for the app to make matches with less input, but I suspect that'd massively increase the size of the coding project.

The main issue is to match existing files to existing subdirs -- to join them with the rest of their type. I've got a boatload of utilities gathered over the years, mostly freeware (and I used to be a real regular at download sites, so I've seen pretty much every type of variation that's out there) but I've never seen anything that could make sense out of hundreds or thousands of orphan files that have mates in subdirs.

The only other consideration (just thought of it) is to somehow deal with filename collisions -- I guess "overwrite (yes/no/all/none/cancel)" is ideal if possible. If it uses a GUI then Windows XP and later already allow for a shift-click on the "no" button as "no to all" for the standard dialog.

[lanux128]

Cavalcader: until the gurus of AHK come up with a better solution, why don't you try this -> File Sieve

File Sieve is a small tool that can take the files from a directory and copy or move them into a new folder, while creating subfolders from A-Z and sorting the files into them based on their name (first letter). The program can come handy if you want to organize files for archiving, or to put them on a CD.

Source: www.snapfiles.com

[Cavalcader]

Thanks, lanux128 -- it's a good idea, and I'd seen similar utilities, but it always struck me that an A-Z designation would tell me little more than the standard Windows alphabetic sort does.

Still, I'm sure that kind of function has its place, especially if you're making shareware CDs for a computer club or something! The program above has lots of potential -- I love the "smart creation" idea, where only folders are created that would contain files. But it would be a rare occasion indeed for someone to want empty A-Z directories.

Edit: I did some digging, and there's a version 3 beta 3 which seems to be getting much closer to what I'm looking for.

There's a new Delimiter mode that "creates a directory based on a delimiter within an item's name. Depending on the mode selected, the directory name is derived from an item's name as read from the start of the string up until the delimiter character, or it's read from the delimiter until the end of the name."

Another new mode is Substring:

Spoiler (click to show/hide)

Purpose
    Destination directories are created from a substring contained within the
    name of the processed items in specification to the options set below.

Options

    StartIndex
        This is the position of the first character to start at.

    Length
        This is the number of characters to return.

    IncludeExtension
        If True, then the extension will be taken into account when
        obtaining a substring from an item's name.

    Catch-All Directory
        If a substring could not be obtained from an item's name,
        the item will be placed into this directory.   

Example
    Item Names
        VoiceMate Professional.exe
        Hydrogen.exe
        Poo.rar

    Options
        StartIndex: 1
        Length: 5
        IncludeExtension: False

    Result
        \Voice\VoiceMate Professional.exe
        \Hydro\Hydrogen.exe
        \Poo\Poo.rar

There's a Words mode too.

Spoiler (click to show/hide)

Purpose
    Items are sorted into directories based on the word(s) contained within
    their name. The worded directories they are placed in depends on the
    options set below.

Options
    First Word
        This is the number of the first word to use.

    Total Words
        This is the number of words that should be used for
        creating the destination directory, starting from First Word.

    Include Extension
        If True, the extension of any files will be taken into account
        as being an actual word itself.

    TrimSpaces
        Once the item's name has been broken down into words,
        they may or may not contain extra spaces at either the start
        or end of their names. Setting this option to True will remove
        these spaces.

    TrimCharacters
        Any characters specified here will be trimmed from the start
        and end of any words. Please note that each character is
        treated separately and not as a string. So for example, entering
        test will not remove "test" from the start and end of words, but
        instead from the characters t, e, s, and t from the start and end.

        Spaces cannot be specified on their own, this is why the
        TrimSpaces option exists.

    Catch-All Directory
        If a word could not be obtained from the item's name,
        then it will be placed into this directory.

Examples
    Item names
        My Test File.exe
        Another Test File Here.exe
        And Finally The Last One.rar

    Options
        First Word: 2
        Total Words: 2
        Include Extension: False

    Result
        Test File\My Test File.exe
        Test File\Another Test File Here.exe
        Finally The\And Finally The Last One.rar

I can see I'm going to have to spend some time experimenting with this puppy. I still don't think it does the matching to already-present subdirs, though, but thanks for the tip!

[app103]

As someone that recently manually sorted 30G of songs by artist, and then by album, while merging the contents of 2 pc's to a 3rd pc, I really could have used a tool like this myself...and I know plenty more would have a use for it too.

Maybe I will wait and see how this goes before attempting to merge the 2 graphics collections.

Great idea, Cavalcader!

[Cavalcader]

Thanks, app103! Lanux128, I spent some time with it, and for what I'm describing it's not quite as powerful as the Directify! tool except that it's got a GUI. It will create a boatload of directories, but it doesn't have a way to only create dirs if there's more than X number of candidates, and it doesn't have any way to fix the results (take orphaned, single-file dirs and group 'em again). The result is that you get a lot of dirs, but no telling how the contents are grouped. The preview or "simulation" mode is helpful, but there's no way to sort it. It defaults to the existing alpha sort based on the original paths, which isn't much help when you're trying out new paths to see how well they work. 

That said, it's got some other really powerful features, including being able to recurse into subdirs for its operations. It'll solve a lot of file "housekeeping" chores, but not the one described in this thread (at least not yet). It may end up including this capability as well, but I noticed two things that make me wonder how far it'll go: there's no way to edit the existing "methods" (like A-Z, delimiter, word, etc.) or create new ones, although there's a mention in the help file that it might be added to a future version. That may be a pretty trivial consideration, but I also noted that the discussion form referred to in the help file from January of this year is closed, at least temporarily. This says to me that the author may have run into some personal issues taking him away from coding and his website for now. (Then again, I could be way off!)

Lotsa hobbyist (or even pro) coders tend to suffer burnout after awhile -- it's tough work on a good day, wrestling with attempts at logical code and dealing with cryptic error messages, operating system or development system quirks, and who knows what else. When you jump online and try to interact with the world along the way (feature requests, help demands and so on -- some people aren't even nice about their requests and won't give enough info about their situation) it can be a heckuva lot to deal with. Here's hoping he's able to keep going with the project!

[lanux128]

i'm glad you liked it, Cavalcader...
yes i agree. it is very difficult for a programmer who codes as a hobby to keep the program updated as frequent as we'd like it to be...

[skrommel]

Try FileToFolder!

I've barely tested it, so please try it out on something unimportant! To use it, save to FileToFolder.ahk and go to www.autohotkey.com and download AutoHotkey.


FileToFolder - Moves files to subfolders, based on their name

Syntax: FileToFolder "<BaseFolder>" "<SubFolder>" <Overwrite> <Copy> "<PartOfFilename>" ["<PartOfFilename>" "++<PartOfFilname>" "--<PartOfFilname>"]

Example: FileToFolder "C:\Download" "Antivirus" "anti" "vir" "++.exe" "--.zip"

  Moves all files containing anti or vir, and containing .exe and not containing .zip
  from C:\Download to C:\Download\Antivirus

  ++ means the filname must contain the pattern
  -- means the filname must not contain the pattern
  Include the word Overwrite to replace existing files.
  Include the word Copy to copy files instead of moving them.


Skrommel

Code

;FileToFolder.ahk
; Moves files to subfolders, based on their name
;Skrommel @2006

If 0<3
  Goto,ABOUT

basefolder=%1%
subfolder=%2%
ormatch=
andmatch=
notmatch=
overwrite=0
copy=0

Loop,%0%
{
  If (A_Index<3)
    Continue
  param:=%A_Index%
  If (param="Overwrite")
  {
    overwrite=1
    Continue
  }
  If (param="Copy")
  {
    Continue
  }  StringLeft,action,param,2
  If (action="++")
  {
    StringTrimLeft,param,param,2
    andmatch=%andmatch%%param%`,
  }
  Else
  If (action="--")
  {
    StringTrimLeft,param,param,2
    notmatch=%notmatch%%param%`,
  }
  Else
    ormatch=%ormatch%%param%`,
}
StringTrimRight,ormatch,ormatch,1
StringTrimRight,andmatch,andmatch,1
StringTrimRight,notmatch,notmatch,1

StringSplit,andmatch,andmatch,`,

count=0
Loop,%basefolder%\*.*,0,0
{
  If A_LoopFileName Not Contains %ormatch%
    Continue
  If A_LoopFileName Contains %notmatch%
    Continue
  Loop,%andmatch0%
  {
    break=0
    pattern:=andmatch%A_Index%
    If A_LoopFileName Not Contains %pattern%
    {
      break=1
      Break
    }
  }
  If break=1
    Continue
  TrayTip,Moving %basefolder%\%subfolder%\%A_LoopFileName%
  count+=1
  IfNotExist,%basefolder%\%subfolder%
    FileCreateDir,%basefolder%\%subfolder%
  If copy=1
    FileCopy,%basefolder%\%A_LoopFileName%,%basefolder%\%subfolder%\%A_LoopFileName%,%overwrite%
  Else
    FileMove,%basefolder%\%A_LoopFileName%,%basefolder%\%subfolder%\%A_LoopFileName%,%overwrite%
}
If copy=1
  MsgBox,0,1 Hour Software by Skrommel - www.dcmembers.com\skrommel,FileToFolder copied %movecount% file(s)
Else
  MsgBox,0,1 Hour Software by Skrommel - www.dcmembers.com\skrommel,FileToFolder moved %movecount% file(s)
Return

ABOUT:
about=FileToFolder
about=%about%`nMoves files to subfolders, based on their name
about=%about%`n
about=%about%`nSyntax: FileToFolder "<BaseFolder>" "<SubFolder>" <Overwrite> <Copy> "<PartOfFilename>" ["<PartOfFilename>" "++<PartOfFilname>" "--<PartOfFilname>"]
about=%about%`n
about=%about%`nExample: FileToFolder "C:\Download" "Antivirus" "anti" "vir" "++.exe" "--.zip"
about=%about%`n
about=%about%`n  Moves all files containing anti or vir, and containing .exe and not containing .zip
about=%about%`n  from C:\Download to C:\Download\Antivirus
about=%about%`n
about=%about%`n  ++ means the filname must contain the pattern
about=%about%`n  -- means the filname must not contain the pattern
about=%about%`n  Include the word Overwrite to replace existing files.
about=%about%`n  Include the word Copy to copy files instead of moving them.
about=%about%`n
about=%about%`nFor more tools and information, please stop by at
about=%about%`n  http://www.donationcoder.com/Software/Skrommel
about=%about%`n
about=%about%`n1 Hour Software by Skrommel @2006
MsgBox,0,FileToFolder,%about%
Return

[skrommel]

I just fineread your post, Cavalcader, and you did actually request a somewhat different tool, with automatic matching of subfolders... But should it use the Or or the And parameter for matching? Or a parameter on it's own?

Skrommel

[AbteriX]

*harrumph*

Reading this thread i am thinking about my start menu with a lot of items 
I have clean up them bevor by hand and have move every folder
into an subfolder with his intial char.
Reading this thread i got fun to play around with AHK
and automat my next claening action with an script 

So i have writen this over the day in my rest times. (i call it 'Stefans 8 hour software'  )

I am no pro! So use this on your own risk and with test files first!!!

The script works in that directory from where it is executed.
There are NO exceptions.... just create a dir (if not already exists) with an inital from 0-9 and from A- Z or smtg like '_'
and move ALL files/folders into that folders. That's all.

The script with comments

; Move To Initial
; Autor: Stefan for www.DonationCoder.com
; Version: 0.1 , 2006-04-25
; Purpose: Move all files and folders of the current directory to an sub folder depending on the initial char of the file/folder name.
; :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

; :::  S E T T I N G S :::
; ::: Overwrite existing files or cancel the move ?
FileMoveDirOverwrite=0
FileMoveOverwrite=0
; 0 (default): Do not overwrite existing files/folders. (Cancel the current move and Skip the current file/folder. You have to do some work manually)
; 1: Overwrite existing files. (Maybe you overwrite older files/folders you wanna keep? Be carefull with this option!)
;:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::


; ::: S C R I P T :::

; lines starting with a semicolon is only comment for your pleasure.

;Loop, FilePattern [, IncludeFolders?, Recurse?]
Loop, *.*,1,0
{

; skip the script-file itselfs, to not move it too.
; IF current LOOP the same as the name of the script ... GoTo start
IF A_LoopFileName=%A_ScriptName%
{
continue
; 'continue' means "Skip current loop and GoTo start"
}

; skip folder/file with name less then 2 char, to avoid loops (i.e. don't move e.g. folder 'A' to folder 'A'...)
;StringLen, OutputVar, InputVar
StringLen, Len, A_LoopFileName
IF (%LEN% < 2) continue


; we need only the first char of the folder/file name to create an new folder with this initial.
;StringLeft, OutputVar, InputVar, Count
StringLeft, InitialDir, A_LoopFileName, 1


; IF NOT ALREADY EXIST %InitialDir% MAKE A NEW folder %InitialDir%
IfNotExist,%InitialDir% 
{
FileCreateDir, %InitialDir%
}

; MOVE Folder with name %A_LoopFileName% TO %InitialDir%
;FileMoveDir, Source, Dest [, Flag]
FileMoveDir, %A_LoopFileName%, %InitialDir%\%A_LoopFileName%  , %FileMoveDirOverwrite%

; MOVE File with name %A_LoopFileName% INTO %InitialDir%
;FileMove, SourcePattern, DestPattern [, Flag]
FileMove, %A_LoopFileName%, %InitialDir% , %FileMoveOverwrite%

;Esc::ExitApp

; go to the begin and execute the next Loop
}

The pur script

; Move To Initial
; Autor: Stefan for www.DonationCoder.com
; Version: 0.1 , 2006-04-25
; Purpose: Move all files and folders of the current directory to an sub folder depending on the initial char of the file/folder name.
; :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

; :::  S E T T I N G S :::
; ::: Overwrite existing files or cancel the move ?
FileMoveDirOverwrite=0
FileMoveOverwrite=0
; 0 (default): Do not overwrite existing files/folders.
; 1: Overwrite existing files. (Maybe you overwrite older files/folders you wanna keep?)
;:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::


; ::: S C R I P T :::
Loop, *.*,1,0
{
IF A_LoopFileName=%A_ScriptName%
{
continue
}
StringLen, Len, A_LoopFileName
IF (%LEN% < 2) continue
StringLeft, InitialDir, A_LoopFileName, 1
IfNotExist,%InitialDir% 
{
FileCreateDir, %InitialDir%
}
FileMoveDir, %A_LoopFileName%, %InitialDir%\%A_LoopFileName%  , %FileMoveDirOverwrite%
FileMove, %A_LoopFileName%, %InitialDir% , %FileMoveOverwrite%

;Esc::ExitApp

}

The compiled exe (non overwrite version)

MoveToInitial.exe (199.9 KB)

----
EDIT:
oh, i have overseen to finish the cancel option (ESC::ExitApp), i will try how to implement this option later.
(This option is for cancel the script if it is executed by accident or run into trouble)

[Cavalcader]

should it use the Or or the And parameter for matching? Or a parameter on it's own?

Hmm, I'm not sure -- do you mean when moving filenames to existing folder named "antivirus" that match the pattern [ANTI [i]or[/i] VIRUS] vs. [ANTI [i]and[/i] VIRUS]?

Maybe a less complicated example is to sort music files as suggested by "app103" -- let's use that as a scenario. It could read a dir and look for very close similarities to existing subdirs, such that a subdir named Bing Crosby would match:

• Bing Crosby - song name.ogg
• Bing Crosby and David Bowie - Little Drummer Boy.wav
• Bob Hope & Bing Crosby - vaudeville sketch.mp3

This type of grouping might be made easier by the standardization of the naming convention; you'd have
  artist space dash space title dot extension
...one can use a delimiter of " - " to make parsing easier and group by artist.

Another application using the original screensaver idea would be, using a subdir named Savers:

• spinning globe saver.zip
• screensaver switcher.exe
• savers to download.txt

In this case, you'd have to somehow tell it to match [root-word]* (resulting in saver* for that one) to have 'em all grouped.

Does that help?

Reading this thread i am thinking about my start menu with a lot of items

Good idea! But the program we talked about above, File Sieve, sounds perfect for that task.
By the way, I like your avatar! Is that a particular comic character, or did you make it?

[AbteriX]

Reading this thread i am thinking about my start menu with a lot of items

Good idea! But the program we talked about above, File Sieve, sounds perfect for that task.
By the way, I like your avatar! Is that a particular comic character, or did you make it?

Hi Cavalcader 
I make this just for fun, that was an task i could manage with my skill 
The Avatar i found somewhere over the rainbow  {i.e. in the net} (reminds me to the 'Gorillaz', or how is this band named *scratchingmyhead* ?)

[Cavalcader]

I make this just for fun, that was an task i could manage with my skill 

I understand. It's good to have a project to practice with, and it's definitely a useful function!

[Cavalcader]

Hey, Skrommel, I think I've got an easier approach.    It's a bit of a kludge, but the end result is the same as described in previous posts. Would it be possible to take the DelEmpty program and add a function to move the files from any folders containing single files to their parent dir and then delete the original (now empty) dir? That should be a heckuva lot smaller project (hopefully) than the original idea. 

[Cavalcader]

...Was it something I said?

[skrommel]

Should be no problem.

Skromme

[Cavalcader]

Awesome! That'll be a huge help for drive "housekeeping". 

Edit: since this will be a mod of the DelEmpty script, I'm locking this topic for eventual disposal.