DonationCoder.com Forum

DonationCoder.com Software => Coding Snacks => Finished Programs => Topic started by: mickblu on March 03, 2010, 09:54 PM

Title: DONE: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 03, 2010, 09:54 PM
I am looking for a very specific organizational ability, namely this:

I have a huge directory (51GB and growing) of images (but I wouldn't want the solution to be image-file specific).

I need to parse the filenames in this directory using standard grep / wildcard rules and then create a series of file folders labeled as per the parsing rule and move all matching files into that subfolder. So for example, if I had a range of files called

BobsPhoto_ABC_0001.jpg    through    BobsPhoto_ABC_0251.jpg

and then similar files but for different names with varying numbers of files, thus

BobsPhoto_AAB_0001.jpg    through    BobsPhoto_AAB_0147.jpg
BobsPhoto_BFG_0011.jpg    through    BobsPhoto_BFG_0025.jpg
BobsPhoto_XMC_0009.jpg    through    BobsPhoto_XMC_4252.jpg
etc.

I would like all the ABC files in the .\ABC folder, all the AAB files in the .\AAB folder etc. or wherever I specified the folders should go.

In an absolutely perfect world there would be a csv file containing

'ABC','Smith, Mary'
'AAC','Brown, Joan"
etc.

and it would create the destination directories according to those rules (if the csv file is present, otherwise name the directories ABC, AAC etc)

If someone knows of a readily available solution (preferably freeware! :Thmbsup:) I'd love to hear about it. All my Googling has produced numerous solutions, none of which do quite what I want.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 03, 2010, 10:50 PM
Are all the original files in one source folder?
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 03, 2010, 10:55 PM
Yes they are.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 03, 2010, 11:41 PM
By chance, have you checked out Skrommel's MoveOut?

https://www.donationcoder.com/Software/Skrommel/index.html#MoveOut

If that doesn't do it for you, I'll write you a custom app.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 04, 2010, 11:56 AM
Hi skwire,

I checked out Moveout and it is indeed closer than much else I have found other than perhaps RoboBasket (http://www.easy-to-use-software.com/) whose author says it can do the job. He said (In your case you can use the "Move to" action and check the "sort into subfolder" option , insert the "name.part2"  substring into the text box. ), but it certainly isn't intuitive as to why I need to do what he said (nowhere is defined what the name, name.part1, name.part2, name.part3, name.lastpart substrings refer to or even where or how one can define them). The software should (if it's designed properly) not need a manual for all but the intellectually challenged  :-\) or it isn't written well enough, IMHO.

MoveOut (as far as I can see) lacks the ability to create the necessary sub-folders and then move the files that meet the wildcard criteria into the self-named newly-created folders.

Did I miss something?

Mike
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 04, 2010, 07:54 PM
MoveOut (as far as I can see) lacks the ability to create the necessary sub-folders and then move the files that meet the wildcard criteria into the self-named newly-created folders.
Did I miss something?
-mickblu (March 04, 2010, 11:56 AM)

Ah, apologies.  I didn't realise that MoveOut didn't create the destination folders.  In that case, would it be possible for you to PM me a simple text listing of your image filenames so I can test against them?  I don't need the full paths...just the names.  However, if there are privacy/sensitivity issues with this request, I certainly understand.  In that case, I'll just generate some random filenames based off the patterns in your initial post.  I've got the main framework of your app already finished;  I just need to do a more work on the flow and then some testing.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 04, 2010, 09:37 PM
OK, I've tried to give you some sense of the file naming conventions which are reasonably strict as long as you appreciate that they will always start with a 2 or 3-letter or number group, then an underscore, then a 2 or 3 letter designator (typically but no hard and fast rule) then an underscore and some more letters and/or numbers and perhaps another underscore or hyphen and some more letters or numbers but this last part is not relevant to the project.

So what we need to create is the following structure:

root (could be anywhere)
then the first grouping creates a major subdir named after it (in the examples shown FJ or MA)
then subdirs off those based on the next grouping
then subdirs off that based on third grouping
then all the files that fit that pattern into those dirs so only three levels deep

Basically Photographer then Model then Project then Image Number

I hope you appreciate that I need an automatic process because (currently) there are 21,128 filers (and growing) so any needed manual intervention makes it a very laborious process indeed!

I forgot to ask you whether this process can be recursive because currently I have a mess of files in this monster dir and (more recently) some measure of subdirs off the monster dir named with project names and their contents need to be processed according to the overall rules, so you don't have to care what the dirs are named, just enter and process (I know, I'm not asking for much...)

I hope this makes it clear?

Mike

Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 05, 2010, 04:07 AM
OK, I've tried to give you some sense of the file naming conventions which are reasonably strict as long as you appreciate that they will always start with a 2 or 3-letter or number group, then an underscore, then a 2 or 3 letter designator (typically but no hard and fast rule) then an underscore and some more letters and/or numbers and perhaps another underscore or hyphen and some more letters or numbers but this last part is not relevant to the project.
-mickblu (March 04, 2010, 09:37 PM)

Well, therein lies the rub.  "Reasonably strict" doesn't equate to "strict" so this will be very difficult to do automatically.  However, read on.

So what we need to create is the following structure:
root (could be anywhere)
then the first grouping creates a major subdir named after it (in the examples shown FJ or MA)
then subdirs off those based on the next grouping
then subdirs off that based on third grouping
then all the files that fit that pattern into those dirs so only three levels deep
-mickblu (March 04, 2010, 09:37 PM)

At its most basic, what you describe above is very easy to do.  Cracking a filename on its underscores, creating folders, and moving said file into said folder is a piece of cake.  However, since some of your files have three groups and others have four, this is much more difficult to do accurately.  Do you understand what I mean?

For instance, your MA_* files mostly have four groups (three that matter and then the irrelevant counter portion) but your FJ_* files mostly have three groups (two that matter and then the irrelevant counter portion).  Normally, I could take num_groups-1 and go with that but I can't, in this case, since there are example in your images where the MA_* files have three groups and the FJ_* files have four groups.

At this point, I don't feel comfortable writing a parser that goes three levels deep since there is no hard standard to the filenames.  There are a few ways to approach this:

1) You strictly standarise your filenames at which point I can make the parser as creative as you would like.

2) You divvy up your files into groups that contains the same amount of "underscore groups" and then I can offer an option to crack the filenames on first, second, or third groups.

3) I write the app so you can form basic *? type of wildcard matching.  Something like "MA_*_*.jpg" gets filtered to some destination folder you specify.  One drawback here is that each match entry can only have one destination folder.  Another drawback, I suppose, is that the entry will need to be entered manually.  On the other hand, the list of entries is easily saved and reused.  This is how I started writing this app before you mentioned your automatic-filename-cracking request.

I hope you appreciate that I need an automatic process because (currently) there are 21,128 filers (and growing) so any needed manual intervention makes it a very laborious process indeed!
-mickblu (March 04, 2010, 09:37 PM)

Sure, I appreciate it but, from my point of the view as the coder who is messing with your files, I have to feel confident that I'm not going to muck them up.

I forgot to ask you whether this process can be recursive because currently
-mickblu (March 04, 2010, 09:37 PM)

Yes, recursive processing is easily doable.

Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 05, 2010, 04:18 PM
Hi Skwire,

I appreciate your dilemma, and I think your solution #3 is the best where you allow me to put in the search criteria in the form of a wild-card string. That wouldn't bother me at all as there are really only the two naming conventions (3 group and 4 group) but at least that way any other eventuality is easily handled.

I look forward to hearing from you,

Mike
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 12:18 PM
Hi Skwire,

I was just wondering where we were at with the "project" as it's been over a week and I hadn't heard back from you.

Sincerely,

Mike
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 04:37 PM
Doh...I'll work on this tonight.  Sorry about that.   :-[
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 04:46 PM
Hey, no worries, I was just curious is all...  :-[
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 04:55 PM
BTW, have you checked out File Sieve 3 (http://software.bootblock.co.uk/?id=filesieve3) at all?  The author is a great coder...I use a few of his apps.  Comical is a favourite.  Anyway, if File Sieve doesn't do what you need, I'll continue work on your custom app.  Please let me know.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 05:31 PM
Nope, haven't come across that one before, and I searched the heck out of the web at the time, but I guess I just didn't hit the right search phrase...

Certainly looks interesting, and probably can do the job, but because of its sophistication it will take some playing with to ascertain whether it will or won't be able to do the sort. I'll play with it a bit and let you know... What really appeals to me is that it supports multiple input directories.

Thanks for the heads-up!
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 05:48 PM
For what it's worth, my custom app for you also has multiple input directories.  Here's a preliminary screenshot:

[ You are not allowed to view attachments ]
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 05:50 PM
Wow! All I can say so far is that it looks about a million times more intuitive than FileSieve!

Out of curiosity, what language are you using to write it?
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 06:30 PM
If FileSieve doesn't end up working out for you, let me know.  All my applications are written in AutoHotkey.  You can find more examples of my work at my website in my sig.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 07:26 PM
Now you've got me interested in AutoHotKey! It seems like it would, with some learning, enable me to automate a lot of the repetitive things I do.

I don't know how old you are so I don't know how much of what I am about to say will make any sense but, here goes:

I studied programming waaaay back in the 70's (BASIC, COBOL and FORTRAN on an NCR mini coding onto 80-column  punch-cards!) which, while they are certainly not current languages, should give me some structural guidelines. I've also coded in dBase IV and similar (Foxbase etc.) and in  VBasic. In fact, I ran the very first BBS here in town on a 300 baud dialup modem using a Z80 computer running the entire system in interpreted BASIC!

If you were to make up this app for me, would you provide it in AHK source or is that too much to hope for? I'd love to be able to learn from a master!

I've been fiddling with FileSieve and it is a lot less intuitive than I would have hoped, and it's not helped by the almost complete lack of documentation other than a few simple examples in a .chm file.

What would it take to convince you to complete the project for me  :-* ?
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 08:23 PM
Hi Skwire,

It seems that FileSieve crashes when faced with the 293,717 files  that I have (and I only pointed it to the "fj" subset, "ma" has another 241,098) so, I guess, your mission, should you choose to accept it sir, is to demonstrate how AHK can do the job!

BTW, FWIW in FileSieve's documentation it says: In this version, it is not currently possible for you to create your own method. It may be possible in a later version which will either provide scripting or plugin support to provide you with this functionality.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 09:36 PM
Now you've got me interested in AutoHotKey! It seems like it would, with some learning, enable me to automate a lot of the repetitive things I do.
-mickblu (March 11, 2010, 07:26 PM)

Oh, my.  I would hate to use a computer without it.  I've built my main "system script" up over the years to the point where vanilla Windows functionality is tough to deal with.

I don't know how old you are
-mickblu (March 11, 2010, 07:26 PM)

I'm 36.  Not too old...not too young.   :P

If you were to make up this app for me, would you provide it in AHK source or is that too much to hope for? I'd love to be able to learn from a master!
-mickblu (March 11, 2010, 07:26 PM)

I'm no master...but you're welcome to the source on this one.

What would it take to convince you to complete the project for me  :-* ?
-mickblu (March 11, 2010, 07:26 PM)

I'll work on it tonight.  There really isn't much left to do other than some GUI bits and testing.

It seems that FileSieve crashes when faced with the 293,717 files  that I have (and I only pointed it to the "fj" subset, "ma" has another 241,098) so, I guess, your mission, should you choose to accept it sir, is to demonstrate how AHK can do the job!
-mickblu (March 11, 2010, 08:23 PM)

I've no doubt AHK can do it.   :)
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 10:07 PM
That will be awesome! Thank you.

Heh, 36! My daughter is 34 and my son is 32 so that gives you a clue how old I am (I'm 62  [ You are not allowed to view attachments ] !)

Anyway, I'm really looking forward to seeing what AHK can do...
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 10:19 PM
To make my testing easier, can you provide me with a text file containing the filenames you're dealing with?
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 10:20 PM
Sure, gimme 5...
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: mickblu on March 11, 2010, 11:47 PM
I guess it took somewhat longer than 5  :-[ but I had to find the right utility to produce a recursive directory list...

Anyway, here it is!
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 11, 2010, 11:49 PM
Nice, thanks.  I've sent you a PM as well.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 12, 2010, 10:30 AM
Give this a test (source/binary included): File Punter (http://skwire.dcmembers.com/apps/snacks/FilePunter.zip)

It's not fully feature complete as of yet but the core functionality is in there.  It should be pretty intuitive.

1) Use the top section to create and add your match filters to the list.
2) Checkmark the rows you want to process.
3) Choose your method at the bottom and click Start.  Progress and such is shown in the statusbar.  You can cancel at any time.
4) All checked rows will be processed in order.  You can move highlighted rows up and down through the list with ctrl-up and ctrl-down before clicking Start.
5) You can delete rows by highlighting and pressing delete.
6) The list is saved/loaded automatically when you close/launch the app.  No other settings are saved at this point.

The three methods you can choose from are:

1) Copy files to destination.  Self-explanatory.
2) Move files to destination.  Self-explanatory.
3) Simulation only.  This mode won't touch your files.  It will just create a text report that will show which files matched your filters and their destinations.  You can use it as a sort of "test run" mode.  Keep in mind that if your filter matches a very large number of files, the resulting text file can get pretty large, too.  Depending on path lengths involved, figure about a meg per 9000 lines/matches.

Put it through its paces and let me know how it works out for you.  If it seems solid enough, I'll add in my usual Skwire Empire stuff and release it as a full-fledged app.  Thanks and apologies for the delay in getting this to a release candidate.

[ You are not allowed to view attachments ]
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: tomos on March 12, 2010, 10:43 AM
I studied programming waaaay back in the 70's (BASIC, COBOL and FORTRAN on an NCR mini coding onto 80-column  punch-cards!) which, while they are certainly not current languages, should give me some structural guidelines. I've also coded in dBase IV and similar (Foxbase etc.) and in  VBasic. In fact, I ran the very first BBS here in town on a 300 baud dialup modem using a Z80 computer running the entire system in interpreted BASIC!
-mickblu (March 11, 2010, 07:26 PM)

Hi Mick,
an off-topic note:
you may be interested in this thread: In rememberance of....The antiquated hardware/software reminiscence thread (https://www.donationcoder.com/forum/index.php?topic=21995.0)
Tom
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on March 12, 2010, 11:45 AM
As an aside, keep in mind that this has had, obviously, minimal testing.  In other words, I'd make some backup copies of your photos first.   :D
Title: IDEA: Extended function for Files2Folder
Post by: magician62 on February 11, 2011, 08:27 AM
Hello, hopefully in this my first post, I get things right.


First  Files2Folder (http://skwire.dcmembers.com/wb/pages/software/files-2-folder.php) is a great utility

The thought I have which may be a new utility or enhancement of the existing.

I often encounter files in the following formats

first part - middle part - last part.extension
or
first part - last part.extension

It would be nice to be able to select a group of files and have the utility folderise by a defined segment, though to make it easier.

option 1
Anything before the first " - "

option 2
Anything after the last " - " (ignoring extension)

There may be other cases, but I suspect it would be more complex to handle.

When run the utility would create folders based on content for the defined option.

If a folder already exists, there is no need to create

Files are then added to the appropriate folder.

In case of duplicates, files are enumerated, or ignored, though in theory this should not happen from an existing folder, but may if further files are added for subsequent processing.


I am guessing there may already be something out there that does this, but I have yet to find it. And then probably nothing with the simplicity of Files2Folder

Thanks in advance

 
Title: Re: IDEA: Extended function for Files2Folder
Post by: skwire on February 11, 2011, 11:42 PM
Hello, hopefully in this my first post, I get things right.
-magician62 (February 11, 2011, 08:27 AM)

Welcome to DonationCoder.  Thanks for posting and enjoy your stay.

First  Files2Folder (http://skwire.dcmembers.com/wb/pages/software/files-2-folder.php) is a great utility
-magician62 (February 11, 2011, 08:27 AM)

I'm glad you find it useful.   :)

The thought I have which may be a new utility or enhancement of the existing.

I often encounter files in the following formats

first part - middle part - last part.extension
or
first part - last part.extension

It would be nice to be able to select a group of files and have the utility folderise by a defined segment, though to make it easier.

option 1
Anything before the first " - "

option 2
Anything after the last " - " (ignoring extension)
-magician62 (February 11, 2011, 08:27 AM)

I think this is well out of scope for Files2Folder but you're in luck.  I have another application that I haven't fully released that should do what you want.  What it lets you is specify source folders and then write a simple regular expression with which you can create destination folders that uses data matched by the regular expression.  If you're not familiar with regular expressions, that description probably won't make much sense so here's an example (based off of yours):

Source folder:         c:\path\to\
Containing files like: first part - middle part - last part.extension
RegEx string:          (.*?) - (.*?) - (.*?).extension
Destination folder:    c:\path\to\??2

Each "(.*?)" represents a matched subpattern.  I've highlighted in red the second matched subpattern and how you could use it to automatically create a destination folder based off of text within the filename.  The app then gives you the option to copy, move or simulate.  Based on the above, it would copy/move said file to the following:

c:\path\to\middle part\first part - middle part - last part.extension

Does that make sense?  Would you be interested in something like this?
Title: Re: IDEA: Extended function for Files2Folder
Post by: skwire on February 11, 2011, 11:55 PM
Also, can you give me more information on the types of files you want to shift around?  Are they music files by chance?
Title: Re: IDEA: Extended function for Files2Folder
Post by: magician62 on February 12, 2011, 05:15 AM
Many thanks for the response. The description I gave was in the simplest form, as I can see other potential later. I am actually sorting pictures

The idea came from using Calibre (http://calibre-ebook.com) and how it stores ebooks based on regular expressions. Unfortunately it puts each item into a subfolder of the group, rather than directly in the group, which would have meant i could have used Calibre to do the grouping.

The problem is selecting the images into sets. Most have one or other of the formats mentioned for the filenames, but there are multitudes of variants, and the list is continually growing. Automating the folderising would reduce the biggest manual part of the process to an insignificant number.

Once grouped, they can be placed in CBR format and loaded into calibre as sets or stored elsewhere.

So it does sound like what you have may work, I can always give it a try, maybe iron any bugs too if they exist
Title: Re: IDEA: Extended function for Files2Folder
Post by: skwire on February 12, 2011, 06:28 AM
Would you mind giving me some examples of the filenames and paths you're working with as well as how/where you would like them to end up?  If they're sensitive names, either change them or PM me the list.  I can then let you know if this application will work for you.
Title: Re: IDEA: Extended function for Files2Folder
Post by: magician62 on February 12, 2011, 07:30 AM
Many thanks for the quick reply.

The structure is in many ways quite simple, and I can adjust some of it if needed as I go, such as moving each set files to source folder, which effectively elliminates the possibilities of collisions.

Once a batch has been processed, I can physically move the generated folders to a new location, before processing another set. They can then be moved on completion, and windows can handle the collisions for me, and i can the adjust the folder name if needed.

In the following examples i can put "myfolder" anywhere needed in the path i.e. drive:\\myfolder if it makes things easier.

Most of the images are either in the format

myfolder\image00001 - location.jpg
or
myfolder\location - image00001.jpg

There are a number where I incorporate date such as

myfolder\image00001 - location 15-02-2004.jpg
or
myfolder\2004-02-15 location - image00001.jpg
or
myfolder\location 15-02-2004 - image00001.jpg

In these cases i can see the extra "-" being a possible problem, but i can work around that as once they are in their sets, I have other tools I can utilise.

There are also a few files in the format

location - 15022004 - image00001.jpg

But i have means to easily remove the first "-" so reducing it to one of the previous variants.


Destination for each set would be
mydestinationfolder\location\

Though as we are only starting in

myfolder\

and there are no existing subfolders, unless there is a programming reason, I can see no reason why the destination can't just be

myfolder\location\
Title: Re: IDEA: Extended function for Files2Folder
Post by: skwire on February 12, 2011, 08:38 AM
In the following examples i can put "myfolder" anywhere needed in the path i.e. drive:\\myfolder if it makes things easier.
-magician62 (February 12, 2011, 07:30 AM)

The application lets you make a list of source/pattern/destination entries and save them.  Sort of like a profile.  You can use simple wildcard matching to move files but, in your case, you will need to use regular expressions since you want to move them to dynamically created destinations.  After setting up your entries, you can then check/uncheck entries as necessary and only run those.  Here's a screenshot:

[ You are not allowed to view attachments ]

Most of the images are either in the format

myfolder\image00001 - location.jpg
or
myfolder\location - image00001.jpg

There are a number where I incorporate date such as

myfolder\image00001 - location 15-02-2004.jpg
or
myfolder\2004-02-15 location - image00001.jpg
or
myfolder\location 15-02-2004 - image00001.jpg

In these cases i can see the extra "-" being a possible problem, but i can work around that as once they are in their sets, I have other tools I can utilise.
-magician62 (February 12, 2011, 07:30 AM)

Shouldn't be a problem with regular expressions.

There are also a few files in the format

location - 15022004 - image00001.jpg

But i have means to easily remove the first "-" so reducing it to one of the previous variants.
-magician62 (February 12, 2011, 07:30 AM)

Again, regular expressions should easily be able to work around this.

Destination for each set would be
mydestinationfolder\location\

Though as we are only starting in

myfolder\

and there are no existing subfolders, unless there is a programming reason, I can see no reason why the destination can't just be

myfolder\location\
-magician62 (February 12, 2011, 07:30 AM)

Yep, not a problem.  The software offers a test mode that creates a text file of what will be move/copied and where.  This way, you can test your regular expression to see if it does what you want.  If you use instant messaging, feel free to add any of the accounts here:

http://skwire.dcmembers.com/wb/pages/contact.php

If you don't mind and, since this app hasn't been released to too many people, I'd like to work with you directly to make sure it does what you want.
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on February 13, 2011, 03:05 PM
Magician62 has been doing some really good testing of File Punter on his image files and, I'm happy to say, we've got File Punter to what I feel comfortable for a Skwire Empire release.  Thanks, magician62!   :Thmbsup:

I also merged his topic into this original File Punter thread.

Website (http://skwire.dcmembers.com/wb/pages/software/file-punter.php) | Download (http://skwire.dcmembers.com/apps/file_punter/FilePunter.zip)
v1.0.4 - 2011-02-13
    * Made simulation mode the default option.  (Thanks, magician62)
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: 4wd on February 13, 2011, 05:53 PM
Thanks, magician32!   :Thmbsup:
-skwire (February 13, 2011, 03:05 PM)

It obviously took it's toll on him :)
Title: Re: IDEA: Organize files into folders based on flename wildcard parsing
Post by: skwire on February 13, 2011, 06:06 PM
It obviously took it's toll on him
-4wd (February 13, 2011, 05:53 PM)

OMG...fixed!  Thanks.  =]
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: magician62 on February 15, 2011, 05:02 PM
One issue found during testing which was not reproducible other than on my own system was the case of only Deskop and Current User appearing in the Browse for Folder Dialogue box for add /edit entry. It was believed that it might be Win7-64.

Well discovered the cause/fix as I also found another until had started to encounter the same issue. elsewhere.  You can read about it yourselves at Browse For Folder -- Missing Computer,and Libraries (http://forums.cnet.com/7723-19411_102-412058.html)

But in summary, there is a limit on the number of icons on the desktop. When you exceed the number say byebye Browse For Folder functionality.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on June 18, 2012, 02:46 PM
Website (http://skwire.dcmembers.com/wb/pages/software/file-punter.php) | Download (http://skwire.dcmembers.com/apps/file_punter/FilePunter.zip)
v1.0.5 - 2012-06-18
    + Added "Create shortcuts" mode.  (Thanks, nkormanik)
    + Added Check/Uncheck toolbar buttons.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: nkormanik on June 21, 2012, 09:07 PM

Two enhancement requests:

1)  Skwire, I do feel it would be quite a bit more reassuring to show the total number of lines/commands selected to be seen somewhere -- status line or elsewhere.  I'd just feel more at ease seeing that number.

2)  Perhaps, also, allow configuration .ini to set default location in which to begin, from which to open text files containing command lines.  My preference is to look in same directory where the executable is.  For whatever reason when I say Open, File Punter wants to look in Desktop.  I keep all the text file commands within the File Punter executable directory buried somewhere else.

Thanks!
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on June 22, 2012, 12:07 AM
Website (http://skwire.dcmembers.com/wb/pages/software/file-punter.php) | Download (http://skwire.dcmembers.com/apps/file_punter/FilePunter.zip)
v1.0.6 - 2012-06-22
    + Last used folder is remembered for File, Open action.  (Thanks, nkormanik)
    + Added status bar section showing total number of rows and total number of
      checked rows.  (Thanks, nkormanik)
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: y0himba on January 03, 2013, 10:46 AM
Not to resurrect a dead thread, but I have been using File Punter a lot.  I was wondering if there is command line support?  I would like ot be able to use task manager to run it at a regular interval to organize files.  The Perl flavored RegEx is beautiful.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on January 03, 2013, 11:08 AM
Unfortunately, no, there isn't any command-line support for this application.   :(
Title: File Punter - BUG
Post by: magician62 on October 26, 2013, 04:20 PM
For source dir you can use a network location

\\NAS\START

but you can not use for destination.

On Update Entry

Error dialogue
Destination DRIVE does not exist. Please chose a valid destination.

At present the only solution is to map the drive to a letter.
Title: Re: File Punter - Feature Requests
Post by: magician62 on November 24, 2013, 07:22 AM
And to add a feature request or two.

When you start to get long list of options, it would be nice to be able to insert a blank line :)

Additionally, it would be nice to be able to apply to folders. (The contents of the folder would go with the folder to it's new destination no matter what their name)

And one other option, a Notes column to allow us a reminder of what the option was for in the first place :)
Title: Re: File Punter - BUG
Post by: skwire on November 25, 2013, 10:10 AM
Error dialogue
Destination DRIVE does not exist. Please chose a valid destination.

At present the only solution is to map the drive to a letter.
-magician62 (October 26, 2013, 04:20 PM)

I did some testing on this and it seems to be an AutoHotkey bug/limitation.  There are a few solutions I can think of:



When you start to get long list of options, it would be nice to be able to insert a blank line :)
-magician62 (November 24, 2013, 07:22 AM)

This shouldn't be a problem.

Additionally, it would be nice to be able to apply to folders. (The contents of the folder would go with the folder to it's new destination no matter what their name)
-magician62 (November 24, 2013, 07:22 AM)

Ehhh...I'll have to give this some thought.  No promises on this one.

And one other option, a Notes column to allow us a reminder of what the option was for in the first place :)
-magician62 (November 24, 2013, 07:22 AM)

Good idea, actually, and easily doable.
Title: Re: File Punter - BUG
Post by: skwire on December 04, 2013, 09:45 PM
I didn't add the blank line request as I felt it was inelegant.  I'll give it some more thought.  Also, if you don't mind, I'd like to merge this thread with the original File Punter thread.

Website (http://skwire.dcmembers.com/wb/pages/software/file-punter.php) | Download (http://skwire.dcmembers.com/apps/file_punter/FilePunter.zip)
v1.0.7 - 2013-12-04
    + Drag-n-drop reordering in the listview is now possible.
    + Added icons to the menus.
    + Added description column.  (Thanks, magician62)
    * Application is now DPI aware.
Title: Re: File Punter - BUG
Post by: magician62 on December 08, 2013, 03:20 PM
Hi Jody, sure merge away :) I am at present splitting my time between several time eating tasks and trying to prepare for the holiday season. I have just downloaded and will test shortly.

The odd thing about the path's is that a network location works for the source just not destination.


It doesn't help that I am still installing software after replacing the HDD and starting from scratch in an attempt to find what program has been causing the computer to effectively stall at times, with no obvious processor or memory issues, then it just continues!
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on April 09, 2014, 12:03 AM
Website (http://skwire.dcmembers.com/fp/?page=file-punter) | Download (http://skwire.dcmembers.com/apps/file_punter/FilePunter.zip)
v1.0.8 - 2014-04-09
    ! Using pipes in pattern strings would screw up their display in the
      dropdown list.  (Thanks, Christian)
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: magician62 on April 09, 2014, 12:40 PM
Hi Jody, a clue to the paths issue, which makes me think it is something to do with the field where we enter the Destination path. The fact that the source path can be entered in the required format lends to a belief something odd is going on.

I can create the path directly in the settings file, which loads OK and functions correctly

\\NSA-4\++++START   (.*?) - (.*?)x(.*?)   \\NSA-4\++++FINISH\??1 [s0??2]   No   Yes


There is one issue with doing this in the saved settings file and that is the entries are a continuous line. Could they be made into one per line this would simplify manual edits to overcome the path issue.

EDIT
Ok there is a second issue, and that seems to be related to the line length.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on April 09, 2014, 01:04 PM
Hi Jody, a clue to the paths issue
-magician62 (April 09, 2014, 12:40 PM)

Please read this post.  It's an AutoHotkey bug.

https://www.donationcoder.com/forum/index.php?topic=21964.msg342772#msg342772
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: kiui on May 13, 2014, 01:24 PM
Hello,
I'm a basic (french) user and trying to work with File Punter... but it doesn't acually.

I've in the same folder such pdf files.

abcdef_ducs_PES_04-2014.pdf
dfglfg_ducs_PES_04-2014.pdf
kiui_ducs_PES_04-2014.pdf
...
the only variable is the first letters (max 6 numbers)

and would like to move thoses files to folders :
abcdef
dfglfg
kiui

It's the right application to use to do this ?
Many thanks for your help.
Julien
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on May 13, 2014, 01:38 PM
Hi, Julien, and welcome to the DonationCoder site.   :D

and would like to move thoses files to folders :
abcdef
dfglfg
kiui

It's the right application to use to do this ?
-kiui (May 13, 2014, 01:24 PM)

Yes, you can use File Punter to move those files how you want.  Have you tried to create a RegEx rule?  If so, can you post a screenshot of it?
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: kiui on May 13, 2014, 01:42 PM
Hello,
Here is the rule :
C:\Users\Administrateur\Desktop\test   (.*?)_ducs_PES_04-2014.pdf   C:\Users\Administrateur\Desktop\clients\??1   No   No   

Thanks
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on May 13, 2014, 01:53 PM
C:\Users\Administrateur\Desktop\test   (.*?)_ducs_PES_04-2014.pdf   C:\Users\Administrateur\Desktop\clients\??1   No   No   
-kiui (May 13, 2014, 01:42 PM)

Based on that, it looks like you have not set that rule as a RegEx rule.  Double-click the rule and enable the "Pattern is a regular expression" checkbox.  Test it again and let me know if that makes it work how you want.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: kiui on May 13, 2014, 02:04 PM
Fantastic ! It works...
Sorry but I didn't get the différence is this box enable or not ?

the second part of my files name is changing, but the rule is the same:
 (.*?)_ducs_PES_04-2014.pdf
 (.*?)_ducs_AEPT_2014.pdf

can i put someting to not have to create a rule for each file "suffix" name.?

And if I can ask one more thing, what's the difference between (.*?) and (\d+) or (.+?) ?

thanks!

(You will never know how much time I will earn in the future with this application, but now i'm loosing a lot to learn  :P )

Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on May 13, 2014, 02:32 PM
Fantastic ! It works...
-kiui (May 13, 2014, 02:04 PM)

Great to hear.   :Thmbsup:

Sorry but I didn't get the différence is this box enable or not ?
-kiui (May 13, 2014, 02:04 PM)

File Punter can use normal DOS-style wildcards or it can use regular expressions.  The problem is that both methods make use of such characters as * or ? but in different ways.  Hence, the checkbox to set which method to use.

can i put someting to not have to create a rule for each file "suffix" name.?
-kiui (May 13, 2014, 02:04 PM)

You sure can.  Try this as your match rule:  

(.*?)_.*

And if I can ask one more thing, what's the difference between (.*?) and (\d+) or (.+?) ?
-kiui (May 13, 2014, 02:04 PM)

Hehehe...the world of regular expresssions.   ;)  There is so much to learn that it might be easier to point you to http://www.regular-expressions.info/tutorial.html (http://www.regular-expressions.info/tutorial.html)

(You will never know how much time I will earn in the future with this application, but now i'm loosing a lot to learn  :P )
-kiui (May 13, 2014, 02:04 PM)

I'm glad you find the program useful.  It's a very powerful program that can, potentially, really mess things up so please make good use of the "test mode" functionality it offers.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: bastik on December 16, 2014, 08:37 AM
@skwire

File Punter is useful, good thing it does not overwrite existing files. Now I'd like to ask if something is already possible or if that turns out to be a feature request.

Explanation of what I am doing:
I have files whose names include $x and $y that are supposed to be part of a folder structure. They are moved or copied from
[Select]
C:\source
to
[Select]
D:\destination_path\$x, $y\some_subfolder

A regular expression (don't know too much about them) looks for $x and $y, brackets like the following () group $x and $y. The destination in File Punter looks like this
[Select]
D:\destination_path\??1 ??2\some_subfolder
File Punter does what I want it to do, in the first place. It moves a file named
[Select]
creative mind 12345.ext
to
[Select]
D:\destination_path\creative mind\some_subfolder\creative mind 12345.ext
That works, that is cool.

What I would like and why I would like it:
The folders are supposed to exist already. Meaning they are present before the copying or moving process even begins. It is cool that File Punter is able to create folders that are not present, however in my scenario there was something wrong if the folder did not exist and had to be created by File Punter. Like the folder present is
[Select]
creative mind
but the files say
[Select]
craetive mind $number
File Punter does its job and creates a folder, which is not its fault, but my misfortune.

Is there a way, for example by modifying my destination
[Select]
D:\destination_path\??1 ??2\some_subfolder
to achieve that File Punter ignores the files. So that if folders with
[Select]
??1 ??2
are not present are not created?

If that is not possible, I'd like to suggest two new methods
[Select]
copy to existing folders
and
[Select]
move to existing folders
Alternatively some other way of telling file punter to not create folders.


Other stuff I wondered while I was writing this:
When I move the files (my use case), remaining files indicate that something went wrong. E.g. the pattern did not catch all files or the destination file is already present. I don't have that "feedback" when I would copy the files. Could there be some kind of log, either where every operation goes into, or one just listing the skipped files, maybe with a reason?

I'm aware that File Punter is not a (re)naming tool, but I realized that I move
[Select]
creative mind 12345.ext
to
[Select]
.../creative mind/subfolder/creative mind 12345.ext
which makes it a bit redundant. Is there a way to use File Punter to strip
[Select]
??1 ??2
from the files during the process?

Thank you for the program and thank you for anything you can say about my problems.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on December 17, 2014, 01:28 PM
@bastik: I've quickly read through the post but I need to devote some time to studying it a bit.  I've got a few other app requests to fulfill first but, rest assured, this is on my agenda.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: bastik on February 13, 2015, 11:55 AM
@skwire

It is nothing I have a strong need for.

So if it is very easy to achieve it would be nice if it could be added. If it is most likely difficult to do, then please don't implement it unless there is another request or you find it extremely useful.

Is there a way to use File Punter to strip (blabla) from the files during the process?
This is totally irrelevant. Having the identifier in the filename makes sure they will be found. This is a non-issue.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: callumm on November 09, 2015, 02:40 PM
Will filepunter punt folders?
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on November 09, 2015, 04:02 PM
Will filepunter punt folders?
-callumm (November 09, 2015, 02:40 PM)

Nope.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: dariotoby on October 21, 2018, 11:23 AM
hi, I recently discovered this program that I think is for me,
File Punter
But I have to ask for help because I do not know anything about strings regx
I'll explain my situation and if someone can suggest the string to insert
I of the mp3 files named example so
Aretha Franklin - Right woman do right man (Cori) (K5) .mp3
Ariana Grande & John Legend - Beauty and the beast (Cori) (K5) - by Maker.mp3

Type in these ways are karaoke files and I have many
I have these files put all in a folder like Download
Then I have already organized files in folders named for Artist in another PC folder
I would like you to move them to existing folders that have the artist's name in another PC folder.
Someone can help me on how to configure the program I read all the theard I try strings but they do not work.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on October 21, 2018, 11:52 AM
Hi, dariotoby, and welcome to the DonationCoder site.   :)

Type in these ways are karaoke files and I have many
I have these files put all in a folder like Download
Then I have already organized files in folders named for Artist in another PC folder
I would like you to move them to existing folders that have the artist's name in another PC folder.
Someone can help me on how to configure the program I read all the theard I try strings but they do not work.
-dariotoby (October 21, 2018, 11:23 AM)

I did create a program to move audio files according to their tags called Tags 2 Folders (http://www.dcmembers.com/skwire/download/tags-2-folders/) but File Punter might be better suited to your case.  One question -- in your filenames, is the artist ALWAYS before the first dash in the file name?  For example:

Aretha Franklin - Right woman do right man (Cori) (K5) .mp3
Ariana Grande & John Legend - Beauty and the beast (Cori) (K5) - by Maker.mp3
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: dariotoby on October 21, 2018, 12:23 PM
Yes is the artist ALWAYS before the first dash in the file name

 
Hi, dariotoby, and welcome to the DonationCoder site.   :)

Type in these ways are karaoke files and I have many
I have these files put all in a folder like Download
Then I have already organized files in folders named for Artist in another PC folder
I would like you to move them to existing folders that have the artist's name in another PC folder.
Someone can help me on how to configure the program I read all the theard I try strings but they do not work.
-dariotoby (October 21, 2018, 11:23 AM)

I did create a program to move audio files according to their tags called Tags 2 Folders (http://www.dcmembers.com/skwire/download/tags-2-folders/) but File Punter might be better suited to your case.  One question -- in your filenames, is the artist ALWAYS before the first dash in the file name?  For example:

Aretha Franklin - Right woman do right man (Cori) (K5) .mp3
Ariana Grande & John Legend - Beauty and the beast (Cori) (K5) - by Maker.mp3
-skwire (October 21, 2018, 11:52 AM)
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on October 21, 2018, 01:16 PM
You could do something like this:

[ You are not allowed to view attachments ]

In case it's not clear, the pattern is: (.*?) -

Or course, you will need to adjust the paths according to your file system (leave the ??1 at the end of your destination path).  After creating an entry like above, click the "Run simulation" button to see if the entry will match and move how you expect.  For instance, in my test setup for you, here's my simulation report (I coloured some bits to show you how it matches everything before the first dash and then creates a folder from that data):

C:\download\music\Aretha Franklin - Right woman do right man (Cori) (K5).mp3 --> c:\music\artists\Aretha Franklin\Aretha Franklin - Right woman do right man (Cori) (K5).mp3
C:\download\music\Ariana Grande & John Legend - Beauty and the beast (Cori) (K5) - by Maker.mp3 --> c:\music\artists\Ariana Grande & John Legend\Ariana Grande & John Legend - Beauty and the beast (Cori) (K5) - by Maker.mp3

Furthermore, I would use the "Copy files to destination" method until you are positive it's matching and moving properly.  Keep in mind that you can really screw things up with FilePunter if you're not sure you know what you're doing.  Hence, all the simulation/test modes I offer in the program.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: dariotoby on October 23, 2018, 08:46 AM
Great fantastic beautiful I did 30 seconds that required work days. You are great
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: nkormanik on April 19, 2020, 01:44 AM
Seems all files are placed into single folder.  Original nested folders as well.  No mirrored folders appear to be created.

Is there a way to copy the original folder structure, too?

Thanks!

Nicholas Kormanik

 
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on April 19, 2020, 10:10 AM
Seems all files are placed into single folder.  Original nested folders as well.  No mirrored folders appear to be created.

Is there a way to copy the original folder structure, too?
-nkormanik (April 19, 2020, 01:44 AM)

For File Punter to recreate the original folder structure, you would have to match parts of the source folder structure you wanted and re-use them to recreate the destination.  However, if you're just looking to copy/mirror over a folder structure and files, File Punter is NOT the app for you.  Use one of the myriad of file/folder synchronization apps out there.  If that's not what you need to do, please give me detailed information on what you need to accomplish.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: nkormanik on April 19, 2020, 09:25 PM
Thanks, Skwire, for clarifying.  Yes, wanted to move all .mp3 files from folder A to folder B, keeping the entire folder structure where the .mp3 files reside.

Will try another app.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on April 19, 2020, 10:46 PM
wanted to move all .mp3 files from folder A to folder B, keeping the entire folder structure where the .mp3 files reside.
-nkormanik (April 19, 2020, 09:25 PM)

A simply copy/cut and paste would not work here?  Am I missing something?
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: nkormanik on April 20, 2020, 12:53 AM
100,000 .mp3 files -- intermixed with other formats, such as .flac -- in a giant tree structure.  Branches of trees are "Artist" and "Album".

I wanted to move ONLY the .mp3 files to another drive, leaving .flac, etc., where they are.

As you've said, a myriad of programs out there to do it.  Such as xcopy, robocopy, richcopy.

Thought it best to use a gui program, to see more clearly what's about to happen... before pressing go.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: 4wd on April 20, 2020, 04:44 AM
https://github.com/Cinchoo/ChoEazyCopy
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: anandcoral on April 20, 2020, 04:46 AM
100,000 .mp3 files -- intermixed with other formats, such as .flac -- in a giant tree structure.  Branches of trees are "Artist" and "Album".

I wanted to move ONLY the .mp3 files to another drive, leaving .flac, etc., where they are.

As you've said, a myriad of programs out there to do it.  Such as xcopy, robocopy, richcopy.

Thought it best to use a gui program, to see more clearly what's about to happen... before pressing go.

-nkormanik (April 20, 2020, 12:53 AM)

I suggest to use Total Commander https://www.ghisler.com/
You can filter on *.mp3 and press Ctrl+B, then F5 copy to new folder.
All sub folders will be created automatically.

Regards,

Anand
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: anandcoral on April 20, 2020, 04:47 AM
https://github.com/Cinchoo/ChoEazyCopy
-4wd (April 20, 2020, 04:44 AM)
:Thmbsup:

Regards,

Anand
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: magician62 on January 17, 2022, 05:45 AM
Is it possible to re-use the output variables?

Such as Destination like

....\??1\??1 ??2
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: skwire on January 17, 2022, 09:18 AM
Is it possible to re-use the output variables?

Such as Destination like

....\??1\??1 ??2
-magician62 (January 17, 2022, 05:45 AM)

Yes.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: magician62 on January 17, 2022, 01:10 PM
Hi Jody.

I simplified my pattern and did a quick test, just in case it was something in the long path I had.

[ You are not allowed to view attachments ]

Simulation result

E:\TEST\start\Groupname - Filename.txt --> E:\TEST\finish\Groupname\??1 \Groupname - Filename.txt

The end result should have been

E:\TEST\start\Groupname - Filename.txt --> E:\TEST\finish\Groupname\Groupname\Groupname - Filename.txt

Obviously this is a stripped down version of what I am trying to do. 

Curiously there is a space after the "??1 " in the simulation.
Title: Re: DONE: Organize files into folders based on flename wildcard parsing
Post by: magician62 on July 23, 2023, 07:35 AM
DELETED