Main Area and Open Discussion > General Software Discussion

Program Whose Time Has Come: virtual folders, collections, file baskets...

(1/8) > >>

tranglos:
What would be the one program you would like to have most that is already possible but doesn't quite seem to exist yet? A Program Whose Time Has Come? For me, it would be a powerful sidekick to file managers: a program to manage virtual folders, a.k.a file collections, a.k.a file baskets/organizers... Hope someone comes up with a catchier name!

The time when the simple folder hierarchy ceased to suffice is long gone, isn't it? I've seen plenty of requests for such functionality on various forums, but strangely no real takers yet. I'll describe my experience with the existing solutions, pretty immature all, but first, here's what the program would do:

A virtual folder is a folder that doesn't exist physically on disk. It is merely a name for a collection of files selected by the user for whatever purpose or gathered automatically according to some criteria. The "whatever" is pretty darn big - I come up with new potential uses for such a program every day. Here are the three main uses of file collections I can see. (Please add others if my three don't exhaust the range of possibilities). Each of these uses requires slightly different behaviors, but it seems possible to accommodate them all in a single application.

Use case 1. Creating ad-hoc file collections in order to do something immediately with the files: zip them, copy them somewhere, encode them to mp3, backup them, email them as attachments, etc. File managers make such tasks easy only as long as all the files you need reside in the same folder (or under the same parent folder, if you can think of a pattern to match the required files). However, if you want to zip or backup files residing in different folders, it becomes a prolonged task. File collections would help a lot here.

These collections are by nature temporary and need not be persisted. Our hypothetical virtual folders program needs to be able to execute external applications passing the collected filenames to them (e.g. to an archiver or an encoder). Some of these features (zipping, copying) could be built in, but the program must be extensible enough to allow user to do any (unforeseen by the author) operations on the file collection.

Use case 2. Creating disk catalogs, i.e. "snapshots" of the current contents of drives or directories. These would be persistent collections and should not be automatically refreshed. (If you want to catalog your CDs or DVDs, there will be a different disk in the cd-rom drive every time, and existing catalogs of previously inserted discs must be preserved untouched). There isn't much users will actually "do" with these collections - just view them, search, save as text, print.

Use case 3. Collections of files arbitrarily chosen by the user to represent a concept or a project. For example, if you are a freelancer working for several clients, you may have one physical directory on disk for each client. Inside each directory there will be subfolders holding different types of files - the files you work on, invoices, old archives and other data. Now, if you wanted to look at all the invoices for all the clients, you'd have to navigate through all the folders in sequence. Instead, you could create a collection (a perfect use case for a "virtual folder") to list all the invoices in one place, so that you don't have to find them manually. The collection could be maintained manually (add each invoice by hand as you create it) or automatically (e.g. add every file that matches the pattern "invoice*.xls" from all subfolders under c:\MyClients). In the same way you could have an automated collection that lists invoices for the current year only, etc.

These collections should be persistent and *may* be automatically refreshed, depending on the semantics of a particular collection (i.e., what a collection represents).

Collections of the third type would probably be most common. You could use them to gather files for a specific project you're working on, or specific files from all your projects so far; you could create playlists, or automatically generate lists of all mp3s by a given artist or with a given genre. You could have a collection of all .lnk files in your Start menu (and use the program a little like FARR), or a collection of all "readme.txt" files for all the software you have written. Or you could have a collection of shortcuts to all uninstallers for the programs you have installed (a little like the Control Panel "Add or remove programs" applet). There are no limits here, and the potential usefulness is, I think, awesome.

The collections come in different types according to how they are created. You would select a collection type that corresponds to how you want to use a collection.

Type 1 Manual collections. You create them by hand, by adding / dragging single or multiple files (or pasting filenames in a dialog box). Refreshing such a collection means checking if the files still exist, optionally removing missing files, and updating the properties (date, size) for all the files, so that the program displays current data.

Type 2: Automatic collections. You specify one or more folders and one or more filename patterns, and the program adds the files automatically to the collection. Refreshing such a collection means re-running the search according to the specified criteria (any previous contents would be cleared). Refresh could happen on opening the collection, on timer or manually. A somewhat extreme example: create a collection where the folder is "C:\", the file pattern is "*.tmp", and the option to include subfolders is enabled. The collection will be automatically refreshed when opened (it'd take some time for a large drive...) and show all the temp files on your C: drive. Press Ctrl+A to select all, then press Delete to remove all your temp files from the drive. (With macro support, the whole process could be entirely automatic and you wouldn't even have to open the collection if the refresh happened on a timer). A somewhat less radical example - create a collection of all mp3 files in your c:\music folder which match the pattern of "Pink Floyd*.mp3". Create a collection of all the invoices, as described earlier, or a colection of all files modified this week, etc.

Type 3: Scripted collections. The program would be extensible by scripting (maybe using JavaScript, Python or Pascal). The script would be responsible for building the collection. It would of course require a little programming, but you could create collections using limitless criteria of your own. Refreshing such a collection means re-running the script. A script could also be used as a bridge between our program and a search engine such as Locate, Everything or Archivarius. You could use the power of the search engine to create your file list, and the script would simply add the search results to your collection - in this case the script itself would be quite simple. (This assumes that Locate, Everything or Archivarius can save their search results to a text file - if they can't, we can pester their makers to add this simple feature). And of course, scripts could also be used to perform operations on files already gathered in collections.

Some notes:

1. I understand Windows 7 has "libraries", which are somewhat like the collections I am describing, but knowing Microsoft, the functionality of their libraries will be quite limited in comparison to what's possible.

2. The program should use a database back-end, since the number of files in a collection is potentially very large. The program should only store links to files (their full paths), and not the contents of the files. (Seems obvious, but one related application does make physical copies of the collected files. Wonder what happens when you start making collections of your ripped DVDs... Argh!)

3. The program should store its data in a way that facilitates backup.

4a. The program should ideally be implemented as a shell extension, so that its functionality is available everywhere you can use Windows Explorer. This would mean it would also be automatically available in your preferred file manager.

4b. Further, it should be transparent to Explorer: it should support the basic file operations and implement them in its specific way. For example, copying a file to a virtual folder should result in the file being added to the virtual collection (i.e., the file should not be physically copied anywhere). Deleting a file from a virtual folder should only delete the collection item, not the physical file - although there must also be a (separate) feature to delete the physical file. Renaming a file should probably just rename the collection item, etc. Moving a file to a collection should probably do nothing (since it violates the whole concept), but moving a file from a collection could perhaps move the physical file from its original location to the new one. I'm not sure about the particulars here; the important thing is that these behaviors should be consistent and should make good sense to the user. You should certainly be able, for example, to drag files from a collection to another program (e.g. to add an email attachment or to zip those files in WinZip, etc.)

4c. As a shell extension, the program should create a virtual drive in the system. All virtual folders (collections) would reside on this drive. It is bad design IMO to put virtual folders among physical directories, since it makes them hard to distinguish and may lead to loss of data through confusion (deleting a physical file when you think you're only removing it from a collection, etc.)

4d. However, the program could also be implemented as a standalone application, though it would reduce its usefulness somewhat.

5. The program should support tagging (as in Tag2Find), rating, marking files with stars or what-not, describing them and specifying aliases, so that you could refer to files by simple aliases rather then their complete, long filenames. Tagging is not sufficient by itself though, since (a) it is time-consuming, (b) it is error-prone, as you will make typos, and (c) tags are useful so long as you remember what tags you have and what they mean. It would be easy for example to forget you already have a given tag, and attach what you think is a new tag to a bunch of files, thereby creating chaos in your collection. Tagging should be an important secondary feature; a collection is merely a named list of files.

6. Ideally, for some uses, the program should be able to detect when a file has been renamed or moved, and update the collection accordingly. This is non-trivial to implement, since monitoring all drives for changes to all files is probably going to put some strain on the system. Also, for certain uses (such as disc catalogs) you do not want such automatic updates. Without this feature however, simply renaming a file could make it invisible to the collection it used to be in, so it's a problem that needs consideration.

7. Working with files in the program should in many ways be indistinguishable from working with a file manager. Enter should open/execute a file; right-click should show the shell context menu for the filetype, etc.

So finally, here's the current state of play. Some file managers implement collections to some degree: DOpus, Xplorer2 and SpeedCommander. I'm not sure what the DOpus support is like - please share, those who know!. In the other two it's quite weak. You have to use separate panels, so they're not well integrated, and usually you can have only one collection (scrap window, whatever) at a time. There is a "temp panel" plugin for Total Commander, but again, its usefulness is almost nil compared to what I've described. I'm not even sure if those collections are persistent; in the TC plugin they are not.

Outside of file managers, there are a few specialized solutions (all commercial):

Virtual Folder: http://www.virtualfolder.net/
- Didn't really work for me at all. It creates a virtual "Z:" drive, which is good. On this drive you can create regular folders, but copying a file to such folders makes a physical clone, a second copy - that doesn't make sense. You have to create a folder, right-click it and make it virtual - but then you cannot open the folder in Explorer! I just don't understand how it's supposed to work, or why the Z drive is created at all, since the screenshots show how to create a virtual folder on your regular drives, among your ordinary folders (bad idea in the first place). I gave up. There is a new incarnation of this program at http://www.virtualdisk.net/ which I have not tried yet.

Tag2Find: http://www.tag2find.com/
- A promising start, but development is on hiatus and may not resume. Some good ideas, but Tag2Find does only tagging, which really involves much more manual labor than just drag-dropping files into collections, and cannot support some of the use cases I've proposed. For example, if you want to backup a collection of files, how do you achieve that if all you have is tags? Possible perhaps, but not nearly as straightforward as dragging a bunch of files from a collection to your backup app. And no automation, so you could not automatically generate a collection of, say, all *.xls" files under some folder. Also, the right-click interface in Tag2Find is awkward. It opens a tiny, tiny edit box for entering tags, but the edit box has no focus, so you have to click it first. Did I say it's tiny? I could see myself use Tag2Find eventually, but not at all in its current state, and the program isn't being developed at the moment.

My SmartFolders: http://www.castlepeaksoftware.com/
- Same idea as Tag2Find, somewhat different implementation. Haven't installed it yet. It's also a shell extension, which is good. However, it's tag-based only, which again limits the possibilities. Looking at the screenshots, I don't like the idea of tags as checkboxes that you click - what happens when you have a hundred or two hundred tags? Or more? (Because if you're only going to have ten tags or so, you probably don't need the program in the first place!) Also, the development doesn't seem to be moving forward. Link to the downloadable file was dead and the author didn't know about that until I emailed - so probably not much interest, either.

Benubird PDF: http://www.debenu.com/benubird/features.html
- The weirdest of them all! Looks promising, with the (completely unnecessary but attractive-looking) ribbon interface. True collections as lists of files, great. Uses SQLite database, great. Claims to monitor files for changes, renames etc., pretty good. Drag a file to a collection - Benubird creates a physical copy of the file under its data folder! What the...? Let me repeat the question: what happens when you create a collection of video files, between 700 MB and 4.5 GB each? And why copy the files in the first place? For what purpose?

That's about what I've been able to find so far. Wait, I've just discovered TaskTracker: http://tasktracker.wordwisesolutions.com/index.html Despite the name, it's apparently a file organizer with virtual folders. First time I've seen this, not tested yet. Last updated in 2007, though.

...And to anyone who's read this far...if you have really read this far... drop me a line next time you're in Poland, I owe you a beer! :)

MilesAhead:
From a programmer's pov I'd also like to see a Folder Selection dialog that didn't force you to either, click open + signs to expand a tree so you can click on the folder, or, type in the name of the folder(why have a gui if you're going to type it in?)

I'd like to see some kind of incremental search.  For example, if the only folder in the system with the name MyDownloads is under C:\EverybodysDownloads then when you type MyDo.. into the folder input line, the strand under C: should be expanded and MyDownloads highlighted.  Clicking 8 times to open a folder, or typing in the entire path is lame.  Probably why many use Windows search who would otherwise turn it off... but it's not that handy for program input as in "select a folder to copy" or whatnot.

They've been doing it the same why for how many decades now? Jeez!!

Shades:
Regarding the virtual folders:
Linux is doing that for years already. Directory Opus is also able to create and manage those (version 9 at least) inside DO.

housetier:
Some time ago (years, but how many?) I read about a file manager that gives you several "views" on your files: you have a calendar view with color-coded entries showing which project's files were worked on; you assign files to projects (nowadays one would say tags), filter by projects, date, age, and so on.

But I can't for the time of my life remember the name!  >:(

It was for Linux though...

tranglos:
Regarding the virtual folders:
Linux is doing that for years already. Directory Opus is also able to create and manage those (version 9 at least) inside DO.
-Shades (July 27, 2009, 01:30 AM)
--- End quote ---

Well, if I dropped Windows for Linux, I'd be out of work, so that's out of consideration :)

DOous - yes, that's why I mentioned it. I've never used it though, so I don't quite know the extent to which it supports collections. I'd be interested to find out which of my postulates are implemented.

Navigation

[0] Message Index

[#] Next page