topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday December 12, 2024, 4:30 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: How do you tag (or even organize) your files?  (Read 64691 times)

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
How do you tag (or even organize) your files?
« on: October 19, 2007, 11:40 PM »
This thread was created to answer a question about the way I tag files (I didn't want to hijack this other thread). But maybe we could all share our systems! [edit : just added that part to make this thread more... encompassing]

Like I said in the other thread, after much pondering, I've decided to append tags directly to filenames.

Some of the reasons for that decision are :

1- they (the tags) are always there and <b>visible</b> ("Filename tags" are "tangible" : can see them all the time, in explorer and any other desktop search tool), (tangibility)
2- they are compatible with any search tools (farr, X1, etc.) (compatibility),
3- they're available in any other modern OS (portability)
4- they're not that cumbersome to manipulate (change, add, remove, modify) with a good renaming utility (BRU, Flash Renamer) and a little bit of RegEx knowledge, and they're also VERY easy to add to a filename on the spot (in the "save" window, or in the "property" window) with AHK's help.. but more about that latter. (flexibility)
5- they're realiable (reliability and... robustness?)


So… Some details: as it is now, my tagging scheme is built around the abcz abbreviation system. I actually used this system awhile back, then stupidly abandoned it because of some wrong language considerations, and then came back to it (on rjbull’s advice — thanks again!).

The syntactic structure of my tags/categories is :

1-   "tgs" (a fixed string that indicates that the word is a tag — it could be something even shorter, like two numbers, but it’s not as convenient with farr)
2-   "X" (a capitalized variable which indicates a metacategory)
3-   "abcz" (the abbreviated theme, following the abcz system).


Following this model, here are some tag examples (just for illustration, won't go into too much detail...):

tgsOimpt (something important)
tgsOru (something I’ve already read)
tgsPphie (something related to philosophy)
etc.

These tags are placed at the end of my strictly formatted file names in which all “info fields” are delimited by specific characters.
e.g.: “Tags strings”  always start and end with a “+” sign. These characters facilitate file renaming, readability, and other file name related operations.

So, a fairly typical filename would look like :

xAR-.Eagleman,David M.(2005 11 09)__Time and the Brain__,,in The Journal of Neuroscience,,[No 45 vol 25]+tgsPscie tgsPbioe tgsPneue tgsPat tgsPthée tgsPPEP+.pdf

That’s 160 character just there. But I don’t care because my work flow has improved dramatically with that technique : I access everything in seconds with farr (or even X1 or Archivarius).


I use AutoHotkey’s help to implement that system. Using AutoHotkey allows me to :

1- make sure I never ever make orthographic mistakes,
2- never use a tag that "doesn't exist" in my "registry",
3- easily search through my tags (just use "search" in my AutoHotkey script) which are classified hierarchically (easy to do in AHK’s script),
4- easily add new tags or modify them,
5- have some tags to always appear together, grouped ; example : if I write "perl", I want AHK to write : "tgsOmi tgsOPERL" (which means : "something personnal, written by me...)
6- …etc. possibilities are uncounted

Of course, I organize my AHK script so that it’s easy on the eyes and pleasant to use (with lots of commented lines -- essential)!



My only gripe with that system is that, compared to a complete software solution like tag2find :

1- it's probably not as easy to replace/edit/change/erase tags once they're in the names,
2- it’s not as sexy as using a nice GUI,
3- and, mostly, there are limits to how many tags I can append to a file name (I've yet to have big problems with that, and hopefully, the 260 char. limit will be obsolete in a few years) …  Like I said in the other thread :

It's not perfect: file names tend to become gigantic and it's pretty important to keep them under control because of the windows 259-260 max path limit (IMO a stupid limitation, even if I understand why — when will that change!!! Amiga’s SFS allowed 32000 ack in 1998)... In my case, the 259 character limit is especially a problem for documents names — I put a lot of info in there : document type, author, date, title, etc. + tags ; I do that for very good reasons (don’t want to hear about windows files metadata capabilities : too inconsistent), it's extremely helpful. For applications, file/folder names length are not as preoccupying since I don't write anything else in the name but the program’s name  +  the tags or categories.)

Anyways, I still manage to be pretty efficient using software like BRU (Bulk Rename Utility) or Flash Renamer.


Regarding the visual aspect & the “folder problem” vixay mentions

But you know. I find that there is a difference in paradigm between tagging, and folder hierarchy. Tagging almost always requires you to search, and doesn't give you a good folder level view, but gives you quicker access to stuff you need. Whereas folders, give you good views, allows exploration, but makes finding things a bit harder. I think when we reach the tags = folders in explorer stage is when it'd be great! (like the delicious extension in firefox!)

yes… I know what you mean. It is a problem. I currently use X1’s ability to save different type of searches in a folder tree, and I do create temporary folders I windows’ file system for current projects. (I usually destroy the folders once I’m finished though). It can make it easier to follow a project, depending on what I’m doing.

I’d actually really like to use “true” virtual folders more, but I found ALL the “virtual folders” solutions that I’ve tried are unreliable : change the name of the file inside the virtual folder, move it to another location in the true folder, etc. and weird things start to happen (e.g. :  tried the “file collections” feature in Directory Opus : I trialed it a while back and it didn’t work as expected).**

Sorry for that long post.   :)

So... How do you tag YOUR files?

**
Spoiler
Actually, yes the best potential solution I found is tag2find (it could also be used to create virtual folders). But it didn’t pass my compatibility tests and portability requirements. To me not only a “certain degree” of portability is important, but I also want to keep some kind of control over my data (even if created with a piece software that's proprietary). In that spirit, I find that file identificator/metadata/tags shouldn't be just lost in the advent of file transfers, file backup, etc. (a likely possibility with all current tagging software I’ve tried). Also, aside from portability, one thing that "bothered" me with tag2find  was that I couldn’t see the tags while I was in explorer (or any other desktop search tools or file manager for that matter) : tags were pretty much invisible unless the tag2find explorer was opened. That made the tags and the "tagging endeavor" a pretty abstract thing... they could only be used with tag2find, but not with X1, farr, etc.

Some of these criticisms might be addressed in the future beta release (end of Octobre, actually), and APIs might be released for software like launchy — hopefully, farr will be able to tap into that opportunity too… We’ll see.

« Last Edit: October 20, 2007, 08:34 PM by Armando »

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag files?
« Reply #1 on: October 20, 2007, 05:23 PM »
I have modified the content of my first post and the title of the thread to make it more encompassing and encourage participation... as I'm really curious about everybody else's strategies...

justice

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,898
    • View Profile
    • Donate to Member
Re: How do you tag files?
« Reply #2 on: October 20, 2007, 08:28 PM »
we need a document management solution for the desktop.
make my documents a database,
with tags, versoin control and notes about where files are copied to out of the system.
I admire your persistance but in the end its just abusing a crap system (the concept of filenames and organisnig by folders).

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag files?
« Reply #3 on: October 20, 2007, 08:33 PM »
we need a document management solution for the desktop.
make my documents a database,
with tags, versoin control and notes about where files are copied to out of the system.
I admire your persistance but in the end its just abusing a crap system (the concept of filenames and organisnig by folders).

I think agree.  If I understand what you say...
If I could program and design something better, I would try to help in this endeavor... that's for sure. (It's the kind of project that would make my mind happy -- still wondering what I'm doing in the theatre field...)

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #4 on: October 21, 2007, 03:17 PM »
In OS X all files have a Spotlight comments field. So in finder one calls up file info (⌘i) and adds space separated word tags. I use &tag notation to know what was a tag and what was a comment.
My launcher Quicksilver, leverages Spotlight for tagging, and this is where things get *cool*. I pattern-match find my file [DENN], then [TAB], then pattern-match my action [AD]d tag..., then type in my tag [ORWELLIAN]:
step1.png

This adds the tag to my file. I can queue up files using [,] multi-selection:
step4.png

The actions I have are — [Add tags...] [Set tags...] [Remove tags...] [Show tags]. Ok, I've tagged my file(s), and I now want to search for some tags. In Quicksilver, a tag catalog is stored so I can pattern-match search my tags:
step2.png

That utilises the core OS Spotlight mechanisms to find my tagged files:
step3.png

The cool thing about this, as Spotlight is a global and core OS service, I can integrate it with other software (or manually tag files if I choose); for example use Ruby/Applescript/Automator to handle an infinitude of further tasks (auto-magically upload tagged files to an FTP server etc.). I can also save "Smart folders" in my Finder so I have pseudo-folders of tagged files available. This workflow is as simple and elegant as it gets ;-)

FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]
« Last Edit: October 21, 2007, 03:26 PM by nontroppo »

vixay

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 140
  • ViXaY
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #5 on: October 22, 2007, 03:36 AM »
Sorry for the delayed response, but it took a while to read the other links. Anyway the Spotlight approach sounds cool, though since i haven't used a mac interface regularly enough i don't get the concepts.

As for Armando, what AHK scripts do you use?

I think the whole issue comes down to, how can we use our computers to be more intelligent and do things for us, rather than have us do things for the computer (so that it makes it easier for us later).

So a database would be a good idea, as long as it was seamless, and didn't have me pulling out my hair! :)

A lot of these document controls do exist, but at a great cost of learning curve, and performance hit. Anyway lets see...
"Drunk on the Nectar of Life!" -me

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #6 on: October 22, 2007, 12:20 PM »
Nontroppo : thanks.

I’m abit in a hurry, but here are some thoughts…

First…I love QuickSilver...**  The tagging possibilities you're showing are nice. They seem similar (in features only, must I emphasize...) to what I've seen from the new explorer/Desktop Search in Vista — meaning : more intelligent usage of metadata, and better integration. But Quicksilver seems to make the integration more fluid and useable. Can't compare right now though.

How much is that metadata portable though ? Is it strictly reserved to OSX and would be lost if read by windows or Linux? My experience with metadata fields for files is that they’re not that reliable and/or portable.
 
The new Tag2find is supposed to offer some kind of integration in explorer and other desktop search tools and launchers like farr (via some specific APIs) too. So, those with XP might be able to soon have a better system to tag files (with tag clouds, statistics, etc.). Still, all these solutions (including the one provided with QuickSilver / MacOSX ) leave me with the same feeling of "insecurity": how permanent is that metadata, that tagging, how portable is it, how robust is it?

Tagging stuff is as time consuming as naming files or classifying them in folders. If all that work is going to be lost by changing between Oss or just backing up, etc., it's not a perfect solution. Tag2Find is supposed to provide some solutions to keep trace of all tags in the the advent of data transfert… we’ll see.

(Nontroppo : in another you talked about the proprietary format of tag2find’s databas. Would you say the OS X format is not proprietary?)

 
If you look at my points in my first post, you'll see why I decided to include everything in file names -- maybe not very sexy and elegant solution, I’ll admit, but the only one I'm sure will pass the portability test relatively well. If I switched to Linux Or Mac OS tomorrow I’d be able to use my system as if it was native.


 
vixay : calling my script a script is actually probably a bit too much :D. I’m mostly using AHKs text replacement capabilities.

Here are some examples :

; COURS PRÉPARATION(s)
:oc:xcp-::xCP-.Mnz ou Nom, Prénom.(Dte)__Nom du projet - Titre, Nom du cours, code etc__"PLUS"TGS sujet organisme compagnie etc"PLUS"{home}{right}{right}{right}{right}{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}+{right}

; Assign Ctrl-shift-Alt-R as a hotkey to restart the script
^+!r::Reload

; Assign Ctrl-shift-Alt-R as a hotkey to restart the script
^!e::Edit

; Revue Éparts (ancien : pjREVÉPA)
:oc:ppreép::tgsPReÉp
:oc:Preép::tgsPÉtRe tgsPunié tgsPdoct tgsPReÉp (tgsPat tgsPthée)
:oc:preép::tgsPÉtRe tgsPunié tgsPdoct tgsPReÉp

; PERFORMATIVITÉ-EFFETS-PRÉSENCE
:oc:pppep::tgsPPEP
:oc:ppep::tgsPÉtRe tgsPunié tgsPdoct tgsPThDo tgsPat tgsPthée tgsPPEP
:oc:Ppep::tgsPÉtRe tgsPunié tgsPdoct tgsPPEP


I think the whole issue comes down to, how can we use our computers to be more intelligent and do things for us, rather than have us do things for the computer (so that it makes it easier for us later).

I always try of course to have my computer do things for me, of course… whether or not I succeed is another story.

So a database would be a good idea, as long as it was seamless, and didn't have me pulling out my hair! 

I personally find problems with traditional databases. Especially when it comes to backup and portability (e.g. : it can be very annoying/infesable to have to backup a 50gb file every night). There might be ways I completely ignore, but from what I’ve seen…
AskSam, Ultra Recall, myBase, General knowledge Base, treepad, and others more PIMed, like Info Select etc., it’s pretty much always the same story.





 
**
Spoiler
... even if I genuinely do prefer Farr (go  mouser go!!! ;D)... e.g. Farr has better file manipulation-browsing possibilities, the result view is more detailed, it'S very customizable, and it searches without an index by default -- a must for someone like me who messes with folders and files all the time.
 
QuickSilver also has some advantages (some are user friendliness, sheer beauty, speed, and great plugins) but none that I miss with farr. I won't go into a detailed comparaison of farr and quicksilver as it'S been months since my last intense quicksilver usage. But it could be fun if you'd do  one! You seem like the perfect person for that.


nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #7 on: October 22, 2007, 06:37 PM »
How much is that metadata portable though ? Is it strictly reserved to OSX and would be lost if read by windows or Linux? My experience with metadata fields for files is that they’re not that reliable and/or portable.
Well, the metadata system is based on the filesystem (FS), HFS+. Each FS has a different way to handle metadata. HFS+ uses resource forks, which are like NTFS alternate data streams IIUC. So if I copy a file to Windows, it will most likely lose the metadata because the FSs are incompatible.

BUT, it is quite easy to handle for Spotlight. There are OS command-line tools that extract this metadata out. It would be trivial I assume to set up a folder action to extract the comments and write them to an XML file. So I could have an drag-n-drop "upload" folder, who's job it is to automagically upload a file(s) to a remote FTP and extract the comments to XML as a sidecar file.

Tagging stuff is as time consuming as naming files or classifying them in folders. If all that work is going to be lost by changing between Oss or just backing up, etc.

Most command-line utilities, and good backup software is fully compatible with HFS+ features. Backup is not an issue.

Nontroppo : in another you talked about the proprietary format of tag2find’s databas. Would you say the OS X format is not proprietary?
It is part of the standard core OS filesystem. OS X has exposed APIs to deal with it, and those are easily used by any application developer or scripter. So it is non-proprietry within-OS X. The problem is obviously platform incompatibilities, which is something we'll be stuck with for a while unfortunately (a unified file system for all OSs would rock, but I suspect they'd squabble for 200years before arriving at a decision!). My XML sidecar file is the best I can come up with, but then Windows software would need to be able to translate that to whatever it uses. I have no idea what tag2find uses. Their FAQ provides no information about where they store their data, but as you say they are writing an API. I'd still prefer that this is built in to the OS personally, but I don't know when Microsoft will get round to implementing this (their next-generation file OS was supposed to come after XP but got pulled IIRC). I don't know how Linux handles metadata either...
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #8 on: October 23, 2007, 01:48 AM »
Thanks nontroppo! Very informative comments.
The problem is obviously platform incompatibilities, which is something we'll be stuck with for a while unfortunately (a unified file system for all OSs would rock, but I suspect they'd squabble for 200years before arriving at a decision!). My XML sidecar file is the best I can come up with, but then Windows software would need to be able to translate that to whatever it uses.
Like I said, this is really why I’ve decided to stick with file names. Of course, I’d prefer another solution… if it’s as portable and robust.


I have no idea what tag2find uses. Their FAQ provides no information about where they store their data, but as you say they are writing an API. I'd still prefer that this is built in to the OS personally, but I don't know when Microsoft will get round to implementing this (their next-generation file OS was supposed to come after XP but got pulled IIRC). I don't know how Linux handles metadata either...

I actually asked them a while back, and this is what they answered :
At the moment, to prevent you from losing your hard work, we provide one basic backup possibility: export to a plain text XML file. The schema of the XML is very basic simple and will for sure prevent you from a "vendor lock in", which we understand nobody really wants. The backup has some downsides at the moment, as the files are stored with absolute path, but it will always allow recovery in case of a disaster, maybe requiring a little bit tweaking with a text editor in case the location of files has changed.

We do not really make a very big secret out of how our tags are stored: at the moment they are stored in two locations: a system-wide tag-database (SQLite) and attached as Alternate Data Stream to the file itself (which is the reason why we can only support NTFS at the moment). Tags can be recovered from backup or the NTFS Alternate Data Stream in case the central database corrupts (which is highly unlikely). Alternate Data Streams are copied together with a file by Windows Explorer, as long as the target volume supports them.
-martin – tag2find team on June 19 2007 -- http://forum.tag2find.com/ftopic85.html

I suspect that the system is now (or will be) even more robust and should integrate with explorer (API’s for Launchy or other Launcher should also soon be available). So… we’ll see.

In terms of OS, OS X seems like the one with the best support for meta data. Linux (Ubuntu, PCLinuxOS…) is getting there to. The biggest problem, like we said, is interoperability, compatibility, portability…

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #9 on: October 23, 2007, 06:56 AM »
The biggest problem, like we said, is interoperability, compatibility, portability…

Aye indeed, what I'd give to lock MS, Apple, and *nix engineers up together in a room for a week and actually think about inter-OS operability.  :Thmbsup:
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #10 on: October 23, 2007, 10:08 AM »
Good idea! 8)

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #11 on: October 23, 2007, 04:38 PM »
On a slightly tangential potential future tagging route, I just stumbled on the Mcnucle server:

http://www.inuron.co...tware-overview.php#3

The general ideas are a metadata driven tagging system at its base (seems to use spotlight?), multiple views of files (via tags / ratings / projects) in smart-folders, and a built-in server to abstract this metadata'd "stuff" out to any machine in the æther. You can send links of individual files to friends they can get from the server, but more interesting is you can share your metadata. Allow family to see photos from a tagged set, it doesn't matter where it is on your machine, you don't need to zip it up, just share it...

Prosaically, you can use this as I use SSH now to get to my work machine from home, except all my tags are preserved and browsable. But what is neat, is that you can aggregate tagged data from machines anywhere. So tagged stuff at work can get aggregated into my laptop tag search results even if I'm in an Airport in China. The server is OS X only (all written in Java so hopefully it will get ported) but the client runs under OS X or Windows and parts in any browser. Currently in beta but very intriguing.
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]
« Last Edit: October 23, 2007, 04:40 PM by nontroppo »

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #12 on: October 25, 2007, 10:52 AM »
Slightly tangential, but definitely interesting. Thanks for sharing that, nontroppo!

nosh

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 1,441
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #13 on: October 25, 2007, 02:41 PM »
I like to think I'm organized but I can see I'm out of my depth here with you super-organized dudes.

Anyway, just to share...

I personally have two ways, as an X2 user I can color code files and even folders depending on any kind of attribute or metadata so it's only limited by the imagination.
I use a few tags that I set in the file comments, rather than filenames coz I prefer to keep the original filenames.
x2 lets me:
1. add/edit comments in bulk
2. display comments in details view (though I see no need for this coz I don't have an overly complicated system of tags and color coding is good enough, not to mention a lot of fun to eff with  ;))   
3. search on comments.

Clipboard01.jpgHow do you tag (or even organize) your files?

If I want to rate a bunch of music files (1-5), I just rate them as they play in Winamp and then batch edit the comments from within winamp using the ml functions, voila! - the files immediately show up with their rating colors in x2. The ratings can be imported back if winamp ever loses its data.

Putting your tags right into the filenames is definitely a safer/more reliable way, though if the names get too long it may actually end up being counterproductive if you're seeing a whole lot of files at once (not that you'd see that many to start with if they have such long names!). My method targeted mainly at comments is not as robust or as effortless as I would like it to be, but it serves the purpose. And colors, did I mention colors!!  :-*

Another relatively simple favorite of mine is to have ahk text replacements for current date+time, this way say if a friend regularly sends me pix I just have one main folder for the person and sub folders named something like "20071026 005818", I like the Japanese date format coz it's best for sorting, v.sharp those guys! :)
« Last Edit: October 25, 2007, 02:50 PM by nosh »

justice

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,898
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #14 on: October 25, 2007, 05:33 PM »
do you have a link for x2 can't find it with google.

tinjaw

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,927
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #15 on: October 25, 2007, 05:39 PM »
do you have a link for x2 can't find it with google.
justice,

I think he is using that as shorthand for xplorer2.

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #16 on: October 25, 2007, 10:01 PM »
Nosh :
Thanks for sharing! There are some interesting aspects in your system (like the color coding, X2 [thanks tinjaw!], etc.). I don't have much time now, but I'll think about all that in... a couple days maybe.  :)

jared1999

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 9
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #17 on: November 05, 2007, 04:17 AM »
Windows XP (and Vista I presume) has built-in metadata support. Properties -> Summary on a file gives you this:

file-summary.png

These fields should be searchable.
« Last Edit: November 05, 2007, 04:19 AM by jared1999 »

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #18 on: November 05, 2007, 05:52 AM »
unfortunately, these fields are NOT always:
1- reliable (not always accessible for all file types, etc.)
2- searchable (through most desktop search software)
3- portable

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #19 on: November 05, 2007, 06:24 AM »
As I understand it, Windows has supported metadata (Alternate data streams, ADS) in some form since 1993 when NTFS was released; it was added to provide compatibility with Apple's file systems metadata and Novell.  In Windows 2000, MS added Author, title fields etc as shown above for XP.

But ADS appear to have been particularly attacked on Windows for security reasons (hiding stuff in "files") -- and so has sadly never really taken off. That critique could apply to any FS, but Windows is more security-concerned. The technology is/has been there but the culture certainly isn't/wasn't. Vista exposes this more clearly (i.e. command line awareness of ADS), and I very much hope the Windows community will standardise and promote ADS for functional metadata across applications. Sadly the current result is each app has its own index, its own metadata representation.

Historical aside: Regarding tagging and metadata, the most elegant original file system was the Be file system from BeOS:
http://en.wikipedia..../wiki/Be_File_System

Metadata was very well integrated, and a search system like a relational database built around it as a core OS function. It has taken years for others to catch up, if at all[1].

----
[1] http://arstechnica.c...os/macosx-10-4.ars/6 -- Apple basically hired the creator of the BeFS, though it took until Tiger to get close to parity. That article (pages 6 and 12) show the battle to go from metadata as possibility to rich metadata as functional reality.
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]

jared1999

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 9
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #20 on: November 05, 2007, 07:49 AM »
unfortunately, these fields are NOT always:
1- reliable (not always accessible for all file types, etc.)
2- searchable (through most desktop search software)
3- portable
Too bad. I should have known there was a reason I haven't seen anyone use those fields. ;)

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #21 on: November 06, 2007, 03:30 PM »
Armando: do you know why ADS doesn't work for some file types? Is it than that programs that open them strip the ADS out?

Is that why Tag2Find stores tags in both a database and the ADS?
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,169
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #22 on: November 06, 2007, 04:55 PM »
Armando, thanks for sharing your system.

My file name tagging differs a bit from yours:
- I use around 20 intuitive (self-constructed) subject matter abbreviations like "tv", "todo", "note", but mostly for different academic subjects.
- I use no tag indicator string apart from a "-". Instead I tend to put all tagged content in a few basic folders and then do searches on those folders. I have no problem finding things without such indicators.

Armando

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,727
    • View Profile
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #23 on: November 06, 2007, 11:48 PM »
Armando, thanks for sharing your system.

My file name tagging differs a bit from yours:
- I use around 20 intuitive (self-constructed) subject matter abbreviations like "tv", "todo", "note", but mostly for different academic subjects.
- I use no tag indicator string apart from a "-". Instead I tend to put all tagged content in a few basic folders and then do searches on those folders. I have no problem finding things without such indicators.

The reason why I use tag indicators and a coherent system are multiple. Here are some of them :
-   I want to be able to find certain groups of tags, regardless of their particular signified – I can’t use a punctuation mark as a identifier because not all desktop search software will take them in consideration
-   I want to be able to use the same tags I use for filenames INSIDE documents (e.g. : in word or pdf comments, etc.), and the tags have to be as visible and unique as possible to ease the search
-   I want to simplify the bulk renaming process (using basic regular expressions)
-   I want to use these same tags to identify projects in Outlook and create project/tasks hierarchies (a missing feature in Outlook)
-   I want to be able to create tags easily AND reconstruct their meaning as easily (so the abbreviations need to follow a system),
-   Etc.

It,s basically a matter of coherence and efficiency in mutiple (different) situations of application

Armando: do you know why ADS doesn't work for some file types? Is it than that programs that open them strip the ADS out?

Really, I’m note sure. But what I know —someone correct me if I’m wrong! — is that in NTFS the data gets written behind the file which might be a problem for certain program (that seam to overwrite it),  but especially if you copy these files to a CD, a DVD or another file system (like fat32) for backup. I don’t know about HFS+ specific tagging strategies.

Also, you’ll notice that the whole business of using the special fields available in explorer (author, comments…) when you click on property, is shaky ast best. All fields are not always available — there’ll be either absent, grayed out, or won’t accept any data! — and sometimes the data entered in the fields don’t even appear in explorer! And these are just a few inconsistencies. Wow.

here's an excerpt from http://www.flexhex.c...ernate-streams.phtml


So When to Use Alternate Streams?
Certainly you should not use alternate streams for storing any critical information. Older file systems are still widely used, and they don't support the advanced NTFS features. If you copy an NTFS file to a USB drive, flash card, CD-R/RW, or any other non-NTFS drive, the system will copy the main stream only and will ignore all the alternate streams. The same is true for FTP/HTTP transfers. No warning is given, and a user, relying on alternate streams, might get a nasty surprise. So the Microsoft reluctance to provide user tools for alternate streams is not all that unfounded.
However alternate streams are still extremely useful. There is a lot of non-critical information that alternate streams is the most natural place to store to. Examples are thumbnails for graphical files, parsing information for program sources, spellcheck and formatting data for documents, or any other info that can be recovered easily. This way the file can be stored on any file system, but keeping the file on an NTFS drive will greatly increase processing speed.

Is that why Tag2Find stores tags in both a database and the ADS?
Well, yes., it seems like it. But I can’t speak for the developers. Like I’ve posted elsewhere, I asked questions to the developer about the tagging system and here’s what on of them answered to me :

At the moment, to prevent you from losing your hard work, we provide one basic backup possibility: export to a plain text XML file. The schema of the XML is very basic simple and will for sure prevent you from a "vendor lock in", which we understand nobody really wants. The backup has some downsides at the moment, as the files are stored with absolute path, but it will always allow recovery in case of a disaster, maybe requiring a little bit tweaking with a text editor in case the location of files has changed.

We do not really make a very big secret out of how our tags are stored: at the moment they are stored in two locations: a system-wide tag-database (SQLite) and attached as Alternate Data Stream to the file itself (which is the reason why we can only support NTFS at the moment). Tags can be recovered from backup or the NTFS Alternate Data Stream in case the central database corrupts (which is highly unlikely). Alternate Data Streams are copied together with a file by Windows Explorer, as long as the target volume supports them.
« Last Edit: November 06, 2007, 11:52 PM by Armando »

nontroppo

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 649
  • spinning top
    • View Profile
    • nontroppo.org
    • Donate to Member
Re: How do you tag (or even organize) your files?
« Reply #24 on: November 07, 2007, 04:50 AM »
Thanks for that info Ralf. The solution HFS+ uses is to generate ._sidecar files if a file with metadata is saved on a non-compatible drive. That way the resource fork is preserved even on non-compatible drives, and will be read back in from the sidecar when it is loaded from such a drive. *BUT* this is seen as file detritus to those on that other FS (who don't know what they are), and there are several programs to clean the ._sidecar files away or people just delete them anyway, breaking that metadata...

I've been playing with Mcnucle for a while for tagging. What is great is the preservation of my tagging schemes no matter from where I access that, though I haven't hammered the system to see how thoroughly it preserves it on a long journey. What I don't like its the limitations for tags - it is very consumer oriented at the moment (few tag choices), so I've decided to wait a while before using it more seriously.

What I do for tagging now though is a free tool Spotmeta. It does the tedious job of metadata scheme construction seamlessly. You can make any type of metadata types (booleans, lists, strings etc) and use them for categorisation. It uses standard OS metadata mechanisms, contextmenu support from anywhere, uses a namespace to stop collisions, and makes spotlight aware of the tag schema thus fully searchable.
FARR Wishes: Performance TweaksTask ControlAdaptive History
[url=http://opera.com/]