topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 4:00 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Three-dimensional file access in Windows, especially by Hard Links  (Read 3997 times)

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 115
    • View Profile
    • Donate to Member
I'd been interested in hard links, a subset of symbolic links; we're speaking of newer Windows versions here.

From this quite recent post: https://blogs.window...symlinks-windows-10/ (December 2, 2016 2:00 pm - Symlinks in Windows 10! - By Yosef Durr / Lead Senior Program Manager), you could infer that they had been made more accessible to common Windows users now, but in the end, I had to set my system to "Developer Mode" indeed, in order to be able to create hardlinks with my macro program (open a cmd window, trigger the necessary commands, auto-close the cmd window); with direct API access it should be smoother, visually, but I don't know how I would do that; in any case, no manual intervention is asked for, it's just the popping up of the cmd window which isn't that pretty.

Hard links are robust when it comes to renaming or moving of the "original" file name (or any one of the "links") - in fact, there is not such an "original" file anymore, but there's that data thing in the hdd, and then just several links to that data body, the original name being just one of them - that's how NTFS works anyway, so these NTFS-only links appear to be a perfect way to do file cloning (instead of creating copies), even though it all doesn't function but within the same drive - I suppose you could install some super drive for this, i.e. a common denominator for anything below it, like multiple hdd's in some NAS, so this one-drive-only limitation doesn't exclude the use of symbolic links from "corporate" use if the "corporation" is a pop-and-mom office or the like.

Of course, there seem to appear big problems with web storage ("cloud"); some people in the web speak of using it for this, but I cannot imagine any inter-office-web use, so they've got probably everything (and only, except perhaps for backup purposes) in the (same) "cloud".

Where things begin to get nasty - hence this thread -, read here:

https://www.zive.cz/...98/?consultanswers=1

cpbbt2  |  15. 08. 2014 09:44  |  Microsoft Windows 8.1 IE 11.0  |  [5.104.22.---]

What you have to consider is the way an application/program works on a file. Some edit the file directly, and they should work perfectly fine with hardlinks. Although I didn't knew the "fileinfo difference" problem until now ... Other applications copy the file and then delete the old one when they save their edited version. This programs obviously break the hard link with that hidden copy operation. What they save is no longer the file they had opened. When they then delete the old version, they remove this single hardlink from the directory. Other hardlinks remain, and so you get two files in the end.

Lots of blah-blah discussion here: https://superuser.co...ntents-do-not-affect but they don't get it any better than that (and they don't even cite the source).

Of course, I encountered the problem described there, immediately, with my very first trial files: You make, from some original file x.xyz, a copy (!) 0x.xyz, then create hard links 0x1.xyz, 0x2.xyz, etc. from 0x.xyz, and then you can play around with these all-non-original "files", i.e. links to the copied original; first copying is important since you will want to extend your playing around to also deleting the first name in the line, the 0x.xyz, in order to see if really there is no difference anymore to the other, later links (since you created these with the mklink command, while the 0x.xyz had been created in the traditional way): You open some of these file aliases, change them, close them, then check if other aliases you open have got those changes or not.

(I did not check what happens when you open that same content body, under two different names, with 2 different programs at the same time, and if that is possible to begin with - probably yes... IF one of the 2 programs comes with the problems described above.)

Now imagine somebody trying to create some office software relying on hard links, and the customer service they would have to do afterwards, with all those angry customers having created multiple alternatives from the same original files...

BUT this being said, hard links are probably a fine, elegant, file-system-only solution for pictures or other files of a certain kind and of which you first must be sure you will never use a program upon, which will create the problems described above; also, the same picture programs could behave differently for just opening / displaying some pictures, and then processing them, for example some dedicated picture viewer but which also has some basic processing features.

So before adopting hard links for pic categorization, you would have to do many verifications if it's viable in your case, but if it is, such a file-system-only solution is without any doubt much superior to any commercial database solution, for pics, since in the latter, you only have your multiplicity of the same pic / document only in that dedicated program - the same is true for dedicated tagging programs of all sorts, except for a single exeception among them which writes the tags into the filenames, but you could do that yourself, with any good file manager (thus offering a good multi-rename function) -, while with hard links, these "clones" would be available from anywhere, from within any file manager, any pic browser, and thus also be available TO any special dedicated program which does something DO to these files, so if you exactly know what program and/or program functionality does, how it behaves in this respect, the use of hard links, together with some automatic check functionality which then, in case, handles the aliases, could certainly be an almost optimized solution.

The same for office use, the only problem being that then such an office software would prescribe the use of some restricted list of "acceptable" applications, either because they or the file formats in question don't represent any problem, or because that centralized program knows how to "react" whenever such an application works upon a file within those "monitored" directories (which would be a prerequisite in order to get such a system functional). (NB: Some file formats may be without problems up to now just because there isn't any application yet which messes around with it in this way.)

Whereas traditional links are totally unreliable; they've egen got some (!) functionality in order to not being broken when the original file (here there is a distinction between original file and link files indeed) is renamed or moved, but from my experience, you should never rely upon it, it works here and there, and then again not, so...

So whatever that MS "Lead Senior Program Manager" says, a really viable solution to the everlasting "it's-just-a-tree-instead-of-a graph" problem inherent to NTFS (not speaking of FAT32 et al.) hasn't been found yet, or in other words, MS don't do their homework.



EDIT August 1st, 2017: Just title change, for the title better reflecting the extended content.
« Last Edit: August 01, 2017, 07:18 AM by ital2 »

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 115
    • View Profile
    • Donate to Member
Three-dimensional file access in Windows, especially by Hard Links
« Reply #1 on: August 01, 2017, 07:16 AM »
I'm surprised there has been no reaction to this post yet since hard links seem to provide a valid third-dimensionality in Windows at least for some use cases.

Some more details. Hard links seem to be no subset of symbolic links, as I wrongly stated above, but info about all those links in the web is really chaotic, and I currently don't grasp the difference (if there is any) between traditional .lnk files and "symlinks", symbolic links. As for applications messing around with, ie breaking hardlinks (ie creating independent file copies instead of treating the original file you want to stay unique), even some plain text editors do this, while others work fine, so you must check every application one by one, and even re-check them after any update, since it's perfectly possible that new versions behave differently from the application's traditional behavior - they call this "progress" then... but without telling you about the unwanted side effects... provided the developers themselves will have discovered them, which isn't necessarily the case...

The main file managers, xplorer2, Directory Opus and XYplorer all seem to create hardlinks from within their respective menus; TotalCommander has got some add-in in order to do this; I don't know other file managers which do it, too, currently, but there might be some.

There is a problem inherent to hardlinks: Since they are all just other denomination/aliases for the original (and unique) file, just like (technically, not by what we'd call it) the very first "hardlink" (see also Leo in https://resource.dop...ight-hardlinks/16493 , see also below re FPV, I didn't know that special feature there), the regular file system list entry of some file is, deleting the very last hardlink of a file deletes the file itself, or in order words, using hardlinks, when you delete some file entry in your file manager or elsewhere, you can never be sure if you aren't deleting the very last entry, ie the file itself, instead of just deleting a reference. (See my post today re vicinity search: Of course, the "Del" key in some hit list should just delete the file reference from the results list, not the file in question: this is so obvious that I even didn't mention that.)

x2 and DO both come with some - unsufficient? - means to try to prevent file deletion (and since I didn't check, other file managers may do similar things here); some list functionality in DO (see https://www.gpsoft.c...ks_and_Junctions.htm ; not using DO, I cannot say how this would work in practice), and an additional column in x2 which seems to be the best solution to the problem so far: In x2, if you haven't got list view but details view, you get columns with additional info (for example also for comments, in x2!!! (but unfortunately, these seem to be specific to x2 and cannot be read by other tools checking for the generic comment meta data strings)) - so far, so common behavior of most file managers -, and there seem to be one column for "number of file system entries for the current file" indeed.

x2's isn't the ideal solution to the problem yet, though, since if there is some file in the current folder and which has got an entry count of 2 in that column you will visually check before any file/entry(!) deletion, and if then you think you can safely delete the file's occurrence in the current folder, the (before the deletion) second entry in that column may come from some instance of the file which is already in the bin, and if then you press "Del", the file itself is lost.

What I don't understand here (I didn't try for myself, take the above from x2's forum): A deletion of an entry is deemed to decrease the counter; how come then that a counter of 2, for 1 entry in your current folder and 1 entry in the recycle bin, is possible? Leo (link above) says what I said above: technically, hardlinks are identical to the very first entry of any file, so technically, when you delete a hardlink, there is NO difference to deletion of the "original" file entry, so HOW come a deleted hardlink in the bin shows in the entry-count list, when at the same time hardlink deletion would reset the counter? (In the bin, it cannot be the "file itself" but must be an entry, since the count is for entries, not for files: if it was, the count would always be 1.) So these cannot be true at the same time and should be investigated further; I suppose that if the bin problem is true, then the count to zero will not work but immediately after deleting the bin, or if the bin problem doesn't exist as described in the x2 forum, it would be very easy to implement a safe function for ANY file manager, with or without visual indication (as in x2, DO...):

For ANY deletion of an entry, instead of forwarding this directly to the system (Windows), the file manager would need to check for the count, and then deliver a warning dialog if the entry count is just 1 for some file - which, of course, is the case in 99,999... of the cases for people who don't make extensive use of hardlinks, since "count 1" would be identical to any regular file entry.


For pictures, video clips, music files and the like (subsequently: "pics"), a file manager isn't ideal since you would want to "consume" those files, look at them, listen to them, and most file managers aren't that very good at that; x2 is particularly bad at it, while both XY and DO aren't bad at all here, so they could replace a dedicated pic (etc., see above my "definition") viewer indeed, DO probably even better than XY, but it would be preferable to have this functionality within your dedicated and preferred viewer application indeed.

For example, I prefer FSIV (FastStone Image Viewer) for pictures, over XY (I don't have DO), and it's not very probably FSIV will EVER get such an entry counter/alarm function, considering that currently, even NOT ONE file manager comes with such a functionality; as for the creational function, I've said it above, you do NOT need this function in-built within the application you're using, you can easily do it with some macro which can be applied everywhere: In fact, two macros: 1) identify the current item (with full path, in your file manager, in your pic tool, whatever, and store it, in the clipboard (bad) or in some variable (much better)), 2) create the entry/hardlink, within the (then) current folder in your file manager or other browser tool (and which could even be another one than the one used in 1), with the name you want it to have (input box, default = original file name, also stored in-between).

It's also understood that macro 2 would be ready to be triggered multiple times, for the same input of macro 1 (multiple aliases, the max number per file is 1023, ie 1024 incl. the "original" one, so this should suffice for most use cases, ha, ha) - it's the deletion counter which is missing from applications, not so much the in-built creational functionality... well, at the end of the day, you could create the check-the-aliases-numbers-for-each-deletion on your own, too, making your own functionality overriding the in-built deletion functions (ie intercepting the Del key, etc.) - that could become a little bit complicated for mass deletions, of course... Also, what about moves to different drives? Without having tried, I assume that the innocent moving of an entry ("original" or (other) "alias") will break ALL entries of that file on the original drive (?) since the Windows copy/move function (which the application in question will trigger then, in most cases - some file managers have "robust copy" and such, which behave differently and do, more or less, their own stuff) simply doesn't cope (yet) with hardlinks?

Of course, what you would like to get instead - or perhaps it even functions this way already -, is the creation of a copy (but which will then not be synched in any way further on of course) on that other drive, while leaving the original file (and all its possible entries) on drive 1 unharmed - but copy, with an additional message indeed, telling you the move's making a copy instead since the original file's listed within more than one location; on the other hand, the copy command when copying to another drive should work as expected even now: It would simply copy the current entry of your file to the other drive; for example, if your "original" was was a.x, and your current alias of the file (always on drive 1) is b.x, and there's also some other aliases for that same file, c.x, etc., you'll get a copy of b.x (not a.x) onto drive 2, and any hardlinks over there would be aliases for that new b.x which in time could differ quite considerably from the b.x, a.x, etc. on drive 1 of course - so, as said before, hardlinks come with a cost anyway...


BUT as said above, for pics and so on they could represent an ideal solution for some. As DO's Leo says in the link, FPV (Fast Picture Viewer) has even got in-built functionality... well, I suppose that's just for the creational side. Now FPV is for photographers' needs only, ie for pre-selecting valid pics from the ones coming from your camera, and it's reason for being (raison d'ĂȘtre) is the fact that Adobe Lightroom - where most photographers will then import their FPV preselection into, anyway - is as sluggish as it is; I'd prefer the PhaseOne contender instead, since I think, after some really dreadful experiences, that the LR workflow is absolutely terrible anyway, also and especially for photographers, but as always, your mileage may vary; in the end, it all comes down to the old question if you prefer your things to be in the file system, or then in some never-ending hybrid state, them being always within the file system, but with cryptic names and not accessible anymore but thru some dedicated database application; for information management / office workflow of office documents, that might even be inevitable in the end (or then, the only other alternative being to put your things into a monster database, no traces of them even left within the file system) - but for media management of all sorts, the philosophy behind Adobe LR is arguable, to say the list - and the FPV developer has correctly realized this, since hardlinks would not be that helpful in LR, but with file-system-based pic applications, they can be invaluable IF, as said before, the (main or third-party) applications don't break the hardlinks (or restore them immediately after breaking them).

(FPV unfortunately has the worst - and worst by miles, that is - file browsing functionality I have ever encountered anywhere, both for browsing as for targeting (moving / placing the hardlink) means, so as a picture file manager or even just (general) picture browser (except for immediate access to just-downloaded, new directories), it's almost unusable, and the usefulness of its hardlink-creation functionality should be intensily hampered by the fact that navigation to the respective target locations in which to then create the hardlinks, is an almost incredibly difficult chore. But then, it preloads pictures, and so, while there is no further real speed advantage for browsing .jpg and similar pictures with it, over some other program (FSIV, DO, XY), it's evident that for multi-mb raw files, the speed advantage is upheld, to a degree, depending also of course on the speed of your respective storage medium.)

Be that as it may be, even for just looking at picture, listening to music, etc., hardlinks can be extremely helpful. Just let me say here upfront that I know that most media files come with more or less numerous possibilities of storing metadata, so there is always the alternative to exclusively rely on those instead, and I acknowledge the fact that before deciding to use hardlinks instead, you should compare the different ways of doing things indeed. But let me explain the advantage of hardlinks over traditional .lnk files (symlinks identical to them?), even for "consuming":

If you link to your files within another folder, that's for categorizing the same (media) file under different aspects, of course: the same picture under "landscapes under rain" and under "Camargue trip Summer 1998", etc., etc., the same jazz/soul... tune under the name of the tune ("Summertime"!, "I love you more than you'll ever know"!) and under the name of the artist - I admit that my music examples aren't that totally pertinent since you could use stored searches instead, to some degree, since here the necessary "tags" would even be in the respective file names, most of the time, but even then, there are variants in spelling - let alone the often almost incredible spelling error in YT tunes which clearly show that the uploaders in question don't know a bout about their hot goods.

I also admit that in many cases and/or for many people, tagging would be preferable to multiple directory entries, i.e. entries into multiple directories, but here again, we've got the old problem that either tags aren't "readable" (ie that the files aren't sortable) but within special, dedicated software (except, partly, for tags in files' alternate data strings (ADS), but they present a certain speed problem), or that you'll get quite ugly and especially quite long file names (if you do the tags there), let alone the fact that then you will not have got a tool yet by which to really well manage / administer those file name tags (hundreds of stored searches? that's not that practical, after all); XY for example comes with a some tag-maintenance/retrieval functionality, but that's for its proprietary tags if I'm not totally wrong here, no such thing for tags in file names (except for mass renaming, ubiquitous in good file managers - as I said elsewhere in this forum, the free FC even has got one of the very best (and intuitive) bulk renamers on the market).

But back to hardlinks vs traditional .lnk files (symlinks now?) in traditional browsing situations; the same is true for office documents and similar. So you will have created traditional links/symlinks or aliases/hardlinks to/of the necessary files in your current "target" folder, let's say a project folder for some work, or simply the "person x" folder, your files being originally stored by (chronology and/or geography of) shootings. You want to look at these pictures grouped by folder, you want to use core/reference data to be included in your work.

With .lnk files (and very probably with symlinks if really they are something different now), the "opening" of any such file here in your "target"/project/special folder will immediately trigger the file manager (or FSIV and the like) to display that "origin" folder of the file you just "opened", which means your "browser", for every (?) non-hardlink file entry, switches to the respective "home" folder of that file, so any "opening" of a linked file anywhere "breaks" your browsing. (I don't know how FPV behaves in this situation: since with very few coding "intelligence" it'd be perfectly possible to NOT do that unwanted shift, it's possible that its developer has implemented the functionality to just follow the link but without switching to an unwanted directory here.)

Which clearly shows the superiority of hardlinks for any such (leisure or work) browsing situation: You stay in your "project"/"current" folder, independently of the original storage location of the very first entry/listing of your file, since for hardlinks (only?), any listing is the "original" one; in other words, if you want to implement your third-dimensionality of file access by file-system means, you seem to be in need of hard links.

Since with file-name-tagging (ie tagging within the file names), which is another alternative, there is no practical implementation of a tag tree in any file manager, picture file browsers and the like; at least "frontend" for file-name-tagging exists indeed, but it's external again to your file manager / picture manager / etc. (just as all the more traditional "tag tools" are): it's TagSpaces, 39$, but I consider it illogical to have file-name-based tags which then must be managed/administered, and accessed/benefitted-from outside of your file management and media browsing workflow: You'll browse from within your current folder displayed by your file or graphics-media manager (office: any file manager, pics: DO, XY, FSIV and others), so if you wanted to do it by tagging instead of folder navigation, you'd nee that functionality there... or then, tagging applications would need to implement the necessary media-file display functionality, which is even less realistic, considering the relevant quality/speed requirements in its realization, let traditional alone navigational needs which by this would not vanish but aren't there: any such setup which forces you to use a tag tool (even incl. satisfactory display), and a navigational (file system folder) tool concurrently, cannot be said "ideal", while a traditional browser (DO, XY, FSIV) is obviously much better overall, once the three-dimensional file access problem in its folder structure has been resolved.

This includes the necessity for very fast (!) creation of the necessary "metadata", be it hardlinks, tags in ADS or any other means. As said, those ADS tags come with a slight speed problem even on retrieval (just try in x2 or with a dedicated tool), while both tags-in-file-names (by the same retrieval technology used in "Everything", ie IF that technology is used for retrieval, not so in other cases) as hardlinks (they are just alternative entries in the file table) are instantaneous; but another problem lies in reducing the time which you need to create hardlinks, and here the navigational ease (or lack of it, see FPV above) plays a prominent role: Ideally you will have more than just one tree and/or list pane at your disposal (as in generic file managers where you have at least two of them, the "current" and the "target" one, more so in DO, doubtful in x2 (additional but non-live panes), but it's not that easily possible in XY and others (just "tabs" for alternative folders but which can be used as link targets by some additional scripting, without "opening" them (which for link creation isn't necessary anyway); in fact, if you have some list, that list could have some 50 or more (depending on your screen resolution) entries concurrently visible, any of which could be a possible link target (list of the multiple folders within a parent folder), or even better, a partly-expanded tree structure (so that the folders listed would not be necessarily sibling folders); just a dozen or so tabs instead, and without easily-readable (ie complete) file names, as possible link targets, isn't the same, far from that...

So here again, and considering its picture display capabilities (you would want to do browsing and link creation at the same time of course, since the need for additional entries will arise from browsing), DO seems to be a rather good solution for quick hard link creation for pictures and other media files or even in general, since a number of n "listers" could contain a considerable number of possible link targets (folder lists), n then being the number of groups of folder siblings; the attentive reader will discover the high interest of "hoisting" in some text databases (like UltraRecall and others) since in such a setup, you'll have a multitude of, not flat-lists, but of possibly-expanded, or expandable, sub-trees, which multiplies the number of readily-available link-targets (ie locations for positioning the link or alias), even though over there, you'll have to "open" these additional tabs first, but then, navigation to the desired link target is almost instant.

Of course, there's always the problem of returning to the "active" folder, and there, to the "active" element (picture, office file, whatever); it's desirable that after each link creation, you'll immediately get there again, in order to smoothly continue your browsing / link making. With tabs and multiple visible panes, this condition is regularly met; not so in a single tree like in FSIV where each (necessary) "selection" of some other tree entry does not only invalidate the selection of your "current" folder (which you browse, and from which you create the links or aliases=hardlinks), but worse, when you (manually of hopefully by script) get back to that, you then have to identify your current file anew, which for folders with hundreds of pictures in them is almost impossible, "manually" ie visually, and scripting with regards to FSIV isn't evident at all, elements in FSIV often not being identifiable.

This leaves us with DO as our first choice, and with XY as our second choice: Both are brilliant with picture display (and probably with video clips, too?), and both offer multiple concurrent link targets (mostly sibling folders), DO displaying them directly, XY hiding them behind tabs (if you don't use the tab folder = parent folder as link target, you must open the tab first, then close it again, in order for the target pane becoming available for other targets again).

Full automatization isn't possible since most of the time, you will first "create the link" (ie store the path of the file to be linked), then designate the target pane/tab (ideally by 1-key shortkeys, and which also opens the tab if it's a tab, not a pane), then manually navigate to the target folder in that pane (arrows up/down), then trigger macro 1 (see above) which creates the link/alias within the target folder (without opening it), and which then also closes the tab again, in the tab situation, but it's evident that in such a workflow, creation of each link/alias will take less than 10 seconds IF you limit the possible targets somewhat, ie if you don't try to switch to "exotic", non-easily-reachable targets, too, but if you try to group your link creation in a way, setting up some "combination" of link targets for some work session, then set up another combination of panes/tabs for some other work session; it becomes evident here that the possibility to store these groups (incl. their current open/closed state) becomes very important, in order to get them immediately available for similar work sessions later on, without having to tediously recreate them again and again (I don't know if such "layouts" of panes and/or tabs are available in the programs mentioned here).

Also, it's evident that link targets you need again and again (for example current projects, in office use, or "standard" picture folders, or even "standard inboxes", ie inboxes in higher-up sub-folders from which then only you will do the more precise distribution of the links/aliases further down into the "real" target folders, later on), should be immediately accessible all the time, either by shortcuts or then in some "standards" pane grouping them, and in which those standard target locations then would be targetable by the very first character of their name (so in order to avoid mishaps, automatic alphanumeric sorting in that pane would be welcome), and by this, even a simple 2-key macro could do all the work for these (intermediate or final) standard target locations: some F-key as trigger, then a...z or 1...0 for up to 36 locations not needing any navigation to access them. (It would be ideal to even have these "standards" multiple, at the very least one such set for office use and another one for pictures, another one for classical music, another one for jazz music, and for the pictures, I could easily imagine several standard sub-lists, too; technically, no problem: Just store them grouped in some special folder identified by your tool), and the special command will work on the only one visible one of these special lists, at any given moment.)

From the above, it also becomes evident that the "aliases" is not so much about other file names, but, exclusively in most situations, about other locations: the same file name, but in multiple folders. And from this, you can imagine two quite simple (non-technical but user-interaction) intermediary (ie waiting for better, technical) solutions to the last-file-standing-and-then-falling (sort of friendly-fire) problem described above: The lesser solution would be to systematically add some special character to any link/alias, while not renaming the original file entry in the same way. Then you will never delete any "last" file inadvertently, wrongly assuming it's "just another alias", but whenever you then delete a file entry without that special character, you will want to kill the file itself - admittedly, without succeeding in doing so whenever some other alias of it survives elsewhere in your file structure.

The probably better but radical solution would obviously be, to introduce abstraction between file storage and file naming: have some bulk folders for anything new of some sort, you could for example have folders for some file extension, and then subfolders by date (e.g. one new subfolder a year, or per month if necessary by the sheer amount of it (professional photographers and the like)) - and then have aliases, exclusively, i.e. put any file in some (pre-sort) target folder, with its original file name, and then into other such target folders, but the very first (and probably, for most files, unique) target folder will already be a "second" location for the unique file; in this set-up, every file entry in an accessible folder will be an alias, a virtual entry, compared with the original file entries, in the (here: deliberately) non-accessible (non-accessed) storage-only folders. Such a system will be as neat as it get, and should be the future of NTFS (or its successor): Then, Windows can automatically assign cryptic original file names to any file, just like it does it currently do to "special" files, and everything (except the system files that is) will be accessible from everywhere, under any file name you will want to assign to it there, the very first such entry, as said, being the first non-cryptic, "human-readable" file name of your choice.

It's evident that such a system would greatly simplify any further file operation, the "really original", the cryptic, file entries just being there for Windows(') administration purposes, and they would very rarely, if ever, be changed. (Technically, it's easy to do this even now: just run a script moving all your files into such "technical folders" (as you know, their respective physical positions on the harddisk will not change with that "move"; as said, by extension, and with a little sub (whenever more than 1,000 files of that extension, create a new folder for the rest), replacing all original file entries by (hard) links at the same time; ditto for any new files then: any new file creation would trigger a little routine creating it at the "centralized" storage, while creating a link where you create it, and you'll have created and will be maintaining your own, 3-dimensional NTFS system.

Ok, that's not considering the "some applications kill hard links" problem described above, neither do I presume anybody will follow my advice here, and I didn't treat the impending recursion problem with folders, apart from and to begin with the fact that I haven't mentioned the problem that you cannot apply hard links to folders anyway but must do it differently for them, but it's evident that the current - virtual-only anyway, as explained above - relationship "1-file-to-1-filename" paradigm in Windows (and prevailing in NTFS) is junk (as is the - equally virual-only - separation into "files" and "folders", too, but explaining this would double the text amount here); and that MS should finally do their so long-overdue homework.

It's all about providing three-dimensionality to a file system, while at the same time avoiding to set up any database - outdoing what the "Everything" indexing does - as the file system's sidecar, this will for avoidance being there for obvious speed and reliability reasons and being perfectly understandable... but perfect three-dimensionality (and including folders, not treated here) is already possible with NTFS if organized differently, and the one-file-referenced-trans-drive problem should be overcome with a "database" (ie with a simili-database, just like the MFT is just such a simili-database, and the necessary change indexing and -processing) indeed.

In a word, what we need is a spiced-up NTFS with handshaking for synching, but in the meantime, our homemade possibilities are far from inexistent; especially media home storage should probably be made with DO and then some little macros for speeding up the link creation process, as described above (and attending what they will implement internally instead), since browsing / viewing AND creating of multiplicity is seemless then - for bigger storage needs, the question would be if a big NAS or similar can be set up or not, with one shared, big MFT, so that for Windows' / hard links' means it's all one drive. And remember, hard links are the file system itself, while metadata / ADS are not stored there but will have to be retrieved separately, hence their incompatibility with large datasets when not indexed - but indexing them for immediate retrieval should always be possible, so there are alternatives.

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,612
    • View Profile
    • Donate to Member
Re: Three-dimensional file access in Windows, especially by Hard Links
« Reply #2 on: August 01, 2017, 08:06 AM »
I'm surprised there has been no reaction to this post yet ...
TL;DR;
If you keep writing blog-size entries like this in the forum, then don't be surprised if there will be less and less response. :-\

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 115
    • View Profile
    • Donate to Member
Re: Three-dimensional file access in Windows, especially by Hard Links
« Reply #3 on: August 01, 2017, 09:57 AM »
What about your programmatically avoiding my threads and posts anyway?

And to depart that interrelational meta level: Some topics are dense, so if you try to treat them in completeness, you have to write quite a bit. There are posts of mine where your remark would have been quite pertinent, but here you chose a very bad example for your point: Any redaction here could have shortened the post only so imperceptibly.

And yes, I know that it gathers quite a bulk of elements, and that reading it cannot be but hard work, but then, my - as said above, partial - development is meant to exist for the few people really interested in the subject of (general, not-media-files-only) 3dimensional file access; for them, there's lots of very lightweight material on the web to be found, and some very technical, very partial info; I've been striving here to give some sort of overview which treats the multiple problems and aspects of the subject, be them obvious or more conceiled. Most of the time, I strive to give new and complete* information, especially whenever my own web searches have been totally unfruitful in this respect.

Thus, I didn't want to bother about 150 people regularly reading/writing here with the intricacies of my findings and musings, but I had hoped and I'm still hoping that

- people researching the subject will be redirected here by google (and then there is some draft awaiting them which doesn't really leave out much anymore, so they will be well served here I dare hope), and that

- some members of the aforementioned group will share their individual means to attack the problem of their file system being flat (information exchange even without reading me for half an hour or longer); my (mild) "surprise" covered that "easy" aspect of the subject then.

*: Often I refer to external info, ie I don't attempt to "explain the obvious", the "readily-available-elsewhere", I just mention its existence in case; as said, the subject here is special in this respect: info putting into perspective, into relation, in a word, synthesis, was from rare to absent, depending on your point of view or your standards, so I've tried to give that missing overview I had missed for reading myself, and that's necessarily as complete and thorough as possible by my means.

But thank you for prompting my clarification, which partly at least applies to other posts of mine.

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 11,186
    • View Profile
    • Donate to Member
Re: Three-dimensional file access in Windows, especially by Hard Links
« Reply #4 on: August 01, 2017, 02:00 PM »
And to depart that interrelational meta level: Some topics are dense, so if you try to treat them in completeness, you have to write quite a bit. There are posts of mine where your remark would have been quite pertinent, but here you chose a very bad example for your point: Any redaction here could have shortened the post only so imperceptibly.

And yes, I know that it gathers quite a bulk of elements, and that reading it cannot be but hard work, but then, my - as said above, partial - development is meant to exist for the few people really interested in the subject of (general, not-media-files-only) 3dimensional file access; for them, there's lots of very lightweight material on the web to be found, and some very technical, very partial info; I've been striving here to give some sort of overview which treats the multiple problems and aspects of the subject, be them obvious or more conceiled. Most of the time, I strive to give new and complete* information, especially whenever my own web searches have been totally unfruitful in this respect.

I think to Ath's point, there is something to be said about topics for mediums, and topics for audiences.  Forums aren't really made for long-form dissertations, but rather short-form discussions.  For long form, there are other mediums for them, i.e. blogs.  I'm not sure if dc still has blogs for members, but if so, then it might work to post your blogs on one, and direct from here to that resource.  It is in the end up to you and the management as to what can be posted, so I'm not even speaking to that.  But rather adding some objective clarification to what Ath may have meant.  Others do the same thing, and get, in general, similar results, so you're of course welcome to continue down that path.  But these seem like blog posts rather than discussion fodder.

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,612
    • View Profile
    • Donate to Member
Re: Three-dimensional file access in Windows, especially by Hard Links
« Reply #5 on: August 01, 2017, 02:52 PM »
... seem like blog posts rather than discussion fodder.
+1 (and then some) :Thmbsup: