For Serious Research: Cadillac of "ClipBoard Managers" vs. "Info/Data Manager"

[nkormanik]

Thanks Dr_Andus.  Great explanation.

TaoPhoenix likes MyInfo ($49, pro-$99).  What other programs have you, and others, found worthy of holding your vast and growing supply of information?

Is this sort of program beyond what we might hope for on donationcoder.com?

Dr_Andus says "if you want to work ... on an on-going basis dynamically (e.g. by constantly analysing, re-organising and synthesising)...."  I'm very interested in this process.  Can it be further explained some?

[dr_andus]

Tomos is probably right in that it also depends on the nature of the data being organised. The tree-like hierarchy clearly works for TaoPhoenix and many other people, otherwise hierarchical software like UltraRecall, RightNote, MyInfo etc. wouldn't be around.

The scenarios I'm talking about have to do with organising a large number of small chunks of text (with occasional images or links to other files and websites), such as a database of quotes and reading notes that one collects over a lifetime. In that situation it is not all that useful to organise these items into trees at the time of collecting or creating them because they are being gathered for later consumption, where they may be re-used for different purposes. It would be more important to label/categorise them, so that they would turn up in relevant searches.

The method I'm using is called the Zettelkasten (slip box) method, developed initially by a sociologist called Niklas Luhmann. If you do a search for it, you can find sites that explain it.

One category of software to implement this system are personal/desktop wikis. I use ConnectedText, but as I said, it's got a steep learning curve, it took me several tries over several years to finally start using it. Zim is another one.

There are some cross-over software that try to combine the tree with a wiki. E.g. Wikidpad, Whizfolders, or Outwiker.

There is also Luedecke's Zettelkasten software.

Piggydb is another interesting approach.

Is this sort of program beyond what we might hope for on donationcoder.com?

Not necessarily. There are some very simple implementations of the Zettelkasten out there, such as using nvALT on Mac or ResophNotes on Windows. Apparently it's even possible to create a desktop wiki using AutoHotkey.

[TaoPhoenix]

TaoPhoenix likes MyInfo ($49, pro-$99).  What other programs have you, and others, found worthy of holding your vast and growing supply of information?

Is this sort of program beyond what we might hope for on donationcoder.com?

-nkormanik (February 15, 2014, 05:39 PM)

Hi Nick.

On this "other topic" a while back I investigated the idea of "Coding Snacks" and "Coding Lunches" and even "Coding Dinners". DC's special niche is that for other people you can usually get a "snack", which is a cute little low level program to do one super-useful thing that you cannot program yourself. One of my favorite examples is the TranDesk desktop Splitter that allows you to force WinXP / other into multiple desktops. Or MilesAhead's BBSS that lets you save the URLs of a bunch of Firefox tabs into a text file. (Somewhere else I swear I saw the other half that lets you load them all back in again.)

The problem is there is a gap in the middle I call "Coding Lunch". Most of the quality well done "NANY" apps are people's labor of love, so maybe you can sneak a small feature in, but otherwise they either work for you or not. It's a bit tricky to get people to make you a medium app here. I've tried.

I do have to say that even leaving off the fancy stuff, I thoroughly respect the backend problems of an info manager program. MyInfo was one of actually TWO programs with the killer feature I wanted - a beautiful Web export. Last year my recommendation would have been the other one. But that one somehow decayed with a deadly core level structural integrity data destroying problem, and I had to let it go.

I totally get how adding features is Additive and support is Cube-Exponential.

Does that help?

[Attronarch]

The method I'm using is called the Zettelkasten (slip box) method, developed initially by a sociologist called Niklas Luhmann. If you do a search for it, you can find sites that explain it.

-dr_andus (February 16, 2014, 04:13 AM)

Looks like an interesting system, especially for complex issues, scientific books, and journal articles. Could you recommend some readings on the topic?

[dr_andus]

Looks like an interesting system, especially for complex issues, scientific books, and journal articles. Could you recommend some readings on the topic?

-Attronarch (February 16, 2014, 03:59 PM)

It was this article by Manfred Kuehn that got me started: Some Idiosyncratic Reflections on Note-Taking in General
and ConnectedText in Particular

He blogs frequently about Zettelkasten.

Christian Tietze's blog posts on the matter are also helpful.

And here is the original Luhmann article: Communicating with Slip Boxes: An Empirical Account

[40hz]

Edit: removed. Link I posted here was mentioned earlier in thread.  

[tomos]

It was this article by Manfred Kuehn that got me started: Some Idiosyncratic Reflections on Note-Taking in General
and ConnectedText in Particular

-dr_andus (February 16, 2014, 05:11 PM)

that was impressive. Will have to read some more about the actual Zettelkasten system, as he doesnt really explain it there in much depth.

While reading, I was wondering if CT+Zettelkasten was useful for writing too - interesting to note he says this article was the first he wrote in CT. Occured to me it might be helpful to have notes in one app and write in another. What's your approach there?

[TaoPhoenix]

Looks like an interesting system, especially for complex issues, scientific books, and journal articles. Could you recommend some readings on the topic?

-Attronarch (February 16, 2014, 03:59 PM)

It was this article by Manfred Kuehn that got me started: Some Idiosyncratic Reflections on Note-Taking in General
and ConnectedText in Particular

He blogs frequently about Zettelkasten.

Christian Tietze's blog posts on the matter are also helpful.

And here is the original Luhmann article: Communicating with Slip Boxes: An Empirical Account

-dr_andus (February 16, 2014, 05:11 PM)

These are some power articles.

But maybe there's "theory" that goes down-level as well as up-level.

I like MyInfo as one of Four Two One winner app that just lets me smash out info. I'm "just a customer" aka no hidden agendas, but as a User I spent over a year and over twenty programs came up short.

Per these articles, years ago I got fascinated by that kind of high level theory. But if the "Implementation" has a fatal flaw, in real life you lose months of your productive life. (Speaking from experience!)

(Your choice of who) have taken smaller goals and implemented them well. In Data Mgt to me that's huge - do a smaller subset tightly and maybe I can work around my edge case, than get too big and have the whole thing implode.

[dr_andus]

While reading, I was wondering if CT+Zettelkasten was useful for writing too - interesting to note he says this article was the first he wrote in CT.

-tomos (February 18, 2014, 07:04 PM)

It's certainly possible, especially since CT v. 6, as you can have unlimited floating windows of notes open, while doing the writing and editing in the main window. CT also has its own dedicated single-pane outliner, which can be combined with the notes to be used as a dual-pane outliner/viewer of notes.

Occured to me it might be helpful to have notes in one app and write in another. What's your approach there?

-tomos (February 18, 2014, 07:04 PM)

This is probably down to individual taste (and budget). I prefer to use CT as my database of notes (Zettelkasten), displayed in my third monitor on my right, while I do my writing in WriteMonkey in the central monitor. CT does have a full-screen writing mode, but it's not as good as WM's or Scrivener's.

I use CT's outliner mainly as a to-do list to track the writing tasks, but I use a Freeplane mind map for the overall outline of a writing project. The nice thing is that it is easy to link specific CT notes to specific Freeplane mind map/outline items, so clicking on a Freeplane node can bring up a corresponding note with a quote or comment in CT.

[dr_andus]

Per these articles, years ago I got fascinated by that kind of high level theory. But if the "Implementation" has a fatal flaw, in real life you lose months of your productive life. (Speaking from experience!)

(Your choice of who) have taken smaller goals and implemented them well. In Data Mgt to me that's huge - do a smaller subset tightly and maybe I can work around my edge case, than get too big and have the whole thing implode.

-TaoPhoenix (February 18, 2014, 08:16 PM)

I really wouldn't want to suggest that the Zettelkasten approach (or desktop wikis) is a magic bullet solution for all kinds of data management problems. If a hierarchical organisation approach works for you, there is no need to abandon that.

My own problem had to do with the organisation, analysis and synthesis of qualitative data for academic research, and the wiki/Zettelkasten solution emerged as a solution to the limitations of traditional academic qualitative data analysis software (so-called QDA or CAQDAS software, such as NVivo, Atlas.ti, QDA Miner etc.).

It took me repeated trials of CT over 2-3 years to figure out how I could use it, and even after that it took another year of resisting the Zettelkasten method until I realised its main benefits. Now I'm kicking myself for not adopting some kind of a similar index-card system at the start of my undergraduate studies. Obviously taking and organising notes is a core academic skill and it probably makes or breaks an academic career especially in the humanities and social sciences.

[dr_andus]

These are some power articles.

Per these articles, years ago I got fascinated by that kind of high level theory. But if the "Implementation" has a fatal flaw, in real life you lose months of your productive life. (Speaking from experience!)

-TaoPhoenix (February 18, 2014, 08:16 PM)

BTW, it's not necessary to adopt the Luhmann approach in its entirety (I'm not even sure I completely understand it). Instead, I've just adopted some of the basic principles, such as:

- bite-size notes (c. 200 words on ave., 500 words max. per note). One major idea per note.
- no hierarchical organisation (flat alphabetical or chronological list). I keep them in a chronological order.
- make it easy to find the notes by:
-- having descriptive (long) titles, inc. date and time note captured, author's name, year of publication, main topics
-- having categories, labels.

- link related notes together by:
-- using categories and attributes in CT (which create meta-pages, if you click on any of the marked-up terms)
-- using direct (wiki) linking.

The advantages of using CT for this is that there are all kinds of other sophisticated annotation and search tools, plus the software is highly modular, so you can arrange it to suit your idiosyncratic needs. But the above principles can be also adopted for use with a variety of other Zettelkasten or database software that don't require you to define a hierarchical position for your notes upfront.

[TaoPhoenix]

...
Instead, I've just adopted some of the basic principles, such as:

- bite-size notes (c. 200 words on ave., 500 words max. per note). One major idea per note.
- no hierarchical organisation (flat alphabetical or chronological list). I keep them in a chronological order.
- make it easy to find the notes by:
-- having descriptive (long) titles, inc. date and time note captured, author's name, year of publication, main topics
-- having categories, labels.
...

-dr_andus (February 19, 2014, 09:10 AM)

We're def. getting into personal prefs.

My answer to tags would be a bit like an Excel/clone page with extra "sort columns". So then if "shuffling data" is your thing, with a medium amount of work up front, your data can be presented hierarchically in X ways.

I'm pretty big on at least fluid hierarchy because I believe almost all data exists in *some* hierarchy. Even if you're not sure if you want to deal with it as Compounds/Molecules/Atoms or Symptom/Causes/Suspected, purely flat data just feels wrong. I think I'm hearing there's need for flexibility in organization, but not all notes are created equal! So even if you "tagged" three different sort methods, it's like in Excel "sort by Column G, or F, or B".

And for Note size, I haven't done word counts (though I'm almost the type to do that!!), but I'm a "Completeness Maven". Short notes bother me because they tend to feel they are not complete. The only unclear part is whether it should all go into one big note (length) or subnotes (structure). So I def. see some style prefs going on!

As a friendly comment, I'm barely able to even imagine how you deal with stuff without either longer notes or structuring! Someone please give me a PM example of a minor-mini data-set for chatting! I basically can't even handle 500 words before it becomes a tree! But ya know, this might be because I merge "Getting Things Done" (GTD) themes into my notes.

So holler at me!

Cheers,

--Tao

[TaoPhoenix]

Heh that means you, Mr. Kormanik, you of the "giant unsorted data chunks". You're the thread opener! Let's play!

[nkormanik]

I bookmarked the various leads provided by the esteemed panel here.  Thank you!  But I've yet to read 'em.  So am still naive.

Maybe the sites mentioned will answer a new question I've come up, but I'll ask it anyway:

Dr_andus earlier mentioned a "lifetime" of notes possible, that it would be nice to have saved all, somewhere.  Well, I totally agree.  An entire life's worth of notes and 'snippets' would likely fit on one of our 'thumb drives', together with lots of programs for slicing and dicing, mixing and matching, analyzing, synthesizing, creating all sorts of output.

Then, supplemented by the Internet, multitudes more possibilities spring forth for our expression, manifestation, whatever we want to do with the collection.

I'm setting up the question....

In my humble opinion, every paper -- scientific, academic, etc. -- and book I download and add to my Library really deserves to be in my grand personal database as well.  Yeah, I didn't type it in, or even copy a 'snippet', but still.  It all deserves to be included.

Just as with my 'notes' being in a single shoebox (=folder), thousands of pdfs, epubs, etc., are in a single shoebox as well.  A huge mass of carefully selected (cough) pdf files, all jammed into my Library folder.  Each item chock-full of valuable insights, findings, information.

At some point soon the technology of personal indexing and instant-searching capabilities will allow us ready access to all that information we've accumulated.  We're not quite there yet.  But soon.  Suppose: a dedicated hyper-fast solid state drive for Library and huge index -- probably at least a terabyte in size..., twelve gigabytes of memory, a screaming multi-core CPU, 64-bit operating system.  And incredible indexing/retrieval software.

The question is: (let's just say)  If we include all of our lifetime of notes, plus all our carefully selected literature, and have truly instantaneous access to all of it (by way of super-indexing/retrieval), how will that affect our creation of output?

Getting back to my original question at the outset.  With the above being said, no need for Cadillac of Clipboard Managers.  Skwire's Clipboard Trap is fine -- one long text file of heterogenous snippets is not a problem.  Don't bother parsing out into various categories and topics.  Keep all notes in one single shoebox (=folder).  The Grand Indexer, regularly run, and the Info/Data Manager would be the way to go.

Right?

[tomos]

^Yeah, I think the thread moved away from the clipboard fairly quickly

Read the Manfred Kuehn link from dr_andus - that was very informative I found - and helpful in it's suggestions as to *how* to take notes. He's not so into the idea of simply copying quotes - more into your opinion of, and comments on, the quotes.

The basic strength of any IM is that each note is separate and can be moved around/"cloned"/copied/connected/tagged/filtered.
So when you say:

one long text file of heterogenous snippets is not a problem.  Don't bother parsing out into various categories and topics.  Keep all notes in one single shoebox (=folder).  The Grand Indexer, regularly run, and the Info/Data Manager would be the way to go.

-nkormanik (February 21, 2014, 03:08 AM)

makes me think: would it not be better to have each "snippet" as a separate note? Mouser's CHS uses a database for clips. I've never really taken advantage of that in any menaingful way, but it might be worth looking at it's capabilities and export options.
OTOH, InfoQube -- and probably many other IM's -- allow the option to separate a pasted text into individual 'items' (with a new item after each carriage-return). Also importing a text file in a similar manner is oten a possibility.

Re the indexing of your PDF's -- I think there are IM's that will allow search of linked files (maybe they're indexed?), or even store a copy within it's own db (that is gonna be dangerously big though imo). But not an option I need/use so I'm not well informed there.

But just index/search is not enough, hence the Info Manager. And I think that's where/what you've got to follow up on.

[dr_andus]

In my humble opinion, every paper -- scientific, academic, etc. -- and book I download and add to my Library really deserves to be in my grand personal database as well.

-nkormanik (February 21, 2014, 03:08 AM)

As Tomos had suggested above (after Manfred Kuehn,) there are some benefits to distinguishing between "collected data" and one's own notes. The former are analogous to books that have been purchased and put on the shelf without having been read, and there is also a chance that they may never get read. They are more like bookmarks than notes. The chances are that these unprocessed bookmarks would overwhelm your own notes in terms of sheer volume, making it difficult to make use of your own notes.

I'd recommend a two-tier system. In tier one (a big box), include all the collected data that may become interesting in the future (e.g. PDFs of articles, collected web pages etc.). In tier two (a much smaller box), include only your own notes (such as selected quotes, comments, your own ideas and opinions). The first tier can be any kind of software that can hold that sort of data together (such as UltraRecall, Surfulater, RightNote, MyInfo etc.), and the second tier some kind of a dedicated database that can deploy the Zettelkasten method (index cards in a slip box with one note/quote per index card) and offer sophisticated tools for analysis and synthesis (writing up), such as ConnectedText, Luedecke's Zettelkasten, Piggydb or other desktop wikis and equivalents.

[IainB]

...and THEN pasted into MyInfo! But if you MUST have a whole webpage, and can live with a little format-squabbling between javascripts, I get this below by going all
Control-A Select All - Control C Copy - Click over to Node - Control V Paste.

-TaoPhoenix (February 10, 2014, 07:51 PM)

That seems to work as you described. I took a copy of the MyInfo download page (http://www.milenix.c...wnloaded?edition=pro) into MyInfo, and it saved it intact (as far as I could see).
Curiously however, having saved that page into MyInfo, when I searched (this is using MyInfo PRO) across all topics for "download" or "downloading", it apparently cannot find either string in that copied web page, though it does find the strings where they exist in other documents.
Similarly, directly searching for those strings in the copied web page draws a nil response.
The strings are definitely there, so it seems that the search function is "blind" to web pages.
Strange, not to be able to search for strings in the saved web pages. There don't seem to be any settings that can affect this search behaviour either.