Author Topic: I'm thinking of going primitive, with discursion into zettelkasten (Read 1,750,448 times)

Dormouse · « **Reply #825 on:** October 23, 2020, 12:23 AM »

Quote from: Nod5 on October 22, 2020, 09:17 PM

a kanban app that reads todo lines from plaintext files.

There was this post too: https://forum.obsidian.md/t/kanban-board-rendered-from-markdown/5184
pointing to a vscode extension TODO.md Kanban Board

wraith808 · « **Reply #826 on:** October 23, 2020, 01:12 AM »

With []() you know exactly how the filename is formatted though.

wraith808 · « **Reply #827 on:** October 24, 2020, 06:49 PM »

This is an example of why I hate using hosted applications that don't interact with files that I control - https://www.yarps.net/yarps-is-in-trouble

panzer · « **Reply #828 on:** October 28, 2020, 10:26 PM »

Anytype – an offline-first private alternative to Notion:
https://anytype.io/

JavaJones · « **Reply #829 on:** October 28, 2020, 11:03 PM »

Quote from: panzer on October 28, 2020, 10:26 PM

Anytype – an offline-first private alternative to Notion:
https://anytype.io/

Yep, I'm in the beta. Mentioned it back on Page 28

https://www.donationcoder.com/forum/index.php?topic=48938.msg440630;topicseen#msg440630

- Oshyan

panzer · « **Reply #830 on:** October 28, 2020, 11:47 PM »

Sorry. My bad.

I'll go whip myself.

JavaJones · « **Reply #831 on:** October 29, 2020, 11:15 PM »

Quote from: panzer on October 28, 2020, 11:47 PM

Sorry. My bad.

I'll go whip myself.

Not at all. In fact I wish people asked more about it, nobody seemed to care at the time (or now, I guess). Not Markdown-ey enough perhaps.

- Oshyan

Dormouse · « **Reply #832 on:** November 13, 2020, 08:56 PM »

My original Plan B when I started mostly using Obsidian was to switch to another program (possibly Foam, even Zettlr or even newer ones) and change all the syntax in the notes as required.

I've been thinking about it again recently. I'm still comfortable with Obsidian. But I look at all the feature requests and what they all have in common is increased complexity in exchange for adding features that would help a small minority of current users and a tiny proportion of users should Obsidian go mainstream. That may not be a problem if they are all added as optional plugins, but I've been here many times before. Reacting to user requests is a good thing, but it has its dangers. The vast majority of current users (at least those who post) are techie (even the ones who seem incapable of understanding what they're doing). There's any number of conversations about Mermaid. No appreciation of KISS or UI at all. There's been one example already - block refs - many wanted them, but the implementation is necessarily a hacky workaround which impacts the underlying document (only alternative would have been a database).

So I reviewed my own usage. There are major features (eg Graph) that I barely use at all. Many others where it seems currently weak, but might improve in the future (though I'm not confident, since the user base is the main potential source of plugins); I deal with that by using other programs. What I've learned by using it is the value of nested vaults and the productivity of wikilinks; and that using many programs on the same file is pretty seamless.

And the major source of friction is markdown. Plaintext is great, markdown grates all the time. Some simple things are just unnecessarily convoluted (whoever thought that counting #s was the bees knees?0 and some simple things it can't do at all (for me multi-coloured highlighting comes high on the list - atm forces me to switch to RTF or other WP format when I need it). Asciidoc looks better, and org-mode much better again (at least if it weren't for the compulsory complexity). But markdown doesn't seem likely to be avoidable in any future Obsidian alternatives.

So then it came to me. Design my own system. I could probably even do much of it with current plaintext/markdown editors and text expanders for conversion (which is what I already do to some extent). But I wouldn't need compatibility since it would only be for me. I could have better interfaces with other file types (I assume Obsidian will eventually, but I've no idea what they might look like). So there we are. A new Plan B. Not Plan A, and it might easily be superseded by something else. But will sit in my mind as something quite feasible and maybe even desirable.

PS
I note that logseq has gone open source.

JavaJones · « **Reply #833 on:** November 14, 2020, 12:09 AM »

If you're willing to have your links, formatting, and other markup be separate from your text, then a stand-off approach like Codex uses may be the way to go. The Codex dev has claimed that he intends to make the stand-off markup exportable, though whether any other app will ever support it fully is questionable. But at least your base text files are untouched. That also means no links in the text files themselves though (e.g. [[link]] ). All that said, Codex is also not locally-based. But parts of it will be open source. So it might be something to keep an eye on...

- Oshyan

Dormouse · « **Reply #834 on:** November 14, 2020, 10:14 AM »

I want to stay local, I like wikilinks in the text, plus the bits of markup and formatting that I use while writing and editing.

I'm happy for formatting for publication to be separate, but markdown doesn't do that anyway.

40hz · « **Reply #835 on:** November 14, 2020, 04:58 PM »

Just out of curiosity …

What are people doing with all this information they’re curating and cataloging with these various pieces of software? To what purpose? Or maybe even: to what avail?

I’m more curious about the individual “business” use cases rather than the supporting technology. Technology and solutions that offer varying degrees of utility aren’t that difficult to run down. God knows there’s tons of software out there. But the reasons to employ said technology can sometimes be less obvious. At least to me.

So help me out. What is/are your goal(s). What’s it all for? What are you guys doing with all this information you’re gathering?

Dormouse · « **Reply #836 on:** November 14, 2020, 10:32 PM »

Quote from: 40hz on November 14, 2020, 04:58 PM

What are people doing with all this information they’re curating and cataloging with these various pieces of software? To what purpose? Or maybe even: to what avail?

Most of the time that isn't what I'm doing. Most of what I have is what I write. Stuff I don't write myself is just linked.

Mainstream uses
Writing and research.
It's a large part of what I do, and always has been.
It's pretty much what it's designed for, though the userbase in Obsidian seems to have a huge number of students.

Related uses
Work and professional stuff. Again really, it's largely writing and research. Just differently dressed.

Anything money related
Research on things to be bought, or suppliers, what's paid.
Ditto for pensions, investments etc. I'll probably record my tax stuff in it this year (that bit being purely local and secure).

Everything else
I'm actually using it for nearly everything.
Eg in the garden, what was planted where and how it did.
Anything at all.

My basic rule is that if something requires collection of information, weighing it up and making a decision - especially if it's something I might have to do again in future - then it goes in unless I can work my way through just by remembering (I'm not trying to give myself unnecessary work); or unless I don't get round to it.
But I'm still working my way around the system, so changes are still likely.

Of course, I'm only using it for everything because, I'm already using it and it works and it's easier to use one process for everything.
And having freedom to do that across the board makes it easier to innovate and adjust.

And even more of course, I couldn't write the above in markdown without adding HTML. And that was deprecated in HTML 5 for a while.

wraith808 · « **Reply #837 on:** November 14, 2020, 11:27 PM »

Quote from: 40hz on November 14, 2020, 04:58 PM

Just out of curiosity …

What are people doing with all this information they’re curating and cataloging with these various pieces of software? To what purpose? Or maybe even: to what avail?

I’m more curious about the individual “business” use cases rather than the supporting technology. Technology and solutions that offer varying degrees of utility aren’t that difficult to run down. God knows there’s tons of software out there. But the reasons to employ said technology can sometimes be less obvious. At least to me.

So help me out. What is/are your goal(s). What’s it all for? What are you guys doing with all this information you’re gathering?

Writing, Research, and Project Management for Technical projects for Home and Work
Writing, Research, and Project Management for Non-Technical projects for Home and Work
To-Dos and Notes from my general days
Pretty much everything at this point. As Dormouse says above, I'm already in it for my projects, so it's just second nature at this point to use it for anything that comes up. Doctor Appointments, Maintenance Directory and Logs, etc.

And my notes are pure MD. I have other documents linked in the repository, but everything as far as the notes is just MD.

JavaJones · « **Reply #838 on:** November 15, 2020, 12:34 AM »

I have a multiplicity of needs and tools. I'd rather have fewer tools, but nothing does everything I want yet.

I have a daily "life logging" practice that is a combination of work and personal items, including some "quantified self" stuff like when I woke up, how much I weighed, what I ate, etc., as well as work periods and tasks done or things accomplished, documentation of thoughts, feelings, ideas, etc. One of the reasons I got interested in Roam-like linking approaches is because it's a lot less work to connect my daily log to individual pages where I can analyze, etc. So now I can link to e.g. [[Weight]] and I have all my weight entries drawn straight from my journal, I don't have to actually open the page to add that day's weight, the "backlinks" feature shows them all for me just by linking. And that way I can read them in my more narrative journal too, without having to open a bunch of related pages to get the full picture of a day. Best of both worlds. That's just one example. Right now I do all of this in Roam, but plan to move either to Logseq or Obsidian, and maybe Codex long-term.

I also keep track of a lot of life, project, and activity stuff like Dormouse and Wraith. Pages for projects, entrepreneurial ideas, etc. I find the outline structure too rigid for this stuff, so I used to use Quip (and still have a lot of data in it), and have been slowly migrating this stuff to Notion. But not sure I like Notion enough to keep it all there... Depending on how well Anytype shapes up, I might be able to move the database stuff there.

Then there is regular data sets with some reasonable amount of commonalities, i.e. properties, that I want to store, sort, etc. Things like Recipes, Articles or Books to read, etc. These all go in Notion databases at present, they used to be simple lists or header-divided pages in Quip. The databases are better, but still not perfect.

Then there is the actual writing I do, mostly for planned blog posts (I say "planned" because I post them very seldom). This I do in Notion right now, it's the closest analog to a "web presentation" I can write in, and it's faster and better than working in Wordpress directly for me.

Ultimately I'd like as much of this stuff as possible to all be in a single system because interrelation and data re-use is incredibly powerful, time-saving, and useful. As a simple example, being able to reference the data in a property/field of a database from regular text, or link to a database entry in entirety from some other page (Notion can do the latter but not the former), etc. Or even embed a database record, or an entire table (Notion can do the latter, not the former).

That covers most of the non-work stuff. For work I chose and implemented Fibery, but that's getting pretty tangential to the topic of this thread.

Shades · « **Reply #839 on:** November 15, 2020, 03:34 AM »

Quote from: Dormouse on November 14, 2020, 10:14 AM

I want to stay local, I like wikilinks in the text, plus the bits of markup and formatting that I use while writing and editing.

I'm happy for formatting for publication to be separate, but markdown doesn't do that anyway.

Sorry to remain harping on about AsciiDoc. But I know that for AsciiDoc there are applications that you can host locally. Whatever documentation you create, it will be automatically translated to (static) webpages and served locally or on a hosted solution. In case that is your thing. The tool I talk about is called: Hugo I see that it also supports MarkDown nowadays.

For those that really wish to introduce versioning into their documentation (technical documentation writers come to mind), for those that have collected a lot of documentation all over the place, this software is capable of searching local or hosted projects for any document and create a complete manual from it, all automatically. I have seen examples of this system working with a hosted GitLab instance with many different repositories. This software is called: Antora (more descriptions here).

Looks to be an AsciiDoc only solution. But I'm sure there is similar software for MarkDown.

panzer · « **Reply #840 on:** November 15, 2020, 06:36 AM »

Quote from: Shades on November 15, 2020, 03:34 AM

But I'm sure there is similar software for MarkDown.

https://www.gitbook.com/
http://daux.io/
https://www.mkdocs.org/
https://docsify.js.org/
https://docpress.github.io/
https://dotnet.github.io/docfx/

Dormouse · « **Reply #841 on:** November 15, 2020, 10:22 AM »

Quote from: Shades on November 15, 2020, 03:34 AM

Sorry to remain harping on about AsciiDoc

I'm happy for you to keep on with it.
For me asciidoc is better than markdown but worse than org-mode .
And none have some features that would be essential for full utility (for me). So, theoretically, I don't mind moving around. But so far I don't see another program that looks as if it will be as good as Obsidian will be.

I do consider Obsidian's block reference method a misstep, but it won't affect me if I don't use it. Apart from that, I think they're doing well.
(I think a better approach would have involved a better file explorer, options for automatically creating new notes from a next command with a folgezettel name that's hidden in editor, and using transclusions for display etc. That would have been elegant and stuck to their first principles, but couldn't have been done with a quick hack.)

Dormouse · « **Reply #842 on:** November 18, 2020, 01:05 AM »

OK. Things have moved on swiftly since my last post and my Plan B has now become Plan A.

I said that I considered Obsidian's block reference method a hack. Not using it meant that it didn't affect me, but it was still something I noted.

I was prompted a few days ago to take a look at exactly what data Obsidian was storing and where it was stored. In the Discord it was stated that data was stored in the vault, but actually most is stored in systems folders predominantly, on Windows, in Appdata\Roaming\ObsidianCache which contains .json files for every vault that has been opened. The .json contains the names of all files in the vault, plus headings, and links - in plain sight. A user might encrypt the vaults themselves but this information would remain easily read.

(This can be dealt with by simply deleting the obsidian file in Roaming every time the program is closed - it will just rebuild it when it is opened again. Presumably there will be a speed penalty which will become noticeable with very large vaults.)

I assume that starting Obsidian calls or creates these files, which are used to provide fast access to items that can be linked. This allows fast responses without having to load the full content of files into memory. JIT method.

When I first used Obsidian all information was kept in the vault. The switch to the system folders came in 0.8.7 and was apparently to address problems with sync programs.

I also noted that one of the recent plugins deleted user data before it was updated to address the issue. Plugins are optional but regarded as a central feature of Obsidian's design.

I drew a number of conclusions.

Until this is fixed, which I assume it will be, there is a potential security issue for any user who puts private information in file names or headings, unless they are confident of the security of their computer.
The method was chosen without any thought to possible security implications, despite a thread on the forum where a number of users were finding ways to circumvent employers restrictions on installing software so that they could install and use Obsidian on work computers (this would increase their risk if their computer were audited).
Together with the block reference methodology, my confidence in the expertise of the developer has reduced.
The developer seems to be responding to pressure to increase the number of features which would only work well in a database. Given that a large proportion of the posting community seems to be made of students, many of whom were either previous users of Roam or aware of its features but not keen to pay Roam's price, I would conclude that this is the group driving the direction of travel.
I see Obsidian currently as being a partial database with linked files. afaics it will end up as a full database program with associated files. They are discussing ways of saving folds between sessions, and it's hard to see how that can be done without a database. The argument will go 'we already have quite a large partial database, what would be lost if we move to a full database model?' and I think that's true. I now see Obsidian as a database program rather than a files based program - I had assumed that this type of stuff was only in a cache which disappeared when the program was closed.

Since I don't want another database program, and since my needs aren't closely concordant to students, it seems likely that it will diverge further and further from something that works for me. So no longer Plan A, just a makedo until I have a better solution. The Plan B from last time is my new Plan A. And I'll maintain an open mind about the possibility of a better Plan B.

40hz · « **Reply #843 on:** November 18, 2020, 11:52 AM »

@ Dormouse, Wraith808, JavaJones:

Thx for taking the time to answer my question in such detail.

Interesting to see how, for you guys at least, this technology definitely is evolving to be an “extra” head. Which is how the authors of the software (especially Obsidian) seem to have envisioned it.

wraith808 · « **Reply #844 on:** November 19, 2020, 03:55 AM »

@Dormouse- that's a no-brainer for me too. It's one of the reasons that I stopped depending on Obsidian pretty quickly, and moved over to the Frankenstein model that I have (Using Foam and Memo extensions). But the plugins with Visual Studio Code has been working really well for me. I wish that I had the ability to embed references- it's one thing that would have solved some issues I've had. But it has worked out fine, especially with previews inline.

And for anyone using VS code, I just found a new model built on top of it - Dendron (https://www.dendron.so/). ; I can't say too much about it yet, but I'm definitely going to take a look to see what's different between that and the others.

Dormouse · « **Reply #845 on:** November 19, 2020, 09:16 AM »

Quote from: wraith808 on November 19, 2020, 03:55 AM

And for anyone using VS code, I just found a new model built on top of it - Dendron (https://www.dendron.so/). ; I can't say too much about it yet, but I'm definitely going to take a look to see what's different between that and the others.

It's designed to have a strong hierarchy, reflecting a folder structure.

Didn't suit me, even if I had been willing to use VSCode.
I'm OK with opening VSCode for one purpose and then closing it again, but that wouldn't suit any Note app.

Dormouse · « **Reply #846 on:** November 19, 2020, 10:01 AM »

Quote from: wraith808 on November 19, 2020, 03:55 AM

@Dormouse- that's a no-brainer for me too. It's one of the reasons that I stopped depending on Obsidian pretty quickly, and moved over to the Frankenstein model that I have (Using Foam and Memo extensions). But the plugins with Visual Studio Code has been working really well for me. I wish that I had the ability to embed references- it's one thing that would have solved some issues I've had. But it has worked out fine, especially with previews inline.

As you know, I have a much more nuanced approach to databases.
I'm OK with Writemonkey 3 with its coterminous files. I have my data in files and the extra features that come from the database (which include, I assume, its very good folding). Most writing programs have some sort of database; some also have files and some save much of the data in files.

But the Writemonkey approach to the database is far superior to that in Obsidian:
It's explicit that everything is in a database, with coterminous files being a selectable option.
The location of the database can be changed.
You can have more than one database.
You can run more than one instance at the same time.
This makes it easy to have a tiered approach to privacy and security.
Even so there are still some reports etc I wouldn't use Writemonkey for.

Obsidian has never been totally clear about what is saved where. Some is saved in the vault folder. But a large part has been moved to a json in a system folder.
It talks about vaults, and how every fault is totally separate, but then the data from every vault is in the same system folder in readable format.
It seems hard coded to only look at one location. If it's empty it writes another set.
And it has just announced saved searches. So a reiteration of the same question in my head - 'What is saved, where?'. I'm sure the answer will be that central json, but I'll have to run a few searches and do a file check to see exactly what's there. And will only be quick because I will know what I'm looking for and can do a search.
For me, it's straightforward poor design and not thinking through the implications of choices. Fixing a small immediate problem - today easier, tomorrow harder, and just don't think about next week.

Most of the immediate problems can be overcome. I control what I use it for. I can remove and encrypt the system file between uses (though that would always irritate me). And I can to a detailed test of every update (though they are weekly, more or less; I think I'll make it less going forward, updating is starting to feel too much effort for a small gain).
The big question about any developing software is where it's going to end up and how confident you can be about both quality and direction. This is where I now favour your Frankenstein model, though my version may look completely different to yours. Obsidian might be some part of it, maybe.

wraith808 · « **Reply #847 on:** November 19, 2020, 01:53 PM »

Quote from: Dormouse on November 19, 2020, 10:01 AM

As you know, I have a much more nuanced approach to databases.
I'm OK with Writemonkey 3 with its coterminous files. I have my data in files and the extra features that come from the database (which include, I assume, its very good folding). Most writing programs have some sort of database; some also have files and some save much of the data in files.

I'm actually not opposed to databases as it might seem. I don't care about something being in a db, if I also have it in files. It's the reason that I still use Scrivener, Writemonkey, and Cintanotes. In all three cases, I have the content of the database/project replicated in plain text elsewhere.

Quote from: Dormouse on November 19, 2020, 10:01 AM

Obsidian has never been totally clear about what is saved where. Some is saved in the vault folder. But a large part has been moved to a json in a system folder.
It talks about vaults, and how every fault is totally separate, but then the data from every vault is in the same system folder in readable format.
It seems hard coded to only look at one location. If it's empty it writes another set.
And it has just announced saved searches. So a reiteration of the same question in my head - 'What is saved, where?'. I'm sure the answer will be that central json, but I'll have to run a few searches and do a file check to see exactly what's there. And will only be quick because I will know what I'm looking for and can do a search.
For me, it's straightforward poor design and not thinking through the implications of choices. Fixing a small immediate problem - today easier, tomorrow harder, and just don't think about next week.

Most of the immediate problems can be overcome. I control what I use it for. I can remove and encrypt the system file between uses (though that would always irritate me). And I can to a detailed test of every update (though they are weekly, more or less; I think I'll make it less going forward, updating is starting to feel too much effort for a small gain).
The big question about any developing software is where it's going to end up and how confident you can be about both quality and direction. This is where I now favour your Frankenstein model, though my version may look completely different to yours. Obsidian might be some part of it, maybe.

You bring up some good points. I haven't opened Obsidian in a long while; at the time that I did it, it seemed that all files were in the Obsidian folder, i.e. if I created files in VS Code, Obsidian seemed to pick it up and pick up the changes if I had both open. I apparently (a) haven't kept up with Obsidian changes, or (b) was just unaware that it stored other files in other locations. That's really concerning to me, and I might just go ahead and uninstall it as it's sort of withered on the vine in my workflow.

Dormouse · « **Reply #848 on:** November 19, 2020, 02:23 PM »

Quote from: wraith808 on November 19, 2020, 01:53 PM

it seemed that all files were in the Obsidian folder, i.e. if I created files in VS Code, Obsidian seemed to pick it up and pick up the changes if I had both open

It will still do this, and the files themselves are just as they were and in the same location.

Quote from: wraith808 on November 19, 2020, 01:53 PM

I apparently (a) haven't kept up with Obsidian changes, or (b) was just unaware that it stored other files in other locations.

The big change came, I believe, in 0.8.7 when some files were switched from the vault folder to a system folder. Originally described as some data, and more recently as metadata, the amount stored has steadily increased as features have been added.

I'd prefer files alone, but I don't object to this in principle. I would have wanted detailed disclosure and much preferred the approach of keeping this 'metadata' in the vault folders.
I might investigate what happens if I leave the system folder where it is but just move the vault 'metadata' for vaults I want to keep secure to an encrypted space (moving it back when I want to access that vault).

The big hit is to my confidence in the developers. I was anticipating subscribing to their Sync service when it came out (e2e encryption, secure cloud) - don't need it but I'd rather contribute if I'm using it a lot - but there's no way I'd trust their implementation now.

Quote from: wraith808 on November 19, 2020, 01:53 PM

I might just go ahead and uninstall it as it's sort of withered on the vine in my workflow.

No obvious reason why not, if you don't use it.

I like the way transclusions work.

superboyac · « **Reply #849 on:** November 19, 2020, 06:04 PM »

Quote from: 40hz on November 14, 2020, 04:58 PM

Just out of curiosity …

What are people doing with all this information they’re curating and cataloging with these various pieces of software? To what purpose? Or maybe even: to what avail?

I’m more curious about the individual “business” use cases rather than the supporting technology. Technology and solutions that offer varying degrees of utility aren’t that difficult to run down. God knows there’s tons of software out there. But the reasons to employ said technology can sometimes be less obvious. At least to me.

So help me out. What is/are your goal(s). What’s it all for? What are you guys doing with all this information you’re gathering?

40....
When I learned about this, the thing that intrigued me the most was how prolific that Lurhman guy was due to using this system. Prolific in terms of churning out books. I also want to continue writing books, so that is my end goal. I remember writing my first set of books how annoying and time=consuming/frustrating it was to organize everything, and then once organized, even keeping track of your thoughts and notes and edits, etc. So the idea of using a system such as this to capture your thoughts and then later to recover it and put quickly together in long-form or book form, is the ultimate goal for me.

I've been an unorganized notetaker for many many years, most of my life. But their all over the place. If I wanted to put something together based on my thoughts, I have to search and find everything, then reread it (because they are long and disjointed), then re-understand my original points, then organize, then edit. Supposedly, this allows us to skip all that once the "zettels" are created and its just a matter of following each zettel (which should already be concise and easy to read/understand) and they are already linked, so you can just churn out books...theoretically.

Now, in practice, much of the last 2 years of the exercise was just to see if this even works with the software and tools etc. Nothing really "took" for me until i saw that software Neuron that I am loving. And the thing there was that it syncs with my local files and presents a very nice looking website instantly with everything there. The softwares technically do the same, but something about the aesthetic of neuron really is working for me.

So now I am just writing away, and hopefully some nice books will come out of it.

Business wise, I don't see many applications. Would employees really use such a tool? Not really. Even if they were research oriented, it still takes a rare kind of dedication to use it all (the markdown, the linking, the curating) it's all very rare. Maybe if the system could be more automated somehow, but still, I struggle to find a good application. Actually, in the neuron forum, one of the users presented it to a company, and he said there wasn't much interest at all, lol.

What's wrong with us? Why do we write and write and curate and write? I don't know, might be a mental problem.