Notetaking and Knowledge Management
Problem 1: Walled Gardens
I have two to three online services with notes, collections of bookmarks, code snippets and so on. Each of them has an easy way in and a hard way out - means I can add data easily, but to port my data to another service is painful (large JSON blob, archive with html files...).
Problem 2: Finding My Stuff
The problem I solved a year ago and the corrosponding website I bookmarked, how do I find this again? Go to each of my services and search, hoping I tagged or described it well enough and in the best case the search engine of the service is somehow smart (fuzzy) and reads my mind. What I end up with is a google-search finding solutions again in the wild and rarely hitting my own archive or services.
The solution comes down to One entry point to find my stuff and in the best case One entry point to record notes and bookmarks. Spreading my data over several services is easy, because most services offer an easy way to add bookmarks from the browser or code snippets from a context menu etc..
My Dream Setup
- Storage is based on text files in my filesystem, so it's easy to keep that in sync via Dropbox or Nextcloud.
- Notes and code snippets are saved as markdown files
- Bookmarks can be added via browser or a service that is able to sync/backup my bookmarks to dropbox
- All markdown files and the bookmark file are getting indexed and every time I search on google or any other search engine, a browser extensions inserts the results from a local search through my files.
My Actual Setup
I have one global
/@docs/ folder in my Dropbox for general notes. That folder includes a variety of markdown files to different topics. The filename usually indicates what's in the files.
Logs.mdCodesnippets and gotchas on how to parse logfiles
CMS.mdall kind of information regarding CMS systems.
CMS Pico.mdinformation about that specific CMS
- and so on...
Projects are organized in folders, so every projects contains a
@docs folder with a markdown file inside, containing notes, ideas, todo lists.
Most of the time there is a YAML header with a
Software and Services
I access notes with Obsidian, Sublime Text (which is open most of the time anyway) and experimented with Typora, Caret, Mark Text or
Zettlr (I haven't settled on one yet, each of those editors has some unique features and lack others) on on my phone I use iA Writer, GTW - Markdown & Text Editoror Drafts 4+.
Copy as Markdown helps me to copy the link of a website, as markdown link.
Code snippets I'm torn about. I like Github Gist to share code, to fork other gists and so on. For that I use Lepton. I also wrote a script, that's backing up all my gists to my local hard drive. But I might save them as markdown as well as described here
Bookmarks I collect with Raindrop. The good thing about this service is, it saves a copy of the page, in case the site goes away (which happens too often) and I connect raindrops.io with Dropbox (via IFTTT), so there is a backup of all bookmarks in my Dropbox.
Search and Discovery
As mentioned before, it's easy to get data in, but hard to get out. The browser is open all the time, so I see that as my entry point for searching. Back in 2011, Google offered Google Desktop which indexed the hard-drive and displayed the results next to google search results. Unfortunately that is gone and I haven't found any good alternative.
What I have setup right now is a little tool that's based on Xapian indexer called Recoll. There is a Windows version, which works ok. Recoll indexes files (pdf, txt, mailboxes) and you can run a fulltext fuzzy search with stemming. Unfortunately, the web interface doesn't work for Windows, so there is no way to get results injected into the google search results page. But I could see that as an viable option. Or I'm running this command and parse everything myself with php.
Public Notes indexed by Google
At the end of the day, I have set up a working system, yet I still end up using google to search for the same stuff over and over again - even starting Recoll seems too much.
The consequence would be to publish my notes so they get publicly indexed. That's what I did with this wiki.
The best local filesystem search is Everything, and their 1.5 alpha version now has Content Indexing. This could be promising, but for displaying search results of text content, you need context, meaning text snippets.
The Obsidian editor seems to become the go to for note taking and building a second brain. The indexed search is great and really useful and working. For my approach I have to move everything in one vault to have it indexed. That might create the problem of losing focus, but I guess with the approach of Recoll or Everything indexing all my harddrive, I already have that problem.
- All Notes In One Folder (I do that for general notes)
- Notes Per Project (yep)
- One Single File With All Notes (I don't think that's very practical)
- Several Large Files About One Topic (that's what I do right now... see above!)
- A Mix Of The Above (yeah, whatever works)
See [[Markdown Editors and Note Taking]]
Other Peoples Thoughts
- Markdown, plain text based.
- Optional todo.txt org-mode
- Bookmark management based on markdown or be able to export to markdown or plain text (even html in a
- dropbox, cloud
- directory based
Personal File System
Theoretical concept of how to store files on a computer.
The act of using a computer (in general) for a purpose only materializes as an afterthought if ever. There is rarely a situation where one prepares the computer beforehand, or actively indicates "this is the purpose I'm using the computer now" before actual use. That is the root of the problem of organizing information. Using folders/directories means "folding N different meanings onto one single, rudimentary abstraction" and it's a shame that this is the only reliable primitive people can use to sort their digital information. What is proposed here is the need for a common model of "User Agent" (in the RFC 8890 sense) to interface humans and storage.
Really good writeup about the problem of how to record and access all movements across the web/research.
Concept of a Knowledgemanagement with Blocks
What's out there
- PHP based, Flat-file based
- writes text files with
- uses its own doku wiki markup
- markdown extensions are available but not up to date and the integration is basic.
- Many extensions
- Just save one html file with all program code and notes.
- Exporting notes possible
- A Good Looking Tiddly Theme for Digital Gardens
- Getting started with TiddlyWiki: a beginner's tutorial - Ness Labs
- How to build a digital garden with TiddlyWiki - Ness Labs
Obsidian (or any Markdown App)
Obsidian seems to be the best option in terms of linking docs and tags.
- Webservice that aims to be everything...
Similar to Mdwiki -
html file + js = formatted markdown docs