Notetaking and Knowledge Management

Problem 1: Walled Gardens

I have two to three online services with notes, collections of bookmarks, code snippets and so on. Each of them has an easy way in and a hard way out - means I can add data easily, but to port my data to another service is painful (large JSON blob, archive with html files...).

Problem 2: Finding My Stuff

The problem I solved a year ago and the corrosponding website I bookmarked, how do I find this again? Go to each of my services and search, hoping I tagged or described it well enough and in the best case the search engine of the service is somehow smart (fuzzy) and reads my mind. What I end up with is a google-search finding solutions again in the wild and rarely hitting my own archive or services.

Solution

The solution comes down to One entry point to find my stuff and in the best case One entry point to record notes and bookmarks. Spreading my data over several services is easy, because most services offer an easy way to add bookmarks from the browser or code snippets from a context menu etc..

My Dream Setup

  • Storage is based on text files in my filesystem, so it's easy to keep that in sync via Dropbox or Nextcloud.
  • Notes and code snippets are saved as markdown files
  • Bookmarks can be added via browser or a service that is able to sync/backup my bookmarks to dropbox
  • All markdown files and the bookmark file are getting indexed and every time I search on google or any other search engine, a browser extensions inserts the results from a local search through my files.

My Actual Setup

Structure

I have one global /@docs/ folder in my Dropbox for general notes. That folder includes a variety of markdown files to different topics. The filename usually indicates what's in the files.

  • Logs.md Codesnippets and gotchas on how to parse logfiles
  • CMS.md all kind of information regarding CMS systems.
  • CMS-Wordpress.md or CMS Pico.md information about that specific CMS
  • and so on...

Projects are organized in folders, so every projects contains a @docs folder with a markdown file inside, containing notes, ideas, todo lists.

Most of the time there is a YAML header with a tags field.

Software and Services

I access notes with Sublime Text (which is open most of the time anyway), Typora, Caret, Mark Text or Zettlr (I haven't settled on one yet, each of those editors has some unique features and lack others) on on my phone I use iA Writer or Drafts 4+.

Copy as Markdown helps me to copy the link of a website, as markdown link.

Code snippets I'm torn about. I like Github Gist to share code, to fork other gists and so on. For that I use Lepton. I also wrote a script, that's backing up all my gists to my local hard drive. But I might save them as markdown as well as described here

Bookmarks I collect with Raindrop. The good thing about this service is, it saves a copy of the page, in case the site goes away (which happens too often) and I connect raindrops.io with Dropbox (via IFTTT), so there is a backup of all bookmarks in my Dropbox.

Search and Discovery

As mentioned before, it's easy to get data in, but hard to get out. The browser is open all the time, so I see that as my entry point for searching. Back in 2011, Google offered Google Desktop which indexed the hard-drive and displayed the results next to google search results. Unfortunately that is gone and I haven't found any good alternative.

What I have setup right now is a little tool that's based on Xapian indexer called Recoll. There is a Windows version, which works ok. Recoll indexes files (pdf, txt, mailboxes) and you can run a fulltext fuzzy search with stemming. Unfortunately, the web interface doesn't work for Windows, so there is no way to get results injected into the google search results page. But I could see that as an viable option. Or I'm running this command and parse everything myself with php.

<?php

exec('recoll.exe')

At the end of the day, I have set up a working system, yet I still end up using google so search for the same stuff over and over again - even starting Recoll seems too much.

Thoughts

  1. All Notes In One Folder (I do that for general notes)
  2. Notes Per Project (yep)
  3. One Single File With All Notes (I don't think that's very practical)
  4. Several Large Files About One Topic (that's what I do right now... see above!)
  5. A Mix Of The Above (yeah, whatever works)

See [[Markdown Editors and Note Taking]]