My Note-Taking Solution

What programs I use to take and keep notes

Posted by Brandon Wong on 14th Oct 2019

The Problem

A few years ago (at least four or five), I was searching for a better way to take notes. At the time, I was simply writing in plain text files with no structure or standard format, and with really bad file names (like "notes.txt", or "2014-07.txt"). These became disorganized really quickly, were very difficult to search through, and very easy to lose or corrupt. After switching to another computer, I would copy the files over - probably in a folder called "backup-from-old-computer-2014-09" or something like that - start a new batch of files, and the process repeats. And don't even bother trying to access them from a different computer than the one they're sitting on - it's not worth the hassle.

What I needed was a way to take notes that was structured, easy to file and organize, and easy to access. There are many notes-taking applications, but most of them are too text-editor-oriented - they are trying to be like physical notebooks, with rich-text formatting added on top, then organization as an afterthought. The number one problem with almost all of them is that they host your data on their servers. If you know anything about me, you know that I try to avoid that, especially if it can be helped.

In short, I was looking for a wiki. In addition to being a web technology (which is right up my alley), a wiki fits the bill. It's easy to search through, easily organized, and can easily link between pages. Futher, if I self-host (which is also my specialty), then I can access it from anywhere I want.

The Solution

In keeping with my tradition of finding awesome programs that have terrible names (check out Gogs, Piwigo, and Scrot), I eventually found a program called TiddlyWiki. I wanted something that was simple to set up and simple to use, and this was both. In it's simplest form, it is a single html file which contains all the markup, styling, code, and content. When you want to save your work, you just "download" a new copy of the html file with your new text included. They also have a node.js server-hosted version, which is what I use. It saves each page into a flat file - more on that later.

TiddlyWiki

TiddlyWiki has a unique interface for a wiki. Each page (called a "tiddler") appears in a box on screen, and there can be many open at a time. You open, edit, and close them as needed. This excellent, since the kinds of notes I take are relatively short, and they are closer to being like post-it notes than a notebook page. It would be a waste of space to fill the screen with a single page, and it's useful to see more than one at a time.

The system is very customizable, very extensible (it accepts any javascript code), and it is well maintained. It also has many plugins available (or, as I mentioned, you can make your own). Examples include a vim-mode editor, a LaTeX renderer, and a markdown renderer (the default is wikitext).

I think TiddlyWiki is ideal, especially for a single user taking lots of notes in small chunks (though it can handle long ones too), in many different categories, and needing to access them from different devices.

Version Control and Timestamping

Because the server-hosted version of TiddlyWiki saves flat files, I realized that I could use git to maintain a version history of every page I create. I set up a cron job to automatically commit all changes to the folder and push it to my host once per day. I use it often enough to justify a daily commit, and if there are no changes, then nothing happens. Because no one has access to the server itself, it's relatively secure. One of the benefits of keeping a version history is that if someone hacks into my wiki and changes things, or the files get corrupt, I can simply revert the unwanted changes. Plus, I now have a history of all the changes I make, but without the complicated database overhead (like in MediaWiki, for example).

Later on, I did a lot of work with a program called opentimestamps. What this does is it commits the unique hash fingerprint of a file onto the bitcoin blockchain (technically, it puts it into a merkle tree with thousands of other files to be included). Because a new block in bitcoin is created approximately every ten minutes, and because each block is cryptographically "locked in" within hours of being created, then any data that is committed into a block at the time it is created will be there forever. It's like having a photo of you and a famous person printed in the newspaper. You can prove to everyone that you met the celebrity and approximately at what time by showing them the newspaper. In short, it allows you to timestamp your files. opentimestamps is easy to use, and absolutely free.

I wrote a script to timestamp each tiddler in my wiki. opentimestamps creates a small file containing the cryptographic proof of commitment. My script compares the modification time of the file to the creation time of the timestamp file. If the tiddler file is newer than the timestamp file (or if the timestamp file doesn't exist), it will timestamp it again. The best part is that I save the timestamp files in git as well. That means not only do I have a full version history of the tiddler files, but I also have a timestamp proof of each version! For now, the only real reason I do it is because I take notes of the ideas I have, as well as writing pieces. If ever I have to prove that I had an idea before someone else, or if there is some sort of copyright/plagiarism dispute, then I have some way of demonstrating that my content existed in a certain format at a certain time.

Conclusion

As I mentioned before, I've been using this setup for at least four or five years. I've used it in class lectures (including complex math notation) and in meetings, for writing drafts, cataloging ideas, compiling birthday wishlists, recording code snippets, and planning projects. So far, it's been great, and I haven't found anything better.