The benefit of journaling is not just reentry, but that you begin to solidify the mental model into a concrete branching of possibilities that is tightly coupled to the specific problem. Your work becomes traversal and mutation of this tree. Several benefits accrue: you begin to see gaps in the tree, and can fill them in. You begin to have confidence in your mental model, recovering the time you used to spend going over the same nodes again and again in a haphazard way. In distributed systems in particular, the work is often detailed, manual, error prone and high latency - with a solid mental model you can get through a checklist of steps with minimum difficulty and high confidence that you didn't miss anything. This ability to take something abstract and make it more concrete on the fly is a critical skill.
Perhaps the greatest barrier to using it is akin to envy. We see others who apparently do this without written materials, in their head. I think we see this as evidence of intellectual superiority and harbor the doubt that using an aid like a journal means we are somehow lacking in skill or ability. This is wrong. Using an aid to map out complex problems isn't a failure, it's essential, especially for problems in systems you've never used before. Over time you may yourself build up your expertise such that you no longer need the aid, but that doesn't signal anything about your intelligence or ability either, only your experience.
I wrote a program (used from a CLI, but I mostly use the GUI I developed for it) to do something similar for my own use: https://github.com/lelanthran/frame/blob/master/docs/FrameIn...
I use it daily.
Interesting tool.
I currently use a TODO text file in the root of each project I'm working on, which I update right before I stop working. The lists are hierarchical, so each task I'm focused on has a parent item, which helps me remind of the bigger picture.
Usually, ths simplest solution is best and your approach exemplifies it: no need for a special application, simply open the text file and then go to work.
I have found, and advise teams I lead or work with, that the tool absolutely does not matter because it is the discipline to use the tool consistently that makes a difference.
Now, having said that, shoving jira down peoples throats with all kinds of rules around tagging and whatever wears people out.
So, yes, a text file, a google doc, linear or a few post-its on the wall.
Agree, though I advise folks that tooling matters tremendously, because a bad tool requires more discipline to continue using than a streamlined tool.
This is very true. I guess I was thinking about all the times new people join a team and want to make changes to the team to fit the tools they prefer (including when a "big org" person rolls in wanting JIRA).
Toil and friction are killers.
In "Secrets of Productive People" Mark talks about building systems and that the lower level / background things should be reliable and without friction.
"Good systems for simple administration will free your mind for more productive work. Ideally you shouldn’t need to have to think about the lower-level stuff at all. Thinking needs to be kept for the high-level systems, which will be designed to fit each particular case. But even then the aim of designing a high-level system is to avoid eventually having to think about that system too."
I agree. The "tool" I have simply maintains text files and provides a streamlined way to display/edit the current one, switch to some previous one and switch back.
I made it to manage context switches. IOW, it's not to serve as a journal, it's to serve as a swap partition for my brain.
and a tool to squash or expand the hierarchy helps alot. jEdit can fold/unfold based on indentation. along with simple prefixes : '-' for info, '>' for todo, '= {date}' for done, etc
Mineis called README.md or notes.txt I use a similar hierarchical format - I use tabs for the hierarchy. I use markdown for most notes, but I find it somewhat incompatible with the tab-based note-taking that I prefer.
I share this in case there are others out there who work the same way. Let's band together to establish a tab-friendly Markdown variant!
It's low-tech, but I've been using notebooks (not too thick, stapled usually). I write a header for each day when I start and then a line or two as I work on tasks and I try to note each time I switch tasks.
I keep the notebooks but rarely look at them once they are filled. Maybe once or twice as I switch to a new notebook and then once again when it comes to yearly review time. A couple of times I have rifled through old ones looking for command line flags but most of the time reading through the notes is enough to jog my memory.
Van Neistat (YouTube) has a video that I can’t find know where he talks about his planning process. He uses full sized poster boards.
Lots of space to lay out your ideas and get a great overview just not very pocketable.
Also a note taker - the physical aspect acts as a kind of cementing the thought. I never look at my old notes either and discard them when the book is complete, it's just an excerise in helping me manage complexity.
A real notebook and pen are the perfect tools for this. Its the only one I know that works, long term.
append only log works, but skimming through 2 months of logs for a specific thing is poor and slow but very useful. I don't do that often but when i do need it sometimes and especially fast, like on a call, with no lead time. Its a superpower.
I now organize my notebook a with a few conventions that make it more useful.
I limit myself to using only the right side of a page for logs from whole week. Each new week gets a new page, always on the right side. Put the date as the page header. Makes skimming easier. Put any important tasks / targets for the week right after.
Further Split right page into 3 columns. First 2 for work logs, third column for recurring weekly/biweekly meeting stuff. Very easy to go back to what was discussed 2 months ago. All logs are usually 1-3 words. Just cues. but everything has a topic subheader.
Left page is free form for detailed stuff. Things i discover, design, 1:1 meeting notes, questions i need answers to, philosophy, anything new on HN, etc. Right page serves as context.
I also do an index on the first page of the book pointing to anything that i find repeatedly useful. Could do page numbers but i put the date as its naturally ordered by the dates on the right page.
Been doing this a while and works perfect. I have everything I need in one notebook, i can carry it with me. a $1 composition book with 50 pages lasts well over six months.
Not a comment about the project itself.
If someone wants to try a similar flow but cannot run the above for any reason, git can be used to achieve something similar. You can also make use of existing tooling around git for shell integration like PS1 and gui.
1. Your main/master branch is your root frame.
2. Child Branches + branch commits themselves are messages.
3. Pop is hard reset of branch to parent or branch switch
The main idea is your log resides in commit messages and not the commit data itself. You can try using commit data too but limit that for shared contextual information.
Wrapping it up(to generate phony changes and running git) in shell aliases or functions should be easy.
But how do you use that flow to _plan_ your work?
I wouldn't see that as a way to plan work, but rather to keep track of what you did or where you were.
In general, I like the idea of ruthlessly tracking what I've done like this. But I think that it is still missing the context of the environment... meaning, if you are editing a file, it would be nice to not only know that you edited file A, but also that you changed line N to X.
I've spent a decent amount of time thinking about this over the years and haven't fully figured out a good solution. I was a wet lab scientist for a long time and we have the tradition/requirement of keeping a lab notebook. This is something that is incredibly helpful where you keep track of what you've done and what you're planning to do. I've missed this when I switched over to mainly computational work. In the past, I've thought about maybe having a loopback FUSE-ish mounted directory to track changes to files would work. But I think something akin to working in a git tracked repository (with these intermittent commits per command) might make this work better!
This is really cool. How do you reorder frames?
If you’re working on a particularly complex frame, how do you maintain context?
Since each frame is a subdirectory directory in ~/.framedb/root, I can simply `mv` them. TBH, I've never yet found a need to rearrange them.
I create child frames using whatever context was written into the current frame.
Love the sound of this tool! Seems a bit of a shame to permanently delete anything that's been completed, sometimes it's really useful to have a record of what you've been working on.
I did originally have that; it lowered the signal:noise ratio.
I've found that there's isn't a need to keep around any frames of context for things that I have completed: generally there's already an artifact from that frame of context anyway (write this function, call that person, design that foobar, etc).
That is one of those novel once in a generation kind of tools and concepts I feel. Outstanding
Really cool tool. Starred
This is really interesting. Thanks for sharing.
It’s great. You should do a separate hn post
I write a lot of notes in fact I write more than I read. It helps. However I have some issues:
1. How much commentary should I write? I try to write not too many notes because I write code, and some say code should be self-documenting. So it's the same old question of how many comments should there be along the code.
2. How do I retrieve a note I've written earlier? I can use tags and search for them but it is not easy to come up with a perfect tag which I would remember later.
3. I have so many notes and by now many of them are out-of-date. I don't want to spend time updating my notes. But if I don't they can become misleading.
There needs to be a balance between "Just do it" and "Write about it". I'm not sure I have the correct balance between those two.
I can see an alternate approach which would be a FORUM where co-workers discuss what they are doing or plan to, or have done . But there too the retrieval might be a problem. However the FORUM-tool would automatically keep track of when something was written and by whom. NOTE: You might benefit from other people's notes just as you can from your own.
Check out obsidian.
It is just folders and markdown so it is ultra portable, but the obsidian editor has tons of useful features like a graph view, autolinking, and a plugin in for anything you can think of.
There are tons of YouTube videos and articles describing different organizing systems and ways to use it.
I like Obsidian, but plain text continues to reign supreme. I prefer nvAlt or my programmers editor, or if on a team, whatever the company provides (e.g. Jira). The embarrassment of options is actually a key problem in itself, since if you do not commit to one, you find your notes spread across various files, tools, and services, totally disjoint in a way that is impossible to work with and difficult to undo.
The ideal tool, which I don't think exists, would combine the immediacy and locality of nvAlt and bidirectionally map to something like Jira for sharing, distributed as a browser plugin and/or a simple server component with local write access.
Feed them all to an LLM?
I've thought about that. The AI should figure it out. But if I don't know what I should ask the AI it cannot much help me.
I wrote in some discussion about whether AI could replace us programmers? I think they cannot because:
AI has the answers. WE have the questions!
I write but never reread. The act of writing helps me organize my thoughts. Maybe I'll reread the last page when coming back to something but that's it.
So my advice is to write when you have a lot on your mind so that you can get it out of your mind. That's it.
Don't mess around with forums. That's for a different problem.
Keep a pen and paper handy always. There can be no barrier to entry or it breaks the flow.
Likewise, I seldom reread. But I do get most out of just writing it down.
It really depends on your work. If you're doing mundane work, keeping notes is just busywork and doesn't really have a payoff.
But if you're constantly trying to solve novel problems, and have episodic ideas that are half-baked, writing notes -- without trying to organize them first -- can be really powerful. For me, I just write them in Logseq and tag them with a few hashtags like #topic1 #topic2 #topic3. It doesn't have to be a perfect tag, just tag it with all the topics you think are relevant.
From time to timeI click a hashtag and revisit all my half-baked ideas -- periodic revisits and curation is key -- I surprise myself when some peripherally connected notes coalesce into a real idea. (Logseq makes this easy because each note is bullet point that can be tagged, and clicking on a tag is like running a query)
This is called the Fieldstone method. (conceptualized by Gerard Weinberg). It's a very useful approach for writers because it recognizes that the best ideas are episodic and don't all come at once, you have to gather the "stones" over a long time before something gels.
https://www.amazon.com/Weinberg-Writing-Fieldstone-Gerald-M/...
I've used it with great success over the years (both at work and in my writing).
Keep trying. Try different methods. What works for me debugging and refactoring distributed systems may not work for you. The thing I tend to drive toward is a single page mind-map-like artifact that is monotonically increasing in density with a focus on system-of-record and data flow. I tend to keep notes about individual tech and important systems in whatever tools are handy, either locally or in a form that are accessible to the team. Often this takes the form of a JIRA page per topic, or an nvAlt note per topic. Tickets aren't great for this because they are ephemeral. These notes are where code snippets, error messages, anything searchable, go. I have something like the Zettlekasten[1] method in the back of my mind when deciding on scope of these support notes. However I may also start from a simple local text file, not committed to the repository, or even attached as a note to a ticket or other ephemeral trigger. But the thing that ties it all together, for me, is that mind map. Once I have that I can truly reason about the system.
1 - https://zettelkasten.de/introduction/
This is the key point - capturing, organizing and retrieving notes has a cost. And I find myself always paying for notes out of my "just do it" budget. Especially when virtually all of the things I'm working on (notebooks, libraries, applications, planning documents) are themselves a form of writing, having yet another place to scatter my thoughts is not helpful at all. It's much more productive to take that thought and put it directly into the project documentation where everyone can benefit. More README, less journal.
I think you need to have a clear separation on what you're trying to achieve. From what you've written, I get the sense that you're combining documentation, wiki and your personal notes together.
For example, code documentation is very subjective. I'd combine code comments with readmes and potentially a separate wiki. All depending on the complexity of it.
For personal notes like reminders or thoughts, there's no need to keep it up to date. It's ordered by date. And when I need it, I roughly remember when I wrote it. If something needs to be updated? Write a new entry today. I use pen and paper for this. This is also where I sketch one-off diagrams and the like.
Separately, I also have a personal wiki for things I learn or teach among other things. Since these are limited in numbers and are quite important, keeping them up to date is not a big task
The problem with writing too much is that it becomes a chore to read, there could be a lot of fluf and a few gems, but you couldn’t tell anyways.
I think the best I've heard this was by a friend who said something akin to this;
Programming is the art of solving problems by codifying complexity. Generally, the self documenting part is the 'defining the problem' portion of the code. But in every problem, there's a certain amount of irreducible complexity, or it wouldn't be a problem.
There's going to be some part of the code where you wish you could make it simpler, or you wish you understood it better, or you wish you could break it down into smaller components but there's no 'good way' to do it in the system you're working in. Or, the way you have working is 'good enough' and it's not worth the investment from some (business needs angle) in making it any better.
This is the portion of the code you should comment, and document, and do so liberally and in detail.
I’ve been seriously considering using Logseq for this reason.
When I first started with Obsidian I used it that way, but the more I put in it the more I started organizing everything. It became less of a journal and more of a repository for long form stuff.
I’m thinking about using both just so I have a dedicated tool just for the journaling side of things.
I use different editors for different purposes, e.g., Obsidian for long form and planning, OneNote for meetings.
I wouldn't overthink it, though, and just use the simplest tools available. I use Sublime Text 3 with a few shortcuts to add the current timestamp, etc. and log everything in a long file. I was too ambitious in the past and wanted to learn how to use Emacs for everything, but it just held me back, and I ended up without any notes.
Also, my unfortunately named thread from 2022: https://news.ycombinator.com/item?id=33359329
do you have a decent system to get entries between(both in and out) Obsidian and OneNote?
I have a similar system to you, but getting things in and out of OneNote is such a massive pain.
Everything I have tried requires significant reformatting (even Word).
I usually don't move notes to and from OneNote. I tried this Obsydian importer, and it is OK for my meeting notes in terms of formatting. However, it has a bug that strips slashes from page titles, which is a bit of an issue for me because I always add dates to titles (e.g., 7/13/2024).
With a few good examples, maybe some LLM could help you with reformatting?
Good luck!
I've been using Logseq as a work journal and it works great. Hashtags help me to track what I need to do and what I've done.
Love Logseq. It’s the best model Ive found for how I want to take notes. I used to get analysis paralysis managing structure or thinking about when to split off a new note. But with Logseq, you just write linearly and the hashtags take care of all that for you while making discovery a great experience.
For those struggling with overwhelming functionality an ux tax of note-taking apps:
Same here. I began using Obsidian when looking for something to drop my collection of Markdown notes into, which was somewhat of an improvement due to the quick search, tags and links, but not life-changing. I tried the Kanban plugin but gave up after a while. Then I read about the MOC concept[0] and started with topic-based index pages using the `dataview` plugin for generating lists of backlinks. Haven't looked back (yet)!
You could also create an index of MOC pages with the same plugin and making sure each MOC have a `#moc` tag, for example by using templates. Then write a query that lists all pages with the `#moc` tag.
For pure TODO lists, I'm a happy user of Taskwarrior since more than a decade.
[0] https://obsidian.rocks/quick-tip-quickly-organize-notes-in-o...
I use Emacs Org Mode with Org Roam for journaling. I’ve customized it extensively to fit my workflow, but there is a risk. Unlike paper, in Emacs there’s a potential distraction lurking around every thought, every entry, even every keystone. I’ve tried going to paper many times to mitigate the risk, but it never sticks. I’m too far down the hole and habituated to change. But if I were starting over, I would choose paper and stay with it. When journaling, you want no distractions. Nothing beats paper.
If I'm working on something particularly complex I basically just do the journaling in a code comment adjacent to what I'm working on. So the first commit may be three lines of code and a huge long winded rambling comment of what I've already tried or thought about. By the time the task is done I've pared away the more speculative/rambling elements of what I wrote, and what's left is typically some extremely well commented code. I think this method results in higher quality code produced faster than if you just try to keep everything in your head. No one's complained yet...!
A good technique I've used as well. Call it "iterative literate programming".
I like that. There are other ways of capturing work in progress adjacent to the code, instead of writing a journal. One of my favourites is to write a failing test - pretty much impossible to overlook or misunderstand on "re-entry" to the task.
Another is to write a temporary commit log, with "WIP" in the first line and a TODO list in the rest of the log. This is good for ephemeral information that would just clutter up the code.
If I do need something like a journal, I have occasionally just written a private gist and put that in a tab on my browser.
To add to your "this is wrong": These others may have themselves solved the problems we are now trying to solve, likely even using a journal. They no longer need a journal since they know how to navigate it, and it appears as superior to us.
This was the biggest revelation in grad school for me, I think.
The professors were in fact not gods of problem solving, they just had the answers. Not just of the problems they brought along (obviously). But also of the handful of problems we’d tend to invent. Of course, if you really catch them flat-footed, they can provide circumspect and sagely advice, and then quickly check Wikipedia to see if anyone has solved your problem.
I mitigate this "problem" somewhat by taking pride in specialized yet (hopefully) readable notations and sometimes even creative tool usage. Like using a project planning tool with a Gantt chart feature to speed up a boot process with many dependencies.
Of course, sometimes an existing tool or notation used for its original purpose is what you need. Maybe manually remove the stuff that doesn't matter.
I had a whole CLI full with bash scripts and a Sublime Text app with self-written plugins that functioned like this at my previous job.
You mean, for journaling, or to automate the process?
Is the process of writing not for reading it but to solidfy memory? Of course the bonus is you could read it, but you really won’t
I sometimes write to document my thoughts - but just as often I write in order to discover what I think. Sometimes it's like the writing is the thinking.
Also, I love Obsidian. It became more useful when learned to stop overdoing it with exploring the endless plugin options, settled on a favorite few, and now mostly write in the daily note, occasionally extracting things to dedicated devnotes which in turn have chronological timestamped entries (and bidirectional links to the corresponding DNs). Highest possible recommendation to find a tool / workflow that suits you, and leverage it.
Experience looks like intelligence to us because we didn’t see the hours our favorite instructors and mentors spent banging their heads against their own notebooks, haha.
It is the rare instructor or mentor that takes pains to NOT show off their expertise, and model the expected behavior, tools, and techniques of the learner where they are at. It is the classic problem of telling someone the goal, but not how to get there. A common error-mode is repeating "the magic" over and over again, expecting it to sink in; it takes extra effort to decompose the magic into teachable, practicable parts. This teaching effort requires meta-cognition and empathy orthogonal to the effort required to become an expert, which is why expert teachers are a rare and precious gift.
This advice is pure gold. Thank you.
I believe it cuts both ways. Writing things down allows you to dump state so that you can make other complex calculations with your working memory. Then, learning to hold more complexity in your head increases your mental bandwidth using the same amount of external state available. Rinse and repeat.