11 replies
3d17h
As a heads up, this seems to break completely if any of a couple of different Chrome extensions I have are enabled (for example, Simplify - https://chromewebstore.google.com/detail/simplify-copilot-au...).
Also, maybe it's just me, but I feel like the sqlite-based thing is, if anything, kind of a downside. Use it for caching and calculation, sure, but I want the source of truth to be just plain markdown files I can take into other apps in 5 years when whatever I'm using now inevitably dies.
I know many note-taking users like the concept of markdown and plain text. So do I. However, I think plain text has its limitations. There are many other types of structured data in life. Just like Word and Excel, they have different responsibilities. No one worries that Excel will not work in a few decades because it's just an offline software. Similarly, no one worries that SQLite will be unable to open or view because it has already been running with billions of instances, becoming part of the infrastructure.
SQLite won't stop working but your format is proprietary. A markdown can be understood even without rendering to HTML but your tables are useless for an user without your app.
You can likewise encode proprietary information in Markdown, in the sense that it can be readable but not necessarily understandable by another application that understands Markdown. Like say for example my application allows for tables that include formulas in cells. Perfectly human-readable Markdown but probably won’t have the same level of fidelity in another application.
This is why I use RTF. It is considerably more flexible than Markdown (which can't do basic stuff like making text red), but not a proprietary format. The application I use (DEVONthink) supports a number of formats including Markdown, HTML and RTF, and documents how to get at the raw files if the application ever becomes unavailable. No use of a proprietary database format. This is for my 30-year files.
I also use RTFD in the same application. This is a variant of RTF which can have embedded images. It's not universally supported like RTF, but there are sufficient third-party editors to leave me confident that I will be able to get at the information. This is for my 5-year files.
A key idea of Eidos is to make each table a real SQLite table, so users can view and modify it through other software or visualize it with tools like Metabase
https://eidos.space/show/sqlite.webp
As far as I can tell, the whole app is open source (AGPL), and sqlite is obviously open source as well. I agree that markdown is more portable, but this setup is far from proprietary.
I may not have made the same decision (I love having my notes in markdown), but I think OP was quite thoughtful about it and has made an informed and reasonable decision.
What logseq is going to do is just dump documents to markdown (right now they use markdown),I think something along these lines is the way to go, store in sqlite and have a constant background process creating markdown files for everything
good idea. all documents are currently stored in the eidos__docs table. The `content` field is used to store the state of lexical documents in JSON format. Additionally, there is also a `markdown` field. This can be viewed by any sqlite software. Every time a document is updated, both fields are updated simultaneously, making it easy to convert to a file.
That's perfect. For what is worth, I like that's stored in sqlite because I can access it easily in a programmatic format if I need to
My general answer to this is to make sure whatever thing I'm building understands Pg COPY TSV syntax - specified by https://www.postgresql.org/docs/current/sql-copy.html under 'text format' - and export to/import from that.
It's nice to have something that plays nicely with awk/cut/diff/etc. and can be committed into a git (or elsevcs) repo.
(I'm not specifically wed to COPY as the form, but given it's well known and documented I've found it a good default)
Interesting, I'll give it a try.