Autogenerating a Book Series from Three Years of iMessages

I never understood why people care to keep their private conversation history in the first place.

One reason that's understandable without relying sentimentality, is they're a record of what you were doing or thinking at a particular time, much like a private diary.

There's been a few times where I've gone back though stuff like chat history to better understand something that I didn't realize the significance of at the time.

I think this is interesting, and not necessarily unpopular. It seems different people just think about this issue differently. I do everything I can to preserve every single chat history that I can. And I would like to have every face-to-face conversation and phone call recorded and easily accessible for that matter. I have a sense that I am the sum of my experiences and I don't want to forget those experiences - it feels like I am somehow less than myself if I don't remember them.

But I've seen that episode of Black Mirror, too. So I wrestle with the desire to perfectly remember everything that I've ever experienced vs the mental and emotional health benefits that clearly come from being able to forget things.

People used to write letters to each other and older folks usually have a drawer full of all of their old letters.

We have learned a lot of history by reading famous peoples' letters long after their deaths.

IMO private messages (as opposed to public posts, blogs, etc) are supposed to be temporary

Why? Privacy and permanency are orthogonal axes. You've never kept a cherished letter or re-read a thoughtful text message?

private messages are not face to face conversations nor phone calls. letters last a long time and would be a more accurate comparison

Because they're cherished memories for one.

Really depends on the mindset when creating the message. If I message on a platform that keeps history, then I write with that expectation, or at least possibility, in mind. Now, this begs the question - is this modification of behavior problematic, does it detract perhaps from the meaning of the communication itself? Maybe.

The barrier, to me, is broken the moment we use technology. I work with this shit, I know how the sausage is made. I know that the phone calls are as encrypted as HTTP, that everyone can always keep record without you knowing, that even if they promise that something will go away, it may be won't, especially because it's a juicier target now just that promise alone. As soon as something is electronic, then it's a record.

I can see both sides. I did actually correspond with people using written letters up until maybe 2004 or so, and in many of those cases, especially old girlfriends and letters from my little sisters when I first went to college, reading them years later was intensely nostalgic.

On the other hand, when I left the Army I moved into a much smaller place, put most of my stuff in storage, then three years later figured anything I hadn't touched or used for three years was something I didn't actually need, and let the facility have all of it. That seems to have included both all those old letters and all of my old photographs. I can't say I actually miss those things. People in here are saying they don't want to forget the past but the reality of forgetting is you don't know you forgot it so it has no perceivable effect once it happens.

To be honest, I'm nostalgic enough as is and don't think I need even more things to hold onto. I already don't watch new television or listen to new music. I'm mentally stuck in 1999 and not sure that's healthy.

are supposed to be temporary

However, this is just an opinion.

Thinking more carefully, I think my opinion goes more like this:

my data collected by others should be temporary, but I should be able to maintain my own data forever if I want.

I agree. But it's more to do with the part of me that cringes at messages I sent and exist forever. The person I was 10 years ago is so different. It feels so jarring reading old messages.

I don't assume those are private or ephemeral.

It tells a story and its the zeitgeist of our generation.

People haven't thought about too much how to preserve something like this.

I personally like the idea and i can imagine exporting this with all / few messages of my mother and having a memory of that time.

WhatsApp lets you export chats as txt, but I guess that's lossy.. e.g. I'm not sure the emojis will be there. Surely no attachments or voice messages.

As for extracting from the backup DB, they'll be encrypted..

This Python package exports WhatsApp backup chat to JSON or HTML:

https://github.com/KnugiHK/WhatsApp-Chat-Exporter

It also links to a Telegram exporter.

For the rest of the world…

This desperately needs to happen — in a way that all messages and media can be exported in a sensibly reviewable format. Heck, I'd just like to be able to archive a backup that I know can be restored in the future on another device.

I love it as art, but it's usefulness is questionable. If you want a hard copy, just copy it to a microsd. Or three if you are worried about losing it.

One of the great disadvantages of private emails (& texts) is the massive amount of correspondence that is lost to future historians. I have books of letters published by Feynmann, Feyeraband, Einstein, etc. Everything is now email that is behind a password, which means we'll likely never have troves of personal letters from which to contextualize modern people who become historical figures in the future.

The same, I expected a whole criminal AI generated novel based on a history of a chat.

Ditto. Exactly what I pictured from the title and I was already thinking how interesting that would be. I’m curious to try something like it now.

Me too. But the real story is way better! Now I want to do the same with my Telegram chat history.

With the EU's DMA law and the preceding GDPR, some services have to offer an API so that your hypothetical service can pull this data. However, iMessage was notably excluded from this law, and then there's the encryption thing where you can't just pull data from e.g. whatsapp.

No, I do too. I've been planning on doing what the author did for quite some time now, and this is one of the unsolved stumbling blocks.

Printing and binding at home is probably the only option. All that's left to figure out is how to make the end result durable haptically pleasant...

the publisher is the business of spitting ink on paper. you should be more unsettled by being MITM'd by data mining companies whose job it is to change behavior via ads or other consensus-building tools.

Maybe try the —rot13 switch.

:-)

I think if I sent my full history of private messages to a publisher, the most unsettled one would be the publisher!

Hm, why can't you restore it? AFAIK you should be able to.

I remember looking into a similar problem and learned that on desktop it was just an encrypted SQLite DB. It was readable with the standard SQLite library.

Not sure of the situation for the mobile backups though!

Author of the post here, thanks so much for making it available! It’s an excellent library and I was thrilled to find it.

Big fan of this library. Thanks for making it!

Thank you for this! I recently was digging into the sqlite files with an idea to monitor them for changes indicating new messages and then extract them. My initial prototype seemed to mostly work, with a few hacks. Next time I look at that idea I’ll switch to your library. Any suggestions or tips around near-time accessing?

Maybe you could use something like GPT 4 vision To include a text description of the image in the transcript

If been thinking about doing just this for a bit now. I plan on showing thumbnails plus a QR code for animations and videos; I have yet to figure out how to make the files accessible in a private and durable manner though.

Maybe they wanted it from/for the other persons perspective for some reason?

It is. I thought the author just mistyped in the post but looking at the repository example it's indeed backwards.

Perhaps the book is a gift for the person on the other side of the messages in question.

Why?

Telegram gives you a JSON export of a full conversation, should not be complicated to adapt the input.

https://github.com/KnugiHK/WhatsApp-Chat-Exporter

Just in case you missed my other comment.

Not my repo.

I often listen to Pocket TTS on the train or when I can't access my device to skip or do much other than play/pause, and oh my god this gets me everytime haha. I am actually thinking of DIY'ing my own web-scraper thing to do a better job at it because especially for scientific articles, it's really rough when it gets to any LaTeX. And then I'm sitting there listening to some very automated sounding voice read off cryptic numbers and greek letters and code and math notation like some kind of Soviet number station (which is kinda cool at first, but gets annoying haha).

I want some kind of local document host that I can run a summarization or filtering script over to extract the portions that are legible to TTS, pipe it into something nice like ElevenLabs (if I was rich) or whatever, and then host a OGG for me to listen to on the go...

are you posting this while having a drink at the Black Sun

Ya bro and then the next level would be to like feed that AI generated play or script and feed it into an AI movie maker. It would totally be a game changer! I'm super stoked and pumped about it

Or, hear me out: you keep the personal communication personal instead of feeding it through a randomness machine.

Ha, I’m not sure it’s quite the same. If my understanding of most couples is correct, this would be more akin to preserving all of their sticky notes e.g. “Pick up milk on the way home”, “You’re getting the kids, right?”, “see you in 20”, etc.

Yet I suppose there’s a certain charm to that, so I hope I don’t sound like too much of a wet blanket.

Or keep it to remind yourself of what a toxic relationship looks like, be that the case.