79 replies
1d10h
The irony of it all is that MongoDB actually matured and you do not have to jump through hoops to have, say, transactions.
I am ALL for Postgres, I even wrote a post about the importance of being fluent in relational databases: https://renegadeotter.com/2023/11/12/your-database-skills-ar...
What grinds my gears is the whiplash caused by this frantic stampede from one hype to the next.
I am just waiting for the AI hype to blow over and for everyone to rediscover Hadoop, or something.
Postgres is no hype, it was already in plateau of productivity when MongoDB came to the web scale scene.
Hadoop just like Mongo had their hyped time in the sun, but RDBMS are far too advanced and versatile than any of them were.
I am not calling Postgres hype - it should have never been NOT hype. It's a reasonable default for most problems,
Now all I read about is how Postgres is awesome, as if it's this great new thing. I guess that makes sense, as the new generation of engineers is rediscovering stable, reliable, lean technologies after a decade of excesses with "exotic" tech.
For grey beards, it's all very odd. Like, "where have you all been?"
I remember Robert C. Martin ("Uncle Bob") going on about how No-SQL will replace literally all SQL and that there is literally not a single use case for relational data and SQL. I wonder if he ever came back on that.
Now, my opinion of Martin in general is not especially high and he's a bit of a controversial figure, but it wasn't just the kids. And Martin is also all about reliable software, so that makes it even more surprising.
He's always been fundamentally a salesman. Hopping on the hype trains lets him sell more.
He's always been fundamentally a zealot who sees most narratives in black and white terms. It's an extremely unfortunate characteristic of a software engineer.
It's the same reason he's so pro Trump I think.
I didn't even know about that; I checked his Twitter, and first thing is a reply to the Baltimore bridge collapse:
https://twitter.com/unclebobmartin/status/177262869295654553...
You really need to be a special sort of individual to go from a collapsed bridge to ranting about pronouns with no prompting or any prior mention of that. What a complete spanner.
From the other side, people like me, see your odd dismissive behaviour in the same light. This bridge and hundreds of other things going wrong throughout the world atm are all symptoms of some sort of rotten problem, and I honestly don't know why you guys are so adamant about dismissing it. Almost as if you want to see it all come crashing down.
Pure evil, as Trump would call it.
Just one problem, huh?
This illustrates what I meant about black and white thinking.
Not sure what angle you're going with. If you want to say there that there is more than one rotten problem at the core of society, go for it. I'd be the first to agree with you.
This is not black and white reasoning. Each problem can be on a continuous plane, from "slightly bothersome" to "there is an absolutely horrible evil lurking here".
Is the rate of these incidents increasing, decreasing, or about the same?
Obviously you can say I'm biased but I think they're increasing.
Agree. Out of control neo-feudal lords taking up all the slack out of every system they can get their tendrils into in the name of efficiency, thus endangering us all in order to extract as much profit as possible out of our entire economic system.
The "other side"'s solution tends to be to further deregulate industry and the economy, and to empower and lower the tax burden of these neo-feudal lords, all while pointing at gays, trans-folk, and immigrants as if they are the reason people can't afford rent and bridges are falling down.
Another man opens the morning newspaper, and after scanning the headline, declares: "Honey, this planet is going to hell".
Yes, a single accident of the type that's been happing sine bridges were invented, pronouns, and unconditional support for an out of control government in Israel are all deeply related...
Or he's a hyper-politicized idiot completely consumed and obsessed by some topics and needs to talk about them all the fucking time at the drop of a hat, whether appropriate or not.
It has nothing to do with "sides"; it has to do with the forceful injection of completely unrelated matters.
That happened to esr, too, although he derailed right after 9/11, and it's been downhill ever since.
The reason he says he voted for Trump is taxes. But that seems like a pretty flimsy reason. Have dems in the past couple decades ever reverted any republican-instated tax cuts for the wealthy/corporations, or even made any moves for tax bumps? In my experience dems are just republicans, except they wave a rainbow flag every now and then.
His words: "I am also self-taught. I never got a degree of any kind. I entered the industry at the tender age of 17 and never looked back. Frankly, school isn’t all it’s cracked up to be — especially in the software field."
https://twitter.com/unclebobmartin/status/107212575854868889...
Which means his opinions are more often than not uninformed opinions and should be taken with a grain of salt.
I think you overestimate college education a little too much. Having a degree doesn't make your opinions automatically better than those that didn't have the chance to get one. You wish it did, but it don't.
When you turn 23 in a couple years, you’ll look back at college and realize you weren’t at the apex of all knowledge back then.
Unpopular opinion, Robert C Martin has caused more harm than good. People take his ideas and blindly follow them no matter the cost. I’m looking at you clean architecture.
Well yeah, solving problems and thinking critically and learning are hard and boring. "Uncle Bob" gives you handy alhorisms you can spam on your next pull request and then feel like you've done something.
There's definitely a missing piece he hasn't explained well enough. I've seen the best code in the world because it was clean architecture and I've also seen some truly atrocious code in the style of "clean architecture".
To me the difference was in recognizing that the architecture advice is implicit - its not actually written down in the code, rather the code exemplifies it - and the business domain is explicitly modeled. When developers name everything "use case" and "interactor" and go crazy with packages and folder structures with architectural names it's a recipe for terribleness.
I also remember Michael Stonebraker basically saying something, but being ignored...Along the lines of "I will see you all map reduce and nosql people in 20 years, after you have all reinvented SQL...."
Not an actual quote but a factual statement
Because he realized you can parse SQL and generate the equivalent of a map reduce job on the backend to execute it.
That's as ridiculous as saying "integer division will be useless in 20 years". Sometimes relational databases really are the optimal solution to certain problems. I really wish CS had better authority figures, although we shouldn't need them in the first place.
Postgres as we know it has only existed for about a decade, since the post-9.x era in 2010-2014 when many of its lauded features were added. Replication, heap-only tuples, extensions, foreign data wrappers, JSON, leveraging multiple indexes, parallel index scans, reindex concurrently, JIT compilation, declarative partitioning, stored procedures, and major WAL improvements are all "recent".
I love Postgres and it's been my go-to default since 2014 and you can pry it from my cold dead hands, but it's more contemporary with Perl 6 than Perl 5 if we're talking grey beards.
Extensions and stored procedures are not; they've been around for longer.
I do not actually use any of the others you listed (well, occasionally bitmap index scans, but it's not a killer). The main difference between 8.x and 15.x for me is the small stream of optimizer features and tweaks (e.g. at some point, it started being able to reorder outer joins).
Postgres 8 supported functions capable of returning tuples (RETURN QUERY), but lacked transaction control and other features that CREATE PROCEDURE added in Postgres 11 (2018).
Extensions in Postgres 8 (and earlier) were also half-baked compared to 9.1 (2011), which added CREATE EXTENSION and other quality-of-life improvements including standardization.
The latest complete implementations are what most people have in mind when talking about them.
Sure, there are improvements, but it's not like Postgres was a radically different beast back then.
Would I pick Postgres 15 over Postgres 8? Yes, any day of the week. But I would also readily pick Postgres 8 over most of today's databases (including e.g. the latest MySQL and MongoDB versions), for most projects, if that was the choice I had.
I would not have risked my business (or budget) on Postgres 8, even with Slony-I. My appreciation for its well-written code and thoughtful architecture simply did not outweigh what it lacked for a reliable operations experience.
MySQL foot guns were mostly under control by 2010, and it was operationally efficient to manage a cluster of MySQL servers in diverse configurations with mostly-known boundaries to what could go wrong.
The Postgres foundations are the same today, but what you can build on top is very different today than it was pre-9.x.
This assumes you need replication, though. You can get pretty far without it, especially given how wide hardware you can get now.
Eh. :-) I think you would have to specify which footguns in particular.
Replication can be for HA, not just for scale. All depends on your business requirements.
Also replication can be good for other operational reasons, such as zero downtime major version upgrades. Again depends on the business need/expectations.
If you're doing tens of millions in revenue and the RDBMS is critical to your operations, I'm a firm believer you need replication. Keep in mind this was the era of spinning rust and a day-old backup could take hours to restore even with a SAN/DAS. A replica was for more than just distributing load, it was for resilience to keep the business healthy and avoid a SPOF.
Risks to the business are more important to me than my enthusiasm for a technology or product.
I operated MySQL clusters in the late 2000s. I still am surprised how many hoops you have to jump through to set up replication in PostgreSQL than what you could do - out of the box - with MySQL 15 years ago.
To be frank, PostgreSQL has evolved a lot since the late nineties. There was a time where people preferred MySQL over it at it seemed to work faster, certain things were easier and so on.
I used postgres in the 90s. The biggest complaint was that postgres's default config needed a change so you could connect to it remotely, as the default, wisely at the time, decided that allowing connections from any IP by default before you changed any security setting whatsoever was a real problem.
The MySQL performance advantages were large, with the small caveat that it was fast as long as you were using MyISAM, which achieved its speed by throwing away many safety features, like working transactions and foreign key constraints. Unsurprisingly, when eventually it got an engine that didn't throw caution to the wind, the performance advantage went away.
I spent a lot of time in the early 2000s showing people whose that many of the issues they were facing with their e-commerce websites were ultimately down to the MySQL engine they had picked. But it's faster! Yes it is, but now your application code has to deal with all the things the engine has now offloaded to your application code.
It's not that Postgres' changes were key to increasing market share, but that as MySQL matured, it had no options but to lose its 'advantages'.
That mirrors my experience. MySQL was faster for use cases where data loss was acceptable. Will anyone die if you lose some web log analysis stats? Probably not. You wouldn’t want to use it for anything where guaranteed consistency was critical, like anything involving money, but it was an alright replacement for things like an in-memory data store.
I say “was”. I hear it’s vastly better now.
If you want a fast DB for web log analysis, you shouldn't look at MySQL or PostgreSQL anyway. ClickHouse will run circles around both of them for the use case of web log analysis.
That was an example of the sort of thing that someone might reasonably have used MySQL+MyISAM for in the past. I wouldn’t use that combination at all today.
MySQL was more popular for a while, but from a technical standpoint Postgres was always ahead.
PostgreSQL didn’t exist in the 1990s. It was called Ingres. Postgres started as a personal project and was first released in 1999, but was unusable. Around 2004 the project started getting popular and hipsters started screaming that MySQL didn’t have stored procedures and wasn’t fully ACID. Acid became a buzzword for PG fanboys, even though data corruption and memory leaks plagued the system for the first decade. It became a stable around 2008 as long as you didn’t mind restarting your database every few days. PostgreSQL didn’t really become a viable option until around 2010.
There was a time PostgreSQL did not run under Windows, and that is IMO what gave MySQL the market share edge.
Without these windowless years, PostgreSQL would be the most used RDBMS right now.
Excited, vocal users make the most noise.
Somehow the entire Mongo DB era passed me through and I never used it once. I used to use MySQL in the 2000s, and switched to PostgreSQL in the 2010s.
You haven't missed much TBH. I think the only case where you'd use it in new projects in DynamoDB if you work with AWS.
Dynamo is not mongo?
No, DocumentDB is Mongo. What I understand the OP parent referred to is the NoSQL hype era, not just Mongo which was probably the most popular one at that time.
My company considered it, but went with Cassandra instead.
They used it at my 1st job, a startup.
They had an ORM which made everything much slower, and the most common operation they did was to add items into lists, which causes the whole document to be resized and requires it to be moved.
It could have been done with 1 sql server, but we needed 4 really heavy mongodb servers instead. And of course our code was buggy so the schema was all over the place.
...and, continuing the cycle next season, devs will (re)discover NoSQL and document DBMS. I do wonder if this phenomenon of trendy tech is somehow integral to advancing the state of the industry, the SWE equivalent of early adopters. "Rediscovering the old" is a side effect, as old tech is used and reinterpreted in new ways. Kind of neat, really.
There’s an entire industry of people selling knowledge of the “next great thing/the one true way," but they need a new thing every few years as people learned whatever the old thing was.
This is what happens when you hire 20-somethings who don't know how computers actually work to build your entire business. You either learn about relational databases in school, or through experience. Relational algebra has been around for half a century, at least. If someone doesn't know about its utility, it's a failure of the education system, or a failure of corporate incentives.
MongoDB is more popular that it ever has been. You can look at their quarterly filings and learn this. The NoSQL space is larger than it ever has been and is the fastest growing segment in the database world.
Postgres is not hype though. There's very few legitimate use cases for not using a RDBMS as a main data store, and Postgres happens to be the most popular nowadays, for good reasons.
The number one reason being that it’s free. In a purely technical comparison it is not at the top of the list.
What's top of the list technically?
Oracle
Oracle is a giant mess, is the sense I got from trying to work with it in the past.
While I am loathe to recommend any Microsoft product, SQL Server is at least coherent, and has actual transactions like Oracle (and not just "checkpoints")
Aren't MSSQL and Oracle just doing redo logs, like MySQL?
What do you consider not actual transactions?
Postgresql keeps dead tuples in place until vacuumed, which seems optimized for rollbacks. But isn't so bad if you're inserting much more than deleting or updating.
Nestable transactions = “real transactions” to me, I guess.
I almost don’t see the point in non-nestable transactions
SQL standardizes SAVEPOINT for such nesting. Is that not sufficient? FWIW Postgres supports it.
Anybody that has ever used Oracle knows how nice it is. But very few people have that experience because their company isn't spending millions of dollars on the database server.
Yes, that was my point.
I get that working with Oracle and their army of lawyers is not cheap or pleasant but the db is still excellent.
I don't think that's true—all the old cases that were legitimate before are still legitimate. The price of both persistent storage and memory has simply come way down to the point where many computational workloads are viable under vertically scalable databases again. I suspect the pendulum will swing the other way one day yet again (and of course there is a rich ecosystem of ways to query horizontally, if not transactionally, with SQL that will likely temper this swing even further).
The price of persistent storage and memory was the same for Mongo as it was for Postgres back when the NoSQL movement happened. Mongo wasn't made of magic scaling fairies and still needed resources. As soon as you cranked up the safety of Mongo to a nearly acceptable level it's performance fell through the floor. The only reason people though it performed amazingly was because of Mongo's rather deceptive marketing.
I use the term "nearly acceptable" because for a long time Mongo's data consistency was absolutely crap (https://jepsen.io/analyses)
Personally I think people used Mongo because it was new, shiny and exciting and it promised performance (though it didn't really deliver for most people).
I am certainly not going to defend mongo ever having being a good choice; I was referring to other "NoSQL" databases (namely, the large number of highly scalable key-value stores).
But you wouldn’t use a key value data store as your main store. It’s a specialized kind of store. Of course Redis and friends have their place. Just not as a main store.
In addition to changes in ephemeral and persistent memory, the other big difference between now and nosql’s heyday is improvements in distributed relational transactional databases (newsql).
We haven’t exactly circled the square on CAP but we’ve certainly bent it.
I've nothing against Postgres but MySQL is a wonderful option that is likely still in far higher and heavier use, when considering open source databases.
I wrote this in another comment, but it's relevant here I think:
" I do scaling and performance work, mostly with Rails apps, but a significant amount of the work is database level and not language specific. I've used both postgres and MySQL (and a few other databases) going back to 2000.
The best thing I can hear from a company when I start is "We use Postgres". If they're using postgres then I know there's likely a far smoother path to performance than with MySQL. It has better tooling, better features, better metadata. "
Right now I would not choose MySQL over Postgres at all, ever. I can't think of a single way it is materially better.
In my mind, MySQL no longer exists. It’s a weird Oracle thing that I wouldn’t touch with a lineman’s pole. MariaDB is what happened after MySQL disappeared.
I think it gained popularity over a decade ago, when it was faster because it was unsafe.
In the end postgres is the better one. With mysql the defaults are bad.
In my experience MySql is more popular. Worked at several companies that used MySql, none that used Postgres.
Kids at universities still use XAMMP to learn databases.
In my experience MySQL is still commonly used in the PHP world, but everything else is mostly Postgres
In my opinion the use case is when your data no longer fits on a single, potentially very large, server. Automatic distribution and replication and rebalancing of data is a tricky problem and something like Cassandra handles those very well.
Or Cockroach DB or similar, if you still need relational capabilities.
Generally with these migrations biggest problem in my experience is that all the operational learnings are not transferrable.
You often want to create expertise and achieve operational excellence with the tech that you start out.
Back in the day we used to say "you choose your database once". You can swap in/out a lot of things, but your data store is really the core.
It is shockingly hard to change your storage engine once you are serving customers, so you really want to get that right.
Exactly, languages and frameworks come and go - but get the data layer right and you’re set.
Are transactions still limited to a single document in Mongo?
Mongo has had multi-document transactions since 2018 (version 4.0), and joins (the $lookup aggregation pipeline stage) since 2015 (version 3.2) [1].
[1] https://www.mongodb.com/evolved#mdbfourzero