Worked on two GraphQL projects; I was quickly cured from the hype. I recognize a lot of points in this article.
In both these projects the GraphQL had started small. I came in during a more mature phase of these projects (2 and 4 years). That's where the requirements are harder, more specific, and overall complexity has grown. Adoption and demand on the API were growing quickly. Hence you logically spend more time debugging, this is true for any codebase.
But GraphQL has everything in it to make such problems even harder. And both these projects had clear signs of "learning-on-the-go" with loads of bad practices (especially for the N+1 problem). Issue descriptions were much vaguer, harder to find in logs and performance issues popped up in the most random places (code that had been running and untouched for ages).
Fun fact; in both these projects the original devs who set it up were no longer involved. Probably spreading their evangalism further elsewhere.
RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
REST API's are a proven solution for the problem of other apps, including front-ends, needing data from a data store. Using JSON is much improved over the days of XML and SOAP. Beyond that there haven't been advancements in technology that cause fundamental shifts in that problem space. There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.
There are so many developers out there that can't stand plugging in proven solutions to problems and just dealing with the trade-offs or minor inconveniences. Nothing is going to be perfect and most likely a lot of the software we write will cease to be running in a decade.
REST APIS suck for nested resources. GraphQL is a huge breakthrough in managing them.
Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.
With internal REST for companies I have seen so many single page specific endpoints. Gross.
You could argue almost any tech solution in a non-pure tech play is largely in consequentially as long as the end goal of the org is met, but managing REST APIS were a huge point of friction at past companies.
Either it goes through a long review process to make sure things are structured "right" (ie lots of opinions that nobody can sync on) or people just throw up rest endpoints willynilly until you have no idea what to use.
GraphQL is essentially the "Black" for Python Syntax but for Web APIs. Ever seen engineers fight over spaces vs tabs, 8 vs 4 spaces, whether a space before a colon? those fights happened a lot and then `black` came out and standardized it so there was nothing to fight over.
GraphqL makes things very clear and standard, but can't please everyone.
Unpopular opinion: I'm actually a fan of singe page specific endpoints. You get much easier debugging, easier to audit security, easier performance optimization an the imho pretty small price to pay is that it's "not elegant" and a bit of backend code
The modern web development practices are just insane.
The GP's idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.
On the other extreme, we have the HTTP 1.0 developers saying something like "networks are plenty of fast, we can waste a bit of it with legible protocols that are easier to make correct", while the HTTP 2.0 ones are all in "we must cram information into every single bit!"
Every place you look, things are completely bananas.
For me, what's crazy is that there are "web" developers who can't just add the endpoint they need while working on a frontend feature, or "web" developers who can't just add an element or a page for testing the backend endpoint.
What ever happened to full-stack developers? The "frontend" and "backend" developer split is so incredibly inefficient that it's really jarring—you take something that should take 2 hours and magically, through tickets, delegation, and waiting for results (then repeat that for debugging, who knows how many times!), make it into a 2-3 day task.
I once reproduced (black-box style!) a two-week effort by a three-man team in six hours of work simply because I had PyCharm and IDEA opened side-by-side and could write code on both sides at the same time.
If someone has a good explanation for why the full-stacks that were once the norm went almost extinct, I'd be happy to give it a read!
For context: I work on large-scale browser apps that are closer in complexity to something like Linear or Obsidian than to your standard WordPress blog with some forms. E.g I'm currently working on a browser-based GIS tool for the financial sector.
I started my career as a full-stack developer, but went all-in on frontend because I felt I was spreading myself too thin. At one point I found that I could choose to be almost good enough at doing two different things or extremely good at one thing. I chose the latter option.
Modern browser apps are complex beasts, at least if you want to do them right. You obviously have to worry about all the technical bits --- HTML, CSS, JavaScript, your view library of choice, platform APIs like <canvas> and WebAudio, cross browser testing, bundle sizes, performance optimizations, techniques like optimistic rendering, all that good stuff.
On top of that, you also need to work closely with designers to make sure they know the features and limitations of the platform(s) they're designing for. More often than not, you end up being a sort of bridge between the backend devs, designers, and product managers.
A lot of times you end up doing design too, whether you like it or not. I've learned a lot about UI/UX design just because I often have to fill in the gaps where a designer forgot to include a certain error state, or didn't test their design on tablet screens, or didn't account for cases where a certain API might not be available.
I tried for many years to learn as much as I could about Django as well as React and friends. But it eventually got too much. I found that I wasn't able to keep up with both ecosystems, and I was producing code that wasn't very good. I could certainly build things quickly because I was familiar with all parts of the stack, but it came at the cost of code quality, security, stability, and robustness. I eventually decided to hang up my backend developer hat and focus exclusively on what goes on inside the browser (which can be a lot by itself these days!)
It's probably possible for a single individual to build a high-quality server-rendered MPA with some forms without making a mess of it. But that says more about how good Rails/Django/Laravel are than about the capabilities of any single individual. I don't think a single person could build a product like Linear end-to-end without cutting corners.
IMO the fact that being a full-stack dev is so taxing is an indication that the stack as a whole is just way too complex and overengineered. Which is rather obvious from looking at the state of affairs on the frontend side of things. Desktop GUI devs don't usually have those problems.
I don’t think this is a fair conclusion; web development is harder than desktop GUI development for various reasons.
For one, clients (mobile, desktop) are drastically different with all sorts of downstream implications:
- Differing screen size requiring responsive design
- Internet speeds magnitudes different
- Intermittent internet
- Uneven feature support due to different versions of browsers
- Everything needs to be JS at the end of the day
Desktop apps generally don’t have to worry about any of these issues.
Also, desktop GUI frameworks are typically either fragmented (one per OS) or don’t look OS-native.
I don't think the claim is that a single developer should be able to build an entire enterprise product, but rather that a single developer should be able to implement the software side of a task end-to-end. The latter is a reasonable expectation when your software is monolithic.
Cutting corners is a feature. I bet the Linear team is as pained as any, internally, at the tradeoffs they're making.
There is no way to know "what to get right" without going through it. So for 80% of the dev cycle the job is to cut corners to get to the 80/20, rinse and repeat forever.
This isn't against excellence and the dream of a beautiful product experience as your reply seems to convey.
Because people have a limited appetite for complexity.
I wrote some front-end stuff back in the days but I've lost track of whatever is happening these days. jQuery to append some stuff took five minutes to learn, but learning react hooks takes a determined effort.
Likewise, adding a field to a graphql type is simple, but doing it with authorization, controlling n+1s, adding tests etc.. requires front-end folks to actually invest time in learning whatever back-end they're dealing with this time.
Everything is just a lot more complicated these days, and if you've been around for a while you may not be excited anymore by the churn, but rather fed up.
This is why the Rails community should be applauded in my book, for their dogged determination that we should keep it a “one person” framework. Yes it may not be as performant, type safe or flashy on the front end but my god it’s productive.
At my startup there are 7 devs who can all do tickets across the stack and as we grow I think it would be good if we could resist the pressure to silo and specialize
It really is. Regrettably, I've drifted away from it in large part because of client requirements for more "modern" and "maintainable" solutions (e.g. Python or Node; I'll take Python every time, thanks). Django comes very close in terms of productivity (and is better in some ways: auth, admin, etc.) but the Rails CLI, generators and community (not sure if this is still relevant) give it the edge.
The "recent" (last 10+ years) movement towards "lightweight" libraries (instead of frameworks) that require you to either reinvent the wheel, copy-and-paste or use some random "getting-started" template every time you start a new project is disheartening. As others have said above, I think it's partially resume-driven-development and people wanting to tinker (both of which I do appreciate).
Something which continues to surprise me is that there hasn't really been a "modern" successor to Rails. Granted, I haven't kept pace with developments in the Node/TypeScript world but, last time I looked, Sails was on the right track but wasn't very actively developed and I was shot down (in favor of Express spaghetti) when I suggested it. There's also a smattering of Rust web frameworks but none that quite fit the bill and come with all of the batteries and opinions included. I keep saying I'm going to do a Summer of Code project which attempts this but ... life.
If you absolutely need 2 people for that, they should be side by side.
But yes, that break-down of the problem is insane. People artificially split an atomic problem in two, and go create all of that extra craziness to try to solve the communication problem they made.
And then people go and push UX and UI tasks into the frontend developer, and ops tasks on the backend since them are there... What is insane again, because those are really incompatible tasks that can't be done with the same mindset.
And since it's standard, backend tools won't support the frontend and the other way around. So the insanity gets frozen on our tooling.
Microservice architectures and the associated explosion in complexity on both ends are to blame. When it takes twice the time to build something, it is natural to hire twice as many developers. Increased complexity drives specialization.
Many organizations would rather pay 3 people $120,000 each instead of paying 1 person $300,000 to do the same work, for a variety of reasons. Some good, some bad.
Yup. Especially when your api really only has 1 (possibly 2 in the case of a mobile app) real clients. People like to pretend their apis are generic but they aren’t. There’s a good argument to stop writing generic apis for your single application.
Typically the endpoints clients are multiple frontend developers. If the frontend is blocked for every page they need waiting on the backend to expose data that massively increases the costs of features and reduces the time for delivery.
This happens anyway.
Not nearly as often with Graphql and happens less and less as your backend and data models stabilizes.
Most of our frontend features now don't have backend changes and we were able to increase the ratio of frontend to backend devs.
My experience is that the problem is avoided in theory, but not in practice.
Making a good API on a large system with many clients is always difficult. GraphQL makes it easier in theory, but if you have average level devs working on it, they’ll make a bigger mess than if they use simple REST. The latter will still be complex, but at least it’s easier to have observability.
Aka never underestimate the power of additional complexity to drive bad use, absent familiarity.
This sounds like more of an org chart problem or a value chain management problem than a technical problem.
That's why we have Conway's law.
Yeah. And a gentle nudge that Conway's Law is about lines of communication in general, not specifically the org chart.
I used to be in charge of the stored procedures that served as the API to a shared database system used by many teams. (So, not exactly the same thing, but I'd argue that the sprocs vs ORM debate isn't entirely dissimilar from the REST/RPC vs GraphQL debate.) My turnaround time for getting database API changes into staging for application teams to play with was typically less than one work day. But that happened because I had a lot of latitude to just get things done, because the company valued rapid delivery, and therefore made sure we had what we needed to deliver things rapidly, both from a technical and a business process perspective. This was in a very highly regulated industry where every change required paperwork that would be audited by regulators, too.
I've also worked at places where this sort of thing took ages. Typically the worst places for work backing up were organizations that used Scrum, because the whole "sprints" thing meant that the _minimum_ time to get something from another team was about 1.5 times the sprint duration. And even then you could only manage that if you could deliver a request that already met all of that particular team's requirements for being able to point it, and could convince their Product Owner that it was a higher priority than whatever their pet project is this quarter, and the stars align so that you perfectly time the submission of your request with respect to that team's Scrum process's frame rule.
The thing I want to point out here is that absolutely none of that bullshit was ever the API technology's fault.
Agreed. Specific endpoints. I was on a project recently where a 40+ yr old dev was going crazy over "pure REST." It's embarrassing. Sure, have your pure REST but FFS create specific endpoints if you they make sense.
You are literally Transferring the REpresented State of a given page/view/screen.
Screen-specific REST endpoints will make their way as a default in to a JS-based framework in 2025 and people will pretend like this is some breakthrough advancement.
And finally HyperText will progress into the futuristic HyperCard
GraphQL gives you this. https://relay.dev/docs/guides/persisted-queries/
I think it's elegant. I loved MVVM in the postback world, and with SPAs, I see view-specific endpoints as the view-model continuing to reside on the server while the view was split off to the client-side.
This is the true purpose of REST. If you need to make multiple requests for a single operation you don't have enough endpoints.
The idea that resources and the underlying data needs to map 1-1 is wrong.
Sounds good in theory. In practice, what I've seen is people don't just do single page endpoints with shared services, they follow this all the way down into the services and so you end up with lots of duplicate looking code. This isn't a problem until the application gets big enough and now you have business logic that should be the same spread out across an app. This leads to weird bugs where something that should work the same way doesn't depending on which page you're accessing things from. Of course, I don't a solution, solving things like this is what makes software engineering hard.
I’ve heard this called the “backend-for-frontend” pattern.
Hardly gross. It is what it is and it’s universal across the domain. I bet Windows has internal APIs or even external ones that were created just for one page/widget/dialog of one app. It’s the nature of things at times.
Its gross because it is a waste.
An engineer had to spend time to make that specific API for that page instead of the frontend consumer using what was already defined and get all the resources with one call and 0 backend engineer needed for that new page.
That backend engineer wrote a single SQL query that joined some tables, ensured indices were used, and always executed in <1ms.
In graphql land you'd be doing multiple SQL queries, "joining" in the API layer, and spending 50ms per API call.
The entire point of graphql servers is that they're basically ORMs (or use an underlying ORM) that turn complex nesting into a single query's worth of joins. It won't beat hand-crafted sql from an expert, but if that's your preferred approach, debates on the relative merits of different query frameworks are all academic to you anyway.
I've never seen this kind of graphql server implementation that can automatically boil down a complex nested query to sensible SQL. It sounds like the best of both worlds. Do you have links?
Typically libraries use a Dataloader + ORM to implement this which gets you pretty far, outside of that some libs like Strawberry with automatically optimize your queries for you if you return Django ORM Querysets.
Any query you were going to build and serve with Rest can be made with these two methods or even a raw dataloader and manual SQL string.
The inventors of GraphQL did not intend it to be mapped directly to a database.
"That would be one way to implement the system in a DB-centric way. However we believe that intermediate application code is pretty critical to any GraphQL implementation." [1]
"GraphQL is a client-server dance that needs server-side capabilities, not just CRUD semantics." [2]
[1] https://news.ycombinator.com/item?id=9879870
[2] https://news.ycombinator.com/item?id=14351800
The Language Libs like Strawberry implement what he is describing by intermediate application code. It doesn't map directly to a database.
GQL implementations don't map to the database but to application code.
Specifically, GQL implementations always map to a `resolve(source, args, context, info)` function (the names can be anything, the types are what matter). In that sense, you also get a standard server interface similar to wsgi/psgi/rack, but much more fine-grained (there can be a resolver for every field in the query).
Symfony API Platform boils to ORM-generated SQL via Doctrine, which is verbose, but not overly clever. So the only link I could give you there would be to the Doctrine query builder docs (I won't subject you to API Platform's docs). I imagine a more sophisticated graphql dialect like what Prisma supports can generate some pretty gnarly sql, but that's ORMs for you. But exposing your data model directly through graphql is also not recommended, same with REST APIs (I can't claim the high ground, my biggest graphql project exposes models directly. The tradeoff was worth it in this case). So in the end you're writing custom controllers/resolvers anyway, and field resolvers are where graphql really starts to shine.
What? GraphQL is purpose built to solve that in 1 Query. Not doing it in 1 query is on you not the protocol.
In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.
A lot of graphql implementation end up moving the n+1 problem to the query resolver.
Every GQL implementation I have seen explicitly has a way to avoid n+1 queries.
1 graphql query maybe. But that translated to a dozen SQL queries.
The point you're missing is that for 1 graphql query the API did N+1 SQL queries, and then also joined them in JavaScript.
In the REST case the front end can switch to the efficient custom endpoint when it is implemented. In the graphql case it will never get any faster because the API has to stay generic.
This is like, an hour per endpoint. For maybe 30 endpoints (high figure for many apps) throughout the lifespan of your application. But let's say 3 hours: you're talking about 90 hours in total, over a period of 2 or more years. It's really not that much.
And GraphQL isn't free either; you need to actually implement that. It pervades your entire stack, too – it's not something you can just "put on top of it".
I think that in many cases, perhaps even most, GraphQL is a "spend 8 hours to automate a half hour task" kind of affair. Are there cases where that's not the case? Probably. Maybe your specific use case is one of them. But for the general case? I'm entirely unconvinced.
An experience engineer knows that a change that takes hour turns to days when a team is running at scale and needs to have CI approvals etc.
Why waste any time?
Rest isn't free. You have to actually implement also that and end up with a more limited API.
GraphQL libs in Python are equally as complex as the FastAPI etc..
And no one will be spending days working on it, they will work on other things while waiting for code reviews and such.
You need to account for the fact that anything can query anything, whereas a more traditional REST-type API will have much more limited codepaths.
This starts with just basic database maintenance and indexing, which in many case will be significantly more complex, to all sorts of other problems. This article already enumerates them so I'm not going to repeat them here.
You can't just handwave all of that away with "oh there's a library you can use".
If you think all of that is worth it then that's fine. As with many things it's a trade-off, and a bit of a judgement call too. But please, don't pretend this complexity doesn't exist, and that the trade-off doesn't exist.
Is the flexibility worth the tradeoffs? Maybe in a company where you are adding new pages all the time with deeply nested relational data needs. But I would argue this is more rare than not. And I often find that frontend engineers aren’t as familiar with database queries and the load they are putting on the system with some of the graphql queries they are making. Flexibility for frontend has its own tradeoffs and I totally understand why a frontend engineer doesn’t want to have to wait for an endpoint to be finished. But this article outlines some of the issues you encounter later as you scale your team and system.
We use a schema first design where I am at and if a frontend person needs a new endpoint because the resource-only endpoints aren’t enough then they submit a pull request to the schema repo with a design for their endpoint they need. It gets approved and boilerplate is auto generated. Yes you have to wait longer, but 90% of the time (for our software) the resource endpoints work great.
Sorry what are the tradeoffs?
I'm not sure where this narrative comes from that GraphQL immediately means that the frontend time will have no idea what they are doing and will induce load on the system.
95% of my nested graphql fields are based on foreign key indexes so its almost no additional load to the system (only 1 query per "level" of GraphQL) to query the nested objects.
I restrict GraphQL to 5 levels and now I have 5 queries per API call instead of 5 Queries over 5 API calls.
The backend team exposes the fields that they know are efficient to query. They can restrict the depth of the GraphQL, number of fields, etc.
Every page is special. Using generalized API is waste of resources. Generalized APIs are hard to use and develop. It takes a genius to create a good generalized API (say SQL) and even those generalized APIs are not usable in web context (because of authorization and DDoS issues).
It's better to think about optimizing of creation specialized APIs.
On the other hand, it may take that engineer less time to create a page-specific endpoint than it would be to create something more generic that might serve other purposes (that may never come to pass), which may also involve talking to other teams, checking what future plans are, etc.
And that's assuming it's a new endpoint; if there's an existing endpoint that does almost what's necessary, they may need to check in with that team about what modifications to the endpoint would be acceptable, etc.
Single-page endpoints aren't great, but often times they're acceptable because they end up being a half-day task instead of a week-long slog.
It's analogous to a specific method in code. No idea why people go nuts over this.
A REST endpoint can be analogous to a specific method in code. just as much as a GraphQL field.
What people are excited about is that the frontend can request all the data it needs at once and the backend can efficiently serve it. Something not possible with REST without reimplementing something similar to GraphQL.
It's definitely possible, just not without tight coordination between frontend and backend. When your development teams are broken into silos with strong separation, this isn't feasible and GraphQL starts to make some sense. Otherwise, you just create an ad hoc endpoint that serves the exact data you need and call it a day.
I suspect that people don't think about it too hard and presume that they should have only one model - that is that their REST model should map directly to their DB model which should map directly to their internal domain model (the idea that there might be two-or-more-domains and not-all-of-them-are-public hasn't even yet crossed their minds).
When you realize that sometimes your REST model may be a view-model that maps to a different storage model optimized for online-transaction-processing, neither of which map to your internal domain model directly (there are adapters for that) _then_ you get somewhere ... but of course you then have to fight off the `transformToRestModel(transformToDomainModel(retrieveDbModel(theId)))` when the three models happily coincide for a time.
Some people treat codebases as an art project rather than engineering project.
I think this explains about three quarters of all engineering problems I've seen in corporate context, give or take.
Single page endpoints is exactly what you want if you have more than 5 engineers in your company anyways.
It ensures that the endpoints are maintainable and future-proof when people are working on different features.
How does GQL prohibit this? It encourages it by focusing on 1 stable API for everyone instead of a custom API endpoint for each case.
It doesn't. That's literally what GraphQL was designed for: To provide a resolver that sits between high latency clients and all of the various REST/RPC services in order to roll up all of the requests at datacenter speeds into one in order to overcome the latency issues.
But you still need all of the various single page REST/RPC endpoints to make use of GraphQL as it is intended. Some developers out there skip the REST/RPC part, making GraphQL their entire service, violating its principles.
If it works it works, but it does not come tradeoff free.
? why do you need a REST endpoint with GraphQL? Nearly every language has a GraphQL engine that integrated directly with the database and exposed functions as GQL directly without REST at all.
You don't, obviously – we already talked about how many don't. But if you don't have a graph of "microservices" in which to query, what do you need GraphQL for? Especially if all you are doing is returning the results from a database. The database will already have its own API. Why not just use it? The native API is always going be better than your attempt to translate.
Because exposing your Database to the outside world is asinine. GQL sits between letting the frontend query semi-customizable queries and not having any customizability to select related resources.
Nobody said you should expose your database to the outside world. Do you not understand the difference between APIs and services?
You said
Where is this API coming from? You have to build it. I'm saying you should make those APIs graphql apis though a language framework and not REST apis because GQL consuming GQL is much better than GQL consuming REST.
Someone has to build it, but as long as you are using a popular DMBS the work is no doubt already done. All you need to do is stick your application-y bits in the middle – same as you have to do with GraphQL. Computers aren't exactly magic. It's all just 1s and 0s at the end of the day. You can do just about anything and make it work. But there are tradeoffs to different approaches. If GraphQL is the best choice for working with databases, why don't DMBSes use it natively?
I'm curious as to why you believe that exposing direct SQL over the database is "asinine" but GQL is fine, given that either one is very generic, and e.g. the security issues are very similar (as the article points out).
I don't think it would hold true for very long, devs will have specific cases which will pile into the definitions.
That's why GraphQL examples usually focus on querying data from users and not how you are going to manage 10 different views of the same data.
That is how REST works but is the opposite of the way GQL works.
You don't pile into existing defintions but extend the current definitions out. A team needing new data doesn't affect the consumption of other teams. which is not the case of REST where if one team needs to change a REST endpoint to return different shape of data, they have to verify the new shape with other teams.
With REST, the endpoints are team based if you have even a semi-competent CTO so you never have this problem, you just check who owns the controller and that's it.
With GQL though, good luck retracing who owns what field and what does it affect if you change the definition of what it returns, especially that you you are not even using it on the same language.
Bonus points here if you are managing something outside of your control like a mobile app.
GQL you would only care about if someone removed a field not if someone added a field. How would adding a field change existing GQL calls return? Doesn't make sense.
Also, Its about 1 line to set up CI to extract all GQL queries from a typescript project and do static analysis to check against the schema.
But again you only care if someone deletes a field, and even if you have to delete it, at least the GQL schema has built in annotations for deprecation, something not in REST.
Deleting things happens all the time though.
Yeah sure you can make it work with anything if you spend the extra effort but the ownership really isn't as defined as in REST.
Is there code which have fuzzy or no ownership? Are there changes which affect other teams? Suddenly those became much harder questions to answer.
Actually deleting happens rather rarely compared to adding.
Its much easier to trace and debug what teams are using GQL fields than REST. What if one team is piggy backing on another teams exising rest endpoint and you dont know? same problem that would require some analysis of all code calling and endpoint to determine if a field is safe to delete. GQL makes this much simpler than REST.
On one side you need to check which folder you are in, on the other side you need to do a static analysis of the whole current and past apps.
I know there's some diverging opinions here but there's one which sounds definitely easier than the other.
As for deletes, I work in a company with a good hundred devs so that happens weekly at least.
The canonical REST solution of query params to add nested fields gets you quite far:
GET /myresource?extra=foo,bar
sure you over fetch a bit if you have multiple accessors.
But agreed, if you have highly nested data especially when accessed with multiple different query purposes then REST might not be the best fit.
I think GraphQL has been positioned as a general purpose tool and for that I am with the author, REST is a better go-to for most usecases.
I guess depending on the context of a simple app one level could be viewed as far.
Any more levels and you have now reinvented GraphQL
OData existed before GraphQL in the wild, it's possible to suggest GraphQL reinvented OData.
https://www.odata.org/
The point is GQL is currently the standard for querying dynamic nested data shapes not that it is the first or was the first.
Look at OData download stats on Pypi it had 2 downlaods the last day. Graphql-core for python? 624,201. that is not even on the same planet.
If you don't use GQL and want a system of querying nested data you will be using a less used protocol that is analogous to GQL so you might as well use the standard.
OData is still currently a standard for querying (among other things) nested data shapes. The point was it is a matter of fact that as a standard it predates GraphQL and as a matter of ecosystem perspective whether or not you think it is a more or less used protocol.
To trade anecdotes for anecdotes: On Nuget, the Microsoft.Data.OData package marked deprecated and marked with CVEs for critical bugs still has an average of 36.3K per day downloads. (Its successors combine for about double that.) The top GraphQL library (of the same name, unadorned) still only has 9.3K per day downloads. In .NET if you want a system of querying nested data (that is also "REST compatible"), why would you use a less used protocol like GraphQL?
Sure, but reinventing the wheel can be good, particularly when the existing wheel technology is oblong, lumpy and constructed of stone and cheese.
I’m just bitter than GraphQL endpoints return 200 on errors. If you’re returning 200 on errors, then you’re not really doing HTTP; you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.
Browser does not allow to access any ports with any protocols.
Often intermediate software gets in the way as well. For example with Kubernetes it's trivial to expose HTTP service using Ingress and it's not trivial to expose other protocols.
Another HTTP plus is that it's trivially secured by TLS and every developer knows API for that. Using TLS for your own protocol or QUIC is absolutely not trivial and probably more like arcane knowledge.
One place I was at we used REST with a hydration service that ran as a proxy in front of the requests. That gave us most of the benefits of GraphQL and we only implemented it for the main resources (users, photos, and maybe one other thing). To minimize latency/faults the hydration service ran locally on every webapp/API server. I wasn't around for far too long after though to see how it turned out as it grew (if it did at all, the company sort-of went defunct--development-wise anyway).
I also recall, we had similar N+1 query problems in the REST API endpoints irrespective of hydrating the returned resources.
The biggest benefit of GraphQL I can see from a user perspective is that it lowers total latency especially on mobile with fewer round trips.
you don't think it's a benefit that you could get the benefits of a "hydration service that ran as a proxy in front of the requests" out of the box?
there's lots of other benefits for GQL: multiple queries per request, mutation/query separation, typed errors, subscriptions support.
That's an established pattern (backend for frontend). Like all patterns there are trade-offs and considerations to make, but it's certainly not a priori "gross".
https://learn.microsoft.com/en-us/azure/architecture/pattern...
If an app's API requires more than a couple of nested structures then, without knowing the specifics of the domain, I would venture a guess at a poorly defined data model is the issue vs. API technology.
For example, a lot of times people build out nice normalized table structures for online transactional apps. The UI/UX is pretty straight forward because end users typically only CRUD an object maybe a couple nested objects at a time. The API's are straight forward as well, likely following single responsibility principles, etc. Then comes along requirements to build UI's for analytics and/or reporting types of things where nearly the entire schema is needed depending on what the end user wants to do. Its the wrong data model for doing those types of things. What should be done is ETL the data from the OLTP schema into a data warehouse style schema where data is de-normalized so that you can build reporting, etc.
As someone pointed out in reply to another comment, GraphQL is "a technological solution to an organizational problem." If that problem manifests as abuse of REST endpoints, you can disguise it with GraphQL, until one day you find out your API calls are slow for more obscure, harder to debug reasons.
With REST, though, that pain is visible to both sides. Frontend engineers generally don't want to make N+1 REST calls in a tight loop; it's a performance problem that they see and that is very visible in their Dev Tools. Backend engineers with good telemetry may not know why they get the bursts of N+1 calls that they see without asking the Frontend or digging into Frontend code, but they can still see the burstiness of the calls and have some idea that something could be optimized, that something is maybe too chatty.
There are multiple ways with REST to handle things: pagination, "transclusions", hyperlinks, and more. Certainly "single page endpoints" is a way as well, no matter how gross it is from REST theory, it's still a pragmatic solution for many in practice.
REST certainly can please everyone, given pragmatic compromises, even if it isn't very clear or standard.
JSON winning over XML is like saying CSV won over MySQL. They aren't equivalent.
Much like CSV, JSON isn't particularly standardised and different parsers and writers will do different things in some situations. Usually it doesn't matter, but when it does you're probably in for a lot of pain.
If you handle structured data and the structures might change over time, JSON isn't a good fit. Maybe you'll opt for JSON Schema, maybe that'll work for your use case, but with XML you can be quite sure it'll be reliable and well understood by generations of developers.
The tooling is generally very good, commonly you can just point your programming language to the XSD and suddenly you have statically typed classes to program against. Perhaps you'd like to store the data in a RDBMS? You can probably generate the DB schema from the XSD. If you want you can just throw JSON into MongoDB instead, but there will be very important tradeoffs. Same goes for UI, you can write some XSLT based on the XML schema and suddenly you get web views directly from API responses. Or you could use those classes you generated and have your GUI code consume such objects.
None of this is as easy with JSON as it is with XML, similar to how many things aren't as easy with CSV as with a RDBMS.
What's missing in ECMA-404? Never had a problem with JSON parsers or writers, using it all day every day for decades. It's crappy in some ways, sure, like lack of full floating point support, but standardization is not an issue.
XML is mostly already lost on the current generation of developers though, much less future developers. Protobuf and cousins generally do typed interchange more efficiently with less complexity.
There are a lot of proponents that some or all of the "JSON5" [1] improvements should be standardized by ECMA as well. Especially because there is a mish-mash of support for such things in some but not all parsers. (Writers are a different matter.) Primarily comments and trailing commas, are huge wish list items and the biggest reasons for all of the other "many" variant parsers (JSONC, etc).
[1] https://json5.org/
This is more of a concern for JSON configs and other such uses that are directly exposed to humans, but not really for machine-generated and machine-consumed data.
It's focused almost entirely on syntax and ignores semantics. For example, for numbers, all it says is that they are base-10 decimal floating point, but says nothing about permissible ranges or precision. It does not tell you that, for example, passing 64-bit numbers in that manner is generally a bad idea because most parsers will treat them as IEEE doubles, so large values will lose precision. Ditto for any situation where you need the decimal fraction part to be precise (e.g. money).
RFC 8259 is marginally better in that it at least acknowledges these problems:
But note how this is still not actually guaranteeing anything. What it says is that implementations can set arbitrary limits on range and precision, and then points out that de facto this often means 64-bit floating point, so you should, at the very least, not assume anything better. But even if you only assume that, the spec doesn't promise interoperability.In practice the only reliable way to handle any numbers in JSON is to use strings for them, because that way the parser will deliver them unchanged to the API client, which can then make informed (hopefully...) choices on how to parse them based on schema and other docs.
OTOH in XML without a schema everything is a string already, and in XML with a schema (which can be inline via xsi:type) you can describe valid numbers with considerable precision, e.g.: https://www.w3.org/TR/xmlschema-2/#decimal
Frankly, if a developer can't figure out XML then they aren't worth their salary. Age is no excuse here; as a developer your job involves figuring out how to work with technology you haven't used before.
Its not JSON over XML. They're saying JSON REST won over _XML and SOAP_.
Those two are kinda orthogonal, and while there was some overlap for adoption, it was fairly common to serve XML over REST early on (because more languages and frameworks had proven-quality XML parsers out of the box, so it was easier for the clients to handle).
JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend. Then other languages caught up with their own implementations, although in some cases it took a while - e.g. for .NET you had to use third-party libraries until 2019.
To claim that pretty much every .NET project adding Newtonsoft.JSON as a first step was somehow a problem is just strange. No adequate team would claim this to be a problem.
XML parsers suffer the same fragmentation issues that JSON parsers do.
Go's XML parser straight-up emits broken XML when trying to output tags that have prefixed namespaces.
Harder to get a promotion when you’re doing something that’s old, boring, and just works.
Especially if your job title has "architect" in it.
The closer you are to the cutting edge, the more sliced up you’re gonna get.
I think us engineers have an inherent desire to try to innovate, and I think that is a good thing. Some problems will require a lot of wrong turns before finding the right path, but that is simply the nature of innovation
Agreed it's a good thing. Writing software today is still a slog that turns into a spaghetti bowl without focused intent by highly skilled devs. I think most devs realize this and really want there to be something better, and the only way to find something good is to find a lot of the bad around it first.
I like the thought experiment of adding a new persisted/editable field to an entity in a web app, if you've done full-stack you know all the layers that need to be touched to accomplish this and how lame most of the work turns out to be. After doing that 20 times while iterating, any dev worth their salt starts to wonder why it sucks so bad still and how it could be made better, and some will actually try.
I think IDLs and gRPC in particular are a meaningful advancement in the problem space. The ecosystem and productivity that they enable via programatic tooling were really eye-opening for me (The OpenAPI ecosystem is, imo, extremely poor). They also have better support for modern techniques like streaming and even load-balanced streaming.
XML still has its usage and is excellent to quickly validate documents, but JSON clearly won in this use case because it was kept simple, stupid.
If only that were true in my experience.
GraphQL is very good for places where frontend and backend developers are isolated from each other (separate teams). Or rather places where you have data-producers and data-consumers as separate teams. If you have a big enough org eventually there will be many of such teams, interdisciplinary teams are not feasible at scale for everything.
It allows teams to work with less communication overhead. It solves more of a human problems than a technical problems, if someone think there is no value in that and bare metal performance is paramount that person never worked in a big org.
In some ways yes, in others no. For example it can be near impossible to see if a deprecated field in a REST API is still being used and by which clients it is being used. With GraphQL this is fairly simple.
Unfortunately GraphQL way of working is very different from normal REST APIs and often requires more complex server-side caching. The N+1 problem needs to be figured out upfront for every data-storage system used in the backend.
The problem is you delegate a lot of the query building to the client, hoping that it will not suddenly change your performance profile by being creative and that you will have not missed an obviously expensive use case coming.
That's a huge bet, especially given that GraphQL is expensive in the first place, and given that the more you grow the API in size, the less you can actually map the cartesian product of all request params.
I'm not sure this is any more or less of a problem for REST APIs. What if your engineers change $client/$server and the new version makes really expensive queries? Well, ask them not to do that, then when some of them inevitably ignore you, start to review their code, terminate long-running queries, batch or pool fanouts so they don't take anything down, monitor new releases and roll back if anything breaks, etc.
If you're providing an external API like GitHub does, then that's a different story and I agree.
If you have separation between front and back end, then the back end team can elect to serve REST APIs which only permit selecting, filtering, grouping and pagination that they know they can support within defined latency bounds for a given traffic level.
Thing get more problematic when there's vertical ownership for a feature, where the UI needs just a few extra things and you end up with a REST response which is fatter and fatter, in the interest of avoiding round trips and client-side joins.
The problem with killing correct queries that take too long is that it shows up as intermittent failure that's dependent on server load and data cardinality. You might not find it in testing and it ships a bad experience to the customer before bugs are found. Whereas APIs which can't be so easily misused make it much harder to ship bugs.
Why do you think that they can't do that with GraphQL? GraphQL isn't open ended. Its a highly restricted syntax for calling nested resources. If a resource is expensive simply don't nest it and make it a top level field and it is the same as REST?
Lots of nested resources are by default efficiently served by GraphQL because they are mostly returning single object foreign keys. Something that would take extra calls with REST.
GraphQL can have the same restrictions and performance guarantees as REST but the later is not necessarily true because in REST there is no standard way to define nested resource access.
I think the point here is, if you have to involve a backend team to add restrictions to the graphql endpoints and try to make good educated guesses where those might be, then the idea of frontend not needing backend engineers to query whatever they need becomes less of an advantage. So is the complexity of setting up graphql and then having your backend team try and make sure no frontend engineers can do terrible queries better for the software than custom rest APIs where needed and standard resource APIs everywhere else. Obviously it depends on the project. But I find the complexity of setting up and managing graphql often isn’t worth the pain, especially with schema first resource api designs and tooling like Google’s AIP linter.
No because if you dont do that you have to involve more engineers anyways to build the REST endpoints and keep modifying the rest endpoints.
GraphQL is also default restrictive (i.e. exposes nothing). You don't need to add engineers to make it restrictive.
In Startups typically:
Without Graphql your "backend" ends up needing to have a lot of work and changes because it is constantly needing to be updated to the needs of the most frequent changes happening on the frontend.With GraphQL the only time you need to update the backend is when those "utility" functions change (i.e. 3rd party api calls, etc) or the data model changes.
So you end up needing substantially less backend engineers.
This is akin to saying that "directly exposing the database is easier, you only have to change things if the data changes".
And yes this is true, but when the data changes, or the environment changes, the paradigm falls apart a bit, no? Which is what the backend code was for, insulation from that.
Yes, so for a short lived, non-scaled application its far easier to do it one way, and maybe that's fine for most small apps (that will never scale far). I suspect a lot of the push back comes from larger, less nimble, more backwards-compat focused organizations/apps.
Far from it actually. I am saying that in practice the data and queries that you perform on your Database actually tend to stabilize and you add less and less as time goes on.
By Allowing the frontend to select what combination of these pre-approved queries that you already approved it can use, you have to do less and less backend work when compared to REST where you have to do backend work for every query combination you want to serve.
I mean saying GQL doesn't scale for big apps is over looking one of the largest Corporate Software Orgs (FB) created and use it in production purposefully for managing large software APIs.
Sure, so you are just filtering raw database access then. That doesn't make it any different - and, you still need to approve and filter these queries, so what exactly have you saved? I.e. either the front end engineers can change these filters, or not, so it amounts to the same thing in the case they can.
That's not a great argument, though, saying a large company with many resources is capable of supporting something does not make it a sustainable technical decision. They likely also have a very specific work structure they use to make it for them.
In fact thats a strong reason not to use it, if it requires enterprise level resources to use it effectively. There is a big difference between technologies that scale to enterprise and technologies that require enterprise...
It still comes down to, if you can achieve 99% of the same thing with autogenerated REST apis and a couple page specific apis, what, exactly, is worth the considerable increase in complexity for that remaining 1%? Making things regularly more complex is a hallmark of failed, bad technologies, and I suspect GraphQL will see the dustbin like SOAP did...
You are bouncing back between it is ony for startups and it requires enterprise level maintenance. It can be used easily for both.
Because you can get 100% by autogenerating GQL APIs and 0 page specific apis.
No, I never said that. You are the one that brought FB into the equation.
Just because it can be used for something does not mean that it should.
I said that that approach doesn't scale well, especially for frequent data/model changes. For small apps, where as you say, you have few data changes, by all means embed your database as closely as possible to you end user code.
Sqlite inside a c app or electron, e.g. No need for any API at all! Just raw query access.
Its nice GQL to generate stuff for small non-changing web apps, I'm sure. But once you get into more performance oriented, data-migration-style stuff, if there's not good support for changing the data and reacting to the environment, then adding complexity (GQL) to an already complex situation is a Bad Idea.
You never said what this 1% was, autogeneration is not a bonus when you already have to manually filter and route things. The simpler solution gets you there as well, with less fuss.
You think you don't have page specific apis, but if you are doing the manual filtering, then you still have them, you are just "hiding" them inside another language, that doesn't have a clear benefit? At least you can't say what it is, without going in circles, another sign GQL is probably ultimately a garbage technology...
But you actually don't need to keep modifying the REST endpoints for most projects, that's what everybody is saying.
The vast majority of projects don't gain anything from this flexibility, because you don't have suddenly a 1000 of farmvilles copy cat that need their own little queries. You just have bob that need an order by.
There's a lot of relevant differences between REST & GraphQL. It is possible to construct a REST endpoint that simply can't do any of those things, and such construction is a mid-level developer task at best. For instance, pagination of "all posts ever" is not uncommon, and clients won't be shocked to deal with it. GraphQL is enough harder to characterize the performance of that it definitely qualifies as a change in quantity that is itself a change in quality. Hypothetically, both approaches are vulnerable to all the same issues, but GraphQL is far more vulnerable.
This is Wrong.
GraphQL only exposes what you ask it to. There are plenty of pagination plugins for GraphQL frameworks just as there are plugins to REST framework.
GraphQL can be restrictive as REST if you want it to be.
The point is GraphQL can be "as restrictive" as REST, but if you want to enable more efficient queries by knowing all the data that the frontend is requesting, you can. But the opposite isn't true of REST. With REST if you want more advanced functionality like that you have to define your own specification.
But then what's the point of using it if it's to get the limitation of REST?
You get something more complex, more expensive to maintain, consuming more resources, and configure it to basically be REST with extra steps.
Idk. Strawberry GQL and most GQL libraries are maybe equally as complex as the REST libraries for the same language. Strawberry and FastAPI I would say are equal in complexity and configuration.
It would be hard for me to say GQL is more expensive or consumes more resources. Opposite of the purpose and most uses of GQL.
In stawberry you make a method per field you want to retrieve, I would say it is indeed more complex and costly.
What? Its a method per collection you want to return. or else it is a type annotation. Exactly as complex as FastAPI or any other typed system.
Sorry, what? The original suggestion was that a developer would change things and it would cause performance problems. That same developer can change either a REST system or a GraphQL system and introduce the same performance issues in the same way, probably by adding a horrible N+1 query, or unbounded parallelism, or unbounded anything else.
Yeah, the client can't change the query if you don't let it specify a query, this is trivially true, but the developer can go break an API endpoint with the exact same result while trying to achieve the exact same business outcome.
The much more constrained input of the REST query means that the effect of changes on the API are much more comprehensible. Performance testing a particular REST endpoint is generally practical, and if a dev doesn't do it, the responsibility is reasonably placed on them. GraphQL means that you may do something you think is completely innocent like changing some index but for some query you didn't anticipate it trashes the performance. The range of things the dev of a GraphQL endpoint must be keeping track of is much larger than a REST endpoint, arguably exponentially so (though generally with a low power factor in practice, the possible queries are still exponentially complicated), and taking on any form of exponential responsibility is generally something that you should do only as a last resort, even if you do think your powers will stay low.
Yes, so the cost benefit here is not in favor of GraphQL. If both technologies ultimately suffer from the same issues (what to do about unpredictable clients), but one is far more complex to implement and use (GraphQL), then there's a clear winner. Spoiler, its not GraphQL.
Page specific endpoints, I would argue, can do 99% of what GraphQL was trying to do. If you want to use it as some sort of template language for constructing page specific endpoints, that could be useful (the same way xml schema is useful for specifying complex xml documents).
But you can optimize a page specific endpoint, and do it with REST-style endpoint to boot.
Having a bunch of "simple" calls and optimizing for the "complex" ones that you need using metrics/analysis is what you should be doing, not creating a complex API that is far harder to break down into "simple" cases.
When you build a GraphQL server, you’re creating a system that outputs page-specific endpoints. They can be generated just-in-time (the default) or at build time (the general recommendation).
The engineering work involved shifts from building individual endpoints to building the endpoint factory. This shift may or may not be worth the trade off, but there are definite advantages, especially from the perspective of whomever is building the client. And once you factor in the ease at which you can introduce partial streaming with defer and streamable (granted they’re still WIP spec-wise), the experience can be pretty sublime.
https://graphql.org/blog/2020-12-08-defer-stream/
This? Yeah, that seems neat, for command/batch queuing.
I'd be curious how it compares to e.g. rest apis returning refs to e.g. webrtc streams or tcp/udp ones for non-browser. I presume the main advantage would be client side.
Even a SQL query can suffer the same fate. Ever tried writing a SQL query against a distributed database that isn’t optimized for that read path?
I think that’s what’s really pointing out the root cause issues here, it’s not purely GraphQL’s problem, it’s the problems inherent to distributed systems.
Obviously depends on the API but a REST API that maps relatively cleanly to database queries is going to make it very clear on both the client and the server when it’s not scaling well.
If, at page load, I’m making 100 HTTP requests to fetch 100 assets then as a client side developer I’m going to know that’s bad practise and that we really ought to have some kind of multi-get endpoint. With GraphQL that gets muddy, from the client side I’m not really sure if what I’m writing is going to be a massive performance drag or not.
I haven't done much more than toy projects in GraphQL. Is there no way to limit the query complexity/cost? Such as a statement timeout in postgres?
I'm not that much into GraphQL but I vaguely remember libraries that provide some kind of atteibutes you apply to entities/loaders and then pre-execution an estimated cost is calculated (and aborted if over a specified threshold).
That's sort of my expectation too -- it would be nuts to provide a user facing system without bounds of some sort.
API Platform for PHP is one of those graphql implementations that has a query cost limiter built in (it's binary, it just rejects queries that go over your configured complexity threshold). Shopify's graphql api is even fancier, where every query costs X amount of a limited number of "credits". The structure of gql itself makes costs easier to estimate (you have as many joins as you have bracket pairs, more or less), and some servers can recognize a directive in the schema to declare the "real" cost.
Ah but that's the beauty of GraphQL, a query can actually fetch data from several systems: the db, the cache, the search engine, etc. It's all abstracted away.
But let's say you have a timeout, and they have a retry, then suddenly, your server is now spammed by all the clients retry again and again a query that worked a week ago, but today is too heavy because of a tiny change nobody noticed.
And even if it's not the case, you can now break the client at any time because they decide to use a feature you gave them, but that you actually can't deal with right now.
To be clear, the main thing that's abstracted away are server round-trips and client-side joins. REST APIs can fetch data from different systems too.
Sure but queries are crafted in the client, that may know nothing about this, while a rest api, the requests are restricted and the queries are more likely under control of the server, which mean the backend will likely decide what fetches what and when.
It takes a lot of work to actually ensure all possible combinations of graphql params hit exactly what you want in the backend, and it's easy to mess with it in the frontend.
There’s a free query depth. There’s ways to do query cost but federating that then becomes really annoying. Coming from the security side, there’s always a long curve of explaining and an even longer curve of mitigating.
I always am happy when I get an elegant query working. Often however I just find I wasted time looking for a clean 1 query solution when iteration by caller was the only solution.
When the client’s data requirements change, isn’t there always a risk that the data loading performance profile will change?
Surely that is always the case, if the client is composing multiple REST requests, or if there’s one RPC method per client page, or with any other conceivable data loading scheme.
Couldn't disagree more. GraphQL encourages tight-coupling--the Frontend is allowed to send any possible query to the Backend, and the Backend needs to accommodate all possible permutations indefinitely and with good performance. This leads to far more fingerpointing/inefficiency in the log run, despite whatever illusion of short-term expediency it creates.
It is far better for the Backend to provide Frontend a contract--can do it with OpenAPI/Swagger--here are the endpoints, here are the allowed parameters, here is the response you will get--and we will make sure this narrowly defined scope works 100% of the time!
Can you explain what you mean by this? The GraphQL API you expose allows only a certain schema. Sure, callers can craft a request that is slow because it's asking for too much, but
- Each individual thing available in the request should be no less timely to handle than it would via any other api
- Combining too many things together in a single call isn't a failing of the GraphQL endpoint, it's a failing of the caller; the same way it would be if they made multiple REST calls
Do you have an example of a call to a GraphQL API that would be a problem, that wouldn't be using some other approach?
The problem is that the client team can - without notice - change their query patterns in a way that creates excess load when deployed.
When you use the "REST" / JSON-over-HTTP pattern which was more common in 2010, changes in query patterns necessarily involve the backend team, which means they are aware of the change & have an opportunity to get ahead of any performance impact.
My blocker on ever using GraphQL is generally if you've got enough data to need GraphQL you're hitting a database of some kind... and I do not generally hand direct query access to any client, not even other projects within the same organization, because I've spent far too much time in my life debugging slow queries. If even the author of a system can be surprised by missing indices and other things that cause slow queries, both due to initial design and due to changes to how the database decides to do things as things scale up, how can I offload the responsibility of knowing what queries will and will not complete in a reasonable period of time to the client? They get the power to run anything they want and I get the responsibility of having to make sure it all performs and nobody on either side has a contract for what is what?
I've never gotten a good answer to that question, so I've never even considered GraphQL in such systems where it may have made sense.
I can see it in something big like Jira or GitHub to talk to itself, so the backend & frontend teams can use it to decouple a bit, and then if something goes wrong with the performance they can pick up the pieces together as still effectively one team. But if that crosses a team boundary the communication costs go much higher and I'd rather just go through the usual "let's add this to the API" discussions with a discrete ask rather than "the query we decided to run today is slow, but we may run anything else any time we feel like it and that has to be fast too".
It seems there is a recent trend of using adapters that expose data stores over graphql automatically, which is kind of scary.
The graphql usage I'm used to works more or less the same as REST. You control the schema and the implementation, you control exactly how much data store access is allowed, etc. It's just like REST except the schema syntax is different.
The main advantage of GraphQL IMO is the nice introspection tools that frontend devs can use, i.e. GraphiQL and run queries from that UI. It's like going shopping in a nice supermarket.
Not particularly scary. For something like Hasura, resolvers are opt-in, not opt-out. So that should alleviate some of your concerns off the bat.
For Postgraphile, it leans more heavily on the database, which I prefer. Set up some row-level access policies along with table-level grant/revoke, and security tends to bubble up. There's no getting past a UI or middleware bug to get the data when the database itself is denying access to records. Pretty simple to unit test, and much more resistant to data leakage when the one-off automation script doesn't know the rules.
I also love that the table and column comments bubble up automatically as GraphiQL resolver documentation.
Agreed about the introspection tools. I can send a GraphiQL URL to most junior devs with little to no SQL experience, and they'll get data to their UI with less drama than with Swagger interfaces IMO. (Though Swagger tends to be pretty easy too compared to the bad old days.)
How does this follow? A client team can decide to e.g. put up a cross-sell shelf on a low-traffic page by calling a REST endpoint with tons of details and you have the same problem. I don't see the difference in any of these discussions, the only thing different is the schema syntax (graphql vs. openapi)
GQL is far more flexible wrt what kind of queries it can do (and yes, it can be constrained, but this flexibility is the whole point). Which means that turning a cheap query into an expensive one accidentally is very easy.
A hand-coded REST endpoint will give you a bunch of predefined options for filtering, sorting etc, and the dev who implements it will generally assume that all of those can be used, and write the backing query (and create indices) accordingly.
Then we just come back full round trip to REST where the backend clearly defines what is allowed and what is returned. So using GraphQL it is unnecessary complicated to safeguard against a caller querying for all of the data and then some. For example the caller queries nested structures ad infinitum possibly even triggering a recursive loop that wakes up somebody at 3am.
It sure is better for the backend team, but the client teams will need to have countless meetings begging to establish/change a contract and always being told it will come in the next sprint (or the one after, or in Q3).
It is true it can cause these kind of problems, but they take far, far, far less time than mundane contract agreement conversations. Although having catastrophic failures is usually pretty dire when they do happen, but there are a lot of ways of mitigating as well like good monitoring and staggered deployments.
It is a tradeoff to be sure, there is no silver bullet.
trying to solve org problems with tech just creates more problems, allthewhile not actually solving the original problem.
Yup. And the solution to that org problem is for the front engineers to slow down, and help out the "backend" engineers. The complexity issues faced by the back-end are only getting worse with time, the proper solution is not adding more complexity to the situation, but paying down the technical debt in your organization.
If your front-end engineers end up twiddling their thumbs (no bugs/hotfixes), perhaps there is time (and manpower) to try to design and build a "new" system that can cater to the new(er) needs.
GraphQL is the quintessential move fast and break things technology, I have worked in orgs and know other people who have done so in other orgs where getting time from other teams is really painful. It is usually caused by extreme pressure to deliver things.
What ends up happening is the clients doing work arounds to backend problems which creates even more technical debt
This is what I wanted to say too. If your backend team is incapable of rapidly adding new endpoints for you, they probably are going to create a crappy graphql experience and not solve those problems either. So many frontend engineers on here saying that graphql solves the problem they had with backend engineers not being responsive or slow, but that is an org problem, not a technology problem.
At TableCheck, our frontend engineers started raising backend PRs for simple API stuff. If you use a framework like Rails, once you have the initial API structure sketched out, 80% of enhancements can be done by backend novices.
I never understood why this was such a big deal... "Hey, we need an endpoint to fetch a list of widgets so we can display them." "Okay." Is that so difficult? Maybe the real problem lies in poor planning and poor communication.
A GraphQL schema is a contract though.
And the REST API can still get hammered by the client - they could do an N + 1 query on their side. With GraphQL at least you can optimize this without adding a new endpoint.
Yes, GraphQL is a "contract" in the sense that a blank check is also a "contract".
You can whitelist queries in most systems though. In development mode allow them to run whatever query, and then lock it in to the whitelist for production. If that type of control is necessary.
It's really not, it's not exposing your whole DB or allowing random SQL queries.
GraphQL does this - it's just called the GraphQL "schema". It's not your entire database schema.
Most REST APIs I've seen in the wild just send the entire object back by default because they have no idea which fields the client needs. Most decent REST API implementations do support a field selection syntax in the query string, but it's rare that they'll generate properly typed clients for it. And of course OpenAPI has no concept of field selection, so it won't help you there either.
With my single WordPress project I found that WP GraphQL ran circles around the built-in WP REST API because it didn't try to pull in the dozens of extra custom fields that I didn't need. Not like it's hard to outdo anything built-in to WP tho...
Not to mention the fact that GraphQL allows anyone messing with your API to also execute any query they want. Then you start getting into query whitelisting which adds a lot of complexity.
What do you mean by "backend developer" ? The one who creates the GraphQL endpoints for UI to consume ?
You should log deprecation warnings.
But also if the client is composing urls/params manually then you are not doing REST, you are doing RPC.
Rest APIs should mainly use HATEOAS hyperlinks to obtain a resource. that is clients almost always call links you have provided in other reponses (starting from a main entrypoint)
I knew we were in trouble when we started having to sort the query criteria in order to support caching of requests.
If I use graphQL again it’ll only be for admin features. Anything where very few users will use it and very infrequently. Preferably in spots where caching works against the workflow. OLAP vs OLTP.
GraphQL is really about reducing friction between teams. High functioning distributed systems all have two qualities in common: work stealing and back pressure. Without back pressure there is nothing to stop the project from running off a cliff.
what's work stealing?
I think two projects having loads of bad practices is too small a dataset to really assume anything, you sort of need to see widespread "bad practices" in the tech to be able to determine that the bad practice is actually the norm practice and there is perhaps a flaw in the tech that encourages that norm.
You're not wrong, but at the same time I think it's a decent data point that getting it right is not straight-forward and that, practical speaking, you at the very least need to think very carefully before actually using it.
"The idea of it" is sometimes fine, but then there's also "the practicality of it", and sometimes that's a very different thing.
Remember the old microkernel vs. monolithic debate; everyone more or less agrees that in principle, a microkernel is better. But the practicality of it is a lot more complex, so monolithic kernels and hybrid ones are much more common. Microservices vs monolithic is essentially the same debate, and I've seen a lot of Microservices with very poor implementations and a lot of problems. That doesn't mean the idea is bad in itself, but if it's hard to execute well, then you do need to be very careful.
There's tons more examples of this. You also see this sort of thing in e.g. politics, where what's "more fair" vs. "what's actually achievable without introducing heaps of bureaucracy and overhead" are sometimes very different things.
In the case of GraphQL, I think it's pretty obvious that the general idea, as described from a high level, is a good one. But the practicalities of it are a lot less straight-forward, as this article explains reasonably well IMHO.
That fun fact bothers me at a fundamental level. I've seen it happen so many times and is really painful when the people were promoted out of the place. Even worse when they are good developers, but often in a cowboy style where they confuse their energy for some fundamental efficiency of what they were doing.
We have similar issues in our codebases - but not just in GraphQL, but also in our PHP and Elixir code, and to some extent our Typescript stuff.
I think the "learning-on-the-go" symptom, where you can sometimes literally read down a file and watch some developer learn the language as they add more and more functions to the file with a gradual increase in skill (or, to put it less charitably, as they slowly get less bad at writing code) is probably a very common thing, and not just a GraphQL issue.
Ah this brings up bitter memories. Team I was managing was roped in to become the first graphql framework (nay platform) the ivory tower (central architecture run by the cto) team was building and trying to shove down the rest of the company. This was during the graphql craze around 5 years ago. The principal engineer leading it was even preemptively promoted to distinguished engineer for future contributions.
Fast-forward 5 years project was disbanded due to massive migration, cost and complexity problems. DE still lauded for experimentation heroism and evangelism. I think he is now pushing the latest flavors of the month to go for the next promo!
Now where does Braid-HTTP fit in?!