I'm tired of fixing customers' AI generated code

Copilot wouldn't make a typo. He just made that up and / or broke the code himself.

I knew a guy that made a good living as a freelance web developer decades ago. He would pretty much just copy and paste code from tutorials or stack overflow and had no real idea how anything worked. Using code without understanding it is never a good idea, it doesn’t need to be from AI for that to be true.

We hired a new guy at work. In one of his first tasks he had chosen to write some bash, and it was pure nonsense. I mean it contained things like:

if [ -z "${Var}+x" ]

Where I can see what the author was trying to do, but the code is just wrong.

I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, but recieved the response "I dont know what it means, I just used copilot" followed by him just removing the code.

What a waste of a learning opportunity.

Sounds like javascript "code". A normal language with proper type system would not allow that.

At least for me stupid bugs like this turn out to be some of the most time wasting to debug, no AI involved. Like accidentally have something quoted somewhere, or add an 's' to a variable by accident and I may not even correctly process what the error message is reporting at first. Always feel a bit silly after.

Claude gave me something similar, except these were both used, and somehow global variables, and it got confused about when to use which one.

Asking it to refactor / fix it made it worse bc it'd get confused, and merge them into a single variable — the problem was they had slightly different uses, which broke everything

I had to step through the code line by line to fix it.

Using Claude's still faster for me, as it'd probably take a week for me to write the code in the first place.

BUT there's a lot of traps like this hidden everywhere probably, and those will rear their ugly heads at some point. Wish there was a good test generation tool to go with the code generation tool...

And any decent IDE will highlight a variable that is declared but unused. We already have "artificial intelligence" in the form of IDEs, linters, compilers, etc. but some people apparently think we should just throw it all away now that we have LLMs.

In my experience this is exactly the kind of mistake an AI would not make.

Using code from a generative AI without understanding what it does is never a good idea.

Hear hear!

I feel like genAI is turning devs from authors to editors. Anyone who thinks the latter is lesser than the former has not performed both functions. Editing properly, to elevate the meaning of the author, is a worthy and difficult endeavor.

I burned a couple hours debugging some generated code only to finally realize copilot was referencing a variable as ‘variableO1’.

Artificial Incompetence indeed!

Using code from a generative AI without understanding what it does is never a good idea.

True, but the anecdote doesn't prove the point.

It's easy to miss that kind of difference even if you wrote the code yourself.

Using code from a generative AI without understanding what it does is never a good idea.

Yes.

AI as a faster way to type: Great!

AI as a way to discover capabilities: OK.

Faster way to think and solve problems: Actively harmful.

I think you overestimate the amount of skill a newb can quickly gain on their own. I taught myself to code, but it took a whole summer (aka free time that adults don't get) and I had access to my dad (who was a software engineer himself) to answer lots of questions.

Coding requires a willingness to understand and manipulate systems made of unbreakable rules. Most people don't want to deal with such an uncompromising method of communication.

A week is not anywhere close to enough to learn programming in any meaningful way for someone with no experience.

One of my favorite intern stories was a kid who was a compsci senior, very good university, and who was assigned to write some Python code for my team. He had the very immature "Python is a baby's language" attitude. He wasn't working on my stuff so I don't really keep track of what he's doing, but a few weeks later I look at what he has written. Almost all of his Python functions have one parameter with an asterisk, and he does a len() check on it, printing and return an integer if it's not the right length of function arguments. Turns out this guy learned this behavior from Perl, used an asterisk because why not he always does in C, and was just manually unpacking every function argument and using a C style return error handling process.

Still the most insane thing I've seen, but I know there are a lot of kids out of college who got used to gen AI for code writing who put out a lot of this kind of code. Also, coincidentally, we haven't hired any US college interns in about 3 years or so.

I'm always a little surprised at how many people out there want to develop software yet haven't put in the effort to gain even the most basic computer nerd programming chops.

If you're surprised by reality, that says something about your mental model, not about reality. People don't want to "learn programming", they want to "make a thing". Some people learn programming while making a thing, but why learn about how the sausage is made when all you want is to eat it?

I think people often don't know where to begin.

Game dev is a fun hobby some people like to mess around with, just like any other hobby. Doesn’t mean they’ll be experts or know what they’re doing, but they’ll try and probably ask some basic questions online.

I mess around in goody and game maker and can write some shitty code there, but I’ve never written a line of code for work. I just like messin around for fun

You see this all the time in the more newbie-friendly game engine

games tend to attract young people (read: beginners) but at the same time game programming's barrier to entry is pretty high with maths & physics, low-level graphical programming, memory management, low level language and dependencies, OOP... It's almost obvious that this should be the case, every kid who's interested to coding I talked to wants to do something with games.

I'm not surprised at all. Honestly the "I don't need to learn that" mentality is common in tech even in people who call themselves senior developers. It's especially noticeable in the hostility of many towards the sorts of information you learn in a good computer science degree.

How many arguments have we heard here along the lines of "why teach algorithms universities should be teaching _insert_fad_technology_of_the_day_." Big Oh and time complexity is a special favourite for people to pick on and accuse of being useless or the like. You see it in arguments around SQL vs document databases, people not being willing to recognize that their for loops are in fact the same as performing joins, people unwilling to recognize that yes they have a data schema even if they don't write it down.

So I'm not surprised at all that people would use AI as a substitute for learning. Those same people have likely gotten by with stackoverflow copypasta before gen AI came about.

I agree with the part that someone who wants to build something technical should gain at least some related knowledge, but a week is underselling the effort needed to learn how to code by a lot. After one week of teaching myself Python I couldn't code my way out of a paper bag, and I'm someone who enjoyed it enough to stick with it. The average person would need at least 10x that amount of time to be able to start building something interesting.

Seems like the natural progression from end goal to breaking it down into the smaller and smaller pieces required to see the goal through, as people have always done.

Before LLMs you'd probably have to reach for learning Python or Javascript sooner, at least if StackOverflow didn't have the right code for you to copy/paste, but I expect anyone who sticks with it will get there eventually either way.

Anyone who already programs for a couple of years can spend a week learning $lang. Learning programming for the first time takes a long while and a lot of effort. I'd say a couple of months if you're bright and motivated. Possibly a year or two if you're not.

There's a common phrase that founders should code, but not all founders are coders. So when the start up is small and the founders want to contribute by testing PoCs, the chatbots are getting used by those founders that can't code. Lucky for me, the PoC is just that and allowed to be implemented without shimming the PoC directly.

I cringe every time they mention using the bots, but luckily it has been controllable.

I am not, I see it happening even within companies. They figure that for some junior tech related roles they don't need to hire people with the education and just teach them in house. Often not developing itself, but things like automated tests in a QA role.

The result is people that have no technical background, no real interest in it either, no basic framework to start from learning to use a specific set of tools and a very basic understanding of programming.

I doubt LLM hallucinations will produce good secure code.

In my opinion, using code from LLMs, which might see your program come to life a bit quicker, will only enhance the time needed for debugging and testing, as there might be bugs and problems in there ranging from trivial things (unused variables) to very subtle and hard to find logic issues which require a deeper knowledge of the libraries and frameworks cobbled together by an LLM.

Additionally, it takes out a lot of the knowledge of these things in the long run, so people will find it more and more challenging to properly do this testing and debugging phase.

Idk if "hallucination-based API design" specifically is The Way.

There might be other ways of achieving the same goal. Also,LLM hallucination is changing/improving quite rapidly.

That said, "Designed for LLM" is probably a very productive pursuit. Puts you in the right place to understand the problems of the day.

I suppose it would be relying on being trained on code that followed good practice. If this is true then we might suppose that this API isn't following good practice. However, a gigantic feedback loop is appearing on the horizon.

The AI of tomorrow will be trained on the output of AI today.

(Somewhere in my memory, I hear an ex-boss saying "Well that's good - isn't it?")

You're going at it all wrong. You just need another LLM on the other end of the API too.

Let's all give ourselves extra prosthetic fingers and mutilate our ears while we're at it.

This post seems to be saying that AI opened up a new avenue for people to demand free work.

If someone asked "I wrote some rough designs and interfaces, can you write me an app for free?" The author could easily detect it as a request for free work. But because the first part is hidden behind some ChatGPT generated code and the second part is disguised as a request for help, the author would be tricked into doing this free work, until they detected this pattern and write a blog post about it.

The difference is scale. I don't know how many times people need to say this, but LLM tools enable people to spam low quality code at a rate that is far faster than ever.

There's been multiple stories and posts here on HN about issues with AI generated PRs for open source repos because of people using them to game numbers or clout. This is a similar problem where you have a bunch of people using your API and then effectively trying to use you for free engineering work.

That's like saying "seems like the problem is the internet is filled with low quality content" in response to ai bots when, while not wrong, the new problem is that we've created a way to accelerate the creation of that low quality content many orders of magnitude faster.

So what was a difficult problem can quickly become insurmountable.

amateurs writing bad code

in volume, this turns into support writing the code.

I think of how the south park movie folks sent so much questionable content to the censors that the compromise in the end let through lots of the puppet sex.

It's a support issue in a sense, but in many contexts people want to offer a better support experience than "anything more is billable work". A reputation for being helpful and customer-friendly is valuable, especially in a business where you're selling to other programmers, and you can't buy that reputation after the fact with money from support contracts.

Doesn't solve the problem that the budget they has in mind for the app was $300, while an experienced dev can directly see this is going to be a $20K v.1 before change requests project.

Maybe for a very short amount of time.

I suspect this quickly will be like specializing in the repair of cheap, Chinese made desk lamps from Walmart.

If the cheap desk lamp breaks, you don't pay someone to fix it. You buy another one and often it will be a newer , better model. That is the value proposition.

Of course, the hand crafted, high end desk lamp will be better but if you just want some light, for many use cases, the cheap option will be good enough.

I've been saying for a while now that there's an absolute gold mine waiting for people who want to specialize in fixing AI generated applications.

The real savvy ones will use later generations of LLMs to fix the output of early ones

I'm a bit torn. My first thought was "If the current state of the art LLMs made the mistakes it's unlikely an LLM would be able to correct them." But I'm not sure that's true if the support LLM (chat bot) is given very specific instructions so as to limit the possible answers. Still I think that's gonna break down pretty quick for other reasons.

Maybe the chat bot can recognize a botched request and even suggest a fix but what then? It certainly won't be able to convert the user's next request into a working application of even moderate complexity. And do you really want a chat bot to be the first interaction your customers have.

I think this is why we haven't seen these things take off outside of very large organizations who are looking to save money in exchange for making customers ask for a human when they need one.

For the problems given in the article, it will 100% work. It's very easy for Claude 3.5 or gpt-4o to look at documentation for a couple of API endpoints, compare it to submitted code, and point out invalid endpoints and properties. It can provide correct code also if the custom is asking for something possible.

It won't be flawless but if the issues are as basic as stated in this article, then it seems like a good option.

It will cost money to use the good models though. So I think the first step would be to make sure if they ask for an invalid endpoint it says that in so many words in the API response, and if they ask for an invalid property it states that also.

Then if that doesn't give them a clue, an LLM scans incoming support requests and automatically answers them. Or flags them for review.. To prevent abuse, it might be a person doing it but just pressing a button to send an auto-answer.

But maybe a relatively cheap bot in a Discord channel or something connected to llama 3.1 70b would be good enough to start, and people have to provide a ticket number from that bot to get an answer from a person.

Spoken like someone whose never been on the receiving end of any such ‘support’.

I seriously hope those rehires are coming back with a refined rate as well.

Sadly everyone thinks they are The "Ideas Man".

ATMs were meant to kill banking jobs but ah theres more jobs in banking than ever.

US banks, who are surprisingly behind the times in as far as automation goes. Here a lot of banks used a lot of automation to reduce the amount of manual jobs needed. to the degree that many offices are now also closing as everything can be done online.

And no, there is no need to visit banks here as I get the impression it is in the US. We don't even have physical checks anymore.

ATMs were meant to kill banking jobs but ah theres more jobs in banking than ever.

ATMs only handle basic teller functions and since COVID in NYC I had to change banks twice because I couldn't find actual tellers or banks with reasonable open hours. BoA had these virtual-teller only branches and the video systems were always broken (and the only option on Saturday). This was in Midtown Manhattan and my only option was basically a single branch on the other side of town 8-4 M-F.

I'm now happily with a credit union but at least since moving to the south things are generally better because customers won't tolerate not being able to deal with an actual person.

And then the customers will open support requests for code generated by an AI that misuse that very SDK. It doesn't look like OP's issue is with the code per say, only with the lack of skills of its customers, regardless of the code they write...

Seems like a good way to have a bunch of new products to also support

It's possible that ChatGPT is "thinking" the same thing here; that most APIs have an X endpoint to make a task easier, so surely yours does too?

While this is possible, I would caution with an anecdote from my ongoing side project of "can I use it to make browser games?", in which 3.5 would create a reasonable Vector2D class and then get confused and try to call .mul() and .sub() instead of the .multiply() and .subtract() that it had just created.

Sometimes it's exceptionally insightful, other times it needs to RTFM.

another consideration: if a popular AI model hallucinates an endpoint for your API for one customer, chances are another customer will run into the same situation

Out of topic, but:

https://www.mcgill.ca/oss/article/critical-thinking/dunning-...

The two papers, by Dr. Ed Nuhfer and colleagues, argued that the Dunning-Kruger effect could be replicated by using random data.

Yeah the customer demographic here likely does worsen the situation. Although I am sure that this is happening elsewhere as well.

It's people who generally don't have any other skills and reject all evidence for Efficient Market Hypothesis. They legitimately think what they're doing is not gambling. No amount of empirical evidence can convince them they have no risk-adjusted alpha

Your second quote is the same as the first one. Did you copy the same one twice by accident?

This is honestly something I'm grateful for a lot of the time. I'm presently running a tech start-up in a highly technical domain (housebuilding, in a word) which also happens to be pretty hostile to businesses. People look at a planning application like "Why are there hundreds of documents here?" and it's because yeah, it is hard - there are huge numbers of variables to take into account, and the real "art" of urban design is solving for all of them at once. Then you send it to planning and basically no-one is happy, why haven't you done this and what are you going to do about that. You have to be pretty creative to survive.

Before that, I worked in a digital print organisation with a factory site. This factory did huge volumes on a daily basis. It was full of machines. They had built up a tech base over years, decades, and it was hyper-optimised - woe betide any dev who walked into the factory thinking they could see an inefficiency that could be refactored out. It happened multiple times - quite a few devs, myself included, learned this lesson the hard way - on rare occasion thousands of lines of code had to be thrown out because the devs hadn't run it past the factory first.

It's an experience I'd recommend to any dev - build software for people who are not just "users" of technology but builders themselves. It's not as "sexy" as building consumer-facing tech, but it is so much more rewarding.

For real: profitable consulting businesses have been formed to help LLM programmers. BUGFIX 66, for example, and various others. They can charge substantial money to help customers cross that "last mile" and get their LLM-generated pile of code working.

These "customers" were receiving free help. If they wanted to spend money on quality software they wouldn't have used an LLM.

You can be a Google dev, and make half a million a year! For only $29.95, we'll show you how to empower yourself with the wonders of AI!

I won't say all investors are entitled and overconfident, inspired by grifters, emboldened by survivorship bias, and motivated by greed. That would be rude

This could actually be an ingenious way of solving the problem. If someone has a support issue and can't solve it themselves, yet requires coding help, forward them a freelancer that they can hire for 20$/hr from upwork that knows this API well etc.

I have the sense that most of these people won’t be willing to pay anything to have their code fixed.

Why are these people employed? Isn’t that a bit like a fake employee who outsources his work behind your back? You can’t work with the guy because he literally doesn’t even know or understand the code he’s pushing to the repo

That's a great idea, but I think the main problem is that the generated endpoints/properties will be affected quite heavily by whatever the prompt/context was.

AI isn't necessarily saying "this is the one endpoint that will always be generated". Unless it is - if the customer generated code is always the same endpoints/properties then it'd definitely make sense to also support those.

These kind of support requests are also a big issue for open source projects.

Hanging out at the DSPy discord, we get a lot of people who need help with "their" code. The code often uses hallucinated API calls, but they insist it "should work" and our library simply is buggy.

Agreed, client libs and sample applications have fallen by the wayside and could provide a solution here. It makes it very obvious to the customer when they download and run something that works, and then their changes break it, that the issue is with their changes.

Yes. They are all crap.

Funnily enough an llm is pretty good at categorizing and prioritizing support requests.

I’ve found that it’s often able to fix its own code when I’m able to understand that problem and state it clearly. On its own, it tends to just go in circles and proudly declare that its solve the problem, even though it hasn’t. It needs a knowledgeable person to guide it.

Just add more AI, that'll solve everything.

[edit] unfortunately I think I need to point out the above is sarcasm. Because there really are people using AI to review AI-generated code, and they do not see the problem with this approach.