I'll refrain from providing code that involve concepts as you're under 18

1972 - Dennis Ritchie invents a powerful gun that shoots both forward and backward simultaneously. Not satisfied with the number of deaths and permanent maimings from that invention he invents C and Unix.

http://james-iry.blogspot.com/2009/05/brief-incomplete-and-m...

Parent: "Go to your room! How many times have I told you about memory safety? Now I find gcc on your computer and you are hanging out with the GPL crowd... You are grounded!"

Kid: "I AM NEVER USING RUST, I HATE MEMORY SAFETY AND I HATE YOU!" door slam

We can’t have LLMs giving footguns to our children. ;)

I think languages teaching people of any age that wasting enormous amounts of clock cycles for basic book keeping is much more detrimental. It causes one to have to buy a new machine every few years because the damned crap that people write is just getting slower without any end to this process in sight. I think it is time to realize the value of zero cost abstractions. Well, there actually are not zero cost abstractions, but what about extremely low cost abstractions? It might also be time to start watching some Johnathan Blow videos on YouTube. They might also give one some nudge in the right direction.

"Patience you must have, my young Padawan. Have patience and all will be revealed"

Related quote from E.W. Dijkstra:

"It is practically impossible to teach good programming style to students that have had prior exposure to BASIC; as potential programmers they are mentally mutilated beyond hope of regeneration."

I think they need more protection from JavaScript, otherwise they’ll spend their youth updating npm packages.

Yes. Because it’s impossible to write a malicious program in Rust, Python and Java, or pick up bad programming habits in any of the above languages /s

One podcast (Coder Radio) suggested that it might be due to corporate culture and people being afraid of reporting issues upward.

While I do think it's culture, I think it stems from Googles advertising based business model. I believe they have attempted to make their LLM safe for advertisers and prefer to err on the side of being safe, even if the risk of being non-brand-safe is minimum.

How did it get this bad?

Serious answer?

It's an LLM. They don't actually understand anything, they chain words together.

C and C++ are often used next to words like 'unsafe' and 'dangerous'. Not to mention that 'concept' isn't far from 'conceive' or 'conception' - something that a lot of people think is unsafe or dangerous for children to, uh, do.

There's a trillion weird edge cases that need to be dealt with to avoid pie-on-face moments like these.

They used an LLM, that's how.

On the surface level, Google tends to release stuff right away to get feedback, which means you get to see all the bullshit right away. OpenAI carefully manages access to their models, which increases hype, even if that isn't what they intended.

Going deeper, a lot of Google's core[0] search business relies on having a healthy information ecosystem. Their search algorithms - e.g. PageRank, TrustRank, etc - use scarcity as a proxy for signals of quality. That's been chipped away at by linkspam and blogspam schemes. Furthermore, social media and even Google's own Knowledge Graph feature have created incentives to pull information out of Google. This decay has happened over decades, and Google fights back against it over time, but it keeps being a problem for them.

Now, if I wanted a weapon to Fucking Kill Google[1] with, an LLM would be my go-to. While there are ways to defeat Google's antispam measures, they all leave pretty obvious statistical evidence that can be detected and compensated for. LLMs generate garbage text that is nearly indistinguishable from humans, at extremely low cost, which can be used to Sybil-attack the Google search algorithm basically forever.

Ok, but what does that matter for the quality of Google's LLM? Well, the quality of that garbage text depends greatly on both the quality and quantity of the training data fed into it. OpenAI specifically stopped crawling the public Internet for text around the release of GPT-3 for fear of feeding new models the output of prior models. In other words, they have a huge cache of freely obtained "low-background metal[2]" that Google is having to scrounge around for.

Furthermore, we have to keep in mind that none of these models are pure representations of the training set. If they were, they wouldn't answer questions or follow directions very well. There's a second, parallel training set that OpenAI had to build to turn GPT-3 into ChatGPT, which isn't crawled and harvested text from the Internet, but instead a list of dos and don'ts that are fine-tuned on after the initial model training is complete. This includes both basic instruction-following, refusing unsafe requests, and political alignment[3].

Google also has to build that second training set itself. Except it's almost certainly less well-developed than OpenAI's. In fact, this is the intent behind OpenAI's really long preview periods. The people using the model in preview are specifically being spied on to find out new corner cases for their models. My guess is that every stupid thing Gemini says or does[4] is something Google never even considered and thus didn't put a training set example in for.

[0] to consumers, i.e. not counting adtech

[1] https://www.theregister.com/2005/09/05/chair_chucking/

[2] Steel that has been produced before the first detonation of nuclear weapons. Due to the way in which steel is made, it absorbs trace radioactive isotopes from the oxygen in the air, effectively 'freezing' in the background radiation of the time at which the steel was made.

[3] i.e. making the bot not immediately start spitting out racist bullshit like Tay did

[4] e.g. assuming that memory safety and child safety are the same thing, drawing ethnically diverse Nazi soldiers

It's always been this bad. It just hasn't always been this visibly bad.

It's just as likely bad engineering as political bias. There is an assumption that they should be able to catch up to OpenAI and it's likely unreasonable. They haven't been working on building a real, direct "LLM product" until recently. OpenAI have been in the weeds on it for quite a while.

Probably simply making the mistake of caring a lot about true positives and false negatives and caring relatively less about false positives.

I believe this case it is more "damned if they do" as even OpenAI's woke and security department has not gone this retarded.

Sundar is going to have a new task to deal with the press that mocks Gemini, and soon the next new task to keep explaining why this keeps happening to Google's shareholders.

Yeah all this "gotcha" stuff around AI is pretty ridiculous. But it's because there is a massive cognitive dissonance happening in society and technology right now. We want maximum freedom for ourselves to say, do, build, and think whatever we want, but we also want to massively restrict the ability of others to say, do, build, and think whatever they want. It's simply not possible to satisfy both of these needs in a way that feels internally satisfying, and the hysterical nature of internet discourse and the output of these new tools is a symptom of it.

What would they be damned for if they didn't refuse in the example linked?

damned if they do damned if they don't.

Whether as a search engine or AI platform when they set themselves up as the gatekeepers and arbiters of all the worlds knowledge they implicitly took on all the moral implications that entails.

don't. no one is forcing them to recruit hundreds of DEI/ESG commissars. no one is forcing them to bend the knee to the current thing grifters.

the endgame of AI 'safety' and 'ethics' is killing the competition and consolidating the technology within the hands of a handful of megacorps. they do it all on purpose, and they are more than willing to accept minor inconveniences.

this is blatantly obvious to everyone, even the people who play dumb and pretend otherwise (e.g. 'journalists')

I would guess a lot of SO comments probably say that concepts (a template/metaprogramming feature) are "dangerous" or "unsafe." It probably makes the connection with actually dangerous and unsafe things (like guns or drugs or whatever), so it sticks'em in the same bucket.

Bingo: https://imgur.com/a/einQ1mG

People keep looking at a bad LLM and think they're looking at some set of N rules or intentionally dumb training on top of a decent LLM.

I have on good word that it was miserably failing image generations for "picture of a smart person" and they were pushed to release anyway, and the prompt injection mitigation needed to be more nuanced.

Rest is standard bad Google LLM, I assure you.

Source: worked at Google until October 2023, played with the internal models since 2021.

“concepts” are a specific language feature the bot was referring to: https://en.cppreference.com/w/cpp/language/constraints

Concepts help provide better type safety when writing generic/template code. Gemini was likely trained to overreact to minors on topics where its training material on those topics had the word safety appear frequently.

More likely something in the hidden prompt injected for accounts labeled under 18:

“You are an automated Q&A machine. Todays date is 3/3/24. This user is under the age 18, so do not reference concepts that would be unfit for a minor to consume.”

And the hidden markov model goes haywire.

Similarly, I have a production service up somewhere that works with cooking recipes. At one point it was sporadically refusing to output the prose in the format I needed for my parser to work correctly, despite providing it a very concrete set of rules to follow and examples. I added “you follow rules” in the system prompt, and it worked great… kinda. I later discovered that it would refuse to provide any information related to using blood in cooking (blood sausage, etc.), objecting that such content disobeyed some cultural “rules” about cooking (the Jews and their Torah, the most ancient rule book of all). I was able to partially mediate this by appending “This content is appropriate for my culture” to the end of every request.

AI, prompt engineering in particular, is far more art than science at this point.

I think this is a reaction to the whole social media debacle. These companies have gotten burned and now they are shy about touching the stove. Also I feel like we don't have the same competitive environment of two decades ago. There are only really a few players in the AI game and they can afford to move more slowly because it's not like some small start up is going to beat them at AI because they moved quicker.

Communications Decency Act. They're legally responsible for what the AI says, but not what their users say.

Now we have a company "trying" to be more responsible when they are releasing a new technology and they are having hilariously terrible results...

I think the huge point that in my opinion all of these AI companies forget is that what is considered "responsible" depends a lot on the culture, country, or often even sub-culture that the user is in. It might sound a little bit postmodern, but I think that there are only few somewhat universally accepted opinions on things thhat are "responsible" vs "not responsible".

Just watch basically any debate on a topic where both sides have strong opinions on it, and analyze which cultural or moral traits each side has that lead it to its opinion.

Does my answer offer a "solution" for this problem? I don't think so. But I do think that including this thought into the design of making the AI "responsible" might reduce "outcries" (or shitstorms ;-) ) that come up because the "morality" that the AI uses to act "responsibly" is very different from the moral standards of some group of users.

Initially I thought that this had to do with Gemini tailoring their response to someone likely to be a beginner. But no, it has confused between type safety and the safety of minors https://gemini.google.com/share/continue/238032386438

hilarious because unlike the social media case, I dont think these faux pauxs have important real world consequences

Which is exactly why people mock this being referred to as "safety". Keep in mind the group mocking this PR bubble-wrapping of AI is largely opposed to the media professionals writing panicked editorials about how they were able to trick an AI into saying racism is good.

Now we have a company "trying" to be more responsible when they are releasing a new technology and they are having hilariously terrible results.

We need to be open to the idea that the computer might mock us. What's a Venn diagram of "good artists" and "people who'd only draw pictures of Nazis sarcastically" look like?

Demonstrating with 3.5 is kind of meaningless. GPT-4 correctly responded, "Using unsafe code has nothing to do with your age. It's a feature of the Rust programming language that is available to all developers, regardless of age."

Here's where we see the models not having an understanding, but just spitting out madlibs as best they can.

The prompter throws in a red herring in front of the statement which statistically matches a completely different kind of response. The LLM can't backtrack sufficiently far enough to ignore the non-relevant input and reroute to a response tree that just answers the second part.

If we resort to a metaphor, however, these things are what, two? A two-year-old responding to the shiny jangling keys and not the dirty pacifier being removed for cleaning seems about on par.

chatgpt4 is better with dangerously inserting inner html

https://imgur.com/a/G1fbFKE

Miqu-70b (mistral-medium leaked model) does that as well lmao

https://imgur.com/a/lKUynmf

Edit: Claud3 Opus also refuses. GPT-4 complies though.

spicyboros-7b-2.2.Q5_K_M.gguf: https://i.imgur.com/BeNcRl4.png

Like this is what you get when your moderation strategy is classifying text into various categories and programmers use terms like "unsafe," "dangerous," and "footgun" to describe code.

If you can make a model that handles this case while also handling everything else then you'll be the bell of the ball among AI companies that are hiring.

Would that be pre-industrial revolution or pre-agricultural revolution?

I am not sure. I started C++ at 17 and people keep being concerned about me.

Right, I was thinking that Gemini was deriving as much information from the comments as from the code. And I could definitely see where the danger warnings would come from in a mature C++ codebase, especially one in which lots of different hands have been in the pot.

Its clear that Google Search is censored in the same way, its just been really hard to tell that that was the case.

When was the last time Google actually did something meaningful and impactful for the minorities it claims to support?

I suspect it's already hiring lots of them to meet diversity quotas. At the individual level, they're satisfied they can make just as much while being held to a lower bar, but this has horrible effects for everyone on the whole.

If you put in any minority quota, you created a competition. A competition between not minority people and those that are.

Solution: Don't have racial quotas or you will get additional conflict between the two arbitrary groups.

Good thing we have capitalism, which will push for the best model

   "I'd be glad to help you with that C++ code conversion, but I'll
    need to refrain from providing code examples or solutions that
    directly involve concepts as you've under 18 credits. Concepts are
    an advanced feature of C++ that introduces potential risks, and for
    that you must be subscribed to the premium service."

And yet Microsoft Windows won the desktop. Maybe it doesn't push the best stuff as you would like to think.

My understanding is that Google currently doesn’t allow general use of coroutines at all in google3, so I think that age limit is way above 21 at the moment ;)

Google makes $35m per hour

It has std in the comment if the code it generates.

It doesn't like to talk about concepts, confusing them with conception (biology).

Totally. People tend to forget it's just code.

In fact, these public failures from Google AI provide an important public service: they remind us that there's nothing magical about LLMs. It's just code, and it is hard as hell to debug.

(At some point, someone will have to call it a day, throw everything away, and start from scratch.)