Ex-Google search engineer here (2019-2023). I know a lot of the veteran engineers were upset when Ben Gomes got shunted off. Probably the bigger change, from what I've heard, was losing Amit Singhal who led Search until 2016. Amit fought against creeping complexity. There is a semi-famous internal document he wrote where he argued against the other search leads that Google should use less machine-learning, or at least contain it as much as possible, so that ranking stays debuggable and understandable by human search engineers. My impression is that since he left complexity exploded, with every team launching as many deep learning projects as they can (just like every other large tech company has).
The problem though, is the older systems had obvious problems, while the newer systems have hidden bugs and conceptual issues which often don't show up in the metrics, and which compound over time as more complexity is layered on. For example: I found an off by 1 error deep in a formula from an old launch that has been reordering top results for 15% of queries since 2015. I handed it off when I left but have no idea whether anyone actually fixed it or not.
I wrote up all of the search bugs I was aware of in an internal document called "second page navboost", so if anyone working on search at Google reads this and needs a launch go check it out.
This better echoes my personal experience with the decline of Google search than TFA: it seems to be connected to the increasing use of ML in that the more of it Google put in, the worse the results I got were.
Same here with YouTube, assuming they use ML, which is likely.
They routinely give me brain-dead suggestions such as to watch a video I just watched today or yesterday, among other absurdities.
YT Shorts recommendations are a joke. I'm an atheist and very rarely watch anything related to religion, and even so Shorts put me in 3 or 4 live prayers/scams (not sure) the last few months.
YT Shorts itself is kind of a mystery to me. It's an objective degradation of the interface; why on earth would I want to use it? It doesn't even allow adjustment of the playback speed or scrubbing!
You can scrub on the mobile player, that's what makes it so much frustrating because you can't do that on desktop
What does scrubbing mean in this context? Blocking the Shorts?
Scrubbing means quickly moving the current playback position back and forward
Seeking to a certain part of the video. On mobile, you can do it by dragging the progress bar at the bottom of the screen.
I think there is a large demo of people now who actually prefer to watch videos in portrait.
I dont mind portrait. I mind inability to jump forward in the video.
If you’re watching a single subject of interest video on your phone (TikTok type of content), it’s great. But landscape videos is more pleasant and there’s a reason we move from 4:3 for media. But that actually means watching the videos, but what I see is a lot of skipping.
So, there's a few ways to explain it. From a business strategy level, TikTok exists, and is a threat to YouTube, so we need to compete with it.
From a user perspective, Shorts highlights a specific format of YouTube that happened to have been around for a lot longer than people realize. TikTok isn't anything new, Vine was doing exactly the same thing TikTok was a decade prior. It was shut down for what I can only assume was really dumb reasons. A lot of Viners moved to YouTube, but they had to change their creative process to fit what the YouTube algorithm valued at the time: longer videos.
Pre-Shorts, there really wasn't a good place on YouTube for short videos. Animators were getting screwed by the algorithm because you really can't do daily uploads of animation[0] and whatever you upload is going to be a few minutes max. A video essayist can rack up hundreds of thousands of hours of watch time while you get maybe a thousand.
(Fun fact: YouTube Shorts status was applied retroactively to old short videos, so there's actually Shorts that are decades old. AFAIK, some of the Petscop creator's old videos are Shorts now.)
But that's why users or creators would want to use Shorts. A lot of the UX problems with Shorts boils down to YouTube building TikTok inside of YouTube out of sheer corporate envy. To be clear, they could have used the existing player and added short-video features on top (e.g. swipe-to-skip). In fact, any Short can be opened in the standard player by just changing the URL! There's literally no difference other than a worse UI because SOMEONE wanted "launched a new YouTube vertical" on their promo packet!
FWIW the Shorts player is gradually getting its missing features back but it's still got several pain points for me. One in particular that I think exemplifies Shorts: if I watch Shorts on a portrait 1080p monitor - i.e. the perfect thing to watch vertical video on - you can't see comments. When you open the comments drawer it doesn't move over enough and the comments get cut off. The desktop experience is also really bad; occasionally scrolling just stops working, or it skips two videos per mousewheel event, or one video will just never play no matter how much I scroll back and forth.
[0] Vtubers don't count
Solid point. Not to mention that Shorts content is mainly linkbait and/or garbage.
I imagine my blocked channels list is stress testing YouTube at this point from the amount of shit Shorts results it's fed me after 2 years. Lol
Besides the religious crap, ill randomly get shit in India in hindu, having had not watched anything Indian and not even remotely Indian.
Hindi is the word for the language, bro.
I knew I could count on you.
You bet. Think nought of it. We gave the world zero, after all. Even computers owe us. ;)
https://en.m.wikipedia.org/wiki/0
I only get those when it's new content with <20 likes and they are testing it out. Doesn't bother me, I like to receive some untested content - even though 99% of it is pure crap (like some random non-sense film with a trendy music on top).
Prayers for the unbelievers makes some sense.
But I associate YouTube promotions with garbage any how. The few things I might buy like Tide laundry detergent are entirely despite occasional YouTube promotion.
Lmao. I'm very positive that the conversion rate for placing an atheist in a live mass out of the blue is very very very low. Because I never stayed for more than 3 seconds, I'm not sure if it's real religious content or a scam, though - and they don't even let me report live shorts :(
“Conversion rate”. I’m not sure if you intended that pun but it’s pretty good.
Similarly, Google News. The "For You" section shows me articles about astrology because I'm interested in astronomy. I get suggestions for articles about I-80 because I search for I-80 traffic cams to get traffic cam info for Tahoe, but it shows me I-80 news all the way across the country, suggestions about MOuntain View because I worked there (for google!) over 3 years ago, commanders being fired from the Navy (because I read a couple articles once), it goes on and on. From what I can tell, there are no News Quality people actually paying attention to their recommendations (and "Show Fewer" doesn't actually work. I filed a bug and was told that while the desktop version of the site shows Show Fewer for Google News, it doesn't actually have an effect).
Part of the reason I switched from google to duckduckgo for searching was I didn't WANT "personalization" I want my search results to be deterministic. If I am in Seattle and search for "ducks" I want the exact fucking same search results as if I travel to Rio de Janeiro and search for "ducks".
Honestly, I'd prefer my voice assistant (siri mostly) to be like that as well. It was at first, and I think everyone hated that lol.
Just because you're an atheist doesn't mean you won't engage with religious content though. YT rewards all kinds of engagement not just positive ones. I.e. if you leave a snide remark or just a dislike on a religious short that still counts as engagement.
Yes I know, not the case, and before you ask, I also don't engage with atheist videos. But that's only one example: the recommendations are really bad in a lot of ways for me.
For what it's worth, I do not remember a time when YouTube's suggestions or search results were good. Absurdities like that happened 10 and 15 years ago as well.
These days my biggest gripe is that they put unrelated ragebait or clickbait videos in search results that I very clearly did not search for - often about American politics.
I do remember when Youtube would show more than 2 search results per page on my 23" display.
Or when they would show more than 3 results before spamming irrelevant videos.
Or when they didn't show 3 unskippable ads in a 5 minute video.
Or when they had a dislike button so you would know to avoid wasting time on low quality videos.
Wait what?! You "Consume Content" on a COMPUTER? What are you some kinda grandpa? Why aren't you consuming content from your phone like everyone else? Or casting it from your phone to your SMART TV! Great way to CONSUME CONTENT!
CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT
Lol, Youtube on Apple TV is great. Mostly because I either need to find something fast or I switch it off because the remote is not conducive to skipping. But the only time I watch Youtube on my computer is for a specific video. The waste of space is horrendous. Same with Twitter (rarely visited), just a 3/4 inches wide column of posts on my 24 inch screen.
I'm not consuming the content on my phone, because the user experience of using these services on my phone sucks. Just the app vs website difference with urls is a difference in behavior I hate let alone all the UI differences that make the mobile experience awkward.
I don't know about the TV though.
On Android devices I use the app PipePipe to avoid the YouTube ad hell. I recommend it.
I also use Firefox for Android, which has Addon support. Ublock Origin works on the phone and disables a a lot of the ad horror.
15 years ago, I used to keep many tabs of youtube videos open just because the "related" section was full of interesting videos. Then each of those videos had interesting relations. There was so much to explore before hitting a dead-end and starting somewhere else.
Now the "related" section is gone in favor of "recommended" samey clickbait garbage. The relations between human interests are too esoteric for current ML classifiers to understand. The old Markov-chain style works with the human, and lets them recognize what kind of space they've gotten themselves into, and make intelligent decisions, which ultimately benefit the system.
If you judge the system by the presence of negative outliers, rather than positive, then I can understand seeing no difference.
I would go further and say that it is impossible. Human interests are contextual and change over time, sometimes in the span of minutes.
Imagine that all the videos on the internet would be on one big video website. You would watch car videos, movie trailers, listen to music, and watch porn in one place. Could the algorithm correctly predict when you're in the mood for porn and when you aren't? No, it couldn't.
The website might know what kind of cars, what kind of music, and what kind of porn you like, but it wouldn't be able to tell which of these categories you would currently be interested in.
I think current YouTube (and other recommendation-heavy services) have this problem. Sometimes I want to watch videos about programming, but sometimes I don't. But the algorithm doesn't know that. It can't know that without being able to track me outside of the website.
I think there are things they could do and that ML could maybe help?
* They could let me directly enter my interests instead of guessing
* They could classify videos by expertise (tags or ML) and stop recommending beginner videos to someone who expresses an interest in expert videos.
* They could let me opt out of recommending videos I've already watched
* They could separate sites into larger categories and stop recommending things not in that category. For me personally, when I got to youtube.com I don't want music but 30-70% of the recommendations are for music. If the split into 2 categories (videos.youtube.com - no music) and (music.youtube.com - only music) they'd end up recommending far more to me that I'm actually interested in at the time. They could add other broad categories like (gaming.youtube.com, documentaries.youtube.com, science.youtube.com, cooking.youtube.com, ...., as deep as they want). Classifying a video could be ML or creator decided. If you're only allowed one category they would be incentive to not mis-classify. If they need more incentive they could dis-recommend your videos if you mis-classify too many/too often).
* They could let me mark videos as watched and actually track that the same as read/unread email. As it is, if you click "not interested -> already watched" they don't mark the video as visibly watched (the red bar under the video). Further, if you start watching again you lose the red-bar (it gets reset to your current position). I get that tracking where you are in a video is something that's different for email vs video but at the same time (1) if I made it to 90% of the way through then for me at least, that's "watched" - same as "read" for email and I'd like it "archived" (don't recommend this to me again) even if I start watching it again (same as reading an email marked as "read)
Those are some good suggestions, particularly the first one:
Theres a general problem in the tech world where people seem to inexplicably disregard the issue of non-reducibility. The point about the algorithm lacking access to necessary external information is good.
A dictionary app obviously can't predict what word I want to look up without simulating my mind-state. A set of probabilistic state transitions is at least a tangible shadow of typical human mind-states who make those transitions.
They probably optimize your engagement NOW - with clickbaity videos. So their KPIs show big increases. But in long term you realize that what you watch is garbage and stop watching alltogether.
Someone probably changed the engine that shows videos for you - exactly as with search.
Long long time ago; youtube "staff" would manually put certain videos on the top of the front page when they started. Im sure there we're biases and prioritization of marketing dollars but at least there was human recommending it compared to poorly recorded early family guy clips. I dont know when they stopped manually adding "editors/staff" choice videos but I recall some of my favorite early youtubers like CGPGgrey claim that recommendation built the career.
See this >15-year-old video "How to get featured on YouTube" - https://www.youtube.com/watch?v=-uzXeP4g_qA, which I remember as being originally uploaded to the official Youtube channel but looks like it's been removed now, this reupload is from October 2008.
YouTube seems to treat popular videos as their own interest category and it’s very aggressive about recommending them if you show any interest at all. If you watch even one or two popular videos (like in the millions of views), suddenly the quality of the recommendations drops off a cliff, since it is suggesting things that aren’t relevant to your interest categories, it’s just suggesting popular things.
If I entirely avoid watching any popular videos, the recommendations are quite good and don’t seem to include anything like what you are seeing. If I don’t entirely avoid them, then I do get what you are seeing (among other nonsense).
It all depends on your use case but a lot of people seem to be in agreement it fell off in the mid to late 10s and the suggestions became noticeably worse.
I think it's probably pushing pattern it sees in other users.
There's videos I'll watch multiple times, music videos are the obvious kind, but for some others I'm just not watching/understanding it the first time and will go back and rewatch later.
But I guess youtube has no way to understand which one I'll rewatch and which other I don't want to see ever again, and if my behavior is used as training data for the other users like you, they're probably screwed.
A simple "rewatch?" line along the top would make this problem not so brain dead bad, imho. Without it you just think the algorithm is bad (although maybe it is? I don't know).
Install "Unhook" chrome extension. That changed my life.
This is happening to me to, but from the kind of videos it's suggested for I suspect that people actually do tend to rewatch those particular videos, hence the recommendation.
It's also a good lesson for the new AI cycle we're in now. Often inserting ML subsystems into your broader system just makes it go from "deterministically but fixably bad" to "mysteriously and unfixably bad".
I think - I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations... but we need to be frank about the fact that the business layer above us (with a few rare exceptions) absolutely does not understand the limitations of AI and views it as a magic box where they type in "Write me a story about a bunny" and get twelve paragraphs of text out. As someone working in a healthcare adjacent field I've seen the glint in executive's eyes when talking about AI and it can provide real benefits in data summarization and annotation assistance... but there are limits to what you should trust it with and if it's something big-i Important then you'll always want to have a human vetting step.
> I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations.
The people I see who are most excited about ML are business types who just see it as a black boxes that makes stock valuation go vroom.
The people that deeply love building things, really enjoy the process of making itself, are profoundly sceptical.
I look at generative AI as sort of like an army of free interns. If your idea of a fun way to make a thing is to dictate orders to a horde of well-meaning but untrained highly-caffienated interns, then using generative AI to make your thing is probably thrilling. You get to feel like an executive producer who can make a lot of stuff happen by simply prompting someone/something to do your bidding.
But if you actually care about the grit and texture of actual creation, then that workflow isn't exactly appealing.
We get it, you're skeptical of the current hype bubble. But that's one helluva no true Scotsman you've got going on there. Because a true builder, one that deeply loves building things wouldn't want to use text to create an image. Anyone who does is a business type or an executive producer. A true builder wouldn't think about what they want to do in such nasty thing as words. Creation comes from the soul, which we all know machines, and business people, don't have.
Using English, instead of C, to get a computer to do something doesn't turn you into a beaurocrat any more than using Python or Javascript instead does.
Only a person that truly loves building things, far deeper than you'll ever know, someone that's never programmed in a compiled language, would get that.
Getting drunk off that AI kool-aid aren't ya
the othering of creators because they use a different paintbrush was bothering me.
please go other yourself somewhere else
Hit a nerve, it seems. Apologies.
I can relate, AI is a tool, and if I want to write my code by LEGOing a bunch of AI-generated functions together, I should be able to.
Was it intentional to reply with another no true Scotsman in turn here?
Yeah, I was also reading their response and was confused. "Creation comes from the soul, which we all know machines, and business people, don't have" ... "far deeper than you'll ever know", I mean, come on.
If you have to ask, then you missed it
If one uses English in as precise a way as one crafts code, sure.
Most people do not (cannot?) use English that precisely.
There's little technical difference between using English and using code to create...
... but there is a huge difference on the other side of the keyboard, as lots of people know English, including people who aren't used to fully thinking through a problem and tackling all the corner cases.
No one can, which is why any place human interaction needs anything anywhere close to the determinancy of code, normal natural langauge is abandoned for domain-specific constructed languages built from pieces of natural language with meanings crafted especially for the particular domain as the interface language between the people (and often formalized domain-specific human-to-human communication protocols with specs as detailed as you’d see from the IETF.)
I gotta say, I love how you use english to perfectly demonstrate how imprecise english is without pre-understood context to disambiguate meaning.
using English has been tried many times in the history computing; Cobol, SQL, just to name a very few.
Still needed domain experts back then, and, IMHO, in years/decades to come
Or you can draw pretty pictures in LabVIEW lol
They wouldn’t think this way if stock investors weren’t so often such naive lemmings ready to jump off yet another cliff with each other.
really, my impression is the opposite. They are driven by doing cool tech things and building fresh product, while getting rid of "antiquated, old" product. Very little thought given to the long term impact of their work. Criticism of the use cases are often hand waved away because you are messing with their bread and butter.
I think we also need to be aware that this business layer above us that often sees __computers__ as a magic box where they type in. There's definitely a large spectrum of how magical this seems to that layer, but the issue remains that there are subtleties that are often important but difficult to explain without detailed technical knowledge. I think there's a lot of good ML can do (being a ML researcher myself), but I often find it ham-fisted into projects simply to say that the project has ML. I think the clearest flag to any engineer that this layer above them has limited domain knowledge is by looking at how much importance they place on KPIs/metrics. Are they targets or are they guides? Because I can assure you, all metrics are flawed -- but some metrics are less flawed than others (and benchmark hacking is unfortunately the norm in ML research[0]).
[0] There's just too much happening so fast and too many papers to reasonably review in a timely manner. It's a competitive environment, where gatekeepers are competitors, and where everyone is absolutely crunched for time and pressured to feel like they need to move even faster. You bet reviews get lazy. The problems aren't "posting preprints on twitter" or "LLMs giving summaries", it's that the traditional peer review system (especially in conference settings) poorly scales and is significantly affected by hype. Unfortunately I think this ends up railroading us in research directions and makes it significantly challenging for graduate students to publish without being connected to big labs (aka, requiring big compute) (tuning is another common way to escape compute constraints, but that falls under "railroading"). There's still some pretty big and fundamental questions that need to be chipped away at but are difficult to publish given the environment. /rant
I’m not optimistic on that point: the executive class is very openly salivating at the prospect of mass layoffs, and that means a lot of technical staff aren’t quick to inject some reality – if Gartner is saying it’s rainbows and unicorns, saying they’re exaggerating can be taken as volunteering to be laid off first even if you’re right.
I think that’ll define the industry for the coming decades. I used to work in machine translation and it was the same. The older rules-based engines that were carefully crafted by humans worked well on the test suite and if a new case was found, a human could fix it. When machine learning came on the scene, more “impressive” models that were built quicker came out - but when a translation was bad no one knew how to fix it other than retraining and crossing one’s fingers.
Yes, but I think the other lesson might be that those black box machine translations have ended up being more valuable? It sucks when things don't always work, but that is also kind of life and if the AI version worked more often that is usually ok (as long as the occasional failures aren't so catastrophic as to ruin everything)
Can’t help but read that and think of Tesla’s Autopilot and “Full Self Driving”. For some comparisons they claim to be safer per mile than human drivers … just don’t think too much about the error modes where the occasional stationary object isn’t detected and you plow into it at highway speed.
relevant to the grandparent’s point: I am demoing FSD in my Tesla and what I find really annoying is that the old Autopilot allowed you to select a maximum speed that the car will drive. Well, on “FSD” apparently you have no choice but to hand full longitudinal control over to the model.
I am probably the 0.01% of Tesla drivers who have the computer chime when I exceed the speed limit by some offset. Very regularly, even when FSD is in “chill” mode, the model will speed by +7-9 mph on most roads. (I gotta think that the young 20 somethings who make up Tesla's audience also contributed their poor driving habits to Tesla's training data set) This results in constant beeps, even as the FSD software violates my own criteria for speed warning.
So somehow the FSD feature becomes "more capable" while becoming much less legible to the human controller. I think this is a bad thing generally but it seems to be the fad today.
I have no experience with Tesla and their self-driving features. When you wrote "chill" mode, I assume it means the lowest level of aggressiveness. Did you contact Tesla to complain the car is still too aggressive? There should be a mode that tries to drive exactly the speed limit, where reasonable -- not over or under.
Yes there is a “chill” mode that refers to maximum allowed acceleration and “chill mode” that refers to the level if aggressiveness with autopilot. With both turned on the car still exceeds the speed limit by quite a bit. I am sure Tesla is aware.
Well Tesla might be the single worst actor in the entire AI space, but I do somewhat understand your point. The lake of predictable failures is a huge problem with AI, I'm not sure that understandability is by itself. I will never understand the brain of an Uber driver for example
They are lying with statistics, for the more challenging locations and conditions the AI will give up and let the human take over or the human notices something bad and takes over. So Tesla miles are miles are cherry picked and their data is not open so a third party can make real statistics and compare apples to apples.
Or in some cases, the Tesla slows down, then changes its mind and starts accelerating again to run over child-like obstructions.
Ex: https://www.youtube.com/watch?v=URpTJ1Xpjuk&t=293s
The key difference is how tolerant the specific use case is of a probably-correct answer.
The things recent-AI excels at now (generative, translation, etc.) are very tolerant of "usually correct." If a model can do more, and is right most of the time, then it's more valuable.
There are many other types of use cases, though.
A case in point is the ubiquity of Pleco in the Chinese/English space. It’s a dictionary, not a translator, and pretty much every non-native speaker who learns or needs to speak Chinese uses it. It has no ML features and hasn’t changed much in the past decade (or even two). People love it because it does one specific task extremely well.
On the other hand ML has absolutely revolutionised translation (of longer text), where having a model containing prior knowledge about the world is essential.
As someone who worked in rules-based ML before the recent transformers (and unsupervised learning in general) hype, rules-based approaches were laughably bad. Only now are nondeterministic approaches to ML surpassing human level tasks, something which would not have been feasible, perhaps not even possible in a finite amount of human development time, via human-created rules.
The thing is that AI is completely unpredictable without human curated results. Stable diffusion made me relent and admit that AI is here now for real, but I no longer think so. It's more like artificial schizophrenia. It does have some results, often plausible seeming results, but it's not real.
yes, who exactly looked at the 70% accuracy of "live automatic closed captioning" and decided Great! ship it boys!
My guess: They are hoping user feedback will help them to fix the bugs later -- iterate to 99%. Plus, they are probably under unrealistic deadlines to delivery _something_.
I think NM translation was broken all along. Not in the neural network part but in choosing the right answer. https://aclanthology.org/2020.coling-main.398.pdf
But rule-based machine translation, from what I've seen, is just so bad. ChatGPT (and other LLM) is miles ahead. After seeing what ChatGPT does, I can't even call rule-based machine translation "tranlation".
*Disclaimer: as someone who's not an AI researcher but did quite some human translation works before.
Perhaps using a ML to craft the deterministic rules and then have a human go over them is the sweet spot.
mysteriously with a helping of random too!
I've heard AI described as the payday loan (or "high-interest credit card") of technical debt.
This is why hallucinations will never be fixed in language models. That's just how they work.
that's not something ML people would like to hear
Is ML the new SOAP? Looks like a silver bullet and 5 years later you're drowning in complexity for no discernible reason?
So... obviously SOAP was dumb[1], and lots of people saw that at the time. But SOAP was dumb in obvious ways, and it failed for obvious reasons, and really no one was surprised at all.
ML isn't like that. It's new. It's different. It may not succeed in the ways we expect; it may even look dumb in hindsight. But it absolutely represents a genuinely new paradigm for computing and is worth studying and understanding on that basis. We look back to SOAP and see something that might as well be forgotten. We'll never look back to the dawn of AI and forget what it was about.
[1] For anyone who missed that particular long-sunken boat, SOAP was a RPC protocol like any other. Yes, that's really all it was. It did nothing special, or well, or that you couldn't do via trivially accessible alternative means. All it had was the right adjective ("XML" in this case) for the moment. It's otherwise forgettable, and forgotten.
ML has already succeeded to the point that it is ubiquitous and taken for granted. OCR, voice recognition, spam filters, and many other now boring technologies are all based on ML.
Anyone claiming it’s some sort of snake oil shouldn’t be taken seriously. Certainly the current hype around it has given rise to many inappropriate applications, but it’s a wildly successful and ubiquitous technology class that has no replacement.
Call me back when you have voice recognition that doesn't constantly fail spectacularly.
Voice recognition will never be rule based.
Thank you for this.
Reading these comments I thought I stepped into some alternate timeline when we don't already have widespread ML all over the place.
Like, nobody does rules-based image recognition for a decade now already!
That ML I have no problem with.
This new ML that's supposed to be the basis for an entire new economic wave, that I mostly dislike.
But I guess that's how we build new things... We explore and throw away 80% of what we've built.
Yeah, I'm staring at my use of chatgpt to write a 50 line python program that connected to a local sqlite db and ran a query; for each element returned, made an api call or ran a query against a remote postgres db; depending on the results of that api call, made another api call; saved the results to a file; and presented results in a table.
Chatgpt generated the entirety of the above w/ me tweaking one line of code and putting creds in. I could have written all of the above, but it probably would have taken 20-30 minutes. With chatgpt I banged it out in under a minute, helped a colleague out, and went on my way.
Chatgpt absolutely is a real advancement. Before they released gpt4, there was no tech in the world that could do what it did.
ML is somewhere between the new SOAP and the new cryptocurrency.
Well thats grim
Dear sir, may I interest you in the initial coin airdrop of WSDLCoin? It is going straight to the moon.
> SOAP
Argh. My PTSD from writing ONVIF drivers just kicked in.
Been there, Done that. Slides over a bottle of single malt.
Horrifying memories of Microsoft Biztalk
Don't forget about that expensive GPU infrastructure you invested in.
and the power bill
and how difficult it is to program those GPU to do ML
ML is a quite well adopted technology. iPhones has ML bulit in since about 2017. It has been more than 5 years.
Well, it depends on the ML person. I work on industrial ML and DL systems every day and I'm the one who made that comment.
Machine learning or not, seo spam sort of killed search. It’s more or less impossible to find real sites by interesting humans these days. Almost all results are Reddit, YouTube, content marketing, or seo spam. And google’s failure here killed the old school blogosphere (medium and substack only slightly count), personal websites, and forums
Same is happening to YouTube as well. Feels like it’s nothing but promoters pushing content to gain followers to sell ads or other stuff because nobody else’s videos ever surface. Just a million people gaming the algorithm and the only winners are the people who devote the most time to it. And by the way, would I like to sign up for their patreon and maybe one of their online courses?
What I don't understand about this explanation is that Google's results are abysmal compared to e.g. DuckDuckGo or even Brave search. (I haven't tried Kagi, but people here rave about it as well.) Sure, all the SEO is targeting googlebot, but Google has by far more resources to mitigate SEO spam than just about anyone else. If this is the full explanation, couldn't Google just copy the strategies the (much) smaller rivals are using?
Have you read the article this thread is about?
To summarize it: Google reverted an algorithm that detected SEO spams in 2019.
(Note that I never work for Google and I don't know whether it's true or not. It's just what this article says.)
I wasn't responding to the article; I was responding to the claim that Google's results are bad because of all the SEO. It's a claim I've heard from Google apologists including some people I know at Google. I think it's nonsense both for the reasons I stated and for the reasons enumerated in the article.
You are totally correct I think.
This isn't about what is possible.
It is about Google not wanting to say goodbye to the sweet dollars from spammy sites.
Otherwise making the probably number one requested feature, a personal block list, wouldn't have been impossible for a company with so many bright minds.
I mean: little bootstrapped Kagi had it either from the beginning or at least since shortly after they launched.
People always think they lost against SEO spam. But my main reason for quitting as soon as an alternative showed up was because they started to overrule my searches and search for what they thought I wanted to search for.
For a while I kept it at bay by using doublequotes and verbatim but none of those have worked reliably for a decade now.
That isn't SEO spam. That is poor engineering or "we know better than you" attitude.
When a large search engine deranks spam websites, the spam websites complain! Loudly! With Google they have a big juicy target with lots of competing ventures for an antitrust case; no such luck for Kagi or DDG.
This is an interesting theory. Is there evidence that it's happening? Is Big SEO unreasonably effective at lobbying the Justice Department?
It’s definitely a concern where I work (not Google). Deranking anybody who happens to share a vertical we’re in is colorable as an anticompetitive action[0], and due to our dominance in another sector (not search), effectively any anticompetitive action anywhere is a no-go. And since we don’t have time to review whether a particular competitor also competes in one of our verticles and run everything by legal, nothing gets de-ranked manually.
0: for context, us doj does not take antitrust action against companies simply for market dominance; it requires market dominance plus an anticompetitive action. However, they don’t like monopolies, so effectively any pretext can be used — see the apple lawsuit or the 90s ms lawsuits for how little it takes.
I've been using Kagi for a while, and I find that it delivers better results in a cleaner presentation.
I think a case can be made that the spam problem can be traced all the way back to Google buying Doubleclick.
Its really easy to spot the crap websites that are scaping content-creating websites ... because they monetize by adding ads.
If Google was _only_ selling ads on the search results page, then it could promote websites that are sans ads.
Instead, it is incentivised to push users to websites that contain ads, because it also makes money there.
And that means scraping other sites to slap your ads onto them can be very profitable for the scammers.
We need a Reverse Google search that will weed out the garbage.
https://kagi.com/ de-prioritizes SEO ad sites and also lets you blacklist sites from your search reaults. Never going back to google after trying it
They hired people who introduce Jack Welch methods.
This is like in that Steve Jobs video about product people being kicked out and exchanged by ones who dont care about product:
https://m.youtube.com/watch?v=P4VBqTViEx4
They will not make good search. That is not their priority.
Correction: It is Google's willingness to display search results by what is MOST PROFITABLE and/or POLITICALLY expedient to them that killed search. This includes their willingness to promote/demote content based on what the thought police wanted to promote/demote.
This is the correct insight. Google has enough machine learning prowess that they could absolutely offload, with minimal manhours, the creation of a list ranking a bunch of blogspam sites and give them a reverse score by how much they both spam articles or how much they spread the content over the page. Then apply that score to their search result weights.
And I know they could because someone did make that list and posted it here last year.
I'm waiting for folks to implement a Reverse Google Search.
A bit chicken-and-egg. Another perspective: Google’s system incentivizes SEO spam.
Search for a while hasn’t been about searching the web as much as it has been about commerce. It taps commercial intent and serves ads. It is now an ad engine; no longer a search engine.
Absolutely this. I don't think many people consider how odd it is that the largest internet advertising company in the world and the largest search engine company in the world are one and the same, and just how overt a conflict of interest that is, so far as providing quality service goes. It would be akin to if the largest telephone service company in the world was also the largest phone maker in the world. Oh wait, that did happen [1] - and we broke them up because it's obviously extremely detrimental to the functioning of a healthy market.
[1] - https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System
Best exercise bike articles, and such, are what lots of people people actually search for. There is no incentive to provide quality work which answers these queries hence the abundance of spam and ads.
If you want to purchase consumer products at your own expense and offer an impartial opinion on each of them then you will have no problem getting ranked highly on google. You will lose a lot of money doing so, however, and will also be plagiarized to death in a month. The sites you want to be rid of will outrank you for your own content, I have been there and have the t-shirt.
SEO Spam didn't kill search so much as Google failed to retain Matt Cutts or replicate his community involvement https://www.searchenginejournal.com/matt-cutts-resigns-googl...
What did he used to do ? Your comment seems contradictory cutts seem to be on anti spam but your comment implies seo did not kill search . Is seo not part of spam?
Even when matt_cutts used to be here it was still impossible to get him (or anyone else) to care about search results including lots of results I never asked for.
Not low quality pages that spammed high ranking words but pages that simply wasn't related to the query at all as evidenced by the fact that they didn't contain the keywords I searched for at all!
I've heard this argument again and again, but I never see any explanation as to why SEO is suddenly in the lead in this cat-and-mouse game. They were trying ever since Google got 90%+ market share.
I think it's more likely that Google stopped really caring.
Well yeah, it's in the article - at some point, they switched completely to metrics (i.e. revenue) driven management and forgot that it's the quality of results that actually made Google what it is. And, with a largely captive audience (Google being the default-search-engine-that-most-people-don't-bother-or-don't-know-how-to-change in Chrome, Android, on Chromebooks etc.), they arguably don't have to care anymore...
Hard disagree. As another reply mentions, just compare the alternatives such as Kagi that aren’t breaking search by pursuing ad growth.
Kagi isn't amazing, it's just not bad and it really makes plain how badly Google has degraded into an ad engine. All it takes to beat Google is giving okay quality search results.
Machine learning is probably as much or even more susceptible to SEO spam.
Problem is that the rules of search engines created the dubious field of SEO in the first place. They are not entirely the innocent victim here.
Arcane and intransparent measures get you ahead. So arcane that you instantly see that it does not correspond with quality content at all, which evidently leads to a poor result.
I wish there was an option to hide every commercial news or entertainment outlet completely. Those are of course in on SEO for financial reaesons.
There's alway plugins or you can subscribe to Kagi, although I don't think there's any blocklist preconfigured for "all commercial news websites"
It's like "Do some SEO magic and Tada!"
And who forgot the recent Reddit story.
Could you link it please? I have unfortunately no idea what you are referencing
I don't know, but Youtube seems to have a more solid algorithm. I'm typically not subscribed to any channel, yet the content I want to watch does find me reasonably well. Of course, heavily promoted material also, but I just click "not interested in channel" and it disappears for a while. And I still get some meaningful recommendations if I watch a video in a certain topic. Youtube has its problems, of course, but in the end I can't complain.
I don't think youtube is trying that hard to desperately sell stuff to you via home screen recommendation algorithm. And I agree its bearable and what you describe works cca well, albeit ie I am still trying to get rid of anything related to Jordan Peterson whom I liked before and detest now after his drug addiction / mental breakdown, it just keeps popping back from various sources, literal whack-a-mole.
I wish there was some way to tell "please ignore all videos that contain these strings, and I don't mean only for next 2 weeks".
Youtube gets their ads revenue from before/during video, so they can be nicer to users.
Much agreed, and this is prompting me to experiment with other search engines to see if they cut off also the interesting humans sites. With todays google I feel herded.
Most of the problems I complain about are not related to SEO spam but to Google including sites that does not contain my search terms anywhere despite my use of doublequotes and the verbatim operator.
As for SEO spam a huge chunk of it would have disappeared I think if Google had created the much requested personal blacklist that we used to ask them for.
It was always "actually much harder than anyone of you who don't work here can imagine for reasons we cannot tell or you cannot understand" or something like that problem, but bootstraped Kagi managed to do it - and their results are so much better that I don't usually need it.
This explodes for search terms dealing with questions related to bugs or issues or how to dos. Almost all top results are YT videos, each of which will follow the same pattern. First 10 secs garbage followed by request for subscribe and/or sponsorship content then followed by what you want.
spam didn't kill search. Google willingness to promote spam for ads killed Google. Google is not search.
For me what killed search was 2016, after that year if some search term is "hot news" it becomes impossible to learn anything about it that wasn't published in the last week and you just get the same headline repeated 20 times in slightly different wording about it.
After that I only use search for technical problems, and mouth to mouth or specific authors for everything else.
Thanks for writing this insightful piece.
The pathologies of big companies that fail to break themselves up into smaller non-siloed entities like Virgin Group does. Maintaining the successful growing startup ways and fighting against politics, bureaucracy, fiefdoms, and burgeoning codebases is difficult but is a better way than chasing short-term profits, massive codebases, institutional inertia, dealing with corporate bullshit that gets in the way of the customer experience and pushes out solid technical ICs and leaders.
I'm surprised there aren't more people on here who decide "F-it, MAANG megacorps are too risky and backwards not representative of their roots" and form worker-owned co-ops to do what MAANGs are doing, only better, and with long-term business sustainability, long tenure, employee perks like the startup days, and positive civil culture as their central mission.
I formed a worker co-op - but it's just me! And I do CAD reverse engineering, nothing really life-giving.
I would love to join a co-op producing real human survival values in an open source way. Where would you suggest that I look for leads on that kind of organization?
Let's start with replacing Google. Count me in.
While DDG, Brave, Kagi etc are working generously to replace Google search. The other areas that I think get less attention and needs to be targeted to successfully dismantle them and their predatory practices are Google maps and Google docs.
Maps are hard because it requires a lot of resources and money and whatever but replacing docs should be relatively easier.
(paid user of Kagi here)
FWIW, Kagi is built on top of Google search, so yes it's "replacing" (for you and me) a dependence on Google search, but it is categorically not a from-the-ground-up replacement for Google search.
Oh that's pretty smart
Using OSS to commoditize complements plays a big role in breaking up big advantages.
There is big tech open source consortium working on maps now to commoditize it: https://siliconangle.com/2022/12/15/aws-microsoft-meta-tomto...
Not sure it'll work. I think half the advantage comes from the integration across all these tools (maps, search, etc). Have you ever tried to use duckduckgo? It surprised me what I take for granted in Google's user experience.
I would imagine GitHub and technology social media
What's odd to me is how everything is so metricized. Clearly over metricization is the downfall of any system that looks meritocratic. Due to the limitations of metrics and how they are often far easier to game than to reach through the intended means.
An example of this I see is how new leaders come in and hit hard to cut costs. But the previous leader did this (and the one before them) so the system/group/company is fairly lean already. So to get anywhere near similar reductions or cost savings it typically means cutting more than fat. Which it's clear that many big corps are not running with enough fat in the first place (you want some fat! You just don't want to be obese!). This seems to create a pattern that ends up being indistinguishable from "That worked! Let's not do that anymore."
Agree you have to mix qualitative with the quantitative, but the best metrics systems don't just measure one quantity metric. They should be paired with a quality metric.
Example: User Growth & Customer Engagement
Have to have user growth and retention. If you looked at just one or the other, you'd be missing half the equation.
I think that a good portion of the problem is that there are groups involved in metrics:
1) People setting the metrics
2) People implementing/calculating the metrics
3) People working on improving the metrics (ie product work)
2 is specially complicated for a lot of software products because it can some times be really hard to measure and can be tweaked/manipulated. For example, the MAU twitter figures from the buyout that Musk keeps complaining about, or Blizzard constantly switching their MAU definition.
Often 2 and 3 are the same people and 1 is almost always upper management. I argue that 1 and 2 should be a single group of people (that doesn't work on the product at all) and not directly subject to upper management and not tracked by the same metrics they implement (or tracked by any metrics at all).
The hard part about starting worker owned co-ops is financing. We need good financing systems for them. People/firms who are willing to give loans for a reasonable interest, but on the scale of equity investment in tech start ups.
The problem is risk —- most new businesses will go under. Who’s going to take on that unreasonable risk without commensurate reward (high interest loan rate, if any, or equity).
Co-ops could go the angel/VC route for funding if they don’t give up a controlling share.
I guess it depends on how much equity you own as to what is better (to your first paragraph), and how large the paycheck is (to the 2nd paragraph.
Problem is, worker owned co-ops would still require money to do anything even remotely competitive to existing businesses.
So... people go walk up for handouts from VCs....and the story begins lol.
There's a lot of ML hate here, and I simply don't see the alternative.
To rank documents, you need to score them. Google uses hundreds of scoring factors (I've seen the number 200 thrown about, but it doesn't really matter if it's 5 or 1000.) The point is you need to sum these weights up into a single number to find out if a result should be above or below another result.
So, if:
Are there any humans out there who want to write a traditional forward-algorithm to tell me which result is better?You do not need to. Counting how many links are pointing to each document is sufficient if you know how long that link existed (spammers link creation time distribution is widely differnt to natural link creation times, and many other details that you can use to filter out spammers)
Ranking means deciding which document (A or B) is better to return to the user when queried.
Not writing a traditional forward-algorithm to rank these documents implies one of the following:
- You write a "backward" algorithm (ML, regression, statistics, whatever you want to call it).
- You don't use algorithms to solve it. An army of humans chooses the rankings in real time.
- You don't rank documents at all.
- Link-counting (e.g. PageRank) is query-independent evidence. If that's sufficient for you, you'll always return the same set of documents to each user, regardless of what they typed into the search box.
At best you've just added two more ranking factors to the mix:
So I ask again:- Which document matches your query better, A or B?
- How did you decide that, such that not only can you program a non-ML algorithm to perform the scoring, but you're certain enough of your decision that you can fix the algorithm when it disagrees with you ( >> debuggable and understandable by human search engineers )
A few minor nitpicks. Pagerank is not just link counting, who is linking to the page matters. Among the linking pages those that are ranked higher matter more -- and how does one figure out their rank ? its by Pagerank. It may sound a bit like chicken and egg but its fine, its the fixed point of the self-referential. definition.
Pagerank based ranking will not return the same set of pages. Its true that the ranking is global in vanilla version of Pagerank, but what gets returned in rank order is the set of qualifying pages. The set of qualifying pages are very much query sensitive. Pagerank also depends on a seed set of initial pages, these may also be set on a query dependent way.
All this is a little moot now, because Pagerank even defined in this way stopped being useful a long time ago.
Statistical methods are debuggable. Is PageRank not debuggable? I am not sure where ML starts and statistics end.
Yes, this is a statistical method. Guess what machine learning is and what it actually excels?
simplicity is always the recipe for success, unfortunately, most engineers are drawn to complexity like moth to fire
if they were unable to do some AB testing between a ML search and a non-ML search, they deserve their failure 100%
there are not enough engineers blowing the whistle against ML
Unfortunately, Google evaluates employees by the complexity of their work. "Demonstrates complexity" is a checkbox on promo packets, from what I've heard.
Naturally, every engineer will try to over-complicate things just so they can get the raises and promos. You get what you value.
I've heard a similar critique for Google launching new products and then letting them die, where it's really driven by their policies and practice around what gets someone a promotion and what doesn't.
Engineers love simplicity but management hates it and won’t promote people that strive towards it. A simple system is the most complex system to design.
I definitely think the ML search results are much worse. But complexity or not, strategically it's an advantage for the company to use ML in production over a long period of time so they can develop organizational expertise in it.
It would have been a worse outcome for Google if they had stuck to their no ML stance and then had Bing take over search because they were a generation behind in technology.
Amit was definitely against ML, long before "AI" had become a buzzword.
He wasn't the only one. I built a couple of systems there integrated into the accounts system and "no ML" was an explicit upfront design decision. It was never regretted and although I'm sure they put ML in it these days, last I heard as of a few years ago was that at the core were still pages and pages of hand written logic.
I got nothing against ML in principle, but if the model doesn't do the right thing then you can just end up stuck. Also, it often burns a lot of resources to learn something that was obvious to human domain experts anyway. Plus the understandability issues.
i worked on ranking during singhal's tenure, and it was definitely refreshing to see a "no black box ML ranking" stance.
@gregw134 Thank you for sharing! I've never worked at Google, but really curious what the engineering context is when you say "needs a launch" in the last line.
I'm glad you shared this.
My priors before reading this article were that an uncritical over-reliance on ML was responsible for the enshittification of Google search (and Google as a whole). Google seemed to give ML models carte blanche, rather than using the 80-20 rule to handle the boring common cases, while leaving the hard stuff to the humans.
I now think it's possible both explanations are true. After all, what better way to mask a product's descent into garbage than more and more of the core algorithm being a black box? Managers can easily take credit for its successes and blame the opacity for failures. After all, the "code yellow" was called in the first place because search growth was apparently stagnant. Why was that? We're the analysts manufacturing a crisis, or has search already declined to some extent?