return to table of content

Testing Generative AI for Circuit Board Design

HanClinto
82 replies
1d1h

This feels like an excellent demonstration of the limitation of zero-shot LLMs. It feels like the wrong way to approach this.

I'm no expert in the matter, but for "holistic" things (where there are a lot of cross-connections and inter-dependencies) it feels like a diffusion-based generative structure would be better-suited than next-token-prediction. I've felt this way about poetry-generation, and I feel like it might apply in these sorts of cases as well.

Additionally, this is a highly-specialized field. From the conclusion of the article:

Overall we have some promising directions. Using LLMs for circuit board design looks a lot like using them for other complex tasks. They work well for pulling concrete data out of human-shaped data sources, they can do slightly more difficult tasks if they can solve that task by writing code, but eventually their capabilities break down in domains too far out of the training distribution.

We only tested the frontier models in this work, but I predict similar results from the open-source Llama or Mistral models. Some fine tuning on netlist creation would likely make the generation capabilities more useful.

I agree with the authors here.

While it's nice to imagine that AGI would be able to generalize skills to work competently in domain-specific tasks, I think this shows very clearly that we're not there yet, and if one wants to use LLMs in such an area, one would need to fine-tune for it. Would like to see round 2 of this made using a fine-tuning approach.

surfingdino
51 replies
1d

This feels like an excellent demonstration of the limitation of zero-shot LLMs. It feels like the wrong way to approach this.

There is one posted on HN every week. How many more do we need to accept the fact this tech is not what it is sold at and we are bored waiting for it get good? I am not say "get better", because it keeps getting better, but somehow doesn't get good.

makk
15 replies
22h56m

That’s a perception and the problem isn’t the AI it’s human nature: 1. every time AI is able to do a thing we move the goalposts and say, yeah, but it can’t do that other thing over there; 2. We are impatient, so our ability to get bored tends to outpace the rate of change.

127
6 replies
19h0m

What goals were achieved that I missed? Even for creative writing and image creation it still requires significant human guidance and correction.

selestify
5 replies
17h28m

This is a great example of goalposts shifting. Even having a model that can engage in coherent conversation and synthesize new information on the fly is revolutionary compared to just a few years ago. Now the bar has moved up to creativity without human intervention.

doe_eyes
3 replies
16h39m

But isn't this goalpost shifting actually reasonable?

We discovered this nearly-magical technology. But now the novelty is wearing off, and the question is no longer "how awesome is this?". It's "what can I do with it for today?".

And frustratingly, the apparent list of uses is shrinking, mostly because many serious applications come with a footnote of "yeah, it can do that, but unreliably and with failure modes that are hard for most users to spot and correct".

So yes, adding "...but without making up dangerous nonsense" is moving the goalposts, but is it wrong?

selestify
1 replies
11h36m

IMO it’s not wrong to want the next improvement (“…but without making up dangerous nonsense”), but it is disingenuous to pretend as if there hasn’t already been a huge leap in capabilities. It’s like being unimpressed with the Wright brothers’ flight because nobody has figured out commercial air travel yet.

surfingdino
0 replies
9h28m

The leap has indeed been huge, but it's still not useful for any anything. The Wright brothers did not start a passenger airline after the first try.

kenjackson
0 replies
11h38m

There are a lot of things where being reliable isn’t as important (or it’s easier to be reliable).

For example, we are using it to do meeting summaries and it is remarkably good at it. In fact, in comparison to humans we did A/B testing with - usually better.

Another thing is new employee ramp. It is able to answer questions and guide new employees much faster than we’ve ever seen before.

Another thing I’ve started toying with it with, but have gotten incredible results so far is email prioritization. Basically letting me know which emails I should read most urgently.

Again, these were all things where the state of the art was basically useless 3 years ago.

127
0 replies
9h37m

No it's not. You can not shift goalposts that do not exist in the first place.

goatlover
3 replies
20h43m

The other side of this coin is everyone overhyping what AI can do, and when the inevitable criticism comes, they respond by claiming the goal posts are being moved. Perhaps, but you also told me it could do XYZ, when it can only do X and some Y, but not much Z, and it’s still not general intelligence in the he broad sense.

refulgentis
1 replies
20h35m

I appreciate this comment because I think it really demonstrates the core problem with what I'll call the "get off my lawn >:|" argument, because it's avowedly about personal emotions.

It's not "general intelligence", so it's over hyped, and They get so whiny about the inevitable criticism, and They are ignoring that it's so mindnumbingly boring to have people making the excuse that "designed a circuit board from scratch" wasn't something anyone thinks or claims an LLM should do.

Who told you LLMs can design circuit boards?

Who told you LLMs are [artificial] general intelligence?

I get sick of it constantly being everywhere, but I don't feel the need to intellectualize it in a way that blames the nefarious ???

ben_w
0 replies
20h11m

Who told you LLMs are [artificial] general intelligence?

*waves*

Everyone means a different thing by each letter of AGI, and sometimes also by the combination.

I know my opinion is an unpopular one, but given how much more general-purpose they are than most other AI, I count LLMs as "general" AI; and I'm old enough to remember when AI didn't automatically mean "expert level or better", when it was a surprise that Kasparov was beaten (let alone Lee Sedol).

LLMs are (currently) the ultimate form of "Jack of all trades, master of none".

I'm not surprised that it failed with these tests, even though it clearly knows more about electronics than me. (I once tried to buy a 220 kΩ resistor, didn't have the skill to notice the shop had given me a 220 Ω resistor, the resistor caught fire).

I'd still like to call these things "AGI"… except for the fact that people don't agree on what the word means and keep objecting to my usage of the initials as is, so it would't really communicate anything for me to do so.

derefr
0 replies
19h33m

ML scientists will tell you it can do X and some Y but not much Z. But the public doesn’t listen to ML scientists. Most of what the public hears about AI comes from businessmen trying to market a vision to investors — a vision, specifically, of what their business will be capable of five years from bow given predicted advancements in AI capabilities in the mean time; which has roughly nothing to do with what current models can do.

slg
1 replies
20h58m

I don’t think the problem is moving the goalposts, but rather there are no actual goalposts. Advocates for this technology imply it can do anything either because they believe it will be true in the near future or they just want others to believe it for a wide range of reasons including to get rich of it. Therefore the general public has no real idea what the ideal use cases are for this technology in its current state so they keep asking it to do stuff it can’t do well. It is really no different than the blockchain in that regard.

surfingdino
0 replies
10h11m

One of the main issues I see amongst advocates of AI is that they cannot quantify the benefits and ignore provable failings of AI.

jodrellblank
0 replies
4h24m

"1. every time AI is able to do a thing we move the goalposts and say, yeah, but it can’t do that other thing over there"

So are you happy that a 1940s tic-tac-toe computer "is AI"? And that's going to be your bar for AI forever?

"Moving the goalposts is a metaphor, derived from goal-based sports such as football and hockey, that means to change the rule or criterion of a process or competition while it is still in progress, in such a way that the new goal offers one side an advantage or disadvantage." - and the important part about AI is that it be easy for developers to claim they have created AI, and if we move the goalposts then that's bad because ... it puts them at an unfair disadvantage? What is even wrong with "moving the goalposts" in this situation, claiming something is/isn't AI is not a goal-based sport. The metaphor is nonsensical whining.

XorNot
0 replies
17h36m

No I'd say it's that people are very bad at knowing what they want, and worse at knowing how to get it.

While it might be "moving the goal posts" the issue is that the goal posts were arbitrary to start with. In the context of the metaphor we put them on the field so there could be a game, despite the outcome literally not mattering anywhere else.

This isn't limited to AI: anyone dealing with customers knows that the worst thing you can do is take what the customer says their problem is at face value, replete with the proposed solution. What the customer knows is they have a problem, but it's very unlikely they want the solution they think they do.

echelon
14 replies
21h58m

I'm in awe of the progress in AI images, music, and video. This is probably where AI shines the most.

Soon everything you see and hear will be built up through a myriad of AI models and pipelines.

slg
10 replies
20h54m

Soon everything you see and hear will be built up through a myriad of AI models and pipelines.

It is so bizarre that some people view this as a positive outcome.

echelon
7 replies
19h18m

These are tools. Humans driving the tools have heart and soul and create things of value through their lens.

Your argument rhymes with:

- "Let's keep using horses. They're good enough."

- "Photography lacks the artistic merit of portrait art."

- "Electronic music isn't music."

- "Vinyl is the only way to listen to music."

- "Digital photography ruins photography."

- "Digital illustration isn't real illustration and tablets are cheating."

- "Video games aren't art."

- "Javascript developers aren't real programmers."

Though I'm paraphrasing, these are all things that have been said.

I bet you my right kidney that people will use AI to produce incredible art that will one day (soon) garner widespread praise and accolade.

It's just a tool.

slg
2 replies
18h47m

The specific phrase used was "everything you see and hear" (emphasis mine). You weren't arguing this would be an optional tool that could be used in the creation of art. You were arguing that this will replace all other art. That isn't an argument that photography is an art equal to painting, it is an argument for it to replace painting.

echelon
1 replies
14h4m

You were arguing that this will replace all other art.

The population of people who want to create art is higher than the people who have the classical skills. By sheer volume, the former will dominate the latter. And eventually most artists will begin to use AI tools when they realize that's what they are -- tools.

slg
0 replies
1h46m

Now combine that with the photography and painting analogy that you made in the previous post. Photography was invented some 2 centuries ago. Do you think the world would be better if every painter of that era, including the likes of van Gogh and Picasso, picked up a camera instead of a paintbrush?

ziml77
1 replies
15h29m

Surely there's some point where it ceases being a tool though. We can't both be making AIs out to be comparable to humans while simultaneously calling them tools. Otherwise people who commission art would be considered artists using a tool.

vasco
0 replies
12h44m

Many many successful artists from the Renaissance until today are not actually artists but just rich people with a workshop full of actual artist they commission works from. The rich person curates.

Many times this also happens with artists themselves. After a point, you are getting way more commissions than you can produce yourself, so you employ a small army of understudies that learn your techniques and make your pieces for you. So what you describe has existed for hundreds of years.

A short list could include old ones like Rembrandt or Rubens and a new ones like Jeff Koons or Damien Hirst.

vunderba
1 replies
17h45m

Just to play devil's advocate - I'm surprised you (and many other people apparently) are unable to tell the operative difference between something like:

1. (real illustration vs digital illustration)

2. (composing on sheet music vs composing in a DAW)

and

3. illustration vs Stable Diffusion

4. composing vs generative music models such as Suno

What's different is the wide disparity between input and output. Generally, art has traditionally had a closer connection between the "creator" and the "creation". Generative models have married two conventionally highly disparate mediums together, e.g. text to image / text to audio.

If you have zero artistic ability, you'd have about as much success using Photoshop as you would with traditional pencil and paper.

Whereas any doofus can type in the description of something along with words like "3D", "trending on artstation", "hyper-realistic,", and "4K" and then proceed to generate thousands of images in automatic1111 which they can flood DeviantArt with in a single day.

The same applies to music composition whether you are laboriously notating with sheet music or dropping notes using a horizontal tracker in a DAW like Logic. If you're not a musician, the fanciest DAW in the world won't make you one.

echelon
0 replies
14h11m

I don't think you realize the sheer scale of people that are working their asses off to leverage AI in their work in creative ways, often times bending over backwards to get it to work.

I spent 48 hours two weeks back (with only a few hours of sleep) making an AI film. I used motion capture, rotoscoping, and a whole host of other tools to accomplish this.

I know people who have spent months making AI music videos. People who painstakingly mask and pose skeletons. People who design and comp shots between multiple workflows.

These are tools.

zo1
1 replies
18h12m

What I find bizarre is people gatekeeping the process that helps get things from imagination onto canvas.

Artists and "creative" people have long held a monopoly on this ability and are now finally paying the price now that we've automated them away and made their "valuable" skill a commodity.

happypumpkin
0 replies
42m

Artists and "creative" people have long held a monopoly on this ability and are now finally paying the price

I've seen a lot of schadenfreude towards artists recently, as if they're somehow gatekeeping art and stopping the rest of us from practicing it.

I really struggle to understand it; the barrier of entry to art is basically just buying a paper and pencil and making time to practice. For most people the practice time could be spent on many things which would have better economic outcomes.

monopoly

Doesn't this term imply an absence of competition? There seems to be a lot of competition. Anyone can be an artist, and anyone can attempt to make a living doing art. There is no certification, no educational requirements. I'm sure proximity to wealth is helpful but this is true of approximately every career or hobby.

Tangentially, there seem to be positive social benefits to everyone having different skills and depending on other people to get things done. It makes me feel good when people call me up asking for help with something I'm good at. I'm sure it feels the same for the neighborhood handyman when they fix someone's sink, the artist when they make profile pics for their friends, etc. I could be wrong but I don't think it'll be entirely good for people when they can just have an AI or a robot do everything for them.

goatlover
1 replies
20h39m

I sincerely hope not. Talk about a dystopian future. That’s even worse than what social media has become.

wewtyflakes
0 replies
19h6m

Why would that be describing a dystopian future? A more generous framing might be to say that incredibly creative feats will be available to more people, and those who are particularly talented will create things that are now beyond our imagination using these tools. Who knows if that is how it will actually play out, but it also does not seem unreasonable to think that it might.

ben_w
0 replies
20h29m

They already are, when using the meaning of "AI" that I grew up with.

The Facebook feed is AI; Google PageRank is AI; anti-spam filters are AI; A/B testing is AI; recommendation systems are AI.

It's been a long time since computers took over from humans with designing transistor layouts in CPUs — I was hearing about the software needing to account for quantum mechanics nearly a decade ago already.

refulgentis
13 replies
21h46m

There's this odd strain of thought that there's some general thing that will pop for hucksters and the unwashed masses, who are sheep led along by huckster wolves who won't admit LLMs aint ???, because they're profiting off it

It's frustrating because it's infantalizing, it derails the potential of an interesting technical discussion (ex. Here, diffusion), and it misses the mark substantially.

At the end of the day, it's useful in a thousand ways day to day, and the vast majority of people feel this way. The only people I see vehemently arguing the opposite seem to assume only things with 0 error rate are useful or are upset about money in some form.

But is that really it? I'm all ears. I'm on a 5 hour flight. I'm genuinely unclear on whats going on that leads people to take this absolutist position that they're waiting for ??? to admit ??? about LLMs.

Yes, the prose machine didnt nail circuit design, that doesn't mean whatever They you're imagining needs to give up and accept ???

shermantanktop
8 replies
21h21m

So what should we make of the presence of actual hucksters and actual senior execs who are acting like credulous sheep? I see this every day in my world.

At the same time I do appreciate the actual performance and potential future promise of this tech. I have to remind myself that the wolf and sheep show is a side attraction, but for some people it’s clearly the main attraction.

refulgentis
6 replies
20h42m

The wolves/sheep thing was to indicate how moralizing and infantalizing serves as a substitute for actually explaining what the problem is, because surely, it's not that the prose machine isn't doing circuit design.

I'm sure you see it, I'd just love for someone to pause their internal passion play long enough to explain what they're seeing. Because I refuse to infantalize, I refuse to believe it's just grumbling because its not 100% accurate 100% of the time, and doesn't do 100% of everything.

shermantanktop
3 replies
19h42m

I am literally right now explaining to a senior exec why some PR hype numbers about developer productivity from genAI are not comparable to internal numbers, because he is hoping to say to his bosses that we’re doing better than others. This is a smart, accomplished person, but he can read the tea leaves.

The problem with hype is that it can become a pathological form of social proof.

anoncareer0212
2 replies
18h59m

I see, I'm sorry that's happening :/ I was lucky enough to transition from college dropout waiter to tech startup on the back of the iPad, 6 years in, sold it and ended up at still-good 2016 Google. Left in 2023 because of some absolutely mindnumbingly banal-ly evil middle management. I'm honestly worried about myself because I cannot. stand. that. crap., Google was relatively okay, and doubt I could ever work for someone else again. it was s t u n n i n g to see how easily people slip into confirmation bias when it involves pay / looking good.

fwiw if someone's really into Google minutae: I'm not so sure it is relatively okay anymore, it's kinda freaky how many posts there are on Blind along the lines of "wow I left X for here, assumed i'd at least be okay, but I am deeply unhappy. its much worse than average-white-collar job I left"

selestify
1 replies
17h24m

Are there any write ups of the newly evil Google experience I can read about? When did things shift for you in the 2016 - 2023 timeframe?

refulgentis
0 replies
17h11m

No, my way of dealing with it is to whine on HN/twitter occasionally and otherwise don't say anything publicly. Feel free to reach out at jpohhhh@gmail, excuse the overly familiar invitation, paying it forward because I would have found talking about that sort of thing f a s c i n a t i n g.

in general id recommend Ian Hickson's blog post on leaving. I can't remember the exact quote that hit hard, something like decisions moved from being X to Y to Z to being for peoples own benefit.

I'd also add there was some odd corrupting effects from CS turning into something an aimless Ivy Leaguer would do if they didn't feel like finance.

photonthug
1 replies
19h37m

I’ll play along. The thing that’s annoying me lately is that session details leaking between chats has been enabled as a “feature”, which is quickly making ChatGPT more like the search engine and social media echo chambers that I think lots of us want to escape. It’s also harmful for the already slim chances of having reproducible / deterministic results, which is bad since we’re using these things for code generation as well as rewriting emails and essays or whatever.

Why? Is this naive engineering refusing to acknowledge the same old design flaws? Nefarious management fast tracking enshittification? Or do users actually want their write-a-naughty-limerick goofs to get mixed up with their serious effort to fast track circuit design? I wouldn’t want to appear cynical but one of these explanations just makes more sense than the others!

The core tech such as it is is fine, great even. But it’s not hard to see many different ways that it’s already spiraling out of control.

refulgentis
0 replies
19h11m

(thank you!) 100% cosign. It breaks my. goddamn. heart. that [REDACTED], the consummate boring boneheaded SV lackey is [REDACTED] of [REDACTED], and can't think outside 6 week sprints and never finishes launching. This is technology that should be freeing us from random opaque algorithmic oppression and enabling us to take charge if we want. I left Google to do the opposite, and I'm honestly stunned that it's a year later and there's nothing on the market that challenges that. Buncha me-too nonsense doing all the shit I hate from the 2010s: bulk up on cash, buy users, do the recurring revenue thing and hope x > y, which inevitably, it won't be.

wruza
0 replies
20h57m

Why should we even?

The problem with everything today is not only that it’s hype-centric, but that that carries away those who were otherwise reasonable. AI isn’t any special in this regard, it’s just “crypto” of this decade.

I see this trend everywhere, in tech, socio, markets. Everything is way too fake, screamy and blown out of proportion.

ben_w
3 replies
21h25m

But is that really it? I'm all ears. I'm on a 5 hour flight. I'm genuinely unclear on whats going on that leads people to take this absolutist position that they're waiting for ??? to admit ??? about LLMs.

Irony: humans think in very black-and-white terms, one could even say boolean; conversely LLMs display subtly and nuance.

When I was a kid, repeats of Trek had Spock and Kirk defeating robots with the liar's paradox, yet today it seems like humans are the ones who are broken by it while the machines are just going "I understood that reference!"

goatlover
1 replies
20h41m

And yet we still don’t have Data or the Holographic Doctor.

ben_w
0 replies
20h37m

You're demonstrating my point :)

When we get to that level, we're all out of work.

In the meantime, LLMs are already basically as good as the scriptwriters made the TNG-VOY era starship computers act.

refulgentis
0 replies
20h44m

Excellent point, it really is what it comes down to. There's people getting hoodwinked and people hoodwinking and then me, the one who sees them for what they are.

exe34
4 replies
22h46m

how long does it take for a child to start doing surgery? publishing novel theorems? how long has the humble transformer been around?

surfingdino
1 replies
10h15m

Nobody is telling an experienced heart surgeon to step aside and let a child plan an open heart surgery. And yet, AI and LLMs in particular are being sold as the tools that can do complex tasks like that. But let's leave complex tasks and have a look at marketing behind one of the tools that's aimed at business. The messaging of one of the ads I'm seeing promises that the tools in question can summarise a 150-page long document into a 5-slide presentation. Now, that sounds amazing, if we ignore the fact that a person who wrote a 150-page document has already prepared an outline and is perfectly capable of summarising each section of the document. Writing a 150-page document without a plan and not being able to organise would mean that people have evolved into content generators that need machines to help them write tables of contents and reformat them into a presentation. Coming back to your child analogy, why would a child be better at summarising content it knows nothing about that the person who wrote it?

exe34
0 replies
7h28m

we do get consultants coming into companies and telling the experienced professionals how to screw up stuff all the time though. i think there are laws with teeth and of course the immediate body to get rid of that helps surgeons maintain the integrity of their profession. when the outcome is far removed from the decision, you do get people like ministers meddling in things they don't understand and leave the consequences for the next administration.

ben_w
1 replies
21h14m

Wall-clock or subjective time?

I think it would take a human about 2.6 million (waking) years to actually read Common Crawl[0]; though obviously faster if they simply absorb token streams as direct sensory input.

The strength of computers is that transistors are (literally) faster than synapses to the degree to which marathon runners are faster than continental drift; the weakness is they need to, too — current generation AI is only able to be this good due to this advantage allowing it to read far more than any human.

How much this difference matters depends on the use-case: if AI were as good at learning as we are, Tesla's FSD would be level 5 autonomy years ago already, even with just optical input.

[0] April 2024: 386 TiB; assuming 9.83 bits per word and 250 w.p.m: https://www.wolframalpha.com/input?i=386+TiB+%2F+9.83+bits+p...

TeMPOraL
0 replies
19h55m

Subjective time doesn't really matter unless something is experiencing it. It could be 2.6 million years, but if the wall-clock time is half a year, then great - we've managed to brute-force some degree of intelligence in half a year! And we're at the beginning of this journey; there surely are many things to optimize that will decrease both wall-clock and subjective training time.

As the saying goes - "make it work, make it right, make it fast".

Kiro
0 replies
4h10m

This post supports your case way less than you think. I've sent it to several EE friends and none have expressed your discontent. The general consensus has been "amazing what AI can do nowadays", and I agree. This would have been complete science-fiction just a couple of years ago.

DHaldane
22 replies
1d1h

My gut agrees with you that LLMs shouldn't do this well on a specialty domain.

But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later.

Overall not sure what to expect, but fine tuning experiments would be interesting regardless.

cjk2
14 replies
1d1h

I doubt it'd work any better. Most of EE time I have spent is swearing at stuff that looked like it'd work on paper but didn't due to various nuances.

I have my own library of nuances but how would you even fine tune anything to understand the black box abstraction of an IC to work out if a nuance applies or not between it and a load or what a transmission line or edge would look like between the IC and the load?

This is where understanding trumps generative AI instantly.

DHaldane
8 replies
1d1h

I doubt it too, but I notice that I keep underestimating the models.

Do you have a challenge task I can try? What's the easiest thing I could get an LLM to do for circuit board design that would surprise you?

cjk2
7 replies
1d1h

Make two separate signals arrive at exactly the same time on two 50 ohm transmission lines that start and end next to each other and go around a right hand bend. At 3.8GHz.

Edit: no VSWR constraint. Can add that later :)

Edit 2: oh or design a board for a simple 100Mohm input instrumentation amplifier which knows what a guard ring is and how badly the solder mask will screw it up :)

bmicraft
4 replies
1d

It would seem to me that the majority of boards would be a lot more forgiving. Are you saying you wouldn't be impressed if it could do only say 70% of board designs completely?

cjk2
1 replies
21h42m

No because it’s hard enough picking up an experienced human’s designs and work with them. A 70% done board is a headache to unwrap. I’d start again.

nurple
0 replies
20h47m

This is how I am with software. There's usually a reason I'm arriving at 70% done, and it's not often because it's well designed and documented...

AdamH12113
1 replies
22h26m

Not the GP, but as an EE I can tell you that the majority of boards are not forgiving. One bad connection or one wrong component often means the circuit just doesn't work. One bad footprint often means the board is worthless.

On top of that, making an AI that can regurgitate simple textbook circuits and connect them together in reasonable ways is only the first step towards a much more difficult goal. More subtle problems in electronics design are all about context-dependent interactions between systems.

nurple
0 replies
20h50m

I hate that this is true. I think ML itself could be applied to the problem to help you catch mistakes in realtime, like language servers in software eng.

I have experience building boards in Altium and found it rather enjoyable; my own knowledge was often a constraint as I started out, but once I got proficient it just seemed to flow out onto the canvas.

There are some design considerations that would be awesome to farm out to genai, but I think we are far from that. Like stable-diffusion is to images, the source data for text-to-PCB would need to be well-labeled in addition to being correllated with the physical PCB features themselves.

The part where I think we lose a lot of data in pursuit of something like this, is all of the research and integration work that went on behind everything that eventually got put into the schematic and then laid out on a board. I think it would be really difficult to "diffuse" a finished PCB from an RFQ-level description.

DHaldane
1 replies
23h28m

Right - LLMs would be a bit silly for these cases. Both overkill and underkill. Current approach for length matching is throw it off to a domain specific solver. Example test-circuit: https://x.com/DuncanHaldane/status/1803210498009342191

How exact is exactly the same time? Current solver matches to under 10fs, and I think at that level you'd have to fab it to see how close you get with fiber weave skew and all that.

Do you have a test case for a schematic design task?

cjk2
0 replies
21h43m

Yeah. But you need $200k worth of Keysight kit to test it.

The point is there’s a methodology to solve these problems already. Is this better? And can it use and apply it?

LeifCarrotson
4 replies
1d

Really? Most of the time?

I find I spend an enormous amount of time on boring stuff like connecting VCC and ground with appropriate decoupling caps, tying output pins from one IC to the input pins on the other, creating library parts from data sheets, etc.

There's a handful of interesting problems in any good project where the abstraction breaks down and you have to prove your worth. But a ton of time gets spent on the equivalent of boilerplate code.

If I could tell an AI to generate a 100x100 prototype with such-and-such a microcontroller, this sensor and that sensor with those off-board connectors, with USB power, a regulator, a tag-connect header, a couple debug LEDs, and break out unused IO to a header...that would have huge value to my workflow, even if it gave up on anything analog or high-speed. Presumably you'd just take the first pass schematic/board file from the AI and begin work on anything with nuance.

If generative AI can do equivalent work for PCBs as it can do for text programming languages, people wouldn't use it for transmission line design. They'd use it for the equivalent of parsing some JSON or making a new class with some imports, fields, and method templates.

scld
2 replies
23h32m

"Looks like you forgot pullups on your i2c lines" would be worth a big monthly subscription hahaha.

oscillonoscope
0 replies
22h41m

There are schematic analysis tools which do that now just based on the netlist

makapuf
0 replies
4h2m

This totally didnt happen to me again recently. But next time I surely won't forget those. (Cue to a few months from now...)

DHaldane
0 replies
23h3m

I've found that for speeding up design generation like that, most of the utility comes from the coding approach.

AI can't do it itself (yet), and having it call the higher level functions doesn't save that much time...

HanClinto
3 replies
1d1h

But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later.

Heh. This is very true. I think perhaps the thing I'm most amazed by is that simple next-token prediction seems to work unreasonably well for a great many tasks.

I just don't know how well that will scale into more complex tasks. With simple next-token prediction there is little mechanism for the model to iterate or to revise or refine as it goes.

There have been some experiments with things like speculative generation (where multiple branches are evaluated in parallel) to give a bit of a lookahead effect and help avoid the LLM locking itself into dead-ends, but they don't seem super popular overall -- people just prefer to increase the power and accuracy of the base model and keep chugging forward.

I can't help feeling like a fundamental shift something more akin to a diffusion-based approach would be helpful for such things. I just want some sort of mechanism where the model can "think" longer about harder problems. If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time. That alone should tell us that LLMs are not intelligent, and they are not "thinking", and they will be insufficient for this going forward.

I believe Yann LeCun is right -- simply scaling LLMs is not going to get us to AGI. We need a fundamental structural shift to something new, but until we stop seeing such insane advancements in the quality of generation with LLMs (looking at you, Claude!!), I don't think we will move beyond. We have to get bored with LLMs first.

pton_xd
1 replies
23h3m

If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time.

Is that true, especially if you ask it to think step-by-step?

I would think the model has certain associations for simple/common board states and different ones for complex/uncommon states, and when you ask it to think step-by-step it will explain the associations with a particular state. That "chattiness" may lead it to using more computation for complex boards.

HanClinto
0 replies
22h53m

> If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time.

Is that true, especially if you ask it to think step-by-step?

That's fair -- there's a lot of room to grow in this area.

If the LLM has been trained to operate with running internal-monologue, then I believe they will operate better. I think this definitely needs to be explored more -- from what little I understand of this research, the results are sporadically promising, but getting something like ReAct (or other, similar structures) to work consistently is something I don't think I've seen yet.

visarga
0 replies
21h51m

I just want some sort of mechanism where the model can "think" longer about harder problems.

There is such a mechanism - multiple rounds of prompting. You can implement diverse patterns (chains, networks) of prompts.

sweezyjeezy
0 replies
1d1h

Some research to the contrary [1] - tldr is that they didn't find evidence that generative models really do zero shot well at all yet, if you show it something it literally hasn't seen before, it isn't "generally intelligent" enough to do it well. This isn't an issue for a lot of use-cases, but does seem to add some weight to the "giga-scale memorization" hypothesis.

[1] https://arxiv.org/html/2404.04125v2

yousif_123123
1 replies
20h39m

One downside for diffusion based systems (and I'm very noob in this) is that the model won't be able to see it's input and output in the same space, therefore wouldn't be able to do follow-up instructions to fix things or improve on it. Where as an LLM generating html could follow instructions to modify it as well. It's input and output are the same format.

HanClinto
0 replies
19h26m

Oh? I would think that the input prompt to drive generation is not lost during generation iterations -- but I also don't know much about the architectural details.

omgJustTest
1 replies
1d

I asked this question of Duncan Dec 22!

If you are interested I highly recommend this + your favorite llm. It does not do everything but is far superior to some highly expensive tools, in flexibility and repeatability. https://github.com/devbisme/skidl

HanClinto
0 replies
22h48m

This tool looks really powerful, thanks for the link!

One thing I've been personally really intrigued by is the possibility of using self-play and adversarial learning as a way to advance beyond our current stage of imitation-only LLMs.

Having a strong rules-based framework to be able to be able to measure quality and correctness of solutions is necessary for any RL training setup to proceed. I think that skidl could be a really nice framework to be part of an RL-trained LLM's curriculum!

I've written down a bunch of thoughts [1] on using games or code-generation in an adversarial training setup, but I could see circuit design being a good training ground as well!

* [1] https://github.com/HanClinto/MENTAT

hoosieree
1 replies
1d

I agree diffusion makes more sense for optimizing code-like things. The tricky part is coming up with a reasonable set of "add noise" transformations.

HanClinto
0 replies
23h44m

The tricky part is coming up with a reasonable set of "add noise" transformations.

Yes, as well as dealing with a variable-length window.

When generating images with diffusion, one specifies the image ahead-of-time. When generating text with diffusion, it's a bit more open-ended. How long do we want this paragraph to go? Well, that depends on what goes into it -- so how do we adjust for that? Do we use a hierarchical tree-structure approach? Chunk it and do a chain of overlapping segments that are all of fixed-length (could possibly be combined with a transformer model)?

Hard to say what would finally work in the end, but I think this is the sort of thing that YLC is talking about when he encourages students to look beyond LLMs. [1]

* [1] https://x.com/ylecun/status/1793326904692428907

eimrine
0 replies
1d1h

I like how you called it holistic, it is maybe the first time I see this word not in a "bad" context.

What about the topic, it is impossible to synthesize STEM things not in the manner an engineer does this. I mean thou shalt to know some typical solutions and have all the calculations for all what's happening in the schematic being developed.

Textbooks are not a joke and no matter who are you - a human or a device.

bottlepalm
15 replies
1d1h

It'd be interesting to see how Sonnet 3.5 does at this. I've found Sonnet a step change better than Opus, and for a fraction of the cost. Opus for me is already far better than GPT-4. And same as the poster found, GPT-4o is plain worse at reasoning.

Edit: Better at chain of thought, long running agentic tasks, following rigid directions.

stavros
10 replies
1d1h

Opus is better than GPT-4? I've heard mixed experiences.

imperio59
8 replies
1d1h

That's because the sample size is probably small and for niche prompts or topics.

It's very hard to evaluate whether a model is better than another, especially doing it in a scientifically sound way is time consuming and hard.

This is why I find these types of comments like "model X is so much better than model Y" to be about as useful as "chocolate ice cream is so much better than vanilla"

stavros
4 replies
1d1h

True, I just tried it for generating a book summary, and Sonnet 3.5 was very bad. GPT-4o is equally bad at that , gpt-4-turbo is great.

netsec_burn
3 replies
1d

This more likely has to do with context length?

Obscurity4340
1 replies
5h58m

How is prose more readable than bullets?

stavros
0 replies
5h54m

* Clearer narrative

* Connection between points

* Flows better

* Eyes don't start-stop as much

r2_pilot
2 replies
1d1h

And both flavors have a base flavor of excrement... Still, since I started using Claude 3 Opus (and now 3.5 Sonnet) a couple of months back, I don't see myself switching from them nor stopping use of LLM-based AI tech; it's just made me feel like the computer is actually working for and with me and even that alone can be enough to get me motivated and accomplish what I set out to do.

skapadia
1 replies
1d1h

"it's just made me feel like the computer is actually working for and with me and even that alone can be enough to get me motivated and accomplish what I set out to do."

This is a great way to describe what I've been feeling / experiencing as well.

r2_pilot
0 replies
14h54m

Just an update on my initial impressions of Claude 3.5 Sonnet. It's a better programmer than I am in Python; that's not saying much, but this is now two nights in a row I've been impressed with what I've created with it.

DHaldane
0 replies
1d1h

It really depends on the type of question, but generally I'm between Gemini and Claude these days for most things.

DHaldane
3 replies
1d1h

That's an interesting question - I'll take a few pokes at it now to see if there's improvement.

DHaldane
2 replies
1d1h

Update: Sonnet 3.5 is better than any other model for the circuit design and part finding tasks. Going to iterate a bit on the prompts to see how much I can push the new model on performance.

Figures that any article written on LLM limits is immediately out of date. I'll write an update piece to summarize new findings.

CamperBob2
1 replies
1d1h

That name threw me for a loop. 'Sonnet' already means something to EEs ( https://www.sonnetsoftware.com/ ).

RF_Savage
0 replies
5h50m

Yeah same here. Thought Sonnet had added some ML stuff into their EM simulator.

seveibar
11 replies
22h56m

I work on generative AI for circuit board design with tscircuit, IMO it's definitely going to be the dominant form of bootstrapping or combining circuit designs in the near future (<5 years)

Most people are wrong that AI won't be able to do this soon. The same way you can't expect an AI to generate a website in assembly, but you CAN expect it to generate a website with React/tailwind, you can't expect an AI to generate circuits without having strong functional blocks to work with.

Great work from the author studying existing solutions/models- I'll post some of my findings soon as well! The more you play with it, the more inevitable it feels!

maccard
4 replies
20h7m

The same way you can't expect an AI to generate a website in assembly, but you CAN expect it to generate a website with React/tailwind

Can you? Because last time I tried (probably about February) it still wasn’t a thing

mewpmewp2
1 replies
16h48m

Depends on the website, right. Because a single index.html can easily be a website which it cam generate.

maccard
0 replies
8h7m

I mean, yeah. But that’s not exactly helpful. Technically a web server can serve plain text which your browser will render so that meets the definition for most people.

I don’t think pedantry helps here, it doesn’t add to the conversation at all.

jamesralph8555
1 replies
13h27m

I tried GPT-4o in May and had good results asking it to generate react+tailwind components for me. It might not get things right the first time but it is generally able to respond to feedback well.

maccard
0 replies
8h9m

That’s not the same as generating a website though. You still need to iterate on the components, and use them.

I agree that using llms for generating things like schemas, components, build scripts etc is a good use of the technology, but we’re no closer to saying “make a saas landing page for X using vercel” and having it ready to deploy, then we were a year ago

HanClinto
3 replies
22h42m

I'd be interested in reading more of your findings!

Are you able to accomplish this with prompt-engineering, or are you doing fine-tuning of LLMs / custom-trained models?

seveibar
2 replies
21h31m

No fine tuning needed, as long as the target language/DSL is fairly natural, just give eg a couple examples of tscircuit React, atopile JotX etc and it can generate compliant circuits. It can hallucinate imports, but if you give it an import list you can improve that a lot.

DHaldane
1 replies
19h6m

I've found the same thing - a little syntax example, some counter examples and generative AI does well generating syntactically correct code for PCB design.

A lot of the netlists are electrically nonsense when it's doing synthesis for me. Have you found otherwise?

seveibar
0 replies
17h32m

Netlists, footprint diagrams, constraint diagrams etc. are mostly nonsense. I’m working on finetuning Phi3 and I’m hopeful it’ll get better. I’m also working on synthesized datasets and mini-DSLs to make that tuning possible eg https://text-to-footprint.tscircuit.com

My impression is that synthetic datasets and finetuning will basically completely solve the problem, but eventually it’ll be available in general purpose models- so it’s not clear if its worth it to build a dedicated model.

Overall the article’s analysis is great. I’m very optimistic that this will be solved in the next 2 years.

crote
1 replies
16h33m

The problem is going to be getting those functional blocks in the first place.

The industry does not like sharing, and the openly available datasets are full of mistakes. As a junior EE you learn quite quickly to never trust third-party symbols and footprints - if you can find them at all. Even when they come directly from the manufacturer there's a decent chance they don't 100% agree with the datasheet PDF. And good luck if that datasheet is locked behind a NDA!

If we can't even get basic stuff like that done properly, I don't think we can reasonably expect manufacturers to provide ready-to-use "building blocks" any time soon. It would require the manufacturers to invest a lot of engineer-hours into manually writing those, for essentially zero gain to them. After all, the information is already available to customers via the datasheet...

seveibar
0 replies
13h16m

This is why me and even some YC backed companies are working toward datasheet-to-component ai. We don’t trust third party, but we do trust datasheets (at least, trust enough to test for a revision)

cjk2
9 replies
1d1h

Ex EE here

> The AI generated circuit was three times the cost and size of the design created by that expert engineer at TI. It is also missing many of the necessary connections.

Exactly what I expected.

Edit: to clarify this is even below the expectations of a junior EE who had a heavy weekend on the vodka.

FourierEnvy
4 replies
1d1h

Why do people think inserting an LLM into the mix will make it better than just an evolutionary or reinforcement model applied? Who cares if you can talk to it like a human?

m-hilgendorf
1 replies
13h33m

imo, it's the same reason that Grace Hopper designed COBOL to write programs instead of math notation.

What natural language processing does is just make a much smarter (and dumber, in many ways) parser that can make an attempt to infer the intent, as well as be instructed how to recover from mistakes.

Personally I'm a skeptic since I've seen some hilariously bad hallucinations in generated code (and unlike a human engineer who will say "idk but I think this might work" instead of "yessir this is the solution!"). If you have to double check every output manually it's not that much better than learning yourself. However, at least with programming tasks, LLMs are fantastic at giving wrong answers with the right vocabulary - which makes it possible to check and find a solution through authoritative sources and references instead of blindly analyzing a problem or paying a human a lot of money to tell you the answer to your query.

For example, I don't use LLMs to give me answers. I use them to help explore a design space, particularly by giving me the vocabulary to ask better questions. And that's the real value of a conversational model today.

thechao
0 replies
10h33m

I think you've nailed a subtly — and a major doubt — I've been been trying to articulate about code helpers from LLMs from day one: the difficulty in programming is reducing a natural language problem to (essentially) a proof. I suspect LLM's are great at transferring style between two sentences, but I don't think that's the same as proof generation! I know work is being done I this area, but the results I've seen have been weird. Maybe transferring style won't work for math as easily as it does for spoken language.

Terr_
1 replies
1d1h

Yeah, when the author was writing about that initial query about delay-per-unit-length, I'm thinking: "This doesn't tell us whether an LLM can apply the concepts, only whether relevant text was included in its training data."

It's a distinction I fear many people will have trouble keeping in-mind, faced with the misleading eloquence of LLM output.

Kuinox
0 replies
16h47m

I think you are looking at the term generalizing and memorisation. It have been shown that LLM generalize, what is important to know is if they generalized it or memorized it.

shrimp_emoji
2 replies
1d1h

It's like a generated image with an eye missing but for circuits. :D

cjk2
1 replies
1d1h

AI proceeds to use 2n3904 as a thyristor.

AI happy as it worked the first 10ns of the cycle.

jeffreygoesto
0 replies
1d

Every natural Intelligence knows that you need to reach out to a 2N3055 for heavy duty. ;)

AdamH12113
8 replies
1d

The conclusions are very optimistic given the results. The LLMs:

* Failed to properly understand and respond to the requirements for component selection, which were already pretty generic.

* Succeeded in parsing the pinout for an IC but produced an incomplete footprint with incorrect dimensions.

* Added extra components to a parsed reference schematic.

* Produced very basic errors in a description of filter topologies and chose the wrong one given the requirements.

* Generated utterly broken schematics for several simple circuits, with missing connections and aggressively-incorrect placement of decoupling capacitors.

Any one of these failures, individually, would break the entire design. The article's conclusion for this section buries the lede slightly:

The AI generated circuit was three times the cost and size of the design created by that expert engineer at TI. It is also missing many of the necessary connections.

Cost and size are irrelevant if the design doesn't work. LLMs aren't a third as good as a human at this task, they just fail.

The LLMs do much better converting high-level requirements into (very) high-level source code. This make sense (it's fundamentally a language task), but also isn't very useful. Turning "I need an inverting amplifier with a gain of 20" into "amp = inverting_amplifier('amp1', gain=-20.0)" is pretty trivial.

The fact that LLMs apparently perform better if you literally offer them a cookie is, uh... something.

neltnerb
4 replies
1d

I think the only bit that looked handy in there would be if it could parse PDF datasheets and help you sort them by some hidden parameter. If I give it 100 datasheets for microphones it really should be able to sort them by mechanical height. Maybe I'm too optimistic.

The number of times I've had to entirely redo a circuit because of one misplaced connection, yeah, none of those circuits worked for any price before I fixed every single error.

DHaldane
3 replies
23h16m

Agree that PDF digesting was the most useful.

I think Gemini could definitely do that microphone study. Good test case! I remember spending 8 hours on DigiKey in the bad old times, looking for an audio jack that was 0.5mm shorter.

robxorb
0 replies
21h18m

Anyone looking for an idea for something potentially valuable to make: ingest PDF datasheets and let us search/compare etc, across them. The PDF datasheet is possibly one of the biggest and most unecessary hurdles to electronics design efficiency.

neltnerb
0 replies
11h14m

Hah, you're not kidding. Literally my comment was inspired by a recent realization that it is not possible to search for a RF connector by footprint size.

That's absurd to me, it took so long to figure out which random sequence of letters was the smallest in overall PCB footprint.

Maybe we found it, we think it's the AYU2T-1B-GA-GA-ETY(HF) but sure would be nice if Digikey had a search by footprint dimensions.

Yet strangely the physical ability of a device to fit into a location you need it is not in the list of things I can search. Takes ten seconds to find the numbers -- after I download and open the PDF file.

https://www.digikey.com/en/products/filter/coaxial-connector...

Just so strange, but so common. And digikey is heads and shoulders above average, McMaster might be the only better one I know of at it and they're very curated.

hadlock
0 replies
22h48m

As I understand it, PDF digestion/manipulation (and particularly translation) has long been a top request from businesses, based on conversations I've had with people selling the technology, so it doesn't surprise me that Gemini excels at this task.

oscillonoscope
0 replies
22h26m

I don't know enough about LLMs to understand if its feasible or not but it seems like it would be useful to make certain tasks hard-coded or add some fundamental constraints on it. Like when making footprints, it should always check that the number of pads is never less than the number of schematic symbol pins. Otherwise, the AI just feels like your worst coworker

lemonlime0x3C33
0 replies
1d

thank you for summarizing the results, I feel much better about my job security. Now if AI could make a competent auto router for fine pitch BGA components that would be really nice :)

doe_eyes
0 replies
23h53m

Yes, this seemed pretty striking to me: the author clearly wanted the LLM to perform well. They started with a problem for which solutions are pretty much readily available on the internet, and then provided a pretty favorable take on the model's mistakes.

But the bottom line is that it's a task that a novice could have solved with a Google search or two, and the LLM fumbled it in ways that'd be difficult for a non-expert to spot and rectify. LLMs are generally pretty good at information retrieval, so it's quite disappointing.

The cookie thing... well, they learn statistical patterns. People on the internet often try harder if there is a quid-pro-quo, so the LLMs copy that, and it slips past RLHF because "performs as well with or without a cookie" is probably not one of the things they optimize for.

kristopolous
4 replies
21h3m

Just the other day I came up with an idea of doing a flatbed scan of a circuit board and then using machine learning and a bit of text promoting to get to a schematic

I don't know how feasible it is. This would probably take low $millions or so of training, data collection and research to get not trash results.

I'd certainly love it for trying to diagnose circuits.

It's probably not really that possible even at higher end consumer grade 1200dpi.

cmbuck
3 replies
20h58m

This would be an interesting idea if you were able to solve the problem of inner layers. Currently to reverse engineer a board with more than 2 layers an x-ray machine is required to glean information about internal routing. Otherwise you're making inferences based on surface copper only.

kristopolous
0 replies
20h19m

Maybe not. I scanned a bluetooth aux transceiver yesterday as a test of how well a flatbed can pick up details. There's a bunch of these on the market and the cheap ones, they are more or less equivalent. It's a CSR 8365 based device, which you can read from the scan. The industry is generally convergent on the major design decisions for some hardware purpose for some given time period.

And the devices, in this case, bluetooth aux transceivers, they all do the same things. They've even more or less converged on all being 3 buttons. When optimizing for cost reduction with the commodity chips that everyone is using to do the same things, the manufacturer variation isn't that vast.

In the same way you can get 3d models from 2d photos because you can identify the object based on a database of samples and then guess the 3d contours, the hypothesis to test is whether with enough scans and schematics, a sufficiently large statistical model will be good enough to make decent guesses.

If you've got say 40 devices with 80% of the same chips doing the same things for the same purpose, a 41st device might have lots of guessable things that you can't necessarily capture on a cheap flatbed

This will probably work but it's a couple million away from becoming a reality. There's shortcuts that might make this a couple $100,000s project (essentially data contracts with bespoke chip printers) but I'd have to make those connections. And even then, it's just a hobbyist product. The chances of recouping that investment is probably zero although the tech would certainly be cool and useful. Just not "I'll pay you money" level useful.

contingencies
0 replies
19h20m

I think good RE houses have long since likely repurposed rapid PCB testing machines to determine common nets using flying CNC probes. The good ones probably don't need to depopulate to do it.

catherd
0 replies
16h27m

As long as you are OK with destructive methods, grinding/sanding the board down gives you all layers. "PCB delayering" is the search term.

dindobre
3 replies
1d1h

Using neural networks to solve combinatorial or discrete problems is a waste of time imo, but I'd be more than happy if somebody could convince me of the opposite.

utkuumur
2 replies
1d1h

There are recent papers based on diffusion that perform quite well. Here's an example of a recent paper https://arxiv.org/pdf/2406.01661. I am also working on ML-based CO. My approach has a close 1% gap on hard instances with 800-1200 nodes and less than 0.1% for 200-300 nodes on Maximum Cut, Minimum Independent Set, and Maximum Clique problems. I think these are very promising times for neural network-based discrete optimization.

dindobre
1 replies
23h32m

Thanks, will try to give it a read this weekend. Would you say that diffusion is the architectural change that opened up CO for neural nets? Haven't followed this particular niche in a while

utkuumur
0 replies
15h24m

I believe it helps but not the sole reason. Because there are also autoregressive models that perform slightly worse. Unsupervised learning + Diffusion + Neural Search is the way to go in my opinion. However, currently, the literature lacks efficient Neural search space exploration. The diffusion process is a good starting point for neural search space exploration, especially when it is used not just to create a solution from scratch but also as a local search method. Still, there is no clear exploration and exploration control in current papers. We need to incorporate more ideas from heuristic search paradigms to neural network CO pipelines to take it to the next step.

amelius
3 replies
20h34m

Can we have an AI that reads datasheets and produces Spice circuits? With the goal of building a library of simulation components.

klysm
2 replies
20h32m

That's the kind of thing where verification is really hard, and things will look plausible even if incorrect.

amelius
1 replies
6h40m

The LLM can verify e.g. transistors by looking at the curves in the datasheet.

klysm
0 replies
5h53m

The LLM can’t verify anything - it just generates what it thinks is plausible

guidoism
2 replies
1d1h

This reminds me of my professor's (probably very poor) description of NP-complete problems where the computer would provide an answer that may or may not be correct and you just had to check that it was correct and you do test for correctness in polynomial time.

It kind of grosses me out that we are entering a world where programming could be just testing (to me) random permutations of programs for correctness.

moffkalast
1 replies
1d1h

Well we had to keep increasing inefficiency somehow, right? Otherwise how would Wirth's law continue to hold?

thechao
0 replies
10h30m

Most of the HW engineers I work with consider the webstack to be far more efficient than the HW-synthesis stack; ie, there's more room for improvement in HW implementation than in SW optimization.

built_with_flux
0 replies
21h26m

flux.ai founder here

Agree with OP that the raw models aren't that useful for schematic/pcb design.

It's why we build flux from the ground up to provide the models with the right context. The models are great moderators but poor sources of great knowledge.

Here are some great use cases:

https://www.youtube.com/watch?v=XdH075ClrYk

https://www.youtube.com/watch?v=J0CHG_fPxzw&t=276s

https://www.youtube.com/watch?v=iGJOzVf0o7o&t=2s

and here a great example of levering AI to go from idea to full design https://x.com/BuildWithFlux/status/1804219703264706578

roody15
1 replies
5h51m

It makes me think of the saying “a jack of all trades a master of none”.

I cannot help but think there are some similarities between large model generative AI and human reasoning abilities.

For example if I ask a physician with a really high IQ some general questions about say anything like fixing shocks on my mini van … he may have some better ideas than me.

However he may be wrong since he specialized in medicine, although he may have provided some good overall info.

Let’s take a lower IQ mechanic who has worked as a mechanic for 15 years. Despite this human having less IQ, less overall knowledge on general topics … he gives a much better answer of fixing my shocks.

So with LLM AI fine tuning looks to be key as it is with human beings. Large data sets that are filtered / summarized with specific fields as the focus.

pylua
0 replies
59m

That’s not really reasoning, right ? Maybe humans rely disproportionate on association in general.

cushychicken
1 replies
1d

I'm terrified that JITX will get into the LLM / Generative AI for boards business. (Don't make me homeless, Duncan!)

They are already far ahead of many others with respect to next generation EE CAD.

Judicious application of AI would be a big win for them.

Edit: adding "TL;DRN'T" to my vocabulary XD

DHaldane
0 replies
23h8m

I promise that we want to stay a software company that helps people design things!

Adding Skynetn't to company charter...

Terr_
1 replies
23h18m

To recycle a rant, there's a whole bunch of hype and investor money riding on a very questionable idea here, namely:

"If we make a really really good specialty text-prediction engine, it could be able to productively mimic an imaginary general AI, and if it can do that then it can productively mimic other specialty AIs, because it's all just intelligence, right?"

ai4ever
0 replies
21h58m

investor money is seduced by the possibilities and many of the investors are in it for FOMO.

few really understand what the limits of the tech are. and if it will even unlock the usecases for which it is being touted.

teleforce
0 replies
13h38m

Too Lazy To Click (TLTC):

TLDR: We test LLMs to figure out how helpful they are for designing a circuit board. We focus on utility of frontier models (GPT4o, Claude 3 Opus, Gemini 1.5) across a set of design tasks, to find where they are and are not useful. They look pretty good for building skills, writing code, and getting useful data out of datasheets.

TLDRN'T: We do not explore any proprietary copilots, or how to apply a things like a diffusion model to the place and route problem.

surfingdino
0 replies
1d

Look! You can design thousands of shit appliances at scale! /s

ncrmro
0 replies
19h42m

I had it generate some opencad but never looked into it further.

djaouen
0 replies
16h56m

Sure, this will end well lol

al2o3cr
0 replies
15h28m

TBH the LLM seems worse than useless for a lot of these tasks - entering a netlist from a datasheet is tedious, but CHECKING a netlist that's mostly correct (except for some hallucinated resistors) seems even more tedious.

MOARDONGZPLZ
0 replies
5h12m

Author mentions prompting techniques to get better results, presumable “you are an expert EE” or “do this and you get a digital cookie” are among these. Can anyone point me to non-SEO article that outlines the latest and greatest in the promoting techniques domain?