Rendered at 21:58:53 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
K0balt 20 hours ago [-]
These are going to be so awesome to buy in 5-8 years!
Right now I’m running 2x 3090 and 128 gb ram on 3.8 flash-next180b (the quant is 97gb) and it hums at120t/s. It’s really a surprisingly good model for general tasks, and I’m soon to see how it does with more agentic kit. For agentic work it does slow down quite a bit when the context fills up… but what do you want for running a 180b model on consumer hardware?
Right now I have it set up as an administrator for our asterisk PBX and radius server, and it does higher level judgment on our cctv footage for notifications to security. It’s been really good so far. Agents are the new UI, everything just works over SSH now.
hypfer 11 hours ago [-]
> Agents are the new UI
Poorly-defined and ever-changing with occasional cases where a button does nothing, does something else, or deletes the whole prod database?
Idk. For a prototype they might be, but the whole reason we have this software thing going on is because we want some level of determinism.
unrented7977 5 hours ago [-]
> Poorly-defined and ever-changing with occasional cases where a button does nothing, does something else, or deletes the whole prod database?
This perfectly sums up all UI/UX decisions I've encountered in the ~ten years prior to AI, yes
K0balt 10 hours ago [-]
Lol. I’m totally with you on agentic UX development. It’s frequently terrible, and you can tell which companies are telling everyone to 10x with AI. But the problem is testing, not the coding. My first stab at a UX render usually sucks too, it’s the iteration that brings perfection. It’s the same for agentic development.
What I meant though wasn’t agents building UX. I meant that in our case at least, the agent is the UX. “put a new extension 117 for Sam at the port next to Jane’s, and have it ring through to his cell when he’s not in the office. Leave the old 117 as is but only ring where Sam is, if that is known” basically, agents just do everything with SSH, and you can talk to them instead of poking around with a mouse. We don’t build ux for internal systems anymore. Local inference is strong enough to be reliable now with the correct tools and policies, and for 10-20k in hardware you are self contained. (Obviously $more if you are going to have a lot of people using it at the same time)
The “one wierd trick” is to build testing jugs for everything. Before a configuration is deployed, it’s tested against a toy instance, all existing functions are tested in all common cases, new functions verified, inputs fuzzed, adversarily abused, and stress tested against dummy loads. If its code, it gets examined against development policy, self containment, explicit interface walls, race conditions, code standards, complexity, factorability, and often A/B testing of different architectural choices. Except cryptography and a couple of other things, we generally prohibit the use of any libraries, it’s just not needed and the code never passes our tests anyway.
People are just YOLOing it and assuming that if it basically functions and looks right then it must be perfect. You’d have the same luck if you told your UX guy “test only the obvious, and if it looks about right at first glance, merge it. Don’t do extensive testing.”
The real thing with AI is that now you can afford to do a ridiculous amount of automated testing, setting up test fixtures, hardware simulators, mannequins of all the external interfaces, fuzzing, etc. Our testing code is 2-3x our product code, and we are shipping 10% of the bugs we shipped with humans, and it’s still 1/4 the cost and 4-10x the speed.
People shipping all the crap that is everywhere now have bad process engineering. Plain and simple. If you try to build software with AI like you used to with people, you’re going to have a bad time. The pipeline is very different.
fps-hero 9 hours ago [-]
> My first stab at a UX render usually sucks too, it’s the iteration that brings perfection
With AI, your iteration cycle gets 10x/100x quicker. For the same developer effort, you can effectively try out 10s of ideas the same period of time.
Iteration also has a human component. When a person works hard and writes the code, they are personally invested in its success. So iteration, or, chucking out someone’s hard work is psychologically damaging and causes human conflicts in teams. If AI wrote it, and AI deleted it, no-one gets upset. All that matters are good ideas. Reducing the human cost of iteration is just as important as the time saving.
And, now you have freed up implementation time, you now have way more time for testing, that thing that everyone wished they had more time to do, with a super intelligent 100x developer (ai) as your coding partner, and a 100x lever for automating regression testing. Bad UX has zero excuses.
PorciiVorbesc 6 hours ago [-]
>With AI, your iteration cycle gets 10x/100x quicker. For the same developer effort, you can effectively try out 10s of ideas the same period of time.
This, except not (just) for UX, but for everything else too. Tonnes of ideas and side project people had on hold for lack of time and/or knowledge, have now been built and trialed thanks to AI.
You see this in the recent boom of reverse engineered multi platform releases of vintage video games that were once tied to a specific console.
Or people reverse engineering their crappy IoT home smart devices to build better multi platform UIs controllers for them.
Which is why I'm really struggling to understand how tech people can still now say with a straight face "AI isn't useful for me". Maybe they just lack imagination?
sqvaard 10 hours ago [-]
kudos. Precisely this. People are doing agentic design backwards very often: YOLO then cry. But, if first you write skills, specs, set up orchestration, architecture and flows, including review layer, discrimination layer and trajectory management layer, then you task the agents with creation of verifiable testing and static analysis rig, based on the golden samples of high-quality manually engineered code, and only then you tell then to write conformant code.
Agents (and LLMs) are inherently non-deterministic. In coding practice, this translates to garbage code in the long run. The solution is simple - make the process as deterministic, verifiable and rigid as possible. Suddenly, you get all the bonuses of agentic flows (insane initial speed, trivial cost, express iterations etc.) with none of the usual problems... and suddenly, humans have hard time matching the quality of the final artifact with hand-written code, because we suck at testing, fuzzing and debugging - because we're basically single-threaded and overfocused. Agents, OTOH, excel at it, and context management is also trivial compared to humans.
hypfer 10 hours ago [-]
This feels like a detached backpedaling pivot to save face tbh.
But anyway.
I'm sure there are people that use the tech wrong. That can be said about any tech in existence.
sqvaard 10 hours ago [-]
Most people, actually. Remember, friend - ~half of the population has two digits in their IQ, by definition.
If we aren't in that half, it's by sheer luck, and nothing else.
fomoz 9 hours ago [-]
It works just fine. I've been using it for over three months myself once I tuned my MCP server. I run a 25-30 agent fleet with zero issues, I control it with Codex or Claude Code.
This isn't something you'd be able to manage with a normal UI since every agent has its own at of settings. My agents come in sets, I usually change a set or multiple sets, 10-15 settings per agent. And my harness hasn't made a single mistake once I worked out all the kinks from MCP.
lukan 13 hours ago [-]
"These are going to be so awesome to buy in 5-8 years!"
Can't wait. Anyone has a idea of what the price now would be?
throwawayffffas 11 hours ago [-]
The closest I can think of is AMDs MI350P it has 144gbs of hbm3.
The press says amds price is 30-40 thousand dollars. On eBay there is exactly 1 going for close to 90 which is crazy because for that amount you could buy a MI350X and all the related hardware to actually use it and it has double the vram.
So I would peg it at 40-80 thousand dollars given it's Intel, and it's lpddr5 not hbm and not gddr7 so performance is likely to be lackluster.
unrented7977 5 hours ago [-]
At work we've got a 96BG Blackwell card serving Qwen3.8:27b at well over 100tok/s. Most of the time it's way faster than codex. Just this morning I set qwen and codex on the same task and Qwen came back flagging problems Sol 6.1 missed.
Local models are going to absolutely eat OAI's lunch before too long. The only moat left is hardware cost, and they can't keep it artificially high for too much longer.
testerius 7 hours ago [-]
I use Claude Opus 5.5 and Codex Sol 6.1 in my job and I love it. I wish be able to run local models which have power at least current Opus 5.5 on medium hardware which does not cost millions $. This is my dream, running LLMs and AI agents to just create things for myself. On my really old hardware I can only run simple and small 1.5B model (Qwen2.5-Coder-1.5B) and well... Ihmo it works fine with offloading to GPU and using llama.cpp but it does a lot of mistakes so I deciced to stop using it.
ttoinou 7 hours ago [-]
Get a m5 pro mac with 48gb vram and you’re good to go for a few years and that computer can be used for others purpose as well
14u2c 24 minutes ago [-]
Dog slow, at last for me. I can't crack 30 tokens/s. I'm trying an Intel B70 next.
mnadkvlb 16 hours ago [-]
How do you do the cctv analysis ?
0x20Fearless 14 hours ago [-]
It’s combined with UniFi I assume, what else has good API, Hooks, or consumer level integration to play around with with.
gotstad 12 hours ago [-]
Frigate is an excellent open alternative.
amelius 9 hours ago [-]
> Agents are the new UI
Funny as I'm using LLMs now to write UIs.
locknitpicker 17 hours ago [-]
> (...) it hums at120t/s. It’s really a surprisingly good model for general tasks, and I’m soon to see how it does with more agentic kit.
This is exactly why I interpret the aggressive price cutting by OpenAI on their Luna model line as a clear sign they are scared shitless of local models, and how they can trigger a "the king has no clothes" moment for AI companies like OpenAI.
Aurornis 16 hours ago [-]
> as a clear sign they are scared shitless of local models,
The number of people with hardware to run a model like that is very small.
It's going to be many years before hardware to run medium size local models is widely available. Running mid-size local models doesn't even register as a factor in pricing decisions right now.
It's such a small part of the market and will be for many years, until RAM and other hardware prices come down.
stymaar 15 hours ago [-]
> The number of people with hardware to run a model like that is very small.
True, but the number of people who pay the non-subsidized API price is also very small, and it's all in big companies who would have no issue buying or renting hardware for that purpose.
roenxi 11 hours ago [-]
Ram prices suggest there is going to be a mighty glut of the stuff in a couple of years. We may easily see 32->64->128 GB system RAM off the back of that and CPUs seem to be moving to unifying VRAM and main memory.
I don't think running a ~200B parameter model is that out of reach, it might be usual by the mid 2030s for people in the tech world. Factor in the crazy improvements in quality given a number of parameters too.
podocarp 8 hours ago [-]
But why? HBM is not reusable by consumers, and we would probably be at ddr6 by the time used ddr5 comes crashing down.
dathinab 8 hours ago [-]
but the number of small to medium sized companies which really want to run (some of) the AI usage under their control and are willing to drop some 100ks on it aren't that few (especial for non US companies given the state the US is in)
locknitpicker 8 hours ago [-]
> but the number of small to medium sized companies which really want to run (some of) the AI usage under their control and are willing to drop some 100ks on it aren't that few
To this I'll add companies who don't mind paying for a non-US service provider to deliver the compute required to run local models.
Just like some users are happy to run GPT6 Luna, some users are happy to run gemma4 et al.
layer8 10 hours ago [-]
> The number of people with hardware to run a model like that is very small.
But there are a lot of companies who do and rent it out as a service.
K0balt 9 hours ago [-]
Idk, I’m running 180b on $2500 of hardware at 120 T/S (single user of course) but I bought it before the clown show. Now it’s probably more like $10k lol
I expect I’ll be able to run 1/2 terabyte models in 5 years for a couple $K in hardware. Inference is going to be a lot like Bitcoin mining. Only the peak of the power/performance curve is going to be profitable. Which means a glut of surplus hardware that can’t make money but makes tons of tokens.
foobarian 8 hours ago [-]
I hope you are right. I've been waiting for GPU prices to ease off ever since ASIC mining took off.
imtringued 15 hours ago [-]
Many years is only long in relation to the AI hype cycle. In terms of a human lifespan it is not that much, maybe 10 years at most and that's pessimistic. We technically live in that future already it is just very unevenly distributed.
daymanstep 14 hours ago [-]
A human lifespan doesn't have that many decades in it either...
locknitpicker 16 hours ago [-]
> The number of people with hardware to run a model like that is very small.
That's not the problem anyone is facing, and analysis anyone is doing. At all.
The problem is the huge AI-related budget that companies who dove head first onto AI are facing.
Therefore the problem everyone is faced with boils down to whether they keep paying the likes of OpenAI a de-facto business tax, or they replace that tax with a one-off CapEx invoice followed by a utilities bill.
And you don't need a frontier model to do most things. In fact, the biggest challenge in using local models in agentic coding is the models supporting these agent-based tools.
> It's going to be many years before hardware to run medium size local models is widely available.
Nonsense. Speaking for myself, my employer already introduced a hard cap on AI credits while rolling out hardware support for local models, and even went to the extents of rolling out workstations with 64GB of RAM as a baseline. Consequently, we are using the likes of gemma4 for a fair share of everyday tasks. This has a transformative effect on your workflows because suddenly you don't ration tokens, and agent orchestration doesn't become a cash guzzler. So far the only constraints are tool support and token throughput, but that's not a concern.
jcheng 13 hours ago [-]
I wonder how this compares to a hard(-ish) cap on Sonnet-and-better models, and unlimited access to Haiku. That’s one of the policies we’re considering.
Anangikwe 14 hours ago [-]
[dead]
w4der 13 hours ago [-]
The average person may not even own a personal computer, has a company-owned notebook with barely 16 GB of RAM and maybe a TPU, and here you are saying that owning a prosumer GPU and more RAM than what most people will buy across their entire lifetimes is common.
Anangikwe 8 hours ago [-]
The average person doesn't have an interest in AI either.
I'm talking small and medium businesses, who have productive gains using AI, where a 3k€ PC is common. We have many.
locknitpicker 8 hours ago [-]
> has a company-owned notebook with barely 16 GB of RAM
Mind you, some local models run on 16GB of RAM, with some headroom to do development work.
f6v 13 hours ago [-]
> Not completely obvious.
I'd have to spend what, 5-10k today to run flash models? It can't be justified as long as I have Luna on 20$ sub. I'd have to be doing so much heavy unsupervised work for the economy to make sense.
The number of people who have 192GB RAM at their disposal is minuscule because there was no reason to have that much just recently unless:
1. You're a professional in some compute heavy field AND
2. You don't use a cloud or in-house enterprise infrastructure
kelnos 14 hours ago [-]
If you think that 192GB of VRAM on a video card that costs 3k€ is "pedestrian", I think you're a bit out of touch with the average consumer.
altmanaltman 14 hours ago [-]
12 months ago buying 192 gigs of ram was pedestrian? Really?
Anangikwe 8 hours ago [-]
Just checked, 192GB of DDR5 was 500€ vat excl.
That's what we paid per for each set of 4x48.
500€ for a production tool seems fairly innocuous.
We're not in the 10k€+ workstation territory here. At least we were not.
vladgur 17 hours ago [-]
At cost of power at $.45kwh here and dual 3090 setup easily approaching 1kw per hour at 120 tok/sec, latest Luna is cheaper per million tokens.
Your electrical costs are likely less than mine though
rft 11 hours ago [-]
A dual 3090 system shouldn't go above 600W wall power while running inference, even less with some weights in RAM as you need to do with "only" 48GB VRAM and this model. Unless you are running water cooling, the cards are likely sitting at 200-250W power limit each anyway so they don't run too hot. This is considering a consumer tier CPU and mainboard with about 90W TDP, not a dual CPU or something like a threadripper that can easily push the power draw to 1kw if it runs the calculations on the RAM weights. My dual 3090 setup usually drew about 200W-300W average with LLM workloads according to my HA historical data.
selicos 16 hours ago [-]
I'm working off a 128gb Strix system on mostly hydro and it's worth it for me. Qwen 3.8 27B and Flash Next, Gemma 4 and MOE models. It Keeps my thinking and methods local while moving at a speed I can actually read and think about.
It can be annoying when you're running something long and realize there is an issue. But hey, I'm not wasting paid tokens. Reset and try again. I've probably thought of a better way by then too.
K0balt 16 hours ago [-]
Yeah electric costs are nil on the microgrid but capacity is limited.
NitpickLawyer 15 hours ago [-]
> as a clear sign they are scared shitless of local models
Partially agree and partially disagree. Qwen 3.8 next (120b + 50b ngrams) is at the upper end of a local model, and serving it at good speeds (esp for pp) is not cheap. So no, oAI are not particularly scared that everyone will suddenly start using local models of this size / quality.
I do agree that having these models in the open is great pressure on the price point of "small" API models (like luna, haiku, etc). And that's good news for us consumers and for the field in general - more competition is always better.
And on the cost estimate, the true power of local models is the guarantee of running them on prem w/o any data going out. Price wise it is still cheaper to go API, for small teams.
zozbot234 13 hours ago [-]
The "upper end" of local models as of today would be something like MiMo 2.6 Pro or Kimi K3, either of which is significantly smarter than Qwen Flash-Next. Roughly comparable in smarts to the old GPT Terra series, not quite as capable as Sonnet. But yes, running these models locally at realtime-usable speeds and without heavy quantization would be even harder. They can be quite tolerable for plain Q&A chat and great for running multiple unattended workloads on a 24/7 basis, but that may be a bit of a niche.
NitpickLawyer 13 hours ago [-]
Yeah, I should have worded that better. I'd say it's at the upper level of what can be ran on plain consumer hw (like a laptop, mac, etc). For ~k3 you'd realistically need dedicated hw anyways. Or a cluster of consumer hw, and that comes with its own problems and drawbacks (esp in pp which matters a lot for agentic stuff).
zozbot234 13 hours ago [-]
Kimi K3 has been demonstrated on a top-of-the-line laptop Mac running at several seconds per token (it has to fetch essentially all experts from SSD, since IIRC the dense part of the model takes around ~105GB on its own!). That's impressive enough if you only care about pure Q&A/chat or pure unattended tasks, but it's not what one would expect for agentic work. You'd need quite a bit of additional RAM to have good caching for the most common experts, and even then that would only help so much. MiMo Pro would be a roughly similar story.
> Or a cluster of consumer hw, and that comes with its own problems and drawbacks (esp in pp which matters a lot for agentic stuff).
Prefill actually ought to work well on a cluster AIUI, since it can leverage both pipeline parallelism and batching quite effectively.
amelius 14 hours ago [-]
Viewed differently, they make it much cheaper to distill their models.
BizarroLand 6 hours ago [-]
With the 2x3090s, do you have them connected with the NVLink connector or are they just running over the PCIe lanes?
robotnikman 5 hours ago [-]
iirc 3090's doesn't have NVLink, or at least the one I have doesn't/
BizarroLand 2 hours ago [-]
It must be the one you have, because 3090's were basically the last consumer level card to have nvlink.
> These are going to be so awesome to buy in 5-8 years!
Literally my first thought!
sixothree 16 hours ago [-]
Are you running club-3090 or something else?
K0balt 16 hours ago [-]
Flash-Next runs on Strata, version 0.1.42, an inference engine built specifically for Qwen3.8-flash-next
suprjami 20 hours ago [-]
With an impressive 1.5TB/s RAM bandwidth you'd think they'd advertise that more.
Marketing department asleep at the wheel.
auspiv 19 hours ago [-]
This is a very low number for a server GPU. My V100 from 2017 is 0.9 TB/s.
A single B300 has almost 8 TB/s.
AMD MI300x from 2023 is 5.3 TB/s
Edit: my bad you are saying that this is an impressive number precisely because it is more or less normal RAM, not VRAM
pixelpoet 17 hours ago [-]
VRAM isn't a type of memory, it's just RAM used for the GPU. So this LPDDR5 is VRAM, just like it can be with my Strix Halo machine.
Havoc 14 hours ago [-]
That's only partially true - the GDDRX you'd find on say a 3090 is physically very different than what you'd find on a strix halo or mac or vanilla ram. e.g. runs 4 voltage levels not 2 so that it can transmit twice the info per cycle
Dylan16807 12 hours ago [-]
Well, what they don't tell you about GDDR6X is that it's 4 voltage levels but it never goes from min to max or vice versa in a single transition (PAM4 MTA). It transfers 1.75 bits per cycle.
And then GDDR7 backed off to PAM3 and sends 1.5 bits per cycle.
There's a good chance [LP]DDR will use the same techniques once it gets into the same speed range.
throwawayffffas 11 hours ago [-]
Yes the cells are pretty much the same as they would be with hbm as well all the magic happens at the bus and the signaling, there is both a reason hbm and gddr have higher bandwidths and tradeoffs.
DDR has better latency hence the preference in CPU workloads where lower latency is more important due to variable workloads.
14u2c 15 hours ago [-]
Very true, though traditionally it did refer to GDDR.
magicalhippo 15 hours ago [-]
And a 5090 has 1.79 TB/s.
So yeah on its own not very impressive.
The large amount of memory means you can serve larger models, or less quantized, but that also means they're going to be slower due to the low bandwidth.
I'm pretty sure the D270 is meant to be a competitor to the RTX Pro 6000, which also has 1.79TB/s memory bandwidth like the RTX 5090. And it will potentially be a competitor (depending on what the MSRP is) to Nvidia GB10 based systems for 'small' scale AI development. It's not meant to run frontier-size models at the speed you'd expect from something like a cluster of H200 cards.
D270:
1.5TB/s memory bandwidth, 480GB memory capacity, 350W power draw
RTX Pro 6000:
1.79TB/s memory bandwidth, 96GB memory capacity, 600W power draw
Slightly less memory bandwidth, but 5x memory capacity and nearly half the power draw.
magicalhippo 2 hours ago [-]
Sure, but if you use that memory for the weights of a single model, it'll be ~5x slower.
alexcz 14 hours ago [-]
Yeah but I think this accelerator is more meant for a small scale deployment. This could probably serve a medium moe llm model at reasonable tk/s plus you could in addition to the llm also run a couple of other models on it in parallel like an embedding, reranker, STT, TTS, decision, image, video, … if you deploy this for a small business they might make use of all of these models but not all at the same time and not with 100s of users but this card could bring a serious broad range of AI capabilities for them.
genxy 17 hours ago [-]
Its bandwidth is around that of 32 DDR5 DIMMS (50ish GB/s). These boards should have built in NICs but they don't because Intel also wants to sell more servers.
Aurornis 16 hours ago [-]
> Marketing department asleep at the wheel.
The marketing department doesn't have to lift a finger in this market. They could quietly drop it for sale without a single announcement and it would be backordered for a year.
rasz 14 hours ago [-]
Companies nowadays care more about valuations and stock spikes than sales. Curiously D270 announcement was followed by 3% INTC drop.
baq 13 hours ago [-]
everybody is long everything that may touch sand in the next couple years, no buyers left.
layoric 20 hours ago [-]
Source? Extremely impressive if true
0x457 3 hours ago [-]
It's not impressive? Thats about "as slow" as M5 Ultra Mac Studio. That's just speed here is prettu much limited by bus width. You get 1.8 TB/s with 5090 or RTX PRO 6000.
Xeon Phi when introd in 2013ish had that much bandwidth.
zamadatix 8 hours ago [-]
Xeon Phi's ranged between ~100-350 GB/s depending on the models.
The thing to note here isn't supposed to be that 1.5 TB/s is record breaking/astounding though, even the consumer 5090 GPU has higher bandwidth than that, it's just capacity@bandwidth is what defines the usefulness of an AI accelerator and this is good on both sides instead of just one. I.e. there are a lot of "cheap" low-capacity@high-bandwidth devices and a lot of "cheap" high-capacity@low-bandwidth devices. What makes this accelerator interesting is not that it has 480 GB of LPDDR5X, it's that it has 480 GB of LPDDR5X at a speed worth having that much for, which has normally been the realm of the likes of Nvidia/AMD and their AI/DC options.
floorsthelimit 6 hours ago [-]
Sorry I was referring to the internal bidirectional ring between all of the cores. Cache != external ddr!
ornornor 11 hours ago [-]
> Marketing department asleep at the wheel.
They tasked an AI agent to it, like everyone else.
jagannathtech 18 hours ago [-]
These things will fly off the shelves with 0 marketing
cherryteastain 12 hours ago [-]
Is this D for Druid, like the B series was for Battlemage/A for Alchemist? Don't think we had C (Celestial) at any point.
xbar 9 hours ago [-]
Yes. They announced they were skipping C.
p0w3n3d 17 hours ago [-]
To be honest these ubiquitous AI hardware news look to me more and more like Maybach commercials.
Not targeting me
blazarquasar 14 hours ago [-]
[dead]
ismailmaj 19 hours ago [-]
1.5 TB/s 1.3 PFLOPS with no fast multi-GPU, it's a consumer card that pretends to be a server card.
It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while.
minraws 19 hours ago [-]
That's the point I believe fast local 2-4 user service for cheap on very large models.
You have to pretend it's for servers to keep investors happy, it would be banks and similar with in house servers that have to run local large models buying these for experiments and cybersecurity. Likely.
ismailmaj 9 hours ago [-]
Reminds me of the early 2000s era where we'd fight in my household if too many people used the internet at the same time, can see that happening here if it's not 2 to 4 users but 10.
minraws 8 hours ago [-]
I mean in companies you can just buy more, and I think AI compute is going to get a lot cheaper in the next 4-5 years, photonics are just around the corner, assuming AI is even 1/10th as good as AGI they should help prove substantially useful for just pushing photonics based computing over a couple decades sooner.
ismailmaj 7 hours ago [-]
Not really, well depends on the price point but it would make more sense given enough users to multi-GPU setup with good interconnect between them to do expert parallelism.
ismailmaj 20 hours ago [-]
Fun to see the LLMism on the product page: "A, not B"
hughw 20 hours ago [-]
Productivity!
ckrapu 17 hours ago [-]
Slop and placeholder text on the page, including this one. They should have reviewed their webpage.
"What software frameworks are relevant to Intel GPU D270?
Intel GPU D270 messaging should reference familiar AI software and serving frameworks such as PyTorch, vLLM, SGLang, and OpenAI Triton, where approved. This helps developers and platform teams understand how the product fits into existing AI workflows."
airspresso 17 hours ago [-]
Noticed this as well! Was looking for details on the software support and this FAQ entry didn't exactly instill confidence...
dr_kiszonka 15 hours ago [-]
I like this. It means a human was involved.
rkagerer 22 hours ago [-]
Can I actually buy this? Should I prefer it over Blackwell?
samuelknight 21 hours ago [-]
You can tell by the shape of the board that it's meant to go into a server chassis behind a stack of 80mm blower fans. It's not meant for consumers, but you will be able to buy it used for $200 off ebay in 7 years to put in your home lab.
segmondy 7 hours ago [-]
you won't be able to buy it for $200 in 7 years. 7 year old GPUs are still going for more than that, and take into account inflation and demand for these things. My bet is that you will not be able to buy it not even for < $4000 in 7 years
littlecranky67 17 hours ago [-]
Wonder if there will be consumer video drivers for these GPUs once the AI hardware shortage is over. Hope to buy a gaming rig one day that doesn't cost a fortune.
qingcharles 15 hours ago [-]
You can vibe-code the drivers. Someone just did that with the Nvidia drivers for Intel Macs, bless their soul.
justincormack 14 hours ago [-]
I dont think it has a video out
littlecranky67 14 hours ago [-]
I remember back in the day of AGP cards around 2002, you could use a 3d card on linux to render the grafix but use a different GPU for the actual output. I used XFree86 on linux with a voodoo banshee (or maybe was a riva tnt2) and a matrox mga200 for its sync-on-green support with my monitor to play games.
I know modern card architecture doesn't support this out of the box, but what is the actuall problem of forwarding a once-rendered frame to another card for output? Sureley DMA transfers PCIe->RAM->PCIe nowaday are fast enough to saturate 60fps at 4K?
rescbr 9 hours ago [-]
Trivial to do this on Linux. I run my desktop on the on-chip Ryzen 9950X iGPU and I offload games to my nVidia dGPU.
I had a crazier setup in the past with the Intel i7-8809G APU: it had an AMD iGPU for desktop connected to the display, nVidia eGPU for games and the Intel iGPU for video encoding.
Of course there is additional latency due to frame copying, but I can’t notice it in practice.
bzzzt 14 hours ago [-]
I had that setup working back in the day with a Voodoo1 which had the pass-trough construction to take over full screen: only on Linux you could read the completed frames and render them in an X window. It was dog slow though.
While transfers have improved a lot the amount data needed for 4K60 with HDR is about 18Gbps. There's no way that doesn't have an impact.
kjs3 10 hours ago [-]
We cooked up a setup where we had a PCI video card for X, and a separate Voodoo 1 or 2 and monitor for 3D (Mesa) rendering. I don't remember the details any more, but it was nice to not take out the desktop when someone borked up their OGL homework.
littlecranky67 12 hours ago [-]
the voodoo1 worked differently as you looped through the video input. The voodoo banshee i mentioned was a separate 2d/3d card (it actually was 3DFx first true, combined 2d/3d card - not an accelerator).
bzzzt 12 hours ago [-]
I know. Voodoo 1 and 2 had the ugly VGA passthrough hack.
But the principle of reading VRAM from one card and copying it to another would remain the same I think.
> We want to render applications on the more powerful card and send the result to the card which has display connected.
rasz 14 hours ago [-]
in latest gamersnexus video https://www.youtube.com/watch?v=e4c2FSSaCsw Wendell from Level1 Techs was showing some older Amazon dedicated server GPUs with no video output that got hacked community driver making it work like laptop dgpu copying rendered video across PCIe.
_zoltan_ 16 hours ago [-]
In 2 years once it fails and Intel shuts it down completely you mean :)
kjs3 10 hours ago [-]
As has been claimed every year for the last 30, at least. Much like predicting the end of the world, one day, if you keep at it, you'll actually be right.
BeetleB 7 hours ago [-]
I think he meant Intel shuts down their graphics division, not Intel shutting down. It's very easy to believe as they've shut down pretty much everything they've tried other than CPUs.
I wonder how they feel about selling their RAM business a few years ago.
m463 18 hours ago [-]
Wonder if today's ram will go up price by then...
manquer 18 hours ago [-]
It is going up one way or another- the price of money will go down - @5% inflation over 8 years the value of today’s dollar will be 67.7c .
kadoban 21 hours ago [-]
> Can I actually buy this?
No. Certainly not now, maybe not ever. That is the least consumer-focused product page I've seen today.
legitronics 21 hours ago [-]
Intel ai gpu division is so aggressively anti “small” consumer. I think it has something to do with low volume and a small sales/distribution network. The data center cards I have from amd and nvidia are grey market at best. I don’t think intel moved enough units to establish alternate channels.
14u2c 15 hours ago [-]
How so? You can walk into Micro Center and buy a B70 right now. Intel is the only one to offer license-free SR-IOV too.
justincormack 14 hours ago [-]
Thats not an AI GPU its a consumer product.
14u2c 5 hours ago [-]
It's a workstation card targeted specifically at inference. It's also their only late gen "AI GPU" arch available, baring the release of the D270 this thread is about.
cheeze 21 hours ago [-]
Serious question, would they bother?
You can work with 10 players that will buy all of them with minimal effort.
derefr 20 hours ago [-]
What I don't understand is: when you only plan for your product to have 10 customers, why even make a marketing page like this for it? At that point it's essentially the hardware equivalent of consultingware, something you sell to your existing contacts through outbound calls and by shoving ads through the B2B equivalent of a post-consumer sales channel (i.e. proactive outreach by your org's internal champion for that customer, to said customer.)
davrosthedalek 20 hours ago [-]
Because they want the hype to make the stock go up.
hughw 20 hours ago [-]
In fairness, they spent 5 cents generating it with Claude.
faragon 1 hours ago [-]
The Intel D270 ([1]) has similar silicon size to an AMD Radeon R9700 ([2]) and the AMD Radeon XT 9070 ([3]), and about twice the matrix performance. So the price is mainly for the memory. Assuming "normal prices", e.g. 600 USD for 128GB, 2250 USD for 480GB, and the GPU chip itself being 600 USD if produced in enough volume, including industrial profit, its "normal price" could be less than 3000 USD. I.e., it is a slow chip with lots of RAM for covering the case of having many resident models, not for running very big models, which is a very interesting use case for AI hobbyists, and likely also for some use cases in the data centers.
Like cool hardware but in the current situation also quite insulting - totally unaffordable for an individual & one more thing to gobble up useful memory + make the computing hardware affordability crisis worse.
fooqux 7 hours ago [-]
The market for home inference is still tiny compared to the burgeoning datacenter market. Of course it's not for individuals.
hurricanepootis 1 days ago [-]
Looks like Intel is choosing to title its new crescent island cards with the D prefix despite being Crescent Island.
Kind of disappointed that Intel is (probably) not going to release consumer class GPUs. Intel's driver stack was seriously getting mature—both on Linux and Windows. It's always nice to have a third option available.
boredatoms 20 hours ago [-]
Its wild to me that intel isnt trying significantly harder on gpus
Theres so much money to be made
ThrowawayR2 20 hours ago [-]
Why make consumer margins on consumer GPUs when they could be making enterprise margins on enterprise GPUs?
jandrese 17 hours ago [-]
One word: Volume.
You can make a nice profit off of the enterprise space, but the big winners are the guys who put the work into servicing the consumer market.
structural 6 hours ago [-]
Once you remove laptops from the picture, the consumer market is small in dollars compared to enterprise and has been for awhile. This didn't used to be the case, but here we are.
Best estimates from last year are on the order of 50 million discrete consumer GPUs vs 7-10 million datacenter/enterprise GPUs. Sounds like the PC market is huge! but with average sales costs of well under 1k USD (a lot of the PC cards are budget variety, included with prebuilt computers, etc). it's only a $30-40b business. Those datacenter GPUs are averaging $50k each before the supply issues with HBM started to inflate everything greatly. But using that as a benchmark, that enterprise business looks like several hundred billions.
NVIDIA's latest filing reported $162b in revenue for datacenter compute in the last FY, so we know it's at least that large. Now add AMD, Intel, and everything going on in China that we don't have great numbers for. We can very conservatively say that the enterprise GPU market is currently >5x larger than consumer, in dollars, and it may be quite a bit larger than that.
jandrese 6 hours ago [-]
This year is far from typical however. The AI bubble is throwing all of the numbers out of whack.
structural 4 hours ago [-]
I agree, which is why I used numbers from 2025 in my assessment, it's too soon to know if inflated 2026 numbers are a new normal or a one time bump. I think calling enterprise 5x larger than consumer based on 2024-2025 numbers is defensible, saying that enterprise was 10-15x consumer market just based on 2026 is yet unproven.
HDThoreaun 2 hours ago [-]
Thats why nvidia is the most valuable company in the world after effectively axing their consumer division?
boredatoms 19 hours ago [-]
I mean, why not both
ThrowawayR2 19 hours ago [-]
"Both" requires more engineering staff on payroll, more wafers to fab/dice/package/test, more boards to manufacture, more inventory to manage, among other things. If their math says it isn't worth it (granted that they don't always get it right), then they don't do it.
zargon 17 hours ago [-]
"Both" requires allocating limited die capacity to products that have over an order of magnitude less margin.
locknitpicker 17 hours ago [-]
> "Both" requires more engineering staff on payroll, more wafers to fab/dice/package/test, more boards to manufacture, more inventory to manage, among other things.
Aren't you describing how a business operates?
I mean, doesn't Intel already hire people for those tasks? Unless you are worried about a specific bottleneck, increasing headcount and production output is what businesses do.
clarionbell 10 hours ago [-]
I believe that their fabs are running at full capacity already at this point. If they were to announce new GPUs, where would they make them?
ank 19 hours ago [-]
Agreed. I was really rooting for them to deliver that third iteration to consumers, and retain them as a competitive option in the low to mid range. Pretty sad to see.
jamesforestwest 14 hours ago [-]
I have to agree, and it's sad
oddIdentities 22 hours ago [-]
At least that driver stack will continue to mature?
zakisaad 22 hours ago [-]
On Linux mainly, and almost strictly for compute workloads (not much need to optimize for 3D workloads given the customer base).
bpye 21 hours ago [-]
They're still using Xe based iGPUs, so 3D still matters.
DiabloD3 21 hours ago [-]
For now. A limited run of Celestials until 2028 when Serpent Lake switches to Nvidia tiles, and a bit longer for the LTS combo (AMD style) IO+GPU die in lower end/embedded products.
Druid is still dead, consumer DGPUs are still, and workstation DGPUs also seems to be dead, and most of the Xe driver team for Windows was laid off, as well as most of the Xe chip design team.
namjh 14 hours ago [-]
Given the rumored numbers are all true, this could be a huge contender to M5 Ultra(or M7 Ultra if we consider possible delays in the schedule?). Low-batch single-user serving is usually bottlenecked by `E(sum over (memory bandwidth by tier) * (memory size by tier))`.
edit: typo
paaloeye 9 hours ago [-]
Leather jacket Jensen must be sweating loads now because every big company and their mums are making AI chips
redanddead 9 hours ago [-]
>sweating loads
Interesting choice of words
paaloeye 9 hours ago [-]
wdym?
richrichardsson 8 hours ago [-]
Possibly:
British reading -> sweating a lot
American reading -> sweating ejaculate
Or possibly something to do with how running work on a model is referred to as a load?
Fuck knows!
paaloeye 9 hours ago [-]
At at the very least, trying to make them
HDThoreaun 2 hours ago [-]
Intels HBM supply is utter shit. AMD is still far behind nvidia. Making gpus without enough memory bandwidth isnt really a threat to nvda
nunzioraciti 19 hours ago [-]
How much is it expected to cost?
downrightmike 17 hours ago [-]
Several hobos worth of organs
betaporter 9 hours ago [-]
But can it run Crysis?
hannasm 19 hours ago [-]
Can intel gift me one?
The cost of a 480gb ram video card seems preposterous to be even marketing at this point.
Intel is going to make a card that costs $1 million dollars.. this is like industrial newsletters bleeding into a public forum
> Commercial Scale Pricing: In the industrial data center sector, battery storage systems are priced per kilowatt-hour (kWh) or megawatt-hour (MWh). Industrial battery storage installations typically cost $300,000 to $450,000+ per MWh of capacity depending on software integration, cooling systems, and structural framing.
Let's all get excited
Havoc 14 hours ago [-]
hmm...wonder where this will land in market segments.
Seems like neither a consumer product nor server. Maybe if they halve the ram and price it reasonably people with desktops will buy it for MoE like prosumer homelab AI enthusiast type situations?
0xB0D 5 hours ago [-]
Intel can really make silicon
Beautiful
laweijfmvo 7 hours ago [-]
Why even call it a GPU anymore?
wmf 7 hours ago [-]
It uses Xe cores that come from GPUs.
aboardRat4 20 hours ago [-]
How many fps in Cyberpunk 2077?
drnick1 20 hours ago [-]
Can It Run Crysis?
2OEH8eoCRo0 19 hours ago [-]
What about Windows Vista with Aero enabled?
downrightmike 17 hours ago [-]
That brain dance where you end up at the Skav's place in a bathtub
arjie 21 hours ago [-]
Seems incredibly little compute for this. Is it meant to be MoE specific?
jauntywundrkind 22 hours ago [-]
They originally said it was going to be 120GB but that partners could ship bigger capacities. So much for that!
I do hope this has a price point that makes it somewhat competitive with a Mac! Consumers have had nearly no choices available for mem capacity, and 20 channels of lpddr5 is one way to get that capacity! Go Intel!
bigyabai 22 hours ago [-]
Both this GPU and the Mac are constrained by a lack of GPU compute. There's basically no rack-scale GPUs with 480gb on-die that have a sub-400W compute element. It's going to be incredibly prefill-constrained on denser model inference just like the Apple Silicon iGPUs are.
This is almost twice better than RTX 5090. This GPU will not be prefill constrained, I feel.
But I am surprised that they were able to pack so much flops into 300W.
kadoban 21 hours ago [-]
> But I am surprised that they were able to pack so much flops into 300W.
I'm not. When you're doing inference you typically reduce power limits anyway if you're optimizing for efficiency. Even if you don't, it doesn't spike to the max of the card.
My 3090 I power it down to 175W or so for single-digit reductions in performance for like, 60% of the wattage, something like that.
shusaku 22 hours ago [-]
It seems like there’s a market out there that is just happy to run the models locally, even if generation is slow.
jchw 21 hours ago [-]
Am I wrong to assume that large MoE models are likely the point anyway? Seems like this card would love DeepSeek V4.1 Flash.
walrus01 13 hours ago [-]
Is there any indicator whatsoever of what this will cost per unit?
kittikitti 16 hours ago [-]
This is a brand new class of GPU that I am excited for, especially because it's rated for air cooling. I hope to see other manufacturers release GPU's with multiple of 100 GB's that can be utilized even in an at-home system's with PCIe 5 support. I probably can't afford this but knowing the engineering is sound at this scale means a similar accelerator maybe attainable.
doublerabbit 21 hours ago [-]
> Efficient token serving
Dense, air-cooled, standard PCIe infrastructure helps maximize tokens per rack and watt, scaling serving capacity and keeping costs in check.
Next the tokens will be asking for tequila and caviar. Bow down to the new token overloads.
nekusar 20 hours ago [-]
Yay more vaporware hardware I get to read about and never see or have a chance to buy.
Gigachad 20 hours ago [-]
This stuff you usually buy second hand in 6 years when the data centers get rid of them.
11 hours ago [-]
singularityisne 14 hours ago [-]
[flagged]
gravypod 18 hours ago [-]
[dead]
porkbellyeee 21 hours ago [-]
Ai generated no doubt
typon 18 hours ago [-]
[dead]
reckreck 14 hours ago [-]
All these big names in tech will monetize so hard on such products, until this becomes obsolete and available to us all. This should be accessible to everyone at a convenient price, governments should force these companies to mass produce them and sell to people, providing a small subsidy. Everyone deserves to run big models locally.
Right now I’m running 2x 3090 and 128 gb ram on 3.8 flash-next180b (the quant is 97gb) and it hums at120t/s. It’s really a surprisingly good model for general tasks, and I’m soon to see how it does with more agentic kit. For agentic work it does slow down quite a bit when the context fills up… but what do you want for running a 180b model on consumer hardware?
Right now I have it set up as an administrator for our asterisk PBX and radius server, and it does higher level judgment on our cctv footage for notifications to security. It’s been really good so far. Agents are the new UI, everything just works over SSH now.
Poorly-defined and ever-changing with occasional cases where a button does nothing, does something else, or deletes the whole prod database?
Idk. For a prototype they might be, but the whole reason we have this software thing going on is because we want some level of determinism.
This perfectly sums up all UI/UX decisions I've encountered in the ~ten years prior to AI, yes
What I meant though wasn’t agents building UX. I meant that in our case at least, the agent is the UX. “put a new extension 117 for Sam at the port next to Jane’s, and have it ring through to his cell when he’s not in the office. Leave the old 117 as is but only ring where Sam is, if that is known” basically, agents just do everything with SSH, and you can talk to them instead of poking around with a mouse. We don’t build ux for internal systems anymore. Local inference is strong enough to be reliable now with the correct tools and policies, and for 10-20k in hardware you are self contained. (Obviously $more if you are going to have a lot of people using it at the same time)
The “one wierd trick” is to build testing jugs for everything. Before a configuration is deployed, it’s tested against a toy instance, all existing functions are tested in all common cases, new functions verified, inputs fuzzed, adversarily abused, and stress tested against dummy loads. If its code, it gets examined against development policy, self containment, explicit interface walls, race conditions, code standards, complexity, factorability, and often A/B testing of different architectural choices. Except cryptography and a couple of other things, we generally prohibit the use of any libraries, it’s just not needed and the code never passes our tests anyway.
People are just YOLOing it and assuming that if it basically functions and looks right then it must be perfect. You’d have the same luck if you told your UX guy “test only the obvious, and if it looks about right at first glance, merge it. Don’t do extensive testing.”
The real thing with AI is that now you can afford to do a ridiculous amount of automated testing, setting up test fixtures, hardware simulators, mannequins of all the external interfaces, fuzzing, etc. Our testing code is 2-3x our product code, and we are shipping 10% of the bugs we shipped with humans, and it’s still 1/4 the cost and 4-10x the speed.
People shipping all the crap that is everywhere now have bad process engineering. Plain and simple. If you try to build software with AI like you used to with people, you’re going to have a bad time. The pipeline is very different.
With AI, your iteration cycle gets 10x/100x quicker. For the same developer effort, you can effectively try out 10s of ideas the same period of time.
Iteration also has a human component. When a person works hard and writes the code, they are personally invested in its success. So iteration, or, chucking out someone’s hard work is psychologically damaging and causes human conflicts in teams. If AI wrote it, and AI deleted it, no-one gets upset. All that matters are good ideas. Reducing the human cost of iteration is just as important as the time saving.
And, now you have freed up implementation time, you now have way more time for testing, that thing that everyone wished they had more time to do, with a super intelligent 100x developer (ai) as your coding partner, and a 100x lever for automating regression testing. Bad UX has zero excuses.
This, except not (just) for UX, but for everything else too. Tonnes of ideas and side project people had on hold for lack of time and/or knowledge, have now been built and trialed thanks to AI.
You see this in the recent boom of reverse engineered multi platform releases of vintage video games that were once tied to a specific console.
Or people reverse engineering their crappy IoT home smart devices to build better multi platform UIs controllers for them.
Which is why I'm really struggling to understand how tech people can still now say with a straight face "AI isn't useful for me". Maybe they just lack imagination?
Agents (and LLMs) are inherently non-deterministic. In coding practice, this translates to garbage code in the long run. The solution is simple - make the process as deterministic, verifiable and rigid as possible. Suddenly, you get all the bonuses of agentic flows (insane initial speed, trivial cost, express iterations etc.) with none of the usual problems... and suddenly, humans have hard time matching the quality of the final artifact with hand-written code, because we suck at testing, fuzzing and debugging - because we're basically single-threaded and overfocused. Agents, OTOH, excel at it, and context management is also trivial compared to humans.
But anyway.
I'm sure there are people that use the tech wrong. That can be said about any tech in existence.
If we aren't in that half, it's by sheer luck, and nothing else.
This isn't something you'd be able to manage with a normal UI since every agent has its own at of settings. My agents come in sets, I usually change a set or multiple sets, 10-15 settings per agent. And my harness hasn't made a single mistake once I worked out all the kinks from MCP.
Can't wait. Anyone has a idea of what the price now would be?
The press says amds price is 30-40 thousand dollars. On eBay there is exactly 1 going for close to 90 which is crazy because for that amount you could buy a MI350X and all the related hardware to actually use it and it has double the vram.
So I would peg it at 40-80 thousand dollars given it's Intel, and it's lpddr5 not hbm and not gddr7 so performance is likely to be lackluster.
Local models are going to absolutely eat OAI's lunch before too long. The only moat left is hardware cost, and they can't keep it artificially high for too much longer.
Funny as I'm using LLMs now to write UIs.
This is exactly why I interpret the aggressive price cutting by OpenAI on their Luna model line as a clear sign they are scared shitless of local models, and how they can trigger a "the king has no clothes" moment for AI companies like OpenAI.
The number of people with hardware to run a model like that is very small.
It's going to be many years before hardware to run medium size local models is widely available. Running mid-size local models doesn't even register as a factor in pricing decisions right now.
It's such a small part of the market and will be for many years, until RAM and other hardware prices come down.
True, but the number of people who pay the non-subsidized API price is also very small, and it's all in big companies who would have no issue buying or renting hardware for that purpose.
I don't think running a ~200B parameter model is that out of reach, it might be usual by the mid 2030s for people in the tech world. Factor in the crazy improvements in quality given a number of parameters too.
To this I'll add companies who don't mind paying for a non-US service provider to deliver the compute required to run local models.
Just like some users are happy to run GPT6 Luna, some users are happy to run gemma4 et al.
But there are a lot of companies who do and rent it out as a service.
I expect I’ll be able to run 1/2 terabyte models in 5 years for a couple $K in hardware. Inference is going to be a lot like Bitcoin mining. Only the peak of the power/performance curve is going to be profitable. Which means a glut of surplus hardware that can’t make money but makes tons of tokens.
That's not the problem anyone is facing, and analysis anyone is doing. At all.
The problem is the huge AI-related budget that companies who dove head first onto AI are facing.
Therefore the problem everyone is faced with boils down to whether they keep paying the likes of OpenAI a de-facto business tax, or they replace that tax with a one-off CapEx invoice followed by a utilities bill.
And you don't need a frontier model to do most things. In fact, the biggest challenge in using local models in agentic coding is the models supporting these agent-based tools.
> It's going to be many years before hardware to run medium size local models is widely available.
Nonsense. Speaking for myself, my employer already introduced a hard cap on AI credits while rolling out hardware support for local models, and even went to the extents of rolling out workstations with 64GB of RAM as a baseline. Consequently, we are using the likes of gemma4 for a fair share of everyday tasks. This has a transformative effect on your workflows because suddenly you don't ration tokens, and agent orchestration doesn't become a cash guzzler. So far the only constraints are tool support and token throughput, but that's not a concern.
I'm talking small and medium businesses, who have productive gains using AI, where a 3k€ PC is common. We have many.
Mind you, some local models run on 16GB of RAM, with some headroom to do development work.
I'd have to spend what, 5-10k today to run flash models? It can't be justified as long as I have Luna on 20$ sub. I'd have to be doing so much heavy unsupervised work for the economy to make sense.
The number of people who have 192GB RAM at their disposal is minuscule because there was no reason to have that much just recently unless: 1. You're a professional in some compute heavy field AND 2. You don't use a cloud or in-house enterprise infrastructure
500€ for a production tool seems fairly innocuous. We're not in the 10k€+ workstation territory here. At least we were not.
Your electrical costs are likely less than mine though
It can be annoying when you're running something long and realize there is an issue. But hey, I'm not wasting paid tokens. Reset and try again. I've probably thought of a better way by then too.
Partially agree and partially disagree. Qwen 3.8 next (120b + 50b ngrams) is at the upper end of a local model, and serving it at good speeds (esp for pp) is not cheap. So no, oAI are not particularly scared that everyone will suddenly start using local models of this size / quality.
I do agree that having these models in the open is great pressure on the price point of "small" API models (like luna, haiku, etc). And that's good news for us consumers and for the field in general - more competition is always better.
And on the cost estimate, the true power of local models is the guarantee of running them on prem w/o any data going out. Price wise it is still cheaper to go API, for small teams.
> Or a cluster of consumer hw, and that comes with its own problems and drawbacks (esp in pp which matters a lot for agentic stuff).
Prefill actually ought to work well on a cluster AIUI, since it can leverage both pipeline parallelism and batching quite effectively.
https://www.fibermall.com/blog/nvlink-compatible-cards.htm
Literally my first thought!
Marketing department asleep at the wheel.
A single B300 has almost 8 TB/s.
AMD MI300x from 2023 is 5.3 TB/s
Edit: my bad you are saying that this is an impressive number precisely because it is more or less normal RAM, not VRAM
And then GDDR7 backed off to PAM3 and sends 1.5 bits per cycle.
There's a good chance [LP]DDR will use the same techniques once it gets into the same speed range.
DDR has better latency hence the preference in CPU workloads where lower latency is more important due to variable workloads.
So yeah on its own not very impressive.
The large amount of memory means you can serve larger models, or less quantized, but that also means they're going to be slower due to the low bandwidth.
https://www.techpowerup.com/gpu-specs/geforce-rtx-5090.c4216
D270: 1.5TB/s memory bandwidth, 480GB memory capacity, 350W power draw
RTX Pro 6000: 1.79TB/s memory bandwidth, 96GB memory capacity, 600W power draw
Slightly less memory bandwidth, but 5x memory capacity and nearly half the power draw.
The marketing department doesn't have to lift a finger in this market. They could quietly drop it for sale without a single announcement and it would be backordered for a year.
The thing to note here isn't supposed to be that 1.5 TB/s is record breaking/astounding though, even the consumer 5090 GPU has higher bandwidth than that, it's just capacity@bandwidth is what defines the usefulness of an AI accelerator and this is good on both sides instead of just one. I.e. there are a lot of "cheap" low-capacity@high-bandwidth devices and a lot of "cheap" high-capacity@low-bandwidth devices. What makes this accelerator interesting is not that it has 480 GB of LPDDR5X, it's that it has 480 GB of LPDDR5X at a speed worth having that much for, which has normally been the realm of the likes of Nvidia/AMD and their AI/DC options.
They tasked an AI agent to it, like everyone else.
Not targeting me
It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while.
You have to pretend it's for servers to keep investors happy, it would be banks and similar with in house servers that have to run local large models buying these for experiments and cybersecurity. Likely.
"What software frameworks are relevant to Intel GPU D270?
Intel GPU D270 messaging should reference familiar AI software and serving frameworks such as PyTorch, vLLM, SGLang, and OpenAI Triton, where approved. This helps developers and platform teams understand how the product fits into existing AI workflows."
I know modern card architecture doesn't support this out of the box, but what is the actuall problem of forwarding a once-rendered frame to another card for output? Sureley DMA transfers PCIe->RAM->PCIe nowaday are fast enough to saturate 60fps at 4K?
I had a crazier setup in the past with the Intel i7-8809G APU: it had an AMD iGPU for desktop connected to the display, nVidia eGPU for games and the Intel iGPU for video encoding.
Of course there is additional latency due to frame copying, but I can’t notice it in practice.
While transfers have improved a lot the amount data needed for 4K60 with HDR is about 18Gbps. There's no way that doesn't have an impact.
> We want to render applications on the more powerful card and send the result to the card which has display connected.
I wonder how they feel about selling their RAM business a few years ago.
No. Certainly not now, maybe not ever. That is the least consumer-focused product page I've seen today.
You can work with 10 players that will buy all of them with minimal effort.
[1] https://www.techpowerup.com/gpu-specs/d270.c4452
[2] https://www.techpowerup.com/gpu-specs/radeon-ai-pro-r9700.c4...
[3] https://www.techpowerup.com/gpu-specs/radeon-rx-9070-xt.c422...
Kind of disappointed that Intel is (probably) not going to release consumer class GPUs. Intel's driver stack was seriously getting mature—both on Linux and Windows. It's always nice to have a third option available.
Theres so much money to be made
You can make a nice profit off of the enterprise space, but the big winners are the guys who put the work into servicing the consumer market.
Best estimates from last year are on the order of 50 million discrete consumer GPUs vs 7-10 million datacenter/enterprise GPUs. Sounds like the PC market is huge! but with average sales costs of well under 1k USD (a lot of the PC cards are budget variety, included with prebuilt computers, etc). it's only a $30-40b business. Those datacenter GPUs are averaging $50k each before the supply issues with HBM started to inflate everything greatly. But using that as a benchmark, that enterprise business looks like several hundred billions.
NVIDIA's latest filing reported $162b in revenue for datacenter compute in the last FY, so we know it's at least that large. Now add AMD, Intel, and everything going on in China that we don't have great numbers for. We can very conservatively say that the enterprise GPU market is currently >5x larger than consumer, in dollars, and it may be quite a bit larger than that.
Aren't you describing how a business operates?
I mean, doesn't Intel already hire people for those tasks? Unless you are worried about a specific bottleneck, increasing headcount and production output is what businesses do.
Druid is still dead, consumer DGPUs are still, and workstation DGPUs also seems to be dead, and most of the Xe driver team for Windows was laid off, as well as most of the Xe chip design team.
edit: typo
Interesting choice of words
British reading -> sweating a lot
American reading -> sweating ejaculate
Or possibly something to do with how running work on a model is referred to as a load?
Fuck knows!
The cost of a 480gb ram video card seems preposterous to be even marketing at this point.
Intel is going to make a card that costs $1 million dollars.. this is like industrial newsletters bleeding into a public forum
> Commercial Scale Pricing: In the industrial data center sector, battery storage systems are priced per kilowatt-hour (kWh) or megawatt-hour (MWh). Industrial battery storage installations typically cost $300,000 to $450,000+ per MWh of capacity depending on software integration, cooling systems, and structural framing.
Let's all get excited
Seems like neither a consumer product nor server. Maybe if they halve the ram and price it reasonably people with desktops will buy it for MoE like prosumer homelab AI enthusiast type situations?
Beautiful
I do hope this has a price point that makes it somewhat competitive with a Mac! Consumers have had nearly no choices available for mem capacity, and 20 channels of lpddr5 is one way to get that capacity! Go Intel!
This is almost twice better than RTX 5090. This GPU will not be prefill constrained, I feel.
But I am surprised that they were able to pack so much flops into 300W.
I'm not. When you're doing inference you typically reduce power limits anyway if you're optimizing for efficiency. Even if you don't, it doesn't spike to the max of the card.
My 3090 I power it down to 175W or so for single-digit reductions in performance for like, 60% of the wattage, something like that.
Next the tokens will be asking for tequila and caviar. Bow down to the new token overloads.