Searching...
19 results for “hardware infrastructure”
Topic4 mentions across episodes
hardware
Topic3 mentions across episodes
infrastructure
Concept2 mentions across episodes
hardware
...now. And I know infrastructure spans, not just hardware, but also, but also software. Yeah. Yeah. For sure. So, listen, it's been it's been an evolving tale. I mean, hardware has historically actually been pretty boring, like networking, silicon. Now
3:46 / 4:48
from transcript3:46 – 4:48
Databases and file system, abstracted storage devices and networking. For accelerated computing, there's a brand new physical device that inference and we are abstracted a large part of it for inference specific workload. Of course, it's training, bu
40:30 / 41:30
from transcript40:30 – 41:30
to make, you know, for for example, you know, how we what would be the best way to utilize the g b 200, g b 300, m b l 72 rack entirely for the giant open source model. Still, I think it's a it's open problem. There are definitely some endeavors in a
39:27 / 40:48
from transcript39:27 – 40:48
movement to the amount of computer bubbles. So they have a lot more, spec'd out on compute rather than the memory movement. So they end up consuming less than what is efficient. And there are many software tricks that all of us try to do, whether tha
32:57 / 34:04
from transcript32:57 – 34:04
...but the hardware is also, deployed in different configurations is, the way I would say it. And then the other aspect of Inference that is becoming really interesting is, reinforcement learning, especially on the critical path of serving because laten
20:15 / 21:17
from transcript20:15 – 21:17
On the flip side, you have you have companies like SK Hynix and Micron who are converting significant amounts significant amounts of their capacity of, you know, sort of commodity DRAM to HBM. Now HBM is still fungible, right, in that if someone hits
1:03:41 / 1:04:47
from transcript1:03:41 – 1:04:47
our NV link between them, our InfiniBand to network it to other boxes, and, you know, welcome to the show. One more piece to talk about the third leg of the stool there, strategy before we get to what it all means that I think you're about to go to.
1:05:14 / 1:06:20
from transcript1:05:14 – 1:06:20
...infrastructure spend less dependent on narrative velocity and more on execution capacity. This environment highlights a shift in where markets perceive durable value. From application layer revenue models toward compute manufacturing, hardware enable
2:52 / 4:03
from transcript2:52 – 4:03
...infrastructure. I just don't think that the data centers like, if you assume that 100% of the data centers at some point in time will need to get reracked and you will need a very different level of power requirement per rack that's gonna be there co
5:15 / 6:15
from transcript5:15 – 6:15
Actually, regular memory, high end, like, server memory that is not HBM is actually higher gross margin than HBM. And the reason for this is because NVIDIA is pushing the memory maker so hard. Right? They want the faster and newer generation of memor
1:05:14 / 1:06:15
from transcript1:05:14 – 1:06:15
you know, stacking HBM is that you have to you know, when you have each wafer, you have to etch, create, like, this thing called a through silicon via so it can connect from the top to bottom, and then you stack them on top of each other. Right? 12 h
17:12 / 18:16
from transcript17:12 – 18:16
...lot about hardware. The other big kind of feature is software. You have to be really, really good at it. In inference, the the the pressure of being CUDA compatible or or being very easy is a lot lesser than on the training side. On training side, yo
36:33 / 38:04
from transcript36:33 – 38:04
Can you, in general, actually just zoom out and also talk about the the hopper architecture, the NVIDIA hopper GPU architecture, and the difference between h 100 and h 800, like you mentioned, the interconnects. Yeah. So there's, you know, Ampere was
1:08:52 / 1:10:03
from transcript1:08:52 – 1:10:03
You're gonna see this happen more and more going forward where the drive to increase performance, you know, whatever two to three times every two, three, four years, We're not gonna be able to do that just because drawing power and water to the site,
18:00 / 19:06
from transcript18:00 – 19:06
a, what what black holes can do, but probably more importantly, b, what what Huawei can do? You wanna walk us through the numbers? What I would just say, I don't think describing the eight seventy as an obsolete chip is is fair. Right? Chips have man
14:07 / 15:29
from transcript14:07 – 15:29
a 100 GPUs in the data center. It was all the same architecture. Starting in September 2022, they broke out the two business lines into different architectures. So there's the hopper architecture named after great computer scientist, Grace Hopper. I
1:06:00 / 1:07:09
from transcript1:06:00 – 1:07:09
Have a podcast?
Run your podcast through Clypt
Get ranked clips, hooks, and ready-to-post copy from your own episodes. Free to try.