Searching...
14 results for “machine learning”
Topic4 mentions across episodes
Machine learning
Concept2 mentions across episodes
machine learning
...Now deep learning is new for this use case, but Ben is you weren't exactly right. It had been around for a long time, a very long time. And deep learning, neural networks,
46:27 / 47:34
from transcript46:27 – 47:34
...deep learning revolution. Network's gotten larger and larger. And at a certain point, the scaling laws appeared where people realized This is a scaling law shirt metaphor. Representing scaling laws where it became more and more formalized that bigger
38:04 / 39:19
from transcript38:04 – 39:19
...learning, neural networks, this was not a new idea. The algorithms had existed for many decades, I think, but they were really, really, really computationally intensive. They required to train the models to do a a deep neural network. You need a lot
47:14 / 48:17
from transcript47:14 – 48:17
scaling the the width of the networks or the hidden layers. And, in effect, it yeah. Pretty much kinda similar to just scaling depth naively. Yeah. And then once we started introducing residual connections, layer norm, these specific architectural ch
6:12 / 7:16
from transcript6:12 – 7:16
...learning was, like, this one anomaly where we continue to use these really shallow networks. And that's particularly true in the setting that we were looking at where you're starting from scratch. You're starting from nothing. Any other perspectives
3:09 / 4:13
from transcript3:09 – 4:13
...learning. Right? Like ImageNet is a great example. Self attention is great from transformers, but they'll also say this is a way you can exploit human labeling of data because it's the humans that put the structure in the sentences. And if you look a
9:00 / 10:14
from transcript9:00 – 10:14
...in both children and in deep learning. There's one, imitation learning, watch and repeat, I e, pretraining, supervised fine tuning, and two, trial and error learning, reinforcement learning. My favorite simple example is AlphaGo. One is learning by i
2:54:42 / 2:55:46
from transcript2:54:42 – 2:55:46
...branch of machine learning and computer science was. Was. Right. So, Ben, as you were saying earlier, neural networks, not a new idea and had all of this great promise in theory, but in practice, just took too much computation to do multiple layers.
31:30 / 32:33
from transcript31:30 – 32:33
...deep learning as it was later called, the work of of it goes back to Fukushima and the Neo Cognitron, the Hinton and Williams, seminal paper and the PDP book, which I encourage people to go back and look. And and, of course, the work of Jan Lacun, an
4:46:40 / 4:48:43
from transcript4:46:40 – 4:48:43
and GPT four OpenAI hasn't announced, but it's rumored that it has about 1,700,000,000,000 parameters that it was trained on. This is a long way from AlexNet here. It's scaling like NVIDIA's market cap. There is this interesting discovery basically t
41:12 / 42:16
from transcript41:12 – 42:16
...with core machine learning algorithms, it's often indirectly led me to problems that are relevant in applications beyond robotics. So for example, about ten years ago, I started working on end to end neural network training for robots. This included
6:10:16 / 6:11:38
from transcript6:10:16 – 6:11:38
label these images. Yeah. And if I'm remembering from our episode, basically, what happened is the AlexNet team did way better than anybody else had ever done. The complete step changed better. I think the error rate went from mislabeling images 25%
9:00 / 10:01
from transcript9:00 – 10:01
Have a podcast?
Run your podcast through Clypt
Get ranked clips, hooks, and ready-to-post copy from your own episodes. Free to try.