Again

It's the next day. 3:54 PM. Starting on 51:10 / 2:25:51 of building micrograd

Actually restarting

6:20 PM 51:10/ 2:25:51 of building micrograd

1. Confirming all my code is the same and not breaking...
2. Quickly typing up Karpathy's new pasted code...

Short break

Snacks and soy milk

Manual backpropagation

7:15 PM, 1:01:48 / 2:25:51 of building micrograd

1. Using Claude to find the typo in my written tanh function.

Addition The two splitting neurons have the same grad value as the "parent" neuron.

Multiplication The children neurons' grad values can be discovered in a sort of "criss-cross" pattern:

1. x2 neuron: grad is found out by getting the other child's data (var 1) times the parent's grad value ("Source code")
2. w2 is just a swap on the "other child's" data (use x2).

maybe this is the "aha" moment I keep hearing about. Dirty cliches. 1:07:10 / 2:25:51 of building micrograd

Saving is important

8:05 PM, 1:08:42

While doing manual backpropagation number dos, I was doing one cell, (o.grad = 1.0, n.grad = 0.5, and so on) deleting the lines which minimized the amount of blocks and lines. It looked nice and clean. I made a little label typo in the second previous block. So I fixed it and ran it. My gradient values were gone.

I fixed it by just pausing at a frame with all of the blocks visible. A bit tedious, now it's all back.

No more manual backpropagation. Neat!

Functions are pretty awesome. Even quicker functions are even cooler

Tiniest of errors

8:30 PM, 1:22:25 / 2:25:51

I've gone crazy reviewing the differences between my code and Karpathy's.. only to find out my backwards function logic needed just one tab press.

Session end

1. Finally understand how to do manual backpropagation (at least to this scale. Need to understand the deeper reasoning behind addition though)
2. Must be careful with Python's quirks with tabs.