Deep learning
Understanding Optimisers Through Hessians
- Jack Norrie
- Machine learning , Deep learning
- 25 May, 2025
During my recent autodiff library project I got the opportunity to implement common optimizers from scratch. Furthermore, while investigating the vanishing gradient problem I benefited greatly from di
read moreA Deep Dive On Vanishing and Exploding Gradients
- Jack Norrie
- Machine learning , Deep learning
- 18 May, 2025
I was first introduced to the vanishing/exploding gradients problem while conducting my Bachelor's thesis. At the time I was reading the textbook "Hands-On Machine Learning with Scikit-Learn and Tenso
read moreBuilding an Autodiff Library
- Jack Norrie
- Machine Learning , Deep Learning
- 10 May, 2025
Until recently, despite my extensive experience with auto-differentiation frameworks, I have never implemented one myself. I believe that implementing a tool that you commonly use yourself can yield g
read moreBackpropagation A Modern Explanation
- Jack Norrie
- Machine learning , Deep learning
- 05 May, 2025
Manual differentiation is prone to errors and does not scale to functions that involve thousands of parameters, e.g. neural networks. This necessitates procedures which are able to take functions and
read more