http://max-amb.com/series/the-making-of-a-mlp/Recent content in The making of a MLP on Max's blogHugoen-gbSun, 17 Aug 2025 00:00:00 +0100http://max-amb.com/blog/the_main_components_of_the_mlp/Sun, 17 Aug 2025 00:00:00 +0100http://max-amb.com/blog/the_main_components_of_the_mlp/<details> <summary>Contents</summary> <nav id="TableOfContents"> <ul> <li><a href="#the-structure-of-the-network">The structure of the network</a></li> <li><a href="#the-new-function">The new function</a> <ul> <li><a href="#arguments">Arguments</a></li> <li><a href="#network-setup">Network setup</a></li> <li><a href="#weight-initialisation">Weight initialisation</a> <ul> <li><a href="#initialisation-options">Initialisation options</a></li> </ul> </li> <li><a href="#bias-initialisation">Bias initialisation</a></li> <li><a href="#returning">Returning</a></li> </ul> </li> <li><a href="#the-forward-pass-function">The forward pass function</a> <ul> <li><a href="#arguments-1">Arguments</a></li> <li><a href="#return-value">Return value</a></li> <li><a href="#initialisation">Initialisation</a></li> <li><a href="#all-layers-but-one">All layers but one</a></li> <li><a href="#the-one-layer">The one layer</a></li> </ul> </li> <li><a href="#conclusion">Conclusion</a></li> <li><a href="#appendix">Appendix</a> <ul> <li><a href="#the-rng-tangent">The RNG tangent</a></li> </ul> </li> </ul> </nav> </details> <p>This is the second iteration in the <a href="http://max-amb.com/series/the-making-of-a-mlp/">series</a> where we are building a Multi-Layer-Perceptron (MLP) from scratch! This post will consider the main sections of the code in the MLP, which are the:</p>http://max-amb.com/blog/the_maths_behind_the_mlp/Wed, 13 Aug 2025 00:00:00 +0000http://max-amb.com/blog/the_maths_behind_the_mlp/<details> <summary>Contents</summary> <nav id="TableOfContents"> <ul> <li><a href="#why-from-scratch">Why from scratch</a></li> <li><a href="#intuition">Intuition?</a></li> <li><a href="#notation">Notation</a></li> <li><a href="#forward-pass">Forward pass</a> <ul> <li><a href="#matrix-notation">Matrix notation</a> <ul> <li><a href="#vector-of-z-values">Vector of z values</a></li> <li><a href="#vector-of-neuron-values">Vector of neuron values</a> <ul> <li><a href="#non-linearity">Non-linearity</a></li> </ul> </li> <li><a href="#conclusion">Conclusion</a></li> </ul> </li> </ul> </li> <li><a href="#backpropagation">Backpropagation</a> <ul> <li><a href="#output-layer-derivatives">Output layer derivatives</a> <ul> <li><a href="#bias-derivative">Bias derivative</a></li> <li><a href="#weight-derivative">Weight derivative</a></li> <li><a href="#delta">Delta</a></li> <li><a href="#representation-of-output-layer-backpropagation-as-a-matrix-operation">Representation of output layer backpropagation as a matrix operation</a> <ul> <li><a href="#added-notation">Added notation</a></li> </ul> </li> <li><a href="#conclusion-of-the-output-layer-derivatives">Conclusion of the output layer derivatives</a></li> </ul> </li> <li><a href="#hidden-layer-derivatives">Hidden layer derivatives</a> <ul> <li><a href="#intuition-behind-the-derivative-sum">Intuition behind the derivative sum</a></li> <li><a href="#derivative-of-the-bias">Derivative of the bias</a></li> <li><a href="#derivative-of-the-weight">Derivative of the weight</a></li> <li><a href="#deltas-again">Delta&rsquo;s again</a></li> <li><a href="#representation-of-hidden-layer-backpropagation-as-a-matrix-operation">Representation of hidden layer backpropagation as a matrix operation</a></li> <li><a href="#conclusion-of-the-hidden-layer-derivatives">Conclusion of the hidden layer derivatives</a></li> </ul> </li> </ul> </li> <li><a href="#summary">Summary</a></li> </ul> </nav> </details> <p>This blog(/tutorial maybe?) is the first part in a walk-through of the process I followed to build a multi-layer-perceptron (MLP) from scratch in rust. Followed by using it to classify the <a href="https://en.wikipedia.org/wiki/MNIST_database">MNIST dataset</a>. The source code for the MLP can be found <a href="https://github.com/max-amb/number_recognition">here</a>.</p>