Understanding Neural Networks

# CHAPTER 5

Understanding Neural Networks #

1. Introduction #

2. Learning Objectives #

• Explain the biological inspiration behind artificial neurons.

• Define Weights and Biases.

• Understand the architecture of Input, Hidden, and Output layers.

• Explain the concept of Forward Propagation.

• Explain how a network "learns" using Backpropagation.

3. Biological vs. Artificial Neurons #

• Biological Neuron: Receives electrical signals through *Dendrites*. If the signal is strong enough, the neuron "fires" and sends the signal down its *Axon* to the next neuron.

• Artificial Neuron (Perceptron): Receives numerical inputs ($X$). It multiplies those inputs by certain values, adds them up, and if the final number is high enough, it passes a mathematical signal to the next neuron.

4. Weights and Biases #

• Weights ($W$): Imagine you are predicting house prices based on Square Footage and Age. Square Footage is very important, so the network gives it a high *Weight* (e.g., $X \times 5.0$). Age is less important, so it gets a low Weight (e.g., $X \times 0.2$).

• Bias ($b$): A base value added to the equation to shift the result. (e.g., Even if Square Footage is 0, a plot of land still has a base value of $50,000).

The fundamental equation inside a single neuron is: Output = (Input * Weight) + Bias

5. Network Architecture (Layers) #

1. 1. Input Layer: The raw data entering the network. If your dataset has 3 columns (Age, Salary, Credit Score), your input layer has exactly 3 neurons.

1. 2. Hidden Layers: The layers in the middle. This is where the magic happens. A network can have 1 hidden layer or 100 (Deep Learning). Each layer extracts more complex patterns from the data.

1. 3. Output Layer: The final prediction. For a binary Yes/No prediction, you only need 1 output neuron.

6. Forward Propagation (Making a Guess) #

1. 1. The raw data enters the Input Layer.

1. 2. The data is multiplied by the Weights, the Biases are added, and the signals pass into the Hidden Layers.

1. 3. Finally, a number pops out of the Output Layer.

7. Backpropagation (Learning from Mistakes) #

1. 1. Loss Function: We use a mathematical formula to calculate exactly *how wrong* the guess was compared to the true answer (100% Cat).

1. 2. Backpropagation: The network sends an error signal backward (from right to left) through the network.

1. 3. Optimizing: Using calculus (Gradient Descent), the network slightly adjusts every single Weight and Bias in the hidden layers. It increases the weights that lead to the correct answer and decreases the weights that caused the error.

*This process of Forward Guessing and Backward Adjusting is repeated thousands of times (Epochs). Slowly, the network "learns" the perfect Weights to identify a Cat!*

8. Neural Network Diagram Visualization #

[Input 1] \
           [Hidden Neuron A] \
[Input 2] -                   - [Output Prediction]
           [Hidden Neuron B] /
[Input 3] /

*Every line connecting the inputs to the hidden neurons represents a distinct "Weight" that the network will adjust during Backpropagation.*

9. Common Mistakes #

• Assuming the network understands the data: A neural network does not know what a "Cat" is. It only knows that certain arrangements of pixel values (numbers) mathematically result in a 1 at the output layer.

• Thinking adding more layers always makes it smarter: Adding 50 hidden layers to a simple dataset will cause massive overfitting. The network will memorize the training data and fail in the real world.

10. Best Practices #

• Start Simple: When designing a neural network architecture, start with just one or two hidden layers. Only add depth if the model is underfitting (failing to learn the patterns).

11. Exercises #

1. 1. If a single neuron receives an input of X = 5, has a Weight of W = 2.0, and a Bias of b = 10, what is the raw mathematical output of that neuron?

1. 2. Explain the difference between Forward Propagation and Backpropagation.

12. MCQ Quiz with Answers #

13. Interview Questions #

• Q: Explain how Weights and Biases interact with Inputs in a standard artificial neuron.

• Q: Without using complex calculus, explain the purpose of Gradient Descent in the context of training a neural network.

14. FAQs #

15. Summary #

16. Next Chapter Recommendation #