Neural Network
Neural Networks are a specific subset of machine learning models inspired by the human brain, consisting of interconnected layers of nodes (neurons). They are composed of an input layer, one or more hidden layers, and an output layer.
There are various types of neural networks, such as feedforward neural networks (FNN), convolutional neural networks (CNN), deep neural networks (DNN) and recurrent neural networks (RNN).
Types of Neural Networks
Feedforward Neural Network (FNN):
- Simplest type where connections between nodes do not form cycles
- Information moves in one direction
- Use Cases: Classification and regression
Deep Neural Network (DNN):
- Type of feedforward neural network with multiple hidden layers (hence "deep").
- DNNs can learn complex patterns in data by stacking layers of neurons.
- Use Cases: Speech recognition, Image classification, NLP, …
Convolutional Neural Network (CNN):
- Specialized type designed for processing structured grid data, such as images.
- Uses convolutional layers to automatically detect and learn spatial hierarchies of features (like edges, textures, and shapes).
- Use Cases: Image recognition, object detection, video analysis
Recurrent Neural Network (RNN):
- Designed for sequential data.
- Have connections that loop back on themselves
- Allows them to maintain a memory of previous inputs.
- Suitable for tasks where context is important.
- Use Cases: Time series prediction, NLP, and speech recognition.
Terminology
- MLP: Multi-Layer Perceptron, the classic type of neural networks
- Layer: A group of neurons
- Weights: Strengths of connections between neurons
- Bias: Inputs multiplied by weights & summed up with an output neuron's bias
- Epoch: Going once through the entire training data
- Activation Function: A nonlinear function that the output goes through
- Binary Classifiers: Take a float vector in and give an integer out
- Supervised Learning: Training by giving input and expected output
- Mini-Batching: Updating weights and biases several times during an epoch
- Learning Rate: How much to update the weights during training
- Gradient Descent: How neural networks are trained.
- Cost Function: Says how wrong the output is for that input.