What is Deep Learning?
Deep Learning is a subset of Machine Learning that uses artificial neural networks with multiple layers to learn complex patterns from data. It powers modern AI applications from image recognition to language models.
How Neural Networks Work
A neural network is inspired by the human brain. It consists of interconnected nodes (neurons) organized in layers that process information.
The Basic Neuron
import numpy as np
def simple_neuron(inputs, weights, bias):
"""
A single artificial neuron.
Multiplies inputs by weights, adds bias, applies activation.
"""
weighted_sum = np.dot(inputs, weights) + bias
# Sigmoid activation function
output = 1 / (1 + np.exp(-weighted_sum))
return output
# Example: Neuron that detects if an email might be spam
inputs = np.array([0.8, 0.2, 0.9]) # [has_links, is_short, has_free]
weights = np.array([2.5, -1.0, 3.0]) # Learned weights
bias = -2.0
output = simple_neuron(inputs, weights, bias)
print(f"Spam probability: {output:.4f}")
# Higher output = more likely spamNetwork Architecture
A deep neural network has an input layer, multiple hidden layers, and an output layer:
import torch
import torch.nn as nn
class DeepNeuralNetwork(nn.Module):
def __init__(self):
super().__init__()
# Input layer: 784 features (28x28 image flattened)
self.layer1 = nn.Linear(784, 256) # Hidden layer 1
self.layer2 = nn.Linear(256, 128) # Hidden layer 2
self.layer3 = nn.Linear(128, 64) # Hidden layer 3
self.output = nn.Linear(64, 10) # Output: 10 classes
self.relu = nn.ReLU()
def forward(self, x):
x = self.relu(self.layer1(x)) # Pass through layer 1
x = self.relu(self.layer2(x)) # Pass through layer 2
x = self.relu(self.layer3(x)) # Pass through layer 3
x = self.output(x) # Final output
return x
model = DeepNeuralNetwork()
print(model)
# Total parameters
total_params = sum(p.numel() for p in model.parameters())
print(f"Total parameters: {total_params:,}")Activation Functions
Activation functions introduce non-linearity, allowing the network to learn complex patterns:
import numpy as np
import matplotlib.pyplot as plt
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def relu(x):
return np.maximum(0, x)
def tanh(x):
return np.tanh(x)
def softmax(x):
exp_x = np.exp(x - np.max(x))
return exp_x / exp_x.sum()
# Compare activation functions
x = np.linspace(-5, 5, 100)
print("Activation function characteristics:")
print(f"Sigmoid output range: [{sigmoid(np.array([-10]))[0]:.4f}, {sigmoid(np.array([10]))[0]:.4f}]")
print(f"ReLU output range: [{relu(np.array([-10]))[0]:.1f}, {relu(np.array([10]))[0]:.1f}]")
print(f"Tanh output range: [{np.tanh(-10):.4f}, {np.tanh(10):.4f}]")
# Softmax example
logits = np.array([2.0, 1.0, 0.5, 0.1])
probabilities = softmax(logits)
print(f"Softmax probabilities: {probabilities}")
print(f"Sum of probabilities: {probabilities.sum():.4f}")Training Neural Networks
Backpropagation
Backpropagation is the algorithm used to train neural networks. It calculates how much each weight contributed to the error and updates weights accordingly.
import torch
import torch.nn as nn
import torch.optim as optim
# Simple training loop
model = nn.Sequential(
nn.Linear(10, 64),
nn.ReLU(),
nn.Linear(64, 32),
nn.ReLU(),
nn.Linear(32, 1)
)
# Loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# Generate sample data
X = torch.randn(100, 10) # 100 samples, 10 features
y = torch.randn(100, 1) # Target values
# Training loop
for epoch in range(100):
# Forward pass
predictions = model(X)
loss = criterion(predictions, y)
# Backward pass (backpropagation)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch + 1) % 20 == 0:
print(f"Epoch {epoch+1}/100, Loss: {loss.item():.4f}")Gradient Descent Explained
def gradient_descent_example():
"""
Visualize gradient descent — how neural networks learn
"""
# Simple function: f(x) = x^2 (minimum at x=0)
x = 5.0 # Starting point
learning_rate = 0.1
print("Gradient Descent to find minimum of f(x) = x²")
for step in range(10):
gradient = 2 * x # Derivative of x²
x = x - learning_rate * gradient
print(f"Step {step+1}: x = {x:.4f}, f(x) = {x**2:.4f}")
print(f"Final: x ≈ {x:.4f} (should approach 0)")
gradient_descent_example()Types of Neural Networks
Convolutional Neural Networks (CNNs)
CNNs are designed for image processing. They use convolutional filters to detect visual features:
import torch.nn as nn
class SimpleCNN(nn.Module):
def __init__(self):
super().__init__()
self.conv_layers = nn.Sequential(
nn.Conv2d(1, 32, kernel_size=3, padding=1), # 28x28x32
nn.ReLU(),
nn.MaxPool2d(2), # 14x14x32
nn.Conv2d(32, 64, kernel_size=3, padding=1), # 14x14x64
nn.ReLU(),
nn.MaxPool2d(2), # 7x7x64
)
self.fc_layers = nn.Sequential(
nn.Linear(64 * 7 * 7, 128),
nn.ReLU(),
nn.Linear(128, 10) # 10 digit classes
)
def forward(self, x):
x = self.conv_layers(x)
x = x.view(x.size(0), -1) # Flatten
x = self.fc_layers(x)
return x
cnn = SimpleCNN()
# Test with random image tensor (batch_size=1, channels=1, height=28, width=28)
test_input = torch.randn(1, 1, 28, 28)
output = cnn(test_input)
print(f"CNN output shape: {output.shape}") # [1, 10]Recurrent Neural Networks (RNNs)
RNNs process sequential data like text and time series:
class SimpleRNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super().__init__()
self.hidden_size = hidden_size
self.rnn = nn.LSTM(input_size, hidden_size, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)
def forward(self, x):
lstm_out, _ = self.rnn(x)
output = self.fc(lstm_out[:, -1, :]) # Last time step
return output
# Example: Sequence classification
model = SimpleRNN(input_size=10, hidden_size=32, output_size=2)
test_sequence = torch.randn(1, 20, 10) # Batch=1, SeqLen=20, Features=10
output = model(test_sequence)
print(f"RNN output: {output.shape}") # [1, 2]Real-World Deep Learning Application
# Complete MNIST digit classifier
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
# Data loading
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,))
])
train_loader = DataLoader(
datasets.MNIST('./data', train=True, download=True, transform=transform),
batch_size=64, shuffle=True
)
# Model
model = nn.Sequential(
nn.Flatten(),
nn.Linear(784, 512),
nn.ReLU(),
nn.Dropout(0.2),
nn.Linear(512, 256),
nn.ReLU(),
nn.Dropout(0.2),
nn.Linear(256, 10)
)
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()
# Train for 5 epochs
for epoch in range(5):
model.train()
total_loss = 0
for batch_X, batch_y in train_loader:
optimizer.zero_grad()
output = model(batch_X)
loss = criterion(output, batch_y)
loss.backward()
optimizer.step()
total_loss += loss.item()
avg_loss = total_loss / len(train_loader)
print(f"Epoch {epoch+1}/5, Average Loss: {avg_loss:.4f}")Frequently Asked Questions
What is the difference between deep learning and machine learning?
Deep Learning is a subset of machine learning that uses neural networks with many layers. Traditional machine learning often requires manual feature engineering, while deep learning automatically learns features from raw data.
How many layers make a network “deep”?
There’s no strict definition, but networks with more than one hidden layer are generally considered “deep.” Modern deep learning models can have hundreds of layers.
What hardware do I need for deep learning?
A modern GPU (NVIDIA RTX 3060 or better) is recommended. For learning, you can use Google Colab’s free GPUs. For production, consider cloud GPUs from AWS, GCP, or Azure.
How long does it take to train a deep learning model?
Training time varies from minutes (simple models on small datasets) to weeks (large models like GPT-4 on massive datasets). A typical image classifier trains in 10-30 minutes on a GPU.
Want to learn more about deep learning? Join our Telegram Community for hands-on deep learning projects, PyTorch tutorials, and discussions with ML engineers.