Deep Learning with CNN

Introduction to CNNs

• Motivation for CNNs over fully connected networks
• Biological inspiration: visual cortex and receptive fields
• Key use-cases of CNNs (image classification, object detection, etc.)

Fundamentals of CNN Architecture

• Input data representation (images as tensors)
• Layers in a CNN:
  • Convolutional Layer
  • Activation Function (ReLU, Leaky ReLU)
  • Pooling Layer (Max Pooling, Average Pooling)
  • Fully Connected Layer
• Feature maps and receptive fields
• Padding, stride, and dilation
• Flattening

Advanced CNN Building Blocks

• Batch Normalization
• Dropout for regularization
• Global Average Pooling
• Skip connections and Residual Blocks (ResNet concept)
• Depthwise Separable Convolution (MobileNet)
• Dilated/Atrous Convolutions

CNN Architectures

• LeNet-5
• AlexNet
• VGGNet
• GoogLeNet (Inception)
• ResNet
• DenseNet
• MobileNet / EfficientNet
• Comparison of architectures (size, speed, accuracy)

Training CNNs

• Loss functions (Cross-Entropy, Hinge Loss)
• Optimizers (SGD, Adam, RMSprop)
• Backpropagation through convolutional layers
• Overfitting and underfitting in CNNs
• Data augmentation techniques
• Learning rate schedules and callbacks

Transfer Learning with CNNs

• Feature extraction vs. fine-tuning
• Using pre-trained models (e.g., from ImageNet)
• Custom classification heads on top of CNN backbones
• Domain adaptation

Object Detection & Localization (CNN-based)

• Image classification vs. object detection
• Sliding Window + CNN approach
• R-CNN, Fast R-CNN, Faster R-CNN
• YOLO (You Only Look Once)
• SSD (Single Shot Detector)
• Anchor boxes and bounding boxes

Semantic Segmentation

• Introduction to segmentation
• Fully Convolutional Networks (FCN)
• U-Net architecture
• SegNet
• Applications in medical imaging, satellite imagery

CNNs in Other Domains

• CNNs for video analysis (3D CNNs)
• 1D CNNs for time-series and NLP
• Spectrograms and audio classification

Visualization & Interpretability

• Visualizing feature maps
• Activation maximization
• Saliency maps
• Grad-CAM

CNN Implementation Tools

• TensorFlow/Keras CNN APIs
• PyTorch for CNNs
• Transfer learning with Keras Applications or torchvision.models
• Model deployment for CNNs

Recent Trends in CNN Research

• Vision Transformers vs CNNs
• Hybrid CNN-ViT models
• Self-supervised learning for CNNs
• Efficient CNN architectures for mobile and embedded systems