MLOps Assignment 1

Deep Learning & Machine Learning Classification

Prakhar Chauhan | Roll Number: B23BB1032

PyTorch + Mixed Precision Training

Project Overview

This project analyzes the performance of Deep Learning models (ResNet-18 & ResNet-50) and Machine Learning models (SVM) on MNIST and FashionMNIST datasets under various configurations.

99.4%
Best MNIST Accuracy
92.6%
Best FashionMNIST Accuracy
8x
GPU Speedup
16
Experiments
# Key Configuration Dataset Split: 70% Train | 10% Val | 20% Test Models: ResNet-18, ResNet-50 (pretrained=False) Optimizers: SGD, Adam Learning Rates: 0.001, 0.0001 Batch Sizes: 16, 32

Q1(a): Deep Learning Classification Results

MNIST Dataset
Batch Size Optimizer Learning Rate ResNet-18 Acc (%) ResNet-50 Acc (%)
16 SGD 0.001 98.0 98.6
16 SGD 0.0001 98.4 98.9
16 Adam 0.001 99.0 99.3
16 Adam 0.0001 99.2 99.4
32 SGD 0.001 97.9 98.4
32 SGD 0.0001 98.3 98.8
32 Adam 0.001 98.9 99.2
32 Adam 0.0001 99.1 99.3
FashionMNIST Dataset
Batch Size Optimizer Learning Rate ResNet-18 Acc (%) ResNet-50 Acc (%)
16 SGD 0.001 88.9 90.4
16 SGD 0.0001 89.4 91.0
16 Adam 0.001 91.38 92.4
16 Adam 0.0001 91.53 92.6
32 SGD 0.001 88.4 89.6
32 SGD 0.0001 89.1 90.6
32 Adam 0.001 90.9 92.0
32 Adam 0.0001 91.1 92.3

Key Findings

  • Adam optimizer consistently outperformed SGD due to adaptive learning rates
  • Lower learning rates (0.0001) improved stability and generalization
  • ResNet-50 achieved slightly higher accuracy than ResNet-18, but with increased computational cost
  • MNIST achieved significantly higher accuracy than FashionMNIST

Q1(b): SVM Classifier Results

Training SVM with varying kernels (Polynomial, RBF) and hyperparameters.

Dataset Kernel Best Hyperparameters Test Accuracy (%) Train Time (ms)
MNIST Poly C=10, degree=2 95.20 5,204.85
MNIST RBF C=10, gamma=scale 95.55 8,001.53
FashionMNIST Poly C=10, degree=2 86.80 6,729.06
FashionMNIST RBF C=5, gamma=scale 87.45 6,820.15

SVM Analysis

  • RBF kernel slightly outperformed Polynomial kernel on both datasets
  • Deep Learning models (ResNet) significantly outperform SVM on these image classification tasks
  • SVM training time is much faster compared to deep learning approaches

Q2: CPU vs GPU Performance Analysis

Comparison of training time and FLOPs count for ResNet models on different hardware using FashionMNIST dataset.

Compute Batch Optimizer LR ResNet-18 Acc R-18 Time (ms) R-18 FLOPs ResNet-50 Acc R-50 Time (ms) R-50 FLOPs
CPU 16 SGD 0.001 89.20% 4,250,000 0.03G 88.50% 13,900,000 0.08G
CPU 16 Adam 0.001 90.10% 4,050,000 0.03G 89.80% 13,600,000 0.08G
GPU 16 SGD 0.001 91.73% 545,392 0.03G 90.40% 1,663,622 0.08G
GPU 16 Adam 0.001 92.04% 533,329 0.03G 91.90% 1,663,071 0.08G

Performance Insights

  • GPU training provides ~7-8× speedup compared to CPU across all models
  • FLOPs remain constant for a given architecture (hardware-independent)
  • Deeper models like ResNet-50 require substantially more computation
  • GPU acceleration is crucial for practical deep learning workflows

Resources & Links

GitHub Repository

View source code and files

Google Colab Notebook

Run experiments interactively

Jupyter Notebook

View the .ipynb file

Documentation

Read the full README