feat: initial FDP content - Data Science Pipeline with Colab notebook

fdp-colab-content/FDP_Student_Stress_Prediction.ipynb

@@ -0,0 +1,486 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 5,
+ "metadata": {
+  "name": "FDP_Student_Stress_Prediction",
+  "kernelspec": { "name": "python3", "display_name": "Python 3" },
+  "language_info": { "name": "python", "version": "3.10.0" },
+  "accelerator": "CPU",
+  "colab": { "provenance": [] }
+ },
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 🎓 FDP: Student Stress Level Prediction\n",
+    "## Data Science Pipeline — Hands-on with Google Colab\n",
+    "\n",
+    "**Speaker:** Kapil Panwar | **Gyannetra Pvt Ltd**\n",
+    "\n",
+    "---\n",
+    "### 🔍 What We'll Do Today:\n",
+    "1. Load & explore a college student dataset\n",
+    "2. Handle categorical data using **One-Hot Encoding** (including Favorite Colors! 🌈)\n",
+    "3. Train a Machine Learning model to predict student stress\n",
+    "4. Evaluate how accurate our model is\n",
+    "5. Interpret results\n",
+    "\n",
+    "> **No installation needed. Just press Shift+Enter on each cell!**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 1: Import Libraries\n",
+    "import pandas as pd\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import seaborn as sns\n",
+    "from sklearn.model_selection import train_test_split\n",
+    "from sklearn.ensemble import RandomForestClassifier\n",
+    "from sklearn.metrics import accuracy_score, classification_report, confusion_matrix\n",
+    "\n",
+    "print(\"✅ All libraries loaded successfully!\")\n",
+    "print(f\"pandas: {pd.__version__}\")\n",
+    "print(f\"numpy: {np.__version__}\")"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 1: Load the Dataset 📂\n",
+    "\n",
+    "We're using a dataset of **300 college students**. It contains information about their department,\n",
+    "attendance, favorite color, study habits, and whether they have high stress or not."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 2: Load Data\n",
+    "# Option A: From GitHub (recommended)\n",
+    "url = \"https://raw.githubusercontent.com/gnpl_admin/fdp-data-science-pipeline/main/fdp-colab-content/student_stress_data.csv\"\n",
+    "\n",
+    "# Option B: If you have the file locally (upload to Colab)\n",
+    "# from google.colab import files\n",
+    "# uploaded = files.upload()\n",
+    "# df = pd.read_csv('student_stress_data.csv')\n",
+    "\n",
+    "df = pd.read_csv(url)\n",
+    "print(f\"✅ Dataset loaded! Shape: {df.shape[0]} rows × {df.shape[1]} columns\")\n",
+    "df.head(10)"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 📊 What does each column mean?\n",
+    "\n",
+    "| Column | Type | Description |\n",
+    "|--------|------|-------------|\n",
+    "| Student_ID | ID | Unique student identifier |\n",
+    "| Department | Categorical | Science, Commerce, Arts, Engineering |\n",
+    "| Attendance_Level | Categorical | Regular, Irregular, Borderline |\n",
+    "| Favorite_Color | Categorical | Red, Blue, Green, Yellow, Purple, Orange |\n",
+    "| Weekly_Study_Hours | Numerical | Hours studied per week (2-25) |\n",
+    "| Assignment_Submission_Pct | Numerical | Assignment submission rate (40-100%) |\n",
+    "| Backlogs_Count | Numerical | Number of pending backlogs (0-5) |\n",
+    "| Stress_Level | **Target** | 0 = Low Stress, 1 = High Stress |"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 3: Explore Data Types & Missing Values\n",
+    "print(\"📋 Data Info:\")\n",
+    "print(\"-\" * 40)\n",
+    "df.info()\n",
+    "print()\n",
+    "print(\"📊 Missing values:\")\n",
+    "print(df.isnull().sum())\n",
+    "print()\n",
+    "print(\"✅ No missing values! Clean data.\")"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 4: Statistical Summary\n",
+    "print(\"📈 Statistical Summary (Numerical Columns):\")\n",
+    "print(\"-\" * 40)\n",
+    "df.describe()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 2: Exploratory Data Analysis (EDA) 🔍\n",
+    "\n",
+    "Let's understand our data before building a model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 5: Target Distribution\n",
+    "stress_counts = df['Stress_Level'].value_counts()\n",
+    "print(\"🎯 Stress Level Distribution:\")\n",
+    "print(f\"Low Stress (0): {stress_counts[0]} students\")\n",
+    "print(f\"High Stress (1): {stress_counts[1]} students\")\n",
+    "print()\n",
+    "\n",
+    "# Simple bar chart\n",
+    "plt.figure(figsize=(6, 4))\n",
+    "colors = ['#4CAF50', '#F44336']\n",
+    "bars = plt.bar(['Low Stress (0)', 'High Stress (1)'], [stress_counts[0], stress_counts[1]], color=colors)\n",
+    "plt.title('Student Stress Level Distribution', fontsize=14, fontweight='bold')\n",
+    "plt.ylabel('Number of Students')\n",
+    "for bar in bars:\n",
+    "    height = bar.get_height()\n",
+    "    plt.text(bar.get_x() + bar.get_width()/2, height, f'{int(height)}', ha='center', va='bottom')\n",
+    "plt.show()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 6: Favorite Color Distribution\n",
+    "print(\"🌈 Favorite Color Distribution:\")\n",
+    "color_counts = df['Favorite_Color'].value_counts()\n",
+    "print(color_counts)\n",
+    "print()\n",
+    "\n",
+    "# Bar chart with actual colors\n",
+    "plt.figure(figsize=(8, 4))\n",
+    "color_map = {'Red': '#FF0000', 'Blue': '#2196F3', 'Green': '#4CAF50',\n",
+    "             'Yellow': '#FFEB3B', 'Purple': '#9C27B0', 'Orange': '#FF9800'}\n",
+    "bar_colors = [color_map[c] for c in color_counts.index]\n",
+    "bars = plt.bar(color_counts.index, color_counts.values, color=bar_colors, edgecolor='black', linewidth=0.5)\n",
+    "plt.title('Favorite Color Distribution', fontsize=14, fontweight='bold')\n",
+    "plt.ylabel('Number of Students')\n",
+    "for bar in bars:\n",
+    "    height = bar.get_height()\n",
+    "    plt.text(bar.get_x() + bar.get_width()/2, height, f'{int(height)}', ha='center', va='bottom')\n",
+    "plt.show()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 7: Department vs Stress Level\n",
+    "print(\"🏫 Stress by Department:\")\n",
+    "dept_stress = pd.crosstab(df['Department'], df['Stress_Level'])\n",
+    "print(dept_stress)\n",
+    "print()\n",
+    "\n",
+    "# Stacked bar chart\n",
+    "dept_stress.plot(kind='bar', stacked=True, color=['#4CAF50', '#F44336'], figsize=(8, 4))\n",
+    "plt.title('Stress Level by Department', fontsize=14, fontweight='bold')\n",
+    "plt.xlabel('Department')\n",
+    "plt.ylabel('Number of Students')\n",
+    "plt.legend(['Low Stress', 'High Stress'])\n",
+    "plt.xticks(rotation=0)\n",
+    "plt.show()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 3: Feature Engineering — One-Hot Encoding 🔄\n",
+    "\n",
+    "### 🎯 Why Do We Need This?\n",
+    "\n",
+    "Machine learning models understand **numbers**, not words like \"Red\" or \"Science\".\n",
+    "But we can't just assign Red=1, Blue=2 because that implies an order (Blue > Red).\n",
+    "\n",
+    "### ✅ Solution: One-Hot Encoding\n",
+    "\n",
+    "Creates a separate column for each category with 0 or 1.\n",
+    "\n",
+    "**Example — Favorite Color encoding:**\n",
+    "\n",
+    "| Original | Red | Blue | Green | Yellow | Purple | Orange |\n",
+    "|----------|-----|------|-------|--------|--------|--------|\n",
+    "| Red      | 1   | 0    | 0     | 0      | 0      | 0      |\n",
+    "| Blue     | 0   | 1    | 0     | 0      | 0      | 0      |\n",
+    "| Green    | 0   | 0    | 1     | 0      | 0      | 0      |\n",
+    "\n",
+    "**Think of it like a light switch panel** — each color has its own switch, exactly one is ON!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 8: One-Hot Encode Categorical Columns\n",
+    "print(\"🔄 Before One-Hot Encoding — Categorical columns:\")\n",
+    "print(df[['Department', 'Attendance_Level', 'Favorite_Color']].head())\n",
+    "print()\n",
+    "\n",
+    "# Apply one-hot encoding\n",
+    "categorical_cols = ['Department', 'Attendance_Level', 'Favorite_Color']\n",
+    "df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=False)\n",
+    "\n",
+    "print(f\"✅ After encoding: {df_encoded.shape[1]} columns (was {df.shape[1]})\")\n",
+    "print()\n",
+    "\n",
+    "# Show the new columns that start with 'Favorite_Color'\n",
+    "color_cols = [c for c in df_encoded.columns if c.startswith('Favorite_Color')]\n",
+    "print(f\"🌈 New one-hot columns for Favorite_Color ({len(color_cols)} total):\")\n",
+    "print(color_cols)\n",
+    "print()\n",
+    "\n",
+    "# Show a sample of the encoded color columns\n",
+    "print(\"Sample of one-hot encoded colors:\")\n",
+    "df_encoded[['Student_ID', 'Favorite_Color_Red', 'Favorite_Color_Blue', \n",
+    "            'Favorite_Color_Green', 'Favorite_Color_Yellow',\n",
+    "            'Favorite_Color_Purple', 'Favorite_Color_Orange']].head(10)"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 9: Visualize One-Hot Encoding\n",
+    "print(\"🎯 Let's see which colors have more stressed students:\")\n",
+    "\n",
+    "# Create a cross-tabulation\n",
+    "color_stress = pd.crosstab(df['Favorite_Color'], df['Stress_Level'])\n",
+    "color_stress['Stress_Rate'] = (color_stress[1] / (color_stress[0] + color_stress[1]) * 100).round(1)\n",
+    "color_stress = color_stress.sort_values('Stress_Rate', ascending=False)\n",
+    "print(color_stress)\n",
+    "print()\n",
+    "\n",
+    "# Bar chart\n",
+    "plt.figure(figsize=(8, 4))\n",
+    "ax = color_stress['Stress_Rate'].plot(kind='bar', color=['#FF0000', '#2196F3', '#4CAF50', '#FFEB3B', '#9C27B0', '#FF9800'],\n",
+    "                                      edgecolor='black', linewidth=0.5)\n",
+    "plt.title('Stress Rate by Favorite Color', fontsize=14, fontweight='bold')\n",
+    "plt.xlabel('Favorite Color')\n",
+    "plt.ylabel('Stress Rate (%)')\n",
+    "plt.xticks(rotation=0)\n",
+    "for i, v in enumerate(color_stress['Stress_Rate']):\n",
+    "    plt.text(i, v + 0.5, f'{v}%', ha='center', fontweight='bold')\n",
+    "plt.ylim(0, 100)\n",
+    "plt.show()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 4: Train a Machine Learning Model 🤖\n",
+    "\n",
+    "We'll use a **Random Forest Classifier** — it's like asking 100 mini-experts to vote, then taking the majority decision. Easy to understand and very powerful!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 10: Prepare Features & Target\n",
+    "# Drop Student_ID (not useful) and Stress_Level (our target)\n",
+    "X = df_encoded.drop(['Student_ID', 'Stress_Level'], axis=1)\n",
+    "y = df_encoded['Stress_Level']\n",
+    "\n",
+    "print(f\"Features (X): {X.shape[1]} columns, {X.shape[0]} rows\")\n",
+    "print(f\"Target (y): {y.shape[0]} values\")\n",
+    "print(f\"\\nFeature columns: {list(X.columns)[:10]}... (+ {len(X.columns)-10} more)\")\n",
+    "print(f\"\\nTarget distribution:\\n{y.value_counts().to_string()}\")"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 11: Split Data into Train & Test\n",
+    "X_train, X_test, y_train, y_test = train_test_split(\n",
+    "    X, y, test_size=0.3, random_state=42, stratify=y\n",
+    ")\n",
+    "\n",
+    "print(f\"📚 Training set: {X_train.shape[0]} students ({X_train.shape[0]/len(df)*100:.0f}%)\")\n",
+    "print(f\"🧪 Test set: {X_test.shape[0]} students ({X_test.shape[0]/len(df)*100:.0f}%)\")\n",
+    "print()\n",
+    "print(\"✅ Training target distribution:\")\n",
+    "print(y_train.value_counts().to_string())\n",
+    "print()\n",
+    "print(\"✅ Test target distribution:\")\n",
+    "print(y_test.value_counts().to_string())"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 12: Train Random Forest Model\n",
+    "model = RandomForestClassifier(n_estimators=100, random_state=42)\n",
+    "model.fit(X_train, y_train)\n",
+    "\n",
+    "print(\"🌲✅ Model trained successfully!\")\n",
+    "print(f\"Using {model.n_estimators} decision trees\")"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 5: Evaluate the Model 📊"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 13: Make Predictions\n",
+    "y_pred = model.predict(X_test)\n",
+    "accuracy = accuracy_score(y_test, y_pred)\n",
+    "\n",
+    "print(f\"🎯 Model Accuracy: {accuracy*100:.1f}%\")\n",
+    "print()\n",
+    "print(\"📋 Classification Report:\")\n",
+    "print(classification_report(y_test, y_pred, target_names=['Low Stress', 'High Stress']))"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 14: Confusion Matrix\n",
+    "cm = confusion_matrix(y_test, y_pred)\n",
+    "\n",
+    "plt.figure(figsize=(6, 5))\n",
+    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', \n",
+    "            xticklabels=['Low Stress', 'High Stress'],\n",
+    "            yticklabels=['Low Stress', 'High Stress'])\n",
+    "plt.title('Confusion Matrix', fontsize=14, fontweight='bold')\n",
+    "plt.xlabel('Predicted')\n",
+    "plt.ylabel('Actual')\n",
+    "plt.show()\n",
+    "\n",
+    "print(\"📊 Confusion Matrix Breakdown:\")\n",
+    "print(f\"✅ True Low Stress: {cm[0][0]} students correctly predicted\")\n",
+    "print(f\"❌ False High Stress: {cm[0][1]} low-stress students predicted as high\")\n",
+    "print(f\"❌ False Low Stress: {cm[1][0]} high-stress students predicted as low\")\n",
+    "print(f\"✅ True High Stress: {cm[1][1]} students correctly predicted\")"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Step 6: Feature Importance 🔑\n",
+    "\n",
+    "What factors most influence student stress? Let's find out!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "# Cell 15: Feature Importance\n",
+    "importance = pd.DataFrame({\n",
+    "    'Feature': X.columns,\n",
+    "    'Importance': model.feature_importances_\n",
+    "}).sort_values('Importance', ascending=False)\n",
+    "\n",
+    "print(\"🔑 Top 10 Most Important Factors:\")\n",
+    "print(\"-\" * 40)\n",
+    "for i, row in importance.head(10).iterrows():\n",
+    "    bar = '█' * int(row['Importance'] * 50)\n",
+    "    print(f\"{row['Feature']:40s} {bar} {row['Importance']*100:.1f}%\")\n",
+    "print()\n",
+    "\n",
+    "# Plot top 10\n",
+    "plt.figure(figsize=(10, 6))\n",
+    "top10 = importance.head(10)\n",
+    "bars = plt.barh(range(len(top10)), top10['Importance'].values, color='#8ab4f8')\n",
+    "plt.yticks(range(len(top10)), top10['Feature'].values)\n",
+    "plt.xlabel('Importance Score')\n",
+    "plt.title('Top 10 Features Influencing Student Stress', fontsize=14, fontweight='bold')\n",
+    "plt.gca().invert_yaxis()\n",
+    "for i, v in enumerate(top10['Importance']):\n",
+    "    plt.text(v + 0.005, i, f'{v*100:.1f}%', va='center', fontweight='bold')\n",
+    "plt.tight_layout()\n",
+    "plt.show()"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## 🎯 Summary — What We Learned Today\n",
+    "\n",
+    "| Step | What We Did | Key Takeaway |\n",
+    "|------|-------------|--------------|\n",
+    "| 1 | Loaded data | CSV files are the standard format |\n",
+    "| 2 | EDA | Visualized patterns before modeling |\n",
+    "| 3 | **One-Hot Encoding** | Converted colors (Red/Blue/Green) to numbers 📌 |\n",
+    "| 4 | Trained model | Random Forest — simple yet powerful |\n",
+    "| 5 | Evaluated | Measured accuracy & confusion matrix |\n",
+    "| 6 | Interpreted | Found most influential factors |\n",
+    "\n",
+    "### 📌 Key Concept: One-Hot Encoding\n",
+    "- Categorical data (like **Favorite Color**) cannot directly go into ML models\n",
+    "- One-hot encoding creates **separate 0/1 columns** for each category\n",
+    "- In pandas: <code>pd.get_dummies(df['Favorite_Color'])</code>\n",
+    "- Think of it as a **light switch panel** — one switch per color!\n",
+    "\n",
+    "---\n",
+    "### 🙏 Thank You!\n",
+    "\n",
+    "**Kapil Panwar**  \n",
+    "Gyannetra Pvt Ltd, Pune, Maharashtra  \n",
+    "*India's Premier AI Consultancy*"
+   ]
+  }
+ ]
+}