486 lines
17 KiB
Plaintext
486 lines
17 KiB
Plaintext
{
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5,
|
||
"metadata": {
|
||
"name": "FDP_Student_Stress_Prediction",
|
||
"kernelspec": { "name": "python3", "display_name": "Python 3" },
|
||
"language_info": { "name": "python", "version": "3.10.0" },
|
||
"accelerator": "CPU",
|
||
"colab": { "provenance": [] }
|
||
},
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 🎓 FDP: Student Stress Level Prediction\n",
|
||
"## Data Science Pipeline — Hands-on with Google Colab\n",
|
||
"\n",
|
||
"**Speaker:** Kapil Panwar | **Gyannetra Pvt Ltd**\n",
|
||
"\n",
|
||
"---\n",
|
||
"### 🔍 What We'll Do Today:\n",
|
||
"1. Load & explore a college student dataset\n",
|
||
"2. Handle categorical data using **One-Hot Encoding** (including Favorite Colors! 🌈)\n",
|
||
"3. Train a Machine Learning model to predict student stress\n",
|
||
"4. Evaluate how accurate our model is\n",
|
||
"5. Interpret results\n",
|
||
"\n",
|
||
"> **No installation needed. Just press Shift+Enter on each cell!**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 1: Import Libraries\n",
|
||
"import pandas as pd\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import seaborn as sns\n",
|
||
"from sklearn.model_selection import train_test_split\n",
|
||
"from sklearn.ensemble import RandomForestClassifier\n",
|
||
"from sklearn.metrics import accuracy_score, classification_report, confusion_matrix\n",
|
||
"\n",
|
||
"print(\"✅ All libraries loaded successfully!\")\n",
|
||
"print(f\"pandas: {pd.__version__}\")\n",
|
||
"print(f\"numpy: {np.__version__}\")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 1: Load the Dataset 📂\n",
|
||
"\n",
|
||
"We're using a dataset of **300 college students**. It contains information about their department,\n",
|
||
"attendance, favorite color, study habits, and whether they have high stress or not."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 2: Load Data\n",
|
||
"# Option A: From GitHub (recommended)\n",
|
||
"url = \"https://raw.githubusercontent.com/gnpl_admin/fdp-data-science-pipeline/main/fdp-colab-content/student_stress_data.csv\"\n",
|
||
"\n",
|
||
"# Option B: If you have the file locally (upload to Colab)\n",
|
||
"# from google.colab import files\n",
|
||
"# uploaded = files.upload()\n",
|
||
"# df = pd.read_csv('student_stress_data.csv')\n",
|
||
"\n",
|
||
"df = pd.read_csv(url)\n",
|
||
"print(f\"✅ Dataset loaded! Shape: {df.shape[0]} rows × {df.shape[1]} columns\")\n",
|
||
"df.head(10)"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### 📊 What does each column mean?\n",
|
||
"\n",
|
||
"| Column | Type | Description |\n",
|
||
"|--------|------|-------------|\n",
|
||
"| Student_ID | ID | Unique student identifier |\n",
|
||
"| Department | Categorical | Science, Commerce, Arts, Engineering |\n",
|
||
"| Attendance_Level | Categorical | Regular, Irregular, Borderline |\n",
|
||
"| Favorite_Color | Categorical | Red, Blue, Green, Yellow, Purple, Orange |\n",
|
||
"| Weekly_Study_Hours | Numerical | Hours studied per week (2-25) |\n",
|
||
"| Assignment_Submission_Pct | Numerical | Assignment submission rate (40-100%) |\n",
|
||
"| Backlogs_Count | Numerical | Number of pending backlogs (0-5) |\n",
|
||
"| Stress_Level | **Target** | 0 = Low Stress, 1 = High Stress |"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 3: Explore Data Types & Missing Values\n",
|
||
"print(\"📋 Data Info:\")\n",
|
||
"print(\"-\" * 40)\n",
|
||
"df.info()\n",
|
||
"print()\n",
|
||
"print(\"📊 Missing values:\")\n",
|
||
"print(df.isnull().sum())\n",
|
||
"print()\n",
|
||
"print(\"✅ No missing values! Clean data.\")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 4: Statistical Summary\n",
|
||
"print(\"📈 Statistical Summary (Numerical Columns):\")\n",
|
||
"print(\"-\" * 40)\n",
|
||
"df.describe()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 2: Exploratory Data Analysis (EDA) 🔍\n",
|
||
"\n",
|
||
"Let's understand our data before building a model."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 5: Target Distribution\n",
|
||
"stress_counts = df['Stress_Level'].value_counts()\n",
|
||
"print(\"🎯 Stress Level Distribution:\")\n",
|
||
"print(f\"Low Stress (0): {stress_counts[0]} students\")\n",
|
||
"print(f\"High Stress (1): {stress_counts[1]} students\")\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Simple bar chart\n",
|
||
"plt.figure(figsize=(6, 4))\n",
|
||
"colors = ['#4CAF50', '#F44336']\n",
|
||
"bars = plt.bar(['Low Stress (0)', 'High Stress (1)'], [stress_counts[0], stress_counts[1]], color=colors)\n",
|
||
"plt.title('Student Stress Level Distribution', fontsize=14, fontweight='bold')\n",
|
||
"plt.ylabel('Number of Students')\n",
|
||
"for bar in bars:\n",
|
||
" height = bar.get_height()\n",
|
||
" plt.text(bar.get_x() + bar.get_width()/2, height, f'{int(height)}', ha='center', va='bottom')\n",
|
||
"plt.show()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 6: Favorite Color Distribution\n",
|
||
"print(\"🌈 Favorite Color Distribution:\")\n",
|
||
"color_counts = df['Favorite_Color'].value_counts()\n",
|
||
"print(color_counts)\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Bar chart with actual colors\n",
|
||
"plt.figure(figsize=(8, 4))\n",
|
||
"color_map = {'Red': '#FF0000', 'Blue': '#2196F3', 'Green': '#4CAF50',\n",
|
||
" 'Yellow': '#FFEB3B', 'Purple': '#9C27B0', 'Orange': '#FF9800'}\n",
|
||
"bar_colors = [color_map[c] for c in color_counts.index]\n",
|
||
"bars = plt.bar(color_counts.index, color_counts.values, color=bar_colors, edgecolor='black', linewidth=0.5)\n",
|
||
"plt.title('Favorite Color Distribution', fontsize=14, fontweight='bold')\n",
|
||
"plt.ylabel('Number of Students')\n",
|
||
"for bar in bars:\n",
|
||
" height = bar.get_height()\n",
|
||
" plt.text(bar.get_x() + bar.get_width()/2, height, f'{int(height)}', ha='center', va='bottom')\n",
|
||
"plt.show()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 7: Department vs Stress Level\n",
|
||
"print(\"🏫 Stress by Department:\")\n",
|
||
"dept_stress = pd.crosstab(df['Department'], df['Stress_Level'])\n",
|
||
"print(dept_stress)\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Stacked bar chart\n",
|
||
"dept_stress.plot(kind='bar', stacked=True, color=['#4CAF50', '#F44336'], figsize=(8, 4))\n",
|
||
"plt.title('Stress Level by Department', fontsize=14, fontweight='bold')\n",
|
||
"plt.xlabel('Department')\n",
|
||
"plt.ylabel('Number of Students')\n",
|
||
"plt.legend(['Low Stress', 'High Stress'])\n",
|
||
"plt.xticks(rotation=0)\n",
|
||
"plt.show()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 3: Feature Engineering — One-Hot Encoding 🔄\n",
|
||
"\n",
|
||
"### 🎯 Why Do We Need This?\n",
|
||
"\n",
|
||
"Machine learning models understand **numbers**, not words like \"Red\" or \"Science\".\n",
|
||
"But we can't just assign Red=1, Blue=2 because that implies an order (Blue > Red).\n",
|
||
"\n",
|
||
"### ✅ Solution: One-Hot Encoding\n",
|
||
"\n",
|
||
"Creates a separate column for each category with 0 or 1.\n",
|
||
"\n",
|
||
"**Example — Favorite Color encoding:**\n",
|
||
"\n",
|
||
"| Original | Red | Blue | Green | Yellow | Purple | Orange |\n",
|
||
"|----------|-----|------|-------|--------|--------|--------|\n",
|
||
"| Red | 1 | 0 | 0 | 0 | 0 | 0 |\n",
|
||
"| Blue | 0 | 1 | 0 | 0 | 0 | 0 |\n",
|
||
"| Green | 0 | 0 | 1 | 0 | 0 | 0 |\n",
|
||
"\n",
|
||
"**Think of it like a light switch panel** — each color has its own switch, exactly one is ON!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 8: One-Hot Encode Categorical Columns\n",
|
||
"print(\"🔄 Before One-Hot Encoding — Categorical columns:\")\n",
|
||
"print(df[['Department', 'Attendance_Level', 'Favorite_Color']].head())\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Apply one-hot encoding\n",
|
||
"categorical_cols = ['Department', 'Attendance_Level', 'Favorite_Color']\n",
|
||
"df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=False)\n",
|
||
"\n",
|
||
"print(f\"✅ After encoding: {df_encoded.shape[1]} columns (was {df.shape[1]})\")\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Show the new columns that start with 'Favorite_Color'\n",
|
||
"color_cols = [c for c in df_encoded.columns if c.startswith('Favorite_Color')]\n",
|
||
"print(f\"🌈 New one-hot columns for Favorite_Color ({len(color_cols)} total):\")\n",
|
||
"print(color_cols)\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Show a sample of the encoded color columns\n",
|
||
"print(\"Sample of one-hot encoded colors:\")\n",
|
||
"df_encoded[['Student_ID', 'Favorite_Color_Red', 'Favorite_Color_Blue', \n",
|
||
" 'Favorite_Color_Green', 'Favorite_Color_Yellow',\n",
|
||
" 'Favorite_Color_Purple', 'Favorite_Color_Orange']].head(10)"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 9: Visualize One-Hot Encoding\n",
|
||
"print(\"🎯 Let's see which colors have more stressed students:\")\n",
|
||
"\n",
|
||
"# Create a cross-tabulation\n",
|
||
"color_stress = pd.crosstab(df['Favorite_Color'], df['Stress_Level'])\n",
|
||
"color_stress['Stress_Rate'] = (color_stress[1] / (color_stress[0] + color_stress[1]) * 100).round(1)\n",
|
||
"color_stress = color_stress.sort_values('Stress_Rate', ascending=False)\n",
|
||
"print(color_stress)\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Bar chart\n",
|
||
"plt.figure(figsize=(8, 4))\n",
|
||
"ax = color_stress['Stress_Rate'].plot(kind='bar', color=['#FF0000', '#2196F3', '#4CAF50', '#FFEB3B', '#9C27B0', '#FF9800'],\n",
|
||
" edgecolor='black', linewidth=0.5)\n",
|
||
"plt.title('Stress Rate by Favorite Color', fontsize=14, fontweight='bold')\n",
|
||
"plt.xlabel('Favorite Color')\n",
|
||
"plt.ylabel('Stress Rate (%)')\n",
|
||
"plt.xticks(rotation=0)\n",
|
||
"for i, v in enumerate(color_stress['Stress_Rate']):\n",
|
||
" plt.text(i, v + 0.5, f'{v}%', ha='center', fontweight='bold')\n",
|
||
"plt.ylim(0, 100)\n",
|
||
"plt.show()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 4: Train a Machine Learning Model 🤖\n",
|
||
"\n",
|
||
"We'll use a **Random Forest Classifier** — it's like asking 100 mini-experts to vote, then taking the majority decision. Easy to understand and very powerful!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 10: Prepare Features & Target\n",
|
||
"# Drop Student_ID (not useful) and Stress_Level (our target)\n",
|
||
"X = df_encoded.drop(['Student_ID', 'Stress_Level'], axis=1)\n",
|
||
"y = df_encoded['Stress_Level']\n",
|
||
"\n",
|
||
"print(f\"Features (X): {X.shape[1]} columns, {X.shape[0]} rows\")\n",
|
||
"print(f\"Target (y): {y.shape[0]} values\")\n",
|
||
"print(f\"\\nFeature columns: {list(X.columns)[:10]}... (+ {len(X.columns)-10} more)\")\n",
|
||
"print(f\"\\nTarget distribution:\\n{y.value_counts().to_string()}\")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 11: Split Data into Train & Test\n",
|
||
"X_train, X_test, y_train, y_test = train_test_split(\n",
|
||
" X, y, test_size=0.3, random_state=42, stratify=y\n",
|
||
")\n",
|
||
"\n",
|
||
"print(f\"📚 Training set: {X_train.shape[0]} students ({X_train.shape[0]/len(df)*100:.0f}%)\")\n",
|
||
"print(f\"🧪 Test set: {X_test.shape[0]} students ({X_test.shape[0]/len(df)*100:.0f}%)\")\n",
|
||
"print()\n",
|
||
"print(\"✅ Training target distribution:\")\n",
|
||
"print(y_train.value_counts().to_string())\n",
|
||
"print()\n",
|
||
"print(\"✅ Test target distribution:\")\n",
|
||
"print(y_test.value_counts().to_string())"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 12: Train Random Forest Model\n",
|
||
"model = RandomForestClassifier(n_estimators=100, random_state=42)\n",
|
||
"model.fit(X_train, y_train)\n",
|
||
"\n",
|
||
"print(\"🌲✅ Model trained successfully!\")\n",
|
||
"print(f\"Using {model.n_estimators} decision trees\")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 5: Evaluate the Model 📊"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 13: Make Predictions\n",
|
||
"y_pred = model.predict(X_test)\n",
|
||
"accuracy = accuracy_score(y_test, y_pred)\n",
|
||
"\n",
|
||
"print(f\"🎯 Model Accuracy: {accuracy*100:.1f}%\")\n",
|
||
"print()\n",
|
||
"print(\"📋 Classification Report:\")\n",
|
||
"print(classification_report(y_test, y_pred, target_names=['Low Stress', 'High Stress']))"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 14: Confusion Matrix\n",
|
||
"cm = confusion_matrix(y_test, y_pred)\n",
|
||
"\n",
|
||
"plt.figure(figsize=(6, 5))\n",
|
||
"sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', \n",
|
||
" xticklabels=['Low Stress', 'High Stress'],\n",
|
||
" yticklabels=['Low Stress', 'High Stress'])\n",
|
||
"plt.title('Confusion Matrix', fontsize=14, fontweight='bold')\n",
|
||
"plt.xlabel('Predicted')\n",
|
||
"plt.ylabel('Actual')\n",
|
||
"plt.show()\n",
|
||
"\n",
|
||
"print(\"📊 Confusion Matrix Breakdown:\")\n",
|
||
"print(f\"✅ True Low Stress: {cm[0][0]} students correctly predicted\")\n",
|
||
"print(f\"❌ False High Stress: {cm[0][1]} low-stress students predicted as high\")\n",
|
||
"print(f\"❌ False Low Stress: {cm[1][0]} high-stress students predicted as low\")\n",
|
||
"print(f\"✅ True High Stress: {cm[1][1]} students correctly predicted\")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## Step 6: Feature Importance 🔑\n",
|
||
"\n",
|
||
"What factors most influence student stress? Let's find out!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Cell 15: Feature Importance\n",
|
||
"importance = pd.DataFrame({\n",
|
||
" 'Feature': X.columns,\n",
|
||
" 'Importance': model.feature_importances_\n",
|
||
"}).sort_values('Importance', ascending=False)\n",
|
||
"\n",
|
||
"print(\"🔑 Top 10 Most Important Factors:\")\n",
|
||
"print(\"-\" * 40)\n",
|
||
"for i, row in importance.head(10).iterrows():\n",
|
||
" bar = '█' * int(row['Importance'] * 50)\n",
|
||
" print(f\"{row['Feature']:40s} {bar} {row['Importance']*100:.1f}%\")\n",
|
||
"print()\n",
|
||
"\n",
|
||
"# Plot top 10\n",
|
||
"plt.figure(figsize=(10, 6))\n",
|
||
"top10 = importance.head(10)\n",
|
||
"bars = plt.barh(range(len(top10)), top10['Importance'].values, color='#8ab4f8')\n",
|
||
"plt.yticks(range(len(top10)), top10['Feature'].values)\n",
|
||
"plt.xlabel('Importance Score')\n",
|
||
"plt.title('Top 10 Features Influencing Student Stress', fontsize=14, fontweight='bold')\n",
|
||
"plt.gca().invert_yaxis()\n",
|
||
"for i, v in enumerate(top10['Importance']):\n",
|
||
" plt.text(v + 0.005, i, f'{v*100:.1f}%', va='center', fontweight='bold')\n",
|
||
"plt.tight_layout()\n",
|
||
"plt.show()"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"## 🎯 Summary — What We Learned Today\n",
|
||
"\n",
|
||
"| Step | What We Did | Key Takeaway |\n",
|
||
"|------|-------------|--------------|\n",
|
||
"| 1 | Loaded data | CSV files are the standard format |\n",
|
||
"| 2 | EDA | Visualized patterns before modeling |\n",
|
||
"| 3 | **One-Hot Encoding** | Converted colors (Red/Blue/Green) to numbers 📌 |\n",
|
||
"| 4 | Trained model | Random Forest — simple yet powerful |\n",
|
||
"| 5 | Evaluated | Measured accuracy & confusion matrix |\n",
|
||
"| 6 | Interpreted | Found most influential factors |\n",
|
||
"\n",
|
||
"### 📌 Key Concept: One-Hot Encoding\n",
|
||
"- Categorical data (like **Favorite Color**) cannot directly go into ML models\n",
|
||
"- One-hot encoding creates **separate 0/1 columns** for each category\n",
|
||
"- In pandas: <code>pd.get_dummies(df['Favorite_Color'])</code>\n",
|
||
"- Think of it as a **light switch panel** — one switch per color!\n",
|
||
"\n",
|
||
"---\n",
|
||
"### 🙏 Thank You!\n",
|
||
"\n",
|
||
"**Kapil Panwar** \n",
|
||
"Gyannetra Pvt Ltd, Pune, Maharashtra \n",
|
||
"*India's Premier AI Consultancy*"
|
||
]
|
||
}
|
||
]
|
||
} |