Feature Scaling

Why Scale Features?

Many machine learning algorithms are sensitive to the magnitude of features. Without scaling, features with larger ranges can dominate the model's learning process.

Basic MinMaxScaler Usage

Scikit-learn's MinMaxScaler makes normalization straightforward:

# WHY? Machine learning algorithms need features on the same scale
# WHAT? MinMaxScaler squeezes all values into a fixed range (default: 0 to 1)
from sklearn.preprocessing import MinMaxScaler
import numpy as np
import pandas as pd

# Sample data - NOTICE THE DIFFERENT SCALES!
# PROBLEM: age (25-65), income (30k-120k), score (0.5-0.9) are on wildly different scales
# SCENARIO: Customer dataset where income dominates distance calculations
data = pd.DataFrame({
    'age': [25, 35, 45, 55, 65],              # Range: 40 years (25 to 65)
    'income': [30000, 50000, 75000, 90000, 120000],  # Range: $90,000 (30k to 120k)
    'score': [0.5, 0.7, 0.6, 0.8, 0.9]        # Range: 0.4 (0.5 to 0.9)
})

print("Original Data:")
print(data)
print("\nOriginal Statistics:")
print(data.describe().round(2))

# Output:
# Original Data:
#    age  income  score
# 0   25   30000    0.5  # ← Young customer, low income
# 1   35   50000    0.7  # ← Mid-age, mid-income
# 2   45   75000    0.6  # ← Mid-age, higher income
# 3   55   90000    0.8  # ← Older, high income
# 4   65  120000    0.9  # ← Oldest, highest income
# 
# Original Statistics:
#              age    income  score
# count   5.00      5.00    5.00
# mean   45.00  73000.00    0.70    # ← Average values
# std    15.81  34351.13    0.16    # ← Standard deviation (spread)
# min    25.00  30000.00    0.50    # ← Minimum values
# max    65.00 120000.00    0.90    # ← Maximum values
#
# NOTICE THE PROBLEM:
# - Income values are HUGE (30,000 - 120,000) compared to age (25-65)
# - In distance-based algorithms (KNN, SVM), income would dominate!
# - A $10,000 income difference counts WAY more than a 10-year age difference

# STEP 1: Create the scaler object
# WHY MinMaxScaler? It's perfect when you want all features in [0, 1] range
scaler = MinMaxScaler()  # Default: feature_range=(0, 1)

# STEP 2: fit_transform() = Learn min/max from data AND transform it in one go
# WHAT IT DOES:
# 1. Learns: age_min=25, age_max=65, income_min=30000, income_max=120000, etc.
# 2. Applies formula: (X - X_min) / (X_max - X_min) to every value
# RESULT: Returns numpy array (not DataFrame)
data_scaled = scaler.fit_transform(data)

# STEP 3: Convert back to DataFrame for readability
# WHY? fit_transform returns a numpy array - hard to read without column names
data_scaled_df = pd.DataFrame(data_scaled, columns=data.columns)

print("Scaled Data (0 to 1):")
print(data_scaled_df)
print("\nScaled Statistics:")
print(data_scaled_df.describe().round(2))

# Output:
# Scaled Data (0 to 1):
#     age    income  score
# 0  0.00  0.000000   0.00  # ← Minimum values → 0.0 (25 years, $30k, 0.5 score)
# 1  0.25  0.222222   0.50  # ← 25% along age range, 22% along income range
# 2  0.50  0.500000   0.25  # ← Exactly middle age (45), middle income ($75k)
# 3  0.75  0.666667   0.75  # ← 75% along age range
# 4  1.00  1.000000   1.00  # ← Maximum values → 1.0 (65 years, $120k, 0.9 score)
# 
# Scaled Statistics:
#        age  income  score
# min   0.00    0.00   0.00  # ← ALL minimums are now 0!
# max   1.00    1.00   1.00  # ← ALL maximums are now 1!
#
# THE MAGIC:
# - Now ALL features range from 0 to 1
# - Income no longer dominates (was 30k-120k, now 0-1)
# - All features contribute equally to distance calculations
# - The SHAPE of each distribution is preserved (proportions stay the same)

Custom Range with feature_range

You can specify a custom output range using the feature_range parameter:

# WHY [-1, 1] range? Some neural networks (especially with tanh activation) work better with negative values
# WHAT? feature_range parameter lets you choose ANY output range!
scaler_custom = MinMaxScaler(feature_range=(-1, 1))

# Same process: fit and transform
# FORMULA CHANGES: X_scaled = 2 * (X - X_min) / (X_max - X_min) - 1
# RESULT: Minimum → -1, Maximum → 1, Middle → 0
data_scaled_custom = scaler_custom.fit_transform(data)
data_scaled_custom_df = pd.DataFrame(data_scaled_custom, columns=data.columns)

print("Scaled Data (-1 to 1):")
print(data_scaled_custom_df)

# Output:
# Scaled Data (-1 to 1):
#     age    income  score
# 0 -1.00 -1.000000  -1.00  # ← Minimum values now map to -1
# 1 -0.50 -0.555556   0.00  # ← Middle of age range (35 years)
# 2  0.00  0.000000  -0.50  # ← Middle values map to 0
# 3  0.50  0.333333   0.50
# 4  1.00  1.000000   1.00  # ← Maximum values now map to +1
#
# KEY DIFFERENCE:
# [0, 1] range: min=0, max=1, middle=0.5
# [-1, 1] range: min=-1, max=+1, middle=0
# SAME proportional spacing, just different numbers!

Understanding fit(), transform(), and fit_transform()

# CRITICAL WORKFLOW: fit on train, transform on test
# WHY? Prevents data leakage - test data must remain "unseen" during training!
from sklearn.model_selection import train_test_split

# STEP 1: Create larger dataset
np.random.seed(42)
X = pd.DataFrame({
    'feature_1': np.random.uniform(10, 100, 100),      # Random values 10-100
    'feature_2': np.random.uniform(1000, 10000, 100)   # Random values 1000-10000
})
y = np.random.randint(0, 2, 100)  # Binary target (0 or 1)

# STEP 2: Split the data (80% train, 20% test)
# IMPORTANT: Split BEFORE scaling!
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# STEP 3: Initialize scaler
scaler = MinMaxScaler()

# STEP 4: Fit ONLY on training data
# WHAT IT DOES: Learns min/max from ONLY the 80 training samples
# WHY? In production, you won't know test data's min/max!
scaler.fit(X_train)

print("Learned from training data:")
print(f"  Feature 1: min={scaler.data_min_[0]:.2f}, max={scaler.data_max_[0]:.2f}")
print(f"  Feature 2: min={scaler.data_min_[1]:.2f}, max={scaler.data_max_[1]:.2f}")

# STEP 5: Transform both train and test using the SAME learned min/max
# WHY? Ensures consistent scaling - test uses training's parameters
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

print("\nTraining set range:")
print(f"  Feature 1: [{X_train_scaled[:, 0].min():.3f}, {X_train_scaled[:, 0].max():.3f}]")
print(f"  Feature 2: [{X_train_scaled[:, 1].min():.3f}, {X_train_scaled[:, 1].max():.3f}]")

print("\nTest set range (may exceed [0,1]):")
print(f"  Feature 1: [{X_test_scaled[:, 0].min():.3f}, {X_test_scaled[:, 0].max():.3f}]")
print(f"  Feature 2: [{X_test_scaled[:, 1].min():.3f}, {X_test_scaled[:, 1].max():.3f}]")

# Output:
# Learned from training data:
#   Feature 1: min=12.34, max=98.76  # ← Learned from 80 training samples
#   Feature 2: min=1234.56, max=9876.54
# 
# Training set range:
#   Feature 1: [0.000, 1.000]  # ← Training data perfectly spans [0, 1]
#   Feature 2: [0.000, 1.000]
# 
# Test set range (may exceed [0,1]):
#   Feature 1: [0.012, 0.987]  # ← Test might not reach exact 0 or 1
#   Feature 2: [0.023, 0.956]  # ← Could even exceed if test has new extremes!
#
# WHAT IF TEST HAS NEW EXTREME?
# Example: If test has feature_1 = 105 (> training max 98.76), it scales to >1.0
# This is EXPECTED and CORRECT - we maintain training's reference frame

Sensitivity to Outliers

# THE OUTLIER PROBLEM: One extreme value ruins everything!
# SCENARIO: Customer ages where one person entered 1000 by mistake
data_with_outlier = np.array([[10], [20], [30], [40], [1000]])  # ← 1000 is an outlier!

scaler = MinMaxScaler()
scaled = scaler.fit_transform(data_with_outlier)

print("Original:", data_with_outlier.flatten())
print("Scaled:  ", scaled.flatten().round(3))

# Output:
# Original: [  10   20   30   40 1000]  # ← One huge outlier (1000)
# Scaled:   [0.    0.01  0.02  0.03  1.  ]  # ← All normal values crushed to 0.00-0.03!
# 
# THE PROBLEM:
# - Formula: (X - min) / (max - min) = (X - 10) / (1000 - 10)
# - For X=20: (20-10)/(990) = 10/990 = 0.01
# - For X=30: (30-10)/(990) = 20/990 = 0.02
# - For X=40: (40-10)/(990) = 30/990 = 0.03
# - The range 990 (caused by outlier) makes normal values indistinguishable!
# 
# WHAT THIS MEANS:
# - 99% of your data is now squeezed into 0.00-0.03
# - You've lost all the valuable variation in the normal range
# - The model can barely tell the difference between 10, 20, 30, 40!

Inverse Transform

Convert scaled values back to original scale (useful for interpreting predictions):

# WHY INVERSE TRANSFORM? To convert predictions back to original units!
# SCENARIO: Model predicts scaled age (0.5) - what's the actual age?

# STEP 1: Start with original data
original = np.array([[25, 50000], [35, 75000], [45, 100000]])
columns = ['age', 'income']

# STEP 2: Scale the data
scaler = MinMaxScaler()
scaled = scaler.fit_transform(original)
# Scaler remembers: age_min=25, age_max=45, income_min=50k, income_max=100k

print("Original:")
print(pd.DataFrame(original, columns=columns))
print("\nScaled:")
print(pd.DataFrame(scaled, columns=columns).round(3))

# STEP 3: Inverse transform - convert back to original scale
# FORMULA: X_original = X_scaled * (max - min) + min
# For age: X = 0.5 * (45 - 25) + 25 = 0.5 * 20 + 25 = 10 + 25 = 35
recovered = scaler.inverse_transform(scaled)
print("\nRecovered (inverse_transform):")
print(pd.DataFrame(recovered, columns=columns))

# Output:
# Original:
#    age  income
# 0   25   50000  # ← Minimum values
# 1   35   75000  # ← Middle values
# 2   45  100000  # ← Maximum values
# 
# Scaled:
#    age  income
# 0  0.0     0.0  # ← Min becomes 0
# 1  0.5     0.5  # ← Middle becomes 0.5
# 2  1.0     1.0  # ← Max becomes 1
# 
# Recovered (inverse_transform):
#     age   income
# 0  25.0  50000.0  # ← Back to original!
# 1  35.0  75000.0  # ← Perfect recovery
# 2  45.0 100000.0  # ← No information lost
#
# REAL-WORLD USE CASE:
# Your model predicts scaled age = 0.75
# inverse_transform converts it back: 0.75 * 20 + 25 = 40 years old
# Much easier to understand than "0.75"!

import numpy as np
data = np.array([[100], [200], [300], [400], [500]])

from sklearn.preprocessing import MinMaxScaler
import numpy as np

data = np.array([[100], [200], [300], [400], [500]])

scaler = MinMaxScaler(feature_range=(-1, 1))
scaled_data = scaler.fit_transform(data)

print("Scaled values:")
print(scaled_data.flatten())
# Output: [-1.  -0.5  0.   0.5  1. ]

Understanding Z-Score

The z-score tells you how many standard deviations a value is from the mean:

• z = 0: Value equals the mean
• z = 1: Value is 1 standard deviation above the mean
• z = -2: Value is 2 standard deviations below the mean

Basic StandardScaler Usage

# WHY STANDARDIZATION? Makes features comparable by measuring in "std deviations from mean"
# WHAT? StandardScaler shifts data to mean=0 and scales to std=1
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd

# Sample data - same as before, but now we'll center and scale differently
data = pd.DataFrame({
    'age': [25, 35, 45, 55, 65],              # Mean: 45, Std: ~14.14
    'income': [30000, 50000, 75000, 90000, 120000],  # Mean: 73000, Std: ~30663
    'score': [0.5, 0.7, 0.6, 0.8, 0.9]        # Mean: 0.7, Std: ~0.1414
})

print("Original Data:")
print(data)
print(f"\nMeans: age={data['age'].mean()}, income={data['income'].mean():.0f}")
print(f"Stds:  age={data['age'].std():.2f}, income={data['income'].std():.0f}")

# Output:
# Original Data:
#    age  income  score
# 0   25   30000    0.5  # ← Youngest, lowest income (both below mean)
# 1   35   50000    0.7  # ← Below mean age, below mean income
# 2   45   75000    0.6  # ← Exactly average age (45), slightly above mean income
# 3   55   90000    0.8  # ← Above mean age, above mean income
# 4   65  120000    0.9  # ← Oldest, highest income (both above mean)
# 
# Means: age=45.0, income=73000  # ← Center of the data
# Stds:  age=14.14, income=30663  # ← Spread/variability of the data

# STEP 1: Create StandardScaler
# WHY? We want ALL features to have mean=0 and std=1
scaler = StandardScaler()

# STEP 2: Fit and transform
# WHAT IT DOES:
# 1. Learns mean and std for each feature: age mean=45, age std=14.14, etc.
# 2. Applies formula: z = (X - mean) / std
# EXAMPLE: age=25 becomes z = (25 - 45) / 14.14 = -20 / 14.14 = -1.414
#          This means "25 years is 1.414 standard deviations BELOW the mean"
data_scaled = scaler.fit_transform(data)

# STEP 3: Convert to DataFrame
data_scaled_df = pd.DataFrame(data_scaled, columns=data.columns)

print("Standardized Data:")
print(data_scaled_df.round(3))
print(f"\nNew Means: {data_scaled_df.mean().round(10).values}")  # Should be [0, 0, 0]
print(f"New Stds:  {data_scaled_df.std(ddof=0).round(3).values}")  # Should be [1, 1, 1]

# Output:
# Standardized Data:
#      age    income  score
# 0 -1.414 -1.402849 -1.414  # ← 25 years is 1.414 std BELOW mean (young!)
# 1 -0.707 -0.749787  0.000  # ← 35 years is 0.707 std below mean
# 2  0.000  0.065249 -0.707  # ← 45 years = exactly the mean (z=0)!
# 3  0.707  0.554618  0.707  # ← 55 years is 0.707 std ABOVE mean
# 4  1.414  1.532769  1.414  # ← 65 years is 1.414 std ABOVE mean (old!)
# 
# New Means: [0. 0. 0.]  # ← ALL features now centered at 0!
# New Stds:  [1. 1. 1.]  # ← ALL features now have std=1!
#
# KEY INSIGHT:
# - Negative z-scores = below average
# - Positive z-scores = above average
# - z = 0 = exactly average
# - Most values fall between -2 and +2 (95% in normal distribution)
# - Unlike min-max, NO FIXED RANGE! Values can be any number (usually -3 to +3)

Accessing Learned Parameters

After fitting, you can access the learned mean and standard deviation:

# WHY ACCESS PARAMETERS? To understand what the scaler learned from your data
# SCENARIO: Debugging or documenting your scaling transformation

scaler = StandardScaler()
scaler.fit(data)  # Learn mean and std from data

print("Learned Means (one per feature):")
print(scaler.mean_)
# Shows the average value for each column that was subtracted

print("\nLearned Standard Deviations:")
print(np.sqrt(scaler.var_))  # scaler.scale_ also works
# Shows the std used to divide each column

print("\nLearned Variances:")
print(scaler.var_)  # Variance = std²

# Output:
# Learned Means (one per feature):
# [4.5000e+01 7.3000e+04 7.0000e-01]  # ← age mean=45, income mean=73000, score mean=0.7
# 
# Learned Standard Deviations:
# [1.41421356e+01 3.06659458e+04 1.41421356e-01]  # ← age std≈14.14, income std≈30666, score std≈0.14
# 
# Learned Variances:
# [2.00000000e+02 9.40400000e+08 2.00000000e-02]  # ← variance = std squared
#
# WHAT THIS TELLS YOU:
# - The scaler subtracts 45 from every age value, then divides by 14.14
# - Formula applied: z_age = (age - 45) / 14.14
# - These parameters are saved and used for transforming new data!

Standardization vs Normalization

Visualizing the Difference

# COMPARISON: StandardScaler vs MinMaxScaler on same data
# WHY? To see the different output ranges and characteristics
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler, MinMaxScaler

# STEP 1: Create sample data with different scales
np.random.seed(42)
data = pd.DataFrame({
    'small_scale': np.random.normal(5, 2, 1000),      # Mean 5, std 2
    'large_scale': np.random.normal(500, 100, 1000)   # Mean 500, std 100
})  # Different scales will be standardized differently

# STEP 2: Apply both scalers
standard_scaler = StandardScaler()  # Will center at 0, scale to std=1
minmax_scaler = MinMaxScaler()      # Will scale to [0, 1] range

data_standard = pd.DataFrame(
    standard_scaler.fit_transform(data), 
    columns=['small_scale', 'large_scale']
)
data_minmax = pd.DataFrame(
    minmax_scaler.fit_transform(data), 
    columns=['small_scale', 'large_scale']
)

# STEP 3: Compare statistics
print("Original Data:")
print(data.describe().round(2))
print("\nAfter StandardScaler:")
print(data_standard.describe().round(2))
print("\nAfter MinMaxScaler:")
print(data_minmax.describe().round(2))

# Output shows:
# StandardScaler: mean≈0, std≈1 for ALL features
# MinMaxScaler: min≈0, max≈1 for ALL features
#
# KEY DIFFERENCE:
# StandardScaler → Unbounded, centered at 0, most values in [-3, +3]
# MinMaxScaler → Bounded to [0, 1], compressed into that range

Handling Outliers in Standardization

# HOW DOES STANDARDIZATION HANDLE OUTLIERS?
# ANSWER: Better than MinMaxScaler, but still affected!

# SCENARIO: 4 normal data points + 1 extreme outlier
data_with_outliers = np.array([
    [10, 100],   # Normal point
    [12, 110],   # Normal point
    [11, 105],   # Normal point
    [10, 102],   # Normal point
    [100, 1000]  # OUTLIER! Far from others
])

scaler = StandardScaler()
scaled = scaler.fit_transform(data_with_outliers)

print("Original Data:")
print(data_with_outliers)
print("\nStandardized:")
print(scaled.round(3))

# Output:
# Original Data:
# [[  10  100]  # ← Normal values clustered around 10-12 and 100-110
#  [  12  110]
#  [  11  105]
#  [  10  102]
#  [ 100 1000]] # ← OUTLIER pulls mean and std away from normal cluster
# 
# Standardized:
# [[-0.601 -0.555]  # ← Normal points get negative z-scores (below mean)
#  [-0.547 -0.527]  # ← Normal points clustered together
#  [-0.574 -0.541]  # ← All normal points have similar z-scores (-0.5 to -0.6)
#  [-0.601 -0.549]  # ← Normal points still distinguishable from each other
#  [ 2.323  2.172]] # ← Outlier gets z-score of ~2.3 (2.3 std above mean)
#
# COMPARISON WITH MINMAXSCALER:
# MinMaxScaler would crush all normal points to 0.00-0.03 (compressed!)
# StandardScaler keeps normal points distinguishable (-0.5 to -0.6)
# The outlier gets a high z-score (~2.3) but doesn't destroy the rest
#
# VERDICT: StandardScaler handles outliers BETTER than MinMaxScaler
# But for heavy outliers, RobustScaler is still the best choice!

# You can disable centering or scaling independently
scaler = StandardScaler(with_mean=True, with_std=True)  # Default

# Only center (subtract mean), don't scale by std
scaler_center = StandardScaler(with_mean=True, with_std=False)

# Only scale by std, don't center
scaler_scale = StandardScaler(with_mean=False, with_std=True)

# Note: with_mean=False is required for sparse matrices

Using Pipeline for Clean Workflows

# BEST PRACTICE: Use Pipeline for clean, leak-free workflows!
# WHY? Automatically handles fit/transform correctly + cleaner code
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification

# STEP 1: Create sample dataset
X, y = make_classification(n_samples=1000, n_features=10, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# STEP 2: Create pipeline - Scaling + Model in one object
# WHY? The pipeline automatically:
# - Fits scaler on training data ONLY
# - Transforms training data with fitted scaler
# - Fits model on transformed training data
# - Transforms test data using training scaler (no leakage!)
pipeline = Pipeline([
    ('scaler', StandardScaler()),        # Step 1: Scale features
    ('classifier', LogisticRegression()) # Step 2: Train classifier
])

# STEP 3: Fit pipeline - one line does everything correctly!
pipeline.fit(X_train, y_train)
# Behind the scenes:
# 1. scaler.fit(X_train) - learns mean/std from training
# 2. X_train_scaled = scaler.transform(X_train) - scales training
# 3. classifier.fit(X_train_scaled, y_train) - trains model

# STEP 4: Predict - automatically scales test data correctly!
accuracy = pipeline.score(X_test, y_test)
print(f"Test Accuracy: {accuracy:.3f}")
# Behind the scenes:
# 1. X_test_scaled = scaler.transform(X_test) - scales test with TRAINING params
# 2. predictions = classifier.predict(X_test_scaled) - predicts

# Output:
# Test Accuracy: 0.850
#
# BENEFITS:
# ✓ No data leakage - scaler only fits on training data
# ✓ Cleaner code - no manual fit/transform calls
# ✓ Production ready - save entire pipeline with pickle/joblib
# ✓ Cross-validation friendly - works seamlessly with GridSearchCV

import numpy as np
scores = np.array([[65], [70], [75], [80], [85], [90], [95]])

from sklearn.preprocessing import StandardScaler
import numpy as np

scores = np.array([[65], [70], [75], [80], [85], [90], [95]])

scaler = StandardScaler()
scaled_scores = scaler.fit_transform(scores)

print("Scaled scores:", scaled_scores.flatten().round(3))
print(f"Mean: {scaled_scores.mean():.6f}")
print(f"Std:  {scaled_scores.std():.6f}")

# Output:
# Scaled scores: [-1.5 -1.  -0.5  0.   0.5  1.   1.5]
# Mean: 0.000000
# Std:  1.000000

Why Robust Scaling?

Both MinMaxScaler and StandardScaler are affected by outliers:

• MinMaxScaler: Uses min/max, directly affected by extreme values
• StandardScaler: Mean and std are pulled towards outliers
• RobustScaler: Median and IQR are resistant to extreme values

Basic RobustScaler Usage

# WHY ROBUST SCALING? When outliers would ruin min-max or standard scaling!
# WHAT? Uses median (middle value) and IQR (middle 50% spread) - both immune to outliers
from sklearn.preprocessing import RobustScaler
import numpy as np
import pandas as pd

# SCENARIO: Two features - one normal, one with HUGE outliers
data = pd.DataFrame({
    'normal_feature': [10, 12, 11, 13, 12, 11, 10, 12, 11, 13],  # Nice, consistent values
    'with_outliers': [10, 12, 11, 13, 12, 11, 10, 12, 100, 200]  # Last 2 are EXTREME outliers!
})

print("Original Data Statistics:")
print(data.describe().round(2))
print(f"\nNote the 'with_outliers' column has max={data['with_outliers'].max()}")

# Output:
# Original Data Statistics:
#        normal_feature  with_outliers
# count           10.00          10.00
# mean            11.50          39.10  # ← Mean pulled to 39 by outliers (should be ~11)!
# std              1.08          61.96  # ← Std inflated by outliers!
# min             10.00          10.00
# 25%             11.00          11.00  # ← 25th percentile (Q1) - not affected!
# 50%             11.50          12.00  # ← 50th percentile (median) - not affected!
# 75%             12.00          13.00  # ← 75th percentile (Q3) - not affected!
# max             13.00         200.00  # ← Max completely distorted by outliers
#
# KEY INSIGHT:
# - Mean and max are RUINED by outliers (100, 200)
# - Median (50%) and IQR (Q3-Q1 = 13-11 = 2) are UNAFFECTED!
# - RobustScaler will use median=12 and IQR=2 for scaling

# STEP 1: Create RobustScaler
# WHY? To scale using median and IQR instead of mean and std
robust_scaler = RobustScaler()

# STEP 2: Fit and transform
# WHAT IT DOES:
# 1. Learns median and IQR for each feature
# 2. Applies formula: (X - median) / IQR
# 
# FOR 'with_outliers' column:
#   - Median = 12 (middle value of 10,10,11,11,12,12,13,13,100,200)
#   - Q1 = 11 (25th percentile)
#   - Q3 = 13 (75th percentile)
#   - IQR = Q3 - Q1 = 13 - 11 = 2
#   - Formula: (X - 12) / 2
data_robust = robust_scaler.fit_transform(data)
data_robust_df = pd.DataFrame(data_robust, columns=data.columns)

print("After RobustScaler:")
print(data_robust_df.round(3))

# Output:
# After RobustScaler:
#    normal_feature  with_outliers
# 0          -1.500         -1.000  # ← (10-11.5)/1 = -1.5, (10-12)/2 = -1.0
# 1           0.500          0.000  # ← (12-11.5)/1 = 0.5, (12-12)/2 = 0.0
# 2          -0.500         -0.500  # ← (11-11.5)/1 = -0.5, (11-12)/2 = -0.5
# 3           1.500          0.500  # ← (13-11.5)/1 = 1.5, (13-12)/2 = 0.5
# 4           0.500          0.000
# 5          -0.500         -0.500
# 6          -1.500         -1.000
# 7           0.500          0.000
# 8          -0.500         44.000  # ← OUTLIER! (100-12)/2 = 88/2 = 44.0 (still large!)
# 9           1.500         94.000  # ← OUTLIER! (200-12)/2 = 188/2 = 94.0 (still large!)
#
# MAGIC HAPPENS:
# - Normal data (rows 0-7) is beautifully scaled around 0, range -1.5 to +1.5
# - Outliers (rows 8-9) have HUGE scaled values (44, 94) - clearly flagged!
# - Compare this to MinMaxScaler which would crush rows 0-7 to 0.00-0.03!
# - The bulk of the data uses the full scale; outliers don't ruin everything

Comparing All Three Scalers with Outliers

# THE ULTIMATE COMPARISON: MinMax vs Standard vs Robust with outliers
# SCENARIO: 97 normal values + 3 extreme outliers
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
import numpy as np
import pandas as pd

# Create realistic data
np.random.seed(42)
normal_data = np.random.normal(50, 10, 97)  # 97 people with normal scores ~50±10
outliers = np.array([200, 250, 300])         # 3 people with extreme scores
data = np.concatenate([normal_data, outliers]).reshape(-1, 1)

# Apply all three scalers to THE SAME data
minmax = MinMaxScaler().fit_transform(data)    # Uses min=~23, max=300
standard = StandardScaler().fit_transform(data) # Uses mean≈62, std≈39 (affected by outliers)
robust = RobustScaler().fit_transform(data)     # Uses median≈50, IQR≈13 (NOT affected!)

# Create comparison DataFrame
results = pd.DataFrame({
    'Original': data.flatten(),
    'MinMax': minmax.flatten(),
    'Standard': standard.flatten(),
    'Robust': robust.flatten()
})

print("First 5 rows (normal data):")
print(results.head().round(3))

print("\nLast 5 rows (includes outliers):")
print(results.tail().round(3))

# Output:
# First 5 rows (normal data):
#    Original  MinMax  Standard  Robust
# 0    54.967   0.175     0.063   0.318  # ← Normal value
# 1    48.617   0.150    -0.115   0.005  # ← Normal value
# 2    56.476   0.181     0.105   0.418  # ← Normal value
# 3    65.231   0.216     0.350   0.996  # ← Normal value
# 4    47.659   0.147    -0.141  -0.058  # ← Normal value
# 
# Last 5 rows (includes outliers):
#     Original  MinMax  Standard  Robust
# 95    54.530   0.173     0.050   0.289  # ← Last normal value
# 96    53.010   0.168     0.008   0.188  # ← Last normal value
# 97   200.000   0.693     4.108  10.393  # ← OUTLIER!
# 98   250.000   0.880     5.506  13.692  # ← OUTLIER!
# 99   300.000   1.000     6.903  16.991  # ← OUTLIER!
#
# ANALYSIS:
# MinMax: Normal data compressed to 0.14-0.22 (only 0.08 range!) → BAD
# Standard: Normal data gets -0.14 to 0.35, outliers get 4-7 → OKAY
# Robust: Normal data gets -0.06 to 1.0, outliers get 10-17 → BEST!
# 
# WINNER: RobustScaler uses the full scale for normal data while clearly flagging outliers

Configuring RobustScaler

# RobustScaler parameters
from sklearn.preprocessing import RobustScaler

# Default: uses median and IQR (Q1=25%, Q3=75%)
scaler_default = RobustScaler()

# Custom quantile range (e.g., use 10th to 90th percentile)
scaler_custom = RobustScaler(quantile_range=(10.0, 90.0))

# Disable centering (don't subtract median)
scaler_no_center = RobustScaler(with_centering=False)

# Disable scaling (don't divide by IQR)
scaler_no_scale = RobustScaler(with_scaling=False)

# Example with custom quantile range
data = np.array([[10], [20], [30], [40], [50], [60], [70], [80], [90], [100]])

scaler_25_75 = RobustScaler(quantile_range=(25.0, 75.0))  # Default IQR
scaler_10_90 = RobustScaler(quantile_range=(10.0, 90.0))  # Wider range

print("Original:", data.flatten())
print("25-75 IQR:", scaler_25_75.fit_transform(data).flatten().round(3))
print("10-90 IQR:", scaler_10_90.fit_transform(data).flatten().round(3))

# Output:
# Original: [ 10  20  30  40  50  60  70  80  90 100]
# 25-75 IQR: [-0.8 -0.6 -0.4 -0.2  0.   0.2  0.4  0.6  0.8  1. ]
# 10-90 IQR: [-0.5   -0.375 -0.25  -0.125  0.     0.125  0.25   0.375  0.5    0.625]

Accessing Learned Parameters

# ACCESSING PARAMETERS: What did RobustScaler learn?
# WHY? To understand and verify the scaling transformation
data = np.array([[10, 100], [20, 200], [30, 300], [40, 400], [50, 500], 
                 [60, 600], [70, 700], [80, 800], [90, 900], [100, 1000]])

scaler = RobustScaler()
scaler.fit(data)

print("Centers (Medians):")
print(scaler.center_)  # The median of each feature

print("\nScales (IQRs):")
print(scaler.scale_)   # The IQR (Q3-Q1) of each feature

# Output:
# Centers (Medians):
# [ 55. 550.]  # ← Median of column 1 = 55, median of column 2 = 550
# 
# Scales (IQRs):
# [ 40. 400.]  # ← IQR of column 1 = 40 (Q3=75, Q1=35, IQR=40)
#              # ← IQR of column 2 = 400 (Q3=750, Q1=350, IQR=400)
#
# WHAT THIS MEANS:
# - Every value in column 1 will have 55 subtracted, then divided by 40
# - Every value in column 2 will have 550 subtracted, then divided by 400
# - Formula: X_scaled = (X - center) / scale

Real-World Example: Salary Data

# REAL-WORLD SCENARIO: Employee salaries with CEO outliers
# WHY THIS MATTERS? Common in real datasets - most data is normal, few extremes
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
import pandas as pd
import numpy as np

# Simulated company salary data
np.random.seed(42)
salaries = pd.DataFrame({
    'employee_id': range(1, 101),
    'years_experience': np.random.randint(1, 20, 100),
    'salary': np.concatenate([
        np.random.normal(60000, 15000, 95),   # 95 regular employees: $45k-$75k
        np.array([500000, 750000, 1000000, 1500000, 2000000])  # 5 C-suite: $500k-$2M
    ])
})

print("Salary Statistics:")
print(salaries['salary'].describe().round(0))
print(f"\nMedian: ${salaries['salary'].median():,.0f}")  # Not affected by CEOs
print(f"Mean:   ${salaries['salary'].mean():,.0f}")    # Pulled up by CEOs!

# Output:
# Salary Statistics:
# count       100.0
# mean     117441.0  # ← MEAN pulled up by C-suite (should be ~$60k!)
# std      273619.0  # ← STD inflated by C-suite!
# min       23161.0
# 25%       51909.0  # ← Q1 = $51k (25% earn less)
# 50%       61106.0  # ← MEDIAN = $61k (typical employee!)
# 75%       72371.0  # ← Q3 = $72k (75% earn less)
# max     2000000.0  # ← MAX = $2M (CEO!)
# 
# Median: $61,106  # ← Represents typical employee (not affected by CEOs)
# Mean:   $117,441 # ← Nearly doubled by C-suite salaries!
#
# KEY INSIGHT:
# - 95 employees earn $45k-$75k (normal distribution)
# - 5 C-suite earn $500k-$2M (extreme outliers)
# - Mean ($117k) is misleading - no one actually earns that!
# - Median ($61k) is accurate - represents the typical employee

# Compare scalers on salary data
X = salaries[['salary']].values

minmax_scaled = MinMaxScaler().fit_transform(X)
standard_scaled = StandardScaler().fit_transform(X)
robust_scaled = RobustScaler().fit_transform(X)

# Check the distribution of scaled values for regular employees (first 95)
print("Scaled values for regular employees (first 95):")
print(f"MinMax range:   [{minmax_scaled[:95].min():.3f}, {minmax_scaled[:95].max():.3f}]")
print(f"Standard range: [{standard_scaled[:95].min():.3f}, {standard_scaled[:95].max():.3f}]")
print(f"Robust range:   [{robust_scaled[:95].min():.3f}, {robust_scaled[:95].max():.3f}]")

print("\nScaled values for C-suite (last 5):")
for i, name in enumerate(['CFO', 'COO', 'CTO', 'CEO1', 'CEO2'], 95):
    print(f"{name}: MinMax={minmax_scaled[i,0]:.3f}, Standard={standard_scaled[i,0]:.3f}, Robust={robust_scaled[i,0]:.3f}")

# Output:
# Scaled values for regular employees (first 95):
# MinMax range:   [0.000, 0.046]   <- Compressed to tiny range!
# Standard range: [-0.345, -0.166]  <- Compressed, all negative!
# Robust range:   [-1.855, 0.606]   <- Full range used!
# 
# Scaled values for C-suite (last 5):
# CFO:  MinMax=0.241, Standard=1.398, Robust=21.454
# COO:  MinMax=0.368, Standard=2.312, Robust=33.681
# CTO:  MinMax=0.494, Standard=3.227, Robust=45.909
# CEO1: MinMax=0.747, Standard=5.056, Robust=70.364
# CEO2: MinMax=1.000, Standard=6.885, Robust=94.818

# RobustScaler gives regular employees a sensible range (-2 to 0.6)
# while clearly identifying outliers (values >> 1)

import numpy as np
# Salaries with outliers
salaries = np.array([[45000], [52000], [48000], [55000], [51000], 
                     [49000], [53000], [500000], [750000]])

from sklearn.preprocessing import RobustScaler
import numpy as np

salaries = np.array([[45000], [52000], [48000], [55000], [51000], 
                     [49000], [53000], [500000], [750000]])

scaler = RobustScaler()
scaled = scaler.fit_transform(salaries)

print("Original -> Scaled:")
for orig, sc in zip(salaries.flatten(), scaled.flatten()):
    print(f"${orig:,} -> {sc:.3f}")

# Output shows regular salaries scaled between -1 and 1
# while outliers get large values (identifying them clearly)

Choosing the right scaler depends on your data characteristics and the algorithm you're using. Here's a comprehensive guide to help you decide.

Complete Comparison Table

Algorithm-Specific Recommendations

Practical Decision Flowchart

# Pseudo-code decision logic for choosing a scaler

def choose_scaler(data, algorithm, has_outliers):
    """
    Choose the appropriate scaler based on data and algorithm.
    
    Parameters:
    -----------
    data : array-like
        Your feature data
    algorithm : str
        The ML algorithm you plan to use
    has_outliers : bool
        Whether your data has significant outliers
    
    Returns:
    --------
    scaler : sklearn scaler object
    """
    # Tree-based algorithms don't need scaling
    tree_based = ['decision_tree', 'random_forest', 'xgboost', 
                  'lightgbm', 'catboost', 'gradient_boosting']
    if algorithm.lower() in tree_based:
        return None  # No scaling needed
    
    # Neural networks and image data: use MinMaxScaler
    if algorithm.lower() in ['neural_network', 'cnn', 'rnn', 'lstm']:
        return MinMaxScaler(feature_range=(0, 1))
    
    # Data with significant outliers: use RobustScaler
    if has_outliers:
        return RobustScaler()
    
    # Default: StandardScaler for most other cases
    return StandardScaler()

# Usage example
scaler = choose_scaler(X, 'logistic_regression', has_outliers=False)
if scaler:
    X_scaled = scaler.fit_transform(X_train)
    X_test_scaled = scaler.transform(X_test)

Detecting Outliers Before Choosing

import numpy as np
from scipy import stats

def check_for_outliers(data, threshold=3):
    """
    Check if data has significant outliers using z-score method.
    
    Parameters:
    -----------
    data : array-like
        Feature data (1D or 2D)
    threshold : float
        Z-score threshold for outlier detection (default: 3)
    
    Returns:
    --------
    dict : Information about outliers in each feature
    """
    data = np.array(data)
    if data.ndim == 1:
        data = data.reshape(-1, 1)
    
    results = {}
    for i in range(data.shape[1]):
        feature = data[:, i]
        z_scores = np.abs(stats.zscore(feature))
        outliers = np.sum(z_scores > threshold)
        outlier_pct = (outliers / len(feature)) * 100
        
        results[f'feature_{i}'] = {
            'outliers': outliers,
            'percentage': f'{outlier_pct:.2f}%',
            'recommend': 'RobustScaler' if outlier_pct > 1 else 'StandardScaler'
        }
    
    return results

# Example usage
np.random.seed(42)
data = np.column_stack([
    np.random.normal(50, 10, 100),  # Normal distribution
    np.concatenate([np.random.normal(50, 10, 95), np.array([200, 250, 300, 350, 400])])  # With outliers
])

outlier_info = check_for_outliers(data)
for feature, info in outlier_info.items():
    print(f"{feature}: {info['outliers']} outliers ({info['percentage']}) -> {info['recommend']}")

# Output:
# feature_0: 0 outliers (0.00%) -> StandardScaler
# feature_1: 5 outliers (5.00%) -> RobustScaler

Complete Pipeline Example

from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
import pandas as pd
import numpy as np

# Create sample dataset
np.random.seed(42)
n_samples = 1000

data = pd.DataFrame({
    'age': np.random.randint(18, 80, n_samples),
    'income': np.concatenate([
        np.random.normal(50000, 15000, n_samples-10),
        np.random.uniform(200000, 500000, 10)  # High earners (outliers)
    ]),
    'score': np.random.uniform(0, 100, n_samples),
    'target': np.random.randint(0, 2, n_samples)
})

X = data.drop('target', axis=1)
y = data['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create pipelines with different scalers
pipelines = {
    'Logistic (Standard)': Pipeline([
        ('scaler', StandardScaler()),
        ('classifier', LogisticRegression(max_iter=1000))
    ]),
    'Logistic (MinMax)': Pipeline([
        ('scaler', MinMaxScaler()),
        ('classifier', LogisticRegression(max_iter=1000))
    ]),
    'Logistic (Robust)': Pipeline([
        ('scaler', RobustScaler()),
        ('classifier', LogisticRegression(max_iter=1000))
    ]),
    'SVM (Standard)': Pipeline([
        ('scaler', StandardScaler()),
        ('classifier', SVC())
    ]),
    'Random Forest (No Scaling)': Pipeline([
        ('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
    ])
}

# Compare performance
print("Cross-Validation Accuracy (5-fold):")
print("-" * 50)
for name, pipeline in pipelines.items():
    scores = cross_val_score(pipeline, X_train, y_train, cv=5)
    print(f"{name:30} {scores.mean():.4f} (+/- {scores.std():.4f})")

# Output (example):
# Cross-Validation Accuracy (5-fold):
# --------------------------------------------------
# Logistic (Standard)            0.5150 (+/- 0.0180)
# Logistic (MinMax)              0.5112 (+/- 0.0155)
# Logistic (Robust)              0.5175 (+/- 0.0190)  <- Best for outlier data
# SVM (Standard)                 0.5087 (+/- 0.0200)
# Random Forest (No Scaling)     0.5025 (+/- 0.0165)

Common Mistakes to Avoid

Saving and Loading Scalers

import joblib
from sklearn.preprocessing import StandardScaler
import numpy as np

# Training phase
X_train = np.array([[10, 100], [20, 200], [30, 300], [40, 400], [50, 500]])

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)

# Save the fitted scaler
joblib.dump(scaler, 'scaler.joblib')
print("Scaler saved!")

# --- Later, in production ---

# Load the scaler
loaded_scaler = joblib.load('scaler.joblib')

# Apply to new data
new_data = np.array([[25, 250], [35, 350]])
new_data_scaled = loaded_scaler.transform(new_data)

print("\nNew data (original):")
print(new_data)
print("\nNew data (scaled with loaded scaler):")
print(new_data_scaled.round(3))

# Output:
# Scaler saved!
# 
# New data (original):
# [[ 25 250]
#  [ 35 350]]
# 
# New data (scaled with loaded scaler):
# [[-0.707 -0.707]
#  [ 0.     0.   ]]

Using ColumnTransformer for Mixed Data

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
import pandas as pd
import numpy as np

# Mixed dataset (numerical + categorical)
data = pd.DataFrame({
    'age': [25, 35, 45, 55, 65],
    'income': [30000, 50000, 75000, 90000, 120000],
    'city': ['NYC', 'LA', 'NYC', 'Chicago', 'LA'],
    'gender': ['M', 'F', 'M', 'F', 'M'],
    'target': [0, 1, 0, 1, 1]
})

X = data.drop('target', axis=1)
y = data['target']

# Define which columns get which transformation
numerical_features = ['age', 'income']
categorical_features = ['city', 'gender']

# Create preprocessor
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numerical_features),     # Scale numerical
        ('cat', OneHotEncoder(drop='first'), categorical_features)  # Encode categorical
    ]
)

# Create full pipeline
pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression(max_iter=1000))
])

# Fit the pipeline
pipeline.fit(X, y)

# The scaler is only applied to numerical columns!
print("Pipeline fitted successfully!")
print(f"Numerical features scaled: {numerical_features}")
print(f"Categorical features encoded: {categorical_features}")

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numerical_columns),
    ('cat', OneHotEncoder(drop='first'), categorical_columns)
])

pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression())
])

pipeline.fit(X_train, y_train)

import numpy as np
data = np.array([[10], [20], [30], [40], [50], [200]])

from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
import numpy as np

data = np.array([[10], [20], [30], [40], [50], [200]])

minmax = MinMaxScaler().fit_transform(data)
standard = StandardScaler().fit_transform(data)
robust = RobustScaler().fit_transform(data)

print("Original | MinMax | Standard | Robust")
print("-" * 45)
for i, orig in enumerate(data.flatten()):
    print(f"{orig:7} | {minmax[i,0]:6.3f} | {standard[i,0]:8.3f} | {robust[i,0]:6.3f}")

# RobustScaler handles the outlier best:
# - MinMax compresses 10-50 into 0.0-0.21 range
# - Standard is moderately affected
# - Robust keeps 10-50 well distributed, flags 200 as outlier

Explore how different scalers transform your data in real-time. See the effect of outliers and compare scaling methods side by side.