Data leakage is the #1 silent bug in beginner ML. It happens when your test data influences the training process - making accuracy look better than it really is.
What is data leakage?
When a preprocessing step like scaling is fitted on all data (including test rows) before the split, the model has indirectly seen the test set. Your accuracy score is optimistic and won't generalise.
The wrong way (what most tutorials show)
python
# WRONG: scaler sees all data before the split
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X) # computes mean/std on ALL data
X_train, X_test = train_test_split(X_scaled) # test is already contaminated
⚠The scaler's mean and std were computed using test rows. Your accuracy score is optimistic - it won't hold up on truly new data.
The right way (what fling always does)
python
# RIGHT: split first, fit preprocessing on train only
X_train, X_test, y_train, y_test = train_test_split(X, y)
pipeline = Pipeline([
("preprocessor", column_transformer),
("model", estimator),
])
pipeline.fit(X_train, y_train)
pipeline.predict(X_test)
Verify yourself
python
model.train()
X_train, X_test, y_train, y_test = model.get_splits()
print(X_train.shape[0]) # e.g. 712 - scaler computed on these rows only