Feature Engineering in 2026: The Complete Guide to Building Better Machine Learning Features

Feature engineering — the process of transforming raw data into meaningful inputs for machine learning models — is often the difference between a mediocre model and a state-of-the-art one. While deep learning has automated some feature extraction, feature engineering remains critical for tabular data, time series, and domain-specific applications where hand-crafted features encode business knowledge that neural networks cannot easily learn from raw data alone.

What is Feature Engineering?

Feature engineering encompasses: feature creation (constructing new variables from existing ones), feature transformation (scaling, encoding, binning), feature selection (choosing which features to include), and feature extraction (reducing high-dimensional data to informative representations). The goal is to represent the underlying problem structure in a way that makes patterns learnable by the chosen algorithm.

Numerical Feature Transformations

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler, MinMaxScaler, PowerTransformer

df = pd.DataFrame({"amount": [100, 500, 10, 10000, 250, 75]})

# Log transform: handles right-skewed distributions (income, spend, counts)
df["log_amount"] = np.log1p(df["amount"])  # log1p handles zeros

# Box-Cox / Yeo-Johnson: makes distribution more Gaussian
pt = PowerTransformer(method="yeo-johnson")
df["amount_normal"] = pt.fit_transform(df[["amount"]])

# Quantile binning: convert continuous to ordinal categories
df["amount_bucket"] = pd.qcut(df["amount"], q=5,
    labels=["very_low","low","medium","high","very_high"])

# Standard scaling: zero mean, unit variance (required by linear models, SVMs, NNs)
scaler = StandardScaler()
df["amount_scaled"] = scaler.fit_transform(df[["amount"]])

# Clipping outliers before scaling
p1, p99 = df["amount"].quantile([0.01, 0.99])
df["amount_clipped"] = df["amount"].clip(p1, p99)

Categorical Feature Encoding

import pandas as pd
from sklearn.preprocessing import OrdinalEncoder
from category_encoders import TargetEncoder, BinaryEncoder

df = pd.DataFrame({
    "city": ["NYC", "LA", "NYC", "Chicago", "LA", "NYC"],
    "plan": ["basic", "premium", "basic", "enterprise", "premium", "basic"],
    "churn": [0, 1, 0, 0, 1, 0]
})

# One-hot encoding: for low-cardinality nominals (<20 categories)
df_ohe = pd.get_dummies(df[["city"]], prefix="city", drop_first=True)

# Ordinal encoding: for ordered categories
oe = OrdinalEncoder(categories=[["basic", "premium", "enterprise"]])
df["plan_ordinal"] = oe.fit_transform(df[["plan"]])

# Target encoding: replace category with mean target value (handles high cardinality)
# IMPORTANT: must use cross-validation to avoid target leakage
te = TargetEncoder(cols=["city"], smoothing=10)
df["city_target"] = te.fit_transform(df["city"], df["churn"])

# Binary encoding: intermediate between OHE and ordinal for high-cardinality
be = BinaryEncoder(cols=["city"])
df_binary = be.fit_transform(df[["city"]])

DateTime Feature Engineering

import pandas as pd

df = pd.DataFrame({"timestamp": pd.date_range("2026-01-01", periods=1000, freq="H")})

ts = df["timestamp"]

# Calendar features
df["hour"] = ts.dt.hour
df["day_of_week"] = ts.dt.dayofweek  # 0=Mon, 6=Sun
df["day_of_month"] = ts.dt.day
df["week_of_year"] = ts.dt.isocalendar().week.astype(int)
df["month"] = ts.dt.month
df["quarter"] = ts.dt.quarter
df["is_weekend"] = df["day_of_week"].isin([5, 6]).astype(int)
df["is_month_start"] = ts.dt.is_month_start.astype(int)
df["is_month_end"] = ts.dt.is_month_end.astype(int)

# Cyclical encoding: hour 23 should be close to hour 0
df["hour_sin"] = np.sin(2 * np.pi * df["hour"] / 24)
df["hour_cos"] = np.cos(2 * np.pi * df["hour"] / 24)
df["dow_sin"] = np.sin(2 * np.pi * df["day_of_week"] / 7)
df["dow_cos"] = np.cos(2 * np.pi * df["day_of_week"] / 7)

# Time since event
df["days_since_epoch"] = (ts - pd.Timestamp("2020-01-01")).dt.days

Lag and Window Features for Time Series

import pandas as pd

# Assume df has columns: date, customer_id, amount
df = df.sort_values(["customer_id", "date"])

# Lag features: previous values
for lag in [1, 7, 14, 30]:
    df[f"amount_lag_{lag}"] = df.groupby("customer_id")["amount"].shift(lag)

# Rolling window features
for window in [7, 14, 30]:
    rolling = df.groupby("customer_id")["amount"].transform(
        lambda x: x.shift(1).rolling(window, min_periods=1))
    df[f"amount_roll_mean_{window}d"] = rolling.mean()
    df[f"amount_roll_std_{window}d"]  = rolling.std()
    df[f"amount_roll_max_{window}d"]  = rolling.max()

# Expanding window (cumulative stats)
df["amount_cumsum"] = df.groupby("customer_id")["amount"].cumsum()
df["amount_txn_count"] = df.groupby("customer_id").cumcount() + 1
df["amount_cum_mean"] = df["amount_cumsum"] / df["amount_txn_count"]

# Trend: is the customer spending more or less recently?
df["spend_trend_7d_vs_30d"] = (
    df["amount_roll_mean_7d"] / df["amount_roll_mean_30d"].replace(0, np.nan)
) - 1

Text Feature Engineering

from sklearn.feature_extraction.text import TfidfVectorizer
from sentence_transformers import SentenceTransformer

texts = ["Machine learning is transforming industry", "Deep learning with PyTorch"]

# TF-IDF: sparse bag-of-words with term frequency weighting
tfidf = TfidfVectorizer(max_features=10000, ngram_range=(1, 2),
                        min_df=5, max_df=0.95, sublinear_tf=True)
tfidf_matrix = tfidf.fit_transform(texts)  # sparse matrix

# Sentence embeddings: dense semantic representations (better for most tasks)
model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = model.encode(texts, batch_size=64, show_progress_bar=True)
# embeddings shape: (n_samples, 384) — dense semantic vectors

# Text statistics as features
import re
df["text"] = texts
df["text_length"] = df["text"].str.len()
df["word_count"] = df["text"].str.split().str.len()
df["unique_word_ratio"] = df["text"].apply(
    lambda t: len(set(t.lower().split())) / max(len(t.split()), 1))
df["has_question"] = df["text"].str.contains(r"?", regex=True).astype(int)

Interaction Features and Polynomial Features

from sklearn.preprocessing import PolynomialFeatures
import pandas as pd

# Manual interaction features (domain-driven)
df["revenue_per_user"] = df["total_revenue"] / df["user_count"].replace(0, np.nan)
df["sessions_per_day"] = df["total_sessions"] / df["days_active"].replace(0, np.nan)
df["conversion_rate"] = df["conversions"] / df["impressions"].replace(0, np.nan)
df["avg_order_value"] = df["revenue"] / df["order_count"].replace(0, np.nan)

# Ratio features: normalize by a denominator to remove scale effects
df["mobile_session_ratio"] = df["mobile_sessions"] / df["total_sessions"].replace(0, np.nan)

# Polynomial features (for linear models to capture non-linear relationships)
poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
X_poly = poly.fit_transform(X_numeric)
feature_names = poly.get_feature_names_out(numeric_cols)

Feature Selection: Removing Noise

from sklearn.feature_selection import SelectFromModel, mutual_info_classif, RFE
from sklearn.ensemble import GradientBoostingClassifier
import pandas as pd

# 1. Filter methods: fast, model-agnostic
mi_scores = mutual_info_classif(X, y, random_state=42)
mi_df = pd.Series(mi_scores, index=feature_names).sort_values(ascending=False)
top_features = mi_df[mi_df > 0.01].index.tolist()

# 2. Wrapper method: Recursive Feature Elimination
rfe = RFE(estimator=GradientBoostingClassifier(), n_features_to_select=20)
rfe.fit(X, y)
selected = X.columns[rfe.support_].tolist()

# 3. Embedded method: use model feature importances
model = GradientBoostingClassifier(n_estimators=200, max_depth=4)
model.fit(X_train, y_train)
selector = SelectFromModel(model, threshold="median", prefit=True)
X_selected = selector.transform(X)
important_features = X_train.columns[selector.get_support()].tolist()

# 4. Variance threshold: remove near-constant features
from sklearn.feature_selection import VarianceThreshold
vt = VarianceThreshold(threshold=0.01)
X_filtered = vt.fit_transform(X)

Feature Stores: Managing Features at Scale

As feature engineering matures in an organization, feature stores become critical infrastructure. A feature store centralizes feature computation, storage, and serving — ensuring that the same features used in training are identical to those served in production (solving the training-serving skew problem):

Feast (open source): Define features in Python, compute with Spark or Pandas, serve from Redis (online) or Parquet (offline). Tecton (managed): Declarative feature pipelines with streaming and batch support, point-in-time correct joins. Hopsworks: Unified feature store with built-in data validation and lineage tracking.

from feast import FeatureStore, Entity, Feature, FeatureView, FileSource, ValueType
from datetime import timedelta

store = FeatureStore(repo_path="feature_repo/")

# Define feature view
customer_features = FeatureView(
    name="customer_stats",
    entities=["customer_id"],
    ttl=timedelta(days=30),
    features=[
        Feature(name="total_spend_30d", dtype=ValueType.FLOAT),
        Feature(name="transaction_count_30d", dtype=ValueType.INT64),
        Feature(name="avg_order_value", dtype=ValueType.FLOAT),
    ],
    online=True,  # Serve from Redis for low-latency inference
    source=FileSource(path="data/customer_stats.parquet",
                      timestamp_field="event_timestamp")
)

# Training: historical features with point-in-time joins
training_df = store.get_historical_features(
    entity_df=training_entity_df,  # customer_id + event_timestamp
    features=["customer_stats:total_spend_30d", "customer_stats:avg_order_value"]
).to_df()

# Serving: real-time features from online store
online_features = store.get_online_features(
    features=["customer_stats:total_spend_30d"],
    entity_rows=[{"customer_id": "cust_12345"}]
).to_dict()

Best Practices for Feature Engineering

Prevent target leakage: The most common mistake in feature engineering. Never include any information that would not be available at prediction time. If predicting churn, features computed after the churn event are leakage. Always respect temporal boundaries.

Document every feature: Every feature should have a clear definition, the business logic it encodes, and the expected relationship to the target. Features that cannot be explained are features that cannot be trusted.

Monitor feature distributions: Production ML failures are often caused by feature drift — the distribution of a feature in production diverging from training. Monitor key feature statistics (mean, std, % null, % out-of-range) in production and alert on significant changes.

Use cross-validation for everything: When using target encoding, feature selection based on importance scores, or any technique that uses the target variable, always apply within cross-validation folds to prevent information leakage from the test set.

Conclusion

Feature engineering remains one of the highest-leverage activities in the machine learning workflow. A well-engineered feature set can turn a simple logistic regression into a production model that outperforms a complex neural network trained on raw data. The best feature engineers combine domain expertise — knowing which transformations capture meaningful patterns — with engineering rigor: reproducible pipelines, leakage prevention, comprehensive testing, and feature store integration for consistent training-serving parity. In 2026, automated feature engineering tools assist but do not replace the judgment and creativity of skilled data scientists who understand both the domain and the data.

Search This Blog

Globe