2nd/10_Wiki/Topics/AI_and_ML/Feature Engineering.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-feature-engineering	Feature Engineering	10_Wiki/Topics	verified	self	FE feature engineering target encoding feature crossing feature store	none	A	0.98	applied	machine-learning feature-engineering preprocessing target-encoding feature-store		2026-05-10	pending	language framework Python pandas / scikit-learn / Featuretools / Feast

Feature Engineering

매 한 줄

"매 raw data 의 model-ready feature 의 transform". 매 numerical scaling, 매 categorical encoding, 매 datetime, 매 text, 매 interaction. 매 modern: 매 deep learning 의 자동 학습 BUT 매 tabular 의 still 매 critical. 매 feature store (Feast) for production.

매 핵심

매 numerical

• Scaling: standard, minmax, robust.
• Power: log, Box-Cox, Yeo-Johnson.
• Bin / discretize: equal-width, quantile.
• Polynomial / interaction.

매 categorical

• One-hot: 매 low cardinality.
• Label / ordinal: 매 ordered.
• Target encoding (mean): 매 high cardinality + leakage care.
• Hashing trick: 매 fixed dim.
• Embedding: 매 NN.

매 datetime

• Cyclic (sin/cos for hour/day).
• Lag features (time series).
• Rolling stats.
• Holiday / weekend.

매 text

• Bag of words / TF-IDF.
• N-grams.
• Embeddings (BERT, sentence-transformers).
• LLM features.

매 응용

1. Tabular ML: 매 critical.
2. Time series: 매 lag / rolling.
3. NLP: 매 embed.
4. Graph: 매 graph features (node degree, ...).

💻 패턴

Standard scale

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().fit(X_train)
X_scaled = scaler.transform(X_test)

Cyclic datetime

import numpy as np
def encode_cyclic(value, max_value):
    return np.sin(2 * np.pi * value / max_value), np.cos(2 * np.pi * value / max_value)

df['hour_sin'], df['hour_cos'] = encode_cyclic(df.hour, 24)
df['dow_sin'], df['dow_cos'] = encode_cyclic(df.day_of_week, 7)

Target encoding (with smoothing)

def target_encode(train, test, col, target, smoothing=10):
    global_mean = train[target].mean()
    agg = train.groupby(col)[target].agg(['mean', 'count'])
    smoothed = (agg['count'] * agg['mean'] + smoothing * global_mean) / (agg['count'] + smoothing)
    return test[col].map(smoothed).fillna(global_mean)

Out-of-fold target encoding (no leakage)

from sklearn.model_selection import KFold
def oof_target_encode(X, y, col, n_folds=5):
    enc = np.zeros(len(X))
    for tr_idx, val_idx in KFold(n_folds, shuffle=True).split(X):
        means = X.iloc[tr_idx].groupby(col).apply(lambda g: y.iloc[g.index].mean())
        enc[val_idx] = X.iloc[val_idx][col].map(means).fillna(y.iloc[tr_idx].mean())
    return enc

Lag features (time series)

def lag_features(df, target_col, lags=[1, 7, 30]):
    for lag in lags:
        df[f'{target_col}_lag{lag}'] = df[target_col].shift(lag)
    return df

Rolling stats

df['amt_roll_mean_7'] = df.groupby('user_id')['amount'].transform(
    lambda s: s.rolling(7, min_periods=1).mean()
)
df['amt_roll_std_7'] = df.groupby('user_id')['amount'].transform(
    lambda s: s.rolling(7, min_periods=1).std()
)

Aggregation per group

agg = df.groupby('user_id')['amount'].agg(['mean', 'std', 'max', 'count']).reset_index()
df = df.merge(agg, on='user_id', suffixes=('', '_agg'))

Interaction (cross)

from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
X_inter = poly.fit_transform(X[['age', 'income']])

Hashing trick (high-card)

from sklearn.feature_extraction import FeatureHasher
h = FeatureHasher(n_features=2**18, input_type='string')
X_hashed = h.transform([str(row.user_id) for _, row in df.iterrows()])

Featuretools (automated)

import featuretools as ft
es = ft.EntitySet('shop')
es.add_dataframe('orders', df=orders_df, index='order_id', time_index='date')
es.add_dataframe('users', df=users_df, index='user_id')
es.add_relationship('users', 'user_id', 'orders', 'user_id')

features, defs = ft.dfs(entityset=es, target_dataframe_name='users',
                         agg_primitives=['mean', 'sum', 'count'])

Feast feature store (production)

from feast import FeatureStore, Entity, FeatureView, Field
from feast.types import Float32, Int64

user = Entity(name='user', value_type=Int64)
user_features = FeatureView(
    name='user_features',
    entities=[user],
    ttl=timedelta(days=1),
    schema=[Field(name='ltv', dtype=Float32), Field(name='tenure', dtype=Int64)],
    source=BigQuerySource(table='proj.user_features'),
)

store = FeatureStore(repo_path='.')
features = store.get_online_features(features=['user_features:ltv'], entity_rows=[{'user': 1}])

LLM-as-feature

def llm_sentiment(text, llm):
    return llm.classify(text, ['positive', 'neutral', 'negative'])

df['llm_sentiment'] = df['review'].apply(lambda t: llm_sentiment(t, llm))

Embedding feature

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
df['title_emb'] = list(model.encode(df['title'].tolist()))

Pipeline (sklearn)

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
preprocessor = ColumnTransformer([
    ('num', StandardScaler(), num_cols),
    ('cat', OneHotEncoder(handle_unknown='ignore'), cat_cols),
])
pipe = Pipeline([('prep', preprocessor), ('model', xgb.XGBClassifier())])
pipe.fit(X_train, y_train)

Anti-leakage

def split_then_fit(X, y):
    """매 ALWAYS split first 의 fit transformer."""
    X_tr, X_val, y_tr, y_val = train_test_split(X, y)
    scaler = StandardScaler().fit(X_tr)  # 매 train only
    X_tr_s = scaler.transform(X_tr)
    X_val_s = scaler.transform(X_val)  # 매 val 의 transform only
    return X_tr_s, X_val_s, y_tr, y_val

매 결정 기준

상황	Approach
Numerical + tree	Often raw OK
Numerical + linear	Scale (Standard)
Cardinality < 50	One-hot
Cardinality > 50	Target encode (OOF) or hash
Time series	Lag + rolling
Production	Feature store (Feast)
Auto	Featuretools / Tsfresh

기본값: 매 manual + 매 OOF target encode + 매 cyclic datetime + 매 leakage prevent + 매 production = feature store.

🔗 Graph

• 부모: Machine-Learning · Data-Preprocessing
• 변형: Target-Encoding · Embeddings
• 응용: Feature-Store · Feast
• Adjacent: Exploratory-Data-Analysis

🤖 LLM 활용

언제: 매 tabular ML. 매 time series. 매 production system. 언제 X: 매 deep learning end-to-end (image, text).

❌ 안티패턴

• Fit on full data: 매 leakage.
• Naive target encode: 매 leakage.
• No cyclic datetime: 매 RNN-only.
• Skip feature store: 매 prod / train skew.
• Over-engineer for tree: 매 little gain.

🧪 검증 / 중복

• Verified (Kuhn Feature Engineering, Kaggle competitions, Feast docs).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-04-20	Auto-reinforced
2026-05-08	Phase 1
2026-05-10	Manual cleanup — categorical / time / interaction + 매 OOF / Featuretools / Feast code