py-evoFE: Automated Evolutionary Feature Engineering for Tabular ML in Python (Genetic Algorithms + Scikit-Learn + Polars) [P]

Hey everyone!

I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets.

- GitHub: https://github.com/tanopereira/py-evoFE

- PyPI: pip install py-evoFE

- License: MIT

The Problem It Solves

Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own.

Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage.

What py-evoFE Does

py-evoFE searches the space of possible feature recipes using genetic programming: 1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3))). 2. 40+ Built-in Transformers: - Non-linear arithmetic & log-ratios - Target encoding (multiclass, pooled, WoE, quantile target encodings) - String similarity (MinHash, Gap encodings) - Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA) - Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring) 3. Performance & Speed: - Vectorized computation powered by Polars and PyArrow. - Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds. - Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation. 4. Island Model & Caruana Ensembling: - Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration. - Post-search greedy Caruana ensembling over island winners' out-of-fold predictions. 5. Interactive Replay Viewer: - Run view(evo.get_recipe()) to generate a self-contained, zero-dependency HTML dashboard replaying the evolutionary search over time. 6. 100% Scikit-Learn Compatible: - Implements fit, transform, predict, and predict_proba. Plugs directly into standard sklearn.pipeline.Pipeline and GridSearchCV.

Quick Example

```python import polars as pl from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split from evofe import EvoFE

Load data

bc = load_breast_cancer(as_frame=True) df = pl.from_pandas(bc.frame) X, y = df.drop("target"), df["target"].to_numpy()

X_train, X_test, y_train, y_test = train_test_split( X.to_numpy(), y, test_size=0.2, random_state=42, stratify=y ) X_train_df = pl.DataFrame(X_train, schema=X.columns) X_test_df = pl.DataFrame(X_test, schema=X.columns)

1. Initialize EvoFE

evo = EvoFE( task="classification", evaluator="lightgbm", # "lightgbm" | "xgboost" pop_size=15, n_generations=10, cv_folds=3, verbose=True, random_state=42 )

2. Fit: Runs evolutionary search

evo.fit(X_train_df, y_train)

3. Inspect evolved recipe

recipe = evo.get_recipe() print(f"Discovered {len(recipe.genes)} high-impact features:") for gene in recipe.genes: print(f" • {gene.to_formula()} -> {gene.output_col}")

4. Transform & Predict

preds = evo.predict(X_test_df) proba = evo.predict_proba(X_test_df) ```

Why not just brute-force feature generation?