Skip to content

pycatdap

AIC-based EDA and ML error analysis library for categorical data.

pycatdap is a Python implementation of CATDAP (CATegorical Data Analysis Program), originally developed by Sakamoto & Katsura (1980) at the Institute of Statistical Mathematics. It extends the classic CATDAP toolkit with modern EDA and ML error analysis workflows.

What makes pycatdap unique

Unlike ydata-profiling, Skrub, DivExplorer, or pysubgroup, pycatdap provides:

  • AIC-based variable relevance ranking — captures the trade-off between information and complexity (unlike Cramér's V or mutual information)
  • AIC-optimal continuous binning — principled binning of continuous variables (vs. arbitrary equal-width or quantile binning)
  • Subset optimization (CATDAP-02) — discovers the best combination of explanatory variables (unlike feature importance ranking)
  • Model-agnostic ML error analysis — works with predictions from any framework (sklearn, LightGBM, XGBoost, PyTorch)

At a glance

import pycatdap

df = pycatdap.datasets.load_titanic()
profile = pycatdap.profile(df, response="Survived")
profile.to_html("titanic_eda.html")

import pycatdap

result = pycatdap.error_analysis(
    df=test_df,
    y_true=y_test,
    y_pred=model.predict(X_test),
)
result.show()  # in Jupyter

import pycatdap

df = pycatdap.datasets.load_heart_disease()
cat = df[["target", "sex", "cp", "exang", "thal"]]
r = pycatdap.catdap1(cat, response_names=["target"])
print(r.aic_order)

This documentation follows the Diátaxis framework:

  • Getting Started — install and run your first analysis in 5 minutes
  • Tutorials — step-by-step learning paths for each feature area
  • How-to Guides — solutions to specific tasks
  • API Reference — complete function and class documentation (auto-generated)
  • Explanation — concepts, mathematical foundations, design rationale

Project status

Citation

If you use pycatdap in research, please cite:

@article{sakamoto1980categorical,
  title={Categorical Data Analysis by AIC},
  author={Sakamoto, Yosiyuki and Katsura, Koichi},
  journal={Mathematical Sciences},
  year={1980}
}