One-call EDA with profile()¶

pycatdap.profile(df, response=...) is the v0.5.0 flagship API. It bundles everything you have seen in notebooks 02 — 07 (overview, variable cards, ΔAIC association matrix, CATDAP-02 top subsets, quality warnings) into a single call, and renders the result as a self-contained HTML report you can email, embed in Confluence, or hand off to a non-Python collaborator.

This tutorial walks through profile() on the Titanic dataset and shows the four output methods (show, to_dict, to_plotly_json, to_html).

Note: this notebook requires pycatdap[plotly] (for the inline Plotly figures) and pycatdap[tutorial] (for the bundled Titanic seaborn variant used in section 2).

In [ ]:

import seaborn as sns

import pycatdap

df = sns.load_dataset("titanic")[
    ["survived", "pclass", "sex", "age", "fare", "embarked"]
].dropna()
df["survived"] = df["survived"].astype(bool)
df["pclass"] = df["pclass"].astype("category")
df.head()

import seaborn as sns import pycatdap df = sns.load_dataset("titanic")[ ["survived", "pclass", "sex", "age", "fare", "embarked"] ].dropna() df["survived"] = df["survived"].astype(bool) df["pclass"] = df["pclass"].astype("category") df.head()

1. Run profile() on a clean dataframe¶

The minimal call needs only a DataFrame. With no response, you still get the overview, variable cards, ΔAIC association matrix, and quality warnings — only the CATDAP-02 top-subsets section is skipped.

In [ ]:

report = pycatdap.profile(df)
report

report = pycatdap.profile(df) report

ProfileResult.show() is the Jupyter-friendly view. In a plain Python shell it falls back to printed tables.

In [ ]:

report.show()

report.show()

2. Add a response to surface ΔAIC and subset rankings¶

Pass response="survived" to:

• populate delta_aic_vs_response on every other VariableCard
• run CATDAP-02 internally and expose the top subsets via report.top_subsets

In [ ]:

report = pycatdap.profile(df, response="survived", top_k_subsets=3)
report

report = pycatdap.profile(df, response="survived", top_k_subsets=3) report

In [ ]:

# Variable cards now carry ΔAIC vs the response (response column itself stays None).
[(c.name, c.kind, c.delta_aic_vs_response) for c in report.variables]

# Variable cards now carry ΔAIC vs the response (response column itself stays None). [(c.name, c.kind, c.delta_aic_vs_response) for c in report.variables]

In [ ]:

report.top_subsets.aic.head()

report.top_subsets.aic.head()

3. Drive quality warnings with the threshold dict¶

The four built-in checks (high_cardinality / constant / id_candidate / high_missing) use sensible defaults. Override any of them via the quality_thresholds= mapping when your data has different shape.

In [ ]:

# Default thresholds: clean Titanic frame has no findings.
default = pycatdap.profile(df)
default.quality_warnings

# Default thresholds: clean Titanic frame has no findings. default = pycatdap.profile(df) default.quality_warnings

In [ ]:

# Tighter thresholds — flag any column missing > 1% of values.
tightened = pycatdap.profile(df, quality_thresholds={"high_missing": 0.01})
[(w.kind, w.column, round(w.metric, 3), w.message) for w in tightened.quality_warnings]

# Tighter thresholds — flag any column missing > 1% of values. tightened = pycatdap.profile(df, quality_thresholds={"high_missing": 0.01}) [(w.kind, w.column, round(w.metric, 3), w.message) for w in tightened.quality_warnings]

4. Serialize for downstream consumers¶

to_dict() produces a JSON-friendly nested dict (round-trips through json.dumps(..., default=str)). to_plotly_json() returns Plotly figure specs that react-plotly.js or LizyStudio can render directly (no plotly Python dep needed).

In [ ]:

import json

as_dict = report.to_dict()
print(json.dumps(as_dict["overview"], default=str, indent=2))

import json as_dict = report.to_dict() print(json.dumps(as_dict["overview"], default=str, indent=2))

In [ ]:

spec = report.to_plotly_json()
sorted(spec.keys())

spec = report.to_plotly_json() sorted(spec.keys())

5. Generate the HTML report¶

to_html(path) renders a single self-contained HTML file with the Plotly figures embedded inline (no external CDN — the file works offline). Use to_html() without path to get the HTML string for custom delivery (email body, Streamlit component, etc.).

In [ ]:

import pathlib
import tempfile

with tempfile.TemporaryDirectory() as tmp:
    out = pathlib.Path(tmp) / "titanic-profile.html"
    report.to_html(out)
    size_kib = out.stat().st_size / 1024
print(f"Wrote {out.name} ({size_kib:.0f} KiB)")

import pathlib import tempfile with tempfile.TemporaryDirectory() as tmp: out = pathlib.Path(tmp) / "titanic-profile.html" report.to_html(out) size_kib = out.stat().st_size / 1024 print(f"Wrote {out.name} ({size_kib:.0f} KiB)")

The HTML written above opens in any browser and shows:

Section	Content
Overview	rows / columns / missing rate / duplicates / memory
Quality warnings	each finding tagged by severity (info / warning) — hidden when empty
Variables	grid of per-column cards: kind / cardinality / top / continuous stats / ΔAIC vs response
Pairwise associations	interactive ΔAIC heatmap (the association_matrix output)
Top subsets	CATDAP-02 single-variable ΔAIC bar — only when response was provided

Cross-references¶

• Single variable summary (describe): Notebook 02
• Single (target, explanatory) pair (target_summary / plot_target): Notebook 06
• Symmetric / multi-variable views (plot_pair, aic_heatmap, association_matrix, association_plot): Notebook 07
• Multivariate subset search (catdap2): Notebook 04

Next steps¶

Continue with target_analysis() and the data-quality suite: Notebook 09