EDA on Titanic with pycatdap¶

This tutorial walks through the v0.3+ exploratory data analysis (EDA) API on the bundled Titanic dataset. By the end of this notebook you'll know how to:

1. Summarize a DataFrame with pycatdap.describe()
2. Visualize individual variables with pycatdap.plot_variable()
3. Inspect missing-value patterns with pycatdap.plot_missing()
4. Connect EDA output to the classic CATDAP-01 / CATDAP-02 pipeline

Background¶

The Titanic dataset (R datasets::Titanic in long form) records 2,201 passengers and crew by Class, Sex, Age (Adult/Child), and Survived (Yes/No). All four variables are categorical, fully observed, and tabulated — which makes it the canonical playground for CATDAP-01.

!!! note "Companion tutorial: real-world messy data" This notebook intentionally uses the clean contingency-table form of Titanic. For a workflow on the messier 891-passenger version with missing values, dtype heterogeneity, and continuous variables, see 05. Real-world EDA on seaborn Titanic.

In [ ]:

import pycatdap

print(f"pycatdap version: {pycatdap.__version__}")
df = pycatdap.datasets.load_titanic()
df.head()

import pycatdap print(f"pycatdap version: {pycatdap.__version__}") df = pycatdap.datasets.load_titanic() df.head()

In [ ]:

df.shape, list(df.columns)

df.shape, list(df.columns)

1. Per-variable summary with describe()¶

pycatdap.describe() returns a DescribeResult with a per-variable summary table emphasizing CATDAP-relevant signals: variable kind, missing counts, cardinality, the most-frequent value, and continuous statistics (if applicable).

The result is rich: it exposes .show(), .to_html(), .to_dict(), and .to_plotly_json() so the same object works in notebooks, HTML reports, JSON APIs, and web frontends like LizyStudio.

In [ ]:

summary = pycatdap.describe(df)
summary

summary = pycatdap.describe(df) summary

In [ ]:

# The full summary table
summary.summary

# The full summary table summary.summary

Reading the table:

• All four columns are detected as categorical (pandas StringDtype after CSV read).
• n_unique shows the cardinality: Sex/Age/Survived are binary; Class has 4 levels.
• top and top_freq reveal the dominant category — e.g., most passengers were Crew Adults.
• missing_rate is 0 — Titanic is fully observed in this canonical form.

2. Single-variable visualization with plot_variable()¶

For categorical variables, plot_variable() produces a frequency bar chart. For continuous variables it produces a histogram. The default kind="auto" infers from dtype; you can also force kind="hist" or kind="bar" explicitly.

In [ ]:

import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 4, figsize=(16, 4))
for ax, col in zip(axes, df.columns, strict=True):
    pycatdap.plot_variable(df, col, ax=ax)
fig.tight_layout()
plt.show()

import matplotlib.pyplot as plt fig, axes = plt.subplots(1, 4, figsize=(16, 4)) for ax, col in zip(axes, df.columns, strict=True): pycatdap.plot_variable(df, col, ax=ax) fig.tight_layout() plt.show()

The Crew column dominates Class, and the Adult/Male combination dominates passenger counts. About one third survived (Survived = Yes).

Plotly backend¶

The same call with backend="plotly" produces an interactive Plotly Figure — ideal for embedding in web dashboards or LizyStudio.

In [ ]:

# Plotly figure (requires pycatdap[plotly])
fig = pycatdap.plot_variable(df, "Class", backend="plotly")
fig.show()

# Plotly figure (requires pycatdap[plotly]) fig = pycatdap.plot_variable(df, "Class", backend="plotly") fig.show()

3. Missing-value pattern with plot_missing()¶

plot_missing() shows the count of missing values per column. The default layout is a vertical bar chart.

In [ ]:

ax = pycatdap.plot_missing(df)
plt.show()

ax = pycatdap.plot_missing(df) plt.show()

Titanic in this bundled form has no missing values, so all bars are at zero. When working with real data, putting plot_missing first lets you decide what to drop or impute before deeper analysis — see 05. Real-world EDA on seaborn Titanic for that workflow end-to-end.

4. Connecting EDA to CATDAP-01¶

The classic CATDAP-01 workflow ranks pairwise variable associations by ΔAIC. With the EDA output as context, we can now interpret CATDAP-01 results more confidently.

In [ ]:

result1 = pycatdap.catdap1(df, response_names=["Survived"])
result1.aic_order["Survived"]

result1 = pycatdap.catdap1(df, response_names=["Survived"]) result1.aic_order["Survived"]

Interpretation: Sex and Class are the most informative explanatory variables for survival (lowest ΔAIC = most informative). Age comes third. This matches historical accounts of "women and children first" — though the EDA reveals that the Crew category (predominantly Male Adults) heavily biases the marginal counts.

In [ ]:

result1.aic

result1.aic

5. Drill into a single pair with target_summary¶

CATDAP-01 ranks all explanatory variables at once. To inspect the most informative pair (Survived × Sex) in detail — counts, proportions, Pearson residuals, and ΔAIC in one object — use target_summary.

In [ ]:

pair = pycatdap.target_summary(df, target="Survived", explanatory="Sex")
print(f"delta_aic = {pair.delta_aic:.3f}")
pair.col_prop.round(3)

pair = pycatdap.target_summary(df, target="Survived", explanatory="Sex") print(f"delta_aic = {pair.delta_aic:.3f}") pair.col_prop.round(3)

The column-proportion table reads "within each Sex category, what fraction died vs survived". Female survival rate is dramatically higher — consistent with the "women first" policy.

For the full target-pair tour (visualizations, continuous explanatories with AIC-optimal binning, ΔAIC ranking across all features, and HTML/Plotly export), see notebook 06.

6. Connecting to CATDAP-02¶

CATDAP-02 finds the best subset of explanatory variables. For Titanic, all four columns are categorical so no continuous pooling is needed.

In [ ]:

result2 = pycatdap.catdap2(
    df,
    pool=[2, 2, 2, 2],
    response_name="Survived",
    accuracy=[0.0, 0.0, 0.0, 0.0],
)

print(f"Base AIC: {result2.base_aic:.2f}")
print("\nTop subsets:")
for s in result2.subsets[:5]:
    print(f"  AIC={s.aic:>8.2f}  vars={s.variables}")

result2 = pycatdap.catdap2( df, pool=[2, 2, 2, 2], response_name="Survived", accuracy=[0.0, 0.0, 0.0, 0.0], ) print(f"Base AIC: {result2.base_aic:.2f}") print("\nTop subsets:") for s in result2.subsets[:5]: print(f" AIC={s.aic:>8.2f} vars={s.variables}")

7. Plotly result spec for web integration¶

Catdap1Result.to_plotly_json() returns a complete Plotly figure spec ready for direct rendering by react-plotly.js or any Plotly renderer. This is the data contract for LizyStudio integration (Issue #21).

In [ ]:

spec = result1.to_plotly_json()
spec.keys(), len(spec["data"]), spec["data"][0]["type"]

spec = result1.to_plotly_json() spec.keys(), len(spec["data"]), spec["data"][0]["type"]

Summary¶

You have now seen the v0.3 univariate EDA API in action:

Function	Purpose
pycatdap.describe(df)	Per-variable summary table
pycatdap.plot_variable(df, col)	Univariate distribution plot
pycatdap.plot_missing(df)	Missing-value pattern
pycatdap.catdap1(df)	Pairwise ΔAIC analysis
pycatdap.catdap2(df, ...)	Best-subset search

Each visualization function supports both backend="matplotlib" (default) and backend="plotly" (with pycatdap[plotly] installed).

Next steps¶

• 03. AIC-optimal binning on iris — try the CATDAP-02 pooling search on continuous variables and see how it picks bin boundaries that beat equal-width cuts.
• 04. Multivariate HelloGoodbye — scale to 56 binary variables and use catdap2(nvar=k) to keep the subset search tractable.
• 05. Real-world EDA on seaborn Titanic — the full messy-data workflow on the 891-passenger version with real missing values.
• HTML reports: try summary.to_html("titanic_summary.html") to save a standalone report.
• Web integration: the .to_plotly_json() methods are designed to feed LizyStudio's react-plotly.js frontend directly.