Skip to content

Choosing the knots

The VAF on the fitting sample always grows with the number of knots: every extra knot enlarges the spline space of each variable, so the in-sample criterion cannot choose it. In 0.2.0 the number of knots is chosen by hand, guided by the transformation plots and the SmallSampleWarning (n < 5 (degree + knots)).

Design decision D12 (status: proposed for 0.3.0, see DESIGN.md) compares four options: manual choice (today); a score(X) method (VAF of the p components on new data, with new rows clipped to the training range) for use with GridSearchCV; a QLPCACV class (grid of 1–4 knots at the percentiles, 5-fold cross-validation, one-standard-error rule, grid limited by n ≥ 5 (degree + knots), results in cv_results_); and an automatic choice inside QLPCA. The recommendation is score() and QLPCACV in 0.3.0 — neither exists in 0.2.0 — and the automatic choice is rejected (cost, reproducibility, small samples, scikit-learn convention). Both depend on the option extrapolate="clip" deferred in D11.

Preliminary evidence

Out-of-sample VAF (%), p = 2, degree 1, default mode; 5-fold cross-validation repeated 4 times (RepeatedKFold(5, 4, random_state=0)); test rows clipped to the training range; VAF = share of the test rows' transformed variance captured by the training loading directions. Mean over the 20 splits, standard error in brackets. Computed by comparison/princals/knot_selection.py (file results/knot_selection.csv).

data 0 knots 1 knots 2 knots 3 knots 4 knots 6 knots
cylinder_0 (n=51) 83.4 (1.1) 94.5 (0.4) 96.5 (0.3) 97.3 (0.3) 97.7 (0.3) 98.3 (0.2)
cylinder_10 (n=51) 82.0 (0.9) 92.8 (0.4) 94.9 (0.3) 95.7 (0.2) 96.2 (0.2) 96.9 (0.2)
cylinder_25 (n=51) 72.8 (0.9) 87.6 (0.3) 89.5 (0.3) 90.6 (0.4) 91.1 (0.5) 91.4 (0.5)
cylinder_10 (n=500) 74.1 (2.4) 93.2 (0.1) 95.6 (0.1) 96.7 (0.1) 97.2 (0.1) 97.8 (0.0)
gas_turbine (n=400) 79.6 (0.5) 81.7 (0.4) 82.0 (0.5) 82.3 (0.4) 82.6 (0.4) 82.3 (0.4)

Open questions recorded in D12: a criterion on the original scale of the variables; knots per variable (greedy search); the location of the knots, not only their number. The Kelmarsh application compares knots at the percentiles with knots placed at observed control regions.