HGB benchmark sweep and artifact planning

2 min read

HGB benchmark sweep and artifact planning

  • Day: 2026-05-17
  • Time: 11:10 to 11:20
  • Project: Dev
  • Workspace: WP 2: Operational
  • Status: Completed
  • Priority: MEDIUM
  • Assignee: Matías Nehuen Iglesias
  • Tags: Histgradientboosting, Hyperparameter-Sweep, Cross-Validation, Overfitting, Artifact, Ml-Experiment

Description

Session Goal

Interpret recent HistGradientBoosting regression sweeps, identify the practical performance ceiling, and decide what to optimize next. A secondary thread defined a deterministic training-frame artifact to support leakage checks and reproducible inspection.

Key Activities

  • Reviewed learning-rate / iteration sweeps for HistGradientBoosting and concluded validation performance plateaus around CV R2 ≈ 0.545-0.547.
  • Compared the effect of L2 regularization and found it has negligible impact on CV R2, train R2, or the overfit gap in the tested region.
  • Consolidated the min_samples_leaf sweep into a defendable benchmark configuration, separating a fast default from a more expensive best-observed candidate.
  • Drafted a run-local training-frame sample artifact: deterministic CSV + metadata capture of the exact training frame used before fitting, with run-manifest registration and tests to prevent leakage or duplicated per-model artifacts.

Achievements

  • Established that further boosting mainly increases overfitting rather than validation gain, implying a predictive ceiling near the observed CV R2 plateau.
  • Narrowed future hyperparameter search priorities to learning_rate, max_iter, and max_leaf_nodes, while treating L2 regularization as secondary.
  • Clarified the next research direction as feature/target/error analysis instead of brute-force model scaling.
  • Defined artifact requirements for reproducible inspection, including forbidden-column checks and a clean separation from prediction/runtime outputs.

Pending Tasks

  • Run a smaller confirmatory experiment focused on the useful region of learning_rate, iterations, and leaf count.
  • Implement and test the training-frame sample artifact with CSV/metadata outputs and manifest registration.
  • Perform feature, target, and residual/error analysis to identify whether gains now depend more on data/feature engineering than model complexity.

Evidence

  • source_file=2026-05-17.sessions.jsonl, line_number=4, event_count=0, session_id=9a3e0f880c6602d2cc84d23d32936e1712d5282e97e22798bb19e1c45c11cc26
  • event_ids: []

Graph View

Backlinks