Developed and Evaluated AI Model Rubrics

• Day: 2025-06-19
• Time: 00:00 to 23:55
• Project: Teaching
• Workspace: WP 2: Operational
• Status: Completed
• Priority: MEDIUM
• Assignee: Matías Nehuen Iglesias
• Tags: Ai Models, Rubric, Evaluation, Machine Learning, Education

Description

Session Goal

The session aimed to develop and evaluate rubric criteria for AI model assessments, focusing on physics-heavy AI challenges, scientific prompt citations, and grading AI model responses.

Key Activities

• Created MECE rubric criteria for evaluating AI challenges in physics, ensuring criteria are atomic and self-contained.
• Established guidelines for citing sources in scientific prompts, distinguishing between common knowledge and niche information.
• Developed a framework for grading AI models using rubrics, aligning quantitative scores with qualitative assessments.
• Proposed a correction scheme for evaluating prediction model responses, including evaluation criteria, common errors, and expected scores.
• Evaluated student performance on bias and variance concepts in machine learning models using a grading scale.

Achievements

• Successfully developed comprehensive rubric criteria and grading frameworks for AI model evaluation.
• Clarified the distinction between bias and overfitting in machine learning, enhancing understanding of model evaluation.

Pending Tasks

• Further refine the rubric criteria for specific AI challenges and model types.
• Implement the proposed correction scheme in actual model evaluations to validate its effectiveness.

Evidence

• source_file=2025-06-19.sessions.jsonl, line_number=0, event_count=0, session_id=df9b58261241853dc896c91c22dcc658274ef4478797a7c62c01ea5a47f628a5
• event_ids: []