📅 2025-05-13 — Session: Refined LLM Evaluation and Tutoring Design
🕒 00:05–00:25
🏷️ Labels: LLM, Jinja2, Python, Tutoring, Education
📂 Project: Dev
⭐ Priority: MEDIUM
Session Goal
The goal of this session was to enhance the design and functionality of LLM evaluation prompts and to shift the focus from evaluation to tutoring in computer science education.
Key Activities
- Streamlined the design for evaluating student responses using a Jinja2 template and a Python evaluator class.
- Proposed modularization of the
evaluator.pyfile to separate system instantiation from student input, allowing for customizable responses and reusable instructions. - Refined the structure of the evaluation prompt to improve readability and alignment with current ChatCompletion formats.
- Adjusted the evaluation prompt to focus on tutoring, promoting active understanding and critical thinking in computer science students.
- Developed a new pedagogical approach to transform the evaluator’s role into a tutor, emphasizing guidance and student support.
Achievements
- Achieved a modular and maintainable design for LLM evaluation prompts.
- Successfully shifted the pedagogical approach from evaluation to tutoring, enhancing the educational experience.
Pending Tasks
- Further testing and integration of the new tutoring-focused prompts with existing educational tools.