Implemented AI Workflow for Evaluation System

• Day: 2024-10-04
• Time: 22:50 to 23:55
• Project: Dev
• Workspace: WP 2: Operational
• Status: Completed
• Priority: MEDIUM
• Assignee: Matías Nehuen Iglesias
• Tags: Ai Evaluation, Openai Api, Json Schema, Error Handling

Description

Session Goal

The session aimed to design and implement an AI workflow for evaluating notebooks using predefined rubrics, integrating OpenAI API, and handling errors effectively.

Key Activities

• AI Workflow Design: Outlined the design and implementation plan for an AI workflow to evaluate notebooks with traffic light evaluations and structured data storage.
• AIEvaluator Class Update: Updated the AIEvaluator class to integrate with the latest OpenAI API, including improved error handling.
• JSON Schema Development: Created a JSON schema for rubric evaluations, categorizing them as ‘green’, ‘yellow’, or ‘red’.
• Schema Extraction Function: Developed a Python function to extract specific consigna schemas from a rubric evaluation schema.
• Error Resolution: Addressed JSON schema errors in OpenAI API calls, ensuring valid schema structuring and error handling.

Achievements

• Successfully designed an AI workflow for notebook evaluation.
• Updated and improved the AIEvaluator class for better API integration.
• Developed a robust JSON schema for rubric evaluations.
• Implemented error handling strategies for OpenAI API calls.

Pending Tasks

• Further testing of the AI evaluation system with real data to ensure robustness.
• Optimization of the error handling mechanisms for different edge cases.

Evidence

• source_file=2024-10-04.sessions.jsonl, line_number=1, event_count=0, session_id=341da457432d781a83bbf7c694ec5babcaf220aa655bb0485286faa05ff4ffa9
• event_ids: []