Automated Jupyter Notebook Execution and Error Handling

• Day: 2023-02-14
• Time: 11:55 to 13:24
• Project: Dev
• Workspace: WP 2: Operational
• Status: Completed
• Priority: MEDIUM
• Assignee: Matías Nehuen Iglesias
• Tags: Jupyter, Automation, Python, Error Handling, Scripting

Description

Session Goal

The session aimed to automate the execution of Jupyter notebooks and improve error handling during the process.

Key Activities

• Explored methods for programmatically running Jupyter notebooks using nbconvert and nbformat.
• Installed ipykernel in an Anaconda environment to support Jupyter functionality.
• Developed Python scripts to execute notebooks in a directory, incorporating error handling to stop execution on errors and continue with the next notebook.
• Enhanced the run_notebooks_in_directory function to support recursive execution and improved error handling using the traceback module.
• Updated code to correctly use the execution_count attribute and fixed string formatting errors in Markdown displays.
• Created scripts to automate the addition of import statements to the first cell of Jupyter notebooks.

Achievements

• Successfully automated the execution of Jupyter notebooks with robust error handling.
• Enabled recursive execution of notebooks across subdirectories.
• Improved error reporting by integrating traceback information.
• Automated import statement additions to ensure necessary modules are loaded in notebooks.

Pending Tasks

• Further testing of the automated scripts in diverse environments to ensure compatibility and robustness.

Evidence

• source_file=2023-02-14.sessions.jsonl, line_number=2, event_count=0, session_id=311956c525081314f43025bdcccc34642df20631b669558eae479003391b7ffb
• event_ids: []