Enhanced Python scripting techniques and file handling

📅 2023-01-20 — Session: Enhanced Python scripting techniques and file handling

🕒 16:50–18:10
🏷️ Labels: Python, File Handling, Ipython, Automation, Data Processing
📂 Project: Dev
⭐ Priority: MEDIUM

Session Goal

The session aimed to explore and enhance Python scripting techniques, focusing on file handling, loop optimization, and code organization.

Key Activities

• File Listing with glob.glob(): Demonstrated how to list files in Python using the glob.glob() function, applicable to both current and specified directories.
• Loop Automation: Showcased the use of for loops for executing yearly commands, employing both fixed lists and range-based approaches.
• Directory Management: Discussed best practices for managing working directories in loops, advocating for setting the directory once before the loop.
• Year Extraction: Provided methods for extracting years from filenames using regular expressions and string slicing.
• DataFrame Export: Illustrated saving pandas DataFrames to CSV with file path and size reporting, using the humanize library for size formatting.
• Variable Management in IPython: Explained defining and importing variables from external Python files into IPython notebooks to enhance code organization.

Achievements

• Clarified the use of glob.glob() for file listing and for loops for automation.
• Established best practices for directory management in scripts.
• Enhanced understanding of file size formatting with humanize.
• Improved code organization through external variable management in IPython.

Pending Tasks

• Further exploration of advanced file handling techniques and optimizations in Python scripting.
• Investigate additional libraries for enhancing file operations and data processing efficiency.