📅 2023-11-16 — Session: DataFrame Manipulation and Optimization with Python
🕒 20:30–22:15
🏷️ Labels: Python, Dataframe, Optimization, Pandas, Numpy
📂 Project: Dev
⭐ Priority: MEDIUM
Session Goal:
The session aimed to manipulate and optimize data structures using Python, focusing on DataFrame operations and quadratic optimization.
Key Activities:
- DataFrame Filtering: Implemented a Python script to filter unrelated course pairs in the lower triangle of a DataFrame using Pandas.
- Quadratic Optimization: Utilized cvxpy to solve a quadratic optimization problem for assigning levels to subjects, aligning with a correlation matrix.
- Data Replacement: Replaced values in a DataFrame greater than 100 in absolute value with NaN using Pandas’
applymap. - Term Differences Calculation: Created a square matrix to calculate differences in academic terms between subjects using Pandas and NumPy.
- Lower Triangle Calculation: Calculated the lower triangle of the difference between DataFrames and replaced zeros with NaN for better visualization.
Achievements:
- Successfully filtered and manipulated DataFrames for course data.
- Applied quadratic optimization to educational data, enhancing curriculum analysis.
- Improved data clarity by replacing out-of-range values with NaN.
Pending Tasks:
- Further exploration of optimization techniques for curriculum development.
- Implementation of more complex data filtering and manipulation strategies.