Implemented data operations and memory monitoring in Python

2 min read

📅 2023-04-25 — Session: Implemented data operations and memory monitoring in Python

🕒 18:50–19:10
🏷️ Labels: Python, Data Operations, Memory Management, Vs Code, Pandas
📂 Project: Dev
⭐ Priority: MEDIUM

Session Goal:

The session aimed to implement various data operations in Python and explore methods for monitoring memory usage.

Key Activities:

  • Code Evaluation Request: Initiated a request for code submission for evaluation purposes.
  • Union Operation for Relations: Implemented the union operation on two relations, R and S, with functions for initialization, tuple retrieval, and closure of relations.
  • Data Operations in Python: Developed Python classes for data operations such as Projection, Distinct, Grouping, Set Union, Set Intersection, and Set Difference using custom iterators.
  • Memory Usage in Pandas: Explained how to monitor memory usage in Pandas DataFrames using the memory_usage() method and pandas.api.types module.
  • Real-Time Memory Monitoring: Outlined methods for real-time memory monitoring in Python using tools like memory_profiler, objgraph, and pympler, and discussed OS tools like top and htop.
  • VS Code Extensions: Listed extensions for Visual Studio Code to monitor and optimize memory usage in Python applications.

Achievements:

  • Successfully implemented several data operations with custom iterators in Python.
  • Clarified methods for monitoring memory usage in Python applications and Pandas DataFrames.
  • Identified useful VS Code extensions for memory management.

Pending Tasks:

  • Submit code for evaluation as initially requested.

Graph View

Backlinks