📅 2023-08-05 — Session: Developed DBML schemas and data manipulation scripts
🕒 21:00–22:15
🏷️ Labels: DBML, Python, Data Transformation, CSV, Data Modeling
📂 Project: Dev
⭐ Priority: MEDIUM
Session Goal: The session aimed to develop and refine DBML schemas for data transformation processes and establish data manipulation scripts using Python.
Key Activities:
- Created a DBML schema outlining the structure for input and transformed data tables, focusing on 1:1 relationships between columns.
- Expanded the DBML representation to include new tables (RFC1, RFC2, RFC3) and their relationships, enhancing the data model.
- Implemented Python scripts using pandas to load CSV files, update and standardize region names, and concatenate multiple CSV files.
Achievements:
- Successfully defined and visualized 1:1 relationships in DBML for various tables, providing a clear data model for transformation processes.
- Developed Python code snippets to efficiently manipulate and update DataFrames, ensuring data consistency and readiness for further analysis.
Pending Tasks:
- Further testing and validation of DBML schemas and Python scripts to ensure robustness in different data scenarios.