📅 2025-02-27 — Session: Backup and Data Cleanup

🕒 16:40–16:55
🏷️ Labels: Backup, Data Management, File Organization, Rsync, Optimization
📂 Project: Dev
⭐ Priority: MEDIUM

Session Goal

The goal of this session was to manage and optimize backup data by merging old backups, cleaning up redundant files, and revamping outdated projects.

Key Activities

  • Merged and cleaned up old backups using structured workflows to ensure no data loss while removing duplicates.
  • Explored faster methods to analyze large directories without using the du command, focusing on tools like find and rsync for efficiency.
  • Developed a detailed plan for cleaning up the Matias_Iglesias_Backup/ directory, including identifying redundant files and archiving large datasets to Google Cloud Storage (GCS).
  • Diagnosed and revamped the old project compute-corr-structures/ by identifying storage issues and reorganizing it into a lean educational repository.
  • Saved processed file names from the /data/processed/ directory into a README.txt file before deletion, ensuring future reference with optional timestamps.

Achievements

  • Successfully merged and optimized backup data, ensuring efficient storage management.
  • Enhanced project organization by revamping outdated repositories and documenting processed files.

Pending Tasks

  • Further optimization of backup directory analysis and cleanup.
  • Continuous monitoring and updating of backup strategies to prevent data redundancy.