Implemented BigQuery Table and Troubleshooting Workflow

2 min read

📅 2025-02-27 — Session: Implemented BigQuery Table and Troubleshooting Workflow

🕒 21:30–21:55
🏷️ Labels: Bigquery, Google Cloud, Data Loading, Troubleshooting
📂 Project: Dev
⭐ Priority: MEDIUM

Session Goal

The primary objective of this session was to create a BigQuery table for census data stored in Google Cloud Storage, troubleshoot command line issues, and manage data access permissions.

Key Activities

  • BigQuery Table Creation: Followed a structured guide to enable the BigQuery API, create a dataset, load data from CSV files, and verify the table’s creation.
  • Troubleshooting bq Command: Addressed issues with the bq command not being recognized, including installation and authentication steps.
  • Data Upload Verification: Checked the status of a BigQuery table after an interrupted CSV upload and reattempted the upload if necessary.
  • Data Access Management: Detailed steps were taken to share and query data in BigQuery, including setting permissions and using SQL for data access.

Achievements

  • Successfully created and verified a BigQuery table for census data.
  • Resolved command line issues with the bq command.
  • Established procedures for granting data access without changing the project settings.

Pending Tasks

  • Further optimization of data loading processes by converting CSV to Parquet for efficiency.
  • Continuous monitoring and troubleshooting of data upload issues as they arise.

Graph View

Backlinks