The PostgreSQL database powering Liqua's IoT telemetry was hitting its limits. Pool sensors report temperature, chemical levels, and flow rates every 30 seconds. At a few dozen pools that's manageable. At hundreds of pools, it's 86,400 writes per pool per day — and PostgreSQL with a general-purpose RDS instance wasn't built for that write profile. We were seeing data loss during peak hours when the write queue backed up.
Why Firestore
Firestore's write throughput scales horizontally with document sharding. More importantly, its data model — collections of documents — maps naturally to "one document per sensor reading" without needing index management. The tradeoff is that complex relational queries are harder, but our IoT data access patterns are simple: get the last N readings for a device, or get all readings in a time range. Firestore handles both well.
The dual-write strategy
We couldn't stop the world to migrate. The approach was dual-write: for a defined period, every new sensor reading was written to both PostgreSQL and Firestore simultaneously. The application read from PostgreSQL during this period. This gave us time to backfill historical data and validate that Firestore reads matched PostgreSQL — before we cut over.
Backfilling 3 years with Cloud Run
The backfill job ran as a GCP Cloud Run function — serverless, so it scaled automatically, and we could run it in parallel chunks by device and time window. We processed 18 months of data in the first weekend, the remaining 18 months over the following week, running during off-peak hours. Total backfill: ~2.3 billion documents. Cloud Run cost for the job: under $40.
Firestore data modelling for time-series
The collection structure matters enormously for Firestore performance. We settled on: /devices/{deviceId}/readings/{timestamp-ISO}. This gives us efficient range queries by timestamp within a device. We added a top-level /daily-summaries/{date}/{deviceId} collection for aggregated views, updated by a Cloud Function that triggers on new readings. This avoided expensive client-side aggregation.
The cutover
After two weeks of dual-write with zero discrepancies in a sample of 10,000 readings per device, we updated the read path to Firestore and stopped writing to PostgreSQL. No downtime. No data loss. The PostgreSQL instance stayed running in read-only mode for two weeks as a fallback, then was decommissioned.