This guide walks through the recommended process for migrating from Data Lake v1 to Data Lake v2 in GRAX, including configuration steps, sequencing, and data retention considerations.
Data Lake v2 provides improved functionality and flexibility over Data Lake v1. The migration process is designed to allow you to enable Data Lake v2 alongside Data Lake v1, validate data, and then safely disable Data Lake v1 without data loss.
GRAX has already enabled Data Lake v2 for users currently on Data Lake v1 and ensured that all objects enabled in v1 are also enabled in v2.
During the initial backfill, objects will show a Processing status.
Once the backfill is complete and the object status shows Current, Data Lake v2 is fully populated and up to date for that object.
This step is critical to prevent data loss or processing gaps.
Before disabling Data Lake v1, ensure the following are complete for Data Lake v2:
Any processing rules are enabled
Any triggers or automations are configured
Any pipelines, queries, or downstream consumers are updated to reference Data Lake v2 data
After confirming the following, you may safely disable the corresponding object in Data Lake v1:
The object is Current in Data Lake v2, and
You no longer need Data Lake v1 data for that object
To disable objects in Data Lake v1, take the following steps:
Click Configure in the upper-right corner of the page
Use the arrow icons (<, >) located between the columns to move all enabled objects into the column on the left
Click Save
Once all enabled objects have been disabled, please reach out to help@grax.com and we’ll set Data Lake v2 as your default version.
The migration is not considered complete until all of the enabled objects in Data Lake v1 have been disabled.
Data Lake v1 data remains in your storage bucket after disabling v1.
GRAX recommends leaving v1 data in place for a period of time to support validation, rollback, or historical reference.
You are free to delete Data Lake v1 data at any time once:
You have migrated objects to v2, and
Your pipelines and queries have been updated to use v2 data
When deleting, be sure to only delete files under parquet/org=X/... in your bucket. Do not delete files in other parts of the bucket.
If you have questions about your specific migration path or would like assistance validating your setup, please reach out to the GRAX Support team:
Email: help@grax.com
Support is available to assist with migration issues
Last updated
Was this helpful?
Was this helpful?
