Bipin UpadhyayaUsing Data Validation before Ingesting DataIn this blog, we run data validation Great Expectations from managed Airflow in GCP (Cloud Composer). We store the validation result in a…4 min read·Dec 6, 2020----
Bipin UpadhyayaDoes Adding a New Python Dependency In Cloud Composer Breaks the Existing DependenciesIn this blog, I outline the steps to make sure any dependencies installed in cloud composer instances don’t break the existing…1 min read·Nov 21, 2020----
Bipin UpadhyayaScheduling Dataflow Pipeline using Cloud Run, PubSub and Cloud Scheduler9 min read·Dec 19, 2019----
Bipin UpadhyayaData Catalog for data discoverability in your organizationData must be able to be discovered and accessed before it can be reused. The value in data resources is not necessarily in the data…4 min read·Nov 13, 2019----