Google Cloud

Notes for google cloud data engineer certification.

The following list is based on the sample case study for the GCDE certification exam: https://cloud.google.com/certification/guides/data-engineer/casestudy-flowlogistic

The case study focuses on a logistics company tracking orders and shipments via rail, truck, aircraft, and ships.

Goals and Motivation

Goals:

Implement real-time inventory tracking system that tracks locations
Perform data analytics on order and shipment logs (structured/unstructured data) to make decisions about deploying resources, targeting customers, and expanding into markets
Predict delays in shipments

Requirements:

Reliable, reproducible environment that scales
Aggregated data in centralized data lake
Historical data used to perform predictive analytics on future shipments
Accurate tracking of worldwide shipments (proprietary technology)
Improvement of business agility and speed of innovation via rapid provisioning of new resources
Analysis and optimization for performance in the cloud
Migration to cloud, if all other requirements met

Deeper reasoning:

Inability to upgrade infrastructure hampering growth and efficiency
Ineffective at moving data around
Need to better understand where/who customers are, what they are shipping
IT is too busy managing infrastructure to organize data/build analytics/implement tracking technology
Penalties for late shipments and deliveries translates into direct correlation between profitability and bottom line

Databases:

Applications:

Storage:

Analytics:

Miscellaneous servers: