Building an integrated analysis platform with GWS
Ecorbit is the largest comprehensive environmental company in Korea. Currently, we are developing and applying smart environmental technologies such as resource-circulating waste management, water management, resource circulation, new and renewable energy, and materials business based on our accumulated technology, know-how and network
Ecorbit is a comprehensive environmental company that performs sewage treatment, landfill, and incineration, so the source data was diverse. They wanted to solve the data silo and integrate analysis of the entire data. Cloocus proposed building a platform that integrates and analyzes scattered data with a centralized data warehouse such BigQuery, and Digital Transformation in conjunction with GWS.
The main objectives of this project are.
- 1. Integrate data scattered across various environments into a cloud environment
: Overcome the data silo phenomenon by using Cloud Functions to store data from various sources, including local PC, in BigQuery.
2. Create dashboards to analyze aggregated data to gain market insights
: Gain insights using Looker Studio, a BI solution that easily connects to various data sources such as BigQuery.
3. Configurate platform that allows users to create reports in a format familiar to users
: Set up a platform that allows users to create, save, anc view daily work logs using Google Sheets to improve the learning efficiency of field users.
Cloocus proposed linking Google Sheets and Google Apps Script to store data collected from each branch in a standardized format. In addition, data from various sources, such as locally created sheets and other vendor’s cloud, are stored in BigQuery using Cloud Functions, and historical data from other vendor’s storage is collected and stored using Dataproc, creating an architecture that utilizes it in conjunction with business data stored in BigQuery. I suggested it.
We also proposed an architecture that uses Cloud Functions to store data from various sources, such as locally created Sheets and other vendor’s clouds, in BigQuery, and collects and stores historical data from other vendor’s storage using Dataproc in conjunction with business data stored in BigQuery.
We also automate existing reporting systems by visualizing integrated data to KPI using Looker Studio.
- Google Cloud Storage
- Cloud Functions
- Cloud VPN
- Cloud NAT
- VPC Service Controls
- Access Context Manager
- Organization Policy
- Google Sheets (with Google Apps Script)
- Looker Studio
Through this project, Ecorbit uses an integrated data pipeline linked to GWS, reducing the time spent on data querying to create reports from approximately 4 hours to 5 minutes, and improving visibility into work performance, enabling quick results sharing among representatives.
In addition, by using the GWS product line to build a work environment similar to the existing one, the burden of additional training for field workers was reduced, and data could be viewed and transmitted with just one button. This not only laid the foundation for cloud-based integration of assets that had been individually managed on local PCs in branches across the country, but also increased work efficiency through office automation.
Since then, Ecorbit, along with BigQuery ML, expects to introduce AI/ML systems that can be easily utilized for branch-specific demand forecasts and overall supply chain management through the Looker Studio dashboard.