Data Engineer – Big Data – Google Cloud (GCP) – Inside IR35
Large logistics company require a big data engineer with specific knowledge of Google Cloud.
You will have experience in the big data ecosystem to work with the Data Engineering and Data Science teams from the Advanced Analytics department in addition to multiple stakeholders at er to design the architecture of batch and streaming data analytics and machine learning.
About the role:
You will be a trusted advisor to solve some of the most complex and forward thinking big data problems. You will work with a variety of departments to gather their data and analytics requirements and you will implement proof of concepts and blueprints to evangelise the most suitable Big Data tools and best data manipulation and engineering procedures across the company data analytics community.
Specifically you will:
- Understand current data infrastructure and sources of data and design efficient and standardized ways to ingest, process, store and analyse these sources of data.
- Implement proof of concepts to demonstrate the value of using the solutions agreed across the Advanced Analytics department.
- Plan infrastructure capacity depending on sources of data to integrate with the data platform and plan and optimize the public cloud usage.
- Identify processes that should be automated to improve the efficiency of the teams and provide solutions to lead to automation in particular in the continuous integration pipelines, data manipulation and infrastructure management.
- Define infrastructure and applications monitoring and log aggregation strategy and architecture to make sure of visibility and governance within the data feeds and analytics applications.
- Ensure that the technical documentation goes from high level details to low level details in a way that is easy to understand to a variety of stakeholders and take ownership in keeping the documentation updated and relevant.
- Provide technical mentorship to engineers, data scientists, sysadmins and other team members in order to guide usage and implementation of data projects in the most efficient way.
This role I based in central London (City) and offer a day rate up to £450 per day (inside IR35). Initial duration 6 months.