Google Cloud Senior Data Engineer

Posted Date:4 months ago
Share
Work location
Germany, NA
Rate
85 €/h; 85 €/h (remote)
Workload
Full time
Work type
Remote
Experience
Start date:ASAP
Duration:4 months

Location

Skills

.Java BIGQUERY BASH GITHub TerraForm

Languages

English (Native)

Languages

English (Native)

Skills

.Java BIGQUERY BASH GITHub TerraForm

Job Description

2 FTE

Location: currently planned remote

Languages: English

Project description:

*** Senior Data Engineer - Retail Data Platform on Google Cloud ***

 

The Retail Data Platform (RDP) provides a constant collection of all Private Bank‘s relevant data in a central Data Core in the Google Cloud. The RDP will provide data warehouse like functionalities for operational and analytical use cases and ensure neartime Data provisioning for multiple Data consumers.

 The scope of the RDP project includes mainly the design and implementation of data load and transfer pipelines from multiple on-premise data sources to the RDP on Google Cloud including batch or stream based data load processes.

Another important part is the data modeling of this data to store in the RDP following the Data Vault 2.0 approach.

 

The main responsibility of this Data Engineer position is to help designing and implementing the Data load and transfer pipelines from the existing on-premise Data Sources to the RDP in the Google Cloud. In that regards experience in designing and implementing such data load processes for batch/ file or for steam based data ingests on the Google Cloud platform is a key to successfull help the RDP team reaching their goals.

 

MUST HAVE Skills:

Senior Experience in working as Data Engineer as part of an agile Development team.

More technical mut have skills - see below

 

DESIRED Skills:

Expert in Google Cloud Platform (Area Data Engineering)

 

MUST HAVE TECHNICAL Skills

Google specific:

• BigQuery

• Google Dataflows

• Cloud Composer / Apache Airflow

• Google Cloud Storage

• Google PubSub

• Google Functions

• Cloud SDK (incl. gsutil, bq, …)

 

• Nice to have: Cloud Spanner, Cloud SQL

 

Programming Languages:

• Java and/or Python. (incl. development frameworks/tools like Maven, venv, pip, …),

* testing best practices

• SQL

• Linux Bash / Windows CMD

• YAML

 

IDEs:

• Idea IntelliJ and / or PyCharm or MS Visual Code

* CI-CD and version control:

• Github (Actions)

• Git

 

IaaS

• Terraform (Enterprise)

• IaaS testing methods

 

Methods:

• Data Vault Modelling

• Test Driven Development

• Modern CI-CD driven development processes

• Agile mindset, working self-organized and in self-organized teams

2 candidates required
Public job

Location

Register to Apply Job