Data Engineer

Job description

Data Engineer

LOCATION – Weena

local conditions apply

Full-time

JOB PURPOSE

Unilever Food Solutions has traditionally been using sales representatives to engage with its customers. In recent years, the role of digital communication channels has significantly increased; using the right combination of both off and online communication channels has become essential in creating the perfect way for customers to interact with Unilever Food Solutions. For the most effective digital interaction, a good data infrastructure at the back end is crucial. To make this happen we are building a leading-edge cloud based Big Data ready data integration platform that collects, deduplicates and processes all customer data in the organization across more than 50 countries. The data is then provided as input for models that predict customer behavior and distributed across different databases and CRM/Salesforce systems. The data platform is critical in driving digital transformation and data-driven marketing across the organization.

To manage and extend this complex data platform we need a data engineer who loves to make sure everything works as expected and participates in the Scrum team to develop and deploy new

WHAT WILL YOUR MAIN RESPONSIBILITIES BE?

These are the tasks of the data engineer within the scrum team:

• Monitor if the orchestrator (Apache Airflow) runs all data pipelines correctly

• Resolve issues and bugs in Azure AKS, Azure Databricks and Apache Airflow that appear in the logs

• Ensure IT security and data privacy controls are in place across the data platform

• Develop new interfaces to and from the data platform by extending the existing Java Spring code

• Develop new data pipelines and innovations required by the business on the data platform

• Convert Databricks PySpark notebooks with data science code to production code in Scala

SKILLS YOU SHOULD HAVE

The data engineer needs to be proficient in:

• Airflow

• Bug fixing

• CI / CD concepts including Agile/Scrum

• Command line / Bash

• Docker

• Git

• Java Spring Boot

• Java Streams

• Kubernetes

• Maven

• SQL

• PySpark

• PyTest

• SBT

• Scala

• ScalaTest

• Spark Datasets

• Communicating in English

EXCITED ABOUT THE ROLE?

To apply, you must do so online. Please do not forget to upload your CV and a motivation letter. Your application will be reviewed against our requirements and we will be in touch shortly after the closing date to provide you with an update on the status of your application.

Please note this is a Direct Search led by Unilever. Applications from agencies will not be accepted, nor will fees be paid for unsolicited CVs.