Lead Data Engineer

Job description

Lead Data Engineer required to work with a Government department. This is an initial 3 month contract, paying £800 a day, inside IR35, hybrid working.

Job Description:

We are looking for an experienced, collaborative, Data Engineer to join the team to scope modernisation of the data pipeline. Specifically, the purpose of this role is to identify how an analytics-engineering-led approach could be implemented technically to ensure there is a robust and maintainable pipeline delivered to high QA standards, how that approach could be congruent with the existing practices of our analytics

Essential

·  Excellent python, SQL and jinja skills

·  Understanding and awareness of R

·  Applied automated testing, for example with pytest

·  Familiar with version control tools, such as git.

·  Experience with cloud platforms, ideally AWS

·  Experience of orchestrations tools, ideally airflow

·  Experience of building CI/CD workflows, ideally with GitHub Actions

·  Experience of reviewing, designing and developing data pipelines

·  Experience of using dbt, including implementing dbt best practices

·  Applied data modelling (conceptual, logical, physical) experience

·  Active SC clearance

Desirable

·  Experience of infrastructure as code, ideally Terraform

·  Data Lake Table Formats, such as Iceberg

·  Kubernetes and container-based workflows

·  Orchestration systems, such as Airflow

·  Experience of guiding others to apply best practices and ensuring practices are up to date with latest developments in data engineering

·  Working in or leading an Agile team

Please apply should you fit the above criteria