Lead Data Engineer

Job description

Job Title: Lead Data Engineer
Location: London
Working Style: Hybrid - 2 days in office and 3 days WFH
Day Rate: £800 Inside IR35
Contract Until: 31st March 2025 (With a high chance of extension)
Security Clearance: SC Clearance
Industry: Central Government

Are you an experienced Data Engineer looking for an exciting opportunity to join a dynamic team in London? Our client, a leading Central Government organisation, is seeking a Lead Data Engineer. This is a temporary role until 31st March 2025, with a competitive day rate of £800 Inside IR35.

As a Lead Data Engineer, you will be responsible for scoping the modernisation of the data pipeline and ensuring a robust and maintainable pipeline is delivered to high QA standards. You will collaborate with the analytics engineering teams to implement an analytics-engineering-led approach that serves multiple customer groups.

Key Responsibilities:

·  Identify opportunities to modernise the data pipeline

·  Implement technical solutions to deliver a robust and maintainable pipeline

·  Utilise your expertise in Python, SQL, and jinja, along with experience in version control tools and cloud platforms

Skills & Experience Required:



Essential:

·  Excellent python, SQL, and jinja skills

·  Understanding and awareness of R

·  Applied automated testing, e.g., pytest

·  Familiarity with version control tools, such as git

·  Experience with cloud platforms, ideally AWS

·  Experience with orchestration tools, ideally airflow

·  Experience in building CI/CD workflows, preferably with GitHub Actions

·  Experience in reviewing, designing, and developing data pipelines

·  Experience with dbt, including implementing best practises

·  Applied data modelling experience (conceptual, logical, physical)

Desirable:

·  Experience with infrastructure as code, ideally Terraform

·  Knowledge of Data Lake Table Formats, such as Iceberg

·  Familiarity with Kubernetes and container-based workflows

·  Experience with orchestration systems, such as Airflow

·  Ability to guide others and ensure best practises are followed

·  Experience in working in or leading an Agile team

Join our client's team and contribute to the modernisation of their data pipeline. This is an exciting opportunity to work on impactful projects and collaborate with a diverse range of professionals. Apply now and take the next step in your data engineering career!

Please note that SC Clearance is required for this role.

Adecco is a disability-confident employer. It is important to us that we run an inclusive and accessible recruitment process to support candidates of all backgrounds and all abilities to apply. Adecco is committed to building a supportive environment for you to explore the next steps in your career. If you require reasonable adjustments at any stage, please let us know and we will be happy to support you.