Data Engineer
PORTUGAL IT development
Job description
ROLE OVERVIEW
We are seeking a talented and experienced Data Engineer to join our growing data team. In this role, you will be instrumental in building, maintaining, and optimizing our data infrastructure, with a strong emphasis on DataOps principles. You will work closely with data analysts, and other engineers to ensure the reliability, scalability, and security of our data pipelines.
KEY RESPONSIBILITIES
· Translate the business requirements into technical specifications, in cooperation with the data product owner.
· Design, develop, and maintain robust and scalable data pipelines using Airflow.
· Utilize dbt for data transformation and modeling, ensuring data quality and consistency.
· Employ Terraform for infrastructure as code (IaC), automating the provisioning and management of our data infrastructure.
· Implement and maintain CI/CD pipelines for data engineering workflows.
· Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks.
· Implement data quality checks and monitoring to ensure data accuracy and reliability.
· Contribute to the development of data engineering best practices and standards.
· Manage and maintain data repositories using Git.
· Automate data-related tasks to improve efficiency and reduce manual effort.
· Cooperate with the support teams and provide L3 support to comply with predefined SLA’s
KEY QUALIFICATIONS
· University degree in computer science, engineering, or comparable field.
· 3-5 years of experience as a Data Engineer or in a similar role.
· Experience with data warehousing concepts and technologies.
· Hands-on experience with Google Cloud Platform (GCP) data services (e.g., BigQuery, Cloud Storage)
· Knowledge of dbt and version management tools (e.g. Git).
· Experience with infrastructure as code (IaC) using Terraform.
· Experience with workflow orchestration tools such as Airflow.
· Strong proficiency in Python and SQL.
· Strong problem-solving and analytical skills.
· Excellent communication and collaboration skills.
PREFERRED QUALIFICATIONS
· Experience with CI/CD pipelines.
· Experience working with SAP platforms as data sources (e.g., SAP ECC, SAP S/4HANA, SAP BW, SAP Datasphere).
· Knowledge of SAP data structures and extraction methods.
· Knowledge of data visualization tools (e.g. Looker).
· Scrum methodology.