Senior Manager - Data Engineering

Job description

Overview

Main Purpose  

·  Manage delivery of reporting and analytics product, work as an SME on products to provide adequate support needed to the developers 
Responsibilities

Accountabilities  

·  Active contributor to code development in projects and services. 
·  Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products. 
·  Build and own the automation and monitoring frameworks that capture metrics and operational KPIs for data pipeline quality and performance. 
·  Responsible for adopting best practices around systems integration, security, performance, and data management defined within the organization. 
·  Empower the business by creating value through the increased adoption of data, data science, and business intelligence landscape. 
·  Collaborate with internal clients (data science and product teams) to drive solutions and POC discussions. 
·  Develop and optimize procedures to “productionalize” data engineering pipelines. 
·  Define and manage SLA’s for data products and processes running in production. 
·  Support large-scale experimentation was done by data scientists. 
·  Prototype new approaches and build solutions at scale. 
·  Research in state-of-the-art methodologies. 
·  Create documentation for learning and knowledge transfer. 
·  Create and audit reusable packages or libraries. 
Qualifications

 

Key Skills/Experience Required  

·  2+ years of overall technology experience that includes at least 2+ years of hands-on software development, data engineering. 
·  2+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala, etc.). 
·  Experience with version control systems like Github and deployment & CI tools. 
·  Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines. 
·  Working knowledge of agile development, including DevOps and DataOps concepts. 
·  Thorough knowledge in Spark, SQL, Python, Databricks and Azure 
·  Excellent communication skills, both verbal and written, and the ability to influence and demonstrate confidence in communications with senior level management. 
·  Comfortable with change, especially that which arises through company growth.nt. 
·  Proactively drive impact and engagement while bringing others along. 
·  Ability to learn quickly and adapt to new skills. 

Differentiating Competencies Required  

·  Strong problem-solving, analytical, and critical thinking skills 
·  Ability to collaborate in cross-functional teams 
·  Ability to manage multiple projects and report simultaneously across different stakeholders in a fast-paced environment 
·  Excellent verbal and written communication skills  
·  Strong attention to detail