Data Engineer

Job description

DESCRIPTION

EU Supply Chain and Trans Tech is looking for a Data Engineer to play a key role in maintaining and transforming EU Supply Chain and Trans data lake into meaningful analytics. The ideal candidate will be passionate about parteniring in a team that drives next generation extremely large, scalable and fast distributed systems on AWS stack (with focus on Redshift). The Data Engineer will help us grow our capability set, drive efficiency, and improve our overall data technology offering.
The ideal candidate has exposure to data modelling, ETL design, and can build optimized data pipelines for a BigData environment. This candidate relishes working with large volumes of data, enjoys the challenge of highly complex technical contexts, and above all else, is passionate about data and analytics. S(h)e should possess excellent communication skills as the candidate will work closely with diverse teams and sometimes senior leadership. In addition, the candidate should demonstrate thought leadership to drive business insights through machine/deep learning initiatives using out elaborate data sets.

Along with complex problems to solve, we provide you world class work environment and a chance to work with few of industry most talented team members in data engineering space and opportunity to contribute and create history while having fun.

Responsibilities
· Implement and test data pipelines from variety of systems (batch and real time).
· Collaborate with the development team and end customers.
· Mentor data engineering interns
· Advise the team on SQL optimization, performance tune when required
· Troubleshoot existing datasets, broken pipelines, data integrity issues
· Ensure quality of data through tested deliverables and auditing
· Understand team, data and software architecture; actively seeks knowledge and solutions

Desired profile

BASIC QUALIFICATIONS

Bachelors in Computer Science, Engineering, Statistics, Mathematics or related field
· 1-3 years of experience in data engineering / business intelligence space
· Curious, self-motivated & a self-starter with a ‘can do attitude'. Comfortable working in fast paced dynamic environment.
· Good understanding of ETL concepts and explosure to large-scale, complex datasets proessing using traditional or map reduce batch mechanism.
· Proficient in writing performant SQL working with large data volumes; ability to understand and action query plans
· Experience with at least one of scripting languages(e.g., UNIX Shell scripting, Python, Perl, Ruby).
· Exposure to AWS stack/cloud computing
· Clear thinker with superb problem-solving skills to prioritize and stay focused on big needle movers

Bachelors in Computer Science, Engineering, Statistics, Mathematics or related field
· 1-3 years of experience in data engineering / business intelligence space
· Curious, self-motivated & a self-starter with a ‘can do attitude'. Comfortable working in fast paced dynamic environment.
· Good understanding of ETL concepts and explosure to large-scale, complex datasets proessing using traditional or map reduce batch mechanism.
· Proficient in writing performant SQL working with large data volumes; ability to understand and action query plans
· Experience with at least one of scripting languages(e.g., UNIX Shell scripting, Python, Perl, Ruby).
· Exposure to AWS stack/cloud computing
· Clear thinker with superb problem-solving skills to prioritize and stay focused on big needle movers