Data Engineer (m/f) - General Electric - City of London - Wizbii

Data Engineer (m/f)

  • By General Electric
  • City of London (United Kingdom)
  • Information systems

Job description

Additional Cities

Cramlington, Stutensee
Career Level

Relocation Assistance


GE Oil & Gas
Business Segment

Oil & Gas Headquarters

Digital Technology

Germany, United Kingdom
Postal Code

NE23 1WW
Role Summary/Purpose

You will be a member of an integrated team of data engineers, software engineers, data scientists and a product owner to deliver successful outcomes driving efficiency and creating new revenue streams
Essential Responsibilities

·  Collaborating with system engineers, data scientists, frontend developers and software developers to implement solutions that are aligned with our stakeholder’s and our own strategic directions

·  Implementing, documenting and supporting data engineering solutions in an agile environment

·  Creating data visualizations using the latest development methods and infrastructure

·  Create custom software components (e.g. specialized UDFs) and analytics applications

·  Usage of Spark 2.x APIs like DataFrames, DataSets, SQL Apis, spark sessions, utilization of latest spark features for scaling of algorithm development

·  Installation of Python and other libraries required for Data engineering on Hadoop environment

·  Following tasks are carried out by Ayush independently

·  AWS EMR/Spark configurations, Monitoring of Spark UI Monitoring for Java Heap, usage of clusters and execution of jobs

·  Usage of AWS S3 buckets for executing of Spark jobs, knowledge of Spark parameters for remote code and application jar execution

·  ETL operation in Spark and Optimal storage of intermediate data on AWS S3 buckets – knowledge of parquet formats, query and read from these files in Spark jobs

·  Creating Bootstrap scripts for any libraries installs for all clusters

·  Generic monitoring of Spark Clusters using Ganglia – reporting of memory usage, CPU stats for Spark Jobs so that Data Engineering /Data scientist team can optimize the scaled spark codes

·  Experience with relational databases, (preferably Oracle, Postgres, Greenplum)

·  Significant experience writing complex SQL queries, strong PL/SQL skills

·  Experience with at least one programming language (preferably Scala, Java or Python)

·  Experience in Unix/Linux environments

·  Good English skills (written and spoken)

·  Positive attitude and team player

Applications from job seekers who require sponsorship to work in the UK are welcome and will be considered alongside all other applications. However, non-EU/EEA candidates may not be appointed to a post if a suitably qualified, experienced and skilled EU/EEA candidate is available to take up the post, as the employing body is unlikely, in these circumstances, to satisfy the Resident Labour Market Test. For further information please visit the UK Border Agency website
Desired Characteristics

·  Variety of languages and tools (e.g. scripting languages) to marry systems together

·  Integration of these libraries to current algorithm/Spark jobs- integration of JUnit, Spark Unit-Testing framework in the spark codes

·  Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala, Phoenix

·  Programing skills in Map/Reduce , Spark jobs –heavy lifting programming skills will be required for custom implementations or specialized implementations and use case based on Algorithms and machine learning models built/configured by Data scientist/Algorithm teams

·  Development of Map/Reduce , Spark Jobs/Pipelines on Hadoop distributed environment in Python and Java - Pyspark and Java-Spark jobs


About Us

GE is the world's Digital Industrial Company, transforming industry with software-defined machines and solutions that are connected, responsive and predictive. Through our people, leadership development, services, technology and scale, GE delivers better outcomes for global customers by speaking the language of industry.
Primary Country

United Kingdom

Here are some other vacancies your might like

Get a mail digest of vacancies that match your profile!
Thousand of vacancies are available for you on Wizbii :)

About General Electric

Inventer le monde industriel de demain, telle est la motivation de General Electric.

Multinationale de renom, GE est leader dans les secteurs énergie et transports dont les activités se répartissent entre GE Energy Infrastructure, GE Technology Infrastructure, GE Capital, GE Consumer & Industrial et NBC Universal. Doté d'importants pôles de recherche, General Electric met donc à profit son savoir-faire et ses connaissance afin produire de l'énergie, bâtir, faciliter les déplacement et améliorer les soins à travers le monde.

GE couvre un large de panel de métiers, niveaux et compétences à travers des fonctions d'ingénierie, marketing, santé, finance, logistique, communication, business développement etc. De ce fait, les offres du simple stage au CDI ne manquent pas en France, notamment sur les sites de Buc et Velizy Villacoublay, près de Paris.

La société offre des voies professionnelles multiples et intéressantes d'un point de vue rémunération, cadre de travail et challenges pour de jeunes stagiaires ou alternants souhaitant mettre à profit leurs compétences au sein d'une entreprise au service de tous pour inventer les solutions de demain.

Rejoignez GE pour construire votre parcours ensemble.

3199 vacancies open at General Electric

Find your next job on Wizbii,
the leading professional platform for youth's employment

Job vacancies

Jobs right at your fingertips!