Lead Data Engineer/Consultant Specialist

Job description

Job description

Some careers shine brighter than others.

If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.

HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.

We are currently seeking an experienced professional to join our team in the role of Consultant Specialist 

In this role, you will:

·  Ingestion and provisioning of raw datasets, enriched tables, and/or curated, re-usable data assets to enable Cybersecurity use cases.
·  Driving improvements in the reliability and frequency of data ingestion including increasing real-time coverage.
·  Support and enhancement of data ingestion infrastructure and pipelines. 
·  Designing and implementing data pipelines that will collect data from disparate sources across the enterprise, and from external sources, transport said data, and deliver it to our data platform.
·  Extract Translate and Load (ETL) workflows, using both advanced data manipulation tools and programmatically manipulating data throughout our data flows, ensuring data is available at each stage in the data flow, and in the form needed for each system, service, and customer along said data flow.
·  Identifying and onboarding data sources using existing schemas and, where required, conducting exploratory data analysis to investigate and determine new schemas

Requirements

To be successful in this role, you should meet the following requirements:

·  Ability to script (Bash/PowerShell, Azure CLI), code (Python, C#, Java), query (SQL, Kusto query language) coupled with experience with software versioning control systems (e.g., GitHub) and CI/CD systems.
·  Programming experience in the following languages: PowerShell, Terraform, Python Windows command prompt and object orientated programming languages
·  Data Acquisition, Cloud-based Data Pipelines (Azure preferred)
·  Data Transport and Data Cleaning
·  Data Engineering pipeline automation, productionisation, and optimisation
·  Technical knowledge and breadth of Azure technology services (Identity, Networking, Compute, Storage, Web, Containers, Databases)
·  Cloud & Big Data Technologies such as Azure Cloud, Azure IAM, Azure Active Directory (Azure AD), Azure Data Factory, Azure Databricks, Azure Functions, Azure, Kubernetes, Service, Azure Logic App, Azure Monitor, Azure Log Analytics, Azure Compute, Azure Storage, Azure Data Lake Store, S3, Synapse Analytics and/or PowerBI 

www.hsbc.com/careers

HSBC is committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and opportunities to grow within an inclusive and diverse environment. Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.

Issued by – HSBC Software Development India