Spark Administrator /Senior Consultant Specialist

Job description

Job description

Some careers shine brighter than others.

If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.

HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.

We are currently seeking an experienced professional to join our team in the role of  Senior consultant Specialist.

In this role, you will:

·  The following as Spark Administrator responsibilities
·  Install, configure, and manage Apache Spark 3.x. clusters.
·  Monitor and optimize Spark cluster performance.
·  Troubleshoot and resolve Spark-related issues.
·  Manage Spark job scheduling and resource allocation.
·  Ensure high availability and reliability of Spark clusters
·  The following as  DevOps Practices:
·  Implement and manage CI/CD pipelines using tools like Jenkins, Nexus, GitHUB CI.
·  Automate infrastructure provisioning and configuration management using tools like Ansible, Ansible Tower Monitor system performance and ensure system availability and reliability.
·  Implement and manage containerization using Docker i.e. build Jupyter notebook.
·  Collaborate with development teams to ensure smooth deployment and operation of applications.
·    The following in Security and Compliance:
·  Code scanning tools SAST, SonarCube , SonatypeIQ
·  Implement security best practices for Spark clusters and DevOps pipelines.
·  Ensure compliance with industry standards and regulations.
·  Collaboration and Communication

Requirements

To be successful in this role, you should meet the following requirements:

·         Bachelor’s degree in computer science, Information Technology, or a related field.
·  Proven experience as a Spark Administrator, with hands-on experience in Spark 3.x.
·  Strong understanding of Spark architecture and internals.
·  Experience with DevOps tools and practices, including CI/CD, configuration management, and containerization.
·  Proficiency in scripting languages such as Python, Bash, or PowerShell.
·  Experience with cloud platforms such as AWS, Azure, or Google Cloud.
·  Strong problem-solving skills and attention to detail.
·  Excellent communication and collaboration skills. Work closely with data engineers, other stakeholders to understand requirements and provide solutions.
·  Document processes, configurations, and procedures.

 

You’ll achieve more when you join HSBC.

www.hsbc.com/careers

HSBC is committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and opportunities to grow within an inclusive and diverse environment. Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.

Issued by – HSDI