Spark/Pyspark Developer

Job description

Position Description:

We are looking for a hands-on Developer with strong experience in building data-intensive applications using Apache Spark, working within Kubernetes-based environments, and integrating with modern DevOps and database systems. The ideal candidate will be part of a cross-functional team that builds scalable and reliable platform solutions

Your future duties and responsibilities:

• Develop and optimize Spark-based data processing pipelines for large-scale data ingestion, transformation, and analytics
• Build, deploy, and manage containerized applications using Kubernetes, preferably within Rancher
• Write infrastructure-aware, modular, and maintainable code for platform components
• Integrate with PostgreSQL databases and optimize SQL queries for performance
• Collaborate with DevOps and platform teams to build robust CI/CD pipelines
• Participate in code reviews, technical design discussions, and Agile ceremonies

Required qualifications to be successful in this role:

• 3+ years of development experience in Apache Spark (Scala or Python)
• Hands-on experience in containerization and orchestration using Kubernetes
• Familiarity with Rancher for managing Kubernetes clusters
• Proficiency in PostgreSQL with strong SQL and database interaction skills
• Exposure to build/deploy tools (e.g., Jenkins, GitHub Actions, ArgoCD, etc.)
• Strong understanding of Unix/Linux environment and shell scripting
• Strong coding, debugging, and analytical thinking skills
• Comfortable working in Agile teams and fast-paced development cycles
• Good communication and a collaborative mindset

Nice to Have:
• Experience developing data flows using Apache NiFi
• Understanding of Helm Charts for Kubernetes package management
• Experience working with cloud-native environments (preferably GCP)

Skills:

·  Apache Spark
·  Application Design
·  Application Development
·  Financial Services
·  Kubernetes
·  Object Oriented Programming
·  Postgre SQL
·  Systems Analysis
·  Technical Analysis
·  Waterfall Model
·  Web Application Development

What you can expect from us:

Together, as owners, let’s turn meaningful insights into action.

Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…

You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.

Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.

You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

At CGI, we recognize the richness that diversity brings. We strive to create a work culture where all belong and collaborate with clients in building more inclusive communities. As an equal-opportunity employer, we want to empower all our members to succeed and grow. If you require an accommodation at any point during the recruitment process, please let us know. We will be happy to assist.

Come join our team—one of the largest IT and business consulting services firms in the world.