QA Analyst Automation

Job description

Position Description:

The QA Analyst is responsible for validating the quality of deliverables. The QA Analyst must demonstrate a strong aptitude for automation and collaborate closely with client teams to structure and industrialize this practice using a [specific] approach.

We aim to foster a greater understanding of quality assurance practices among all our contributors. We are committed to optimizing different types of tests and developing a quality assurance process from the very beginning of the delivery cycle, tailored to the context of each project.

Your future duties and responsibilities:

AI Systems and Models Testing Design and execute comprehensive test strategies for AI systems and models, including prompt engineering, output evaluation, and bias/safety testing. Develop deep understanding of LLM behavior—tokenization, embeddings, attention mechanisms, and inference—to anticipate failure modes. Construct effective prompts, recognize hallucinations and off-target outputs, and assess quality across accuracy, tone, coherence, and bias dimensions. Apply evaluation metrics specific to generative AI and establish appropriate thresholds.

Test AI systems integrated with RAG pipelines and knowledge bases, validating data quality and retrieval accuracy as they impact model outputs. Understand vector database mechanics, similarity search thresholds, embedding drift, and test edge cases including near-duplicate documents, sparse vs. dense embeddings, and performance under scale. Leverage LangChain and LangGraph frameworks to read code, understand chain and graph construction, identify failure points, and write test harnesses. Validate integration points using MCPs, testing tool availability and error handling.

Test Strategy and Planning Define and execute comprehensive test strategies for securitization platforms, ensuring coverage across functional, regression, integration, and performance testing. Establish testing standards and best practices that span both traditional QA and AI-specific validation.

Test Automation and Framework Development Design, build, and maintain automated test suites to accelerate release cycles and improve coverage. Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.

Securitization Lifecycle QA Validate end-to-end deal workflows including setup, structuring, processing, and distributions. Ensure data integrity across upstream and downstream systems through reconciliation testing and reporting.

Release and Regression Testing Coordinate regression testing for platform releases, patches, and infrastructure changes. Ensure stability and backward compatibility, particularly during critical processing windows.

Cross-Functional Collaboration Partner with developers, business analysts, and product owners to clarify requirements, identify edge cases, and ensure testability of new features. Lead defect triage sessions, prioritize issues based on business impact, and maintain clear documentation through to closure.

Quality Leadership and Reporting Define and monitor key quality indicators including defect density, test coverage, and automation rates. Present findings to leadership and recommend improvements. Mentor junior QA team members and foster a culture of quality across the team.

Production Support Provide production support during critical processing windows, investigate incidents, and coordinate root cause analysis and remediation efforts.

Qualifications

Experience 7+ years of quality assurance or quality engineering experience, with at least 3 years in a lead or senior capacity. Strong domain knowledge in securitization, capital markets, or similar asset classes.

Technical Skills Hands-on experience with test automation tools (Selenium, Robot Framework, Playwright, or similar). Proficiency in programming languages including Java and Python, with demonstrated framework implementation expertise. Hands-on API automation and backend system validation experience. Proficiency in database query development, data validation, and reconciliation testing. Experience with CI/CD pipelines and DevOps practices (Jenkins, GitHub, or similar).

AI and ML Competencies Core understanding of LLM architecture and behavior. Hands-on experience with LangChain and/or LangGraph frameworks. Knowledge of RAG pipelines, vector databases, and agentic solutions. Familiarity with Model Context Protocols (MCPs) and integration testing. Understanding of bias, safety, and red-team testing methodologies for AI systems.

Soft Skills and Mindset Demonstrated passion for automation, continuous learning, and solving open-ended problems. Excellent analytical, problem-solving, and communication skills. Ability to manage multiple priorities in a fast-paced, deadline-driven environment. Ability to mentor and lead teams while maintaining focus on quality excellence.

Required qualifications to be successful in this role:

Core Quality Engineering Must Haves:

• Experience: 5+ years of professional experience in Quality Engineering and Automation.

• Multi-Platform Testing: Proven expertise in Web (Selenium, Playwright, or Cypress), Mobile (Appium, XCUITest etc), and API testing (Postman, RestAssured).

• Service Virtualization: Experience with tools like WireMock or similar technologies to simulate complex backend dependencies.

• Cloud Testing Ecosystem: Hands-on experience with cloud-based cross-browser and mobile testing platforms such as Perfecto, LambdaTest, or Sauce Labs.

• DevOps Mastery: Deep understanding of CI/CD pipelines (GitHub Actions, Jenkins, or Azure DevOps) and a "Quality as Code" mindset.

• Version Control: Expert knowledge of Git branching and merging strategies

 

Strong written and verbal communication skills in English

 

Nice to Have:

Gen AI in QE: Experience leveraging Gen AI (e.g., Copilot, Claude, or specialized QA agents) for test script generation, maintenance, and edge-case discovery.

Performance & Observability: Experience with performance testing and production monitoring using tools like Splunk or Dynatrace.

Contract Testing: Experience with Pact or similar tools for microservices validation.

CGI provides a reasonable salary range estimate for this position. The calculation of this range depends on various factors, including skill level, geographic market, experience, education, and professional licenses and certifications. Compensation decisions are made on a case-by-case basis. A reasonable salary range estimate is between $60,000 and $115,000. This position is currently vacant.

Bilingualism (French and English) is required for this position due to the nature of the role requiring interaction with national and global clients.

Skills:

·  English
·  French
·  Artificial Intelligence
·  Bitbucket
·  GIT
·  Jira
·  Python
·  Robot Framework

What you can expect from us:

Together, as owners, let’s turn meaningful insights into action.

Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…

You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.

Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.

You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

At CGI, we value the strength that diversity brings and are committed to fostering a workplace where everyone belongs. We collaborate with our clients to build more inclusive communities and empower all CGI partners to thrive. As an equal-opportunity employer, being able to perform your best during the recruitment process is important to us. If you require an accommodation, please inform your recruiter.

To learn more about accessibility at CGI, contact us via email . Please note that this email is strictly for accessibility requests and cannot be used for application status inquiries.

Come join our team—one of the largest IT and business consulting services firms in the world.