Data Engineer

Job description

Introduction
As a Data Scientist at IBM, you will help transform our clients’ data into tangible business value by analyzing information, communicating outcomes and collaborating on product development. Work with Best in Class open source and visual tools, along with the most flexible and scalable deployment options. Whether it’s investigating patient trends or weather patterns, you will work to solve real world problems for the industries transforming how we live.

Your Role and Responsibilities
Develop ETL pipelines for data preprocessing and clensing, integrate data pipelines with generative AI frameworks, work closely with data scientist to understand requirements and translate to design and implentation, understand , evaluate and use various large language models for generative AI use cases, use various data generation techniques to generate training data for large language models and machine learning models, manual and automated evaluation of generative AI models, apply prompt engineer techniques as required by the use case.

Required Technical and Professional Expertise
Skills:
·  Python,FastAPI,LangChain, Python packages for data science (tensorflow, scikit-learn,keras etc.,) Jupyter Notebook, Natural Language Processing, LangChain, Flask,FastAPI

Preferred Technical and Professional Expertise
Skills:
·  IBM watsonx.ai, Large Language Models, generative AI