AI Data Pipeline Specialist

Introduction

watsonx Code Assistant is an exciting offering from IBM that strives to revolutionize enterprise software development with Generative AI. We need your expertise, your motivation and your collaboration to take watsonx Code Assistant to the next level.

Your Role And Responsibilities

AI Data Pipeline Specialist for watsonx Code Assistant, you will be responsible for :

• Collecting and cleansing data that is used as training data for building and customizing code generation models
• Creating and maintaining automated data pipelines for data collection and cleansing, using components from watsonx.ai and watsonx.data (among other technologies)
• Having a “data ops” mindset to ensure the pipelines are reliable and provide proper data lineage
• Prototypeing data pipelines using Python and Jupyter Notebooks
  
  

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• 2+ years of experience in Data Engineering in Python
• 2+ years of experience in modern data pipeline technologies
  
  

Preferred Technical And Professional Experience

• Data curation for generative AI models
• Data Lakehouse technologies, e.g. Milvus
• watsonx.ai
• Data Warehousing
• Big Data Management: Manage big data infrastructure and execute data engineering tasks for efficient data processing.