Lead Architect

Job description

Introduction
At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk.

Your Role and Responsibilities
We're looking for software development candidates that want to turn ideas into reality, with a focus on innovation and development of new technology. Our strategic Product Development team creates and delivers exciting database features that impact the business. You'll work alongside global and local multidisciplinary teams of Developers, Architects, Engineers and Analysts focused on the next generation of solutions.

Do you enjoy working and thrive in a challenging yet collaborative team environment? If so, you'll fit right in with our team. You employ creative thinking and collaboration as the foundation of your daily approach. Ideally, you'll possess relational database knowledge (including SQL) and experience across multiple operating systems (Linux, Unix, and Windows). Passionate about technology and software, you have extremely strong analytical, problem-solving and troubleshooting skills.

Responsibilities:
·  Lead the development of advanced tools for casting and converting data from Delta Lake to Apache Iceberg manifests, ensuring robust data integrity throughout the transformation process.
·  Drive metadata migration, copy, or synchronization strategies across diverse platforms, orchestrating seamless schema transitions to support comprehensive data interoperability.
·  Architect and develop migration tools to facilitate efficient data movement into Iceberg buckets, enabling flexible integration with multiple platforms and systems.
·  Engineer solutions to optimize data handling and storage methodologies, addressing performance gaps, and improving overall system efficiency.
·  Spearhead the implementation and support of Python-based User-Defined Functions (UDFs) within a micro-service architecture, ensuring versatility and scalability across environments.
·  Design and implement client interfaces, leveraging technologies like Arrow Flight, to cater to varied BI use cases and tools, ensuring optimal performance and features.
·  Provide expert guidance on data integration and ETL methodologies, overseeing and contributing to complex data transformation processes.
·  Establish and enforce warehouse and lakehouse operational best practices, ensuring adherence to industry standards and compliance requirements.
·  Act as a subject matter expert, advising on diverse BI use cases and providing strategic guidance for project requirements. We strive to continue to make IBM Data & AI a place where you want to be – Inclusive. Creative. Collaborative. This fosters a culture where you are motivated to produce the best results possible...We’re hiring…..JOIN US

Required Technical and Professional Expertise

·  15+ years of experience in software development
·  Extensive experience in developing and leading data integration initiatives.
·  Proficiency in Python and other scripting languages for tool development and automation.
·  Expertise in data warehousing and lakehouse concepts, with hands-on experience in their implementation.
·  In-depth understanding of cloud-based data platforms and technologies.
·  At least 7 years experience in Python Backend Programming
·  At least 7 years experience in Relational Database and SQL
·  At least 2 years experience in Open Source Technologies
·  Experience using Linux, Unix, and Windows operating systems
·  Experience with SQL Data bases (Db2, Oracle, SQL Server etc)
·  Strong analytical, problem-solving, and decision-making skills.
·  Excellent communication and leadership abilities, capable of mentoring and collaborating within a team environment.

Preferred Technical and Professional Expertise

·  Proven experience with complex data migration and synchronization projects.
·  Advanced knowledge of BI tools and their integration across diverse environments.
·  Familiarity with performance optimization methodologies for data handling and storage.
·  Previous experience in architecting and implementing UDF frameworks within scalable architectures.