IBM

Senior Cloud Native Architect - DataBricks

IBM

July 21, 2021

Introduction
Software Developers at IBM are the backbone of our strategic initiatives to design, code, test, and provide industry-leading solutions that make the world run today - planes and trains take off on time, bank transactions complete in the blink of an eye and the world remains safe because of the work our software developers do. Whether you are working on projects internally or for a client, software development is critical to the success of IBM and our clients worldwide. At IBM, you will use the latest software development tools, techniques and approaches and work with leading minds in the industry to build solutions you can be proud of.
Your Role and Responsibilities
As Senior Cloud Native Architect – DatabBricks is expected to be functionally knowledgeable in DataBricks Platforms and Apache Spark including NoSQL Technology areas and hands-on in either data management or open-source programming. As an Architect in our Data and AI team, you will provide best-fit architectural solutions for one or more projects; assist in defining scope and sizing of work; and anchor Proof of Concept developments. You will provide solution architecture for the business problem, platform integration with third party services, designing and developing complex features for clients' business needs. You will collaborate with some of the best talent in the industry to create and implement innovative high quality solutions, participate in Pre-Sales and various pursuits focused on our clients' business needs.
You will also contribute in a variety of roles in thought leadership, mentorship, systems analysis, architecture, design, configuration, testing, debugging, and documentation. You will challenge your leading edge solutions, consultative and business skills through the diversity of work in multiple industry domains.
Responsibilities:
  • Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks
  • Delivering and presenting proofs of concept to of key technology components to project stakeholders.
  • Developing scalable and re-usable frameworks for ingesting of geospatial data sets
  • Integrating the end to end data pipleline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
  • Working with event based / streaming technologies to ingest and process data
  • Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
  • Evaluating the performance and applicability of multiple tools against customer requirements
  • Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
Kyndryl_India
Required Technical and Professional Expertise
  • 12-15 years of overall experience with 5+ years of experience in Azure (Data Bricks) & Hadoop ecosystem (Ex: Hive, Spark, Kafka, hBase, Oozie, Sqoop etc)
  • 3 to 5 years of hands on experience in architecting, designing, and implementing data ingestion pipes for batch, real-time, and streams on Azure cloud platform at scale
  • 3 to 5 years of hands on experience with Azure cloud especially with HD Insight (Azure), Spark, Blob storage etc technologies.
  • 1 to 3 years of hands on experience in using Infoworks, Nifi, and Databricks.
  • Experience in data wrangling, advanced analytic modeling, and AI/ML capabilities is preferred.
  • Having experience in extracting/querying/Joining large amount data sets at scale is essential.
  • Experience in data wrangling, advanced analytic modeling, and AI/ML capabilities is preferred.

Preferred Technical and Professional Expertise
  • 1 to 3 years of experience in Java/Python/Scala.
  • Having exposure to R and ML technologies are a plus.
  • Knowledge of SAS, Teradata, Oracle etc other databases is a plus

About Business UnitThis position currently sits within Global Technology Services (GTS) Infrastructure Services (IS) or a shared services function supporting GTS.
As announced in October 2020, IBM intends that its managed infrastructure services business of the GTS organization will become an independent company named Kyndryl by the end of 2021, creating two distinct and powerful market-leading companies.
Together, we will advance the vital systems that power the digital economy. Serving over 4,600 technology-intensive, highly regulated customers, including over 75% of the Fortune 100, our people will design, run, and manage the most modern and reliable technology infrastructure that the world depends on today.
We will work flexibly and in partnership with our customers to amplify business outcomes while always pushing ourselves to improve and meet all opportunities. Come join our team of diverse, devoted, and empathetic technology experts who are at the center of discovering what’s next.
Please note: The final decision if this position will transition from IBM to Kyndryl is yet to be confirmed.
Your Life @ IBMWhat matters to you when you’re looking for your next career challenge?
Maybe you want to get involved in work that really changes the world? What about somewhere with incredible and diverse career and development opportunities – where you can truly discover your passion? Are you looking for a culture of openness, collaboration and trust – where everyone has a voice? What about all of these? If so, then IBM could be your next career challenge. Join us, not to do something better, but to attempt things you never thought possible.
Impact. Inclusion. Infinite Experiences. Do your best work ever.
About IBMIBM’s greatest invention is the IBMer. We believe that progress is made through progressive thinking, progressive leadership, progressive policy and progressive action. IBMers believe that the application of intelligence, reason and science can improve business, society and the human condition. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 380,000 IBMers serving clients in 170 countries.
Location StatementFor additional information about location requirements, please discuss with the recruiter following submission of your application.
Being You @ IBMIBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.