Azure HDInsight is one of the fastest growing Azure services that offers popular open-source big data technologies, such as Spark, Hadoop, Kafka, HBase and many others in a form that is easy for customers to use. HDInsight enables broad range of scenarios such as extract, transform, and load (ETL), data warehousing, machine learning, and IoT. 

As a DevOps Engineer, you will be responsible for the development and upkeep of a best in class build and release system, constantly improving its efficiency and availability as well as ensuring it meets the compliance and security requirements. 

The Azure HDInsight Build & Release Engineering team is constantly striving to modernize and improve our CI/CD infrastructure, allowing engineering teams to improve the productivity of the development cycle. The team’s goal is to leverage the best-in-class CI/CD, packaging and deployment tools, both open source and internal to build a first class build and release infrastructure for the Azure HDInsight products. 

Responsibilities

As an engineer on a highly motivated, you will help develop and support our distributed deployment and build systems.  

• Engage and collaborate with development teams to optimize build and delivery solutions. 
• Help drive the adoption of emerging cloud-based Azure Dev Ops tools and technologies. 
• Contribute to the development and optimization of complex build systems and deployment pipelines. 
• Diagnose, troubleshoot, and resolve emergent build issues. 

• Develop and maintain documentation of the build and deployment processes. 
• Train support personnel to assist in deploying, monitoring, and maintaining build systems. 

Qualifications

• Bachelor's degree in Computer Science, related degree, or equivalent experience. 
• 4+ years of experience in a DevOps with focus on developing and maintaining distributed build systems. 

• Experience with build tools such as Maven, Gradle, SBT and BigTop. 
• Understanding of YAML to define Azure DevOps pipelines. 
• Proficiency in one or more scripting languages (PowerShell, Python, Bash) 
• Experience in Linux development environments and knowledge about OS-specific packaging such as dpkg/rpm. 
• Experience in using tools such as Git, Jenkins, Docker, and Kubernetes. 

GOOD TO HAVE: 

• Experience with building and maintaining CI/CD pipelines of Apache projects such as Apache Spark, Hadoop, Hive etc. 
• Experience in developing data pipelines and Workflow management using technologies like Kafka, Spark, Azkaban. 

These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. 

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.