If you are looking for the opportunity to build highly scalable services and rich data insights on Azure’s largest distributed platforms, then the Azure Data Governance team is the right place for you to come and take on your next challenge.

Our team owns a data pipeline that processes tens of terabytes of data per day and stores multiple petabytes of data. Our challenge is to do this efficiently, reliably, and with low latency. Our customers need to understand the data that they own and keep that data compliant with GDPR and other privacy regulations. Our job is to provide the infrastructure to do this, to produce meaningful insights about their level of compliance, and to create understandable actions that they can take to improve.

Privacy is one of the top challenges for our industry today. We are not satisfied with the status quo, and we are continually improving. Microsoft also keeps innovating, keeps growing its need for data governance, and so we will keep improving the capabilities of our system to meet the needs of all our partners.

Responsibilities

We are looking for engineers who have a track record of success, who deliver results, and who (preferably) have a background in large scale data processing systems. Everyone on the team owns significant areas of the system, directly contributes to the vision, direction, and implementation of new features, and are key to the success of our products. You will collaborate on your designs across our partner engineering teams and work closely with our PMs to design, build, and maintain a growing set of features that will serve the needs of our customers.

Qualifications

Basic Qualifications:

• Bachelor’s / Master’s degree in Computer Science or Engineering or Mathematics or IT technical discipline
• Architect, implement and optimize scalable infrastructure that enables …
• Data extraction across a wide variety of data sources
• Data ‘cleaning’, preprocessing and transformation for descriptive, predictive and prescriptive modeling
• Data Validation framework from source to endpoints ensuring data quality and integrity
• Data monitoring for reliable and fault tolerant data systems
• Automation of Data Integration / ingestion Pipelines and Orchestration in streaming and batch modes
• Design and Implement a Data platform that is Highly available with Data Intensive operations

• Ensure compliance and sovereignty of data per applicable policies

Preferred Qualifications:

• 5+ years of experience architecting, designing, and operating large scale big data services.

Preferred Qualifications: 

• 5+ year experience with Azure Big Data stack (Data Lake, Spark, P ySpark , Hadoop) and distributed processing like Apache Spark, Azure Data Factory, Azure Automation or equivalent
• 5+ years development experience on large-scale (Big) data solutions including databases (SQL/No-SQL) and ETL / Data pipelines using cloud services.
• Hands-on expertise on Enterprise Python, complex SQL development and Kusto query language
• Modern DevOps practices (including Git, CI/CD, Azure DevOps/VSTS)
• Experience working in a fast-paced/competitive environment, and have a deep desire to work collaboratively, solve problems with groups, find win/win solutions and celebrate successes.
• Long term strategic thinking and strong problem-solving skill

Working experience on any of the below technologies will be an added advantage:

• Azure Synapse Analytics
• High throughput logging systems/databases like Azure Log Analytics, ELK Stack or similar
• Service-oriented architecture using microservices and Kubernetes
• Azure SDK, PowerShell
• Experience with any analytical / visualization tool (Power BI, Cognos, OBIEE, etc.) is a plus.

AZDAT #ENGGJOBS #DTPJOBS

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.