Responsibilities:

• Design and build effective, user-friendly data pipelines, tooling, and automation to accelerate Machine Learning at BlackSwan Technologies
• Collaborate with ML teams to drive the data infrastructure roadmap
• Support company’s internal ML teams with data engineering best practices
• Build complex automated data pipelines for creating datasets for ML products, facilitate efficient data management
• Build data extraction & scraping pipelines for creating datasets from open web data
• Build basic data analytics pipelines
• Assist in the search for relevant datasets for various ML, NLP tasks
• Help establish standards, practices for managing ML teams’ data infrastructure
• Write clean and tested code that can be maintained and extended by fellow engineers
   

Requirements:

• A strong predilection for good software and the processes that make it
• 3+ years of experience as a Data Engineer working in Python
• Good working SQL knowledge and experience working with relational databases, as well as working familiarity with a variety of databases
• A successful history of manipulating, processing, and extracting value from large datasets
• Experience building and optimizing data pipelines & architectures
• Strong analytical skills related to working with structured datasets
• 4+ years experience with AWS or other public cloud platforms (GCP, Azure, etc.) and corresponding data tools (e.g. AWS Athena, etc.)
• Excellent verbal and written communication skills for effective communication in a multicultural environment with teams spread throughout the world
   

Nice To Have:

• Experience managing and building data pipelines with private dumps of large knowledge bases such as Wikidata
• Experience with Big Data processing tools
• Experience with NLP and Machine Learning product development
• Familiarity with Knowledge Engineering, Graph technologies & Graph databases, Knowledge Graphs