Our Designer in the Netpulse team will use cutting edge technologies to design and build robust enterprise solutions using governed Design patterns and frameworks.

 
•    Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using Scala Spark, NiFi, HBase
•    Design and develop pipelines for Streaming and Batch processes
•    Assuring end to end data availability and quality
•    SPARK performance/tuning/optimisation
•    Resolving problems in complex data pipelines with multiple technologies
•    Developing scalable and re-usable frameworks for ingestion and transformation of large data sets
•    Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data 
•    Helps to resolve technical problems
•    Working with other members of the project team to support delivery of additional project components
•    Evaluating the performance and applicability of multiple tools against requirements
•    Developing tangible and ongoing standardisation, simplification and efficiency of engineering processes, reviewing and revising continuous improvement opportunities
•    Ensuring that all data acquired is fully described/understood and communicated using appropriate tools
•    Excellent oral and written communication skills for all levels of an organisation

Essential Skills

•    Direct experience of building data piplines using GCP Native technologies and Spark
•    Experience building data warehouse solutions using ETL / ELT frameworks
•    Experience with GCP Native technologies, Apache Kafka, Hive, HBase and Nifi for use with streaming and event-based data
•    Experience working with structured and unstructured data
•    Comprehensive understanding of data management best practices including demonstrated experience with data profiling, sourcing, and cleansing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking and matching.
•    Experience of migrating data pipelines to Big Query.
•    Extensive skills in SQL, both at production grade and at analytical level, gained through intensive application in a commercial business environment
•    Ability to capture business requirement and transform it to low level design that can be actioned by a developing team. 
 
Desirable skills

•    Experience of building large scale data pipelines on at least one Cloud Platform (GCP preferred)
•    Experience of working with Telco data
•    Vendor and stakeholder management experience
•    GCP Big Data Architecture certification
•    Cloud migration experience 
•    Experienced in deploying data solutions and cloud infrastructure via CI/CD pipelines
•    Experienced in deploying Infrastructure as Code (Terraform / Cloudformation etc)
•    Knowledge of REST/Graph APIs and how they can be used in a data environment.
•    Knowledge of Docker/Kubernetes, and how these can be used to simplify deployments.

We’re looking to pay a great compensation package (depending on experience) for this position. We also offer plenty of extras to sweeten the deal, which could include things like bonuses, life assurance cover, health care and lots of flexible benefits.  
 
Also, every employee has their personal development supported with a LinkedIn learning account; plus other role specific learning available through our award-winning digital learning platform - O2 Campus. 
 
We also believe a great work-life balance is important, so we’re open to considering flexible working arrangements. Like to know more, feel free to raise it.  
 
Join us and we’ll encourage you to be bold every day. So take a deep breath, your career is about to go to exciting new places. 
 
If you have any questions around the role then please email ResourceTUK@o2.com who will be happy to help.