A rapidly growing green energy technology business is seeking a Junior Data Engineer to carry out data engineering activities on their Azure Databricks Data Management Platform.
The role will be part of a small data management team setup in 2021. The team will continue to build and then support the new data platform architecture, built on Azure with the Databricks Lakehouse concept and Azure IOT services as its prime technical stack. Your role will be to build robust, secure data pipelines and data models to enable trusted datasets for data consumers across the company. This will include analysts and scientists whose activities will include ML models to drive digital twin capabilities.
This role will be remote/work from home. The role is paying up to £42,000 plus very good benefits, whilst also offering fantastic career progression opportunities.
Essential Job Functions
- Support the build of and maintain ingestion data pipelines to Delta lake through python based Databricks notebooks
- Support the build and maintain IOT data streaming pipeline from source to Cloud
- Construct and maintain data domains and models including customer engagement
- Maintain metadata and data lineage pipelines into data business and technical catalog services
- Support data scientists and analysts through delivering of trusted, secure datasets
- Work with non-technical data consumers to understand and deliver business requirement
- Use and build data monitoring and auditing tools for rapid issue identification and root cause analysis
- You'll be given lots of training opportunities to build your skillset
Experience/Minimum Requirements
- Some experience in data engineering
- Some experience in coding, ideally in Python
- Some experience and/or understanding of ETL activities
- Solid database and IT skills and a real enthusiasm for working with data
- Fast learner with a self-development mindset, initiative and self-motivation
- Team player
Desirable but not essential
- Experience in an agile software or data team
- Any Public Cloud hands-on experience
- Background in technology, engineering or manufacturing