Data Platform Engineer

Data Platform Engineer – Data Engineering / ML Data / Video Data / Python / GDPR / ASPICE

Take the next step in your career now, scroll down to read the full role description and make your application.

We are currently partnered with a global technology and engineering organisation developing advanced data platforms and machine learning technologies for large-scale visual and video datasets. As part of their continued investment in machine learning and data infrastructure, they are looking to hire aData Platform Engineer – Video Data & ML Dataset Management to own the end-to-end platform for managing a large and growing collection of video data used for machine learning.

Key responsibilities

• Design and build automated ingestion pipelines for large video and media files
• Implement validation, metadata extraction, deduplication, registration, and quality checks during ingestion
• Design and maintain a robust relational metadata catalogue covering videos, people, recording campaigns, cameras, vehicles, and dataset usage
• Ensure strong data integrity through appropriate database relationships, constraints, validation, and lifecycle management
• Implement GDPR and consent-management capabilities for video datasets
• Link individuals to recorded video data and track consent status, expiry, retention, and relevant ownership information
• Automate notifications and workflows relating to consent and compliance requirements
• Maintain appropriate audit trails and traceability for sensitive data
• Establish dataset versioning and reproducible train/test/validation tracking linked to ML experiments

Key requirements

• Strong professional experience in data engineering and building production-quality data pipelines
• Experience with data ingestion, ETL, workflow orchestration, and automation
• Strong relational database design and data modelling skills
• Understanding of database constraints, relationships, data integrity, and lifecycle management
• Strong Python programming skills for data pipelines, automation, xlxgzvr and API development
• Experience handling large media or binary datasets
• Experience with containerisation and secure deployment practices
• Experience with dataset versioning and reproducibility for machine-learning workflows
• Experience with data cataloguing and metadata-management platforms
• Experience integrating low-code platforms or internal tooling for data entry, workflow management, and visualisation
• Practical experience with video and media processing

Keywords

By applying to this role, you understand that we may collect your personal data and process it in line with our privacy policy.