Data Platform Reliability Engineer

Job Summary

We are hiring a specialized Data Platform Reliability Engineer to focus on the development, operation, and stability of our data processing platform. This role is crucial for ensuring high data quality, maintaining platform security and compliance (GDPR), and pioneering innovative development enablement tools (GitHub Codespaces, K8s networking). The ideal candidate combines a deep understanding of operations and modern development methodologies with a commitment to empowering development teams.

Key Responsibilities

Development Enablement & Platform Access

• Simplify Debugging Workflow: Implement solutions to significantly streamline the debugging experience for developers. This includes enabling reliable access to cloud resources (e.g., databases, Event Hubs, CosmosDB) from GitHub Codespaces.
• K8s Networking: Design and maintain the solution for secure resource access, specifically utilizing Kubernetes port forwarding to connect through an HAProxy pod that acts as a secure intermediary to cloud resources.
• Networking Expertise: Deep hands-on experience configuring and troubleshooting HAProxy for internal routing and security within the K8s ecosystem.

Data Quality, Compliance, and Automation

• GDPR Compliance: Act as a technical expert in handling customer data access and deletion requests, ensuring the platform and pipelines meet all GDPR requirements.
• Operational Automation: Develop and deploy automated operational tasks via GitHub Actions, including:
  • Implementing processes for automated TLS certificate rotation across services.
  • Designing and deploying logic for the automatic scale-in/scale-out of K8s resources based on time-of-day usage (e.g., scaling down components only needed during business hours).
• Data Quality Focus: Actively support and develop the underlying data processing platform to continuously improve the data quality of charging stations.

Mentorship and Technical Training

• Conduct short, focused technical training sessions (KATAs) for engineers on key tooling and methodologies, including the use of DBT (Data Build Tool) and advanced HAProxy configurations.

Required Technical Qualifications

• Kubernetes (K8s) & Networking: Expert knowledge of K8s operations, service meshes, and specific experience with internal network routing solutions like HAProxy.
• Developer Tooling: Strong familiarity with GitHub Codespaces and leveraging K8s for local development and debugging scenarios.
• CI/CD & Automation: Proficiency in developing complex automation workflows using GitHub Actions and scripting (e.g., Bash/Python).
• Compliance: Practical experience handling data privacy and security requirements, especially GDPR (data access and deletion procedures).
• Data Tooling: Familiarity with modern data stack tools, particularly DBT, and its role in data transformation and quality.

Education and Experience

• 5+ years of experience in a DevOps, Site Reliability, or Data Platform engineering role.
• Proven ability to bridge the gap between development efficiency, operational reliability, and data compliance.