Medior AWS & Databricks Platform EngineerDatamole

Key Responsibilities

• Design, develop, and maintain data processing pipelines using Apache Spark, Delta Lake, and related technologies within the Databricks ecosystem, ensuring scalability, reliability, and operational excellence.
• Design and model structured and unstructured datasets, including implementing efficient table layouts and partitioning strategies to support performant, scalable, and maintainable data workflows.
• Implement and automate cloud infrastructure using Infrastructure as Code (Terraform, AWS CDK), supporting repeatable and consistent environment provisioning.
• Contribute to the architecture and evolution of a next-generation data platform, applying industry best practices such as the AWS Well-Architected Framework.
• Collaborate closely with data engineers, analysts, and ML practitioners to deliver platform capabilities that support analytical and machine learning workloads. 
• Develop and maintain CI/CD pipelines using GitHub Actions, incorporating automated testing, linting, and artifact management.
• Enhance platform observability, including metrics, logging, alerts, and tracing, ensuring issues are identified and resolved proactively.
• Document platform architecture, operational processes, and standards, enabling alignment within the engineering team and supporting ongoing maintenance.
• Participate in code reviews and provide guidance to colleagues, helping foster a collaborative, supportive, and growth-oriented engineering culture.

‍