Job Description

We are seeking an experienced Senior Data Platform Engineer to join our team. The ideal candidate should have extensive experience with Pyspark, Airflow, Presto, Hive, Kafka, and Debezium, and should be passionate about developing scalable and reliable data platforms.

Responsibilities:

• Design, develop, and maintain our data platform architecture using Pyspark, Airflow, Presto, Hive, Kafka, and Debezium.
• Develop and maintain ETL processes to ingest, transform, and load data from various sources into our data platform.
• Work closely with data analysts, data scientists, and other stakeholders to understand their requirements and design solutions that meet their needs.
• Implement and maintain data governance policies and procedures to ensure data quality, privacy, and security.
• Continuously monitor and optimize the performance of our data platform to ensure scalability, reliability, and cost-effectiveness.
• Keep up-to-date with the latest trends and technologies in the field of data engineering and share knowledge and best practices with the team.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 5+ years of experience in data engineering or related fields.
• Strong proficiency in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium.
• Experience with data warehousing, data modeling, and data governance.
• Experience working with large-scale distributed systems and cloud platforms (e. g., AWS, GCP, Azure).
• Strong problem-solving skills and ability to work independently and collaboratively.
• Excellent communication and interpersonal skills.