Job Description

The primary accountabilities of the Sr Data Engineer are the creation and maintenance of data pipelines and analytics. Secondary accountabilities include cloud platform administration and consulting with business stakeholders.

Responsibilities:

• Create data pipelines sourcing data from clinical and operational systems that feed RapidAI's analytic data repository.
• Assist in the creation of data architecture elements to enhance & amp; extend RapidAI's data storage, processing, and visualization capabilities.
• Collaborate with QA engineering and other teams to ensure compliance with respect to FDA and other regulatory body requirements.
• Leverage git to manage code across the development lifecycle inclusive of pull requests, merges, commits, and reviews.
• Establish CI/CD pipelines to deploy code leveraging modern CI/CD tools and methodologies.
• Collaborate with RapidAI's DevOps team to develop Infrastructure as Code deliverables.
• Automate testing through the creation of Unit and Integration test routines.
• Assist in the management and administration of public cloud infrastructure inclusive of componentry hosted by AWS, Google / Looker, and Snowflake.
• Build data visualization systems to display complex quantitative data in straightforward and easy-to-understand dashboards and analytics (see Stephen Few & amp; Edward Tufte).
• Provide 3rd-level support to resolve customer-facing issues in collaboration with Professional Services and Engineering Support teams.
• Establish data wrangling algorithms, routines, and DAGs to process incoming datasets through the creation of automated and change-resilient data pipelines.
• Author technical documentation
• This position does not have direct reports.
• Perform any other related activities assigned by reporting manager.

Requirements:

• Bachelor's Degree (Graduate Degree preferred) related to computer science or informatics in life science; other STEM disciplines will be considered.
• 8+ years of experience in analytics and data engineering, preferably in a start-up environment.
• Exceptional communication skills relevant to engaging a globally distributed and culturally, geographically, & amp; time zone-diverse team.
• Expertise in developing data pipelines using Python-based DAG environments such as Flyte and Airflow.
• Exposure to developing data pipelines using Rust crates inclusive of Polars, DataFusion, and Ballista.
• Expertise in SQL as applied to the creation and retrieval of analytic (OLAP) data sets.
• Expertise with cloud-based database systems such as Snowflake, Redshift, DynamoDB, and similar.
• Expertise in relational & amp; dimensional data modeling.
• Experiencing developing data lake-based analytic solutions leveraging AWS.
• Expertise in authoring analytic dashboards using Looker or similar (Tableau, Sisense, etc. ).
• Familiarity with Jenkins, GitHub Actions, CircleCI, TravisCI, or other CI/CD platforms.
• Familiarity using Terraform (preferred), Pulumi, and/or AWS CloudFormation.
• Understanding of Agile software development practices including common Scrum/Kanban SDLC management platforms such as Jira, Asana, YouTrack, or similar.
• Exposure to Spark, Dask and other distributed analysis & amp; parallel processing platforms.
• Experience with containerization technologies such as Kubernetes.
• Experience with data quality tools such as Great Expectations, Soda, PyDeequ, or Griffin
• Experience working with on-premises (Linux preferred) and public cloud (AWS preferred) compute environments.
• Familiarity with EMR and PACS systems data lifecycle (data capture, data analysis, etc) is a bonus.
• Demonstrated ability to prioritize and deliver on multiple responsibilities simultaneously.
• Understanding of HIPAA, GDPR, and other regulatory frameworks guiding the use of personally identifiable datasets.