[Remote] Principal Data Platform Engineer (Databricks)

Note: The job is a remote job and is open to candidates in USA. Prominence Advisors is a healthcare and life sciences technology and consulting firm dedicated to helping organizations leverage their data. They are seeking a Principal Data Platform Engineer to lead the architecture and delivery of modern data platforms, focusing on client engagement and mentoring team members.

Responsibilities

• Own the end-to-end architecture and delivery of scalable data solutions, with a strong emphasis on Databricks-based platforms and modern cloud ecosystems
• Lead design and implementation of data pipelines, data models, and transformation frameworks that support analytics, reporting, and advanced use cases
• Serve as a primary client-facing technical leader, building trusted relationships and guiding stakeholders through complex data challenges and solution decisions
• Translate ambiguous business requirements into clear, actionable technical architectures and delivery plans
• Establish and enforce best practices across data engineering, including g data ingestion (pipeline orchestration, testing, and optimization) and DevOps
• Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks
• Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support to ensure successful project outcomes
• Collaborate with cross-functional teams including data architects, analysts, and client stakeholders to ensure alignment and value delivery
• Identify risks, proactively address challenges, and ensure high-quality, timely delivery across engagements
• Contribute to internal capability building, including reusable frameworks, accelerators, and thought leadership
• Support business development efforts by shaping technical solutions, contributing to proposals, and participating in client discussions

Skills

• 7+ years of experience in data engineering, with demonstrated progression into technical leadership and architecture ownership
• Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing
• Advanced proficiency in SQL and Python, with strong experience building and optimizing large-scale data pipelines
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including data services and infrastructure design
• Strong understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing
• Experience with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt)
• Proven ability to lead technical delivery while remaining hands-on when needed
• Strong client-facing experience, including requirements gathering, solution design, and executive communication
• Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments
• Healthcare data experience (e.g., Epic, HL7, FHIR, claims data) is strongly preferred
• Experience with CI/CD, DevOps practices, and infrastructure-as-code tools is a plus

Benefits

• Competitive Salaried and Hybrid Compensation Plans
• Health Care Plan (Medical, HSAs, Dental & Vision)
• Retirement Plan (401k)
• Life Insurance (Basic, Voluntary & AD&D)
• Dependent & Health Savings Accounts
• Short Term & Long Term Disability
• Paid Time Off (Vacation/Sick & Public Holidays)
• Training & Development Fund
• Work From Home
• Charitable Giving to Causes You Believe In

Company Overview

• Prominence Advisors is a team of healthcare experts with extensive knowledge in healthcare analytics, data warehousing, data visualization. It was founded in 2013, and is headquartered in Lincolnshire, Illinois, USA, with a workforce of 51-200 employees. Its website is http://www.prominenceadvisors.com.