Principal Data Pipeline Engineer

We offer a competitive salary, comprehensive benefits package, and a dynamic work environment where your contributions will be valued and recognized

Job Title: Principal Data Pipeline Engineer

Location: San Francisco, California, United States

Salary: $320000 - $360000

Position Overview: We are seeking a highly skilled and experienced Principal Data Pipeline Engineer to join our dynamic team. As the Principal Data Pipeline Engineer, you will be responsible for designing, building, and maintaining our data pipelines to ensure the smooth and efficient flow of data through our systems. Your expertise in data engineering and pipeline development will be crucial in driving our data-driven business decisions and improving our overall data infrastructure.

Key Responsibilities:

• Design, develop, and maintain scalable and reliable data pipelines to support the collection, storage, and processing of large volumes of data
• Collaborate with cross-functional teams to understand data requirements and design solutions that meet business needs
• Optimize data pipelines for performance and scalability, ensuring timely delivery of data to downstream systems
• Troubleshoot and resolve issues related to data quality, consistency, and integrity
• Monitor data pipelines and implement proactive measures to ensure data accuracy and availability
• Ensure compliance with data governance and security policies and procedures
• Keep up-to-date with the latest data engineering technologies and best practices, and make recommendations for improvement
• Mentor and provide technical guidance to junior data engineers on best practices, coding standards, and development methodologies

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• Minimum of 8 years of experience in data engineering, with at least 3 years in a leadership role
• Strong experience in designing and developing data pipelines using modern data processing frameworks such as Spark, Hadoop, or Kafka
• Expertise in SQL and database technologies such as MySQL, Oracle, or MongoDB
• Proficiency in one or more programming languages, such as Python, Java, or Scala
• Hands-on experience with cloud platforms such as AWS, Azure, or Google Cloud
• Experience with data modeling, ETL development, and data warehousing concepts
• Proven track record of delivering complex data engineering projects on time and within budget
• Excellent problem-solving and analytical skills, with a strong attention to detail
• Strong communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams

If you are a highly motivated and driven data engineering professional looking for an exciting opportunity to make a significant impact, we would love to hear from you. Apply now and join our team of talented individuals working towards a common goal of driving data innovation and business success.