Lead Data Pipeline Engineer

If you are passionate about data and have a desire to make an impact through your work, we would love to hear from you

Job Title: Lead Data Pipeline Engineer

Location: San Francisco, California, United States

Salary: $247000 - $272000

Overview: We are seeking a highly motivated and experienced Lead Data Pipeline Engineer to join our dynamic team. As the Lead Data Pipeline Engineer, you will be responsible for developing and maintaining our data pipelines, ensuring efficient and reliable data flow between various systems. You will work closely with our data scientists, software engineers, and other cross-functional teams to design, build, and manage our data pipelines. Your expertise in data integration, ETL processes, and data warehousing will be crucial in driving our data-driven initiatives and delivering valuable insights to our business.

Key Responsibilities:

• Design, develop, and maintain robust, scalable, and efficient data pipelines to support our data-driven initiatives.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Evaluate and recommend appropriate tools and technologies for data integration, ETL processes, and data warehousing.
• Ensure data quality and integrity throughout the entire data pipeline.
• Troubleshoot and resolve any issues related to data pipelines, including data discrepancies, performance bottlenecks, and data quality issues.
• Monitor and optimize data pipeline performance to meet SLAs and business needs.
• Develop and maintain documentation for data pipelines, including data flow diagrams, data dictionaries, and data lineage.
• Mentor and guide junior members of the data engineering team.
• Stay updated with the latest trends and advancements in data engineering and proactively identify opportunities to improve data pipeline processes and technologies.

Qualifications:

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• Minimum of 5 years of experience in data engineering, data integration, ETL processes, and data warehousing.
• Strong proficiency in SQL and experience working with relational databases such as MySQL, PostgreSQL, etc.
• Hands-on experience with data integration tools such as Informatica, Talend, or Pentaho.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Experience with data warehouses such as Redshift, Snowflake, or BigQuery.
• Strong understanding of data modeling concepts and data warehouse design principles.
• Experience with data visualization tools such as Tableau, Power BI, or Looker is a plus.
• Excellent problem-solving skills and ability to troubleshoot and resolve complex data pipeline issues.
• Strong communication and interpersonal skills with the ability to collaborate effectively with cross-functional teams.
• Proven ability to lead and mentor a team of data engineers.

Join our team as the Lead Data Pipeline Engineer and be a part of our exciting journey towards becoming a data-driven organization.