Data Pipeline Specialist

If you are a data-driven individual with a passion for building robust and efficient data pipelines, we would love to hear from you

Title: Data Pipeline Specialist

Location: New York, New York, United States

Salary: $92 Per hour

Job Description: As a Data Pipeline Specialist, you will be responsible for designing, implementing, and maintaining efficient and scalable data pipelines for our organization. You will work closely with our data engineering and data science teams to ensure the smooth and timely flow of data from various sources to our data warehouse and other data systems.

Key Responsibilities:

• Design and develop end-to-end data pipelines to support the collection, processing, and storage of large volumes of data from various sources.
• Collaborate with data engineering and data science teams to understand data requirements, identify data sources, and implement optimal solutions.
• Build and maintain data pipelines that are robust, fault-tolerant, and scalable to handle large and complex datasets.
• Ensure data quality and integrity by implementing data validation and monitoring processes in the pipelines.
• Troubleshoot and resolve issues with data pipelines, ensuring minimal downtime and maximum efficiency.
• Monitor pipeline performance and identify opportunities for optimization and automation.
• Develop and maintain documentation for data pipelines, including design, processes, and data flow diagrams.
• Stay updated with the latest trends and technologies in data engineering and data warehousing to improve our data pipeline infrastructure.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
• Proven experience (2+ years) in designing, building, and maintaining data pipelines in a production environment.
• Strong knowledge of SQL and experience working with relational databases such as MySQL, PostgreSQL, or Oracle.
• Hands-on experience with ETL tools such as Informatica, Talend, or SSIS.
• Proficient in at least one scripting language (Python, Perl, Shell) for data manipulation and automation.
• Familiarity with cloud-based data platforms (AWS, Azure, Google Cloud) and their data services (S3, Redshift, BigQuery).
• Experience with data modeling, data warehousing, and data architecture concepts.
• Strong analytical and problem-solving skills with a keen attention to detail.
• Excellent communication and collaboration skills to work effectively with cross-functional teams.

Why Work With Us:

• Opportunity to work with cutting-edge technologies and complex data sets.
• Collaborative and diverse work environment.
• Competitive salary and benefits package.
• Professional development and growth opportunities.
• Flexible work arrangements and a healthy work-life balance.

Apply now and be a part of our dynamic team as a Data Pipeline Specialist.