Pyspark Developer

If you are a self-motivated and passionate Pyspark Developer looking to join a fast-growing company, we would love to hear from you

Job Title: Pyspark Developer

Location: San Francisco, California, United States

Salary: $171000 - $208000

Job Summary: We are seeking a highly skilled and experienced Pyspark Developer to join our dynamic team. As a Pyspark Developer, you will be responsible for designing, developing, and maintaining data processing pipelines using Pyspark. You will work closely with the data engineering team to build efficient and scalable solutions for data ingestion, transformation, and analysis. The ideal candidate will have a strong background in Python and experience working with big data technologies.

Key Responsibilities:

• Design and develop data processing pipelines using Pyspark to support various data analytics and machine learning use cases.
• Collaborate with data engineers to identify and implement efficient data processing techniques.
• Write efficient and optimized code to handle large datasets and complex data transformations.
• Work with data scientists and analysts to understand business requirements and translate them into technical solutions.
• Troubleshoot and debug code to ensure data accuracy and reliability.
• Monitor and optimize the performance of data pipelines to meet SLAs.
• Maintain and enhance existing Pyspark codebase.
• Stay updated with the latest trends and advancements in Pyspark and big data technologies.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience working as a Pyspark Developer or in a similar role.
• Strong expertise in Python programming language.
• Hands-on experience working with big data technologies such as Hadoop, Spark, and Kafka.
• Experience with data warehousing and ETL tools.
• Proven track record of building and optimizing data pipelines for efficient data processing.
• Good understanding of data structures, algorithms, and data modeling.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Ability to work in a fast-paced and dynamic environment.

Preferred Qualifications:

• Master's degree in Computer Science, Information Technology, or a related field.
• Experience with cloud-based big data platforms such as AWS, GCP, or Azure.
• Knowledge of machine learning techniques and frameworks.
• Familiarity with Agile methodologies.
• Certifications in Pyspark or big data technologies.

We offer a competitive salary, benefits package, and a collaborative work environment. Apply now and be a part of our innovative team!