Pyspark Developer

If you are a highly motivated and skilled Pyspark Developer looking for an exciting opportunity to work on challenging projects, we encourage you to apply for this role

Job Title: Pyspark Developer

Location: San Francisco, California, United States

Salary: $171000 - $208000

Position Overview: We are seeking a highly skilled and experienced Pyspark Developer to join our dynamic team. As a Pyspark Developer, you will be responsible for designing, developing, and maintaining data processing pipelines using Apache Spark and Pyspark. You will work closely with our data scientists and data engineers to build robust and scalable solutions for our big data processing needs.

Key Responsibilities:

• Design, develop, and maintain data processing pipelines using Pyspark and Apache Spark
• Collaborate with data scientists and data engineers to understand data requirements and translate them into technical solutions
• Optimize and tune data processing pipelines for performance and scalability
• Troubleshoot and resolve issues related to data processing and data quality
• Develop and maintain ETL processes and data integration solutions
• Write efficient and high-quality code following industry best practices
• Perform code reviews and provide constructive feedback to team members
• Stay up-to-date with the latest advancements in Pyspark and Apache Spark technologies and incorporate them into our data processing pipelines

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field
• Minimum of 3 years of experience working as a Pyspark Developer
• Strong understanding of Apache Spark and Pyspark internals and architecture
• Experience with data processing and data transformation using Pyspark
• Proficiency in SQL and experience with relational databases
• Strong programming skills in Python and experience with object-oriented programming
• Experience with big data technologies such as Hadoop, Hive, or Kafka is a plus
• Familiarity with cloud technologies and experience with AWS or Azure is a plus
• Excellent problem-solving and analytical skills
• Ability to work independently and in a team environment
• Excellent communication and collaboration skills

Why Join Us:

• Opportunity to work with cutting-edge technologies and solve complex data processing challenges
• Collaborative and innovative work environment
• Competitive salary and benefits package
• Opportunities for professional growth and development
• Flexible work schedule and remote work options

Join our team and be a part of our mission to make a meaningful impact in the world of big data.