Spark Developer

If you are a self-motivated individual with a passion for big data and have a track record of delivering high-quality solutions, we want to hear from you

Position: Spark Developer

Location: San Francisco, California, United States

Salary: $171000 - $208000

Job Summary: We are seeking a highly skilled Spark Developer to join our dynamic team. As a Spark Developer, you will be responsible for designing, developing, and implementing complex data processing pipelines using Apache Spark. You will work closely with cross-functional teams to understand business needs and translate them into scalable and efficient solutions. The ideal candidate is a quick learner, detail-oriented, and has a strong background in big data technologies.

Key Responsibilities:

• Design, develop and implement data processing pipelines using Apache Spark
• Collaborate with data engineers, data scientists, and business analysts to understand data requirements and translate them into technical solutions
• Optimize and tune Spark jobs to improve performance and scalability
• Monitor and troubleshoot Spark jobs to ensure data accuracy and consistency
• Develop and maintain ETL processes to extract, transform and load data from various sources
• Create and maintain technical documentation for developed solutions
• Stay up-to-date with the latest trends and advancements in big data technologies
• Participate in code reviews and provide constructive feedback to team members
• Mentor and guide junior developers in the team

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Systems, or related field
• Minimum of 3 years of experience in developing data processing pipelines using Apache Spark
• Strong proficiency in Scala or Python
• Solid understanding of big data concepts and technologies such as Hadoop, Hive, and Kafka
• Experience with data warehousing and ETL processes
• Proven experience in performance tuning and optimization of Spark jobs
• Familiarity with cloud platforms such as AWS or Azure is a plus
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills
• Ability to work in a fast-paced and dynamic environment

Join our team of talented developers and be a part of our exciting journey towards building innovative data-driven solutions. Apply now and take your career to new heights as a Spark Developer!