Spark Engineer

Join our team and be a part of a fast-paced, innovative, and collaborative work environment

Position: Spark Engineer

Location: San Francisco, California, United States

Salary: $190000 - $224000

Overview: We are seeking a highly skilled Spark Engineer to join our dynamic and growing team. As a Spark Engineer, you will be responsible for designing, implementing, and maintaining our company's Spark infrastructure. You will also be involved in developing and optimizing Spark applications for large-scale data processing and analysis. This is an exciting opportunity for someone who is passionate about working with cutting-edge technology and is looking to make a significant impact in the field of big data.

Key Responsibilities:

• Design and implement scalable and efficient Spark infrastructure for data processing and analysis
• Develop and optimize Spark applications for batch and streaming data processing
• Troubleshoot and debug any issues related to Spark infrastructure and applications
• Collaborate with cross-functional teams to understand business requirements and design solutions that meet those requirements
• Monitor and analyze performance metrics of Spark jobs and make necessary adjustments to improve efficiency and scalability
• Stay updated with the latest advancements in Spark technology and propose new ideas for enhancing our company's data processing capabilities
• Actively participate in code reviews and provide constructive feedback to team members
• Mentor and guide junior members of the team in Spark best practices and coding standards

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• Minimum of 3 years of experience working with Spark and Hadoop ecosystem
• Strong programming skills in Java, Scala, or Python
• Experience with big data tools such as HDFS, Hive, and Kafka
• Experience with cloud platforms such as AWS, Azure, or Google Cloud
• Familiarity with data warehousing concepts and tools
• Excellent problem-solving and analytical skills
• Ability to work independently as well as in a team environment
• Strong communication and interpersonal skills

Preferred Qualifications:

• Experience with machine learning and data mining techniques
• Knowledge of SQL and database systems
• Familiarity with Agile methodologies
• Certifications in Spark or Hadoop

We offer competitive salary and benefits packages, as well as opportunities for career growth and development. If you are passionate about big data and have a strong desire to learn and contribute, apply now for the position of Spark Engineer and take the next step in your career!