Spark Engineer

If you are a self-motivated and results-driven individual with a passion for big data and a desire to grow your skills, we would love to hear from you

Job Title: Spark Engineer

Location: New York, New York, United States

Salary: $107 Per hour

Position Overview: We are seeking a highly skilled and experienced Spark Engineer to join our dynamic team. As a Spark Engineer, you will be responsible for developing and maintaining our Spark-based data processing pipelines. You will play a critical role in ensuring the scalability, reliability, and efficiency of our data infrastructure. This is a great opportunity for someone who is passionate about big data and eager to make an impact in a fast-paced and innovative environment.

Key Responsibilities:

• Design, develop, and maintain Spark-based data processing pipelines.
• Optimize and tune Spark jobs for performance and scalability.
• Troubleshoot and debug production issues related to Spark jobs.
• Collaborate with cross-functional teams to gather requirements and deliver solutions.
• Evaluate and implement new technologies to enhance our data infrastructure.
• Monitor and analyze data pipeline performance and provide recommendations for improvement.
• Ensure data quality and integrity by implementing data governance best practices.
• Document and communicate technical solutions and processes to both technical and non-technical stakeholders.
• Stay up-to-date with the latest developments in Spark and big data technologies.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Minimum of 3 years of experience in a similar role, preferably in a data-intensive environment.
• Extensive hands-on experience with Spark, including Spark SQL, Spark Streaming, and Spark MLlib.
• Strong understanding of distributed computing principles and algorithms.
• Proficient in programming languages such as Scala, Java, or Python.
• Experience with data warehousing and ETL processes.
• Familiarity with cloud platforms such as AWS or Azure.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.

Why Join Us:

• Opportunity to work with cutting-edge technologies and make an impact in a rapidly growing company.
• Collaborative and inclusive work culture that values diversity and encourages innovation.
• Competitive salary and benefits package.
• Professional development and training opportunities.
• Flexible work arrangements.

Apply now and join our team of talented and dedicated Spark Engineers.