Senior Spark Developer

We are looking for a passionate and self-motivated individual who can thrive in a fast-paced and dynamic environment

Job Title: Senior Spark Developer

Location: New York, New York, United States

Salary: $107 Per hour

Position Overview: We are seeking a highly skilled Senior Spark Developer to join our dynamic team. As a Senior Spark Developer, you will be responsible for designing, developing, and implementing complex data processing solutions using Apache Spark. The ideal candidate will have a strong background in data engineering and experience working with large datasets. You will work closely with cross-functional teams to design and deliver scalable, reliable, and efficient data solutions.

Key Responsibilities:

• Design and implement data processing solutions using Apache Spark
• Develop and maintain ETL pipelines to process both structured and unstructured data
• Optimize and tune Spark applications for performance and scalability
• Troubleshoot and resolve any issues related to data processing and ETL pipelines
• Collaborate with data scientists and data analysts to understand their requirements and provide technical solutions
• Identify and implement best practices for data architecture, data modeling, and data quality
• Continuously evaluate and recommend new technologies, tools, and frameworks to improve data processing and analytics capabilities
• Mentor and guide junior team members in Spark development best practices
• Work closely with DevOps team to ensure smooth deployment and monitoring of Spark applications
• Keep up-to-date with industry trends and advancements in big data technologies

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field
• Minimum of 5 years of experience in data engineering and development using Apache Spark
• Proficient in programming languages such as Scala, Python, or Java
• Strong understanding of big data concepts and distributed computing principles
• Hands-on experience with Hadoop ecosystem and related technologies (HDFS, Hive, Impala, etc.)
• Experience with cloud-based data platforms such as AWS, Google Cloud, or Microsoft Azure
• Familiarity with relational and NoSQL databases
• Proven track record of delivering complex data solutions in a fast-paced environment
• Excellent problem-solving and analytical skills
• Strong communication and collaboration skills
• Experience in Agile development methodologies is a plus

Why Join Us:

• Opportunity to work with cutting-edge technologies and solve complex data problems
• Collaborative and inclusive work culture that values diversity and encourages innovation
• Competitive salary and benefits package
• Career growth and learning opportunities
• Flexible work arrangements and work-life balance

If you have a strong background in Spark development and are looking for a challenging and rewarding opportunity, we would love to hear from you. Apply now and be a part of our team of talented data professionals!