Key Responsibilities:

Design, develop, and maintain highly scalable and reliable data processing pipelines using Apache Spark.
Collaborate with data scientists and data engineers to understand data requirements and build data solutions that meet business needs.
Optimize and tune existing Spark jobs for performance and efficiency.
Troubleshoot and debug issues in the Spark environment and provide timely resolutions.
Monitor and maintain the health and stability of the Spark cluster.
Stay up-to-date with the latest advancements in Spark and recommend new technologies and tools to improve our data processing pipeline.
Develop and maintain technical documentation for Spark jobs and processes.
Work closely with cross-functional teams to ensure seamless integration of Spark with other systems and applications.

Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Minimum of 3 years of hands-on experience working with Spark in a production environment.
Strong programming skills in Scala or Java.
Extensive knowledge of Apache Spark architecture and internals.
Experience with SQL, NoSQL databases, and data warehousing concepts.
Familiarity with data engineering tools and technologies such as Hadoop, Hive, Kafka, and HDFS.
Proven track record of designing and implementing scalable and performant data processing pipelines.
Excellent analytical and problem-solving skills.
Ability to work independently and in a team environment.
Strong communication and interpersonal skills.

Dental insurance
Competitive salary and benefits package.
Opportunity to work with cutting-edge technologies and solve complex data challenges.
Dynamic and collaborative work environment.
Professional development and growth opportunities.

Hashtags