Key Responsibilities:

Design, develop, and maintain data processing solutions using Apache Spark
Write efficient and optimized Spark code to process large datasets
Collaborate with data scientists and data engineers to understand data requirements and implement solutions accordingly
Troubleshoot and resolve issues related to Spark performance, data quality, and data integration
Optimize Spark jobs for performance and scalability
Develop and maintain data pipelines using tools such as Apache Airflow
Work with cross-functional teams to understand business requirements and translate them into technical solutions
Stay up-to-date with the latest trends and advancements in Spark technology and data processing techniques
Provide technical guidance and mentorship to junior team members
Document processes, procedures, and solutions to ensure knowledge sharing and transfer within the team

Qualifications:

Bachelor's degree in Computer Science, Engineering, or a related field
Minimum of 3 years of experience working with Apache Spark
Proficiency in programming languages such as Scala, Python, or Java
Strong understanding of big data technologies such as Hadoop, Hive, and Kafka
Experience with data pipeline tools such as Apache Airflow or Apache NiFi
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform
Hands-on experience with SQL and NoSQL databases
Excellent problem-solving and troubleshooting skills
Strong communication and teamwork skills
Ability to work in a fast-paced and dynamic environment
Passion for learning and staying updated with the latest technologies and techniques in the data processing industry

Join our team of talented professionals and be a part of our exciting journey towards innovation and success.

Hashtags