Key Responsibilities:

Design and develop data processing systems using Spark and related technologies
Collaborate with data engineering and data science teams to understand business requirements and design efficient data pipelines
Implement and maintain data quality checks and data validation processes
Optimize data pipelines for performance, scalability, and reliability
Troubleshoot and resolve issues related to data processing and data quality
Stay updated with the latest technologies and industry trends to continuously improve data processing systems
Communicate effectively with cross-functional teams and stakeholders to gather requirements and provide regular updates on project progress
Document technical specifications and procedures for data processing systems

Qualifications:

Bachelor's degree in Computer Science, Engineering, or a related field
Minimum of 3 years of experience in data engineering, data warehousing, or a related role
Strong experience with Spark and related technologies such as Hadoop, Hive, and Kafka
Proficient in programming languages such as Scala, Java, or Python
Hands-on experience with SQL and database design
Knowledge of data modeling and ETL processes
Excellent problem-solving and analytical skills
Ability to work independently and in a team environment
Strong communication and interpersonal skills

Preferred Qualifications:

Master's degree in Computer Science, Engineering, or a related field
Experience with cloud platforms such as AWS, Azure, or Google Cloud
Familiarity with machine learning and data science concepts
Experience with agile development methodologies

If you are passionate about data processing and have a strong background in Spark and related technologies, we would love to hear from you. Apply now and join our innovative team as a Spark Developer.

Hashtags