Key responsibilities:

Design, develop and maintain data pipelines using Pyspark
Collaborate with data scientists and data analysts to understand their data needs and translate them into technical requirements
Optimize and tune data pipelines for performance and scalability
Troubleshoot and resolve data-related issues in a timely manner
Ensure data quality and integrity through data validation and testing
Develop and maintain technical documentation for data pipelines and data infrastructure
Keep up-to-date with new technologies and industry trends, and make recommendations for their implementation in our data processes
Participate in code reviews and contribute to continuous improvement of our data processes and coding standards
Mentor and provide guidance to junior developers in the team

Qualifications:

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field
Minimum of 3 years experience as a Pyspark Developer or a similar role
Strong knowledge of Pyspark and the Hadoop ecosystem
Experience with data warehousing and data modeling concepts
Proficient in SQL and experience with database management systems such as MySQL, PostgreSQL, or Oracle
Experience with cloud platforms such as AWS or Azure
Excellent problem-solving and analytical skills
Strong communication and collaboration skills
Ability to work in a fast-paced and constantly evolving environment
Experience with Agile methodologies is a plus

Join our team and be a part of our exciting journey to transform data into valuable insights.

Hashtags