Key Responsibilities:

Design and develop data pipelines using Pyspark for data ingestion, transformation, and processing.
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
Build and optimize data processing algorithms using Pyspark to support data analytics and reporting.
Troubleshoot and debug complex data pipeline issues and performance bottlenecks.
Continuously monitor and improve the performance, scalability, and reliability of Pyspark applications.
Develop and maintain technical documentation for Pyspark code and processes.
Stay updated with the latest advancements in Pyspark and big data technologies and identify opportunities for implementing them in our projects.

Qualifications:

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Minimum of 3 years of hands-on experience in developing data processing pipelines using Pyspark.
Strong proficiency in Python programming language and experience in working with big data technologies such as Hadoop, Spark, and Hive.
Experience in building ETL workflows and data pipelines using tools like Airflow or Luigi is a plus.
Knowledge of SQL and experience in working with relational databases is required.
Experience with cloud computing platforms like AWS, Azure, or Google Cloud is preferred.
Strong problem-solving and analytical skills with the ability to work in a fast-paced and dynamic environment.
Excellent communication and interpersonal skills to collaborate with cross-functional teams.
Ability to work independently and handle multiple projects simultaneously.

We offer a competitive salary and benefits package.
Work with cutting-edge technologies and a talented team of professionals.
Opportunities for career growth and continuous learning.
A collaborative and inclusive work environment that values diversity and promotes work-life balance.

Apply now and become a part of our innovative and dynamic team!

Hashtags