Key Responsibilities:

Design, develop, and deploy Pyspark applications and tools to support our data processing needs.
Collaborate with cross-functional teams to understand the business requirements and translate them into technical solutions.
Build and maintain efficient and scalable data pipelines to process large volumes of data from various sources.
Perform data analysis and identify opportunities for performance improvements and optimizations.
Troubleshoot and debug issues with data pipelines, data quality, and performance.
Develop and implement best practices for Pyspark coding, testing, and deployment.
Conduct code reviews to ensure code quality and adherence to coding standards.
Keep up-to-date with the latest Pyspark and big data technologies and trends.
Mentor and coach junior team members to help them grow their Pyspark skills.

Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
3+ years of experience working as a Pyspark Developer or in a similar role.
Strong experience with Pyspark and the Hadoop ecosystem (HDFS, Hive, SparkSQL, etc.).
Proficiency in programming languages such as Python, Java, or Scala.
Experience with data processing and ETL tools such as Apache Airflow, Apache NiFi, etc.
In-depth understanding of data structures, algorithms, and data modeling concepts.
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.
Ability to work independently and in a team environment.

As a Pyspark Developer, you will have the opportunity to work on challenging projects and make a significant impact on our data infrastructure. If you are passionate about big data and have a strong track record of delivering high-quality Pyspark solutions, we would love to hear from you!

Hashtags