Pyspark Developer

If you are a self-motivated and results-driven Pyspark Developer with a passion for working with big data, we want to hear from you

Job Title: Pyspark Developer

Location: New York, New York, United States

Salary: $100 Per hour

Job Overview: We are seeking a highly skilled Pyspark Developer to join our dynamic team. As a Pyspark Developer, you will be responsible for designing, developing, and implementing data processing workflows using Pyspark within our company's data ecosystem. You will collaborate with cross-functional teams to understand business requirements and translate them into efficient and scalable solutions. The ideal candidate should have a strong background in data engineering, excellent programming skills, and a passion for working with big data.

Key Responsibilities:

• Develop and maintain data processing workflows using Pyspark in a distributed environment.
• Collaborate with data scientists and analysts to understand business requirements and design efficient data solutions.
• Optimize and tune Pyspark jobs for performance and scalability.
• Monitor and troubleshoot data processing pipelines to ensure data accuracy and integrity.
• Develop and maintain data ingestion pipelines from various data sources.
• Work with DevOps teams to deploy and manage Pyspark applications in production environments.
• Continuously evaluate and incorporate new tools and technologies to improve data processing workflows.
• Document code, processes, and procedures for future reference and knowledge sharing.

Qualifications:

• Bachelor's or Master's degree in computer science, software engineering, or a related field.
• Proven experience working with Pyspark in a production environment.
• Strong proficiency in Python and SQL.
• Experience with big data technologies such as Hadoop, Spark, and Kafka.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Knowledge of data warehousing concepts and dimensional modeling.
• Strong understanding of software development principles and agile methodologies.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Ability to work in a fast-paced and dynamic environment.

Perks and Benefits:

• Competitive salary and benefits package.
• Opportunity to work with cutting-edge technologies and big data.
• Continuous learning and career development opportunities.
• Collaborative and inclusive work environment.
• Flexible work arrangements.
• Company-sponsored social events and team-building activities.

Join our team and make a significant impact on our company's data-driven decision-making processes. Apply now and be a part of our exciting journey towards success.