Pyspark Developer

If you are a self-motivated and passionate Pyspark Developer looking for an exciting new challenge, we would love to hear from you

Job Title: Pyspark Developer

Location: New York, New York, United States

Salary: $100 Per hour

About the Role: We are seeking a highly skilled and experienced Pyspark Developer to join our dynamic team. As a Pyspark Developer, you will be responsible for designing, developing, and maintaining data processing pipelines using Pyspark. This is a great opportunity for someone with a strong background in data engineering and a passion for working with big data.

Key Responsibilities:

• Design and develop data processing pipelines using Pyspark on large datasets
• Collaborate with data scientists and other developers to understand business requirements and translate them into technical solutions
• Build and optimize data pipelines for data ingestion, transformation, and storage
• Troubleshoot and debug issues in data pipelines and provide timely resolutions
• Write efficient and scalable code to process and analyze large datasets
• Develop and maintain ETL processes to ensure data quality and consistency
• Monitor and improve the performance and scalability of data pipelines
• Stay updated with the latest advancements in Pyspark and big data technologies and implement them in our data infrastructure
• Work closely with cross-functional teams to deliver high-quality data solutions within strict deadlines
• Document data pipelines, processes, and best practices for future reference

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• Minimum of 3 years of experience in designing and developing data processing pipelines using Pyspark
• Strong understanding of big data concepts and experience working with large datasets
• Proficiency in Python programming language and hands-on experience with Pyspark
• Knowledge of SQL and experience working with relational databases
• Familiarity with cloud platforms such as AWS, Azure, or GCP
• Experience with data warehousing and ETL processes
• Excellent problem-solving and analytical skills
• Strong communication and collaboration skills
• Ability to thrive in a fast-paced and dynamic work environment

Why Work With Us:

• We offer a competitive salary and benefits package
• Opportunity to work with cutting-edge technologies and solve complex problems
• Collaborative and inclusive work culture that values diversity and innovation
• Plenty of opportunities for career growth and development
• Flexible working hours and remote work options

Apply now and join our team of talented and driven individuals!