Pyspark Developer

We offer a competitive salary and benefits package, as well as opportunities for growth and professional development

Job Title: Pyspark Developer

Location: New York, New York, United States

Salary: $100 Per hour

Position Overview: We are seeking a highly skilled Pyspark Developer to join our dynamic team. As a Pyspark Developer, you will be responsible for designing, developing, and implementing complex data processing pipelines using Pyspark. You will also be responsible for troubleshooting and optimizing existing code to improve performance and scalability. The ideal candidate will have a strong background in data engineering and a passion for working with big data.

Key Responsibilities:

• Design, develop, and maintain data processing pipelines using Pyspark
• Collaborate with data scientists and analysts to understand business requirements and translate them into technical solutions
• Optimize existing code to improve performance and scalability
• Troubleshoot and resolve any issues related to data processing pipelines
• Develop and maintain technical documentation
• Stay current with industry trends and best practices in big data processing and analytics

Requirements:

• Bachelor's degree in Computer Science, Engineering, or a related field
• 2+ years of experience developing data processing pipelines using Pyspark
• Strong knowledge of Python and SQL
• Experience with Hadoop, Spark, and other big data technologies
• Familiarity with cloud platforms such as AWS or Azure
• Experience with data warehousing and data modeling
• Excellent problem-solving and analytical skills
• Strong communication and collaboration skills
• Ability to work independently and in a team environment
• Detail-oriented with a focus on delivering high-quality work

Preferred Qualifications:

• Master's degree in Computer Science, Engineering, or a related field
• Experience with machine learning and data science tools such as TensorFlow, PyTorch, or Scikit-learn
• Knowledge of Agile methodologies
• Experience with data visualization tools such as Tableau or Power BI
• Experience with ETL tools such as Informatica or Talend

If you are a passionate Pyspark Developer with a strong background in data engineering and a desire to work with cutting-edge technologies, we encourage you to apply for this exciting opportunity. Join our team and be a part of shaping the future of big data processing and analytics.