If you are a self-motivated, analytical thinker with a passion for data processing and a desire to work with cutting-edge technologies, then this is the perfect opportunity for you
Job Title: Spark Engineer
Location: New York, New York, United States
Salary: $107 Per hour
Position Overview: We are seeking a highly skilled and experienced Spark Engineer to join our team. As a Spark Engineer, you will be responsible for developing, implementing, and maintaining Spark-based solutions that support our company's data processing needs. You will work closely with our data scientists, data analysts, and software engineers to design and optimize complex data pipelines and systems.
Key Responsibilities:
- Design, develop, and maintain Spark-based data processing solutions to support our company's data needs.
- Collaborate with cross-functional teams to define and implement data processing requirements.
- Optimize and tune Spark applications to ensure high performance and scalability.
- Troubleshoot and resolve performance issues and system failures.
- Monitor and analyze data processing jobs to identify areas for improvement and implement solutions.
- Develop and maintain documentation for data processing workflows and systems.
- Stay updated on industry trends and best practices related to Spark and big data processing.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
- Minimum of 3 years of experience working with Spark and big data processing.
- Strong proficiency in Scala or Java programming languages.
- Experience with building and optimizing data pipelines using Spark and related tools such as Spark SQL, Spark Streaming, and Spark MLlib.
- Knowledge of distributed computing concepts and frameworks, such as Hadoop and MapReduce.
- Familiarity with cloud computing platforms, such as AWS or Azure.
- Experience with data warehousing and data modeling.
- Excellent problem-solving and troubleshooting skills.
- Strong communication and collaboration skills.
- Ability to work in a fast-paced and dynamic environment.
- Experience with Agile methodologies is a plus.
Join our team as a Spark Engineer and help us drive our company's data-driven initiatives to new heights. Apply now!