If you are passionate about Big Data and have a strong track record of building and maintaining robust data pipelines, we would love to hear from you
Title: Big Data Backend Engineer
Location: New York, New York, United States
Salary: $138 Per hour
Job Description: We are looking for a highly skilled Big Data Backend Engineer to join our dynamic and innovative team. As a Big Data Backend Engineer, you will play a crucial role in designing, implementing, and maintaining our company's Big Data infrastructure. Your primary responsibility will be to develop and optimize data pipelines, ensuring the smooth and efficient flow of data from various sources into our data warehouse.
Key Responsibilities:
- Design, develop, and maintain data pipelines to process and store large volumes of structured and unstructured data.
- Build and maintain ETL processes to ingest data from various sources into our data warehouse.
- Develop and optimize data storage solutions, including data lakes and data warehouses, to support our growing data needs.
- Collaborate with cross-functional teams to understand data requirements and implement solutions that meet business needs.
- Troubleshoot and resolve issues related to data quality, data consistency, and data integrity.
- Monitor and optimize the performance of data pipelines to ensure timely and accurate delivery of data.
- Design and implement data security and access control measures to protect sensitive data.
- Stay updated with the latest Big Data technologies and tools and make recommendations for their integration into our existing data infrastructure.
- Provide technical guidance and mentorship to junior engineers on the team.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Minimum of 5 years of experience in building and maintaining large-scale Big Data solutions.
- Strong experience with Big Data technologies such as Hadoop, Spark, Kafka, and Hive.
- Proficient in programming languages such as Java, Python, and Scala.
- Experience with cloud platforms such as AWS, Azure, or Google Cloud.
- Hands-on experience with SQL and NoSQL databases.
- Knowledge of data modeling and data warehousing concepts.
- Familiarity with Agile development methodologies.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
Join our team and be a part of an exciting journey towards transforming data into valuable insights.