Key Responsibilities:

Design and implement large-scale data processing systems using Hadoop, Spark, and other big data technologies
Develop and maintain data pipelines to ingest, process, and store large volumes of data
Work closely with data scientists and analysts to understand their data needs and provide relevant solutions
Optimize and troubleshoot data pipelines to ensure smooth and efficient processing
Collaborate with cross-functional teams to define data requirements and ensure data consistency and accuracy
Design and implement data security and data governance policies to ensure data integrity and compliance
Monitor and maintain the health of the big data infrastructure, including performance tuning and capacity planning
Stay up-to-date with emerging big data technologies and make recommendations for their implementation
Document and communicate technical solutions to both technical and non-technical stakeholders

Requirements:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field
Minimum of 3 years of experience as a Big Data Engineer or a similar role
Strong programming skills in languages such as Java, Python, or Scala
In-depth knowledge of big data technologies, including Hadoop, Spark, Kafka, and NoSQL databases
Experience with cloud-based big data solutions, such as AWS or Azure
Familiarity with data warehouse concepts and tools, such as Snowflake or Redshift
Proven track record of designing and implementing large-scale data processing systems
Experience with data modeling, data integration, and data warehousing
Strong analytical and problem-solving skills
Excellent communication and collaboration skills
Ability to work in a fast-paced and dynamic environment
Passion for learning and staying up-to-date with the latest technologies and trends

Hashtags