Key Responsibilities:

Design, develop, and maintain data pipelines, data lakes, and data warehouses using Azure Data Bricks
Create and manage data ingestion processes from various data sources
Develop and implement data quality and data validation processes
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
Work closely with data scientists to deploy machine learning models in production
Optimize and monitor data pipelines to ensure high performance and availability
Troubleshoot and resolve data-related issues in a timely manner
Stay updated with the latest trends and advancements in Azure Data Bricks and data engineering to continuously improve data solutions

Qualifications:

Bachelor's or Master's degree in Computer Science, Information Systems, or a related field
Minimum of 5 years of experience as a Data Engineer or Data Developer
Strong experience with Azure Data Bricks, Azure Data Factory, and Azure Data Lake
Proficient in SQL and Python for data manipulation and analysis
Experience with data warehousing and data modeling concepts
Knowledge of ETL processes and tools
Familiarity with data governance and data security best practices
Excellent problem-solving and analytical skills
Strong attention to detail and ability to work with large datasets
Excellent communication and collaboration skills
Experience with Agile methodologies is preferred

Hashtags