The project focuses on designing and delivering scalable cloud-based data engineering solutions on Microsoft Azure. The role will support large-scale data integration, analytics, machine learning, AI solutions, and business intelligence through modern data platforms and automated ETL/ELT pipelines.
Design, develop, and optimize ETL/ELT solutions using Azure Databricks, LakeBase, and Spark.
Build and deploy large-scale data pipelines supporting analytics, machine learning, predictive modeling, reporting, and data products.
Integrate structured and unstructured data from databases, APIs, FTPs, cloud storage, and various file formats.
Develop and optimize complex SQL queries and data models.
Build AI-based solutions for business needs, process automation, and operational efficiency.
Improve data reliability, integrity, performance, and overall platform quality.
Contribute to data architecture, engineering standards, frameworks, and best practices.
Provide technical guidance and mentorship to other data engineers.
Must have
7+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes.
5+ years of hands-on Python experience and 2+ years of experience with SQL, complex data schemas, and query performance optimization.
At least 2 years of hands-on experience with Redis-backed state management and event-driven processing, as well as building cloud solutions using Azure Databricks with Unity Catalog, Snowflake, Azure Functions, and Service Bus.
Practical experience with Terraform-based DevOps automation and CI/CD tools including GitHub Actions, Git, Artifactory, and Sonar is required.
Mandatory Skills (only names):
Azure Databricks
Apache Spark
Python
SQL
ETL/ELT
Data Engineering
Data Modeling
Redis
Snowflake
Azure Functions
Azure Service Bus
Terraform
GitHub Actions
Git
Artifactory
Sonar
Nice to have
Excellent communication skills
Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related discipline.
Strong understanding of Data Lake, Delta Lake, and Data Warehousing architectures, as well as advanced data modeling techniques including slowly changing dimensions, aggregation, partitioning, and indexing.
Exposure to LLM/AI solution delivery is preferred.
Ability to independently troubleshoot and performance-tune large-scale enterprise systems, collaborate effectively across technical and business teams, mentor engineers, and communicate technical concepts to both technical and non-technical audiences.
Languages
English: C1 Advanced
Seniority
Senior
...healthier communities, now and for generations to come, we want you to be a part of our team. Job Summary Memorial Hermann TMC Blood Bank seeks a Medical Technologist or MLT to join our team on Night shift. Schedule can be either 4-10s or 3-12s. Will discuss in...
...What: Were hiring flexible virtual assistants to support a range of business functions including admin, finance, HR, and operations. When... ...-based expectations. Salary: depending on skillset and experience. Position Overview: Were seeking adaptable and skilled...
...About the Position : Housekeeping/Custodial Workers have the integral role and basic function of the caring for and cleaning of Griffin Memorial Hospital. Positions in this job family are assigned routine work involving sweeping, mopping, dusting, disposing of trash,...
...in a dynamic, mission-focused environment. The ideal candidate is an experienced professional capable of providing precise interpretation and translation of operational, intelligence, and law enforcement information whilemaintainingthe highest standards of accuracy...
...accomplished scientific leader to join the Quantitative Pharmacology and Pharmacometrics - Immuno-Oncology (QP2-IO) team as-Senior Director, Advanced Pharmacometrics. QP2-IO is part of the Global Clinical Development organization and is responsible for clinical...