Overview

Lead Data Engineer Jobs in Colombo District, Western Province, Sri Lanka at HCLTech Sri Lanka

Title: Lead Data Engineer

Company: HCLTech Sri Lanka

Location: Colombo District, Western Province, Sri Lanka

HCLTech Sri Lanka | Lead Data Engineer

We are HCLTech, one of the fastest-growing large tech companies in the world and home to 219,000+ people across 60 countries, supercharging progress through industry-leading capabilities centered around Digital, Engineering and Cloud.

We are looking for a skilled Lead Data Engineer with strong experience in Databricks, PySpark, Azure Data Factory (ADF), Logic Apps, and Apache Airflow to design, build, and maintain scalable data pipelines and cloud-based data solutions. The candidate should have experience handling large datasets and building reliable ETL/ELT workflows.

Position: Lead Data Engineer

Experience: 5+ years

Location: Sri Lanka

Key Responsibilities:

  1. Design, develop, and optimize data pipelines using Databricks and PySpark.
  2. Build and manage ETL/ELT processes for structured and unstructured data.
  3. Develop and maintain Azure Data Factory (ADF) pipelines and data integration workflows.
  4. Implement and support Azure Logic Apps for workflow automation and system integrations.
  5. Create, schedule, and monitor workflows using Apache Airflow.
  6. Work with cloud-based data platforms and support end-to-end data movement.
  7. Optimize data processing performance and improve scalability.
  8. Collaborate with business, analytics, and application teams to gather requirements.
  9. Ensure data quality, security, governance, and monitoring.
  10. Troubleshoot production issues and provide operational support.

Key Requirements:

  • Strong experience in Databricks and PySpark
  • Highly proficient in Azure Data Factory (ADF) and Azure Logic Apps
  • Hands-on experience with Apache Airflow
  • Strong SQL skills and data modeling concepts
  • Experience in ETL/ELT development and orchestration
  • Knowledge of Azure Cloud services
  • Experience with performance tuning and monitoring
  • Understanding of CI/CD and deployment processes
  • Familiarity with version control tools (Git)

Good to have Skills:

  • Experience with Delta Lake
  • Knowledge of Azure Data Lake Storage (ADLS)
  • Experience with Python
  • Exposure to DevOps practices
  • Understanding of data warehousing concepts

Apply

Upload your CV/resume or any other relevant file. Max. file size: 800 MB.