Data Engineer

CTEC is a leading technology firm providing modernization, digital transformation, and application development services to the U.S. Federal Government. They are seeking a Data Engineer to support cloud data platforms, ETL development, and data migration projects, primarily using Azure Databricks.

Location: McLean, VA

Responsibilities

• Design, develop, and maintain scalable ETL pipelines and data workflows for cloud-based data platforms.
• Build, optimize, and maintain data processing solutions using Azure Databricks and lakehouse architectures.
• Support phased data migration from legacy databases to Azure Databricks environments.
• Implement layered lakehouse data architectures (bronze, silver, gold) in Databricks.
• Develop data processing notebooks, workflows, and transformations using Python and PySpark.
• Develop data validation, reconciliation, and testing processes.
• Integrate Databricks data platforms with analytics and reporting tools.
• Support data governance initiatives including metadata management, data catalog, encryption, access controls, and compliance.
• Maintain source control and CI/CD pipelines for Databricks workflows.
• Collaborate with data architects, solution architects, business analysts, and reporting teams.
• Provide ongoing support for Databricks workflows and troubleshoot data processing issues.
• Mentor junior data engineers and contribute to documentation.

Requirements

• 7+ years of experience in data engineering, ETL development, or large-scale data integration.
• Strong experience designing and developing ETL pipelines in Azure Databricks.
• Proficiency in SQL and Python, with experience using PySpark.
• Experience with cloud-based data platforms, data lakes, and lakehouse environments, preferably on Azure.
• Experience implementing layered lakehouse data architectures.
• Familiarity with Spark-based big data processing frameworks.
• Experience supporting data migration to Databricks platforms.
• Experience integrating Databricks with BI tools like Power BI.
• Knowledge of data governance, metadata management, and data security.
• Experience with source control and CI/CD pipelines.
• Understanding of SDLC and Agile methodologies.

Benefits

• Paid vacation & Sick leave
• Health insurance coverage
• Career training
• Performance bonus programs
• 401K contribution & Employer Match
• 11 Federal Holidays

Additional Information

• Must be a U.S. citizen and able to obtain a Public Trust clearance.