F
Data Engineer
Fa Etvl Saasfaprod1
in
2d ago
65%
अच्छा
नौकरी विवरण
ETL/ELT Pipeline Development: Build and maintain data pipelines extracting from Martech Platforms. MarTech & CDP Integration: Manage Customer Data Platforms (CDPs) and associated tools Identity Resolution & Attribution: Manage user stitching, PII-compliant data handling Data Quality & Governance: Ensure data lineage, schema validation, and reliable monitoring using orchestration tools like Apache Airflow
Key Responsibilities Design, develop, and support batch and incremental data ingestion pipelines into the data lake. Develop and optimize complex SQL queries, data transformations, and ETL workflows. Build, maintain, and monitor Airflow DAGs for orchestration and workflow automation. Analyze and troubleshoot data quality, data reconciliation, and pipeline execution issues. Perform end-to-end data lineage analysis and trace data from source systems to downstream consumers. Debug data discrepancies by investigating source systems, transformation logic, and target datasets. Support data migration and modernization initiatives from legacy platforms to cloud-based data platforms. Work closely with business stakeholders, architects, and upstream/downstream teams to understand data requirements. Validate data completeness, accuracy, and consistency across multiple systems. Participate in production support activities and resolve data-related incidents within SLA timelines. Develop reusable data engineering components and recommend process improvements. Required Skills Strong Expertise SQL / PL/SQL Complex joins, CTEs, analytical functions Query optimization and performance tuning Data modeling and database concepts BigQuery Query development and optimization Partitioning and clustering concepts Data loading and transformation techniques ETL/Data Engineering Data ingestion patterns (Batch and Incremental) Data transformations and validations Data quality checks and reconciliation Good to Have Apache Airflow Developing and troubleshooting DAGs Scheduling, monitoring, and dependency management Python Basic to intermediate scripting Data processing and automation Cloud Platforms (Preferably GCP) Cloud Storage BigQuery Composer/Airflow
Preferred Candidate Profile Strong analytical and problem-solving skills. Ability to independently investigate and resolve data issues. Good communication and stakeholder management skills. Experience in cloud data lake or warehouse environments. Exposure to customer, transactional, or enterprise-scale datasets is preferred. Screening Focus Areas Advanced SQL/PLSQL ETL & Data Pipeline Design BigQuery Concepts Data Debugging & Root Cause Analysis Airflow DAG Development Python Basics Data Migration & Data Validation Source-to-Target Traceability and Data Lineage Must Have: SQL/PLSQL, BigQuery/Cloud Data Warehouse, ETL, Data Analysis & Debugging Good to Have: Airflow DAG Development, Python, GCP Services Kafka, Pub/Sub optional