Текст заказа
Job Description: Lead Data Engineer (GCP Dataproc & BigQuery)
Location: Edison, NJ (Onsite)
Employment Type: Full-Time
Experience Required: 10+ Years
Position Overview
We are seeking a highly skilled Lead Data Engineer with deep expertise in Google Cloud Platform (GCP), specifically Dataproc and BigQuery, to lead the design, development, and optimization of enterprise-scale data platforms and analytics solutions. This is an onsite role based in Edison, NJ.
The ideal candidate will possess strong technical leadership capabilities, exceptional communication skills, and a proven ability to solve complex data engineering challenges. This individual will work closely with business stakeholders, architects, data scientists, and engineering teams to deliver scalable, high-performance data solutions that drive business outcomes.
Key Responsibilities
* Lead the design and implementation of large-scale data engineering solutions on GCP.
* Architect and develop robust ETL/ELT pipelines using Dataproc, BigQuery, and related GCP services.
* Build scalable data ingestion, transformation, and processing frameworks for structured and unstructured data.
* Optimize data models, query performance, and processing efficiency within BigQuery environments.
* Drive cloud migration and modernization initiatives from on-premise or legacy platforms to GCP.
* Collaborate with business stakeholders to understand data requirements and translate them into technical solutions.
* Mentor and guide data engineers through code reviews, design discussions, and best practice adoption.
* Establish data quality, governance, security, and operational monitoring standards.
* Troubleshoot and resolve complex production issues while ensuring platform reliability and performance.
* Communicate effectively with technical and non-technical stakeholders, including executive leadership.
Required Qualifications
* 10+ years of experience in Data Engineering, Data Warehousing, or Big Data ecosystems.
* 5+ years of hands-on experience with Google Cloud Platform (GCP).
* Strong expertise in:
* Google BigQuery
* Google Dataproc
* Cloud Storage (GCS)
* Cloud Composer / Airflow
* Pub/Sub
* Dataflow (preferred)
* Advanced SQL development and query optimization skills.
* Strong experience with Python, PySpark, or Spark-based data processing frameworks.
* Experience building enterprise-scale batch and real-time data pipelines.
* Strong understanding of data modeling, dimensional modeling, and data warehouse concepts.
* Experience with CI/CD, DevOps practices, and cloud-native architecture patterns.
Leadership & Communication Requirements
* Exceptional verbal and written communication skills.
* Proven ability to lead client-facing discussions and technical workshops.
* Strong stakeholder management and requirements-gathering experience.
* Ability to influence architecture decisions and drive consensus across teams.
* Demonstrated capability to present complex technical concepts to business audiences.
* Strong mentoring and team leadership skills.
Problem-Solving Expectations
The successful candidate will:
* Exhibit outstanding analytical and troubleshooting abilities.
* Drive root cause analysis and resolution for complex data platform issues.
* Identify optimization opportunities across architecture, performance, and cost management.
* Provide innovative solutions to business and technical challenges.
* Thrive in fast-paced, highly collaborative environments.
Preferred Qualifications
* Experience with Terraform, Infrastructure as Code (IaC), and cloud automation.
* Exposure to data governance and metadata management tools.
* Experience with Snowflake, Databricks, or other modern cloud analytics platforms.
* GCP Professional Data Engineer certification.
* Experience leading distributed or offshore teams.
Education
* Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or related field.