Connexion Inscription gratuite
Postuler

Publié il y a 9 jours

Data Engineer

@ HCLTech

Toutes ses offres

Vancouver, Colombie-Britannique

Temps plein
Postuler
SIGNAUX

En bref

HCLTech is hiring a full-time Data Engineer to build and run large-scale data pipelines on big data platforms. The role centres on ETL/ELT work on Hadoop and Databricks, with performance tuning and CI/CD for data workflows.

Responsabilités :

  • Build and maintain ETL/ELT pipelines on Hadoop and enterprise data platforms
  • Develop PySpark and Python data processing jobs on Cloudera and Databricks
  • Optimize pipeline performance and cost through partitioning and compute tuning
  • Contribute to CI/CD for data workflows and code reviews
  • Partner with Product Managers to scope new data features

Résumé des exigences :

  • Hands-on production data engineering on Hadoop and Databricks
  • Strong Python, Spark and SQL skills for big data products
  • Experience with Airflow, NiFi or Talend orchestration
  • Working knowledge of Git, automated testing and release pipelines
  • Strong debugging and stakeholder communication skills

Outils et compétences

DatabricksApache SparkPySparkHadoopHiveImpalaApache AirflowApache NiFiTalendCloudera
Source et dates

Type de source non vérifié · apify-linkedin

Date publiée par l’employeur
2026-11-01
Première détection par notre source
2026-11-11
Mise à jour du dossier source
2026-11-11

Ces dates décrivent le dossier source, pas une vérification en direct du recrutement. Consultez l’offre originale.

Détails du poste

DomaineDéveloppement logiciel
Heures par semaine40 (estimé)
Milieu de travailBureau
PositionSurtout assis
Exigences physiquesphysique faible · ordinateur élevé
LanguesEnglish

Résumé et détails rédigés par l’IA à partir de l’offre; l’offre de l’employeur fait foi.

Postuler
ChampValeur
Offres sur Jobsy18
Description

Job Summary

Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at large scale, pushing the boundaries of data processing and model enablement.

•

Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.

•

Develop high-performance data processing jobs using PySpark/Spark, Python on data platforms such as cloudera and databricks.

•

Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patterns

•

Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases.

•

Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features

Ideal Candidate Qualifications:

•

Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).

•

High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.

•

Hands-on experience with cloud data platforms such as databricks, snowflake (databricks preferred)

•

Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.

•

Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.

•

Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.

•

Experience developing Java based applications is an added advantage.

LONGDESCRIPTION section. 2 of 6.

Section Title: Key Responsibilities

Key Responsibilities

Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at large scale, pushing the boundaries of data processing and model enablement.

•

Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.

•

Develop high-performance data processing jobs using PySpark/Spark, Python on data platforms such as cloudera and databricks.

•

Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patterns

•

Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases.

•

Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features

Ideal Candidate Qualifications:

•

Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).

•

High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.

•

Hands-on experience with cloud data platforms such as databricks, snowflake (databricks preferred)

•

Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.

•

Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.

•

Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.

•

Experience developing Java based applications is an added advantage.

LONGDESCRIPTION section. 3 of 6.

Section Title: Skill Requirements

Skill Requirements

Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at large scale, pushing the boundaries of data processing and model enablement.

•

Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.

•

Develop high-performance data processing jobs using PySpark/Spark, Python on data platforms such as cloudera and databricks.

•

Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patterns

•

Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases.

•

Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features

Ideal Candidate Qualifications:

•

Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).

•

High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.

•

Hands-on experience with cloud data platforms such as databricks, snowflake (databricks preferred)

•

Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.

•

Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.

•

Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.

•

Experience developing Java based applications is an added advantage.

SKILL section. 4 of 6.

Section Title: Must Have Skills

Must Have Skills

  • Click Enter to show the proficiency description of Data EngineeringData Engineering
  • Click Enter to show the proficiency description of DatabricksDatabricks
  • Click Enter to show the proficiency description of Apache SparkApache Spark
  • Click Enter to show the proficiency description of PySparkPySpark
  • Click Enter to show the proficiency description of PythonPython
  • Click Enter to show the proficiency description of CI/CDCI/CD

SKILL section. 5 of 6.

Section Title: Good to have Skills

Good to have Skills

LONGDESCRIPTION section. 6 of 6.

Section Title: Other Requirements

Other Requirements

Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at large scale, pushing the boundaries of data processing and model enablement.

•

Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.

•

Develop high-performance data processing jobs using PySpark/Spark, Python on data platforms such as cloudera and databricks.

•

Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patterns

•

Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases.

•

Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features

Ideal Candidate Qualifications:

•

Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).

•

High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.

•

Hands-on experience with cloud data platforms such as databricks, snowflake (databricks preferred)

•

Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.

•

Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.

•

Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.

•

Experience developing Java based applications is an added advantage

Emplois semblables

  • Data Engineer

    North Vancouver, BC
    40–60 $/hTemporaire

    Robert Half

    5+ ansCandidates need 5+ years as a business analyst, data analyst or data engineer, with strong SQL, advanced Excel and Power BI skills. Experience with ERP systems and finance, procurement or inventory processes is required.

    SQL, Excel, Power Query, Power Pivot, Power BI, Oracle

  • Senior Data Engineer

    Victoria, BC
    Télétravail

    Beatport

    5+ ansBeatport needs candidates with 5+ years of direct data engineering experience, expert SQL (ideally MySQL or Postgres dialects), dbt, and production ETL/ELT ownership with orchestration tools such as Airflow or Composer. Candidates also need Python, strong attention to data integrity for financial or regulatory reporting, and clear communication with technical and non-technical stakeholders.

    dbt, BigQuery, Google Cloud Platform, Airflow, Cloud Composer, Cloud Run

  • Senior Data Engineer

    Vancouver, BC

    Capgemini

    Capgemini needs candidates with strong or expert Spark (PySpark) skills, hands-on pipeline development experience in Azure, and experience with orchestration tools such as ADF or Airflow. They also need SQL and denormalized data modeling skills for big data systems.

    PySpark, Databricks, Jupyter Notebooks

    Senior Data Engineer - Vancouver

    Vancouver, BC

    Capgemini

    Candidates need strong, expert-level Spark (PySpark) skills, hands-on Azure data pipeline experience, and knowledge of orchestration tools such as ADF or Airflow. Experience with SQL, denormalized data modeling for big data, and Jupyter, Colab or Databricks is also expected.

    Spark, PySpark, Databricks, Jupyter Notebooks, Airflow

  • Data Engineer (Intermediate)

    Kelowna, BC

    KF Aerospace

    2+ ansCandidates need a degree or diploma in computer science or a related field, or equivalent experience, plus 2 to 5 years in data engineering or data warehousing. Strong SQL, SQL Server, Microsoft Fabric, Azure Synapse and Power BI skills are required.

    SQL Server, Microsoft Fabric, Azure Synapse, Power BI, Azure Data Factory, Azure DevOps

  • Data Engineer

    Vancouver, BC
    Temps plein

    The Semios Group

    3+ ansCandidates need 3 to 5 years of data engineering experience on cloud platforms and production pipelines. They must know advanced SQL, Python or Node, at least one data warehouse, and workflow tools such as Prefect or Dagster.

    BigQuery, Redshift, Snowflake, Postgres, Cloud SQL, Cloud Run

  • Data Engineer

    Langley, BC
    Temps plein

    Pattison Food Group Ltd

    2+ ansCandidates need a STEM bachelor's degree, at least 2 years of building production data pipelines in the cloud, and strong SQL and Python/PySpark skills. Experience with Databricks, Spark, Delta Lake, Azure Data Factory and Git-based CI/CD is required.

    Databricks, Delta Lake, Azure Data Factory, Python, PySpark, SQL

  • Data Engineer, Amazon Prime

    Vancouver, BC
    Temps plein

    Amazon Development Centre Canada ULC

    3+ ansCandidates need at least three years of data engineering experience, including data modelling, warehousing and building ETL pipelines. They also need a track record of processing large datasets and extracting value from them. Experience with AWS services such as Redshift, S3, Glue, EMR and Kinesis is preferred.

    AWS Glue, Amazon EMR, Amazon S3, AWS Lambda, Amazon DynamoDB, Redshift

Lieux et modes de travail

Lieux

Tout le CanadaTélétravail · Hybride · Sur place · Sur le terrain

Astuce : ajoutez le télétravail pour élargir la recherche.

Exclure des lieux

Mode de travail

Offres de cette entreprise dans les résultats

Voici les offres de cette entreprise chargées dans votre recherche. Elle peut avoir d’autres postes ouverts.