Postuler

Publié il y a 4 mois

Principal Databricks Data Engineer

@ Astra North Infoteck Inc.

Toronto, Ontario

12+ ans d’expérience
Postuler
SIGNAUX

En bref

Astra North Infoteck Inc. is hiring a Principal Databricks Data Engineer with 12 to 18 years of data engineering experience to lead the migration of Cloudera platforms to Databricks Lakehouse architecture. The role designs enterprise data warehouse and lakehouse layers for finance and risk data, mentors senior engineers and works with finance, risk and governance stakeholders.

Responsabilités :

  • Lead Cloudera to Databricks transformation initiatives
  • Design and govern enterprise data warehouse and lakehouse architectures
  • Refactor Hive and Impala logic into PySpark and Spark SQL pipelines
  • Ensure reconciliation, audit integrity and data governance
  • Mentor senior engineers and set engineering standards

Résumé des exigences :

  • 12 to 18 years of data engineering experience
  • 5+ years hands-on Databricks and Apache Spark at scale
  • 8+ years enterprise data warehouse and data lake platforms
  • Strong Cloudera (CDH/CDP), Hive, HBase and Impala experience
  • Finance and risk data model experience

Outils et compétences

Source et dates

Type de source non vérifié · zoho

Date publiée par l’employeur
2026-07-13
Première détection par notre source
2026-10-11
Mise à jour du dossier source
2026-10-11

Ces dates décrivent le dossier source, pas une vérification en direct du recrutement. Consultez l’offre originale.

Détails du poste

NiveauChef d’équipe
DomaineDéveloppement logiciel
Expérience12 an(s) et plus
Milieu de travailBureau, Domicile
PositionSurtout assis
Exigences physiquesphysique faible · ordinateur élevé

Résumé et détails rédigés par l’IA à partir de l’offre; l’offre de l’employeur fait foi.

Astra North Infoteck Inc.

ChampValeur
Offres sur Jobsy426
Description

Principal Databricks Data Engineer​

Experience Required: 12–18 Years

Key Requirements

• 12–18 years of overall Data Engineering experience

• 8+ years of experience with enterprise Data Warehouse and Data Lake platforms

• 5+ years of hands-on experience with Databricks and Apache Spark at scale

• Strong experience modernizing legacy Cloudera platforms including:

• CDH/CDP

• Hive

• HBase

• Impala

• Spark

• Modernize Cloudera platforms to Databricks Lakehouse architecture

• Redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to Cloud Object Storage and Delta Lake

• Refactor legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines

• Ensure data reconciliation, audit integrity, and consistency during migration

• Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures

• Implement layered data architecture including:

• Raw / Landing Layer

• Curated / Conformed Layer

• Semantic / Consumption Layer

• Modernize traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures

• Strong experience with finance and risk data models including:

• General Ledger

• Sub-ledger

• Financial Hierarchies

• Credit Risk

• Liquidity Risk

• Market Risk

• Enable reporting capabilities including:

• Aggregation

• Drill-down

• Drill-back

• Build and manage semantic and consumption layers for BI, reporting, and analytics

• Define business metrics, dimensions, hierarchies, and KPIs

• Experience with:

• Databricks SQL

• Delta Tables

• dbt or similar frameworks

• Develop and optimize large-scale data pipelines using:

• PySpark

• Spark SQL

• Delta Lake

• Implement Medallion Architecture including:

• Bronze Layer

• Silver Layer

• Gold Layer

• Optimize workloads using:

• Z-ORDER

• OPTIMIZE

• Caching

• Cluster Configuration Tuning

• Implement:

• Data Governance

• Data Quality Frameworks

• Reconciliation Controls

• Exception Handling

• Establish data lineage and metadata management

• Ensure data security, access control, and compliance standards

• Experience with AWS or Azure cloud platforms

• Experience with CI/CD pipelines using:

• Git

• Terraform

• Jenkins

• Azure DevOps

• Familiarity with:

• Apache Airflow

• Databricks Workflows

• Experience with dbt is an advantage

• Act as a technical authority and lead enterprise architecture decisions

• Mentor senior engineers and establish engineering standards

• Collaborate with finance, risk, analytics, and governance stakeholders

• Translate complex data structures into business-ready insights

Nice-to-Have Skills

• Experience in BFSI, Capital Markets, or Regulatory Reporting

• Exposure to:

• SAP Finance

• Oracle Financials

• SAP S/4HANA

• Experience supporting AI/ML workloads

• Databricks and Cloud Certifications

Key Responsibilities

• Lead Cloudera to Databricks transformation initiatives

• Shape enterprise finance and risk data platforms

• Support regulatory, management, and analytical reporting systems

Essential Skills

• Databricks

• Apache Spark

• PySpark

• Spark SQL

• Delta Lake

• Lakehouse Architecture

• Cloudera (CDH/CDP)

• Hive

• HBase

• Impala

• Data Warehouse & Data Lake

• Medallion Architecture

• Databricks SQL

• dbt

• AWS / Azure

• Airflow / Databricks Workflows

• CI/CD

• Terraform

• Jenkins

• Git

• Finance & Risk Data Models

• Enterprise Data Architecture


Emplois semblables

  • Critical Environment Electrical Engineering

    Toronto, ON
    86k–169k $/anSur placeTemps plein

    Microsoft: Every company has a mission.

    2+ ansCandidates need a bachelor's or master's degree in electrical or computer engineering plus relevant engineering experience, and must be registered as a P.Eng. in Ontario. Critical environment and electrical infrastructure experience is preferred.

    Blueprints, Building management systems, Switchgear, UPS systems, Standby diesel generators

    Senior Critical Environment Mechanical Engineer

    Toronto, ON
    114k–204k $/anSur placeTemps plein

    Microsoft: Every company has a mission.

    7+ ansMicrosoft requires 7+ years of related engineering experience, or a relevant degree plus 3 to 4+ years of experience, and a P. Eng. registration in Ontario. Candidates must be able to obtain Reliability Status screening and meet Microsoft Cloud background checks.

    CAD modeling tools, Predictive modeling software

  • Receptionist/Administrative Support

    Toronto, ON
    20–22 $/hContrat, Temporaire

    adecco

    1+ ansCandidates need 1 to 3 years of reception, front desk or office administration experience. They should communicate well in writing and speaking, be organized, service-oriented, and comfortable with Microsoft Word, Excel, Outlook and Teams.

    Microsoft Word, Microsoft Excel, Microsoft Outlook, Microsoft Teams, audiovisual equipment

    Executive Assistant

    Toronto, ON
    35–40 $/hContrat, Temporaire

    adecco

    5+ ansCandidates need 5+ years of directly related executive assistant experience, preferably in finance. They must be advanced in Microsoft Word, PowerPoint, Excel and Outlook, and have Concur experience (Coupa is an asset). Strong written and verbal communication, discretion and the ability to switch priorities quickly are required.

    Concur, Coupa, Microsoft Word, Microsoft PowerPoint, Microsoft Excel, Microsoft Outlook

    Medical & Legal Customer Service Representative

    Toronto, ON
    28 $/hContrat, Temporaire

    adecco

    Candidates need a high school diploma and bilingual English and French skills, both verbal and written. Experience in customer service, legal, healthcare, or insurance work is preferred, along with strong attention to detail and comfort working remotely on their own device.

    Microsoft Office, Outlook, Excel

  • Physician Reviewer (MD/DO) - Medical AI Evaluation

    Toronto, ON +4
    TélétravailTemps partiel

    Mindrift: Mindrift (mindrift.ai) is the expert-sourcing platform built by Toloka, a Nebius Group (Nasdaq: NBIS) company and a global leader in data services…

    Mindrift needs candidates with an MD or DO, an active unrestricted US medical license, and board certification or completed residency. Candidates need broad multi-system clinical experience, strong reasoning, and C1-level written English.

  • Senior GES Payments Specialist - 1 year contract

    Toronto, ON
    Temps plein

    Royal Bank of Canada: Royal Bank of Canada is a global financial institution with a purpose-driven, principles-led approach to delivering leading performance.

    Cotation en bourse: TSX:RY · NYSE:RY · Source vérifiée: 2026-11-02

    2+ ansCandidates need a bachelor's degree and 2 to 5 years of financial industry or payments processing experience, ideally with sanctions work. They must know SWIFT and ACH transactions and US, EU, UK and Canadian sanctions rules, and work well under high pressure. ACAMS certification is an asset.

    SWIFT, ACH

    Senior GES Payments Specialist - 6 month contract

    Toronto, ON
    Temps plein

    Royal Bank of Canada: Royal Bank of Canada is a global financial institution with a purpose-driven, principles-led approach to delivering leading performance.

    Cotation en bourse: TSX:RY · NYSE:RY · Source vérifiée: 2026-11-02

    2+ ansRoyal Bank of Canada needs candidates with a bachelor's degree and two to five years of financial or payments processing experience, ideally with sanctions work. They must know SWIFT, ACH and the US, EU, UK and Canadian sanctions rules, and communicate well in writing and speaking. An ACAMS certification is an asset.

    SWIFT, ACH

Lieux et modes de travail

Lieux

Tout le CanadaTélétravail · Hybride · Sur place · Sur le terrain

Astuce : ajoutez le télétravail pour élargir la recherche.

Exclure des lieux

Mode de travail

Offres de cette entreprise dans les résultats

Voici les offres de cette entreprise chargées dans votre recherche. Elle peut avoir d’autres postes ouverts.