Connexion Inscription gratuite
Postuler

Publié il y a 9 jours

Big Data Engineer (PySpark / Spark / Kafka)

@ Astra North Infoteck Inc.

Toutes ses offres

Toronto, Ontario

Contrat
Postuler
SIGNAUX

En bref

Astra North Infoteck Inc. is hiring a contract Big Data Engineer to design, build and support enterprise-scale data platforms. The work covers batch and real-time pipelines, Kafka streaming and ETL/ELT workflows built on the Spark and Hadoop ecosystem. The role is hybrid, with 3 days a week in the office.

Responsabilités :

  • Design and optimize scalable data pipelines with PySpark, Spark, Hadoop and NiFi
  • Build and support Kafka streaming and event-driven applications
  • Create and tune ETL/ELT workflows for large datasets
  • Monitor and troubleshoot Spark jobs and data pipelines

Résumé des exigences :

  • Hands-on PySpark and Apache Spark experience
  • Apache Kafka and Hadoop ecosystem experience
  • Apache NiFi experience
  • Strong SQL and Python skills

Outils et compétences

PySparkApache SparkApache KafkaHadoopHiveApache NiFiAirflowJenkinsGitJIRA
Source et dates

Type de source non vérifié · zoho

Date publiée par l’employeur
2026-11-01
Première détection par notre source
2026-11-02
Mise à jour du dossier source
2026-11-02

Ces dates décrivent le dossier source, pas une vérification en direct du recrutement. Consultez l’offre originale.

Détails du poste

NiveauIntermédiaire
DomaineDéveloppement logiciel
Déplacementsavion : aucun · route : aucun
Milieu de travailBureau
PositionSurtout assis
Exigences physiquesphysique faible · ordinateur élevé

Résumé et détails rédigés par l’IA à partir de l’offre; l’offre de l’employeur fait foi.

Postuler

Astra North Infoteck Inc.

Toutes ses offres
ChampValeur
Offres sur Jobsy421
Description

Job Description: Big Data Engineer (PySpark / Spark / Kafka)

Work Mode: Hybrid – 3 Days a Week in Office

Job Overview

We are seeking an experienced Data Engineer with strong expertise in Big Data technologies to design, develop, and support enterprise-scale data platforms. The ideal candidate should have hands-on experience in PySpark, Apache Spark, Kafka, Hadoop ecosystem components, and Apache NiFi, along with a strong understanding of data ingestion, transformation, and real-time processing frameworks.

Key Responsibilities

  • Design, develop, and optimize scalable data pipelines using PySpark, Spark, Hadoop, and Apache NiFi.
  • Build and maintain batch and real-time data processing solutions.
  • Develop and support Kafka-based streaming applications and event-driven architectures.
  • Create and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
  • Develop complex SQL queries for data extraction, transformation, validation, and troubleshooting.
  • Implement data ingestion solutions from databases, APIs, files, and streaming sources.
  • Monitor, troubleshoot, and enhance the performance of Spark jobs and data pipelines.
  • Collaborate with architects, business analysts, and development teams to deliver high-quality data solutions.
  • Support platform upgrades, deployments, testing, certification, and production releases.
  • Ensure data quality, governance, security, and operational excellence across data platforms.

Mandatory Skills

  • PySpark
  • Apache Spark (Spark SQL, DataFrames)
  • Apache Kafka
  • Hadoop Ecosystem (HDFS, Hive, YARN)
  • Apache NiFi
  • SQL
  • Python

Preferred Skills

  • Spark Streaming
  • Airflow / Oozie
  • Hive
  • Scala
  • Jenkins, Bitbucket, Git
  • JIRA, Confluence
  • Cloud Platforms (GCP/AWS/Azure)
  • Data Warehousing Concepts and Dimensional Modeling

Emplois semblables

  • Senior Technology Architect

    Toronto, ON
    Contrat

    Randstad Canada

    This position demands a seasoned technology architecture authority with extensive full-lifecycle systems design experience, robust cyber security and cloud governance expertise, and a…; Establish quality metrics,…

  • Director, Fraud Attack Response & Insights - Americas

    Toronto, ON
    Temps plein

    Canada: Mastercard Canada ULC, MCI

    •Deep fraud operations expertise; •Payment and payment fraud experience , with advanced experience in payment fraud analytics, fraud intelligence, network or product security and cross-functional governance;…

  • Data Architect

    Toronto, ON
    Contrat

    CONFLUX SYSTEMS

    15+ ansMust-Have Skills: AWS & SAS; Experience: 10-15 Years; Looking for experienced Data Architects with strong hands-on experience in AWS and SAS

    AWS, SAS, Reporting, Leadership, Decision making, Cloud technologies

  • Director of Strategic Partnerships

    Toronto, ON
    Temps plein

    CDC Global Solutions

    Required Experience:; Significant CDMO experience - essential; Experience managing technical R&D programs, client-facing projects, or complex development workstreams

    Leadership, Robotics

  • Senior Director Sales & Marketing

    Toronto, ON
    Temps plein

    HappyGlobal Service

    10+ ansMinimum Qualifications; Bachelor’s degree required; MBA preferred; 10–15+ years of progressive experience in Sales, Marketing, or Commercial Leadership

    Sales, Leadership, Customer segmentation, Forecasting, Customer relationship management, Performance management

  • Sr Sales Rep - Retail Lighting Products

    Toronto, ON
    Temps plein

    LEDVANCE

    5+ ansEDUCATION AND EXPERIENCE:; Bachelor’s degree in business or related field, masters preferred; 5+ years’ of relevant experience in the Consumer-Packaged Goods Retail Channel

    Sales, Presentation skills, Supply chain management, Communication skills, Negotiation, Conflict resolution

  • Marketing Business Systems Analyst

    Toronto, ON
    Temps plein

    Fantuan 饭团

    1+ ansBusiness Requirements Analysis & System Optimization; Collaborate with the Marketing team to gather, analyze, and document business requirements, translating them into actionable system solutions; Participate in system…

    Analyse business requirements, Customer relationship management, Data analysis, Solve ICT system problems, Data visualization, Decision making

  • Finance Lead

    Toronto, ON
    Temps plein

    Generis

    5+ ansSkills and Qualifications; 5+ years of progressive finance experience, with at least 2 years in a leadership role; Proven ability to manage full-cycle accounting and FP&A

    CPA, Leadership, Team building, Communication skills, Microsoft Excel, Intuit QuickBooks

Lieux et modes de travail

Lieux

Tout le CanadaTélétravail · Hybride · Sur place · Sur le terrain

Astuce : ajoutez le télétravail pour élargir la recherche.

Exclure des lieux

Mode de travail

Offres de cette entreprise dans les résultats

Voici les offres de cette entreprise chargées dans votre recherche. Elle peut avoir d’autres postes ouverts.