Lead Data Engineer

About the Company

Hartree Partners is a leading global merchant commodities firm specialising in energy, commodities and associated industries. Jointly owned by senior management and funds managed by Oaktree Capital Management, LP, Hartree has over 28 years of experience in proprietary trading and principal investments. The firm employs over 4,600 individuals worldwide. Find out more about us by visiting our website at:

http://www.hartreepartners.com/

About the Role

A Global Commodities Firm Senior Data Pipeline Engineering Lead.

Company and Location: Hartree Partners.

Position Type: Full-time.

Reporting Line: Head of Finance Technology.

We are looking for an experienced Senior Data Pipeline Engineering Lead to shape and deliver the data technology that underpins Finance, Operations, Risk and Compliance across our trading business. You will lead a small team of engineers while remaining close to the technology, designing scalable data pipelines and cloud-native solutions that move critical trade, valuation, settlement and reference data through validation, enrichment and reconciliation into the platforms that depend on it. This is an opportunity to combine hands-on engineering leadership with AWS architecture and the practical application of AI and agentic technologies. You will help turn complex financial and operational processes into reliable, well-controlled data products, while improving automation, data quality and operational efficiency.

Responsibilities

• Lead the design and delivery of batch and event-driven data pipelines supporting Finance, Operations, Risk and Compliance.
• Own the AWS data architecture for Finance Technology, creating scalable, secure, resilient and cost-effective solutions across storage, compute, orchestration and integration.
• Lead, mentor and develop a small team of data engineers, setting technical direction and establishing high engineering standards across design, development and code quality.
• Partner with Finance, Operations, Risk, Treasury and Compliance stakeholders to translate business needs into clear technical designs, delivery plans and priorities.
• Build robust frameworks for data validation, enrichment, reconciliation and exception management, including quarantine, replay, versioning and auditable adjustments.
• Develop practical AI and agentic solutions to improve areas such as exception triage, proposed corrections and operational workflow automation, with appropriate controls and human oversight.
• Embed strong data and audit controls covering completeness, accuracy, timeliness, lineage, control evidence, approvals, segregation of duties and record retention.
• Establish effective CI/CD, infrastructure as code, automated testing, observability and alerting across data pipelines and platforms.
• Take ownership of production stability and daily data delivery, driving effective incident management, root cause analysis and continuous improvement.
• Work closely with architecture, data governance, information security, infrastructure and vendor partners to support the wider Finance Technology strategy.
• Lead delivery from architecture and design through implementation and production support, managing priorities, dependencies and delivery risks.
• Evaluate emerging technologies and AI capabilities, using proofs of concept and clear business cases to identify opportunities that deliver measurable value.
• Maintain high-quality architecture documentation, design decisions, data contracts and operational runbooks.

Qualifications

• Requirements: education, competencies & skills.
• 10+ years’ experience in data engineering or software engineering, including at least three years leading teams delivering enterprise-scale data platforms.
• Experience within commodities, energy trading or capital markets.
• Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics or a related discipline.

Required Skills

• Strong experience designing and delivering cloud-native data architectures on AWS.
• A strong track record of building enterprise-scale data platforms supporting areas such as accounting, settlements, treasury, risk and compliance.
• Hands-on engineering experience building production-grade data pipelines.
• Practical experience developing AI or agentic solutions, including LLM-powered applications, tool use, retrieval-augmented generation, evaluation and guardrails.
• Strong data modelling skills across transactional and analytical workloads.
• Experience designing data quality, validation, reconciliation, lineage and audit-control frameworks within a regulated environment.
• Strong understanding of event-driven architectures and modern integration patterns.
• Proven ability to lead complex technology delivery from architecture and design through to implementation and production support.
• Strong stakeholder management and communication skills, with the ability to explain technical decisions and trade-offs clearly to both technical and business audiences.
• A hands-on leadership style, with the judgement to balance delivery speed with resilience, control, auditability and data quality.

Preferred Skills

• Preferred Requirements.
• Experience delivering enterprise technology solutions within banking or financial services.
• Experience with ETRM platforms such as Allegro, Aspect or similar.
• Understanding of the trade lifecycle and its relationship with Finance, Treasury, Compliance, and market and credit risk.
• AWS certification, such as AWS Certified Solutions Architect – Professional or AWS Certified Data Engineer.
• A track record of applying new technologies pragmatically, with clear consideration for business value, risk and operational controls.
• Tools & Technologies AWS (S3, Athena, Glue, Lambda, Step Functions, MSK, RDS/Aurora PostgreSQL, ECS/EKS) • Python • SQL • Apache Airflow (including Amazon MWAA) • Apache Kafka • Apache Spark • PostgreSQL • Terraform or AWS CloudFormation/CDK • Git and CI/CD pipelines • LLM APIs and agent frameworks • Vector databases • REST APIs • JSON • Parquet • Jira • Confluence • ETRM systems (Allegro, Aspect or similar).

Equal Opportunity Statement

Hartree Partners is an equal opportunities employer. We are committed to ensuring that all applicants and employees are treated fairly and are not discriminated against on the grounds of age, disability, gender reassignment, gender identity, marriage and civil partnership, pregnancy and maternity, race (including colour, nationality, and ethnic or national origins), religion or belief, sex, or sexual orientation. Recruitment and employment decisions are based on merit and business needs. We will make reasonable adjustments for candidates and employees with disabilities.