Senior Data Engineer – AI & Data Platforms

Location: Dublin, Ireland
Working Model: Hybrid
Salary: Competitive, dependent on experience
Employment Type: Permanent

About the Opportunity

Intellect Talent is supporting a global business partner in the search for a Senior Data Engineer to join their growing technology and AI organisation.

Our business partner is a global, AI-first enterprise transformation organisation with more than 25 years of experience. They work with over 100 global clients, including major multinational and Fortune 500 organisations, across sectors including financial services, insurance, healthcare, retail, manufacturing, higher education, and logistics.

Their work focuses on helping enterprises move from AI experimentation to scalable business outcomes, combining expertise across enterprise AI, digital engineering, cloud modernisation, platform integration, data engineering, and generative AI.

This is an exciting opportunity for a senior engineer who wants to work at the intersection of modern data platforms and enterprise AI.

The Role

We are looking for a Senior Data Engineer to help design, build, and scale the data foundations behind the next generation of AI-powered services.

You'll combine strong hands-on data engineering expertise with a focus on data quality, reliability, scalability, and performance, working across Databricks, PySpark, Spark, Hadoop, Hive, SQL, and modern Lakehouse architectures.

The role also has a significant AI engineering component. You'll contribute to the design and productionisation of LLM-powered applications, RAG pipelines, tool-calling integrations, and agentic workflows.

This is a hands-on senior position where you'll contribute across architecture, development, testing, debugging, optimisation, and production delivery, while also providing technical guidance and mentoring to other engineers.

What You'll Be Doing

Data Engineering & Quality

• Design, develop, and validate scalable data pipelines across ingestion, transformation, and consumption layers.
• Work with Databricks, PySpark, Spark, Hadoop, Hive, and Lakehouse architectures.
• Perform source-to-target reconciliation, data profiling, and data quality validation.
• Define and implement data quality frameworks, metrics, controls, and automated checks.
• Develop and optimise ETL/ELT workflows for performance, scalability, and reliability.
• Use advanced SQL for data profiling, reconciliation, root-cause analysis, complex joins, CTEs, window functions, and aggregations.
• Monitor data across cloud-based Lakehouse environments.
• Investigate production issues and perform root-cause analysis.
• Implement proactive monitoring, observability, and automated quality controls.
• Identify and resolve performance bottlenecks across data pipelines and services.

AI & GenAI Engineering

• Contribute to the architecture and development of AI-powered enterprise services.
• Build and productionise LLM-powered applications and RAG/retrieval systems.
• Work with orchestration frameworks such as LangChain and LlamaIndex.
• Develop tool-calling and function-calling integrations.
• Implement agentic workflows covering planning, reasoning, memory, grounding, and human-in-the-loop processes.
• Contribute to AI evaluation, safety, reliability, and monitoring frameworks.
• Work with enterprise data sources to create reliable and grounded AI experiences.
• Help take AI capabilities from experimentation through to production.

Technical Leadership

• Contribute to team-level technical architecture and design decisions.
• Evaluate technical trade-offs and translate business requirements into scalable solutions.
• Work closely with engineers, architects, analysts, and other stakeholders.
• Provide technical guidance and mentorship to junior and mid-level engineers.
• Participate in design reviews, code reviews, testing, and engineering quality initiatives.
• Promote secure coding, observability, automation, and reliable engineering practices.

What We're Looking For

We're looking for an experienced engineer who combines strong data engineering fundamentals with an interest and/or proven experience in AI and modern cloud technologies.

Essential Experience

• 6+ years' experience in Data Engineering, Data Quality Engineering, Data Testing, or related software/data engineering roles.
• Strong hands-on experience with Databricks and PySpark.
• Strong experience with Spark and the Hadoop ecosystem, including Hive and HDFS.
• Advanced SQL skills.
• Experience working with Data Warehouses, Data Lakes, and Lakehouse architectures.
• Understanding of Star Schema, Snowflake Schema, and dimensional modelling.
• Experience with at least one major cloud platform: AWS, Azure, or GCP.
• Strong software engineering experience building scalable and distributed systems.
• Experience with Java, Spring Boot, REST APIs, or similar backend technologies.
• Experience building and supporting production-grade data pipelines.
• Experience working in an agile engineering environment.
• Strong understanding of data quality, validation, reconciliation, and production support.

AI / GenAI Experience

Experience with several of the following:

• LLM-powered applications.
• RAG and retrieval systems.
• LangChain, LlamaIndex, or comparable orchestration frameworks.
• Tool/function calling.
• Agentic workflows.
• AI evaluation and testing.
• Grounding and context management.
• Human-in-the-loop approaches.
• AI safety and reliability mechanisms.

We're particularly interested in candidates who have experience taking AI solutions beyond experimentation and into production environments.

Nice to Have

• Automated data testing frameworks.
• Data observability and monitoring platforms.
• CI/CD for data engineering workloads.
• Delta Lake and Unity Catalog.
• Airflow, Kafka, or other Apache ecosystem technologies.
• Docker and Kubernetes.
• AWS and/or Azure.
• TypeScript or React.
• Experience working with strategic technology partnerships or technology ecosystems.

What You'll Get

• 23 days' paid holiday plus 1 company holiday.
• Up to 5 days of company-paid sick leave on a rolling basis.
• Up to 16 weeks of paid maternity leave, combining statutory and company pay, subject to eligibility.
• 2 weeks' paternity leave in line with company policy and the statutory scheme.
• Up to €140 per month towards private medical insurance.
• Employee pension contributions of up to 3% of salary.
• The opportunity to work on large-scale data, AI and GenAI transformation programmes.
• Exposure to modern Databricks, Lakehouse, LLM, RAG and agentic AI technologies.
• The opportunity to contribute to technical architecture and influence engineering practices.
• A collaborative environment working alongside experienced engineers, architects, and technology professionals.

Why This Opportunity?

This role offers the opportunity to work on challenging engineering problems where enterprise data engineering and AI are coming together.

You'll have the chance to work with modern data platforms while contributing to the development of production AI capabilities, including RAG and agentic systems.

It's a strong opportunity for a Senior Data Engineer who wants to expand their impact beyond traditional data pipelines and become involved in the development of AI-powered enterprise solutions.

Apply Now

If you're a Senior Data Engineer with strong Databricks/PySpark experience and you're interested in working at the intersection of data engineering, cloud and AI, we'd love to hear from you.

Apply now or contact Intellect Talent for more information.