QA Data Engineering Lead
Tralee, IE
SMBC Group is a top-tier global financial group. Headquartered in Tokyo and with a 400-year history, SMBC Group offers a diverse range of financial services, including banking, leasing, securities, credit cards, and consumer finance. The Group has more than 130 offices and 80,000 employees worldwide in nearly 40 countries. Sumitomo Mitsui Financial Group, Inc. (SMFG) is the holding company of SMBC Group, which is one of the three largest banking groups in Japan. SMFG’s shares trade on the Tokyo, Nagoya, and New York (NYSE: SMFG) stock exchanges.
In the Americas, SMBC Group has a presence in the US, Canada, Ireland, Mexico, Brazil, Chile, Colombia, and Peru. Backed by the capital strength of SMBC Group and the value of its relationships in Asia, the Group offers a range of commercial and investment banking services to its corporate, institutional, and municipal clients. It connects a diverse client base to local markets and the organization’s extensive global network. The Group’s operating companies in the Americas include Sumitomo Mitsui Banking Corp. (SMBC), SMBC Nikko Securities America, Inc., SMBC Capital Markets, Inc., JRI America, Inc., SMBC Leasing and Finance, Inc., Banco Sumitomo Mitsui Brasileiro S.A., and Sumitomo Mitsui Finance and Leasing Co., Ltd.
This is a hybrid role, requiring the successful candidate to attend our Tralee office at least three (3) days in a work week.
Role Description
SMBC is seeking an experienced QA Data Engineering Lead to ensure data quality and integrity across our Databricks platforms. This is a hands-on technical leadership role focused on validating complex data pipelines and large-scale datasets using advanced SQL, Python, PySpark and Databricks.
The role will lead the overall data QA strategy and testing approach while collaborating with global technology and business teams. Responsibilities will include testing governance, ETL and data validation strategy, release readiness, quality risk and defect management and ensuring testing activities adhere to established SDLC processes and controls.
This role will report to the Director of Quality Assurance Testing.
Role Objectives: Delivery
QA Leadership & Test Strategy
- Own the overall data QA strategy and testing governance, ensuring appropriate standards, controls and quality measures are established throughout delivery.
- Lead a team of Database QA Engineers and provide direction on testing strategy, coverage and execution.
- Develop comprehensive testing plans for complex data pipelines, ETL processes and data warehouse solutions.
- Review business and technical requirements to identify testable conditions, data-quality risks, gaps and acceptance criteria.
- Provide senior stakeholders with clear assessments of data quality, testing progress, risks and release readiness, escalating significant issues where required.
- Support release planning and ensure appropriate data testing coverage ahead of production releases.
- Drive continuous improvement across QA processes, data-testing standards and automation practices.
- Ensure testing activities align with established SDLC processes and controls.
Data & ETL Testing
- Lead and oversee ETL and database testing activities, remaining hands-on where required.
- Design, develop and execute ETL test cases to validate data extraction, transformation and loading processes.
- Validate data integrity as it moves through the Bronze, Silver and Gold layers of Databricks Medallion Architecture.
- Perform end-to-end data validation and reconciliation across multiple stages and systems.
- Write and execute advanced SQL queries using window functions, complex joins and CTEs to perform detailed data validation.
- Develop advanced Python scripts using PySpark and Pandas to validate large datasets and identify data-quality issues and defects.
- Review Databricks jobs and pipeline runs to ensure ETL processes complete successfully and meet required data-quality standards.
- Investigate pipeline failures and data inconsistencies and work with engineering teams to support resolution.
Test & Defect Management
- Define, document and track data-testing scenarios and test cases to ensure appropriate coverage across data pipelines and integrated systems.
- Identify, document and manage defects through Jira.
- Collaborate with Data Engineers, developers and wider technology teams to investigate issues and support timely remediation.
- Provide oversight of significant data-quality issues and drive appropriate escalation and resolution.
Qualifications and Skills
- Bachelor’s degree in Computer Science, Engineering, Data or a related discipline, or equivalent relevant experience.
- 7+ years of experience across database, ETL, data warehouse or data-quality testing, with demonstrated experience leading complex QA delivery.
- Proven experience leading QA teams and providing technical direction across complex data environments.
- Strong hands-on experience with Databricks, advanced SQL and Python, including PySpark and Pandas.
- Strong SQL capability including window functions, complex joins and CTEs for large-scale data validation.
- Strong experience with Databricks Notebooks and cluster management.
- Strong understanding of ETL processes, data pipelines, data warehouse concepts and end-to-end data validation.
- Experience validating data across Databricks Medallion Architecture, including Bronze, Silver and Gold layers.
- Experience working within Azure or AWS environments.
- Experience developing test strategies, establishing QA governance and assessing release readiness.
- Experience with Jira and working within Agile/Scrum delivery environments.
- Proven ability to introduce and improve data-testing processes, standards and automation practices.
- Strong analytical and problem-solving skills with the ability to operate across complex data environments.
- Strong communication and stakeholder management skills with experience working across global teams and engaging senior stakeholders.
Advantageous Experience
- Experience with CI/CD pipelines and automated testing within data engineering environments.
- Experience with Git-based version control.
- Experience developing or supporting automated data-testing frameworks.
- Exposure to enterprise-scale data transformation or migration programmes.
Additional Requirements
SMBC’s employees participate in a hybrid workforce model that provides employees with an opportunity to work from home, as well as, from an SMBC office. SMBC requires that employees live within a reasonable commuting distance of their office location. Prospective candidates will learn more about their specific hybrid work schedule during their interview process.
SMBC provides reasonable accommodations during candidacy for applicants with disabilities consistent with applicable federal, state, and local law. If you need a reasonable accommodation during the application process, please let us know at accommodations@smbcgroup.com.