Data Engineer
Department: Technology: Data Technology
Employment Type: Permanent - Full Time
Location: London or Manchester UK
Description
About us
We're Transform. We turn data and tech into competitive advantage and social impact for the world's most complex organisations, both public and private.
Organisations like Toyota Lexus, Anglian Water, the Cabinet Office, Transport for London, Public Health England, HM Courts and Tribunal Service, Santander, Department for Education and Domino’s.
We're not here to generate more business for ourselves. We're here to create startling, ambitious outcomes for clients - leaving them with the capabilities they need to take it from there.
We love creating real change. Making a big impact on people's lives. Optimising services, driving business growth or realising social change.
To do that, we built an inclusive, high-performance culture where the world's most talented people do the best work of their careers.
That's what gets us out of bed every day. We hope you feel the same.
We're part of the Next 15 group, a multi-national organisation spanning over 25 brands. Brands like ours, looking to stand out from the crowd as they deliver best in class work for their clients.
The Role:
The Data Engineer will be responsible for working as part of an Agile delivery squad with other data engineers and senior data engineers. Working to deliver data engineering on a range of data products for use in insight, reporting and analytics including AI, working closely with product owners, business analysts, data architects, data analysts, delivery projects managers and data scientists.
Key Responsibilities
* Delivering data engineering on a range of creation of data products, complex marketing and research processes
* Suggest any new technologies and approaches that improve what we do including the use of AI tools in data engineering if applicable
* Managing quality of own outputs
* Working with product owners, scrum masters, business analysts to understand needs, and best implement that with our available technology solutions
* Working with a range of data sources such as CRM, sales, research, HR data including structured and unstructured data.
* Building Single Customer Views or data products or Data Warehouses/Data Lakes to merge disparate data sets for analysis/reporting, marketing or sales activation
* Building and configuring data ingestion methods from APIs, flat files, streaming, public data sources and survey data
Skills, Knowledge and Expertise
* 3 – 4 years’ experience with Azure Data Factory, Python, Azure SQL, Databricks and Pyspark and a range of associated Python libraries for data processing and manipulation
* Experience with a range of database technologies including data lakes such as Azure SQL or SQL Server, Azure Data Lake
* Must have experience of using Azure Databricks and Medallion architecture
* Understanding of Azure DevOps and CI/CD processes
* Knowledge of data warehouse or data lakes with SCVs and building of data products for analysis and insight
* Knowledge of data ingestion of streaming data using Databricks, Deltalake and Pyspark
* Understanding and application of statistical methods
* Ability to work well as part of a team using the agile methodology
Desirable Skills:
* Azure Data Factory and Databricks in an Azure Cloud environment and use of Azure DevOps
* Knowledge of other ELT/ETL tools such as Informatica DEI/BDM and Talend would be beneficial
About us and our benefits
Our culture is built on:
* Being one team; looking out for each other as a team
* The world never stops changing and we’ll never stop learning
* Infinity & Beyond; we’re crazy ambitious, not afraid to fail along the way (it’s just falling with style)
* Integrity is everything; we’re honest with each other and with our clients
* Diversity is our superpower; everyone’s story is different, and we celebrate differences
* We’re all in; we take on tough, complex challenges with lots of moving parts and we’re committed to the work, our clients and each other
Our benefits include:
* Holiday entitlement, 28 days with the option to buy/sell up to 5 days
* Day off (on or in the week of) your birthday
* Pension eligibility, up to 5% matched contributions
* Private healthcare
* Life assurance
* Enhanced maternity and enhanced paternity and shared parental leave
* Cycle to work & electric car schemes
* Gym & retail discounts
* Regular social events/activities
* A range of other benefits from our flexible benefits package
At Transform we believe in a culture of inclusivity, we celebrate difference and believe that diversity makes our business more relevant, our work more meaningful and our people more empowered. We’re committed to equality of opportunity for all, and we actively seek applications from all ethnicities, orientations, beliefs, gender identities + those with neurodiverse traits and disabilities. We want you to have the best opportunity to show us who you are and what you can do. Please do let us know if you need any adjustments now or at any point in the recruitment process.
If this sounds like you, please get in touch to discuss in more detail.
#J-18808-Ljbffr