In this episode, we speak to Senior Performance Manager Lorna Power who summarises her recent assessment of UK Aid Match education projects and what intervention techniques appeared to be most effective. Read Lorna's full assessment here: https://www.ukaidmatch.org/wp-content/uploads/2021/07/Rapid-Evidence-Review-Education.pdf?utm_source=Podcast&utm_medium=Podcast&utm_campaign=Education+RER&utm_id=The+Learning+Post
Read Lorna's full assessment of education projects
Ben Anderson (00:10):
Welcome to the Learning Post, a podcast dedicated to sharing insights from the UK Aid Match and UK Aid Direct funds. My name is Ben Anderson, and I'm the Communication Specialist for UK Aid Match and your host for this episode. In this episode, I speak to Senior Performance Manager for UK Aid Match, Lorna Power, who has conducted a rapid evidence review of previous UK Aid Match education projects to see which intervention techniques have been particularly effective. A rapid evidence review referred to as an RER throughout the podcast is a study based entirely on UK Aid Match project documentation intended to produce fast, actionable results. When you started this RER, what were you looking to find out? And what evidence did you use?
Lorna Power (00:58):
So when we first decided to do this type of RER on education projects, what we were really trying to find out is looking at projects, look at what they've done and trying to find out what works and what doesn't work. This was simply so that we could generate the learning and share the learning with any grant holder, any future grant holder, who was thinking of implementing a very similar education project. Maybe I just need to clarify what we mean by education projects, because actually all of our Match projects have some element of education.
Lorna Power (01:31):
A lot of them will have adult education and training elements. But the projects that we were looking at for this particular RER were very much education projects that were focus focused on the formal education system. So working with schools and working with teachers, trying to reform and improve that formal education. And most of them were working in what we call compulsory basic education, which is either pre-primary or primary education.
Lorna Power (02:01):
So it was very focused. We're very dependent on evidence that is collected from our grant holders. So all the evidence that we used for this RER was evidence that had been shared with us by our grant holders. The evidence we used for this particular RER were the evaluations that they had conducted at the end of the project.
Lorna Power (02:31):
This was evidence sort of looking back, reflecting on the whole of the project. For most of them, it was 36 months. There were two types of evaluations. One was the internal evaluation conducted by the grant holder themselves. What we call the project completion document. The other one was the independent evaluation conducted on projects by external organisations. What we were able to do was to sort of look at both of those and compare it and extract the lessons learned both lessons about things that had worked and hadn't worked.
Ben Anderson (03:04):
I guess, this kind of exercise highlights how important those evaluations that grant holders undertake and the data they collect.
Lorna Power (03:12):
Absolutely, Ben. Yeah, I think because we are very much dependent on the quality of the evidence I did to us by grant holders because we don't collect our own. So if we don't have good quality evidence from grant holders, what it means is that we can't sort of collate and aggregate and share good quality evidence about what works and what doesn't for any future grant holders. Those evaluations are really very important.
Ben Anderson (03:37):
While you were going through this evidence, what kind of conclusions were you able to come through by looking through previous education projects?
Lorna Power (03:43):
There were a lot of conclusions actually. Because of the sample that we were looking at, as I said, it was very much about formal education, working with teachers, reforming school systems, which is the FCDO priority. We had quite a small sample, but what was interesting is that there were a lot of very common issues that emerged from the evidence of that small sample. I think if anyone's really interested in looking at all of the conclusions, maybe go to have a look at the RER itself.
Lorna Power (04:13):
But I think for me, the three main conclusions that had the most value for me and I think for future grant holders, the first one I think is what we call a whole school approach. So all of these grant holders at the end of the project were saying, it's really important if you are trying to reform education systems, if you are working with schools and teachers that you work with the whole school. What some of these projects had done is selected to work with just a few teachers in the school.
Lorna Power (04:43):
And they thought that if we just work with these few teachers and we'll train up these few teachers, and then they can share the training with all the other teachers in the school. Generally, this didn't happen. They trained up the few teachers, but then because they hadn't put any systems in place for those teachers to train and all the other teachers, it just didn't happen.
Lorna Power (05:04):
So what it meant was within those schools, they never reached what we call critical mass. So there was never enough teachers doing the same thing to tip over into everyone for implementing this new practice. So all of them recommended in future working with the whole school, all the teachers in the school, because then you'll reach that critical mass. The other thing they found was that because they weren't working with all the teachers, they were just working with some, it tended to create a little bit of resentment within the school.
Lorna Power (05:36):
It divided the school a little bit with the teachers that had been trained and received the new resources and those that hadn't. The other important thing about the whole school approach is don't just focus on the teachers. You do need to involve the school principals, the school management. Because if the school management know what you are expecting their teachers to do, they can support them. But if you don't, the school principals aren't involved and aren't aware, they don't know how best to support their teachers.
Lorna Power (06:04):
The second thing that emerged that I think is really important is about not just working with the school, but seeing the school, seeing education as a kind of a whole community effort. Children won't go to school if their parents don't make them go to school. So it's really important in any education intervention about working with parents and the community to ensure they understand the value of education, that there is economic returns, social returns, not just for the community, but for the whole country. Often parents don't always understand that so you have to educate them about it.
Ben Anderson (06:42):
Those first two points would really impact the long term impact and what would happen after the project ended as well.
Lorna Power (06:50):
Yeah. So if you change people's mindsets about the value of education, if you change the way they think, that changes very long term. It's not just at change, that happens for the duration of the project. But if you change mindsets, that's very long term. And also, change mindsets for generation after generation. That kind of understanding of the value of education goes from generation to generation. The last thing that I would say emerged of the real value from this.
Lorna Power (07:17):
And for me, I definitely think this is really good learning for all projects, not just education projects. But it's about really focusing on your interventions in terms of not just what you do, but your geographic focus. What most of these education projects that we looked at were doing, they had a vast geographic spread. They were trying to work with too many schools and too many teachers.
Lorna Power (07:43):
So they were working with between 40 and 80 schools and a few teachers within each school. And it was just too big. They didn't have the staff, they didn't have the capabilities. So they were doing a lot of things at quite a superficial level. And so what they found were a lot of the independent evaluations were saying is because the geographic spread was too big and the intervention spread was too big, it negatively impacted the achievements that you could make.
Lorna Power (08:11):
And I think this really goes back. When I first started working in development, I was managing education projects. One evaluator said to me, "If the secret of French cooking is butter, butter, butter, the secret of development projects and education is focus, focus, focus." And that came across very, very strongly in the evaluations of these education projects that we looked at. So it's about working in depth, not working at breadth and if you can work in depth with, you know, a few schools and all the teachers in the schools and the communities around those schools, create those changes, and then you can start scaling up from there.
Ben Anderson (08:53):
Oh, that's really interesting. Thank you very much for your time, Lorna.
Lorna Power (08:56):
Thank you.
Ben Anderson (09:01):
To find out more about Lorna's work analysing previous UK Aid Match education projects, head over to the resources page on the UK Aid Match website and search rapid evidence review education. In the next episode, we'll be discussing the most important factors when determining effective adaptation to the COVID-19 pandemic. Here's a sneak preview.
Speaker 3 (09:23):
I think the most surprising aspect of this review was that there was only one factor that had really any significant impact.
Ben Anderson (09:33):
Don't forget to subscribe to the Learning Post so you don't miss it. Thanks for listening.