Whilst PoL in theory is a powerful tool to support professional learning and collect data about practice (e.g. assessment) we needed evidence about its efficacy in practice. This post provides an overview of the PoL in Early Years Pilot, explores its impact and draws out lessons for PoL's implementation in other settings.
As illustrated in the diagram below PoL consists of there components:
- A four step, iterative process which includes: the agreement of aspirational targets (Granules) and development of a shared understanding of what those granules are and what you would see happening in practice if they were being achieved (Practices); a period during which people make claims against the granules and practices; and mechanisms for reviewing progress against the granules.
- A framework of granules (aspirational targets) and practices (what you would see happening if each granule was being achieved)
- A digital tool (Lengo) which records details of claims made against the framework
For a fuller explanation of PoL in theory see https://halfbaked.education/what-is-point-of-learning-pol-moving-beyond-standardised-testing/.
In mid-2018 we worked with a commercial day nursery to explore PoL's impact. Initially we met with members of the management team and explained PoL (in theory). The three members of management and seven other staff in the nursery agreed to take part in the research. The focus was on using PoL to support staff development (rather than as an assessment tool).
Ethical clearance was obtained from the Open University's Human Research Ethics Committee and BERA guidelines on educational research were followed.
The process that was followed during this pilot varied from the ideal PoL process:
- For Step 1 three members of the management team came up with the granules with minimal input from other members of staff.
- For Step 2 it proved impossible for all the staff to meet to discuss the granules or practices because there was only one whole staff meeting every six weeks. The researchers therefore met individually with each member of staff to
- (a) ask how they felt about the granules and whether they wanted to suggest any other ones, and
- (b) discuss what one would see if each of the granules were being achieved. It was made clear that the descriptions of practices were meant to be exemplars rather than comprehensive.
- For Step 3 seven members of staff plus the nursery manager used Lengo to make claims against the framework that had been developed during Steps 1 and 2 - this took place over a 12 week period starting in August 2018.
- Step 4 included two elements: firstly the staff who were using Lengo to make claims talked with each other about what they were doing; secondly, one of the researchers had one to one interviews with each of the three managers and the seven staff who had used Lengo in December 2018/January 2019.
As is noted in Figure 2, the level of management engagement during the period when staff were making claims was very low:
- Whilst the centre manager (Chloe) said she had talked with staff about using PoL during this period, only one of the other members of staff could recall this having happened, and that was on an occasion when she raised PoL with Chloe.
- Chloe, who was acting as the Assessor (click here for an explanation of the different levels of claim), made four claims in total - all of which were given to one member of staff over a period of about five minutes.
- Although a summary of the data about the claims that had been made was emailed to the centre manager on a regular basis, and she had access to the full data on the system, both she and the senior manager both reported that no use had been made of the data.
Furthermore, relatively few claims were made over the 12 week period - with one member of staff (Molly) only having two claims in total. Sara, who made the most self-claims, had 26 claims in total (i.e. an average of three claims per week, though in practice most of the claims were made in concentrated blocks of time).
Given that the pilot had not followed the recommended process, the low level of management engagement during the period that staff were making claims, and the low level of claims made overall, it seemed unlikely that PoL would have had a noticeable impact within the setting.
Analysis of the data from the interviews with the management team seemed to confirm this. Sophie, who covered for the centre manager when she was on leave, noted that she had played no part in the implementation of PoL other than contributing to the discussion of the granules. Nonetheless Sophie felt that using PoL had helped staff understand what the management team expectations were about their practice - what she called 'recognition'.
Chloe, the centre manager, didn't feel that using PoL had had much impact, though she noted that it did allow staff to provide evidence to show that they were achieving the targets that she had set, that it enhanced their reflection on practice, and that it had helped increase their self-esteem.
The individual interviews with staff provided a more positive picture overall. In relation to the initial one to one discussions of the granules and practices (see Figure 4) many of them commented on how the targets reflected things that they already did as a normal part of their work, but that by having to explain what you would see if they were meeting each target they had to make explicit what previously had been tacit knowledge.
Making claims (see Figure 5) seemed to make them notice their own and each other's practice, and increased the extent to which they acknowledged or praised 'good practice', which made them feel good about themselves.
Using PoL also increased the level of discussion between staff about their practice (see Figure 6). Molly, felt that the initial discussions had been helpful for her as a new member of staff. More experienced staff felt it made them think and increased talk about the granules both in terms of quantity and depth.
Staff also noted that using PoL helped provide 'solid proof' that they were doing what the management wanted, and it extended their practice (see Figure 7). Interestingly, the impact of PoL being used in the nursery seemed to extend beyond those staff who were actually using it, and there were hints that even when they stopped making claims their engagement with PoL continued to have an impact.
However, these positive comments need to be balanced against the low level of claims made and the fact that most of the staff involved stopped using PoL when the number of children in the nursery increased (after the school holidays). Thus, using PoL was clearly not seen as a priority activity when staff were under pressure.
Overall, it would seem that despite the problems with how PoL was implemented in practice there is evidence that it did have some positive impacts on individual staff in terms of their self-esteem, reflection, and practice.
A number of other issues about the implementation of PoL emerged from this pilot (see Figure 8). Some of these are 'not new' - e.g. we know that senior management involvement is important for any initiative of this sort. Others, such as the advantages of using the web-based version of Lengo rather than the mobile app only emerged as a result of the practical experience of using the software.
Clearly this is a very small scale pilot study and further research is needed to more fully explore the potential of PoL, particularly as an alternative form of assessment.
I am particularly interested in exploring the use of PoL with secondary school students, with a focus on what might (misleadingly) be called '21st century' skills and attributes (e.g. creativity, problem solving, team work, empathy, resilience). Get in touch if you would be interested in collaborating on this ...
