Assessment IN learning: What, Why, When and How?

Date: 29 August 2009, Saturday
Time: 10 am - 12 noon
Presented by: Associate Professor Sivakumar Alagumalai
Click HERE to see synopsis

Chanced upon this talk while surfing the web for something else... well, What's "Assessment IN Learning"? I wonder. In fact, am pretty familiar with "Assessment of Learning" and "Assessment for Learning"... well, as it summarises in the website:

Gibbs (1999) highlight that assessment can happen at three crucial stages of learning. Assessment AS learning, Assessment FOR learning and Assessment OF learning can occur either as standalone processes or be implemented concurrently to gauge the learning cycle.

• Assessment as Learning – occurs when students reflect on and monitor their progress to inform their future learning goals.
• Assessment for Learning – occurs when teachers use inferences about student progress to inform their teaching.
• Assessment of Learning – occurs when teachers use evidence of student learning to make judgements on student achievement against goals and standards.

It's pretty obvious that Students are central in Assessment AS Learning - when they are the ones who handles the monitoring/review processes whereas Assessemnt FOR & OF learning is largely managed from the teachers - when often, we would bring in terms like "Formative Learning" and "Summative Learning" respectively.

Alagumalai (1999, 2005, 2008) highlighted that assessment IN learning is equally or more important that just assessment as/for/in learning. He advanced that ‘Assessment in Learning’ occurs when teacher and student(s) conjointly set learning targets and milestones in the learning process instituting appropriate diagnostic / remediation / enrichment processes. This includes responding to any cue or feedback provided during the learning process and formulation of strategies to enable transfer of learning for future use. Alagumalai et al., (2009) highlight assessment in learning is pivotal for pattern recognition and novel problem solving, and feedback facilitates and enhances the extension of situated and tacit knowledge.

Learnt an analogy between FORMATIVE and SUMMATIVE assessment, which is nicely summed up by Robert Stake summed up:
When the cook tastes the soup, that's FORMATIVE;
When the guests taste the soup, that's SUMMATIVE.
Click HERE to see more.

Isn't that very real?
Teacher, being the classroom instructional leader, is the COOK while students are the dishes to be prepared. How good a dish depends on the quality of the ingredients, the supporting hardware (e.g. ovens, pots and pans) and very importantly, the skills and experience of the cook!

• Ingredients are equivalent to the learning materials that we prepared for and used by the students. Quality matters. Choice matters. For instance, we talked about using questions to scaffold students' learning. Did we craft the appropriate kind of questions to elicit the desired responses? The choice of ICT resources - did we use choose the appropriate kind of resources to bring about the intended kind of learning experiences? E.g. while we wanted students to carry out investigation, would something that's dynamic and allow them to manipulate serve better than just a video clip to illustrate 'investigation'?
• The supporting hardware, I would consider them as the learning environment. The set-up that helps bring about the process. For instance, accessibility to technology means (e.g. online portal, computers)
• Last not not least, the most important factor - the teacher factor - the competencies! Without this, all fail! (of course, there's another school of thought says, students perform well under the worst teachers because they will seek external help as the only means to survive! Hahaha...). In fact, "Teacher Competency" is also emphasized in the Teachers' Mass Lecture this year, by Ms Ho Ping (DGE) as well as Michael Barber (will talk about it in another blog post).

In fact, Siva illustrated the 3rd point through another case study carried out (I think, in Australia): When surveys were carried out with teachers as well as students on 3 areas: Content Expertise, PCK (Pedagogical Content & Knowledge) and Assessment views. Interesting but "logical" finding:

• Teachers were asked the topics they find (i) easy to teach (ii) OK to teach (iii) Hard to teach. Similarly, students were asked the topics that they find (i) easy to learn (ii) OK to learn (iii) hard to learn.
• Findings show that there's high correlation! (more than 0.9!). In fact, that links teachers' competency to how well students learn too! For a topic that the teacher finds easy to teach, of course, students benefit most - and probably there's where enduring understanding comes in - assuming that the teacher is able to help the student to connect the content beyond the curricular context. Similarly, for something that the teacher is not strong at (or in the first place, did not learn well), then of course, the same applies to the students!
• Fully agree with this, as I could find myself going through these 3 modes too! (as a learner & as a teacher).

It's an interesting lecture that comes with several provocating questions, which set us to think greater in depths...

Siva highlighted several paragraphs in the OECD (2009) Education Today: The OECD Perspective: (p44) E-learning has not yet revolutionised learning and teaching in higher education systems: The current immaturity of on-line learning is demonstrated by low adoption of content management systems. This refers to electronic content being split into “learning objects”, to be manipulated and reconstituted for multiple pedagogic purposes: only 6.6% of those responding to the UK-based Observatory on Borderless Higher Education (OBHE) survey of 122 Commonwealth institutions reported institution-wide adoption in 2004. ICT has had more impact on administrative services than on the fundamentals of teaching and learning.

Well, my immediate reactions to the above (text that I underlined) are....

• In fact, online learning begins with adoption of content management systems (CMS), which is the strategy I think Singapore schools adopt.
• While LMS/CMS were introduced in the late 90s, it's a good-to-have for schools, and mostly taken up by schools that were gungho to give it a try and being ahead of others. To some extent, I think it's evolved from CD-ROMs, which are less accessible. Schools start to leverage on the internet acccesibility when of bandwidth becomes a standard provision. by the end of the 1st IT Masterplan. To some extent, it was a novelty in its early days - as an alternative to richer content for students. As far as schools have an option, there's always a choice to consider that later (i.e. being pushed down the priority list).
• I think, what drives high adoption of CMS (and the use of LMS) is really the pandemic, SARS. It's an awakening call that technology would be one key vehicle to leverage on to enable learning continues to take place despite of school closure. From that point onwards, all secondary schools were expected to subscribe or provide a means to ensure learning could take place without attending classes physically. Gradually, primary schools also follow... as (I guess) the industries uncover high potential in CMS and recognises the primary levels offer a big market. So, all these leads to high adoption nationwide.
• High adoption is only one milestone to the entirety. How CMS is being incorporated into the learning environment (the entire ecosystem) is another big area to look at. Does it serve as a passive platform or resource? Merely dishing out clips after clips, questions after questions (from its database)? Does it really have an active role apart from being just a supplementary resource?
• It's no doubt that technology impact is more observable in administrative services - but many a time, its role is no other than a productivity tool - from basics tasks to enable efficient and quick references and retrieval to the higher end tasks like analysis and use of algorithms to generate scenarios and facilitate prediction.
• On the other hand, looking at what we are trying to strive at... at least as of what's spelt out in the 2nd & 3rd ICT Masterplans, technology is powerful when it's well harnessed. Yes, I use the word harness as it's still pretty vague how we could really use it - I guess, what we have uncover is only the tip of the iceberg. There could be more effective or better ways we could use technology to bring about more effective learning or acquisition of skills (I'm referring to the 21st century skills, life skills). How to? What strategies to employ? Still largely in the blackbox... though we are on the way to undercover bit by bit. I think I'm fortunate as my current working environment looks at providing the necessary features & conditions that's conducive for such uncovering processes to take place.

Siva also brought our attention to PISA (Programme for International Student Assessment). Well, heard about PISA last year, that both Science and Maths units in CPDD seem to be looking into this.

PISA "... to examine how well they (students) can extrapolate from what they have learned and apply their knowledge in novel settings, both in school and non-school context..."... focus... "increasing concerning with what students can do with what they learn and not merely with whether they have mastered specific curricular content"

Just a quick thought - yes, it's something that we are striving to achieve - students being able to apply the knowledge they acquired or the skills they mastered beyond the curricular context (when the knowledge and skills are introduced to them). As Siva shared his experience about learning the various kinds of 'energies' in the different sciences however these were all presented to the learner in a compartmentalised manner instead of 'linking' them up to show how energy exists in different context! This brings out another point, the importance of providing students with the Conceptual lense where they could see what they learn beyond the learning context, and they also learn about its real world application (Hey! Isn't this what we are doing at SST now?)

Another question that sets us thinking... Is what the examination system emphasize in congruence to the expectation of the society/industry (i..e beyond the national exam)?

• To-date, national exams focus on individual's ability and performance; whereas collaboration is out of the assessment picture.
• So, what message is the exam system sending to the school - making sure the students are able to tackle problems and tasks on their own? (since the exam does not award marks for collaboration!).
• On one hand, we are emphasizing cultivating in students the 21st century skills; why is it not reflective in our exam system? (OK, this is something beyond us... and it's been debated over and over again... let us see when will the "CHANGE" take place).
• Of course, at the back of the school's mind would be: We recognise the importance of the 21st century skills (that are not emphasized in the pen-and-paper exam) and we are willing to deploy/stretch our resources in these areas - however, if students do not perform in national exam, will we be penalised for not directing our attention and resources to the 'right' place? I wonder.

Next question... that always arises when we question the accuracy and validity of assessment (especially during exams) - what's the difference between 49 and 50 marks? Is that 1 mark difference significant? And why are we so sure that one who scored "50" would have met the expectation (which is reflected as the passing criteria) while "49" does not? >> A score is different from a measure.

Another interesting thing shared is the dynamics amongst the various entities in the "ecosystem" that attribute to the assessment directly & indirectly. Managed to capture only part of the diagram...

Also found something similar in the web: Click HERE to see.