ENGAGING WITH COGNITIVE LEVELS: A PRACTICAL

APPROACH TOWARDS ASSESSING THE COGNITIVE

SPECTRUM IN MATHEMATICS
Ryna du Plooy, Caroline Long
University of Pretoria

In the interest of further progression beyond the Intermediate Phase, conceptual

mastery of various crucial mathematical ideas is necessary. If teaching, learning and
assessment remain on the factual recall and operational efficiency levels,
progression to the more advanced levels of mathematical activity is severely
restricted. In this paper we advocate the integrated assessment of conceptual
understanding, across the dimensions of mathematical understanding and on
different cognitive levels. We propose a framework for teachers to use in constructing
classroom assessments that accommodate the dimensions of understanding at various
cognitive levels, to prepare learners for the Senior Phase.
FOCUS AND SIGNIFICANCE OF THE PAPER
This paper is the result of du Plooy’s research quest for a practical and reliable way to
comply with the CAPS requirement to assess mathematics at different cognitive
levels. The study is supervised by Long. In this study some principles informing
mathematics teaching and learning design which use specific metacognitive strategies
in division at the Intermediate Phase (Grade 6) are investigated.
What emerged while developing the research instruments for the research design was
the necessity to enter into the intervention phase with a clear picture of the
participants’ competency. Firstly, a baseline assessment had to be set enabling a
diagnosis of the existing voids in the participants’ understanding of the multiplicative
structures in real life situations. Secondly, a pre-intervention assessment and a post-
intervention assessment had to be set, if the effect of using the proposed
metacognitive strategies was to be investigated.
The insights in this paper originated from informal action research in classroom
practice and were theoretically grounded through a formal literature review. The
practical application of this research finding, culminating in a framework, is that
teachers can use this framework to set and evaluate their own assessments according
to the requirements of the Intermediate Phase curriculum, for baseline, diagnostic,
formative and summative assessment, but particularly to understand how assessment
of cognitive levels is done practically within a real classroom setting.

THEORETICAL ASPECTS OF THE FRAMEWORK

A number of theoretical and practical aspects were taken into account while
designing the assessments for the study:
 • The South African mathematics curriculum. The Curriculum and Assessment
Policy Statement (CAPS) (Department of Basic Education (DBE), 2012) was
the point of departure in the study;
• The CAPS (DBE, 2012) spells out affective and cognitive goals for
mathematics learning. This research is situated in the cognitive domain of
mathematics teaching and learning and therefore had to take into account the
curriculum requirement of ascribing cognitive levels in assessment (DBE ,
2012, p. 296);
• The theoretical antecedents for the cognitive levels used in CAPS were studied
in order to apply them in setting the assessments. Similarities were found
between the approach in CAPS and the revised form of Bloom’s taxonomy for
learning, teaching and assessing (Anderson & Krathwohl, 2001; Anderson,
2002). In the revised taxonomy, Anderson and Krathwohl (2001) describe
Bloom et al.’s original categories (factual knowledge, conceptual knowledge
and procedural knowledge) in action words, and add the metacognitive
category of knowledge to the original three categories;
• Mathematics is a complex subject which requires an intricate process of
teaching and learning, and multidimensional understandings (Usiskin, 2012)
such as understanding the various mathematical representations, the properties
of mathematical concepts, the application of operations within certain
problems and the understanding of algorithms or methods. Assessment would
therefore need to be articulated according to these dimensions of
understanding, in addition to the levels of understanding proposed by
Anderson and Krathwohl (2001).

THE RELATED LITERATURE

For the purpose of this paper, we focus primarily on the literature regarding cognitive
levels of mathematics understanding with a secondary focus on a single selected
theory of mathematics understanding, as follows:
Cognitive levels of mathematics understanding
Apart from the different mathematics areas, the CAPS (DBE, 2012, p. 296) also
describe four cognitive levels at which assessment has to be conducted. These levels
are: knowledge (25%), routine procedures (45%), complex procedures (20%) and
problem solving (10%). The four cognitive levels used in CAPS, correspond directly
with the Subject Assessment Guidelines of the 1999 TIMSS taxonomy of categories
of mathematical demand (Stols, 2013, p. 13). The cognitive categories used in this
international test need to be interpreted carefully if they are to be applied in
classroom assessment.
For the mathematics teachers, as the users of CAPS, these categories leave room for
individual interpretation when an assessment is set. We therefore identified a need for
clarity about the concept of cognitive levels as it is explained in the literature.
Linn (2002, pp. 28-37) uses the term “level of cognitive demand” for what CAPS
terms “cognitive levels”. He discusses the approach taken by the International
Assessment of Educational Progress (IAEP), using the three cognitive levels
conceptual understanding, procedural knowledge and problem solving. He points out
the similarities between the IAEP approach and the five cognitive process categories
called performance expectations in TIMSS 1995, namely understanding, routine
procedures, and problem solving, investigating and communicating.
It was however Anderson and Krathwohl’s (2001) revision of Bloom’s taxonomy of
educational objectives, that helped the first author to conceptually delineate cognitive
levels in assessment. Of the four cognitive categories, namely factual knowledge,
procedural knowledge, conceptual knowledge and metacognitive knowledge, the first
three have been used in the assessment matrix in Table 1 (Dimensions of
understanding and levels of understanding, du Plooy, 2014). The term “factual recall”
is used for the first level, meaning “to bring to the fore, knowledge aspects that have
been stored in the learners’ memories”. For the second level “procedural efficiency”
is used, adopting Hiebert and Carpenter’s (1992) description of procedural
knowledge as “a sequence of actions…the manipulation of written symbols in a step-
by-step sequence” (p. 78). The third level is termed “conceptual grasp”. This use of
the term is in direct contrast with Feuerstein’s (Feuerstein & Rand, 1974, Feuerstein
et al, 2006) construct of “episodic grasp of reality”, where the latter refers to the
incidental and isolated experience of reality, unrelated in time and space to other
experiences. The construct “conceptual grasp” as used in this study denotes the
interrelatedness of an idea with other ideas to form a coherent mental unit, which can
be conceptualized as one concept, but large enough to contain different sub-concepts.
“Rate” as a mathematical concept is a good example, with speed, unit price,
population density and so on, as sub-concepts (see par 3.2).
Usiskin’s theory of mathematical understanding
Mathematical activity, according to Usiskin (2012, pp. 2-3, 19) consists of concepts
and problems. Within a mathematical problem, several concepts would come to the
fore, as is demonstrated in this paper. Usiskin explains that a concept can be used as
the unit of analysis, and it is of such a nature that it can be analysed systematically
(Usiskin, 2012, p. 15). Usiskin’s perspective applies to the research of a concept, and
when I interpret it from a teaching point of view, a concept may then be seen as a
mathematical unit that can be taught and learned systematically.
Furthermore, Usiskin distinguishes five dimensions 1 (Usiskin. 2012) that constitute
mathematics understanding, of which I use four in my assessment matrix of
understanding in Table 1 du Plooy, 2014), namely the skill-algorithm understanding,
the property-proof understanding, the use-application understanding and the
representation-metaphor understanding. In this paper I use this distinction as
described by Usiskin (2012, pp. 4-9), as follows:
• The skill-algorithm dimension or procedural understanding essentially entails
knowing how to get an answer.
• The property-proof dimension involves the identification of the mathematical
properties that underlie why the particular method of obtaining the answer
worked.
• The use-application dimension implies knowing when to apply a specific
operation.
• The representation-metaphor dimension requires the ability to communicate a
mathematical concept by means of an appropriate representation.
The importance of mathematics learning on all cognitive levels
Pantziara and Philippou (2011) draw upon existing studies to make a broad
distinction between procedural and conceptual knowledge of mathematics (pp. 61-
83). Their research reveals that learners have a better command of some of the
problematic mathematical areas like fractions if conceptual knowledge has been
developed alongside procedural knowledge. Voutsina (2011, p. 196) describes the
relationship between procedural- and conceptual knowledge, and problem solving
skills as an iterative process. She found that changes in young children’s problem
solving behaviour stem from the dynamic interaction between procedural and
conceptual knowledge.
Inferring from the literature and du Plooy’s own experience it seems entirely
plausible that failure to make the cognitive transition from knowledge and routine
procedures to complex procedures and problem solving, may account for
deteriorating mathematic competence at the more senior levels.
Although factual knowledge and procedural efficiency are important constituents of
mathematics proficiency, the transition to the more advanced cognitive levels does
not automatically follow on the direct recall of mathematical facts or even on
computational efficiency (arriving at the correct answer) in mathematical procedures.
This transition rather hinges on true and multidimensional understanding of
mathematics concepts. The teaching, learning and assessment processes relating to
the conceptual grasp of mathematical constructs are therefore crucial for mathematics
progression.

1
The fifth dimension, the cultural-historical dimension, is omitted, as it is not relevant in the context of this study.
Subsequently, mathematics assessments have to be created in such a way that they
provide for the demonstration of conceptual grasp in addition to factual recall and
operational efficiency.

AN ILLUSTRATION OF THE METHOD USED IN THIS PAPER

In setting the assessments it was useful to evaluate test items against the four
dimensions of understanding, thus ensuring representivity across the spectrum of
mathematics understanding. Furthermore, in the practice of doing this, the need was
soon experienced to express, within a specific dimension of understanding, the level
of understanding needed to succeed in solving a mathematical problem.
A matrix in Table1(du Plooy, 2014). was subsequently developed according to which
test items could be plotted, not only in terms of its dimension of mathematical
understanding, but also on the cognitive level that the problem requires. In this matrix
Usiskin’s (2012) four selected dimensions of understanding are juxtaposed with the
first three cognitive levels of Anderson and Krathwohl(2001) as explained above,
resembling the IAEP Mathematics Framework for 9- and 13-Year-Olds (Linn, 2002,
p.35). Anderson and Krathwohl’s metacognitive level is not used, since it is not used
in the CAPS. The use of this matrix is demonstrated on a test item based on a real-life
problem as it would be posed to a Grade 6 learner, as follows:
“Ms Buti drove from Bela-Bela to Bethulie. She started from Bela-Bela at 9:30 am,
with the odometer of her car on 88 888 km. She arrived in Bethulie at 16:30, with the
odometer on 89 525 km. What was the average speed of Ms Buti’s journey?”
The main concepts involved in the above problem can be set out as follows:
Average speed

Distance Time

“Analogue time” and “digital time” are additional concepts without which the
problem cannot be solved, because the problem statement makes use of both.
In the example above, we question whether the factual recall of the term “average
speed” on its own can ensure the successful solution of the problem. Here much more
understanding is required. We now analyse the above mathematics problem
according to the dimensions of understanding (Usiskin, 2012), relating to each
concept to the level of understanding that it requires. Broad guidelines are given
below as an indication of some possible plotting of concepts onto the matrix, but
through an interactive session we shall enrich this list and amend it with more content
on each level:
The concept “average speed” requires multidimensional mathematical understanding
on different cognitive levels:
 a. To understand the meaning of “average speed” and the elements it entails
Cognitive level: factual recall
Mathematical dimension: representation-metaphor
b. To understand the elements that “average speed” entails
Cognitive level: factual recall
Mathematical dimension: property-proof
c. To understand which operation to use to calculate the average speed
Cognitive level: factual recall, conceptual grasp
Mathematical dimension: use-application
d. To understand how to calculate “average speed”
Cognitive level: operational efficiency
Mathematical dimension: skill-algorithm
The concept “distance” needs multidimensional understanding:
a. To understand the role of “distance” in calculating average speed
Cognitive level: factual recall
Mathematical dimension: representation-metaphor
b. To understand which operation to use in calculating the distance
Cognitive level: conceptual grasp
Mathematical dimension: use-application
c. To understand how to calculate “distance” correctly
Cognitive level: operational efficiency
Mathematical dimension: skill-algorithm
The concept “time” needs multidimensional understanding:
a. To understand the role of “time” in calculating average speed
Cognitive level: factual recall
Mathematical dimension: representation-metaphor
b. To understand which operation to use in calculating the time using the
available information
Cognitive level: conceptual grasp
Mathematical dimension: use-application
c. To understand how to calculate “time” correctly
Cognitive level: operational efficiency
Mathematical dimension: skill-algorithm
Following such an analysis, we can, by way of a practical exercise with participants,
map the above mathematics problem on a matrix according to the dimensions of
understanding relating to each concept (Usiskin, 2012), at the level of understanding
that it requires, as follows:
Dimensions of understanding (Usiskin, 2012)
Use / Skill / Representation Property /
Levels of application algorithm / metaphor proof
Factual recall Factual recall Factual recall Factual recall
under- (the rule (eg steps in the (the meaning of (attributes of
standing applicable to the division average speed) average speed)
(Adapted calculation of calculation)
from average speed)
Anderson Operational Operational Operational Operational
& efficiency efficiency efficiency efficiency
(calculating (formulating an
Krathwohl,
towards a answer km/h)
2001) correct answer)
Conceptual Conceptual Conceptual Conceptual
grasp grasp grasp grasp
(this situation (e.g. what rate
requires entails)
division)

Table 1: Dimensions of understanding and levels of understanding (du Plooy, 2014 ©)

The above matrix is proposed as a tool in ensuring the representivity of an assessment

item across the spectrum of mathematics understanding, on all cognitive levels
required for thorough understanding of a concept. It has the potential of extending the
present context to a more inclusive concept like “rate”. In fact, the transition to the
Senior Phase and its increased emphasis on more abstract concepts, make the
conceptual grasp of the idea of “rate” an absolute necessity.

REFERENCES
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching and assessing: A
revision of Bloom’s taxonomy of educational objectives: Complete edition. New York: Longman.
Anderson, L. W. (2002). Curricular alignment: A re-examination. Theory into Practice, 41(4), 255-260.
Retrieved from http://www.unco.edu/cetl/sir/stating outcome/documents/Krathwohl.pdf
Department: Basic Education South Africa. (2012). National curriculum statement (NCS). Curriculum and
policy statement: Intermediate Phase. Grades 4-6. Mathematics. Pretoria, South Africa: Author.
Feuerstein, R. & Rand, Y. (1974). Mediated learning experiences: An outline of the proximal etiology for
differential development of cognitive functions. In Journal of the International Council of Psychology.
9/10. 7–37.
Feuerstein, R., Falik, L. H., & Feuerstein, R. Jnr. (2006). Definitions of essential concepts & terms, a
working glossary. Jerusalem: The International Center for the Enhancement of Learning Potential.
Hiebert, J. & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A. Grouws (Ed.),
Handbook of research on mathematics teaching and learning: a project of the National Council of
teachers of mathematics. 65-97. New York: Macmillan.
Linn, R. L. (2002). The measurement of student achievement in international studies. In A. C. Porter & A.
Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement.
Washington: National Academy Press.

Pantziara, M. & Philippou, G. (2011). Levels of students’ “conception” of fractions. Educational Studies of
Mathematics, 79. 61-83. Springer Science and Business Media.
Schoenfeld, A. H. (1994). Mathematical thinking and problem solving. London: Routledge.
Stols, G. (2013). An investigation into the opportunity to learn that is available to Grade 12 mathematics
learners. South African Journal of Education, 33(1), 1-18. Retrieved from
www.ajol.info/index.php/saje/article/viewFile/85005/74976
Usiskin, Z. P. (2012). What does it mean to understand some mathematics? 12th International Congress on
Mathematical Education held in Seoul, Korea, 8-15 July 2012, 502-521. Seoul, Korea: COEX.
Vye, N. J., Delclos, V. R., Burns, M. S., & Bransford, J. D. ( 1988). Teaching thinking and problem solving:
illustrations and issues. In R. J. Sternberg & E. E. Smith, The psychology of human thought. 337-366.
Cambridge: University Press.
Voutsina, C. (2011). Procedural and conceptual changes in young children’s problem solving. Educational
Studies of Mathematics, Vol. 79, 193-214. Springer Science and Business Media.