08 Hmef5053 T4

Topic How to Assess?
4 Essay Tests
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Identify the attributes of an essay question;
2. Explain the purpose of using essay test;
3. List the advantages and limitations of essay questions;
4. Identify those learning outcomes that are appropriately assessed
using essay questions; and
5. Construct well-written essay questions that assess given learning
objectives.
INTRODUCTION
In Topic 3, we discussed in detail the use of objective tests in assessing students.
In this topic, we will examine the essay test. The essay test is a popular technique
for assessing learning and is used extensively at all levels of education. It is
also widely used in assessing learning outcomes in business and professional
examinations. Essay questions are used because they challenge students to create
their own responses rather than simply selecting a response. Essay questions
have the potential to reveal students abilities to reason, create, analyse and
synthesise, which may not be effectively assessed using objective tests.
Copyright Open University Malaysia (OUM)

TOPIC 4 HOW TO ASSESS? ESSAY TESTS 81
4.1 WHAT IS AN ESSAY QUESTION?

According to Stalnaker (1951), an essay is a test item which requires a response
composed by the examinee usually in the form of one or more sentences of a
nature that no single response or pattern of responses can be listed as correct, and
the accuracy and quality of which can be judged subjectively only by one skilled
or informed in the subject. Though the definition was provided a long time ago,
it is a comprehensive definition. Elaborating on this definition, Reiner, Bothell,
Sudweeks and Wood (2002) argued that to qualify as an essay question, it should
meet the following FOUR criteria:
(a) The learner has to compose rather than select his or her response or answer.
In essay questions, students have to construct their own answer and decide
on what material to include in their response. Objective test questions
(MCQ, true-false, matching), on the other hand, require students to select
the answer from a list of possibilities.
(b) The response or answer the learner provides will consist of one or more
sentences. Students do not respond with a yes or no but instead have
to respond in the form of sentences. In theory, there is no limit to the length
of the answer. However, in most cases, its length is predetermined by the
demand of the question and the time limit allotted for the test question.
(c) There is no one single correct response or answer. In other words, the
question should be composed so that it does not ask for one single correct
response. For example, the question Who killed JWW Birch? assesses
verbatim recall or memory and not the ability to think. Hence, it cannot
qualify as an essay question. You can modify the question Who killed
JWW Birch? Explain the factors that led to the killing. Now, this is an
essay question that assesses students ability to think and give reasons for
the killing supported with relevant evidence.
(d) The accuracy and quality of students responses or answers to essay

questions must be judged subjectively by a specialist in the subject. The
nature of essay questions is such that only specialists in the subject can
judge to what degree responses (or answers) to an essay question are
complete, accurate and relevant. Good essay questions encourage students

82 TOPIC 4 HOW TO ASSESS? ESSAY TESTS
to think deeply about their answers that can be judged only by someone
with appropriate experience and expertise in the content area. Thus,
content expertise is essential for both writing and grading essay tests. For
example, the question List three reasons for the opening of Penang by the
British in 1789 requires students to recall a set list of items. The person
marking or grading the essay does not have to be a subject matter expert to
know whether the student has listed the three reasons correctly as long as
the list of three reasons is available as an answer key. For the question To
what extent is commerce the main reason for the opening of Penang by the
British in 1789?, a subject matter expert is needed to grade or mark the
answer to this essay test question.
4.2 TYPES OF ESSAY TESTS

Generally, there are two types of essay tests coursework essay and examination
essay which are used in educational institutions.
(a) Coursework Essay

This may consist of the following:
(i) Essay Outlines

This is a short summary or outline of a particular topic (perhaps about
500 words). It assesses the ability to organise material, construct
coherent arguments and select relevant information from a wide field
of study.
(ii) Standard Essays

These are full papers of anything from 1,000 to 2,500 words in length.
They assess the ability to describe, analyse the relationship between
ideas and events, give a coherent account of a topic, select and weigh
evidence in support of an argument, diagnose and suggest solutions
to problems, solve familiar types of problems, express critical
judgements and make comparisons.
(iii) Extended Essays

These are full papers of between 2,500 and 5,000 words in length.
They assess the ability to solve less familiar problems and to analyse
or critically evaluate less familiar materials. Hence, this type of essay
requires more extensive preparation or research.

Coursework essays are intended to assess students ability to research a

topic thoroughly and meticulously, and to handle a mass of material. They
evaluate students ability to answer at length on a chosen topic. Of
particular concern are the points the student makes, how he or she has
related them, the way in which his or her essay is organised, and the value
he or she attaches to different aspects of the topic. A coursework essay is a
written response and students are expected, for the most part, to use their
own words and communicate their ideas clearly and persuasively.
(b) Examination Essay

This is the short essay written as part of a formal examination. For example,
in a two- to three-hour examination, students may be required to answer
about three essay questions allotting about 3545 minutes per question. In
practice, there is much variation in the number of questions asked and the
duration of the examination. In some situations, there is a choice of
questions while in other situations there is no choice. The reason for
controlling choice is to ensure that students are examined over a
comparable range.
The discussion that follows focuses on the Examination Essay i.e. the use
of essay questions in examinations which are commonly closed-book
settings. We will discuss further about the Coursework Essay in Topic 5
under Projects.
ACTIVITY 4.1
Select a few essay questions that have been used in tests or
examinations. To what extent do these questions meet the criteria of an
essay question as defined by Stalnaker (1951) and elaborated by Reiner,
Bothell, Sudweeks and Wood (2002)?
4.3 WHY ARE ESSAY QUESTIONS USED?

Essay questions are used to assess learning because of the following reasons:
(a) Essay questions provide an effective way of assessing complex learning

outcomes. They allow one to assess students ability to synthesise, organise,
and express ideas, and evaluate the worth of ideas. These abilities cannot be
effectively assessed directly with other paper-and-pencil test items.

(b) Essay questions allow students to demonstrate their reasoning. These

questions not only allow students to present an answer to a question but
also to explain how they have arrived at their conclusions. This allows
teachers to gain insight into a students way of viewing and solving
problems. With such insight, teachers can detect problems students may
have with their reasoning process, and help them overcome these problems.
(c) Essay questions provide authentic experiences. Constructing responses is

closer to real life than selecting responses as in the case of objective tests.
Problem solving and decision-making are vital life competencies which
require the ability to construct a solution or decision rather than selecting
a solution or decision from a limited set of possibilities. In the work
environment, it is unlikely that an employer will give a list of four
options for a worker to choose from when the latter is asked to solve a
problem. In most cases, the worker will be required to construct a response.
4.4 DECIDING WHETHER TO USE ESSAY

QUESTIONS OR OBJECTIVE QUESTIONS
Essay questions should strive for higher-order thinking skills
The decision whether to use essay questions or objective questions in

examinations can be problematic for some educators. In such a situation, one has
to go back to the objectives of assessment. What kinds of learning outcomes do
you intend to assess? Essay questions are generally suitable for the following
purposes:
(a) To assess students understanding of subject matter or content; and
(b) To assess thinking skills that require more than simple verbatim recall of
information by challenging the students to reason with their knowledge.
It is challenging to write test items to tap into higher-order thinking. However,

students understanding of subject matter or content, and many of the other
higher-order thinking skills, can also be assessed through objective items. When
in doubt about whether to use an essay question or an objective question, just
remember that essay questions are used to assess students ability to construct
rather than select answers.

To determine what type of test (essay or objective) to use, it is helpful that you
examine the verb(s) that best describe the desired ability to be assessed (refer to
Topic 2). These verbs indicate what students are expected to do and how they
should respond. They serve to focus on the students responses and channel
them towards the performance of specific tasks. Some verbs clearly indicate that
students need to construct rather than select their answer (e.g. to explain). Other
verbs indicate that the intended learning outcome is focused on students ability
to recall information (e.g. to list). Perhaps, recall is best assessed through
objectively scored items. Verbs that test for understanding of subject matter or
content or other forms of higher-order thinking, but do not specify whether the
student is to construct or select the response (e.g. to interpret) can be assessed
either by essay questions or objective items.
ACTIVITY 4.2
Compare, explain, arrange, apply, state, classify, design, illustrate,
describe, name, complete, choose, defend and name. Decide which of
the verbs in the list above are best assessed by essay questions or
objective tests or both objective and essay questions.
4.5 WHAT ARE SOME LIMITATIONS OF ESSAY

QUESTIONS?
While essay questions are popular because they enable the assessment of higher-
order learning outcomes, this format of evaluating students in examinations has
a number of limitations which should be kept in mind.
(a) One purpose of testing is to assess a students mastery of subject matter. In

most cases, it is not possible to assess the students mastery of the complete
subject matter domain with just a few questions. Because of the time it
takes for students to respond to essay questions and for markers to mark
students responses, the number of essay questions that can be included in a
test is limited. Therefore, using essay questions will limit the degree to
which the test is representative of the subject matter domain, thereby
reducing content validity. For instance, a test of 80 multiple-choice
questions will most likely cover more of the content domain than a test of
three to four essay questions.

(b) Essay questions have limitations in reliability. While essay questions allow
students some flexibility in formulating their responses, the reliability of
marking or grading is questionable. Different markers or graders may vary
in their marking or grading of the same or similar responses (inter-scorer
reliability) and one marker can vary significantly in his or her marking or
grading consistency across questions depending on many factors (intra-
scorer reliability). Therefore, essay answers of similar quality may receive
notably different scores. Characteristics of the learner, length and legibility
of responses, and personal preferences of the marker or grader with regard
to the content and structure of the response are some of the factors that may
lead to unreliable marking or grading.
(c) Essay questions require more time for marking student responses. Teachers
need to invest a large amount of time to read and mark students responses
to essay questions. On the other hand, relatively little or no time is required
for teachers to score objective test items like multiple-choice items,
matching exercises, etc.
(d) As mentioned earlier, one of the strengths of essay questions is that they
provide students with authentic experiences because students are
challenged to construct rather than select their responses. To what extent
does the short time normally allotted to test affect student response?
Students have relatively little time to construct their responses and this
time limit does not allow them to give appropriate attention to the complex
process of organising, writing and reviewing their responses. In fact, in
responding to essay questions, students use a writing process that is quite
different from the typical process that produces excellent writing (draft,
review, revise, evaluate, etc.). In addition, students usually have no
resources to aid their writing when answering essay questions (dictionary,
thesaurus, etc.). This disadvantage may offset whatever advantage accrued
from the fact that responses to essay questions are more authentic than
responses to multiple-choice items.

4.6 SOME MISCONCEPTIONS ABOUT ESSAY

QUESTIONS IN EXAMINATIONS
Other than the limitations of essay questions discussed earlier, there are also
some misconceptions about this format of assessment:
(a) By Their Very Nature, Essay Questions Assess Higher-order Thinking

Whether or not an essay item assesses higher-order thinking depends on
the design of the question and how students responses are scored. An
essay question does not automatically assess higher-order thinking skills. It
is possible to write essay questions that simply assess recall. Also, if a
teacher designs an essay question meant to assess higher-order thinking
but then scores students responses in a way that only rewards recall
ability, that teacher is not assessing higher-order thinking. Teachers must
be well trained to design and write higher-order thinking questions.
(b) Essay Questions are Easy to Construct

Essay questions are easier to construct than multiple-choice items because
teachers do not have to create effective distractors. However, that does not
mean that good essay questions are easy to construct. They may be easier to
construct in a relative sense, but they still require a lot of effort and time.
Essay questions that are hastily constructed without much thought and
review usually function poorly.
(c) The Use of Essay Questions Eliminates the Problem of Guessing

One of the drawbacks of objective test items is that students sometimes
get the right answer by guessing which of the presented options is correct.
This problem does not exist with essay questions because students need
to generate the answer rather than identifying it from a set of options
provided. At the same time, the use of essay questions introduces bluffing,
another form of guessing. Some students are good at using various
methods of bluffing (vague generalities, padding, name-dropping, etc.) to
add credibility to an otherwise weak answer. Thus, the use of essay
questions changes the nature of the guessing that occurs, but does not
eliminate it.

(d) Essay Questions Benefit All Students by Placing Emphasis on the

Importance of Written Communication Skills
Written communication is a life competency that is required for effective
and successful performance in many vocations. Essay questions challenge
students to organise and express subject matter and problem solutions
in their own words, thereby giving them a chance to practise written
communication skills that will be helpful to them in future vocational
responsibilities. At the same time, the focus on written communication
skills is also a serious disadvantage for students who have marginal writing
skills but know the subject matter being assessed. If students who are
knowledgeable in the subject obtain low scores because of their inability to
write well, the validity of the test scores will be diminished.
(e) Essay Questions Encourage Students to Prepare More Thoroughly

Some research seems to indicate that students are more thorough in
their preparation for examinations using essay questions than in their
preparation for objective examinations such as those using multiple-choice
questions. However, after an extensive review of existing literature and
research on this topic, Crook (1988) concluded that students extent of
preparation is based more on the expectations teachers set upon them
(higher-order thinking and breadth and depth of content) than the type of
test questions they expect to be given in examinations.
ACTIVITY 4.3
Compare the following two essay questions and decide which one
assesses higher-order thinking skills.
(a) What are the major advantages and limitations of solar energy?
(b) Given its advantages and limitations, should governments spend

money developing solar energy?
SELF-CHECK 4.1
1. What are some limitations in the use of essay questions?
2. Suggest other weaknesses of using essay questions in

examinations.
3. List some of the misconceptions about essay questions.

4.7 SOME GUIDELINES FOR CONSTRUCTING

ESSAY QUESTIONS
When constructing essay questions, whether it is for coursework assessments or
examinations, the most important thing is to ensure that students have a clear
idea of what they are expected to do after they have read the question or problem
presented.
Below are specific guidelines that can help you improve existing essay questions
and create new ones.
(a) Clearly Define the Intended Learning Outcome to be Assessed by the

Question
Knowing the intended learning outcome is crucial for designing essay
questions. In specifying the intended learning outcome, teachers clarify the
performance that students should be able to demonstrate as a result of what
they have learned. The intended learning outcome typically begins with a
verb that describes an observable behaviour or action that students should
demonstrate. The focus is on what students should and should not be able
to do in the learning or teaching process. Reviewing a list of verbs can help
to clarify what ability students should demonstrate, thereby defining the
intended learning outcome to be assessed (refer to subtopic 4.8)
(b) Avoid Using Essay Questions for Intended Learning Outcomes that are
Better Assessed with Other Kinds of Assessment
Some types of learning outcomes can be more efficiently and more reliably
assessed with objective tests than with essay questions. Since essay
questions sample a limited range of subject matter or content, are more
time-consuming to score, and involve greater subjectivity in scoring, the
use of essay questions should be reserved for learning outcomes that cannot
be better assessed by some other means.

Example:
Learning Outcome: To be able to differentiate the reproductive habits of

birds and amphibians
Essay Question: What are the differences in egg laying characteristics

between birds and amphibians?
Note: This learning outcome can be better assessed by an objective test.
Objective Item:
Which of the following differences between birds and amphibians is
correct?
Birds Amphibians
A Lay a few eggs at a time Lay many eggs at a time
B Lay eggs Give birth
C Do not incubate eggs Incubate eggs
D Lay eggs in nest Lay eggs on land
(c) Clarity About the Task and Scope

Essay questions have two variable elements the degree to which the task
is structured and the degree to which the scope of the content is focused.
There is still confusion among educators as to whether more structure (of
the task required) and more focus (on the content) are better than less
structure and less focus. When the task is more structured and the scope of
content is more focused, two problems are reduced:
(i) The problem of student responses containing ideas that were not
meant to be assessed; and
(ii) The problem of extreme subjectivity when scoring student answers or

responses.
Although more structure helps to avoid these problems, how much and
what kind of structure and focus to provide is dependent on the intended
learning outcome that is to be assessed by the essay question. The process
of writing effective essay questions involves defining the task and
delimiting the scope of the content in an effort to create an effective
question that is aligned with the intended learning outcome to be assessed
by it (as illustrated in Figure 4.1).

Figure 4.1: Alignment between content, learning activities

and assessment tasks
Source: Phillips, J. A., Ansary Ahmed, & Kuldip Kaur. (2005). Instructional design
principles in the development of an e-learning graduate course. Paper presented at
The International Conference in E-Learning. Bangkok, Thailand
This alignment is absolutely necessary for obtaining student responses that

can be accepted as evidence that a student has achieved the intended learning
outcome. Hence, the essay question must be carefully and thoughtfully written in
such a way that it elicits student responses that provide the teacher with valid
and reliable evidence about the students achievement of the intended learning
outcome. Failure to establish adequate and effective limits for students answers
to the question may result in students setting their own boundaries for their
responses. This means that students might provide answers that are outside the
intended task or address only a part of the intended task. If this happens, then
the teacher is left with unreliable and invalid information about the students
achievement of the intended learning outcome. Also, there is no basis for
marking or grading students answers. Therefore, it is the responsibility of the
teacher to write essay questions in such a way that they provide students with
clear boundaries for their answers or responses.
SELF-CHECK 4.2
1. When would you decide to use an objective item rather than an

essay question to assess learning?
2. What is the difference between the task and the scope of an essay
question?

Example:
(Improving Clarity of Task and Scope of Essay Questions)
Weak Essay Question:

Evaluate the impact of the Industrial Revolution on England.
The verb is evaluate, which is the task the student is supposed to do. The
scope of the question is the impact of the Industrial Revolution on England.
Very little guidance is given to students about the task of evaluating and the
scope of the task. A student reading the question may ask:
(a) The impact on what in England? The economy? Foreign trade? A

particular group of people? (The scope is not clear.)
(b) Evaluate based on what criteria? The significance of the revolution? The
quality of life in England? Progress in technological advancements? (The
task is not clear.)
(c) What exactly do you want me to do in my evaluation? (The task is not

clear.)
Improved Essay Question:

Evaluate the impact of the Industrial Revolution on the quality of family life in
England. Explain whether families were able to provide for the education of
their children.
The improved question determines the task for students by specifying a

particular unit of society in England affected by the Industrial Revolution
(family). The task is also determined by giving students a criterion for
evaluating the impact of the Industrial Revolution (whether or not families
were able to provide for their childrens education). Students are clearer about
what must be done to evaluate. They need to explain how family life has
changed and judge whether or not the changes are an improvement for the
children.

(d) Questions that are Fair

One of the challenges that teachers face in composing essay questions is
that because of their extensive experience with the subject matter, they may
be tempted to demand unreasonable content expertise on the part of the
students. Hence, teachers need to make sure that their students can be
expected to have adequate material with which to answer the question
(Stalnaker, 1952, p. 520). In addition, teachers should ask themselves if
students can be expected to adequately perform the thought processes
which are required of them in the task. For assessment to be fair, teachers
need to provide their students with sufficient instruction and practice in the
subject matter required for the thought processes to be assessed.
Another important element is to avoid using indeterminate questions. A

question is indeterminate if it is so unstructured that students can redefine
the problem and focus on some aspect of it with which they are thoroughly
familiar, or if experts in the subject matter cannot agree that one answer is
better than another. One way to avoid indeterminate questions is to stay
away from vocabulary that is ambiguous. For example, teachers should
avoid using the verb discuss in an essay question. This verb is simply too
broad and vague. Moreover, teachers should also avoid including
vocabulary that is too advanced for students.
(e) Specify the Approximate Time Limit and Marks Allotted to Each Question
Specifying the approximate time limit helps students allocate their time in
answering several essay questions. Without such guidelines students may
feel at a loss as to how much time to spend on a question. When deciding
the guidelines for how much time should be spent on a question, keep the
slower students and students with certain disabilities in mind. Also make
sure that students can be realistically expected to provide an adequate
answer in the given and/or suggested time. Similarly, state the marks
allotted to each question so that students can decide how much they should
write to answer the question.

(f) Use Several Relatively Short Essay Questions Rather than One Long One
Only a very limited number of essay questions can be included in a test
because of the time it takes for students to respond to them and the time it
takes for teachers to grade the student responses. This creates a challenge
with regard to designing valid essay questions. Shorter essay questions are
better suited to assess the depth of student learning within a subject
whereas longer test essay questions are better suited to assess the breadth
of student learning within a subject. Hence, there is a trade-off when
choosing between several short essay questions or one long one. Focus on
assessing the depth of student learning within a subject limits the
assessment of the breadth of student learning within the same subject and
focus on assessing the breadth of student learning within a subject limits
the assessment of the depth of student learning within the same subject.
When choosing between using several short essay questions or one long
one, also keep in mind that short essays are generally easier to mark than
long essays.
(g) Avoid the Use of Optional Questions

Students should not be permitted to choose one essay question to answer
from two or more optional questions. The use of optional questions should
be avoided for the following reasons:
(i) Students may waste time deciding on an option; and
(ii) Some questions are likely to be harder which could make the
comparative assessment of students abilities unfair.
The issue of the use of optional questions is debatable. It is often practised,

especially in higher education, and students often demand that they be
given choices. The practice is acceptable if it can be assured that the
questions have equivalent difficulty levels and the tasks as well as the scope
required by the questions are equivalent.

Improving Essay Questions Through Preview and Review
The following steps can help you improve the essay item before and after you
administer it to your students.
PREVIEW (before handing out the essay question to the students)
Predict Student Responses

Try to respond to the question from the perspective of a typical student.
Evaluate whether students have the content knowledge and the skills
necessary to adequately respond to the question. After detecting possible
weaknesses of the essay questions, repair them before handing them out in the
exam.
Write a Model Answer

Before using a question, write model answer(s) or at least an outline of major
points that should be included in an answer. Writing the model answer allows
reflection on the clarity of the essay question. Furthermore, the model answer
serves as a basis for the grading of student responses. Once the model answer
has been written, compare its alignment with the question and the intended
learning outcome, and make changes as needed to assure that the intended
learning outcome, the question, and the model answer are aligned with one
another.
Before using the question in a test, ask a person knowledgeable in the subject
to critically review the essay question, the model answer and the intended
learning outcome to determine how well they are aligned with each other.
REVIEW (after receiving the student responses)
Review Student Responses to the Essay Question

After students have answered the questions, carefully review the range of
answers given and the manner in which students seem to have interpreted the
question. Make revisions based on the findings. Writing good essay questions
is a process that requires time and practice. Carefully studying the student
responses can help to evaluate students' understanding of the question as well
as the effectiveness of the question in assessing the intended learning
outcomes.

SELF-CHECK 4.3
1. Why should you specify the time allotted for answering each
question?
2. Why should you avoid optional questions?
3. What is meant when it is said that questions should be fair?
4. What should you do before and after administering a test?
4.8 VERBS DESCRIBING VARIOUS KINDS OF

MENTAL TASKS
Using the list suggested by Moss & Holder (1988) and Anderson & Krathwohl
(2001), Reiner, Bothell, Sudweeks and Wood (2002) proposed the following list of
verbs that describe mental tasks to be performed (see Table 4.1). The definitions
specify thought processes a person must perform to complete the mental tasks.
Note that this list is not exhaustive and local examples have been introduced to
illustrate the mental tasks required in each essay question.
Table 4.1: Verbs, Definitions and Examples
Verbs Definitions and Examples

Analyse Break material into its constituent parts and determine how the parts
relate to one another and to an overall structure or purpose.
Example: Analyse the meaning of the line He saw a dead crow, in a
drain, near the post office in the poem The Dead Crow.
Apply Decide which abstractions (concepts, principles, rules, laws, theories,
generalisations) are relevant in a problem situation.
Example: Apply the principles of supply and demand to explain why
the consumer price index (CPI) in Malaysia has increased in
the last three months.
Attribute Determine a point of view, bias, value, or intent underlying the
presented material.
Example: Determine the point of view of the author in the article about
her political perspective.
Classify Determine which category belongs to something.
Example: Classify the organisms into vertebrates and invertebrates.
Compare Identify and describe points of similarity.
Example: Compare the role of the Dewan Rakyat and Dewan Negara.

Compose Make or form by combining things, parts, or elements.

Example: Compose an effective plan for solving flooding problems in
Kuala Lumpur
Contrast Bring out the points of difference.
Example: Contrast the contribution of Tun Hussein Onn and Tun Abdul
Razak Hussein to the political stability of Malaysia.
Create Put elements together to form a coherent or functional whole, reorganise
elements into a new pattern or structure.
Example: Create a comprehensive solution for the traffic problems in
Kuala Lumpur.
Critique Detect consistencies and inconsistencies between a product and relevant
external criteria; detect the appropriateness of a procedure for a given
problem.
Example: Judge which of two methods is the best way for solving a
problem.
Defend Develop and present an argument to support a recommendation, to
maintain or revise a policy, programme, or propose a course of action.
Example: Defend the decision to raise fuel prices by the government.
Define Give the meaning of a word or concept; place it in the class to which it
belongs and distinguish it from other items in the same class.
Example: Define the term chemical weathering".
Describe Give an account of; tell or depict in words; represent or delineate by a
word picture.
Example: Describe the contribution of Zaba to the development of
Bahasa Melayu.
Design Devise a procedure for accomplishing some task.
Example: Design an experiment to prove that 21% of air is composed of
oxygen.
Differentiate Distinguish relevant from irrelevant parts or important from
unimportant parts of presented material.
Example: Distinguish between supply and demand in determining
price.
Explain Make clear the cause or reason of something; construct a cause-and-
effect model of a system; tell how to do; tell the meaning of.
Example: Explain the causes of the First World War.
Evaluate Make judgments based on criteria and standards; determine the
significance, value, quality, or relevance of; give the good points and the
bad ones; identify and describe advantages and limitations.
Example: Evaluate the contribution of the microchip in
telecommunications.

Generate Come up with alternative hypotheses, examples, solutions, proposals,

etc. based on criteria.
Example: Generate hypotheses to account for an observed phenomenon.
Identify Recognise as being a particular person or thing.
Example: Identify the characteristics of the Mediterranean climate.
Illustrate Use a word picture, a diagram, a chart, or a concrete example to clarify a
point.
Example: Illustrate the use of catapults in the amphibious warfare of
Alexander.
Infer Draw a logical conclusion from presented information.
Example: What can you infer happened in the experiment?
Interpret Give the meaning of; change from one form of representation (e.g.
numerical) to another (e.g. verbal).
Example: Interpret the poetic line, The sound of a cobweb snapping is
the noise of my life.
Justify Show good reasons for; give your evidence; present facts to support
your position.
Example: Justify the American entry into World War II.
List Create a series of names or other items.
Example: List the major functions of the human heart.
Predict Know or tell beforehand with precision of calculation, knowledge, or
shrewd inference from facts or experience what will happen.
Example: Predict the outcome of a chemical reaction.
Propose Offer for consideration, acceptance, or action; suggest.
Example: Propose a solution for landslides along the North-South
Highway.
Recognise Locate knowledge in long-term memory that is consistent with
presented material.
Example: Recognise the important events in the road to independence in
Malaysia.
Recall Retrieve relevant knowledge from long-term memory.
Example: Recall the dates of important events in Islamic history.
Summarise Sum up; give the main points briefly.
Example: Summarise the ways in which man preserves food.
Trace Follow the course of; follow the trail of; give a description of progress.
Example: Trace the development of television in school instruction.

ACTIVITY 4.4
Discuss the following with your coursemates.
(a) Select some essay questions in your subject area and examine
whether the verbs used are similar to those in the list given in
Table 4.1. Do you think the tasks required by the verbs used are
appropriate?
(b) Do you think students are able to differentiate between the tasks
required in the verbs listed?
(c) Are teachers able to describe to students the tasks required by

using these verbs? Explain.
Checklist For Writing Essay Questions
Can the item be better assessed with a different kind of assessment?
Is the question aligned with the intended learning outcome?
Should the question be split into short essay questions?
Does the question contain a clear task and a specified scope?
Is the question worded and structured clearly?
Do the students know the recommended time for each question?
Have you written a model answer or an outline of the answer?
Do you have a person knowledgeable in the subject to review the

question?

4.9 MARKING AN ESSAY QUESTION

Marking or grading of essays is a notoriously unreliable activity. If we read an
essay at two different times, the chances are high that we will give the essay a
different grade each time. If two or more of us read the essay, our grades will
likely differ, often dramatically so. We all like to think we are exceptions, but
study after study of well-meaning and conscientious teachers show that essay
grading is unreliable (Ebel, 1972; McKeachie, 1987). Eliminating the problem is
unlikely, but we can take steps to improve grading reliability. Using a scoring
guide or marking scheme helps control the shifting of standards that inevitably
take place as we read a collection of essays and papers. The two most common
forms of scoring guides used are the analytic and holistic method.
(a) Analytic Method

Analytical marking is the system most frequently used in large-scale public
examinations (such as the SPM and STPM) and also by teachers in the
classroom. Its basic tool is the marking scheme with proper mark
allocations for elements in the answer (as shown in Figure 4.2). The
marking scheme consists of a list of the major elements the teacher believes
students should include in the ideal answer. In the example, marks are
allocated for describing each of the five factors (three marks for each factor)
and one mark is given for providing a relevant example. Marks are
also allotted to the introduction and conclusion, which are important
elements in an essay.

Figure 4.2: Sample of a marking scheme using the analytical method
The marker reads and compares the students answer with the marking
scheme. If all the necessary elements are present, the student receives the
maximum number of points. Partial credit is given based on the elements
included in the answer. In order to arrive at the overall exam score, the
instructor adds the points earned on separate questions.
Identify in advance what will be worth a point, and how many points are
allocated for each question. Inform your students, so that they do not give
more (or less) than necessary, and they know precisely what you are
looking for. If students come up with an unexpected but correct example,
give them the point immediately and add that point to your answer key, so
the next student will get the point too.

(b) Holistic Method (Global or Impressionistic Marking)

The holistic approach to scoring essay questions involves reading an entire
response and assigning it to a category identified by a score or grade. This
method involves considering the students answer as a whole and judging
the total quality of the answer relative to other student responses or the
total quality of the answer based on certain criteria that you develop. Think
of it as sorting into bins. You read the answer to a particular question and
assign it to the appropriate bins. The best answers go into the exemplary
bin, the good ones go into the good bin, and the weak answers go into the
poor bin. Then, points are written on each paper appropriate to the bin it
is in. It is based on an overall impression, not point by point using the
rubric or marking scheme (as shown in Table 4.2). You cannot be much
finer than five divisions; six at most. You have to re-read everything to
ensure that all the papers in each of the five piles are roughly the same; all
in the middle piles are roughly the same, and less good than the top pile,
but less bad than the bottom pile.
You can develop a description of the type of response that would illustrate
each category before you start, and then try out this draft version using
several actual papers. After reading and categorising all of the papers, it is a
good idea to re-examine the papers within a category to see if they are
similar enough in quality to receive the same points or grade. It may be
faster to read essays holistically and provide only an overall score or grade,
but students do not receive much feedback about their strengths and
weaknesses. Some instructors who use holistic scoring also write brief
comments on each paper to point out one or two strengths and/or
weaknesses so students will have a better idea of why their responses
received the scores they did.
ACTIVITY 4.5
1. Compare and contrast the analytical method and holistic method
of marking essays.
2. Which method is widely practised in your institution? Why?
3. Do you think there would be a difference in marking an answer

using the two methods? Explain.

Table 4.2: Sample of a Marking Scheme Using the Holistic Method
Level of
General Presentation Reasoning, Argumentation
Achievement
EXEMPLARY Addresses the question Demonstrates an accurate
(10 pts) States a relevant argument and complete understanding
of the question
Presents arguments in a
logical order Uses several arguments and
backs arguments with
Uses acceptable style and examples, data that support
grammar (no errors) the conclusion
GOOD Combination of above traits, Same as above but less
(8 points) but less consistently thorough, still accurate
represented (12 errors) Uses only one argument and
example to support
conclusion
ADEQUATE Does not address the question Demonstrates minimal
(6 pts) explicitly, though does so understanding of question;
tangentially still accurate
States a somewhat relevant Uses a small subset of
argument possible ideas for support of
Presents some arguments in a the argument.
logical order
Uses adequate style and
grammar (more than 2 errors)
POOR Does not address the question Does not demonstrate
(4 pts) States no relevant arguments understanding of the
question; inaccurate
Is not clearly or logically
organised Does not provide evidence to
support response to the
Fails to use acceptable style question
and grammar
NO ANSWER
(0 pts)

Suggestions for Marking or Scoring Essays

Here are some suggestions for marking or scoring essays:
(a) Grade the papers anonymously. This will help control the influence of our
expectations of the student on the evaluation of the answer.
(b) Read and score the answers to one question before going on to the next
question. In other words, score all the students responses to Question 1
before looking at Question 2. This helps to keep one frame of reference and
one set of criteria in mind through all the papers, which results in more
consistent grading. It also prevents an impression that we form in reading
one question from carrying over to our reading of the students next
answer. If a student has not done a good job on the first question, we may
let this impression influence our evaluation of the students second answer.
But if other students papers come in between, we are less likely to be
influenced by the original impression.
(c) If possible, try to grade all the answers to one particular question without
interruption. Our standards might vary from morning to night, or one day
to the next.
(d) Shuffle all the papers after each item is scored. Changing the order of
papers this way reduces the context effect and the possibility that a
students score may be the result of the location of the paper in relationship
to other papers. If Rakeshs B work is always following Jamals A
work, then it might look more like C work and his grade would be lower
than if his paper were somewhere else in the stack.
(e) Decide in advance how you are going to handle extraneous factors and be
consistent in applying the rule. Students should be informed about how
you treat such things as misspelled words, neatness, handwriting, grammar
and so on.
(f) Be on the alert for bluffing. Some students who do not know the answer
may write a well-organised coherent essay but one containing material
irrelevant to the question. Decide how to treat irrelevant or inaccurate
information contained in students answers. We should not give credit for
irrelevant material. It is not fair to other students who may also have
preferred to write on another topic, but instead wrote on the required
question.
(g) Write comments on the students answers. Teacher comments make essay
tests a good learning experience for students. They also serve to refresh
your memory of your evaluation should the student question the grade.

(h) Be aware of the order in which papers are marked which can have an
impact on the grades awarded. A marker may grow more critical (or more
lenient) after having read several papers, thus the early papers may receive
lower (or higher) marks than papers of similar quality that are scored later.
(i) Also, when students are directed to take a stand on a controversial issue,
the marker must be careful to ensure that the evidence and the way it is
presented is evaluated, NOT the position taken by the student. If the
student takes a position which differs from that of the marker, the marker
must be aware of his or her own possible bias in marking the essay.
An essay question is a test item which requires a response composed by the

examinee usually in the form of one or more sentences of a nature that no
single response or pattern of responses can be listed as correct, and the
accuracy and quality of which can be judged subjectively only by one skilled
or informed in the subject matter.
There are two types of essay based on their function: coursework essay and
examination essay.
Essay questions provide an effective way of assessing complex learning

outcomes.
Essay questions provide authentic experiences because constructing

responses are closer to real life than selecting responses.
It is not possible to assess a students mastery of the complete subject matter

domain with just a few questions.
Essay questions have two variable elements the degree to which the task is
structured and the degree to which the scope of the content is focused.
Whether or not an essay item assesses higher-order thinking depends on the

design of the question and how students responses are scored.
Specifying the approximate time limit helps students allocate their time in
answering several essay questions.
Avoid using essay questions for intended learning outcomes that are better
assessed with other kinds of assessment.

Analytical marking is the system most frequently used in large-scale public

examinations and also by teachers in the classroom. Its basic tool is the
marking scheme with proper mark allocations for elements in the answer.
The holistic approach to scoring essay questions involves reading an entire

response and assigning it to one of several categories, each given a score or
grade.
Analytical method Marking scheme

Complex learning outcomes Mental tasks
Constructed responses Model answer
Essay Rubric
Grading Task and scope
Holistic method Time-consuming
Inter-scorer reliability
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching,

and assessing: A revision of Blooms taxonomy of educational objectives.
Boston, MA: Allyn & Bacon.e
Crooks, T. J. (1988). The impact of classroom evaluation practices on

students. Review of Educational Research, 58(4), 438481.

08 Hmef5053 T4

Uploaded by

Copyright:

Available Formats

08 Hmef5053 T4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

08 Hmef5053 T4

Uploaded by

Copyright:

Available Formats

Topic How to Assess?

Copyright Open University Malaysia (OUM)

4.1 WHAT IS AN ESSAY QUESTION?

(d) The accuracy and quality of students responses or answers to essay

Copyright Open University Malaysia (OUM)

4.2 TYPES OF ESSAY TESTS

(a) Coursework Essay

(i) Essay Outlines

(ii) Standard Essays

(iii) Extended Essays

Copyright Open University Malaysia (OUM)

Coursework essays are intended to assess students ability to research a

(b) Examination Essay

4.3 WHY ARE ESSAY QUESTIONS USED?

(a) Essay questions provide an effective way of assessing complex learning

Copyright Open University Malaysia (OUM)

(b) Essay questions allow students to demonstrate their reasoning. These

(c) Essay questions provide authentic experiences. Constructing responses is

4.4 DECIDING WHETHER TO USE ESSAY

The decision whether to use essay questions or objective questions in

(a) To assess students understanding of subject matter or content; and

It is challenging to write test items to tap into higher-order thinking. However,

Copyright Open University Malaysia (OUM)

4.5 WHAT ARE SOME LIMITATIONS OF ESSAY

(a) One purpose of testing is to assess a students mastery of subject matter. In

Copyright Open University Malaysia (OUM)

Copyright Open University Malaysia (OUM)

4.6 SOME MISCONCEPTIONS ABOUT ESSAY

(a) By Their Very Nature, Essay Questions Assess Higher-order Thinking

(b) Essay Questions are Easy to Construct

(c) The Use of Essay Questions Eliminates the Problem of Guessing

Copyright Open University Malaysia (OUM)

(d) Essay Questions Benefit All Students by Placing Emphasis on the

(e) Essay Questions Encourage Students to Prepare More Thoroughly

(b) Given its advantages and limitations, should governments spend

2. Suggest other weaknesses of using essay questions in

3. List some of the misconceptions about essay questions.

Copyright Open University Malaysia (OUM)

4.7 SOME GUIDELINES FOR CONSTRUCTING

(a) Clearly Define the Intended Learning Outcome to be Assessed by the

Copyright Open University Malaysia (OUM)

Learning Outcome: To be able to differentiate the reproductive habits of

Essay Question: What are the differences in egg laying characteristics

Note: This learning outcome can be better assessed by an objective test.

(c) Clarity About the Task and Scope

(ii) The problem of extreme subjectivity when scoring student answers or

Copyright Open University Malaysia (OUM)

Figure 4.1: Alignment between content, learning activities

This alignment is absolutely necessary for obtaining student responses that

1. When would you decide to use an objective item rather than an

Copyright Open University Malaysia (OUM)

Weak Essay Question:

(a) The impact on what in England? The economy? Foreign trade? A

(c) What exactly do you want me to do in my evaluation? (The task is not

Improved Essay Question:

The improved question determines the task for students by specifying a

Copyright Open University Malaysia (OUM)

(d) Questions that are Fair

Another important element is to avoid using indeterminate questions. A