Advances in Social Science, Education and Humanities Research, volume 434

International Conference on English Language Teaching (ICONELT 2019)

ESP Course Book Evaluation from the Perspectives

of Teachers, Cadets, and Graduates: The Case of
Maritime English
Latifa Ika Sari Ria Hermina Sari
Politeknik Ilmu Pelayaran Semarang Politeknik Ilmu Pelayaran Semarang
Semarang, Indonesia Semarang, Indonesia
latifa.ika@gmail.com ria.hermina.sari@gmail.com

Abstract—Course books are considered as important communications. Maritime English has several sub
elements in the language teaching and learning process. This languages: (1) General English, (2) General ME, (3) ME for
study was aimed to evaluate a course book used in Preparation Navigation, (4) ME for Marine Engineering, (5) ME for
Class of Maritime English Test in one of Merchant Marine Ship’s Documentation and Correspondence, 6) ME for Radio
Polytechnic from three perspectives: teachers, graduates, and Communication, (7) Standard Marine Communication
cadets. This study employed a qualitative descriptive analysis.
There were 12 teachers, 111 cadets and 14 graduates involved
Phrase (SMCP), (8) ME for IMO Conventions, Regulations,
in this study to give their opinions. Textbook evaluation Manuals, etc., (9) ME for Semiotic Systems, (10) ME for
checklist developed by Daoud and Celce-Murcia (1979) was Visual Aids and Others. It is important to note that
adopted with several adjustments and used as a guide in linguistically, Maritime English is not a separate language
collecting data. There were five aspects of the book to be but just a conventional label for a subset or realization of
analyzed: subject matter, vocabulary and structure, exercise, English language appropriate to a specific maritime setting
illustration, and physical make-up. The data were [1].
recapitulated, analyzed, and described qualitatively. The The International Convention on Standards of Training,
findings revealed that the book covers various materials, Certification and Watch keeping for Seafarers (STCW) has
including maritime vocabulary and grammar, but they are not
organized systematically. The exercises are sufficient, but some
stated that seafarers should have the abilities to understand
parts need more addition. The illustration and the physical orders and to communicate in English with others in relation
make-up need to be improved to enhance the process of to duties on board. It is no doubt that communication is
language teaching and learning. It is concluded that the considered as a significant aspect in the maritime field.
coursebook investigated in this study needs several revisions Communication breakdown in our daily life can cause
and improvement. conflict and embarrassment, but communication breakdown
at sea can cause much more serious effects such as damage
Keywords: coursebook, evaluation, Maritime English to valuable cargoes, sea pollution, crew injury, or even death.
I. INTRODUCTION Realizing that the crew’s English competency is highly
linked to the safety at sea, many shipping companies require
A. Background seafarers to have good English competence and adequate
Maritime English (ME) is the international working knowledge of Maritime terminologies. Therefore, English
language in the maritime industry. It is a language used at proficiency becomes the major qualification for shipping
sea for communication among crew members. On the ship, companies to recruit seafarers for their vessels. Thus,
all manuals, publications, and instructions are written in maritime graduates are expected to show the ability to
English. In vessels with a multinational crew, Maritime conduct effective and accurate communication. Maritime
English becomes the only means of communication to schools today are faced with the biggest challenge of
achieve safety and security of voyages. In this context, developing seafarers with high standards of competence and
Maritime English is said as lingua franca, a language that professionalism with good English communication skills [2].
systematically used to make communication possible In relation to the significance of Maritime English for
between people not sharing a mother tongue [1]. graduates, several maritime colleges and polytechnics in
Standardized language at sea is very important in which Indonesia have set a regulation of cadets taking Maritime
agreement or convention is fundamental. The use of ME is English Test before they register for graduation. ISF Marlins
supported by the international community and developed by English Language Test for Seafarers has been used by one of
the International Maritime Organization (IMO). According to Merchant Marine Polytechnic in Indonesia as the prerequisite
Demydenko [1], ME is a System of Specialized Sub for graduation registration since the middle of 2018. Cadets
languages. Sub languages, or languages of restricted domains, have to reach a minimum score 70 for both Nautical Studies
are most frequently observed in technical and scientific Program and Marine Engineering Program.

Copyright © 2020 The Authors. Published by Atlantis Press SARL.

This is an open access article distributed under the CC BY-NC 4.0 license -http://creativecommons.org/licenses/by-nc/4.0/. 56
 Advances in Social Science, Education and Humanities Research, volume 434

ISF Marlins English Language Test for Seafarers is vocabulary and grammar do not receive too much attention
recognized as an online-based standardized test for Maritime in both books [7].
English. The test consists of a total of 85 questions which are Sarem, Hamidi, and Mahmoudie conducted an evaluation
broken down into the following categories: listening on a coursebook of English for International Tourism.
comprehension (25 questions), grammar (30 questions), Different from the previous researchers, they used a checklist
vocabulary (15 questions), different sound and pronunciation developed by Daoud and Celce-Murcia (1979). Similar to the
(9 questions), reading (1 question) and time and numbers (5 previous one, they employed qualitative descriptive analysis
questions). The test is used worldwide by shipping using self-report study. The result of their evaluation showed
companies to assess Maritime English competence of that the coursebook has excellent layout and physical
seafarers. Considering that the test has special characteristics appearance. The contents of the book are mainly skill-based,
and difficulties, the institution has provided a preparation and the activities and exercises are mostly speaking and
class for the cadets before they take the test. The preparation listening with some writing and reading exercises. The
class is conducted for 10 meetings in which each meeting coursebook also has advantages by providing review unit,
consists of 100 minutes. The cadets are given materials audio CD, listening scripts and grammar glossary [8].
related to Maritime English in a more specific way and they Other researchers Azarnoosh and Ganji evaluated ESP
are also exposed to various question types related to the test. Management Coursebook. Similar to the two previous
However, the result of the first batch who took the real test studies mentioned before, they also employed a qualitative
showed a disappointing result. From 192 cadets, only 47% descriptive analysis. It was also a self-reporting study using a
passed the test, while 53% of them were considered failed. checklist developed by Miekley (2005). The result of their
That condition arose self-reflective questions from the evaluation showed that the book lacks visual materials and
institution: what actually went wrong? various activities to reinforce learners’ vocabulary and
grammatical knowledge. There is also no activity for
B. The Needs for ESP Coursebook Evaluation
practicing listening, how to pronounce words and sentences
Evaluation towards education program is important in
correctly. It also does not provide activities to encourage
order to make judgement or decision for improvement.
students’ writing ability. They recommended that the book
According to Hutchinson and Waters in [3] “evaluation is a
should be modified for improvement [4].
matching process, matching needs to available
Another recent study was conducted by Chegeni, Kamali,
solution.”.ReaDickins and Germanie in [4] elaborated that
and Noroozi [9]. They conducted an evaluation on General
evaluation is “a means by which we can gain a better
English textbook “Four Corners”. Similar to Sarem, Hamidi
understanding of what’s effective, what’s less effective and
and Mahmoudie [8], they used checklist developed by Daoud
what appears to be no use at all”. When an educational
and Celce-Murcia (1979). They also employed qualitative
program does not achieve the expected outcomes, evaluation
descriptive analysis for their research. They found that the
needs to be conducted on the aspects of the program.
book “Four Corners” is well organized and uses authentic
In ESP context such as Maritime English, coursebooks
language. It gives opportunities for learners to practice the
have a significant contribution to the process of teaching and
four skills: speaking, writing, listening and reading. Many
learning. According to Ghalandari and Talebinejad [5],
colorful pictures make the appearance look interesting for
materials in ESP classroom play a crucial role in exposing
students. The book also includes review units, audio CD,
learners to language. For most teachers, coursebooks provide
listening script, self-study units, and grammar glossaries.
the foundation for the content of the lessons, the required
The studies conducted by previous researchers indicate
skills to be taught, and the language practices that students
that it is highly necessary to review the coursebook used in
conduct during class activities. Coursebooks are probably the
the classroom. Realizing the importance of coursebook
best option among other sources of materials for teachers as
evaluation, the researchers want to evaluate the coursebook
guidance of teaching during the course.
used in the Preparation Class of Maritime English Test
Ellis in [6] differentiated between two types of material
conducted in one of Merchant Marine Polytechnic in
evaluation: predictive evaluation and retrospective evaluation.
Indonesia. Different from the previous researches, this study
The former refers to making a decision based on the
will involve three different perspectives of the major
objectives for which the materials are going to be used, while
elements in the education process:teachers, cadets, and
the latter involves further evaluation of the materials to know
graduates in order to obtain a comprehensive evaluation. The
whether they have worked well for the determined purposes.
research questions guiding this study are: (1) How
Several researchers have been conducted evaluations on
appropriate is the content of the book from the perspectives
ESP textbooks or teaching materials. Nezhad, Atarodi, and
of teachers, cadets and graduates? (2) How appropriate are
Khalili evaluated Mechanical Engineering textbooks used in
the vocabulary and structure provided by the book from the
Iran. They used a checklist developed by Miekley (2005) to
perspectives of teachers, cadets and graduates? (3) How
evaluate two textbooks published by SAMT and OXFORD.
appropriate are the exercises provided by the book from the
They found that the OXFORD textbook is better in terms of
perspectives of teachers, cadets and graduates? (4) How
physical appearance and the wide range of activities
helpful are the illustrations provided by the book from the
provided. While the SAMT textbook excels in providing
perspectives of teachers, cadets and graduates? (5) How good
authentic materials for learning. They also found that

57
 Advances in Social Science, Education and Humanities Research, volume 434

is the physical make-up of the book from the perspectives of to-date and interesting for cadets or not, and whether the
teachers, cadets and graduates? materials cover an appropriate balance of the four language
skills or not. In this first aspect, the teachers gave the most
II. METHOD contribution of evaluation. Majority of them agreed that the
book covered various up-to-date topics and materials which
Materials
were needed by cadets to complete the Maritime English
The materials used for the evaluation purposes in this Test. However, there were several things that needed
study was the coursebook of Preparation Class for Marlins improvement, especially the organization of the materials.
English Test. The coursebook was composed by lecturers of For ten meetings of the course, there were too many
Maritime English and compiled from several sources. There materials included in the book with a lack of variation in the
are three parts of the book: the main part which consists of activity. The book also did not provide an appropriate
instructions and exercises, and two supplement parts of balance of the four language skills: reading, listening,
grammar and pronunciation modules. The book has speaking and writing. It was probably because the book was
altogether 117 pages. designed for a test preparation which did not assess speaking
A. Participants ability.
The participants involved in this study were: (1) 12 The graduates and cadets had similar perspectives
English teachers who taught Preparation for Marlins English towards the subject matter of the book. They did not
Test program; (2) 111 cadets of Nautical Study and Marine complain about this aspect. However, fifteen percent of them
Engineering Study who joined the Preparation for Marlins perceived that the materials of the textbook were less
English Test program. They were randomly selected; (3) 14 interesting, less challenging and less motivating for them due
Graduates of Semarang Merchant Marine Polytechnic who to the lack of variation.
attended at least five sessions of the preparation class and
In designing a coursebook, the subject matter should be
had taken the real Marlins English Test. organized carefully. A coursebook can become guidance for
Instrument both teachers and learners through the course as long as it is
designed and organized well. In the case of ESP materials, it
The authors used a questionnaire adapted from Daoud
is important that the materials included in the book are
and Celce-Murcia’s checklist which has been widely used
suitable with the learners’ needs, learning objectives of the
for evaluating textbooks. There are five major sections of course and methodology [6]. A textbook can be a teacher, a
evaluation: (1) Subject matter, (2) Vocabulary and structures, map, a resource, a trainer and an authority. As a teacher, the
(3) Exercises, (4) Illustration, and (5) Physical make up [10]. book gives students relevant information about the materials.
Few adjustments were made to suit the purpose of this study. As a map, it shows an outline of the materials which guides
One significant adjustment was providing a comment students and teachers during the course. As a resource, the
column for each section to obtain qualitative responses. For book provides materials and activities for teachers and
the cadets and graduates, the questionnaire was written in students. As a trainer, the book can give valuable instruction
Bahasa Indonesia and for the teachers, the questionnaire was and guidance for novice teachers [11].
written in English (See appendix).
B. Vocabulary and Structures
Data Collection and Analysis This aspect deals with how the book provides and
The questionnaires for the English teachers were presents vocabulary and grammar section, including the
distributed using Google form and the teachers were asked variation, the organization, and the suitability for cadets. All
to complete the questionnaire in not more than three days. teachers agreed that the book provided systematic
Google form was also used for the graduates since they did organization for grammar from the simple to more
not study at the campus anymore. Actually, there were 30 complicated one. The grammar section also contained
graduates that were given the questionnaires, but only 14 of examples and simple explanation that helped students to
them who completed and submitted the questionnaires back. understand the materials. The book also provided a various
For the cadets, the questionnaires were printed on papers. maritime vocabulary that cadets needed. However, 59% of
the teachers perceived that the vocabulary was not presented
Initially, there were 125 questionnaires distributed for cadets,
systematically. Some of them suggested that there should be
but only 111 of them were completed and thus could be
vocabulary building activity included in the book and to add
analyzed. lists of glossaries at the end of each unit. Another weakness
The results of the questionnaires were then recapitulated of the book according to the teacher was that the grammar
and described. In this study, the authors used qualitative and the vocabulary were not reviewed in the next chapter as
descriptive analysis with simple descriptive statistics. reinforcement. The teachers perceived that cadets need drills
and repetitions in learning because they could forget things
III. FINDINGS AND DISCUSSION
easily. Based on teachers’ observation, some cadets also did
Subject Matter not have sufficient knowledge of grammatical structure.
The subject matter deals with the topics and materials Thus, the grammar section should receive more attention.
presented in the book, including the variation of topics, the This is in line with the perception of the graduates, that it
organization of the materials, whether the materials are up- was necessary to review vocabulary and grammar in the next

58
 Advances in Social Science, Education and Humanities Research, volume 434

chapter of the book. However, in general, the graduates and the illustrations were helpful for them and for the cadets.
the cadets were happy with the vocabulary and the grammar Teachers could use the book illustration to explain the
section. Only twelve percent of cadets who perceived that the materials to the cadets so that they can understand the
language used in the book was rather difficult. This might be materials easily. The teachers agreed that the illustrations in
caused by the low English proficiency of the cadets the book were simple, clear and not confusing. The
themselves that hinder their understanding of the materials. illustrations were also interesting because they consisted of
However, this argument needs further investigation for colourful pictures and diagram.
verification.
Graduates and cadets were also happy with the
In ESP like Maritime English, vocabulary aspect needs to illustration in the book. However, some of them complained
be organized carefully. Maritime English contains abundant that some of the pictures were not clear due to poor picture
of maritime vocabulary that has to be learned by cadets. resolution. Therefore, it is necessary to select good quality
Vocabulary in Maritime English is related to the standard pictures to be included in the book. Despite the problem with
words and phrases used by all seafarers around the world. picture resolution found in some pictures, both graduates and
Structure or grammar is also important as a tool which cadets agreed that the illustrations were interesting and
enables seafarers to construct sentences using vocabularies helpful for them in understanding the materials.
that they have learned. Combination of accurate vocabulary
and correct grammar will contribute to the safety of Pictures, diagrams, or charts are great tools to enhance
communication at sea. Quoting Viskota and Kalibota, the process of teaching and learning. They can help teachers
“increasing efforts at teaching Maritime English at maritime to deliver educational content more effectively. They can be
colleges and universities worldwide has contributed to used to reinforce the material being studied. It can increase
preventing maritime accidents” [12]. cadets’ interest in learning and makes the learning process
more enjoyable [14].
C. Exercises
This aspect includes sufficiency and variation of the E. Physical Make-up
exercise and whether it can help cadets to understand the The last section on the questionnaire used in this study is
materials. The exercise of the book had been made in line the physical make-up. It is related to the attractiveness of the
with the real Maritime English test to make cadets familiar book appearance. It also deals with the durability of the
with the types of questions. There were listening activities, cover and the paper used, the font size, and the printing
filling gaps, image multiple choice, making sentences using quality. In general, the teachers agreed that the book had a
jumbled words, labelling pictures, word categorization, good physical make-up. The font size and the printing
choosing the odd word, time and number, choosing a word quality were excellent, and nothing needs to be concerned
with a different sound, and reading. related to those aspects. They also perceived that the cover
was quite attractive. However, several improvements need to
Majority of teachers agreed that the exercises could help be made on the supplement books. They suggest that the
cadets to obtain a better understanding of the materials. main coursebook and the supplement books of grammar and
There were varieties in the exercise format, so cadets could pronunciation should be put together, instead of making them
practice various type of exercises. However, some teachers as separate books.
argued that the exercise of making sentences using jumbled
words needed more attention. Based on their experience, Graduates and cadets had the same perception towards
cadets had more difficulty in doing this kind of exercise the physical make-up of the book. They agreed that in
compared to the other kind. Therefore, it is necessary to add general, the book had good physical make-up: the font size
more exercises and examples. was proportional, and the printing quality was good.
However, some of them said that the cover of the book was
The graduates and cadets perceived no significant not interesting. It seemed that the cover of the book needs to
problem with the exercises. They agreed that in general the be redesigned to make it more attractive to cadets.
exercises provided by the book were sufficient. The exercises
also enable cadets to understand the materials more easily. In this context, the jargon ‘don’t judge a book by its
cover’ [15] seemed not appropriate. In fact, the cover of the
Exercise is important in a coursebook to check cadets’
coursebook influenced cadets’ interest in learning. People
understanding after they are taught certain materials. It can
also serve as a tool for cadets to practice their ability. It is make the first judgement from the appearance and the first
necessary to arrange exercises, assignments, and activities in impression is highly influential for the later perception and
the coursebook appropriately. Repetition is essential in attitude.
learning vocabulary and grammar because knowledge and
III. CONCLUSION
understanding might not be internalized just in one meeting.
Repetition can enhance the quality of knowledge and also the The process of English Language Teaching and Learning
strength of it [13]. involves many elements that interconnected each other.
Textbook or course book is considered as one of the
D. Illustration important elements because it can be the source of learning
The illustrations in the book were intended to help cadets for students and guidance of teaching for teachers. Even
to understand the materials. From the perspective of teachers, though high attempts have been made in arranging textbooks,
still there are no perfect books made for the first time. In this

59
 Advances in Social Science, Education and Humanities Research, volume 434

case, evaluation becomes an essential part of education and REFERENCES

cannot be separated from the process of teaching and
learning itself. [1] Demydenko, Nadiya., “Teaching Maritime English: A Lingustic
Approach” in Journal of Shipping and Ocean Engineering,” 2,
This study was aimed to evaluate a course book used for 249-254, 2012.
Maritime English Test Preparation Class from three
[2] Navarro, Jomarie D., Garbin, Zenaida Z., Agena, Edwin M,
perspectives: teachers, cadets, and graduates. This study Gracia Olympo B,. “Maritime Students’ English Proficiency and
employed a qualitative descriptive analysis in which the Their Feedback in Instructional Materials” in Asia Pacific
researcher adapted evaluation checklist developed by Daoud Journal of Maritime Education, 1 (1), 63-81, 2015.
and Celce-Murcia [10]. There were five aspects to be [3] Azarnoosh, Maryam., Zeerathpishe, Mitra., and Faravani,
analyzed, including subject matter, vocabulary and structure, Akram., Issues in Coursebook Evaluation, Boston: Brill Sense,
exercise, illustration, and physical make-up. 2018.
[4] Azarnoosh, Maryam., Ganji, Maryam., “ESP Book Evaluation:
The findings of the study showed that in terms of subject The Case of Management Coursebook” in International Journal
matter, the organization of the materials needs to be revised. of Secondary Education, 2 (4), 61-65, 2014.
There were too many materials for a ten-meeting course, and [5] Ghalandari, Shiva., Talebinejad, M. Raza., “Medical ESP
it caused difficulties for teachers when delivering the Textbook Evaluation in Shiraz Medical College” in Education
Research Journal, 2, 20-29, 2012.
materials. In terms of vocabulary and structure, the book
[6] Karimnia, Amin., Jafari, Fatemeh Mohammad., “Critical
provided a systematic organization for grammar including Textbook Evaluation: The Case of Visual Arts Textbook” in
examples and explanation. The book also provided various Sustainable Multilingualism, 2, 219-236, 2017.
maritime vocabulary but unfortunately, it was not presented [7] Nezhad, Hadi Yaghoubi., Atarodi, Isa., Khalili, Maryam., “A
systematically. Thus, it is suggested to add vocabulary Checklist-Based Evaluative and Comparative Study of ESP
building activity and glossary at the end of every unit to Books: The Case of Mechanical Engineering” in Journal of
make it more systematic. The exercise part was important Foreign Language Teaching and Translation Studies, 2 (2), 50-
69, 2013.
because it could be used to check cadets’ understanding. It
[8] Sarem, Saeid Najafi., Hamidi, Hadi., Mahmoudie, Rezyan., “A
could also make cadets familiar with the type of questions in Critical Look at Textbook Evaluation: A Case Study of
the real test. The book provided a sufficient number of Evaluating an ESP Course-Book: English for International
exercises with various activities. However, the exercise of Tourism” in International Research Journal of Applied and
making sentences using jumbled words needs to be added in Basic Sciences, 4 (2), 372-380, 2013.
number because cadets’ face difficulty when doing this kind [9] Chegeni, Nastaran., Kamali, Bahrooz., Noroozi, Atousa.,
of exercise compared to other kinds of exercises. The “General English Textbook Evaluation: A Closer Look at ‘Four
Corners’” in Theory and Practice in Language Studies, 6 (12),
illustration was important to enhance the process of language 2325-2330, 2016.
teaching and learning. Cadets also found it helpful in
[10] Daoud, A., Celce-Murcia, M., “Selecting and Evaluating A
understanding the materials even though some of the pictures Textbook” in M. Celce-Murcia and L. McIntosh (Eds.)
were not clear due to poor resolution. In the aspect of Teaching English as A Second or Foreign Language, 302-307,
physical make-up, the font size and the printing quality were Cambridge, MA: Newbury House Publisher, 1979.
good, but the cover needed to be redesigned to make it more [11] Bojanic, Radic Biljana B., Topaloy, Jagoda P., “Textbook in the
attractive for cadets. EFL Classroom: Defining, Assessing, and Analyzing” in
Collection of Papers of The Faculty of Philosophy, 3, 137-153,
Finally, the findings of the present studies have 2016.
significant implications for materials development, [12] Viskota, Adelija Culic., Kalebota, Sara., Maritime English –
especially in Maritime English. In designing a coursebook, What Does It Communicate? in Transaction on Maritime
Science, 2, 109-114, 2013.
teachers and material developers need to consider several
aspects. Subject matter should cover various topics or [13] Nation, I.S.P., Learning Vocabulary in Another Language,
Victoria University of Wellington: English Language Institute,
materials that are needed by cadets and should be in line with 1999.
the course objectives. The materials should be organized [14] Philominraj, Andrew., Jeyabalan, David., Vidal-Silva, Cristian.,
systematically according to topics and level of difficulty. In “Visual Learning: A Learner Centered Approach to Enhance
ESP especially in Maritime English course, the vocabulary English Language Teaching” in English Language Teaching, 10
and grammar should be presented in a systematic way and (3), 54-62, 2017.
provided with sufficient exercises. The illustration and [15] Adam, Tess, Judging A Book by Its Cover: Are First Impression
physical make-up of the book should be prepared and Acccurate?, Undergraduate Honors Theses University of
Colorado, Boulder, 2012.
designed well because it can enhance cadets’ interest and
motivation in learning.

60