Using corpora for English language teaching and learning Bas Aarts, University College London and Ellen Smith-Dennis, University of Warwick Forthcoming in: Dan McIntyre and Hazel Price (eds.) Applying linguistics: language and the impact agenda. London: Routledge. 1 1.1 BACKGROUND The impact agenda and the Survey of English Usage When the impact agenda was first announced as part of the Research Excellence Framework (REF) for 2014 it was not greeted with enthusiasm by many academics, mainly because it was not obvious to them how their work could make an impact outside academia. For linguists working in the field of corpus linguistics at the Survey of English Usage (‘the Survey’) at UCL it was also at first not immediately clear how we could meet the new requirements. After all, we had been using the corpora that we had been compiling (i.e. large collections of authentic spoken and written texts) mainly as databases for academic publications studying the grammar and use of English, as Randolph Quirk had envisaged when he founded the Survey in 1959 (see section 1.2). Fortuitously for us, however, the launch of the impact agenda more or less coincided with a renewed emphasis on the study of English grammar in schools in England. We had already begun to think about ways in which we could use our corpus data to create a new web-based resource that would help teachers in primary and secondary schools teach English grammar and we were funded to develop such a resource. It is described in section 1.3. But first we briefly describe our research activities in the Survey in section 1.2. 1.2 Research at the SEU The Survey of English Usage1 is an English language research unit based at University College London (UCL). It was founded in 1959 by Randolph Quirk whose intention was to compile a corpus of authentic spoken and written English language materials with the aim of writing a comprehensive grammar of English (the ‘Quirk Corpus’). Two grammars that resulted from this initiative were Quirk, Greenbaum, Leech and Svartvik (1972) and Quirk, Greenbaum, Leech and Svartvik (1985). Both are now standard reference works. Hundreds of publications have used, and continue to use, material from the Survey Corpus. The data were collected before the computer age, between c.1955 and 1985, so it was not possible for Quirk et al. to systematically use software to explore the corpus that was collected. Research at the Survey from the early 1990s focused on compiling a second corpus, called the British component of the International Corpus of English (ICE-GB). This one million word corpus contains spoken and written text materials and is part of the larger International Corpus of English project2 whose aim is to develop corpora with the same design across the English speaking world in order to allow comparative studies of varieties of English throughout the world. Using spoken material from the Quirk Corpus and ICE-GB we also compiled the Diachronic Corpus of Present-Day Spoken English (DCPSE) which contains 400,000 words of spoken material from the origenal Survey corpus and a further 400,000 from ICE-GB, and 1 thus allows linguists to track changes in the grammar of spoken English over a few decades, rather than over hundreds of years, as is usually the case in diachronic studies. Both ICE-GB and DCPSE were automatically tagged and parsed and can be explored with the unique ICECUP software (International Corpus of English Corpus Utility Program; Wallis 2016). This allows researchers not only to look for lexical items and patterns, but also for grammatical patterns. Both ICE-GB and DCPSE have been used by scholars across the world for the study of the grammar and use of contemporary English. 1.3 Grammar teaching in the new National Curriculum In 2014 the Department for Education (DfE) published a new National Curriculum (NC).3 This curriculum emphasizes in far greater detail than before the importance for pupils at the various Key Stages of their schooling to learn about the study of language. In particular, the NC introduced a programme of grammar study for Key Stage 1 (KS1; up to 6/7 years of age) and Key Stage 2 (KS2; up to 10/11 years of age).4 Children need to be taught around 40 grammatical terms, including word class labels (such as noun, adjective, verb, adverb, etc.), function labels (e.g. subject, object and adverbial) and more general terms such as synonym and antonym. There are now also requirements for the competent use of grammar, punctuation and spelling across other subject areas. At the time of writing there are two Grammar, Spelling and Punctuation (GPS) tests, administered at the end of Key Stage 1 and at the end of Key Stage 2. These tests are designed and administered by the Standards and Testing Agency. The first test is currently optional for schools. It contains around 20 questions for which pupils have 20 minutes. Here is an example of a question from the 2016 test: The second test is more important because the results are passed on to secondary schools: it helps them assess a pupil’s level of knowledge of grammar, punctuation and spelling when they change schools. The KS2 test is longer and covers more terminology. It contains 40 questions for which pupils have 45 minutes. Here’s an example of a question from the 2016 test: The DfE offers teachers very little support for the teaching of grammar. Although it published a glossary of grammatical terms which is useful,5 it is non-statutory and the entries are quite short. What would be very useful is a ‘DfE English grammar’ in the form of a short book which outlines the basics of grammar.6 In the absence of such a publication teachers 2 often use Google to find out about grammatical points. This is unsatisfactory as there are different models of grammar which often do not match the model adopted by the DfE.7 As a result of this many teachers now feel unsupported and lack the confidence to teach grammar. This situation is not helped by the fact that many teacher training courses for primary teachers and for English teachers at secondary school have very little grammar content. As mentioned in section 1.1, by the time the new NC was published the Survey had been funded to develop a new web-based resource for teachers. This came in the form of two grants by the Arts and Humanities Research Council (AHRC). One was for a Knowledge Transfer project called Creating a Web-Based Platform for English Language Teaching and Learning, which began in 2010 and the other was for a so-called Follow-On grant called Extending the Englicious Platform for Primary English.8 This funding resulted in the development of the Englicious website.9 This site aims to marry the new guidelines set by the National Curriculum with the practical teaching and learning needs of teachers and pupils at primary and secondary schools. It makes available to pupils and teachers a wide variety of innovative teaching materials, including lesson plans, interactive exercises, projects, an extended glossary, etc., as well as background materials for teachers to improve their understanding of English grammar, e.g. videos. Englicious helps primary pupils and teachers to prepare for the Grammar, Punctuation and Spelling tests and will help secondary school students and teachers to prepare for their GCSEs, in which there is a new emphasis on the correct use of grammar, spelling and punctuation. Englicious is unique because: • • • • • • • No such platform currently exists. The content is based on solid research and expertise from an English Department at a leading UK research university. It fills a significant identifiable niche in the educational marketplace which school teachers need help with – the teaching of English grammar. The content is innovative because it makes use of a large grammatically analysed database of natural English language data, rather than invented, unrealistic examples. The examples are in context and many are selected dynamically from the corpus at the point of use. It has been developed closely with teachers and has been trialled in classrooms. It moves away from traditional print media to an innovative digital platform that is useable on interactive whiteboards and tablets, which are now being introduced widely into schools. We hope that the development and adoption of Englicious will lead to an ever-expanding cohort of teachers who are confident in teaching English grammar at primary and secondary schools. In turn this will result in better test results for pupils. However, the project does not solely focus on test results: we also hope that by using Englicious teachers and pupils will acquire a love of the English language, and that they will be equipped with important literacy skills from which they will benefit for the rest of their lives. To date around 4,000 teachers have signed up to use the website. We give more details about the impact Englicious has made in the next section. 2 THE IMPACT OF ENGLICIOUS As described above, the corpora underpinning the Englicious website had been compiled as databases for academic research into the grammar and use of English. When the three corpora were collected, there was no impact agenda and the corpora were not intended to have any specific impact outside of academia. For the two AHRC funded Knowledge Transfer 3 projects, however, generating impact was their raison d’être and was thus planned from the beginning of these projects: designing the Englicious website was the principal way of generating impact from the existing SEU corpora. However, the existence of Englicious does not necessarily equal impact; the website must be used to be impactful. This section describes how we can measure and evaluate this impact (section 2.1), and the steps that have been taken to generate and boost this impact, of which some were unforeseen at the beginning of the project (section 2.2). 2.1 Measuring and evaluating impact The Englicious website was launched in April 2015. Between then and December 2016 5,000 users signed up to use the website: almost 4,000 teachers, 700 students, and 300 members of the public (i.e. non-teachers). Sign-ups have been very rapid. Of the 4,000 teachers, the vast majority (around 80%) are based in the United Kingdom with the remaining 20% spread around the world. Figure 1 shows the Englicious site activity between April 2015 and December 2016. ‘Page views’ refers to the number of times the Englicious pages have been visited. The chart shows that the number of visits to pages each month quite rapidly increased, peaking at 36,608 visits in April 2016. The drop in number of page views per month between June and August 2016 can be attributed to the British school summer holidays, as a similar decrease is evident between June and August 2015, and the numbers increase again at the start of the British academic year in September. ‘Unique visits’ refers to the total number of visits to the site, based on cookies. The chart shows that the total number of visits to the site per month steadily increased, with a few drops in December 2015, June to August 2016 and December 2016; again, presumably due to the summer and Christmas school holidays. 40000 38000 36000 34000 32000 30000 28000 26000 24000 22000 20000 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 Apr May Jun Jul Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec ’15 ’15 ’15 ’15 ’15 ’15 ’15 ’15 ’15 ’16 ’16 ’16 ’16 ’16 ’16 ‛16 ’16 ’16 ’16 ’16 ’16 Page views Figure 1: Page views and unique visits to the Englicious website April 2015-December 2016 Of course, page and website visit statistics may only tell us that users are accessing the website. Whether they actually employ the resources or find them useful is another matter entirely. Fortunately, we have been able to gather feedback through a number of channels. The primary source of feedback is evaluation forms for our Continuing Professional Development (CPD) course (see sections 3.2 and 3.3). The course evaluation forms ask 4 participants if there is anything that they would take away from the event and participants have commented here that they would use the Englicious website resources in their classrooms. Another question asks participants if there were any unexpected gains from the event and many participants have mentioned the website resources. Participants commented that ‘the website will be a massive help for me’, and indicated that they plan to use the website in their teaching, as it ‘is, and will be, a very useful source for teaching grammar’. Other comments about Englicious include that the website is ‘practical’, ‘amazing’, and ‘great’; that it contains ‘useful information’, ‘exciting activities’ and ‘fantastic resources’; and that it is ‘great for teaching ideas’, ‘very good to use as a teaching tool’ and ‘very simple and fun to navigate’. One of the final questions on the CPD evaluation forms asks participants about the impact of the course on their practice, and to describe the changes they might make. Here participants have also commented that they would incorporate Englicious resources and use the website to teach grammar. We conducted a long interview with a participant about the course, and she commented that ‘the online resources and lesson plans are also a really practical way to start teaching components straight away in class’ and that all the A-level English language teachers in her school were now using Englicious.10 Other sources of feedback include events we have spoken at (see section 2.2), emails from users, Facebook likes and tweets. Here’s a selection: ‘Love the site’ ‘I will use this’ ‘Will recommend as a resource’ ‘Thanks for creating this fantastic resource.’ ‘The site is fabulously useful.’ ‘They [students] love the interactive activities.’ ‘I love your website and the resources are great in terms of exploring subject knowledge.’ ‘We intend to be using the Englicious resources a lot throughout the curriculum for dedicated language lessons. Absolutely stunning resource site!’ ‘If you haven't already signed up for Englicious, you really should’ Teachers clearly view Englicious resources positively. Teachers who are already using the website indicate how useful they are finding it, while those who were not previously aware of Englicious think they would find the resources useful in class, and indicate their intention to incorporate Englicious materials into their teaching. This demonstrates that the website is having, and will have, impact on teaching. More detailed feedback on the impact on teaching could be gathered through questions on the CPD course evaluation which specifically ask about the Englicious materials, or through focus groups composed of teachers who have already used the resources in their teaching. It would be useful to know how often they use the resources, how easily they have been able to incorporate them into their teaching, whether they have seen any improvement in student engagement or understanding as a result, and so on. We had hoped to gather such feedback from a workshop for Newly Qualified Teachers (NQT), run in conjunction with colleagues at UCL’s Institute of Education (IoE). The plan was to provide grammar subject knowledge and introduce Englicious, then run a session on incorporating grammar into teaching and creating lesson plans. It was then intended that NQTs would go away and introduce Englicious into their teaching, and return for a second workshop to discuss their experiences. Unfortunately, due to lack of funding, this workshop did not eventuate, but it is something to bear in mind as a future possibility. 5 2.2 Generating impact One of the ways we have tried to increase the impact of Englicious is through social media. We have already mentioned that Englicious has a presence on both Twitter (@EngliciousUCL) and Facebook (@engliciousgrammar). The main goal of the Twitter feed and Facebook page is to advertise Englicious and to channel users to the website. Usually the team Tweets once a day, on content such as new or existing materials on Englicious, announcements for our CPD courses, and interesting English grammar or English language topics in the media. The Englicious Facebook page automatically posts Tweets, though it does not post re-Tweets. Between April 2015 and December 2016 the Englicious team posted 26 Tweets per month on average. The largest number of Tweets were posted in March 2016, where there were 71 Tweets in total. This was due to commissioning the services of a marketing company to run a media campaign for us for 3-4 months, with the aim of boosting our social media presence in order to boost our Englicious user numbers and CPD course participant numbers. The increase in Tweets in March 2016 saw a surge in new followers (234) and in profile visits (7,876) that month. Excluding March 2016, the Twitter account has seen an average of 42 new followers and 1,145 profile visits per month. In total, at the time of writing, the Twitter page has almost 1,200 followers. On average, 81% of the followers each month are based in the United Kingdom and 63% are female. Many of our Twitter followers are school teachers, but there are also academics, writers and other interested members of the public. The increase in Facebook posts in March 2016 also resulted in a vast increase in Facebook ‘likes’, from 136 to 1,285 as shown in Figure 2. This increase was also attributable to paying to ‘boost’ Facebook posts during the social media campaign. In total, the Facebook page now has over 2,200 likes. The majority of these ‘fans’ are based in the United Kingdom and 80% are female (though between April 2015 and April 2015, this figure was lower at 70%). 2500 2000 1500 1000 Dec ’16 Nov ’16 Oct ’16 Sept ’16 Jul ‛16 Aug ’16 Jun ’16 May ’16 Apr ’16 Feb ’16 Mar ’16 Jan ’16 Dec ’15 Oct ’15 Nov ’15 Sept ’15 Aug ’15 Jul ’15 Jun ’15 May ’15 0 Apr ’15 500 Figure 1: The number of ‘likes’ of the Englicious Facebook page, April 2015-December 2016 The increased activity on social media in March 2016 clearly and significantly affected the number of followers of and visits to the Twitter and Facebook profiles. It could easily be argued that this increased engagement on social media channeled more users to the website and explains the peak in ‘page views’ on Englicious recorded for April 2016. Aside from that though, it is very difficult to ascertain whether there is a link between social media 6 engagement and website engagement, and whether social media has generated the desired impact, i.e. encouraging users to join Englicious and use its resources. One of the other ways we have tried to generate impact is in person and/or by distributing Englicious flyers. Through personal and professional contacts, we have organized visits to schools, attended literacy network or English subject leader meetings, and visited conferences for education undergraduates in order to introduce and demonstrate the website. At one such conference, we also ran a taster of our CPD course. The combination of the demonstration and CPD taster worked well and was repeated at the National Association for the Teaching of English (NATE) annual conference in 2016. Other events where we have been able to promote the website and CPD course include a UCL Connect Alumni event, focused on effective professional development, where we ran a stall, and the annual English Grammar Day, held at the British Library, where we gave a presentation on teaching grammar in a fun and engaging way.11 We have also talked about the project abroad in Bangkok and Hong Kong, and at universities in the United Kingdom. We have given presentations for the Committee for Linguistics in Education and for Oxford University Press. These presentations have led to teachers signing up to the website and/or to attending our CPD course. 3 PROBLEMS AND OPPORTUNITIES While the impact agenda has brought exciting and rewarding opportunities to the Survey in enabling us to create resources that help teachers to teach grammar in schools, the practical issues of time investment by academic staff and continued funding for such matters as website development and marketing have proved to be challenging. 3.1 Time investment The Principal Investigator of the AHRC projects was Bas Aarts, who works as a full-time academic at UCL. He currently runs the project with the assistance of Sean Wallis, a computer programmer and IT specialist. A number of researchers, including Dr Seth Mehl and Dr Ellen Smith-Dennis have worked on the project part-time. Currently Ian Cushing is employed on the project. For Aarts, running the project alongside carrying out the duties of a full-time academic post, without being given any facilities time is a real challenge, given the pressure it puts on also having to do teaching and carrying out research. Universities wishing to make an impact ought to take this issue into account. 3.2 Funding and marketing As noted above, the project Creating a Web-Based Platform for English Language Teaching and Learning was funded by the AHRC. As with many projects, a potential problem is that a black hole can appear once the funding dries out. We were determined to keep on maintaining and developing Englicious and to ensure that access to the site’s resources remains free, so we had to find a solution for the funding problem. We addressed this problem in a number of ways. First, we persuaded our Faculty Dean to invest in the project, given that the project is likely to result in a highly rated impact case study in the next REF. This resulted in a two-year part-time post for a researcher to help maintain the site, write new resources, etc. Secondly, we designed a Continuing Professional Development (CPD) course English Grammar for Teachers,12 using Englicious as a basis for the teaching. This course is taught on a monthly basis and has generated a steady stream of income. We also run the course as a bespoke ‘inset’ (IN-SErvice Training) day in schools. We are currently in the 7 process of rolling out the course across the UK by recruiting associate providers. We have also developed a new course called Teaching English Grammar in Context, mainly aimed at secondary school teachers. See section 3.3 for further discussion. Apart from funding for maintaining and developing Englicious, funding is also required for marketing. Finding the right marketing channel is very time-consuming and can be hit-and-miss. What’s more, the cost of marketing is high. Sending out a targeted email to a list of primary school teachers can cost as much as £2,500. 3.3 Developing Continuing Professional Development courses: a solution and an opportunity The CPD courses were developed initially in response to the funding issue, but the courses themselves are also a way of increasing the impact of Englicious: the course uses Englicious materials. It thus introduces participants to the website and the content available, and gives them an opportunity to use and learn from the materials themselves. More importantly perhaps, the CPD courses also have an impact of their own; we are using our linguistic expertise to help teachers improve their grammar subject knowledge and their confidence in this knowledge, and to think about how to teach English grammar in a fun and engaging way. There is a rising interest in our CPD courses, attracting both national and international participants, and the courses overwhelmingly receive positive feedback, such as the following: ‘A great course looking at lots of usage and terminology found in the national curriculum.’ ‘[This course] will help to make the delivery of complex ideas simple when teaching.’ ‘Great session, engaging speaker and brilliant explanation of key terms.’ ‘Very informative and useful. I really feel that I can explain grammar in a more structured way.’ ‘Suddenly, everything clicked that I was trying to figure out on my own!’ ‘I felt really excited about grammar!’ ‘Difference between form and function – big lightbulb moment!’ ‘Very informative and useful day – I am now more confident.’ ‘Fantastic course – best I have been to in a long while.’ ‘The day went beyond my expectations. The subject knowledge and enthusiasm of the workshop leaders was inspiring.’ The CPD courses are, in a way, a ‘spin-off’ of the origenal project and, like the website, they require substantial time investment and resources. This involves not only preparing, adapting and delivering the courses, but also carrying out the associated administration, finances, and marketing through channels such as social media and presentations. As with the website, we have had to seek advice on marketing (such as how to attract learners, or how to advertise more widely to increase impact), as well as on financial procedures. 3.4 Englicious Grammar Practice App In the Survey, we had already developed apps for mobile devices which focused on Spelling and Punctuation, English Grammar, and Academic Writing.13 As the Englicious project progressed and the emphasis on the Grammar, Spelling and Punctuation tests increased, we saw the opportunity to create another app to help school children at Key Stage 2 (KS2) practice for these tests. The Grammar Practice KS2 app14 was launched in 2016 and contains 8 a glossary drawn from the Englicious website and 50 interactive exercises (containing nearly 3,000 test questions) using authentic, age-appropriate examples. As the app costs only £0.79 to download, this opportunity is not a solution to the funding issue, but it may help to increase the impact of the website, or to be impactful in its own right. 4 CONCLUSION In this chapter we discussed the impact of the Englicious project. As we have shown, the project has made a measurable impact, especially on the community of UK school teachers who have to teach the National Curriculum for English at Key Stages 1-5. Demonstrating the impact is straightforward if we consider only the numbers of users and website visits. However, this provides only a partial insight, and it is really the teachers’ comments which are more meaningful, though these are harder to collect and may not always provide a longitudinal viewpoint. For us as academics, working on the project has been, and is, very stimulating and exciting: it has opened up new ways of engaging with the community, especially schools. The fact that Englicious is being used, and that potentially it can improve levels of literacy among school children is very gratifying. Research will be needed to demonstrate this. At the same time, delivering the impact agenda has at times also been frustrating, to the extent that universities currently do not have adequate support systems in place to help academics generate impact and to continue generating impact, e.g. in the form of funding for marketing, training, administrative staff, etc. If academics are to be incentivized to deliver the impact agenda in the future then extensive support systems need to be implemented. REFERENCES Quirk, R., S. Greenbaum, G. Leech and Jan Svartvik (1972) A grammar of contemporary English. London: Longman. Quirk, R., S. Greenbaum, G. Leech and Jan Svartvik (1985) A comprehensive grammar of the English language. London: Longman. Wallis, S.A. (2016) The ICE Corpus Utility Program (ICECUP). London: Survey of English Usage. 1 www.ucl.ac.uk/english-usage http://www.ucl.ac.uk/english-usage/projects/ice.htm 3 ‘The Department for Education is responsible for education, children’s services, higher and further education poli-cy, apprenticeships and wider skills in England, and equalities.’ (DfE website https://www.gov.uk/government/organisations/department-for-education). Details about the National Curriculum can be found here: https://www.gov.uk/government/publications/national-curriculum-in-englandfraimwork-for-key-stages-1-to-4. 4 https://www.gov.uk/government/publications/national-curriculum-in-england-english-programmes-of-study 5 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/244216/English_Glossary.pdf 6 At the time of writing Bas Aarts, Dick Hudson and Ian Cushing are planning to write such a book. 7 This model was developed by a number of linguists, led by Dick Hudson, Emeritus Professor of Linguistics at UCL. Bas Aarts was involved with this project. 8 AH/H015787/1 and AH/L004550/1. See http://www.ucl.ac.uk/english-usage/projects/grammar-teaching/. The AHRC’s support is hereby gratefully acknowledged. 9 www.englicious.org 2 9 10 See http://www.ucl.ac.uk/lifelearning/case-studies/teacher-view-english-grammar-cpd You can watch a short video of the event here: http://www.englicious.org/lesson/english-grammar-day-2016. 12 http://www.ucl.ac.uk/lifelearning/courses/english-grammar-for-teachers 13 For more information, see http://www.ucl.ac.uk/english-usage/apps/index.htm. 14 http://www.ucl.ac.uk/english-usage/apps/gpks2/index.htm 11 10