European Journal of Operational Research 290 (2021) 99–115

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Production, Manufacturing, Transportation and Logistics

Forecasting and planning during a pandemic: COVID-19 growth rates,

supply chain disruptions, and governmental decisions ✩
Konstantinos Nikolopoulos a,∗, Sushil Punia b, Andreas Schäfers c,1, Christos Tsinopoulos a,
Chrysovalantis Vasilakis c,d
a
Durham University Business School, United Kingdom
b
Indian Institute of Technology Delhi, India
c
forLAB, Bangor Business School, Wales
d
Université Catholique de Louvain (IRES) (Belgium), Institute for the Study of Labor (IZA) (Germany), University of the Aegean (Greece)

a r t i c l e i n f o a b s t r a c t

Article history: Policymakers during COVID-19 operate in uncharted territory and must make tough decisions. Opera-
Received 5 June 2020 tional Research – the ubiquitous ‘science of better’ – plays a vital role in supporting this decision-making
Accepted 3 August 2020
process. To that end, using data from the USA, India, UK, Germany, and Singapore up to mid-April 2020,
Available online 8 August 2020
we provide predictive analytics tools for forecasting and planning during a pandemic. We forecast COVID-
Keywords: 19 growth rates with statistical, epidemiological, machine- and deep-learning models, and a new hybrid
Forecasting forecasting method based on nearest neighbors and clustering. We further model and forecast the excess
COVID-19 demand for products and services during the pandemic using auxiliary data (google trends) and simu-
Pandemic lating governmental decisions (lockdown). Our empirical results can immediately help policymakers and
Excess demand planners make better decisions during the ongoing and future pandemics.
Lockdown © 2020 Elsevier B.V. All rights reserved.

1. Introduction and motivation number of cases and deaths is still exhibiting significant growth
in many countries, with the Americas (most notably the USA and
First spotted in Wuhan in China, the ongoing COVID-19 pan- Brazil) been in the pandemic’s epicenter.3
demic has triggered the most severe recession in nearly a cen- Our generation has never met anything remotely similar to
tury and, according to the OECD’s latest Economic Outlook,2 it this pandemic. Despite HIV/AIDS been associated with far more
has been causing enormous damage to people’s health, jobs, and deaths,4 the speed with which COVID-19 can kill even-perfectly-
well-being. COVID-19 has affected almost all countries in the world healthy humans (sometimes within just a few days), and the un-
and, has practically put the entire planet on hold for more than precedented disruption in work and social life that it has brought
2 months. At the time this paper was being revised, the number (getting workers furloughed for months, and the vulnerable part
of confirmed global cases was more than 13 million; the number of the population in strict isolation for 12 weeks), makes this pan-
of deaths crossed the mark of 50 0,0 0 0 in late June 2020 – stand- demic unique.
ing at 571,689 as of 13–7–2020 (WHO, 2020). Unfortunately, the Furthermore, due to this pandemic and the associated global
healthcare crisis, supply chains have faced significant disruptions
in the upstream, while hoarding and panic buying caused equally
✩
We conducted this research in April-June 2020 during the COVID-19 pandemic.
significant disruptions to the downstream. The balance of supply
The aim is to provide tools for immediate use during the pandemic: desperate times
call for desperate academic measures, and as such this is our direct response to in- and demand was further impacted by the travel restrictions and
form practice. We employ a phenomenon-based research methodological approach, lockdowns implemented by several countries worldwide. Due to
engaging in an early phase of a scientific inquiry, observing, researching, and pro- these disruptions, short-term real time forecasts (daily and weekly)
viding solutions for a developing and novel phenomenon. about the pandemic and its effect on the supply chain have be-
∗
Corresponding author.
come a very important managerial and policy-making imperative.
E-mail addresses: kostas.nikolopoulos@durham.ac.uk (K. Nikolopoulos),
SchaefAnd@gmx.de (A. Schäfers), chris.tsinopoulos@durham.ac.uk (C. Tsinopou- Mid-and long-term forecasts are essential too for supply chain
los). planning (at monthly, quarterly and annual frequency). However,
1
Opinions expressed are solely my own and do not express the views or opinions
of my previous or current employers.
2 3
http://www.oecd.org/newsroom/global-economy-faces-a-tightrope-walk-to-recovery. https://coronavirus.jhu.edu/
4
htm https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1326444/

https://doi.org/10.1016/j.ejor.2020.08.001
0377-2217/© 2020 Elsevier B.V. All rights reserved.
100 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

research on these is more likely to be conclusive after the first demic. Therefore, this study provides a methodological contribu-
wave of the pandemic is over, when more – and more reliable – tion as it illustrates how to perform such a forecasting exercise. A
supply chain data becomes available. prerequisite for this is that data from the academic and policymak-
An accurate forecast of the evolution of new cases enables the ing community becomes available in accessible formats.6
more effective management of the resulting excess demand across For the remainder of this paper in Section 2 we review the liter-
the supply chain. Common sense and recent experience suggest ature while in Section 3 we present our empirical forecasting com-
that the acceleration and progression of COVID-19 across coun- petition. In Section 4 we provide models for estimating the excess
tries drives changes in immediate actual needs (healthcare and demand and respective supply chains disruptions. In the final sec-
food) and in consumer behavior (for example panic buying and tion we provide our conclusions and implications for practice.
overstocking at home5 ). Such changes put an enormous strain to
the respective supply chains. For instance, when consumers start 2. Background literature
panic buying dry pasta, eventually, the whole supply chain involv-
ing eggs, flour, wheat, is affected. A phenomenon, which is likely to In Section 2.1, we provide a targeted review on different tech-
be significantly exacerbated by the well-known implications of the niques and methods used for the forecasting of the evolution of a
Bullwhip effect (Chen, Drezner, Ryan & Simchi-Levi, 20 0 0; Kahn, pandemic. After that, in Section 2.2, we provide a review of the
1987; Lee, Padmanabhan & Whang, 1997; Wang & Disney, 2016). literature on forecasting the demand and supply in a supply chain
Therefore, forecasting during the pandemic becomes essential in view of the evolution of a pandemic. In the last sub-section, we
for effective governmental decision making, for managing supply present our research questions and our methodological approach.
chain resources, and for informing very difficult political decisions
as, for example, imposing a lockdown or curfews. Yet, forecast- 2.1. Forecasting the evolution of a pandemic7
ing the evolution of the pandemic i.e. the growth in the number
of cases per country, or even to greater spatial detail, is a com- Forecasting methods for pandemic evolution can be divided
plex task because of the limited history of pandemic data and the into time-series methods, compartmental epidemiological models,
multidimensionality of the problem. For instance, there are sev- agent-based models, metapopulation models, and approaches in
eral, and at times unknown, factors that affect the contagiousness metrology (Nsoesie et al., 2013). A recent addition to this long list
and the severity of the disease. To that end, forecasting in real is machine learning (ML) and deep learning (DL) methods (Yang
time and while new data becomes available is a complex exercise et al., 2020). Soebiyanto, Adimi and Kiang (2010) proposed the
for both government and supply chain managers (Beliën & Forcé, use of ARIMA models for one-step ahead forecasting of influenza
2012; Nikolopoulos, 2020). weekly cases. Andersson et al. (2008) proposed the use of re-
Epidemiologists have been applying traditional models for out- gression methods for the prediction of the peak time and volume
break prediction (Nsoesie, Marathe & Brownstein, 2013; Yang (of cases) for a pandemic and provided promising empirical evi-
et al., 2020). Applied mathematicians, decision scientists, and oper- dence to that end from seven outbreaks (in Sweden). Shaman and
ational researchers have been employing time-series, and machine- Karspeck (2012) used the Kalman filter based SIR epidemiological
learning techniques. As a result, for COVID-19, since the on- model to forecast the peak time of influenza and claimed that the
set of the crisis, a few statistical and regression-based forecasts peak can be predicted 6–7 weeks in advance.
have been available online (Al-Shammari et al., 2020; Team IHME An extensive evaluation of multiple time series methods for
COVID-19 & Murray, 2020a, 2020b). Yet, and despite the contribu- forecasting the evolution of an epidemic (Hantavirus) with data
tion of these models for predicting the progress of the virus and its from CDC8 was performed by Yaffee et al. (2008) in which they
impact on the supply chain, their proliferation generates confusion. compared casual methods with 16 time-series univariate meth-
The most profound manifestations of this confusion have been the ods and found that univariate methods were better at prediction
different approaches taken by companies and governments to deal than causal models. For COVID-19, Petropoulos and Makridakis
with the pandemic, e.g. timings and extents of lockdown, processes (2020) applied ETS (Hyndman, Koehler, Snyder & Grose, 2002)
of reopening the economy etc. and the differing, and often confus- models for predicting the evolution of the number of cases at a
ing, views about the onset of a second wave. This has been ex- global scale. They reported very successful results in terms of real
acerbated by the wider recognition that different countries and, accuracy both for their point forecasts and the prediction intervals
even, different regions are structurally diverse. Thus, using a single they provided. This is an open-access article in PLOS ONE that has
forecasting model may not accurately predict how the pandemic already drawn significant attention with 70,852 views up to 29–
evolves. As a result, there is an emergent and urgent need for, on 05–2020 while available online for only 2 months, providing evi-
the one hand, more of these models (Petropoulos, Makridakis, Assi- dence of the interest and importance of such quantitative studies
makopoulos & Nikolopoulos, 2014) and, on the other, a methodol- for academia and practice. Finally, there has been a series of stud-
ogy that enables decision makers to select the one, which is likely ies focusing on predicting deaths in the USA and European coun-
to be the more applicable in their own context. tries for the next few months of the first wave of the COVID-19
To address this need, in this article we forecast the growth of pandemic (Team, IHME COVID-19 & Murray, 2020a, 2020b).
the pandemic at the country-level and evaluate 52 time-series, epi- Furthremore, researchers and software companies have also
demiological, machine-learning, and deep-learning techniques. Fur- rapidly during the COVID-19 pandemic developed live-simulators
thermore, we propose a new hybrid forecasting method tailored to which make use of simulation models9 integrating governmental
the task that is using cross-country information. To achieve gener- decisions (e.g. lockdown) and have been made available online via
alizable results, we use data from a diverse set of countries (UK, freely accessible websites and portals.
USA, India, Germany, and Singapore), and perform a rolling fore-
casting evaluation consisting of 46 daily and 6 weekly forecasts. 6
Source code of our forecasting models is freely available upon request.
Our research can easily be extended into all the countries affected 7
We focused only on peer-reviewed and preprints in the literature review. For
by the pandemic. We further use these forecasts in order to esti- portals for live-prediction, reference is made in the introductory section.
8
mate the excess demand for products and services during the pan- https://www.cdc.gov/hantavirus/index.html
9
https://exchange.iseesystems.com/public/isee/covid- 19- simulator/index.html#
page6https://forio.com/app/jeroen_struben/corona- virus- covid19- seir- simulator/
5
https://www.economist.com/britain/2020/03/21/how- panic- buying- is- affecting- index.html#decisions.htmlhttps://metasd.com/2020/03/interactive-coronavirus-
supermarkets models/https://metasd.com/2020/03/community-coronavirus-model-bozeman/
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 101

2.2. Supply chain disruptions due to a pandemic 2.3. Research questions, methodological approach, and implications
for theory
Supply chain disruptions have been known to cause significant
challenges and can affect organization performance (Hendricks & Considering the targeted literature presented, our research aims
Singhal, 2003). Famous incidents, such as the tsunami that hit to address the following research questions:
Japan in 2011 and the financial crisis of 2008 have illustrated how
the interconnectedness and global nature of the supply chains can R1. What are the best models for forecasting the evolution of the
pandemic at the country-level?
amplify even the smallest of “glitches” (Hendricks & Singhal, 2003).
As a result, there have been several studies that attempt to ex- R2. How can we forecast the excess demand for products and ser-
plain the antecedents of resilient supply chains, both at the net- vices during the pandemic, before even actual supply and demand
work (Kim, Chen & Linderman, 2015) and the organization levels data become available?
(Bode, Wagner, Petersen & Ellram, 2011). Pettit, Croxton and Fiksel
(2019) and Pettit, Fiksel and Croxton (2010) offer a good review of We need to emphasize that we address the aforementioned re-
the literature on supply chain resilience that predates COVID-19. search questions during the pandemic and not after it, and thus
The severity of the business disruption of COVID-19 pandemic the urgency and importance of our ongoing research. This caveat
has challenged much of our previous understanding of what con- constitutes a contribution by itself, as it evidences the ubiquitous-
stitutes a resilient supply chain. Recent reports have clearly in- ness, responsiveness, and the timeliness of OR research.
dicated that this crisis has led to the rapid deterioration of sev- We deploy an exploratory methodological approach in order
eral business and economic indicators, including productivity and to find the best forecasting methods (the ‘horses for courses’ –
global GDP (Harris, 2020). In addition, a few studies also esti- Petropoulos et al., 2014) – as we do not prescribe which meth-
mated the impact of COVID-19 on the labor demand, a 16.24% de- ods/models we expect to perform better via a set of formal hy-
crease in the demand of working hours (Castro, Duarte & Brinca, potheses. Then via a series of simulations we forecast the excess
2020). These impacts are due to the imposition of travel and trade demand of products and services, i.e. the excess demand that is
(Baveja, Kapoor & Melamed, 2020) restrictions and the shutting driven from the growth of COVID-19 cases. Our analysis covers
down of work places. a major part of the current wave of the pandemic, the period
As a result, Araz, Choi, Olson and Salman (2020) asserted that from the 22 January 2020 to 15 April 2020. From a methodolog-
COVID-19 is, probably the most severe disruption to the global sup- ical standpoint, we contribute to the stream of Phenomenon-based
ply chain in the last decade. Ivanov (2020), who considered the research as we engage in a very early phase of a scientific inquiry,
pandemic and the respective supply chain risks, provided a simu- observing, researching, and providing solutions for a developing a
lation model for global supply chain disruption and predicted the novel phenomenon (von Krogh, Rossi-Lamastra & Haefliger, 2012).
severity of COVID-19 s impact on supply chain performance. Sim- We further contribute both to the fields of Operations Research
ilarly, Team IHME COVID-19 and Murray (2020a) predicted that (OR) and Supply Chain Management (SCM). For the former, we pro-
COVID-19 will place unprecedented stress on hospitals, ICUs, and vide an exhaustive empirical investigation that identifies the most
ventilators, and that the overall demand will be beyond the health- accurate method for forecasting growth rates during a pandemic.
care system’s current capacity. In a follow up study, Team IHME We do so during the phenomenon and before the start-growth-
COVID-19 and Murray (2020b) predicted the impact of COVID-19 maturity-decline sequence is complete. We contribute to the lat-
on hospitals and deaths for Europe and US and suggested measures ter, the field of SCM, by providing an input (the demand forecasts
to temporarily increase the supply of critical products and services. for the new cases and the selected products), which is essential
Govindan, Mina and Alavi (2020) presented a decision support sys- to decision-making algorithms that involve stock-control, replen-
tem to manage the demand for healthcare supplies based on physi- ishment, advance purchasing, and even rationing,10 i.e. situations
cians’ knowledge and Fuzzy Inference System (FIS). They claimed that require a mean forecasted demand over the lead-time. We fur-
that the use of their propositions leads to efficient and accurate ther provide simulations for the excess demand for products and
managing of the supply chain disruptions in case of an outbreak. services during the pandemic.
Finally, Hobbs (2020) assessed the implications of COVID-19 Finally, we contribute to the theory of predictive analytics, as
on the food supply chains and reported that demand and supply we propose new data-driven predictive methodologies. We do so
shocks created during a pandemic are due to a shift in consumer by building on theory from non-parametric regression smooth-
behaviors. For instance, the sudden panic buying shift to ready- ing on Nearest Neighbors (Härdle, 1990), and by using machine-
meals caused demand shocks, which then led to labor shortages, learning clustering approaches. We capitalize on the experience of
and disruptions of the transportation network. Furthermore, re- those countries where the outbreak of the pandemic came earlier
strictions on cross-border goods movement led to further supply to forecast the evolution of the pandemic. We also contribute to
side shocks to the food supply chains. As a result, it would be rea- policymaking as we take into account the impact of political deci-
sonable to conclude that COVID-19 will have long-lasting effects on sions – specifically the enforcing of a lockdown/curfew11 - on both
consumer habits and supply chains. the evolution of the pandemic and the resilience of the affected
In summary, COVID-19 has put some significant and unprece- supply chains.
dented strain on global supply chains across most product cate-
gories. Past literature on forecasting and on supply chain disrup- 3. Forecasting the evolution of the pandemic
tion has been able to provide some indication of the factors that
can lead to it. However, and at the same time, it has exposed Following the influential12 empirical forecasting evaluation at
some of the challenges associated with identifying and respond- the global level of Petropoulos and Makridakis (2020), we perform
ing to significant changes in the demand patterns during a pan- our empirical forecasting analysis at the country-level. This is also
demic. The ability to forecast excess demand during the pandemic the most common geographical level for decision-making during
early could, however, has significant implications for both supply
chain managers and policy makers. The former can benefit from 10
https://uk.reuters.com/article/us- health- coronavirus- britain- supermarke/
early warnings about where resources will be needed and the lat- panic- buying- forces- british- supermarkets- to- ration- food- idUKKBN21511M
ter from a data driven approach to government interventions, e.g. 11
https://en.wikipedia.org/wiki/National_responses_to_the_COVID-19_pandemic
by prioritizing critical supply chains. 12
81,563 views to date in just over 3 months
102 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

the pandemic. Although at the time of writing there was data from Table 1
Forecasting methods.
215 countries we decided to focus our study on five of these. We
did so for both brevity and for providing a clearer illustration of Category Method
the benefits of the methods we used. The countries we selected Time-series Naïve, Moving Averages (four models 2,3,4,7),
are: Germany, India, Singapore, the United Kingdom, and the USA SES, ETS, ARIMA, Theta, TBATS, ANN_AR,
as they cover a wide range of national systems and government G&M (1985)-Damped trend (Gardner &
responses. More specifically: McKenzie, 1985), Holt - Trend, ns-HW
(non-seasonal Holt-Winters), ARFIMA,
GARCH(1,1) (six models, wih: GED, SGED,
• Germany because it is the country with the best response in NORM, SNORM, STD, SSTD), ARIMAx,
Europe. This is despite neighboring with badly affected coun- Naïve-d with drift (ten models with step of
0.1 for the drifta )
tries and being very close to the epicenter of the outbreak
Machine Learning Multiple linear regression (MLR), Ridge
in Europe: Italy. Germany is also of interest as it followed a regression, Decision Trees (DT), Random
very aggressive testing policy early on, trying to identify each forest (RF), Neural Network (NN), Support
and every case as early as possible. As of 12/07/2020 a total vector machine (SVM).
of 200,047cases with 9135 deaths have been confirmed, bring- Deep Learning Long-Short Term Memory networks (LSTM)
Others Splines, Sigmoid, Partial Curve Nearest
ing the deaths per capita at 109/1 M of population, much lower Neighbor methods (PC–NN), Multivariate
than most G20 countries. Clustering based Partial Curve Nearest
• India because it is the most populous country in the world still Neighbor methods (CPC–NN)
affected by the pandemic (with a population more than 1380 M, Epidemiological SIR (two models with: beta = 1.16,
gamma = 0.38b ; beta = 1.4, gamma = 0.3c )
second largest in the planet). On this basis we did not include
a
China because it is considered to have completed the first wave For the Naïve with drift approaches the reader can revisit Nikolopoulos, Buxton,
Khammash and Stern (2016).
in April 2020. As of 12/07/2020 a total of 888,944 cases has b
https://science.sciencemag.org/content/368/6492/742.abstract.
been reported (third-most in the world) with 23,333 deaths. c
https://www.r-bloggers.com/sir-model-with-desolve-ggplot2/.
• Singapore because it is the country with one of the most ad-
vanced healthcare systems in the planet,13 a claim supported
profoundly during this pandemic as well. Despite Singapore not We collected all data on COVID-19 cases and respective health-
employing extremely strict lockdown measures, it had a very care and socioeconomic variables from credible international pub-
aggressive testing approach, deployed a very effective tracking licly available sources (listed in Appendix A).
mobile application14 and has been in the forefront of tech-
nology adoption and development e.g. it developed a wear- 3.1. Forecasting methods
able device to obtain better results than those achieved by the
mobile app15 . As of 12/07/2020 a total of 46,283 cases has We used a set of 52 models (from more than 20 methods21 ),
been reported and a total of only 26 deaths, with 42,285 con- ranging from simple to complex, and from time-series and epi-
firmed recoveries. This performance equals to a mortality rate demiological, to machine- and deep-learning. We produced fore-
at 0.06%, which is less than long term average for the seasonal casts for the growth rates at various stages of the pandemic for
flu (0.1%16 ) for which both a vaccine17 and a first-line antivi- each nation. In total we produced forecasts 46 times for daily data
ral treatment18 is available, rendering this country’s response and 6 times for weekly data. We identified the top-three methods
arguably the best in the planet. This is despite Singapore be- per country that exhibit the smaller Mean Absolute Scaled Error
ing one of the first hit by the pandemic, right after China and (MASE), and used the equal-weighted combination of these meth-
Taiwan. ods for the follow-up simulations in Section 4. We used the sim-
• the UK because it has been the most-affected country in Europe ple average of forecasts, as it is a simple and effective method for
and the one with the most deaths per capita at 660 deaths per combining forecasts (Makridakis & Winkler, 1983). We used MASE
1 M of population (worst among countries with population over as our primary accuracy metric (Hyndman & Koehler, 2006) be-
15 M). As of 12/07/2020 the UK had reported a total of 289,603 cause it is scale-independent and widely accepted metric for fore-
cases and 44,819deaths. The UK is also of interest as it has the cast evaluations (Makridakis, Spiliotis & Assimakopoulos, 2020).
largest public healthcare system in Europe19 (and 2nd largest In Table 1 we provide a list of competing models. For details on
single-payer healthcare system in the world). It also followed these popular methods, the interested reader may revisit either the
a different approach early in the pandemic – aiming for “herd article on the latest forecasting competition (the M4 competition
immunity” rather than virus containment.20 – Makridakis et al., 2020) or the free online forecasting textbook
• And finally, the USA because it has been the most-affected from Hyndman and Athanassopoulos.22 For the more advanced
country in the world by the outbreak (up to the time of machine- and deep-learning methods we provide a brief descrip-
this submission). As of 12/07/2020 it had reported a total of tion in Appendix A.
3,415,573 cases and 137,797 deaths.
3.1.1. A new forecasting proposition: nearest neighbor approaches for
forecasting the evolution of a pandemic
13
The new proposition is data-driven and designed to use his-
https://www.who.int/whr/20 0 0/en/
14
https://www.tracetogether.gov.sg/
torical data from several countries to produce better forecasts for
15
https://www.forbes.com/sites/johnkoetsier/2020/06/05/ a target country. This is a classic adaptation of a Nearest Neigh-
singapore- building- wearable- tracking- device- for- citizens- because- phone- based- bor approach (Härdle, 1990; Kyriazi & Thomakos, 2020). We have
covid- 19- tracking- isnt- good- enough/#827c2f6e72c8 named it Partial Curve Nearest Neighbor Forecasting (PC–NN) be-
16
https://khn.org/news/fact-check-coronavirus-homeland-security-chief-flu- cause it tries to find similarities in between parts of curves (from
mortality-rate/
17
https://www.cdc.gov/flu/prevent/keyfacts.htm
18 21
https://en.wikipedia.org/wiki/Oseltamivir In our terminology, a method for e.g. MA (Moving Averages), is been partici-
19
https://en.wikipedia.org/wiki/National_Health_Service_(England) pating in our empirical competition with four different models: 2-MA, 3-MA, 5-MA,
20 7-MA.
https://www.nationalgeographic.com/science/2020/03/
22
uk- backed- off- on- herd- immunity- to- beat- coronavirus- we- need- it/ https://otexts.com/fpp3/
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 103

Fig. 1. An example of COVID-19 cases and a fitted smooth curve (USA).

the start of the time series of a pandemic in a country until the VI. Finally, using the PC–NN groups identified in Step V for a
date the forecast is made, as depicted in Fig. 1). The method in- country A and the (N + 1 )th period naïve forecasts for these
volves the following steps: countries, the (N + 1 )th period forecast is produced for coun-
try A using a simple average of all PC–NN’s naïve forecasts.
I. Collecting the data for a period of T days on daily cases growth In the case of PC–NN3 we can either use equal weightings
for a set of N countries. The data vectors for N countries can be for the three neighbors (PC–NN3ew), or uneven/triangular ones
represented as: [Y1 ; Y2 ; . . . ; YN ]. (PC–NN3uw). This would mean that a 50% weighting is given to
II. Fitting a smooth curve to each data series from each country the nearest neighbor and 25% to the other two. For future re-
separately, [Y1 ; Y2 ; . . . ; YN ]. This is done by selecting the best fit search, we would recommend employing the next actual values
(min squared distances optimization) from a pool of moving av- of the neighbors instead of a naïve forecast.
erages methods like 2-MA, 3-MA, 4-MA, and 5-MA
III. Calculating the daily changes in the smooth curve (fitted in Step We further extend this approach by using a multivariate dataset
II) for each nation, [Y1 ; Y2 ; . . . ; YN ]. and a clustering algorithm. We performed the clustering with
IV. Comparing the daily changes curve of country (A) to those of data on socioeconomic, climate, and COVID-19 related factors and
other countries. To do so in a simple and effective manner, we grouped them according to whether they are facing, or they are
normalized the data and we calculated the Euclidean distance about to face similar challenges. We used the K-means23 cluster-
between curves as the squared root of the sum of squared dif- ing algorithm to find the clusters of the countries. The countries
ferences between the selected country’s curve (e.g. India) and that are in the same cluster will probably face a similar situa-
those of others. Based on the values of the (ri j ), we selected tions and challenges related to COVID-19 in the future, especially
the nearest neighbors. if they adopt similar policies. We consider this a very important
feature of this forecasting method as it allows the clustering. The
NN Distance F orumulation : policy implication here is that there may be policies that some can
For a country (A ), learn from some but not from others. We call this method here-
 2  2  2 after: Clustering and Partial Curves and Nearest Neighbor Forecasting
N Ndist = [ YAt1 − Y2t1 + YAt2 − Y2t2 + . . . + YAtT − Y2tT ; (CPC–NN). We use similar notations for the variations of this latter
extension too: CPC–NN1, CPC–NN3ew, CPC–NN3uw, CPC–NN5, and
 2  2  2 CPC–NNall.
YAt _1 − Y3t _1 + YAt _2 − Y3t _2 + . . . + YAtT − Y3tT ;
3.2. Country-level forecasting
 
2  2  2
YAt1 − YN t1 + YAt2 − YN t2 + . . . + YAtT − YN tT , We produced forecasts for the growth rates at various stages of
the pandemic for each of the five nations. In total we conducted
or this process 46 times for daily data and 6 times for weekly data
 for the period of 22 January 2020 to 15 April 2020. We derived the
T 
 2 time series of percentage daily changes in COVID-19 cases from the
N Ndist = [rA2 , rA3 , . . . , rAN ], rAB = YAt − YBt
i i daily new case series. We calculated the daily percentage growth
i=1
with the following equation.
= 1, 2, . . . , T , and A = B.  C ases − C asest

t+1
Daily Growth in % = ∗ 100
V. Defining groups using the r1 j values for a country (A): PC–NN1 Casest
(A and its closest nation), PC–NN3 (A and its two closest coun-
tries), PC–NN5 (A and its 4 closest countries), and PC–NN-all 23
The reader may revisit the theory of the k-means at https://stanford.edu/∼
(all countries). cpiech/cs221/handouts/kmeans.html
104 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

Table 2
Forecasting accuracy: MASE and SMAPE (Relative median errors to Naïve) for all methods across all weeks (days) and across
all countries, by proposed method. [Methods listed alphabetically].

Weekly Daily
Model
RelMdMASE RelMdSMAPE RelMdMASE RelMdSMAPE

MA3 (best MA for weekly) 1.1527 1.2385 0.3402 0.3438

MA7 (best MA for daily) 1.2443 1.2265 0.2602 0.3336
ARFIMA 1.3828 1.4207 0.2709 0.3373
ARIMA 1.3828 1.4209 0.3297 0.3371
ARIMAx 2.0763 1.9343 0.3798 0.3926
DT 1.4219 1.5663 0.3400 0.3445
ETS 1.0427 1.2353 0.2813 0.2284
G&M (1985)–Damped trend 1.8048 1.4603 0.3459 0.3640
GARCH(1,1) model with SGED(best GARCH for daily) 1.2893 1.5166 0.2064 0.2160
GARCH(1,1) model with SSTD (best GARCH for weekly) 1.2236 1.3250 0.3312 0.4423
holt – trend 2.0588 1.6877 0.3543 0.3470
LSTM 1.9028 1.4825 0.3170 0.3885
MLR 1.4504 1.4187 0.3284 0.3515
Naive-d 0.1 (best Naïve-d for weekly) 1.0015 1.0022 1.0754 1.0687
Naive-d 0.9 (best Naïve-d for daily) 1.6488 1.5895 0.4130 0.5409
NN 1.2093 1.3848 0.3164 0.3745
NN_AR 1.9233 1.6155 0.3877 0.4467
ns-HW 1.3949 1.4220 0.2778 0.2443
RF 1.4180 1.5813 0.5024 0.4814
Ridge 1.4621 1.3408 0.3438 0.3147
SES 1.3949 1.4211 0.2765 0.3372
Sigmoid 2.1129 1.8262 1.5698 1.9833
Splines (CV) 0.9846 1.2572 1.2826 1.1985
SVM 1.2678 1.1858 2.1718 1.8120
TBATS 2.2840 1.9044 0.3597 0.4578
Theta 1.4200 1.5195 0.3844 0.4546
SIR-1 3.4664 2.0000 11.8852 2.0000
SIR-2 3.2383 2.0000 10.6193 2.0000

We used the forecasting methods listed in Table 1 for forecast- revised its social distancing measures based on forecasts and actual
ing for all the countries across all time periods. We used the death data for the pandemic every 3 weeks.
and recovery rates as the independent variables for the multivari- For the daily data the picture is very different, with many
ate forecasting methods. We calculated the Mean Absolute Scaled methods outperforming the Naive method. The GARCH(1,1) model
Error (MASE) and the Symmetric Mean Absolute Percentage Er- with SGED with 0.2064 ranks first and MA7 with 0.2602 for
ror (SMAPE) for each iteration (Makridakis et al., 2020; Shankar, MASE ranks second. On the other hand, for SMAPE, GARCH(1,1)
Ilavarasan, Punia & Singh, 2019). We calculated the relative errors model with SGED with 0.2160 ranks first and ETS with 0.2284
by dividing with the corresponding error from the naïve method ranks second. The two epidemiological models did not perform
(Punia, Nikolopoulos, Singh, Madaan & Litsiou, 2020). We report in well.
Table 2 the relative (to naïve) medians for: MASE (RelMdMASE), However, for the weekly data at country level (Table 3), the av-
and SMAPE (RelMdMAPE).24 erage of top-three methods performs significantly better than the
Since we are interested in finding the overall ‘winner’ across the naïve forecast. Most of the relative errors are less than 0.5 indi-
competing methods, we need to evaluate the methods across the cating the large performance improvement over naïve by the pro-
five countries simultaneously. To do so, we produced forecasts for posed methodology, and the respected anticipated benefits of com-
all competing methods, for each period and country. We then cal- binations (Makridakis et al., 2020).
culated the medians of the forecasting errors across all countries. One key conclusion from Table 3, is that the level of error is not
Table 2 includes the results.25 the same across all countries and for some it is easier to forecast
We observe from Table 2 that the performance of the Naïve than others. For example, for Singapore the error on the weekly
method was very difficult to beat for the weekly data: only data is 0.1260 and the one for daily 0.1292. At the other end, for
Splines (CV) did better. This led us to develop models using the the UK the error on the weekly data is 0.2674 for the USA 0.3032
PC–NN/CPC–NN method. In Table 3 and in the next subsection we (on the daily ones). Therefore, a key conclusion is that forecasting
demonstrate that these models do outperform all other methods at the country level is more likely to lead to effective local guid-
for weekly data, but at a computational cost. Given that many pol- ance and would need to consider different underlying time series.
icy decisions are taken weekly, a weekly frequency and forecasting A second key conclusion is that different methods perform bet-
horizon becomes very important for planning. For instance, the UK ter in different countries. For example, for Germany Naïve and two
variants of Naïve with drift are the top-performing models; while
for the USA the two variants of PC–NN3 and CPC–NN3uw are the
24
We do find similar results when we are using the medians of ME and RMSE, as top-performing ones. Thus, the forecasting evaluation needs to be
well as the averages of them. Average errors can be used in parallel with median performed in every country separately: this is a consistent result
errors to help identify in which countries we do face more extreme errors (when
Avg>>Md).
with the ‘horses for courses’ doctrine (Petropoulos et al., 2014) as
25
For each of the 5 countries (Germany, USA, UK, India, and Singapore) according well as the Makridakis forecasting competitions (Makridakis et al.,
to the Wilcoxon test (pairwise) the difference of the performance of the top three 2020). In Table 3 we present the results on the weekly data, as
methods (for each country) and their combination is statistically significant differ- this is the frequency where all methods competed. This is because
ent(and better) than the remaining methods at 95% confidence across all metrics. the PC–NN/CPC–NN family is computationally intensive and thus
In between the top three methods there were much less differences, most of the
times not statistically significant.
we only evaluated it over the six weekly forecasts.
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 105

Table 3
Relative median errors across all weeks, from the average of the top-3 performing methods for each country.a

Country Weekly Daily Top-3 models

RelMdMASE RelMdSMAPE RelMdMASE RelMdSMAPE (Weekly forecasts)

Germany 0.1758 0.2264 0.2573 0.2672 Naive-d 0.5; Naive-d 0.4; Naive
India 0.1484 0.1252 0.2357 0.2727 CPC–NN3uw; PC–NN5; CPC–NN3ew
Singapore 0.1260 0.1290 0.1292 0.1363 PC–NN5 PC–NNall; PC–NN3ew
United Kingdom 0.2674 0.2792 0.2221 0.2584 Naïve; PC–NNall; CPC–NNall
USA 0.1907 0.2254 0.3032 0.2877 PC–NN3uw; PC–NN3ew; CPC–NN3uw
a
For the sake of robustness checks, we also repeated this analysis (Tables 2 and 3) for 22 countries as well with similar results.

Table 4
Forecasting uncertainty: MSIS, Cover Rate and ACD as in Makridakis et al. (2020) (Relative median errors to Naïve)
for all methods across all weeks (days) and across all countries, by proposed method. [Methods listed alphabetically].

Method Weekly Daily

RelMdMSIS Cover Rate ACD RelMdMSIS Cover Rate ACD

MA2 1.0014 1.0000 0.0500 0.8424 0.9956 0.0456

MA7 0.9068 1.0000 0.0500 0.7444 0.9913 0.0413
ARFIMA 0.7311 1.0000 0.0500 0.6981 0.9652 0.0152
ARIMA 0.7664 1.0000 0.0500 0.7034 0.9652 0.0152
ARIMAx 0.6880 0.8696 0.0804 0.4657 0.9620 0.0120
DT 0.7219 1.0000 0.0500 0.5194 0.9851 0.0351
ETS 0.8070 1.0000 0.0500 0.7081 0.9652 0.0152
G&M (1985)-Damped trend 0.9558 1.0000 0.0500 0.7167 0.9783 0.0283
GARCH(1,1) model with SGED 0.4814 1.0000 0.0500 0.3693 0.9340 0.0160
GARCH(1,1) with NORM 0.6485 1.0000 0.0500 0.5706 0.9522 0.0022
holt - trend 0.9233 1.0000 0.0500 0.7146 0.9696 0.0196
LSTM 0.8146 0.9500 0.0000 0.7017 1.0000 0.0500
MLR 0.7209 1.0000 0.0500 0.6991 0.9652 0.0152
Naive-d 0.1 0.9823 1.0000 0.0500 0.9996 1.0000 0.0500
Naive-d 1.0 0.8947 1.0000 0.0500 1.0334 0.9737 0.0237
NN 0.7915 1.0000 0.0500 0.9714 0.9769 0.0269
NN_AR 0.6864 0.9667 0.0167 0.6963 0.9609 0.0109
ns-HW 0.8306 1.0000 0.0500 0.7095 0.9652 0.0152
RF 0.7982 1.0000 0.0500 0.7987 0.9817 0.0317
Ridge 0.8135 1.0000 0.0500 0.8250 0.9742 0.0242
SES 0.8306 1.0000 0.0500 0.7094 0.9652 0.0152
Sigmoid 0.9354 1.0000 0.0500 0.9486 0.9315 0.0185
Splines (CV) 0.7974 1.0000 0.0500 0.7892 0.9260 0.0240
SVM 0.8413 1.0000 0.0500 0.8486 0.9264 0.0236
TBATS 0.7011 1.0000 0.0500 0.7031 0.9565 0.0065
Theta 0.8306 1.0000 0.0500 0.7094 0.9609 0.0109

Following the analysis and recommendations of Makridakis to changes in both the spread COVID-19 and the decisions policy
et al. (2020) we further calculated the prediction intervals for our makers are taking. The algorithm selects the number of clusters
competing methods. We also assessed the respective performance based on maximum height in the dendrograms. Many other clus-
with MSIS, Cover Rate and ACD in a very similar fashion as in tering algorithms could be employed before forecasting (see for ex-
the M4 competition. The results are presented in Table 4. The ample Vangumalli, Nikolopoulos & Litsiou, 2019), but we leave this
GARCH(1,1) model with SGED gives the best MSIS for both weekly for future research.
and daily data -with a perfect cover rate for the weekly; while for Table 6 presents the performance of PC–NN/CPC–NN. The re-
the daily data LSTM and Naive-d 0.1 present a perfect cover rate of sults are better than the Naïve method, with eight models outper-
100%. The cover rate is the most comprehensive metric as it mea- forming the best method of Table 2 (Splines (CV)) for the weekly
sures the percentage of times that the next actual observation falls data. PC–NNall is the overall winner with MASE of 0.3604 and
within the forecasting prediction interval and thus gives a perfect SMAPE of 0.4703. Only the PC–NN/CPC–NN methods that picked
score (100%). only one nearest neighbor are performing worse than Naïve.
This concludes our investigation for R1, as we have identified
3.3. Forecasting with PC-NN/CPC-NN many models and combinations that perform better than the stan-
dard forecasting benchmarks at multiple frequencies.
We first produce forecasts with the five models for PC–NN for
the weekly data following the steps prescribed in 3.1.1. Then we 4. Forecasting the excess demand for products and services
proceed at implementing the five models for CPC–NN by using
the K-means algorithm for clustering the multivariate data we col- In this section, we advance our work towards addressing the
lected. The data consists of the variables listed in Table 5. second research question (R2), which aims at exploring how we
We performed the clustering at each step of the rolling fore- can forecast the excess demand for products and services during
casting evaluation because we expect clusters to change with the the pandemic. In normal conditions, the demand for some of these
evolution of the pandemic in different countries. Fig. 2 presents products and services is relatively non-volatile and, as a result,
the results for four instances of the clustering of countries. The does not exhibit complex patterns. It is, thus, not very difficult to
clusters evolve over time and countries are changing clusters due forecast. This is especially so for products in more mature mar-
106 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

Table 5
List of variables used for clustering for the CPC–NN method.

Name Description

Travel restrictions When no ban (0), for days with ban (1)
Reproductive number Average R value of the country
Temperature Average temperature of the country
Humidity Average humidity of the country
Population density Population density
Population median age Median age of the population
Lung disease Deaths by lung diseases per 100,000 people
Diabetes Diabetes prevalence of the population
Coronary heart disease Deaths by heart disease per 100,000 people
Distance Wuhan Distance as vector from Wuhan
Air pollution Air pollution measured by PM 2.5
GDP spending on healthcare Percentage of GDP spent for the healthcare sector in US dollars
Global healthcare ranking Development ranking from 0 to 100
Growth of cases on daily basis Normalized growth in percent

Fig. 2. Clusters of Countries at different points of time.

Table 6 4.1. Modeling excess demand

Forecasting performance of the PC–NN & CPC–NN
models on weekly data.
We consider the excess demand for the quantity of different
Method RelMdMASE RelMdSMAPE products and services including groceries, electronics, automotive
PC–NNall 0.3604 0.4703 and fashion. We start by considering the following equation as our
PC–NN3ew 0.4436 0.4928 benchmark model.
CPC–NN3uw 0.4664 0.5683
PC–NN5 0.4821 0.4080
QDt = aCov19t−b (1)
PC–NN3uw 0.5827 0.6299
CPC–NNall 0.6262 0.6767
CPC–NN5 0.6434 0.5800 Where QDt is the quantity of the excess demand at time t. Cov19t−b
CPC–NN3ew 0.8499 0.5949 is the growth rate of incidents of COVID-19 that took place at time
Splines (CV) 0.9846 1.2572 t-b with b being the respective lag. We assume that the effect on
Naive-drift-0.1 1.0015 1.0022
the quantity demanded will take place after society becomes aware
of the evolution of the infectious disease. Parameter a captures the
effect of Cov19 on QDt .
kets such as pasta, rice, toiletries. However, during a pandemic, If a government decides to impose measures to reduce the
we expect the purchasing behaviors will become significantly more spread of the virus, it could force a lockdown. The lockdown could
volatile because of consumer biases on the potential for scarcity generate further anxiety and as a result further change in con-
(Chandon & Wansink, 2006). In such cases, customers become less sumer behavior. To capture this effect, we introduce a dummy,
able to evaluate both their own inventory of supplies and the risk which takes the value of one (1) after the date that the govern-
of scarcity of the products they are planning to panic. This leads ment imposed lockdown and zero (0) before.
to “panic buying” (Tsao, Raj & Yu, 2019), which was particularly
prevalent in the COVID-19 pandemic (Gray, 2020). QDt = aCov19t−b + nDt (2)
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 107

Table 7 4.4. Simulations

Daily Google Trends data extracted.

Sector\Product Product 1 Product 2 Product 3 Considering a 21 day Lockdown (3 weeks), starting from week
Grocery Bread Meat Vegetables 1 (thus Lag =3), we estimate the excess demand from the lagged
Electronics TV sets Smartphone Notebook values of COVID-19 growth rates in a country. We simulate four
Fashion Shoes Dresses Handbags product categories for all five countries: Groceries (P1), Electronics
Automotive New Car Used car Car rental (P2), Fashion (P3), and Automotive (P4).
Fig. 3 depicts the projected excess demand of the different sec-
tors due to the pandemic from the onset of the lockdown.
We further investigate the impact of moving the lockdown over
Table 8
the weeks to create alternative scenarios (Fig. 4). We consider four
Estimation of parameter a. ∗ ∗ ∗ , ∗ ∗ , and ∗ indicate statistical significance at the 1%,
5% and 10% levels respectively. Robust standard errors presented in the parenthesis. scenarios: (a) no lockdown, (b) lockdown from week 1, (c) lock-
down from week 2, and (d) lockdown from week 3. We focus
Country\Industry Grocery Electronics Fashion Automotive
on the more critical products, that of Product category 1 – Gro-
∗∗∗ ∗∗ ∗∗∗
Germany 14.3 (3.1) 12.9 (2.1) 4.2 (1.9) 0.5∗ ∗ ∗ (0.2) ceries, as these are essential during the pandemic. We provide the
India 16.4∗ ∗ ∗ (2.4) 8.9∗ ∗ ∗ (3.2) −3.9∗ ∗ ∗ (1.0) −5.8∗ ∗ ∗ (2.3) simulations for groceries (P1) for the remaining four countries in
UK 14.9∗ ∗ ∗ (2.0) 9.3∗ ∗ ∗ (3.4) 1.0∗ ∗ ∗ (0.2) −6.3∗ ∗ ∗ (2.4)
USA 15.9∗ ∗ ∗ (5.3) 8.0∗ ∗ ∗ (2.1) −0.2∗ ∗ ∗ (0.0) −2.2∗ (1.2)
Appendix C.
Singapore 13.7∗ ∗ ∗ (4.3) 4.1∗ ∗ ∗ (2.0) −0.8∗ ∗ ∗ (0.2) −0.5∗ ∗ ∗ (0.1) Our results show that the onset and the amount of the excess
demand are dependent upon the type of product and the timing of
Model: Country_search_trend ∼ Country_growth_normalized − 1.
the lockdown. Demand for groceries (P1) and electronics (P2) be-
comes excessive, whereas that for fashion (P3) and automotive re-
lated items (P4) reduces (Fig. 3). These trends have been confirmed
4.2. Forecasting excess demand by articles in the daily press. Furthermore, Fig. 4, shows that for
groceries, the earlier the lockdown is imposed, the higher the ex-
For the estimation of the demand quantities, we use as a proxy cess demand. Finally, the longer the lockdown lasts the higher the
the searches for products from the Google trends (Jun, Yoo & Choi, cumulative excess demand. We find similar results for India, the
2018) of four different sectors (Groceries, Electronics, Fashion, Au- UK, the USA and Singapore (Appendix C).
tomotive) for the five countries we research. We decided to use Our results therefore point to various directions for both the
auxiliary data as confirmed supply chain demand data will not be process of forecasting and the management of the supply chain.
available for the months to come and as such no demand model- First, we demonstrate that the process of forecasting during the
ing would be possible until then. This is not an option for policy- pandemic needs to be dynamic and to take into account the
makers however and to that end we believe we provide here an changes in the external circumstances. Research that focuses on
essential set of tools to inform decision making. responses to humanitarian crises data (van der Laan, van Dalen,
For the values of variable COVID-19 in Eq. (2) we use the av- Rohrmoser & Simpson, 2016) has also argued for a flexible ap-
erage of the top-3 forecasts prepared in Section 3. We then use proach to forecasting. As more information becomes available and
ordinary least squares to estimate the coefficients in Eq. (2). We decisions about the response to the pandemic are being taken, the
model the excess demand over and above normal stable demand. approach to forecasting needs to be readjusted. Therefore, our re-
We make the implicit assumption that the products we are look- sults extend those for the management of more localized human-
ing at follow a relatively stable average demand in the long-run. itarian crisis by illustrating the implications for forecasting at the
Since we are focusing on the impact of the COVID-19 on the sup- time of a global pandemic.
ply chains of these products we assume that the pandemic leads Furthermore, our results illustrate the challenge of making
to an intermittent demand pattern over and above the mainstream forecasts and making supply chain decisions for products where
(Nikolopoulos, 2020). consumers need to make judgments about their own immediate
needs. In the case of groceries, previous research indicates that
when consumers make estimates about their own inventory lev-
4.3. Using Google trends data to estimate the coefficients a and n for els (e.g. the amount of toilet paper they have at home), they
different sectors do so with unrealistic assumptions and limited data (Chandon &
Wansink, 2006). As a result, they are very likely biased and influ-
To estimate a, we need the demand of the relevant products enced by the external environment. Our results forecast that sim-
and the growth of the confirmed COVID-19 cases. Since demand ilar effects are at play with other product categories such as elec-
patterns and data are not available yet, we extracted the Google tronics, where consumers have to make evaluations about the ca-
search trends for certain goods to get an estimation of how the pability of their own equipment and the potential for scarcity, e.g.
demand changed on a daily basis during the COVID-19 pandemic the combined effect of fear of failure of one’s own laptop and the
as shown in Table 7. potential for stockouts.
We used several consumer products per sector, which allowed Therefore, we can make two recommendations because of our
us to get a more holistic trend. We chose sectors that have dif- results. The first for policy makers and relates to efforts to secure
ferent underlying supply chains. We extracted Google trends data high volumes of inventory for products in those categories (P1
for a 90-day window, starting from the beginning of February and and P2) before the lockdown. Our analysis shows that this should
ending on the 30th of April 2020. We estimated parameter a by not be based only on data of actual needs, but should take into
running regressions between the daily growth and the daily search account consumers’, often biased and at times irrational, behavior.
trend, resulting in Table 8. The second recommendation is for supply chain managers of com-
To consider the impact of imposing lockdowns, we use the panies in the product categories we analyzed above. In addition to
same data from Google trends and add the variable lockdown as the preparations for fluctuations in demand, particularly in view of
a binary classifier. It takes the values of 0, when there is no travel a lockdown, our results indicate that the approach to forecasting
ban or curfew, and 1 when a curfew or travel ban are in place. needs to continuously adjust to take into account the changing
Re-running the regressions leads to Table 9: needs. This would imply changes to the forecasting models as well.
108 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

Fig. 3. The impact of Lockdown in Germany: excess demand for products due to the pandemic.

Fig. 4. The impact of alternative Lockdown decisions in excess demand for Groceries in Germany.
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 109

Table 9
∗∗∗ ∗∗ ∗
Estimation of parameter a and n. , , and indicate statistical significance at the 1%, 5% and 10% levels, respectively. Robust standard errors presented
in the parenthesis.

Country\ Grocery Electronics Fashion Automotive

Industry
α n α n α n α n
∗∗∗ ∗∗∗ ∗∗∗ ∗∗ ∗∗ ∗∗ ∗∗
Germany 26.1 (10.4) 88.5 (14.5) 36.7 (18.1) 108.7 (52.3) −30.3 (15.3) 75.7 (38.3) −28.3 (15.0) 63.1∗ ∗ (34.2)
India 8.9∗ ∗ ∗ (4.7) 77.0∗ ∗ ∗ (20.4) 12.4∗ ∗ ∗ (6.3) 65.0∗ ∗ (32.1) −21.5∗ ∗ (12.2) 53.4(25.0) −21.5∗ (12.2) 47.7∗ ∗ (26.0)
UK 15.8∗ ∗ ∗ (3.9) 83.9∗ ∗ ∗ (18.3) 16.4∗ ∗ ∗ (7.7) 70.3∗ ∗ ∗ (35.1) −21.3∗ ∗ (13.6) 61.0∗ ∗ (36.5) −28.6∗ ∗ (15.0) 60.8∗ ∗ (31.2)
USA 19.4∗ ∗ ∗ (4.6) 88.5∗ ∗ ∗ (43.2) 27.7∗ ∗ ∗ (12.1) 92.4∗ ∗ ∗ (45.7) −26.6∗ ∗ (14.0) 68.3∗ ∗ (38.2) −30.8∗ ∗ (16.3) 73.9∗ ∗ ∗ (35.5)
Singapore 6.2∗ ∗ ∗ (2.3) 65.2∗ ∗ ∗ (25.7) 12.4∗ ∗ ∗ (6.9) 53.8∗ ∗ ∗ (22.3) −14.8∗ (8.1) 45.8∗ ∗ (23.2) −12.8∗ ∗ (6.6) 40.4∗ ∗ ∗ (19.8)

Model: Country_search_trend ∼ Country_growth_normalized + Lockdown − 1.

This concludes our investigation for R2, as we have identified and timing of prevention strategies. However, the use of infec-
ways to forecast the excess demand for products and link that to tious disease forecasts for decision-making is challenging because
governmental decisions. most existing infectious diseases require different methods for dif-
ferent countries. Each forecasting model has limitations. Further-
5. Conclusions, implications for practice and policymaking, and more, data may not be reliable because it may have been recorded
the future during the emergency situations. As a result, comparing forecasts
at the country level remains challenging, potentially limiting the
This paper has examined urgently and extraordinarily the pre- development and utility of forecasts.
dictability of COVID-19 growth in five countries and modeled Despite these limitations, COVID-19 forecasts provide indica-
the dependent short-term supply chain disruptions. We evalu- tions and quantify the needs that appear in an emergency, and
ated existing state-of-the-art and proposed new data-driven meth- thus more research should be directed towards identifying the best
ods for forecasting pandemic evolution while working with lim- forecasting models for all geographical contexts and temporal fre-
ited, volatile, and constantly revised data. Countries have differ- quencies.
ent healthcare systems, run the COVID-19 tests in different places
(hospitals, GPs, community centers, airports), apply different poli-
cies (track and trace, lockdowns, legislation, etc.), test with differ- Appendix A. Data sources
ent devices and protocols, and report differently new cases and
deaths (including or excluding deaths at home or in care homes). The following publicly available data sources has been used.
All these complicate and limit the extent of accuracy that can be
achieved from forecasting models. There is, therefore, an immedi- • Confirmed, recovered and deceased cases were obtained
ate need for a homogenous credible database to enable more accu- from Johns Hopkins university, this data set is derived from
rate and comparable forecasting by the academic community, pol- multiple sources, including WHO and national governmen-
icy makers and supply chain professionals. Nevertheless, forecast- tal organization and is updated on a daily basis: (https:
ing remains an essential part of many decision-making processes, //data.humdata.org/dataset/novel-coronavirus-2019-ncov-cases)
and as such, this motivates us further for this research endeavor. and https://coronavirus.jhu.edu/ (Johns Hopkins Coronavirus
We also modeled the excess demand for products and services dur- Resource Center). Initially the process was done manu-
ing the pandemic via using auxiliary data (Google trends) as actual ally at a daily basis and at a later stage automated via
supply & demand data are not yet publicly available: our mod- https://github.com/CSSEGISandData/COVID
els rightly predicted the panic buying effect and respective excess • Climate information a s well as the reproduction number is de-
demand for groceries and electronics during the current wave of rived from the Covid 19 reseaerch team from Beihan univer-
COVID-19. sity in China: (http://covid19-report.com/#/forecasting, https:
Many operational decisions are affected by our research includ- //github.com/bigscity/nCov-predict)
ing those associated with planning, production, shipping, stock- • The information about the specific date of travel restrictions
control (Prak, Teunter & Syntetos, 2017), ordering, and allocating and curfews by country were obtained from Mayer Brown:
of resources (Nikolopoulos, Metaxiotis, Lekatis & Assimakopoulos, Mayer Brown’s COVID-19 Global Travel Restrictions by Country
2003). They are all decisions where an accurate forecast is an es- (https://www.mayerbrown.com/-/media/files/perspectives-eve-
sential input and as such, our study is relevant. Furthermore, the nts/publications/2020/02/mayer-brown_covid19-
results of our research can inform government decisions. We show global- travel- restrictions- by- country2.pdf)
that the earlier a lockdown is imposed, the higher the excess de- • Information about populations, including median age, popula-
mand will be for groceries. Furthermore, the longer the lockdown tion density were obtained from the world population review
lasts the higher the cumulative excess demand and thus the higher (https://worldpopulationreview.com/countries/median-age/)
the need for planning for production and inventory. Consequently, • Rate of lung diseases per 10 0.0 0 0 inhabitants by coun-
a policy recommendation for the governments will be to secure try was obtained from: (https://www.worldlifeexpectancy.com/
high volumes of inventory for such products before the lock- cause- of- death/lung- disease/by-country/)
down; and if not possible, consider radical interventions such as • Rate of coronary heart diseases per 10 0.0 0 0 inhabitants by
rationing. country was obtained from: (https://www.worldlifeexpectancy.
During a health emergency response, leaders need to make a com/cause- of- death/coronary- heart- disease/by-country/)
numerous critical decisions for the supply chain, and for preven- • Diabetes prevalence by country derived from the world
tion strategies (Glasser, Hupert, McCauley & Hatchett, 2011). The bank:(https://data.worldbank.org/indicator/SH.STA.DIAB.ZS)
decisions occur in a rapidly changing environment and they might • Percentage of GDP spent on healthcare by country was obtained
be misinformed or biased. Consequently, forecasting becomes an from the world bank: (https://data.worldbank.org/indicator/SH.
essential tool for helping and providing guidance for the utility XPD.CHEX.GD.ZS)
110 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

• The PM 2.5 concentration as metric for air pollution by country LSTM

was derived from the world bank:(https://data.worldbank.org/
indicator/EN.ATM.PM25.MC.M3) The LSTM networks are state-of-the-art sequencing modeling
• Import data from OECD: https://stats.oecd.org/Index.aspx? methods which comes under deep learning. The sequence mod-
DataSetCode=BTDIXE# eling feature of LSTM can be used for time-series forecasting spe-
• Google trends: https://trends.google.com cially to model non-linear time series variations. The LSTM were
implemented using Keras library in R (Chollet, 2015). The work of
Appendix B. Description of machine- and deep-learning Punia, Nikolopoulos et al. (2020) was followed for implementation
forecasting methods and hyperparameter optimization of the LSTM networks.

Decision tree Ridge regression

The decision tree is supervised machine learning algorithm Ridge regression is an advanced regression technique that al-
used for the classification and regression application. We used the lows to perform L2 regularization i.e. adding penalty equals to
continuous variable, regression decision tree with classification and square of coefficients along with minimizing the sum of squared
regression tree (CART) algorithm. The Caret package from R is used error between actual and forecast. The linear ridge regression was
for the implementation of the method (Kuhn, 2008). The parame- implemented using ridge library in the R.
ter optimization was performed using grid search.
Support Vector Machines (SVM)
Random forest
SVM are the machine learning techniques that is based on clas-
Random forest was developed by Breiman, (2001); Ho, sification and regression algorithms and can be used for the fore-
(1995) and it generates multiple random samples and perform the casting purposes using regression method. SVM were implemented
bagging of decision tree applied on random sample of data, thus using e1071 package in R. The “linear” kernel were used along with
called random forest. The algorithm is implemented using Caret “eps-regression” type from the parameters for the implementation
package in R (Kuhn, 2008) and grid search was used to search best of the method.
combination of parameters. The literature is referred for optimal
implementation of the random forest fore forecasting (Fischer & Splines and sigmoid
Krauss, 2018; Punia, Singh & Madaan, 2020).
The splines are used to fir a smoothing function to the data
Artificial Neural Network (ANN) just like the regression. Different smoothing splines can be fitted
to the data using different non-linear functions and best one can
ANN have three layers for data modeling, namely, an in- be selected for the purpose of forecasting. We have used the sig-
put layer, an output layer, and hidden layers. The inputs moid, logistics functions to fit the data. The functions smooth.spline
and outputs (yt ) are modeled through yt = αo +
(X , X , . . . , X p )
and nls (non-linear least square estimates) were used from the base

q1 2
j=1
α j g ( βo j +
p
β X ) + εt , where α s and β s are connection
i=1 i j t−i
package of the R.
weights, p is the number of input nodes and q is the number of
hidden nodes. The output from the ANN is a non-linear function Appendix C. Groceries demand (simulations) for India, UK,
that maps the inputs to outputs with the help connection weights. USA, and Singapore [Simulations for P2, P3 and P4 are
ANN were applied for the forecasting using death rate and recov- available upon request]
ery rate as the input and cases growth as the output variable in R.
K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 111

Fig. 5. The impact of alternative Lockdown decisions in excess demand for Groceries in India.
112 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

Fig. 6. The impact of alternative Lockdown decisions in excess demand for Groceries in UK.
K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 113

Fig. 7. The impact of alternative Lockdown decisions in excess demand for Groceries in US.
114 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115

Fig. 8. The impact of alternative Lockdown decisions in excess demand for Groceries in Singapore.

References Chandon, P., & Wansink, B. (2006). How biased household inventory estimates dis-
tort shopping and storage decisions. Journal of Marketing, 70(4), 118–135.
Al-Shammari, A. A. A., Ali, H., Al-Ahmad, B., Al-Refaei, F. H., Al-Sabah, S., & Ja- Chen, F., Drezner, Z., Ryan, J. K., & Simchi-Levi, D. (20 0 0). Quantifying the bullwhip
mal, M. H. (2020). Real-time tracking and forecasting of the COVID-19 out- effect in a simple supply chain: The impact of forecasting, lead times, and in-
break in Kuwait: A mathematical modeling study. MedRxiv 05.03.20089771. formation. Management Science, 46(3), 436–443.
https://doi.org/10.1101/2020.05.03.20089771. Chollet, F. (2015). Keras. GitHub. https://github.com/fchollet/keras
Andersson, E., Schiöler, L., Frisén, M., Kühlmann-Berenzon, S., Linde, A., & Rubi- Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory net-
nova, S. (2008). Predictions by early indicators of the time and height of the works for financial market predictions. European Journal of Operational Research,
peaks of yearly influenza outbreaks in Sweden. Scandinavian Journal of Public 270(2), 654–669. https://doi.org/10.1016/j.ejor.2017.11.054.
Health, 36(5), 475–482. Gardner, E. S., Jr., & McKenzie, E. D. (1985). Forecasting trends in time series. Man-
Araz, O. M., Choi, T.-. M., Olson, D., & Salman, F. S. (2020). Data an- agement Science, 31(10), 1237–1246.
alytics for operational risk management. Decision Sciences In press Glasser, J. W., Hupert, N., McCauley, M. M., & Hatchett, R. (2011). Modeling and pub-
https://onlinelibrary.wiley.com/doi/abs/10.1111/deci.12443<https://nam03. lic health emergency responses: Lessons from SARS. Epidemics, 3(1), 32–37.
safelinks.protection.outlook.com/?url=https%3A%2F%2Fonlinelibrary. Govindan, K., Mina, H., & Alavi, B. (2020). A decision support system for demand
wiley.com%2Fdoi%2Fabs%2F10.1111%2Fdeci.12443&data=02%7C01% management in healthcare supply chains considering the epidemic outbreaks:
7Ccorrectionsaptara%40elsevier.com%7Ca36bb37f70594954e86208d840329cf7% A case study of coronavirus disease 2019 (COVID-19). Transportation Research
7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C637329933415158652& Part E: Logistics and Transportation Review, Article 101967.
sdata=nDupIt8CVdxPkP%2FTk3fPPKu4UPQkzMlBnKTZSzLNMzs%3D&reserved=0>. Gray, A. (2020). Toilet roll mania boosts sales of Andrex maker Kimberly-
Baveja, A., Kapoor, A., & Melamed, B. (2020). Stopping COVID-19: A pandemic- Clark Financial Times. Accessed 14/07/2020 https://www.ft.com/content/
management service value chain approach (SSRN Scholarly Paper No. ID 8075d66f-06ba-4aea-aac9-cac59b79f1e1
3555280). doi: 10.2139/ssrn.3555280 Härdle, W. (1990). Applied nonparametric regression. Cambridge: Cambridge Univer-
Beliën, J., & Forcé, H. (2012). Supply chain management of blood products: A litera- sity Press.
ture review. European Journal of Operational Research, 217(1), 1–16. Harris, R. (2020, May). Covid-19 and productivity in the UK. Durham University Busi-
Bode, C., Wagner, S. M., Petersen, K. J., & Ellram, L. M. (2011). Understanding ness School https://www.dur.ac.uk/research/news/item/?itemno=41707.
responses to supply chain disruptions: Insights from information processing Hendricks, K. B., & Singhal, V. R. (2003). The effect of supply chain glitches on share-
and resource dependence perspectives. Academy of Management Journal, 54(4), holder wealth. Journal of Operations Management, 21(5), 501–522.
833–856. Ho, T. K. (1995). Random decision forests. Document Analysis and recognition,
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. 1995.. In Proceedings of the Third international conference on document analysis:
Castro, M.F., .Duarte, J.B., .& Brinca, P. (2020). Measuring sectoral supply and demand 1 (pp. 278–282). IEEE.
shocks during COVID-19. doi: 10.20955/wp.2020.011 https://research.stlouisfed. Hobbs, J. E. (2020). Food supply chains during the COVID-19 pandemic. Canadian
org/wp/more/2020-011 Journal of Agricultural Economics/Revue Canadienned’agroeconomie.
 K. Nikolopoulos, S. Punia and A. Schäfers et al. / European Journal of Operational Research 290 (2021) 99–115 115

Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accu- Punia, S., Nikolopoulos, K., Singh, S., Madaan, J., & Litsiou, K. (2020). Deep learn-
racy. International Journal of Forecasting, 22(4), 679–688. ing with long short-term memory networks and random forests for demand
Hyndman, R., Koehler, A. B., Snyder, R. D., & Grose, S. (2002). A state space frame- forecasting in multi-channel retail. International Journal of Production Research.
work for automatic forecasting using exponential smoothing methods. Interna- https://doi.org/10.1080/00207543.2020.1735666.
tional Journal of Forecasting, 18(3), 439–454. Punia, S., Singh, S. P., & Madaan, J. K. (2020). From predictive to prescriptive an-
Ivanov, D. (2020). Predicting the impacts of epidemic outbreaks on global sup- alytics: A data-driven multi-item newsvendor model. Decision Support Systems,
ply chains: A simulation-based analysis on the coronavirus outbreak (COVID- Article 113340. https://doi.org/10.1016/j.dss.2020.113340.
19/SARS-CoV-2) case. Transportation Research Part E: Logistics and Transportation Shaman, J., & Karspeck, A. (2012). Forecasting seasonal outbreaks of influenza. Pro-
Review, 136. https://doi.org/10.1016/j.tre.2020.101922. ceedings of the National Academy of Sciences of the United States of America,
Jun, S.-. P., Yoo, H. S., & Choi, S. (2018). Ten years of research change using Google 109(50), 20425–20430. https://doi.org/10.1073/pnas.1208772109.
Trends: From the perspective of big data utilizations and applications. Techno- Shankar, S., Ilavarasan, P. V., Punia, S., & Singh, S. P. (2019). Forecasting container
logical Forecasting and Social Change, 130, 69–87. throughput with long-short-term-memory networks. Industrial Management and
Kahn, J. (1987). Inventories and the volatility of production. American Economic Re- Data Systems, 120(3), 425–441.
view, 77(4), 667–679. Soebiyanto, R. P., Adimi, F., & Kiang, R. K. (2010). Modeling and predicting seasonal
Kim, Y., Chen, Y.-. S., & Linderman, K. (2015). Supply network disruption and re- influenza transmission in warm regions using climatological parameters. PloS
silience: A network structural perspective. Journal of Operations Management, one, 5(3). https://doi.org/10.1371/journal.pone.0 0 09450.
33(Supplement C), 43–59. Team, IHME COVID-19 health service utilization forecasting, & Murray, C. J. (2020a).
Kuhn, M. (2008). Caret package. Journal of Statistical Software, 28(5), 1–26. Forecasting COVID-19 impact on hospital bed-days, ICU-days, ventilator-days
Kyriazi, F., & Thomakos, D. D. (2020). Distance-based nearest neighbour forecast- and deaths by US state in the next 4 months. MedRxiv, 2020.03.27.20043752.
ing with application to exchange rate predictability. IMA Journal of Management doi: 10.1101/2020.03.27.20043752
Mathematics dpz016. https://doi.org/10.1093/imaman/dpz016. Team, IHME COVID-19 health service utilization forecasting, & Murray, C. J. (2020b).
Lee, H. L., Padmanabhan, V., & Whang, S. (1997). Information distortion in a supply Forecasting the impact of the first wave of the COVID-19 pandemic on hospi-
chain: the bullwhip effect. Management Science, 43, 546–558. tal demand and deaths for the USA and European Economic Area countries.
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020). The M4 Competition: MedRxiv, 2020.04.21.20074732. doi: 10.1101/2020.04.21.20074732
10 0,0 0 0 time series and 61 forecasting methods. International Journal of Fore- Tsao, Y.-. C., Raj, P. V. R. P., & Yu, V. (2019). Product substitution in different weights
casting, 36(1), 54–74. and brands considering customer segmentation and panic buying behavior. In-
Makridakis, S., & Winkler, R. L. (1983). Averages of forecasts: Some empirical results. dustrial Marketing Management, 77, 209–220.
Management Science, 29(9), 987–996. van der Laan, E., van Dalen, J., Rohrmoser, M., & Simpson, R. (2016). Demand fore-
Nikolopoulos, K. (2020). We need to talk about intermittent demand forecasting. Eu- casting and order planning for humanitarian logistics: An empirical assessment.
ropean Journal of Operational Research. https://doi.org/10.1016/j.ejor.2019.12.046. Journal of Operations Management, 45(1), 114–122.
Nikolopoulos, K., Buxton, S., Khammash, M., & Stern, P. (2016). Forecasting branded Vangumalli, D.R., .Nikolopoulos, K., & Litsiou, K. (2019). Clustering, forecasting and
and generic pharmaceuticals. International Journal of Forecasting, 32(2), 344–357. cluster forecasting: using k-medoids, k-NNs andrandom forests for cluster selec-
Nikolopoulos, K., Metaxiotis, K., Lekatis, N., & Assimakopoulos, V. (2003). Integrating tion. Bangor Business School Working Paper Series BBSWP/19/16, https://www.
industrial maintenance strategy into ERP. Industrial Management & Data Systems, bangor.ac.uk/business/research/workingpapers.php.en
103(3), 184–191. von Krogh, G., Rossi-Lamastra, C., & Haefliger, S. (2012). Phenomenon-based research
Nsoesie, E. O., Marathe, M., & Brownstein, J. S. (2013). Forecasting peaks of seasonal in management and organisation science: When is it rigorous and does it mat-
influenza epidemics. PLoS Currents, 5(Outbreaks). ter? Long Range Planning, 45(44), 277–298.
Petropoulos, F., & Makridakis, S. (2020). Forecasting the novel coronavirus COVID-19. Wang, X., & Disney, S. M. (2016). The bullwhip effect: Progress, trends and direc-
PloS one, 15(3), Article e0231236. https://doi.org/10.1371/journal.pone.0231236. tions. European Journal of Operational Research, 250(3), 691–701.
Petropoulos, F., Makridakis, S., Assimakopoulos, V., & Nikolopoulos, K. (2014). Horses World Health Organization (WHO). (2020). Coronavirus disease (COVID-19)—
for Courses’ in demand forecasting. European Journal of Operational Research, Situation Report – 119.
237(1), 152–163. Yaffee, R. A., Nikolopoulos, K., Reilly, D. P., Crone, S. F., Wagoner, K. D., & Dou-
Pettit, T. J., Croxton, K. L., & Fiksel, J. (2019). The evolution of resilience in supply glass, R. J. (2008). An experiment in epidemiological forecasting: A comparison
chain management: A retrospective on ensuring supply chain resilience. Journal of forecast accuracies of different methods of forecasting deer mouse population
of Business Logistics, 40(1), 56–65. https://doi.org/10.1111/jbl.12202. density in Montana. Center for Economic Research, 1(2011), 16–19 December.
Pettit, T. J., Fiksel, J., & Croxton, K. L. (2010). Ensuring supply chain resilience: De- Yang, Z., Zeng, Z., Wang, K., Wong, S.-. S., Liang, W., Zanin, M., et al. (2020). Modi-
velopment of a conceptual framework. Journal of Business Logistics, 31(1), 1–21. fied SEIR and AI prediction of the epidemics trend of COVID-19 in China under
Prak, D., Teunter, R., & Syntetos, A. (2017). On the calculation of safety stocks when public health interventions. Journal of Thoracic Disease, 12(3), 165–174.
demand is forecasted. European Journal of Operational Research, 256(2), 454–461.