This Is Not a Drill

American Psychologist

This Is Not a Drill: Anxiety on Twitter Following the 2018

Hawaii False Missile Alert

Nickolas M. Jones and Roxane Cohen Silver

Online First Publication, July 25, 2019. http://dx.doi.org/10.1037/amp0000495

CITATION

Jones, N. M., & Silver, R. C. (2019, July 25). This Is Not a Drill: Anxiety on Twitter Following the 2018

Hawaii False Missile Alert. American Psychologist. Advance online publication.

http://dx.doi.org/10.1037/amp0000495

This Is Not a Drill: Anxiety on Twitter Following the 2018 Hawaii False

Missile Alert

Nickolas M. Jones and Roxane Cohen Silver

University of California, Irvine

The accuracy of emergency management alerts about dangerous threats to public safety is key

for the protection of life and property. When alerts of imminent threats are believed to be real,

uncontrollable, and impossible to escape, people who receive them often experience fear and

anxiety, especially as they await the threat’s arrival (i.e., incubation of threat). However, what

are the consequences when an alert turns out to be a false alarm? We explored psychological

reactions (i.e., anxiety) to the 2018 Hawaii false ballistic missile alert using Twitter data from

users across the state (1.2 million tweets, 14,830 users) 6 weeks before and 18 days after the

event. We demonstrated that anxiety expressed on Twitter increased 4.6% on the day of the

false alert and anxiety during the 38-min alert period increased 3.4% every 15 min. In

addition, users who expressed either low, medium, or high prealert anxiety exhibited

differential anxiety responses postalert, differential stabilization intervals (when anxiety

stopped decreasing after the all-clear), and different postalert baselines relative to their

prealert levels. Low prealert anxiety users expressed more anxiety at the onset of the alert and

for longer relative to other groups. Moreover, anxiety remained elevated for at least 7 days

postalert. Taken together, findings suggest that false alarms of inescapable and dangerous

threats are anxiety-provoking and that this anxiety can persist for many people after the threat

is dispelled. We offer several recommendations for how emergency management agencies

should best communicate with the public after false alerts are transmitted.

Keywords: false alarm, Twitter, anxiety, collective trauma, social media

When danger in one’s community is imminent, people

often rely on local and state emergency management orga-

nizations for information to assess the severity of the threat

and respond with appropriate measures to secure personal

safety and protect property. Indeed, the success of emer-

gency management personnel to effectively communicate

risk information to at-risk populations plays an important

role in saving lives (Rodríguez, Díaz, Santos, & Aguirre,

2007). However, when officials charged with disseminating

information about impending threats falter, this might lead

to a disaster (even if the threat becomes innocuous; Gilbert,

1998), and there may be a number of unintended conse-

quences. For example, a lack of regular updates during an

emergency can elicit distress and other negative psycholog-

ical outcomes among individuals under threat (Jones,

Thompson, Dunkel Schetter, & Silver, 2017).

False alarms are one example of how emergency man-

agement agencies might stumble. They occur when a trans-

mitted warning of an impending threat is no longer relevant,

such as when a hurricane changes course and no longer

threatens a geographic area. False alarms also occur when

an emergency organization broadly transmits a warning

about an impending threat that does not actually exist, such

as false active shooter reports. The implications of these

types of false alarms have been studied by researchers

across several disciplines. This body of work demonstrates

how false alarms are related to loss of organizational cred-

ibility (Dow & Cutter, 1998; Ripberger et al., 2015), behav-

ioral outcomes like diminished protective behavior among

individuals under threat (Ripberger et al., 2015), and in-

creased loss of life in tornado-prone areas with a high false

alarm ratio (Simmons & Sutter, 2009). However, little work

has directly examined the psychological impact of exposure

to potentially life-threatening events that turn out to be false.

Can a false alarm of an impending disaster itself be a form

of collective trauma (i.e., a large-scale natural or anthropo-

X Nickolas M. Jones, Department of Psychological Science, University

of California, Irvine; X Roxane Cohen Silver, Department of Psycholog-

ical Science, University of California, Irvine.

Nickolas M. Jones is now at the Department of Psychology, Princeton

University.

Correspondence concerning this article should be addressed to Roxane

Cohen Silver, Department of Psychological Science, University of Cali-

fornia, Irvine, 4201 Social & Behavioral Sciences Gateway, Irvine, CA

92697-7085. E-mail: [email protected]

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

American Psychologist

0003-066X/19/$12.00 http://dx.doi.org/10.1037/amp0000495

genic disaster that affects many people)? If so, how do

people respond?

Research in psychology offers some clues about how

individuals might respond to false alarms of an approaching

catastrophe. Early experimental work on false alarms found

that when the threat of a strong electric shock was perceived

to be real, imminent, uncontrollable, and impossible to

escape, individuals experienced a heightened physiological

fear response (i.e., heightened heart rate) and subjective

tension (Breznitz, 1984, 1985). During the moments in

which participants anticipated the threat’s arrival, subjective

apprehension and worry increased over time (i.e., the incu-

bation of threat; Breznitz, 1968, 1971, 1984). This work

also demonstrated that when warnings of threat were can-

celled, it took time for individuals to recover from their

heightened state (Breznitz, 1984), suggesting that a cancel-

lation of a threat warning does not immediately remedy the

psychological consequences of having been warned.

Attention to fear and worry in this research is reminiscent

of studies that highlight anxiety as a key psychological

reaction to collective traumas such as terrorist attacks and

natural disasters (Norris et al., 2002). For some individuals,

lingering anxiety can cross clinical thresholds, developing

into posttraumatic stress disorder (PTSD; Norris et al.,

2002). Collective traumas often occur without warning and

the threat they pose can sometimes be ambiguous. The

inherent uncertainty in some disaster contexts (Gilbert,

1998) can elicit psychological distress depending on how it

is appraised by those who experience it (Folkman, Lazarus,

Dunkel-Schetter, DeLongis, & Gruen, 1986). For example,

limited experience with a threat may increase situational

ambiguity, thereby exacerbating anxiety (Lazarus, 1966).

Thus, during dangerous or life-threatening situations in

which information is lacking and ambiguity is high, uncer-

tainty about the impending outcome may lead to anxiety

(Taha, Matheson, & Anisman, 2014) and other negative

psychological outcomes (Jones et al., 2017), especially

among individuals for whom ambiguity is intolerable (Br-

eznitz, 1984; Chen & Hong, 2010; Rosen, Knäuper, &

Sammut, 2007).

When communities experience collective traumas, there

may be marked variation in how they respond, and this

variation may depend on characteristics of the individuals

who reside within them. For example, researchers found

that after 9/11, older adults had a steeper decline in their

posttraumatic stress symptoms relative to younger individ-

uals (Holman, Silver, Mogle, & Scott, 2016). Other re-

searchers have focused on participants’ past negative expe-

riences to understand postevent responses. For example, in

a national sample of Americans surveyed across subsequent

waves following 9/11, Seery, Holman, and Silver (2010)

found that the experience of zero or many negative life

events was associated with poorer psychological outcomes

over time relative to individuals with some negative life

events (i.e., the association was quadratic). Further evidence

suggests that negative psychological states, measured be-

fore a collective trauma, may be relevant as well. Among

both youth and adults, experiencing negative psychological

states (e.g., anxiety) before a collective trauma was associ-

ated with an increased risk of developing PTSD in its

aftermath (Asarnow et al., 1999; Nolen-Hoeksema & Mor-

row, 1991). Overall, these studies suggest that, when pos-

sible, community-based studies of the impact of collective

traumas should disaggregate individuals by their preevent

vulnerabilities (e.g., characteristics or psychological states)

to assess whether postevent outcomes differ in a meaningful

way.

Variation in psychological responses to collective trauma

has been studied mostly using traditional research method-

ologies (e.g., surveys and interviews). However, rigorously

studying the psychological impact of a collective trauma is

often difficult because of a lack of preevent data and chal-

lenges entering the field in a timely manner (e.g., securing

funding and ethics board approval), among other hurdles

(Jones, Wojcik, Sweeting, & Silver, 2016; Silver, 2004).

Some social scientists have circumvented these challenges

by using social media data to explicate psychological re-

sponses to collective traumas. Researchers have shown that

big data from social media platforms, usually Twitter (e.g.,

“tweets”), are particularly useful for evaluating community

responses to school shootings (Doré, Ort, Braverman, &

Ochsner, 2015; Jones et al., 2016, 2017), terrorist attacks

(Gruebner et al., 2016; Lin, Margolin, & Wen, 2017), nat-

ural disasters (Gruebner et al., 2017; Murthy & Longwell,

2013), and other collective adversities (De Choudhury,

Monroy-Hernandez, & Mark, 2014).

Nickolas M.

Jones

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

JONES AND SILVER

Twitter data offer unprecedented opportunities for theo-

retical insight because analyses of tweets, which contain the

thoughts and feelings of Twitter users, possess several

strengths compared with traditionally collected data. First,

access to Twitter data is free to anyone with some technical

know-how. Second, these data are an ecologically valid

observational data source and can circumvent some biases

inherent in traditional data collection methods, like surveys

(e.g., low participation rates) or interviews (e.g., demand

characteristics). Third, because Twitter data are archival by

design, this enables an examination of emotion expression

before and after collective traumas across an extended time

frame that can be aggregated with granularity not possible

with traditionally collected data. Fourth, Twitter data can

also be explicitly linked to a user’s location via geo-

coordinates if a user opts in to making this information

public, or a user’s location can be inferred based on the

accounts they follow (see Jones et al., 2016).

The methodological strengths of working with Twitter

data may also be useful for studying how individuals re-

spond to false alarms of potentially life-threatening events.

To date, the theoretical insights from experimental studies

of false alarms have not been validated against large-scale,

observational data. Thus, Twitter data were harnessed to

explore anxiety responses to the 2018 Hawaii false ballistic

missile alert. At 8:07 a.m. on January 13, 2018, Hawaii

residents and visitors received an emergency alert from the

Hawaii Emergency Management Agency over the radio,

television, and on their smartphones stating that a ballistic

missile was inbound to Hawaii, that people should seek

shelter, and that this alert was “NOT A DRILL” (caps in

original). Media reports cited increased anxiety among res-

idents during the ordeal (Silva, 2018), and some residents

reached out to their loved ones to say goodbye (Pactol,

2018). Presumably, many individuals thought that death or

destruction was inevitable. However, a second message was

transmitted 38 min later stating that there was “no missile

threat or danger” and indicating that the original message

had been a “false alarm.”

This study addresses three main hypotheses. Consistent

with other Twitter studies of collective trauma, we hypoth-

esized that anxiety would increase at the time of the alert

and remain elevated in the immediate aftermath. Second,

consistent with the incubation of threat phenomenon, we

hypothesized that anxiety would increase incrementally dur-

ing the time from the transmission of the missile alert to the

“all clear” 38 min later. Finally, because prior research

demonstrates the importance of preevent vulnerabilities, we

hypothesized that individuals exhibiting either low or high

prealert anxiety might be at greater risk for experiencing

anxiety after the alert. Specifically, we hypothesized that

individuals who exhibited low and high anxiety before the

alert would exhibit the greatest increase in anxiety after the

alert, stabilize later, and exhibit higher postalert anxiety

relative to their prealert baseline.

Method

Twitter Data Collection and Measures

Using procedures developed in prior research (Jones et

al., 2016), Twitter data generated by users likely to be

Hawaii residents were obtained. First, Twitter accounts

(n ⫽ 46) operated by local government and commercial

organizations (e.g., city hall, local radio stations) that were

likely to be followed by Hawaii residents were identified in

the days following the alert. Next, the rtweet package (Kear-

ney, 2017) for R Software (R Core Team, 2018) was used to

interface with Twitter’s Application Programming Interface

(API) and scrape the most recent 5,000 followers of each

local account. If an account had fewer than 5,000 followers,

all followers were downloaded. After filtering out non-

English-language user accounts, user accounts created after

December 2017 (because they would not have 6-weeks of

prealert data), and both “private” and “verified” accounts

(likely belonging to businesses, celebrities, and other public

figures), a list of 32,239 user accounts was retained. Next,

18 days after the false missile alert, this list was read into an

R script that interfaced with the Twitter API and down-

loaded the most recent 800 tweets generated by each user to

obtain enough tweets dating back to 6 weeks before the

alert. It should be noted that this cost-free method for

sourcing Twitter data offers advantages over other methods

of obtaining tweets from Twitter (e.g., standard level search

API). Specifically, it circumvents restricted access to all but

a fraction of public tweets and the restriction of only being

Roxane Cohen

Silver

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

FALSE ALERT ANXIETY

able to download tweets generated in the past 7 days from

a given search.

In all, 1.2 million tweets representing 14,830 users who

tweeted within a 9-week window around the false alert

(6 weeks before and 18 days after) were downloaded. Data

collection procedures for this study were reviewed by the

University of California, Irvine’s Institutional Review

Board and assessed not to constitute human subjects re-

search.

Measures

Anxiety expression. A custom R script was used to

compare the words in each tweet with a list of 114 anxiety

words (e.g., afraid, scared, worried) available in the LIWC

program (Pennebaker, Booth, Boyd, & Frances, 2015;

Tausczik & Pennebaker, 2010). The words threat and alarm

were removed from the dictionary because these words

were specifically used by government and emergency man-

agement personnel to refer to the missile alert event. Each

tweet was then coded dichotomously such that it was as-

signeda1ifitcontained at least one anxiety word; all other

tweets were coded 0. This approach allowed for a propor-

tion of tweets with anxiety to be calculated across analytic

time frames, and it provided a measure of anxiety expres-

sion that compensated for differential counts of tweets gen-

erated at each time-unit of analysis (e.g., minutes, hours,

days).

Prealert anxiety. Users with prealert tweets (n ⫽

8,746) were grouped as low, medium, or high anxiety based

on their average proportion of prealert anxiety tweets gen-

erated 7 days before the alert. These proportions were then

standardized across all users and the z scores were used to

group users based on where they fell along this standardized

distribution. Users in the “low” group (n ⫽ 6,849) did not

express any anxiety on Twitter before the alert. Users placed

into the “medium” group (n ⫽ 1,394) expressed some

anxiety, up to just below one standard deviation; users in the

“high” anxiety group (n ⫽ 503) expressed anxiety greater

than or equal to one standard deviation relative to all other

users.

Analytic Strategy

Data were cleaned and organized in R using tidytext

(Silge & Robinson, 2016), and descriptive visualizations

were created in R using ggplot2 (Wickham, 2009) and

employed the generalized additive model smoothing func-

tion to depict a nonlinear line-of-best-fit across time. Tra-

jectories of anxiety before and after the alert were evaluated

at several time scales. All statistical analyses were con-

ducted in Stata 14.2 (College Station, TX) using procedures

outlined by others (Jones et al., 2016; Mitchell, 2012).

9-week window. The proportion of daily anxiety was

calculated across the 6 weeks preceding the alert and the

18 days that followed. Before attempting to model trajec-

tories of anxiety across time, a change-point analysis was

conducted in R using the changepoint package (Killick,

Haynes, & Eckley, 2016) to determine the discrete time

point at which anxiety decreased to a stable level after the

missile alert. This method employs the pruned exact

linear time algorithm (Killick & Eckley, 2014) to identify

when the mean and variance of a variable deviate over

time. In other words, the algorithm evaluates the mean

and variance in a block at the start of time, and then

evaluates whether they are significantly different from

the mean and variance calculated in the next block of

time. The package then displayed the time values where

the algorithm identified significant changes in daily pro-

portions of anxiety.

The change-point analysis indicated that anxiety in-

creased on the day of the missile alert and stabilized 2

days later. To evaluate these nonlinear changes over time,

a piecewise regression approach (Kim, Fay, Feuer, &

Midthune, 2000) with a discontinuity analysis (Thistleth-

waite & Campbell, 1960) was used. This approach is well

suited for modeling nonlinear changes in time-series data

because piecewise regressions allow for separate regres-

sion lines to be estimated for specified time intervals in

a unified statistical model. Specifically, time is blocked

into meaningful intervals by analytic knots, or dummy-

coded markers. A knot was placed on the day of the alert

(January 13, 2018, at ⬃ 8:07 a.m. HST) so that an esti-

mate of what anxiety expression would have been (had

the alert never happened) could be compared with actual

anxiety expression at the alert’s onset. A knot was also

placed 2 days after the event, as the change-point analysis

revealed this to be a meaningful interval at which anxiety

levels stabilized across all users. Importantly, the piece-

wise regression analysis was clustered around each user

to account for within-user propensity for anxiety expres-

sion on Twitter.

12-hr window. Time across a period spanning 6 hr

before and after the missile alert was parsed into 15-min

intervals (cf. Jones et al., 2017), and the proportion of

tweets with anxiety was calculated for each interval. A

piecewise regression analysis was conducted on all

tweets in this time frame to evaluate whether anxiety

increased the moment the alert was transmitted and the

extent to which anxiety continued to increase during

the 38-min alert period. Thus, a knot was placed at the

moment the alert was transmitted and 45 min later

(roughly 7 min after the “all-clear” was transmitted). In

addition to evaluating immediate changes in anxiety re-

sulting from the missile alert, this approach also allowed

for an analysis of the slope of anxiety expression during

the 38-min alert period.

14-day window. Hourly proportions of tweets with

anxiety generated in a 14-day window around the alert (7

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

JONES AND SILVER

days before and after) were calculated for each prealert

anxiety group (i.e., low, medium, high) to provide a more

fine-grained analysis of the immediate impact of the alert

and its sustained effect across the following week for

each prealert anxiety group. To determine the discrete

time point at which anxiety leveled off for each group, a

change-point analysis was conducted for each group in-

dividually. Postalert change points were used to place a

second knot in each piecewise regression to accurately

model changes in anxiety over time for each group. Thus,

a knot was placed at the moment the alert was transmitted

and at the group-specific change point after the alert,

when anxiety stabilized.

To evaluate whether the event had a lasting effect on each

prealert anxiety group in the days that followed, we con-

ducted three ordinary least squares regression analyses in

which each group’s prealert average proportion of anxiety

expression (baseline anxiety) was compared with its post-

alert average (new baseline), after stabilization in anxiety

occurred.

Figure 1. Daily anxiety among Hawaii Twitter users 6 weeks before and 18 days after the false missile alert

tweets

⫽ 1.2 million; n

users

⫽ 14,830). See the online article for the color version of this figure.

Figure 2. Anxiety on Twitter, 6 hr before and after the ballistic missile alert (n

tweets

⫽ 20,338; n

users

⫽ 4,415).

See the online article for the color version of this figure.

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

FALSE ALERT ANXIETY

Results

First, daily proportions of anxiety across the full range

of available data were examined. Consistent with the

hypothesis that anxiety would increase on the day of the

alert, across all users, anxiety increased 4.6% (standard-

ized b ⫽ .25, standard error [SE] ⫽ .01, t ⫽ 19.62, p ⬍

.001; see Figure 1). In terms of percent change, this jump

represents a 160% increase in anxiety expression.

Anxiety was also hypothesized to increase incrementally

during the period from the release of the alert until the

“all-clear” 38 min later, during which time users awaited the

attack (i.e., incubation of threat; Breznitz, 1968, 1984).

Consistent with this hypothesis, the results from 4,415 users

(20,338 tweets) who tweeted in this time frame indicated

that anxiety increased 3.4% in each 15-min block during the

alert period (standardized b ⫽ .12, SE ⫽ .02, t ⫽ 5.90, p ⬍

.001; see Figure 2), until the all-clear was transmitted, at

which point anxiety expression began to decline over time.

In a window 7 days before and after the missile alert (see

Figure 3), the low prealert anxiety group’s anxiety increased

9.5% at the onset of the alert period (standardized b ⫽ .49,

SE ⫽ .02, t ⫽ 22.87, p ⬍ .001). The medium prealert

anxiety group also increased in anxiety expression (5.8%;

standardized b ⫽ .30, SE ⫽ .03, t ⫽ 8.34, p ⬍ .001).

Although the high prealert anxiety group was hypothesized

to exhibit an increase in anxiety expression, this group’s

anxiety decreased 8.8% (standardized b ⫽⫺.46, SE ⫽ .07,

t ⫽⫺5.93, p ⬍ .001) at the onset of the alert.

Postevent stabilization rates also differed across groups

(see Figure 4). As expected, the low prealert anxiety group

took the longest to stabilize (⬃41 hr after the false missile

alert). However, the high prealert anxiety group stabilized

immediately after the alert was transmitted, the same point

at which a significant drop in anxiety was observed for this

group; the medium prealert anxiety group stabilized 23 hr

postevent.

Each group’s postalert baseline (after stabilization) was

compared with its prealert baseline. The low prealert anxi-

Figure 3. Hourly anxiety by Twitter users with low, medium, and high prealert anxiety, 7 days before and 7

days after the false missile alert (n

tweets

⫽ 324,010; n

users

⫽ 8,746). See the online article for the color version

of this figure.

Figure 4. Change point analyses for each prealert anxiety group, in

which Time 169 (vertical dashed line) is the moment the alert was trans-

mitted: (a) group with low prealert anxiety (n ⫽ 6,849); (b) group with

medium prealert anxiety (n ⫽ 1,394); (c) group with high prealert anxiety

(n ⫽ 503). See the online article for the color version of this figure.

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

JONES AND SILVER

ety group, for whom anxiety expression before the alert was

zero across a 7-day period, exhibited a new anxiety baseline

that was 2.5% higher than its prealert baseline (standardized

b ⫽ .13, SE ⫽ .004, t ⫽ 28.15, p ⬍ .001). The medium

prealert anxiety group’s new baseline was less than 1.0%

lower than its prealert baseline (standardized b ⫽⫺.01,

SE ⫽ .005, t ⫽⫺2.11, p ⫽ .035). The high prealert anxiety

group exhibited a new baseline 10.5% lower than its prealert

baseline (standardized b ⫽⫺.55, SE ⫽ .05, t ⫽⫺10.76,

p ⬍ .001).

Discussion

For the first time, real-time psychological responses to a

false alarm event have been captured using a big-data ap-

proach with large-scale observational Twitter data. Consis-

tent with prior Twitter studies of collective trauma in which

expressions of negative affect have been explored (De

Choudhury et al., 2014; Doré et al., 2015; Gruebner et al.,

2016, 2017; Jones et al., 2016, 2017; Lin et al., 2017;

Murthy & Longwell, 2013), this study reveals a marked

increase in anxiety among likely Hawaii residents that lin-

gered well after the missile threat was dispelled. In the most

fine-grained analysis, results were consistent with the incu-

bation of threat hypothesis (Breznitz, 1968, 1971, 1984),

which states that anxiety experienced in anticipation of a

threat will increase in the moments during which one waits

for the threat to arrive. Specifically, anxiety increased 3.4%

every 15 min until the transmission of a message 38 min

later reporting that the initial alert was a false alarm. Sur-

prisingly, this trend was not thwarted by corrective tweets

posted by the state’s emergency management agency and by

a local congressional representative shortly after the alert’s

transmission explaining that the missile alert was a mistake.

Despite these corrections being retweeted by 35,000 users,

the incubation of threat phenomenon persisted. This is

likely the case because users either (a) did not see the

messages dispelling the threat, or (b) saw them but did not

believe them, as evidenced by a user who tweeted: “I’m still

sheltering twenty minutes [later] who knows who’s right?”

Alternatively, this “incubation of threat” trend could sim-

ply reflect the fact that unique users became aware of the

missile alert at different points in time during the alert

period. For example, if a large number of users woke up to

the alert 35 min into the ordeal and generated tweets ex-

pressing anxiety, this would explain the observed increase

in anxiety during the alert period. This alternative explana-

tion was evaluated among 556 users who tweeted in at least

two of the three 15-min intervals during the alert period. We

found that anxiety increased incrementally across the alert

period (at nearly the same rate) among these users as well.

Overall, this suggests that evaluating tweets generated in the

alert period, in aggregate, sufficiently captured the experi-

ence of the incubation of threat.

Consistent with early work suggesting that the cancella-

tion of a threat does not immediately remedy reactions to a

threatening situation (Breznitz, 1984), our results suggest

that the experience of a false alarm may have a lingering

impact on some individuals well after the threat is dispelled.

The analysis of the entire sample across a 9-week window

revealed that anxiety remained elevated for at least 2 days

following the alert. When disaggregating users by their

preevent propensity for expressing anxiety words, disparate

patterns of postalert anxiety expression were uncovered. For

example, for the group of users in this sample who did not

express any anxiety before the event (i.e., the low prealert

anxiety group), anxiety increased the most and lingered the

longest, relative to other groups, before stabilizing to a new

baseline level 2.5% higher than what it was before the

missile alert. Insofar as anxiety expression on Twitter can be

assumed to be reflective of a user’s life experience (Jones et

al., 2016), this pattern is consistent with evidence demon-

strating that people who are likely to have had lives devoid

of psychologically impactful negative experiences are at

increased risk of negative psychological outcomes follow-

ing a traumatic event (Seery et al., 2010). Moreover, the

lingering presence of anxiety well after the threat was

dispelled may be driven by some users in this group engag-

ing in perseverative cognition, defined as the chronic acti-

vation of the cognitive representation of a psychological

stressor (Brosschot, Gerin, & Thayer, 2006).

Research also suggests that individuals with pretrauma

vulnerabilities (e.g., negative psychological states; Asarnow

et al., 1999; Nolen-Hoeksema & Morrow, 1991) may be at

increased risk for negative psychological outcomes after a

traumatic event. However, our results show a different

pattern. The high prealert anxiety group exhibited a de-

crease in anxiety following alert period, declining to a new

baseline level 10.5% lower than the group’s prealert base-

line. It should be noted that this pattern is unlikely to be the

result of regression to the mean because several prealert

observations were available with which to accurately ac-

count for natural variations in anxiety expression and iden-

tify the “high” prealert anxiety group. However, diminished

anxiety after the alert is consistent with downward counter-

factual thinking (Byrne, 2016), such that users in the high

prealert anxiety group may have recognized how much

worse things could have been had the missile threat been

real. It could also be indicative of near-miss relief (Sweeny

& Vohs, 2012), a phenomenon observed when an aversive

event is avoided.

Alternatively, we offer a few potential explanations that

may be particularly relevant to this group. For example, a

pattern of generalized worry about future negative events,

characteristic of anxious individuals, may have buffered

those in the high prealert anxiety group from experiencing

even greater anxiety as a result of the alert (for a review of

such “upsides” of worry, see Sweeny & Dooley, 2017). It

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

FALSE ALERT ANXIETY

could also be the case that lower anxiety expression in this

group reflects a process by which the threat of death via a

ballistic missile put disruptive daily stressors into perspec-

tive. This supposition is consistent with the notion of a

leveling theory of human adaptation to life events in which

people react differently to positive and negative experi-

ences, and how one responds depends on the totality of a

person’s life. For example, hedonic leveling (Lucas, Clark,

Georgellis, & Diener, 2003) posits that happy individuals

have less to gain when they experience positive life events

(e.g., marriage), whereas unhappy people have more to gain

from positive life events. The pattern for this group is also

reminiscent of adaptation level theory (Brickman, Coates, &

Janoff-Bulman, 1978), which highlights the importance of

the contrast between what life is like after a significant

event compared with what life was like beforehand. Insofar

as these theories operate similarly with respect to negative

life events (e.g., the false missile alert), anxious individuals

may have more to appreciate when they experience a near-

miss and thus express less anxiety on social media after

having “survived” what would have undoubtedly been con-

strued as a deadly situation.

Although our results mirror those in other studies of

Twitter data that reveal increased negative emotion after a

collective trauma, we acknowledge several limitations.

First, Twitter users are not necessarily representative of the

general population, as they tend to be younger (aged 18–29

years) and from urban locations (Pew Research Center,

2018). Second, we recognize that counting anxiety words

embedded in tweets may not be a perfect measure of actual

felt anxiety. Nonetheless, there are myriad face-valid exam-

ples of expressions of anxiety in the data we collected (e.g.,

“I’m scared guys”; “This is some scary [expletive]”). There

is also a tradition in psychology of relying on the words

people use to provide a window into their psychological

state (for a review, see Pennebaker, Mehl, & Niederhoffer,

2003; Tausczik & Pennebaker, 2010). For example, re-

searchers have linked depression to the use of first-person

pronouns and negative emotion words (Rude, Gortner, &

Pennebaker, 2004) and linked word usage to motivation

(Pennebaker & King, 1999) and to traditional personality

constructs (i.e., the Big 5; Schwartz et al., 2013), respec-

tively. On balance, we believe that capturing anxiety words

in this false alarm context reflects an important psycholog-

ical signal despite the potential error inherent in a dictionary

word count approach, and we maintain a person’s prealert

word usage pattern to be an acceptable proxy when other

information is unavailable.

We also acknowledge that the method we used to source

Twitter users precluded our ability to distinguish psycho-

logical responses of temporary visitors to the state com-

pared with permanent residents. Moreover, the data we had

available to us limited our ability to examine nuanced

responses to the alert based on individual characteristics that

we could not assess. Thus, the tweets we collected, although

informative, are not linked directly to any person-level

psychological data (e.g., personality, history of negative life

events). In addition, it is possible that individuals with

military connections, or Native Hawaiians versus those who

recently relocated, may have experienced differential levels

of anxiety in response to the false alert that our methodol-

ogy could not capture. Other methods (e.g., survey research)

might supplement the kinds of data we examine here to

further explore these other factors (see Jones et al., 2017, for

an example of this technique).

Finally, there is no guarantee that all users in our sample

were residents of Hawaii (although ⬃50% of users men-

tioned a Hawaii locale in their Twitter profile); we did not

rely on geolocation data because less than 1% (.07%) of

tweets in our data were geocoded. However, the method we

used to source locally generated Twitter data has performed

well in other studies of collective traumas (see Jones et al.,

2016, 2017). In addition, the presence of heightened anxi-

ety, despite the error inherent this method, suggests that the

effects we demonstrated would likely have been even

greater had we captured tweets generated by residents ex-

clusively.

Despite these limitations, our results lead to several rec-

ommendations for mitigating the psychological impact of

impending threats, false or otherwise. Early theorizing on

false alarms posited the false alarm or “cry wolf” effect in

which individuals may not believe the next threat warning

because they lose faith in the credibility of systems or

agencies responsible for disseminating them (Breznitz,

1984). Empirical work supports this. For example, residents

of tornado-prone areas who perceive their local false alarm

ratio to be high are less trusting of the National Weather

Service (Ripberger et al., 2015). Other work indirectly

shows that such perceptions can be deadly (Simmons &

Sutter, 2009). Credibility loss is particularly important to

combat in risk-prone locales like Hawaii, where residents

remain in targeting range of a ballistic missile from North

Korea, live near active volcanoes, and sometimes experi-

ence destructive weather events. In such a locale, it is

critical to maintain the public’s continued reliance on and

trust in emergency management systems for information

about impending danger.

When emergency systems falter, research shows that

credibility loss can be mitigated by a clear explanation of

why the false alarm occurred in the first place (Breznitz,

1984; Fischhoff, 2011). In the days and weeks following the

Hawaii false missile alert, the media reported that the alert

was sent in error by an employee with whom the agency had

several past issues. This reporting likely raised questions

about the agency’s disciplinary procedures but may have

assured the public that the entire affair was a fluke. Further

reporting assured the public that new safeguards were in

place to prevent one person from having the authority to

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

JONES AND SILVER

transmit any message on the statewide emergency system

(Wamsley, 2018).

During a crisis, social media can be an important channel

through which critical updates are transmitted. Indeed, the

value of using social media accounts to transmit critical

updates, in tandem with outlined procedures, has been ac-

knowledged by the Department of Homeland Security for

almost a decade (Silver & Fischhoff, 2011). Agencies

should also increase public outreach efforts to ensure that

community members are connected to critical information

channels during a crisis. Such measures would serve to

bolster the public’s trust in reporting agencies. We also

believe it is good advice for the public to seek out and

follow verified social media accounts belonging to their

local emergency management agencies as soon as possible

so that they will have access to the most up-to-date infor-

mation about any potential threat during a real emergency.

In addition to revealing new information channels to

follow, false alarms may also raise awareness of potential

threats. For example, some individuals in our sample ex-

pressed the realization of not being prepared: “Definitely

scary. I live in a small studio a few blocks from the beach

I was like take shelter where? I’m definitely not prepared.”

In the months before the false alert, aggressive rhetoric

between government leaders heightened fears that a military

conflict might be imminent. As a result, in December 2017,

Hawaii began performing emergency drills and siren testing

to prepare for such an eventuality, although what to do

during an attack was not clear to all residents (Kelkar,

2018). In our sample, many users expressed confusion about

what actions to take and were unsure where to seek shelter

when instructed to do so. It is prudent for members of the

public to educate themselves about emergency preparations

and for emergency management agencies to use all avail-

able channels to transmit concrete recommendations for

protective action (Fischhoff, 2011).

Insights from our data also highlight the importance of the

interface between emergency management agencies and the

news media. Users in our sample searched for information

on traditional media channels to no avail, evidenced by

tweets like “missile alert in Hawaii but no news coverage”

and “nothing on the news. looked it up on twitter and people

are as confused as we are.” These tweets highlight the

crucial importance of information dissemination via the

news media during a crisis as the public relies on the media

for critical updates (Jones et al., 2017).

Although the missile alert was an unusual event, false

alarms of impending threats are not uncommon. For exam-

ple, near the end of 2018, a false active-shooter alert was

transmitted to the Walter Reed National Military Medical

Center. The alert was transmitted widely without the words

“exercise” or “drill” and was accordingly believed to be a

legitimate warning. This false alert precipitated an hour-

long lockdown of the hospital and was ultimately charac-

terized as an “improper use of a mass notification system”

(Martinez, 2018). As mass-communications technologies

are developed to warn people of life-threatening events at

both the local (e.g., schools) and national (e.g., presidential

alerts) levels, such occurrences may increase if systems

operate without clear procedures and proper oversight.

Free and open access to public Twitter data, coupled with

Hawaii’s false missile alert, provided an opportunity to

study, for the first time, how several thousand people re-

sponded psychologically to the threat of an inescapable,

impending tragedy. Although it is fortunate we were able to

study this phenomenon without loss of life, we show that for

many users, the anxiety elicited by this false alarm lingered

well beyond the assurance that the threat was not real, which

may have health consequences over time for some individ-

uals (Holman et al., 2008). Thus, our results reveal how

potently frightening a crisis period can be and highlight how

intense experiences like this may have lasting effects that

become even clearer after disaggregating users by their

preevent psychological state. This event serves as an exam-

ple of how accidental crisis communication can become a

disaster (Gilbert, 1998) and should inform emergency man-

agement agencies how they can better serve the communi-

ties they are charged with protecting.

References

Asarnow, J., Glynn, S., Pynoos, R. S., Nahum, J., Guthrie, D., Cantwell,

D. P., & Franklin, B. (1999). When the earth stops shaking: Earthquake

sequelae among children diagnosed for pre-earthquake psychopathol-

ogy. Journal of the American Academy of Child & Adolescent Psychi-

atry, 38, 1016–1023. http://dx.doi.org/10.1097/00004583-199908000-

00018

Breznitz, S. (1968). “Incubation of threat” in a situation of conflicting

expectations. Psychological Reports, 22, 755–756. http://dx.doi.org/10

.2466/pr0.1968.22.3.755

Breznitz, S. (1971). A study of worrying. The British Journal of Social &

Clinical Psychology, 10, 271–279. http://dx.doi.org/10.1111/j.2044-8260

.1971.tb00747.x

Breznitz, S. (1984). Cry wolf: The psychology of false alarms. Hillsdale,

NJ: Erlbaum.

Breznitz, S. (1985). False alarms: Their effects on fear and adjustment.

Issues in Mental Health Nursing, 7, 335–348. http://dx.doi.org/10.3109/

01612848509009460

Brickman, P., Coates, D., & Janoff-Bulman, R. (1978). Lottery winners and

accident victims: Is happiness relative? Journal of Personality and Social

Psychology, 36, 917–927. http://dx.doi.org/10.1037/0022-3514.36.8.917

Brosschot, J. F., Gerin, W., & Thayer, J. F. (2006). The perseverative

cognition hypothesis: A review of worry, prolonged stress-related phys-

iological activation, and health. Journal of Psychosomatic Research, 60,

113–124. http://dx.doi.org/10.1016/j.jpsychores.2005.06.074

Byrne, R. M. (2016). Counterfactual thought. Annual Review of Psychology,

67, 135–157. http://dx.doi.org/10.1146/annurev-psych-122414-033249

Chen, C. Y., & Hong, R. Y. (2010). Intolerance of uncertainty moderates

the relation between negative life events and anxiety. Personality and

Individual Differences, 49, 49–53. http://dx.doi.org/10.1016/j.paid.2010

.03.006

De Choudhury, M., Monroy-Hernandez, A., & Mark, G. (2014). “Narco”

emotions: Affect and desensitization in social media during the Mexican

drug war. In Proceedings of the 32nd Annual ACM Conference on

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

FALSE ALERT ANXIETY

Human Factors in Computing Systems (pp. 3563–3572). New York, NY:

ACM.

Doré, B., Ort, L., Braverman, O., & Ochsner, K. N. (2015). Sadness shifts

to anxiety over time and distance from the national tragedy in Newtown,

CT. Psychological Science, 26, 363–373. http://dx.doi.org/10.1177/

0956797614562218

Dow, K., & Cutter, S. L. (1998). Crying wolf: Repeat responses to

hurricane evacuation orders. Coastal Management, 26, 237–252. http://

dx.doi.org/10.1080/08920759809362356

Fischhoff, B. (2011). Communicating about the risks of terrorism (or

anything else). American Psychologist, 66, 520–531. http://dx.doi.org/

10.1037/a0024570

Folkman, S., Lazarus, R. S., Dunkel-Schetter, C., DeLongis, A., & Gruen, R. J.

(1986). Dynamics of a stressful encounter: Cognitive appraisal, coping, and

encounter outcomes. Journal of Personality and Social Psychology, 50,

992–1003. http://dx.doi.org/10.1037/0022-3514.50.5.992

Gilbert, C. (1998). Studying disaster. In E. L. Quarantelli (Ed.), What is a

disaster? Perspectives on the question (pp. 11–18). New York, NY:

Routledge.

Gruebner, O., Lowe, S. R., Sykora, M., Shankardass, K., Subramanian, S. V.,

& Galea, S. (2017). A novel surveillance approach for disaster mental

health. PLoS ONE, 12, e0181233. http://dx.doi.org/10.1371/journal.pone

.0181233

Gruebner, O., Sykora, M., Lowe, S. R., Shankardass, K., Trinquart, L.,

Jackson, T.,...Galea, S. (2016). Mental health surveillance after the

terrorist attacks in Paris. The Lancet, 387, 2195–2196. http://dx.doi.org/

10.1016/S0140-6736(16)30602-X

Holman, E. A., Silver, R. C., Mogle, J. A., & Scott, S. B. (2016). Adversity,

time, and well-being: A longitudinal analysis of time perspective in adult-

hood. Psychology and Aging, 31, 640–651. http://dx.doi.org/10.1037/

pag0000115

Holman, E. A., Silver, R. C., Poulin, M., Andersen, J., Gil-Rivas, V., &

McIntosh, D. N. (2008). Terrorism, acute stress, and cardiovascular health:

A 3-year national study following the September 11th attacks. Archives of

General Psychiatry, 65, 73–80. http://dx.doi.org/10.1001/archgenpsychia

try.2007.6

Jones, N. M., Thompson, R. R., Dunkel Schetter, C., & Silver, R. C.

(2017). Distress and rumor exposure on social media during a campus

lockdown. Proceedings of the National Academy of Sciences of the

United States of America, 114, 11663–11668. http://dx.doi.org/10.1073/

pnas.1708518114

Jones, N. M., Wojcik, S. P., Sweeting, J., & Silver, R. C. (2016). Tweeting

negative emotion: An investigation of Twitter data in the aftermath of

violence on college campuses. Psychological Methods, 21, 526–541.

http://dx.doi.org/10.1037/met0000099

Kearney, M. W. (2017). rtweet: Collecting Twitter data. Retrieved from

https://cran.r-project.org/package⫽rtweet

Kelkar, K. (2018). Hawaii sounds first nuclear warning siren since Cold

War. Retrieved from https://www.pbs.org/newshour/nation/hawaii-

sounds-first-nuclear-warning-siren-since-cold-war

Killick, R., & Eckley, I. A. (2014). Changepoint: An R package for

changepoint analysis. Journal of Statistical Software, 58, 1–19. http://

dx.doi.org/10.18637/jss.v058.i03

Killick, R., Haynes, K., & Eckley, I. A. (2016). Changepoint: An R

package for changepoint analysis. Retrieved from https://CRAN.R-

project.org/package⫽changepoint

Kim, H. J., Fay, M. P., Feuer, E. J., & Midthune, D. N. (2000). Permutation

tests for joinpoint regression with applications to cancer rates. Statistics

in Medicine, 19, 335–351. http://dx.doi.org/10.1002/(SICI)1097-

0258(20000215)19:3⬍335::AID-SIM336⬎3.0.CO;2-Z

Lazarus, R. S. (1966). Psychological stress and the coping process. New

York, NY: McGraw-Hill.

Lin, Y.-R., Margolin, D., & Wen, X. (2017). Tracking and analyzing

individual

distress following terrorist attacks using social media streams.

Risk Analysis, 37, 1580–1605. http://dx.doi.org/10.1111/risa.12829

Lucas, R. E., Clark, A. E., Georgellis, Y., & Diener, E. (2003). Reexam-

ining adaptation and the set point model of happiness: Reactions to

changes in marital status. Journal of Personality and Social Psychology,

84, 527–539. http://dx.doi.org/10.1037/0022-3514.84.3.527

Martinez, G. (2018). Navy: Active shooter alarm at Walter Reed caused by

“improper use of a mass notification system”. Retrieved from http://

time.com/5465194/walter-reed-active-shooter-navy-mass-notifications/

?utm_source⫽emailshare&utm_medium⫽email&utm_campaign⫽

email-share-article&utm_content⫽20181128

Mitchell, M. N. (2012). Interpreting and visualizing regression models

using Stata. College Station, TX: Stata Press.

Murthy, D., & Longwell, S. A. (2013). Twitter and disasters: The uses of

Twitter during the 2010 Pakistan floods. Information, Communication &

Society, 16, 837–855. http://dx.doi.org/10.1080/1369118X.2012.696123

Nolen-Hoeksema, S., & Morrow, J. (1991). A prospective study of depres-

sion and posttraumatic stress symptoms after a natural disaster: The

1989 Loma Prieta earthquake. Journal of Personality and Social Psy-

chology, 61, 115–121. http://dx.doi.org/10.1037/0022-3514.61.1.115

Norris, F. H., Friedman, M. J., Watson, P. J., Byrne, C. M., Diaz, E., &

Kaniasty, K. (2002). 60,000 disaster victims speak: Part I. An empirical

review of the empirical literature, 1981–2001. Psychiatry: Interpersonal

and Biological Processes, 65, 207–239. http://dx.doi.org/10.1521/psyc

.65.3.207.20173

Pactol, C. C. (2018). False missile alert causes panic. Retrieved from

https://themolokaidispatch.com/false-missile-alert-causes-panic/

Pennebaker, J. W., Booth, R. J., Boyd, R. L., & Frances, M. E. (2015).

Linguistic Inquiry and Word Count: LIWC2015. Austin, TX: Penne-

baker Conglomerates.

Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use

as an individual difference. Journal of Personality and Social Psychol-

ogy, 77, 1296–1312. http://dx.doi.org/10.1037/0022-3514.77.6.1296

Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psycho-

logical aspects of natural language. use: Our words, our selves. Annual

Review of Psychology, 54, 547–577. http://dx.doi.org/10.1146/annurev

.psych.54.101601.145041

Pew Research Center. (2018). Social media use in 2018. Retrieved from

http://www.pewinternet.org/2018/03/01/social-media-use-in-2018/

R Core Team. (2018). R: A language and environment for statistical

computing. Vienna, Austria: R Foundation for Statistical Computing.

Retrieved from http://www.R-project.org/

Ripberger, J. T., Silva, C. L., Jenkins-Smith, H. C., Carlson, D. E., James,

M., & Herron, K. G. (2015). False alarms and missed events: The impact

and origins of perceived inaccuracy in tornado warning systems. Risk

Analysis, 35, 44–56. http://dx.doi.org/10.1111/risa.12262

Rodríguez, H., Díaz, W., Santos, J. M., & Aguirre, B. E. (2007). Commu-

nicating risk and uncertainty: Science, technology, and disasters at the

crossroads. In H. Rodríguez, E. L. Quarantelli, & R. R. Dynes (Eds.),

Handbook of disaster research (pp. 476–488). New York, NY: Spring-

er. http://dx.doi.org/10.1007/978-0-387-32353-4_29

Rosen, N. O., Knäuper, B., & Sammut, J. (2007). Do individual differences

in intolerance of uncertainty affect health monitoring? Psychology &

Health, 22, 413–430. http://dx.doi.org/10.1080/14768320600941038

Rude, S., Gortner, E. M., & Pennebaker, J. (2004). Language use of

depressed and depression-vulnerable college students. Cognition and

Emotion, 18, 1121–1133. http://dx.doi.org/10.1080/02699930441

000030

Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones,

S. M., Agrawal, M.,...Ungar, L. H. (2013). Personality, gender, and

age in the language of social media: The open-vocabulary approach.

PLoS ONE, 8, e73791. http://dx.doi.org/10.1371/journal.pone.0073791

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

JONES AND SILVER

Seery, M. D., Holman, E. A., & Silver, R. C. (2010). Whatever does not

kill us: Cumulative lifetime adversity, vulnerability, and resilience.

Journal of Personality and Social Psychology, 99, 1025–1041. http://dx

.doi.org/10.1037/a0021344

Silge, J., & Robinson, D. (2016). tidytext: Text mining and analysis using

tidy data principles in R. Journal of Open Source Software, 1, 37.

http://dx.doi.org/10.21105/joss.00037

Silva, D. (2018). Hawaii false alarm: Ensuing chaos is teachable moment,

experts say. Retrieved from https://www.nbcnews.com/news/us-news/

hawaii-false-alarm-ensuing-chaos-teachable-moment-experts-say-

n838836

Silver, R. C. (2004). Conducting research after the 9/11 terrorist attacks:

Challenges and results. Families, Systems, & Health, 22, 47–51. http://

dx.doi.org/10.1037/1091-7527.22.1.47

Silver, R. C., & Fischhoff, B. (2011). What should we expect after the next

attack? American Psychologist, 66, 567–572. http://dx.doi.org/10.1037/

a0024893

Simmons, K. M., & Sutter, D. (2009). False alarms, tornado warnings, and

tornado casualties. Weather, Climate, and Society, 1, 38–53. http://dx

.doi.org/10.1175/2009WCAS1005.1

Sweeny, K., & Dooley, M. D. (2017). The surprising upsides of worry.

Social and Personality Psychology Compass, 11, e12311. http://dx.doi

.org/10.1111/spc3.12311

Sweeny, K., & Vohs, K. D. (2012). On near misses and completed tasks:

The nature of relief. Psychological Science, 23, 464–468. http://dx.doi

.org/10.1177/0956797611434590

Taha, S. A., Matheson, K., & Anisman, H. (2014). H1N1 was not all that

scary: Uncertainty and stressor appraisals predict anxiety related to a

coming viral threat. Stress and Health: Journal of the International

Society for the Investigation of Stress, 30, 149–157. http://dx.doi.org/10

.1002/smi.2505

Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning

of words: LIWC and computerized text analysis methods. Journal of

Language and Social Psychology, 29, 24–54. http://dx.doi.org/10.1177/

0261927X09351676

Thistlethwaite, D., & Campbell, D. (1960). Regression-discontinuity anal-

ysis: An alternative to the ex post facto experiment. Journal of Educa-

tional Psychology, 51, 309–317. http://dx.doi.org/10.1037/h0044319

Wamsley, L. (2018). Worker who sent Hawaii false alert thought missile

attack was imminent. Retrieved from https://www.npr.org/sections/

thetwo-way/2018/01/30/581853255/hawaii-missile-drill-stated-this-is-

not-a-drill-resulting-in-false-alert

Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New

York, NY: Springer-Verlag. Retrieved from http://ggplot2.org

Received December 28, 2018

Revision received March 28, 2019

Accepted May 2, 2019 䡲

This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

FALSE ALERT ANXIETY