Tweeted Anger Predicts County-Level Results of the 2016 United States Presidential Election

Two years after Donald Trump has been sworn into office, America is more than ever divided about its 45th president (Brownstein, 2018). The results of the midterm elections as the first major electoral test of Donald Trump’s presidency show that a great number of U.S. citizens oppose his person and political agenda, while many still support his actions, regardless of controversial debates and turmoil surrounding his presidency. But who elected Trump in the first place? In the aftermath of the election, different attempts to characterize Trump’s supporters have been made. A number of empirical studies already offered some compelling explanations for why Donald Trump won the presidency, ranging from status threat among privileged Americans (e.g., whites, Christians, and men) to economic anxiety to racism and sexism (Brenan, 2018; Mutz, 2018; Rothwell, 2016; Schaffner, Macwilliams, & Nteta, 2018). A possible explanation that has been put forward by popular media outlets, but has so far not been tested empirically, is that voters’ negative emotions—and anger in particular—played a significant role in the 2016 election. The Washington Post, for instance, ran articles with the headlines: ‘Inside the anger that gave us Trump’ and ‘A new theory for why Trump voters are so angry’. In these articles it was speculated that Trump attracted voters who were angry about the general way how politics were done in the US, because his complaints and outbursts gave voice to their personal feelings. In line with this idea, the present research examines, first, whether Trump was more successful in U.S. counties where citizens expressed more negative emotions—and particularly anger—on social media. Thereby, we make use of the unique possibilities of social media to capture and store emotional expression from a large number of citizens and examine in a quasi-prospective design whether emotional expressions on Twitter, conceptualized as an indicator of stable regional differences in emotional experiences, relate to actual voting behavior at county-level. In these analyses, we control for plausible third variables suggested by previous studies, such as counties’ economic situation, level of education, and percentage of minority and female population (Brenan, 2018; Mutz, 2018; Rothwell, 2016; Schaffner et al., 2018). Further, we examined one possible explanation for why negative emotions and anger in particular predict relative vote choice for Donald Trump. Previous research shows that voters prefer political messages that they match their current emotional state (Roseman, Abelson, & Ewing, 1986). Based on this research, we argue that Trump was more successful in counties where residents expressed more negative emotions and anger on social media, because he frequently expressed these emotions in his presidential campaign. In line with this idea, we tested whether Trump’s campaign was exceptional in the expression of negative emotions and anger compared to the campaigns of his opponent, Hillary Clinton, and his forerunner, Mitt Romney. Bernecker, K., et al. (2019). Tweeted Anger Predicts County-Level Results of the 2016 United States Presidential Election. International Review of Social Psychology, 32(1): 6, 1–14. DOI: https://doi.org/10.5334/irsp.256


Introduction
Two years after Donald Trump has been sworn into office, America is more than ever divided about its 45th president (Brownstein, 2018).The results of the midterm elections as the first major electoral test of Donald Trump's presidency show that a great number of U.S. citizens oppose his person and political agenda, while many still support his actions, regardless of controversial debates and turmoil surrounding his presidency.But who elected Trump in the first place?In the aftermath of the election, different attempts to characterize Trump's supporters have been made.A number of empirical studies already offered some compelling explanations for why Donald Trump won the presidency, ranging from status threat among privileged Americans (e.g., whites, Christians, and men) to economic anxiety to racism and sexism (Brenan, 2018;Mutz, 2018;Rothwell, 2016;Schaffner, Macwilliams, & Nteta, 2018).
A possible explanation that has been put forward by popular media outlets, but has so far not been tested empirically, is that voters' negative emotions-and anger in particular-played a significant role in the 2016 election.The Washington Post, for instance, ran articles with the headlines: 'Inside the anger that gave us Trump' and 'A new theory for why Trump voters are so angry'.In these articles it was speculated that Trump attracted voters who were angry about the general way how politics were done in the US, because his complaints and outbursts gave voice to their personal feelings.
In line with this idea, the present research examines, first, whether Trump was more successful in U.S. counties where citizens expressed more negative emotions-and particularly anger-on social media.Thereby, we make use of the unique possibilities of social media to capture and store emotional expression from a large number of citizens and examine in a quasi-prospective design whether emotional expressions on Twitter, conceptualized as an indicator of stable regional differences in emotional experiences, relate to actual voting behavior at county-level.In these analyses, we control for plausible third variables suggested by previous studies, such as counties' economic situation, level of education, and percentage of minority and female population (Brenan, 2018;Mutz, 2018;Rothwell, 2016;Schaffner et al., 2018).
Further, we examined one possible explanation for why negative emotions and anger in particular predict relative vote choice for Donald Trump.Previous research shows that voters prefer political messages that they match their current emotional state (Roseman, Abelson, & Ewing, 1986).Based on this research, we argue that Trump was more successful in counties where residents expressed more negative emotions and anger on social media, because he frequently expressed these emotions in his presidential campaign.In line with this idea, we tested whether Trump's campaign was exceptional in the expression of negative emotions and anger compared to the campaigns of his opponent, Hillary Clinton, and his forerunner, Mitt Romney.

The Role of Emotions in Political Preference
For a long time research in political science largely neglected the role of emotions in the formation of political preference, and assumed that people approach political information in a ' cold', rational manner (for reviews see Glaser & Salovey, 1998;Marcus, 2000).Over the past decades, however, empirical evidence accumulated suggesting that political preferences are strongly influenced by how people feel about political candidates or issues.This research indicates that emotions and mood states, whether caused by the political object in question or an independent event, strongly affect voter's preferences apart and even beyond rational considerations (Glaser & Salovey, 1998;Marcus, 2000).For instance, research demonstrated that people are more likely to support political incumbents if their favorite sports team wins, presumably because this independent event can induce a positive mood, which becomes falsely attributed to the current political situation (Healy, Malhotra, & Mo, 2010).Other research showed that emotional reactions regarding candidates more strongly predicted overall evaluations of these politicians than personality judgments or political partisanship (Abelson, Kinder, Peters, & Fiske, 1982;Anderson & Granberg, 1991;Finn & Glaser, 2010;Granberg & Brown, 1989;Marcus, 1988;Sullivan & Masters, 1988).
Another line of research suggests that campaigns can trigger certain emotions and thereby affect the way voters engage with the information presented (Brader, 2005;Marcus & MacKuen, 1993).Studies, for instance, showed that anxiety triggered by a campaign increases attention and promotes elaborate processing of campaigns' informational content (Marcus & MacKuen, 1993).Thereby, anxiety discourages voters to rely on more habitual cues, such as the political partisanship, to form their political preferences.In contrast, enthusiasm stimulates positive engagement with a campaign and its claims, and encourages voters to stick to their habitual political preferences (Marcus & MacKuen, 1993).Building upon these correlational findings, Brader (2005) conducted two experimental studies and found that anxiety cued by music and pictures stimulated vigilance and increased the reliance on the evaluations of issues presented over existing preferences.In other words, cued anxiety supported persuasion of voters.Cued enthusiasm, on the other hand, motivated engagement with a candidate and his campaign and fostered reliance on preexisting preferences (Brader, 2005).In sum, these findings suggest that emotional cues in political campaigns affect how voters engage with political messages and how much they take this information into consideration when judging political candidates.
Other research suggests that the effect of emotional appeals in campaigns also depends on the emotion voters bring to the situation.More specifically, studies on the concept of emotional resonance found that voters are more receptive to a political message if it matches their current emotion (Roseman et al., 1986).Mood congruence effects have first been described in the context of learning and memory, where early studies found that mood-congruent information is more attended, better memorized, and easier retrieved than mood-incongruent information (Blaney, 1986;Bower, 1981;Parrott & Sabini, 1990).Studies on visual perception further suggest that resonance effects are stronger in the case of matching emotions (e.g., sad-sad, angry-angry) compared to a simple match in affective valence (e.g., negative-negative) (Niedenthal & Setterlund, 1994).Roseman and colleagues (1986) were the first to document emotional resonance effects in the context of political information.They provided subjects with political statements of different emotional tones and found that voters who were induced a happy mood beforehand responded most favorably to statements including happy appeals, while anger-induced voters responded most favorably to statements including angry appeals (Roseman et al., 1986).More recent research on the presidential election in 2008 further suggests that anxiety might have cross-resonated with hope present in Obama's campaign, perhaps serving a palliative function (Finn & Glaser, 2010).Overall, these findings suggest that political campaigns should be particularly effective if they resonate with voters' emotions.
In line with this idea, we argue that Donald Trump might have been particularly successful in attracting voters who are more inclined to experience negative emotions and anger in particular by expressing those emotions in his campaign.To test this proposition, we first used computerized text analyses to examine emotions spontaneously expressed by US citizens on social media to predict actual vote choice for Donald Trump on the county level.Second, we analyzed emotions expressed in Donald Trump's campaign and compared it to the campaigns of the other presidential candidates in the elections of 2012 and 2016.Emotional resonance hypothesis would predict that the emotions which predict vote choice at county-level should also be dominant in Trump's campaign.Although our approach does not provide a direct test of the phenomenon of emotional resonance, because data is not assessed on the level of the individual, our county-level analyses may provide novel insights for the question who voted for Donald Trump (see Zerhouni, Rougier, & Muller, 2016, for a similar approach in France), while the analyses of campaign data may suggest one possible explanation for why a particular group of voters did so.

Language on Social Media as Measure of Emotions
Today, with about 500 million tweets per day, Twitter is among the most popular social networks.Being widely used and publicly available, tweets provide researchers with valuable insights into people's emotions, thoughts, and behavior.A vast body of research suggests that by analyzing language parameters (e.g., use of pronouns or other function words) and content of what is said, one can reliably infer sensible information about a person, such as demographic characteristics or personality traits ( Boutyline & Willer, 2017;Ireland & Pennebaker, 2010;Pennebaker, Mehl, & Niederhoffer, 2003).Research further shows that psychological states, such as being stressed or depressed, as well as emotional states, such as anger or fear, are reflected in people's language and, thus, can be inferred from oral or written speech (Johnson-Laird & Oatley, 1989;Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007;Pennebaker et al., 2003;Schwartz et al., 2013;Stone, Dunphy, Smith, & Ogilvie, 1966).
Past research utilized the availability of language samples on social networks like Twitter to predict relevant outcomes on the level of the population.For instance, a study showed that emotional language on Twitter is a reliable predictor of heart disease mortality in U.S. counties, and even predicts these outcomes over and above traditionally used indicators, such as counties' economic situation (Eichstaedt et al., 2015).Other studies used emotional language on Twitter to predict the prevalence of depression and well-being in different populations around the globe (De Choudhury, Counts, & Horvitz, 2013;Schwartz et al., 2013).More relevant for the present research, however, is a study that used emotional language on Twitter to forecast the results of a federal election in Germany (Tumasjan, Sprenger, Sandner, & Welpe, 2011).Analyzing sentiment, that is the emotional negativity versus positivity, of tweets directly referencing a political party or a politician, the study found that positive and negative emotions associated with a politician corresponded with voters' political preference (Tumasjan et al., 2011).Whereas this study predicted political preference from emotions toward candidates or parties-in other words, attitudes-the current research aimed to go one step further and predict vote choice at the county-level from emotions spontaneously and independently expressed by US citizens on social media six years prior to the election.
Although emotions are most often conceptualized as fluctuating states, there is extensive empirical work and theoretical considerations on trait anger and trait anxiety suggesting that people differ in the frequency and intensity with which they experience these emotions (Barnes, Harp, & Jung, 2002;Spielberger, Krasner, & Solomon, 1988;Spielberger, Reheiser, & Sydeman, 1995;Wilkowski & Robinson, 2010).Speaking for the validity of these concepts, research has linked trait anger and anxiety to basal processes like genetic expression, neurological functioning, and attentional biases (Bishop, 2009;Etkin et al., 2004;Harmon-Jones, 2007;Honk et al., 2001;Schinka, Busch, & Robichaux-Keene, 2004).Further, longitudinal studies on the life-span development of affect demonstrate that people are remarkably stable in their level of positive and negative affect over time spans of up to 23 years (Charles, Reynolds, & Gatz, 2001).While traits are grounded in genetic expression and neurological functioning, they are also heavily influenced by culture (Linton, 1945;Markus & Kitayama, 1991;Triandis, 1989), and as such not only differ between individuals but also between continents, countries, and also between regions and groups within one country (Grimm, Church, Katigbak, & Reyes, 1999;Na et al., 2010).For instance, studies suggests that there are regional differences within the U.S. with regard to the Big Five personality traits (Rentfrow, 2010).Further, research on regional differences in social orientation (i.e., individualism-collectivism) shows that Japanese culture fosters the expression of socially engaging emotions (e.g., friendly feelings and guilt), while North American culture fosters socially disengaging emotions (e.g., pride and anger; Kitayama, Mesquita, & Karasawa, 2006).Individualism and related cultural practices, on the other hand, differ also between regions and subgroups within the U.S. (Kitayama, Conway III, Pietromonaco, Park, & Plaut, 2010;Suizzo et al., 2008).
In sum, this research suggests that there might be stable regional differences within the U.S. with regard to the frequency of experiencing and expressing certain emotions like anger and anxiety.Language on social media should also reflect these differences which would allow using them to predict outcomes over longer time periods.Evidence that emotional language data predict election results at the county level would contribute to the experimental and correlational research summarized above in two ways: It would indicate that emotions not only directly influence political behavior, but also allow considering election behavior as one of the most influential political actions.

The Present Research
The first aim of the present research was to examine whether Donald Trump was more successful in counties where people expressed more negative emotions and anger on Twitter.To test this hypothesis, we merged an open-source data set of 148 million tweets collected in 2010 and mapped across 1,347 U.S. counties with the county-level results of the 2016 presidential election.Using a machine learning approach, we predicted the county-level election results with different emotional language indicators and simultaneously controlled for variables that either resemble possible confounds or predictors already tested in previous research (e.g., counties' socioeconomic status, minority population, health status).Additionally, we controlled for the results of the 2012 (and 2008) presidential election in order to show that expression of negative emotions and anger was not a mere predictor of voting conservatively (at least within the past eight years) but rather predicted vote choice for Donald Trump specifically.Further, following up on the idea that modern campaigns focus on voters' emotions rather than their rationale (Marcus, 2000), we explored whether language signaling cognitive (dis-)engagement predicted the election results.
Examining whether emotional resonance might explain why certain emotions predict success of Donald Trump at the county level, we tested whether Donald Trump's campaign was exceptional in the expression of the very same emotions.To this end, we collected transcribed campaign speeches of the four presidential candidates of 2012 and 2016 and pre-election tweets of Donald Trump and Hillary Clinton.Both, speeches and tweets were compared in terms of emotional expression.

Method
We used four different kinds of archival data.First, we used an open-source data set of tweets by U.S. citizens coded for different language features at the county level, which also included data about county characteristics (e.g., socioeconomic status).Second, we retrieved countylevel results for the presidential elections of 2012 and 2016 from an online data repository.Third, we retrieved transcribed campaign speeches of Donald Trump, Hillary Clinton, Mitt Romney, and Barack Obama from an open online repository.Fourth, we extracted Twitter data from Donald Trump and Hillary Clinton via web application programming interface (API).Using these different sorts of archival data allowed testing the role of emotions in the presidential election of 2016 on the basis of behavioral data from a large sample of U.S. citizens and an almost complete inventory of campaign speeches and pre-election tweets.

Procedure and Measures
Twitter data from counties.As to the county data, we used an existing open-source data set from Eichstaedt et al. (2015), who provide detailed information about data collection and modeling technique in their method section and supplemental material.Because this information is available elsewhere, we will provide only the most relevant information about data collection and preparation of this dataset here: The open-source data set created by Eichstaedt et al. (2015) was based on a random sample of 10% of tweets that Twitter made available to researchers.From this dataset a sample of 826 million tweets overall (collected between June 2009 and March 2010) were mapped to counties in the U.S. using users' self-reported locations in their user profiles.Overall, Eichstaedt et al. (2015) were able to map 148 million tweets across 1,347 counties (for details see the 'Mapping Tweets to Counties' section in their supplemental material).
County characteristics.Further, Eichstaedt et al. (2015) collected various county characteristics from U.S. officials and also made these data available to other researchers.From the American Community Survey (U.S. Census Bureau, 2009) they obtained the percentage of married residents and median income.The same source provided information about high school and college graduation rates, which were used to create an index of counties' level of education.Further, from the U.S. Census Bureau, they collected percentages of female, African American, and Hispanic residents living in a county (U.S. Census Bureau, 2010).Last, they collected age-adjusted mortality rates for atherosclerotic heart disease (AHD) from the Centers for Disease Control andPrevention (2009, 2010) for more detailed source information see Table S1 in the supplemental material of Eichstaedt et al. (2015).Because counties' AHD mortality rates were predicted by multiple Twitter language categories, including anger (Eichstaedt et al., 2015), we used it as control variables in our analyses.
Election data.County-level results of the presidential elections in 2012 and 2016 were available at github.com(https://github.com/tonmcg/County_Level_Election_Results_12-16/blob/master/US_County_Level_ Presidential_Results_08-16.csv) and were downloaded on 3rd April 2017.According to the contributors, the results of the 2012 election were published by "The Guardian" and 2016 election results were obtained from results published by Townhall.com.Because no information was provided about the source for the 2008 election results, they were not used as a control variable in the main analyses.But additional analyses revealed that controlling for the county-level results of 2008 did not change any of the results reported here.
Twitter data from candidates.Further, to be able to analyze language of the online campaigns in the 2016 election, we downloaded 3,200 tweets from the public Twitter accounts of Hillary Clinton and Donald Trump.Tweets were downloaded on 4th April 2017 via web API using the twitteR package (Gentry, 2015) in R version 3.3.3(R Core Team, 2017).Only tweets posted before Election Day were analyzed.For Donald Trump, these were n = 2,439 and for Hillary Clinton n = 3,127.As 3,200 is the maximum number of tweets that can be downloaded by a single user at a time the difference in the number of tweets per candidate is due to differences in twitter activity between Election Day and 4th April 2017.
Speech data from candidates.Transcripts of campaign speeches of Donald Trump (n = 77), Hillary Clinton (n = 55), Mitt Romney (n = 95), and Barack Obama (n = 105) were obtained from the UCSB American Presidency Project (http://www.presidency.ucsb.edu/index.php).For Donald Trump and Hillary Clinton additionally transcripts of the three presidential debates were downloaded on 5th April 2017 from github.com(https://github.com/paigecm/2016-campaign) and were originally obtained from the New York Times transcripts.
Because words can have multiple meanings (e.g., when negated) and automatic language software is limited in detecting these changes in meaning, Eichstaedt et al. (Eichstaedt et al., 2015) had two independent raters evaluate 200 tweets of their data set and compared their ratings with the results of the automatic language analyses.The accuracy level ranged between 55% to 89% for the different dictionaries (for more detailed information see Table S2 in the supplemental material of Eichstaedt et al. [2015]).
Data analysis.Our analyses involved a series of steps: First, we calculated partial correlations between the six Twitter language parameters (four emotional, two cognitive) and election results in 2012 and 2016, controlling for various county characteristics with regard to demographics (i.e., percentages of married, female, African American, Hispanic residents), and socioeconomic status (i.e., median income, education) and health (i.e., AHD mortality).Next, to test whether emotional language on Twitter significantly predicts the election results in 2016, we applied different machine learning techniques in R version 3.3.3(R Core Team, 2017).More specifically, we trained and crossvalidated a multiple regression model using the ' caret' package (Kuhn et al., 2016) and in that model, simultaneously controlled for the results of the 2012 election and various county characteristics.To address the possible problem of multicollinearity and to test which predictors actually contributed to the prediction of the election results, we applied an elastic net regression (Zou & Hastie, 2005) using the 'glmnet' package (Friedman, Hastie, & Tibshirani, 2010).This method allows the selection of a subset of predictors out of a larger pool of predictors based on their independent contribution to the predictive performance of a model.Thereby, it suggests the most parsimonious model with the best prediction.
In a second set of analyses, we tested whether Trump's campaign was exceptional in the expression of emotions that predicted his success on the county level.To this end, we compared mean values of emotional language indicators among the four presidential campaigns of 2012 and 2016.Because campaign data represent an (almost) complete inventory of tweets and campaign speeches rather than a sample, we did not apply inferential statistics.Instead, we calculated the mean differences between candidates' campaigns with regard to the six Twitter language categories and regard them as the true mean difference of the population (of speeches and tweets).The mean effect size (Cohen's d) for tweets and campaign speeches will be interpreted according to convention (Cohen, 1988) and in terms of probability of superiority (Lakens, 2013).

Preliminary Analyses
First, we tested whether negative emotions (e.g., loneliness, jeal ousy) and anger played a special role in the presidential election in 2016.To this end, we calculated partial correlations between Twitter language parameters and election results controlling for various county characteristics (i.e., percentages of female population, African American population, Hispanic population, married residents, education, median income, age-adjusted AHD mortality).As we tested 12 correlations, we applied a familywise Bonferroni-correction to the significance threshold to control the type 1 error, α = .05/12= .004.
Table 1 displays the partial correlations (zero-order correlations are presented in the additional files in Table S1).Results show that the expression of negative emotions and anger were positively and significantly correlated with vote choice for Donald Trump, while being not significantly related to conservative vote choice in 2012.According to convention both effects were small in size (Cohen, 1988), however, post-hoc power analyses with GPower (Faul, Erdfelder, Lang, & Buchner, 2007) confirmed that the dataset was sufficiently powered to detect the positive correlation between anger and vote choice for Trump, 1-β = .99(r = .15,N = 1,347, α = .003,two-tailed).Anxiety and positive emotions were both not significantly related to election results, neither in 2012 nor in 2016, speaking for the particular role of negative emotions and anger.
Mirroring the results for emotion language parameters, results further show that language signaling cognitive engagement (e.g., interest, alertness) is negatively correlated with vote choice for Donald Trump, whereas cognitive disengagement (e.g., tiredness, boredom) is positively related to conservative vote choice in both presidential elections.These results provide initial evidence that Donald Trump (but not his forerunner Mitt Romney) was more successful in counties where residents tweeted more negative emotions and anger, speaking for the particular role of these emotions in the 2016 election.

Predicting Election Results From Twitter Language and County Characteristics
Next, we aimed to test whether Twitter language indicators predict the election results in 2016.To this end, we applied a multiple linear regression with 10-fold cross-validation Note: *Significant at p < .004,N = 1,347.rs = Partial correlations controlling for county characteristics (i.e., percentages of female population, African American population, Hispanic population, married residents, education, median income, age-adjusted AHD mortality).(Picard & Cook, 1984).Cross-validation means that the data is randomly split into ten sets (folds).Repeatedly, nine sets are used to train a regression model.The prediction performance of the model is then evaluated by comparing the predicted with the observed outcome in the tenth (holdout) set.This process is repeated ten times such that each set serves as holdout set once.The final model parameters and the prediction accuracy are averaged over the ten repetitions.This approach is superior to an ordinary multiple regression without cross-validation, because it makes an actual prediction of data that is not used to train the model (Picard & Cook, 1984).At the same time, cross-validation procedures circumvent overfitting a model to one specific set of data, which increases the likelihood that the model also accurately predicts new data.
For testing the significance of the 14 regression weights associated with the predictors, we again applied a familywise Bonferroni correction to the significance threshold, α = .05/14= .004.Results of the cross-validated regression model are summarized in Table 2. First, the model performed extremely well in predicting vote choice for Donald Trump with on average 93% of variance explained in the validation samples (M R² = 0.927, SD R² = 0.008, M RMSE = 0.041, SD RMSE = 0.002).The main contribution to the prediction was counties' conservative vote choice in 2012, which positively predicted vote choice for Donald Trump in 2016, reflecting the stability in the U.S. presidential elections (see also Fig. S1 in the supporting information).With regard to county characteristics, results show that Donald Trump received significantly less votes in counties with higher female, African American, and Hispanic population, as well as in counties with higher level of education and higher median income replicating previous findings (Brenan, 2018;Rothwell, 2016).Also in counties with higher levels of AHD mortality Trump received relatively more votes.
Despite having controlled for previous election results and various county characteristics, four out of six Twitter language parameters significantly predicted county-level vote choice for Donald Trump.In line with our hypothesis, Donald Trump received relatively more votes in counties  where people twittered more anger and negative emotions.Anxiety and positive emotions on the other hand did not predict vote choice.These results suggest that Trump attracted voters in counties where people were more likely to express anger and negative emotions, but not in counties where voters tweeted words signaling anxiety or positive emotions.
Results further show that in counties where people used more language signaling cognitive engagement, Donald Trump received fewer votes under consideration of the results of his forerunner.However, he also received fewer votes in counties where people used language signaling cognitive disengagement.This result might be considered as surprising given that the results of the partial correlations in Table 1 show that cognitive disengagement was overall positively related to conservative voting behavior in both presidential elections.Thus, this relationship should be considered with caution given that that the sign of the effect reverses between partial correlation and the multiple regression model.
Next, we examined whether Twitter language parameters should be included in the most parsimonious model predicting the election results.Therefore, we applied an elastic net regression to our set of 14 predictors and used 10-fold cross-validation to derive at the model's regularization parameters (λ = 0.00028, α = 0.55).Results showed that the most parsimonious model to predict the results of the election did not include anxiety, positive emotions, and cognitive disengagement.Among the county characteristics the percentage of female population was dropped as predictor.The fact that anger, negative emotions, and cognitive engagement were kept as predictors indicates that they independently contribute to the prediction of the election results despite of their high intercorrelation (see Table S1 in the additional files).
As a last step, we ran a 10-fold cross-validation regression model including only anger, negative emotions, and cognitive engagement to estimate how much of the variance in the election results of 2016 can be explained by these three predictors alone.Results showed that a model trained on these Twitter language parameters alone on average predicted 8% of the variance in the election results (M R² = 0.079, SD R² = 0.028, M RMSE = 0.147, SD RMSE = 0.003), which is according to Cohen (1988) a medium-sized effect.

Comparing Emotional Expression in Presidential Campaigns
Next, we tested whether the expression of anger and negative emotions was particularly characteristic for Trump's online campaign and compared emotional expression in his pre-election tweets with those of Hillary Clinton.Descriptive statistics and effect sizes are summarized in Table 3. Tweets of Donald Trump did not contain more anger than those of Hillary Clinton and only contained slightly more negative emotions.According to convention the effect size for negative emotions was small and relates to a 53% chance that a random tweet of Donald Trump had more negative emotion words than a random tweet of Hillary Clinton (Cohen, 1988;Lakens, 2013).Unexpectedly, Trump's tweets also contained more posi-tive emotions with 61% probability of superiority. 1 As expected, tweets did not differ in terms of anxiety.For language signaling cognitive (dis-)engagement, we did not entertain a specific hypothesis and found no difference regarding these language variables in tweets.
Last, we compared Donald Trump's campaign speeches to those of Hillary Clinton, Mitt Romney, and Barack Obama.For the sake of readability, we will interpret the averaged effect size comparing Trump's campaign to the other campaigns of the other presidential candidates in 2016 and 2012 (d speech in Table 3).As expected, Donald Trump expressed consistently more anger and negative emotions in his campaign speeches than Hillary Clinton, Mitt Romney, and Barack Obama.Both averaged effect sizes were large in size with 81% and 69% probability of superiority.With regard to anxiety there was a difference with Trump expressing more anxiety-related words in his speeches (70% probability of superiority); however, this difference was mainly driven by Obama's campaign speeches in 2012, which had very low instances of anxiety-related words (d speech reduces to 0.43 and 62% probability of superiority when not considering Obama's campaign speeches).Further, there was a small difference in speeches with regard to positive emotion words with 62% probability of superiority for Trump's campaign speeches (reduced to 0.21 and 55% when not considering Obama's campaign speeches).
We also explored whether speeches differed with regard to language signaling cognitive (dis-)engagement.However, there was only a small effect size with regard to Trump using fewer words signaling cognitive engagement (56% probability of superiority).Again, the campaign speeches of Barack Obama stood out in this regard, because Hillary Clinton and Mitt Romney both used considerably more language signaling cognitive engagement than Donald Trump (d speech increases to -0.32 and 59% probability of superiority when not considering Obama's campaign speeches).However, Donald Trump used consistently more words signaling cognitive disengagement with a 68% probability of superiority.
In sum, these results suggest that the expression of anger and negative emotions was characteristic for Trump's campaign speeches and supports the idea that emotional resonance may explain why the expression of these emotions on social media predicted vote choice for Donald Trump on the county level.

Discussion
Previous research already provided some compelling explanations for Trump's triumph in the presidential election in 2016, such as status threat among the privileged, economic anxiety, racism or sexism (e.g., Brenan, 2018;Mutz, 2018;Rothwell, 2016;Schaffner et al., 2018).The present research tested the role of negative emotionsand particularly anger-as a new possible explanation for his success.Using a large data set of tweets and the actual outcome of the election at the county level, our findings showed that Trump received more relative votes in counties where residents tweeted more anger and negative emotions on social media.Election results were unre-   lated to the expression of anxiety and positive emotions.
Our findings therefore corroborate the widely discussed idea that particularly the angry voters supported Trump.Importantly, the effects of negative emotions and anger were significant when controlling for various county characteristics that reflect previously established explanations for Trump's success (e.g., counties' economic situation, level of education of the population, proportion of female and minority population).Speaking for the validity of our data we replicate this previous work, showing that Trump received relatively less votes in counties with worse economic situation, lower average education, and higher proportion of minorities (i.e., African American, Hispanic population).Further, by controlling for these known predictors of the election results we show that the consideration negative emotions and anger opens a new perspective on the question of who voted for Trump.Finally, these variables can be ruled out as possible third variables that would explain the relationship between anger expression on social media and Trump's success.
It is also important to note that we controlled for counties' election results in the 2012 presidential elections (and also the results of 2008 in additional analyses).The finding that emotional language variables still significantly contribut to the prediction of results in 2016 suggests that counties where more anger and negative emotions are expressed on social media, are not more likely to vote more conservatively per se.Rather their conservative vote choice increased from 2012 to 2016, when Trump was running for presidency.This finding supports the exceptional role of emotions in this particular election.
The exploratory finding that Trump was less successful in counties with higher levels of cognitive engagement also speaks for the prominent role of emotions.Counties in which language on Twitter suggested that their inhabitants were interested in the world around them preferred Hillary Clinton over Donald Trump (who in her campaign also used more language signaling cognitive engagement).The effect of cognitive engagement was significant despite having controlled for the 2012 results.This again suggests that cognitive engagement does not predict less conservative vote choice per se, but rather it predicted a decrease of counties' conservative vote choice between 2012 and 2016.Because cognitive disengagement was dropped as predictor in the selection analyses and the sign of the effect changed between partial correlations and the regression model, we refrain from interpreting the results for this predictor.

Theoretical Implications
The county-level prediction of election results by emotional language on social media is in itself interesting and sheds light on this particular election.Of more general interest, however, are the possible underlying mechanisms that may have led to these relationships.One explanation for why Trump was more successful in attracting voters in counties in which people tweeted more anger and negative emotions offers the concept of emotional resonance.Previous research suggests that people are generally more attentive and receptive to information that matches their current emotions (Blaney, 1986;Bower, 1981;Niedenthal & Setterlund, 1994;Parrott & Sabini, 1990).Further, research suggests that political appeals are especially effective when they resonated emotionally with people's emotional state (Roseman et al., 1986).Following this argument, U.S. citizens who were more inclined to experience high levels of anger should have been particularly attracted by a campaign that matched their predominant emotional state.As our findings suggest, this was the case for Trump's campaign: He expressed more negative emotions and anger in his campaign, particularly his speeches, than his direct opponent Hillary Clinton and his forerunners Mitt Romney and Barack Obama in 2012.There were only small differences in campaigns with regard to anxiety and positive emotions, especially compared to Hillary Clinton as direct opponent and Mitt Romney as forerunner.In other words, a campaign that was exceptional in the expression of anger and negative emotions was more successful in regions where people experience and express these emotions more often.Thus, our findings support the emotional resonance hypothesis for anger and negative emotions and suggest that political campaigns are more successful if they resonate with voters' predominant emotional state.Going beyond previous research the present study examined actual voting behavior and analyzed emotions spontaneously expressed by U.S. citizens on social media and by presidential candidates in their campaign speeches and on social media.
However, the present findings have to be regarded as an indirect test of emotional resonance, because this phenomenon has so far been documented on the level of the individual and for emotions induced shortly before the evaluation of political messages (Roseman et al., 1986).Our results involve correlations on a higher level of aggregation (county level) and mean level differences on a lower level of aggregation (level of speeches/tweets).It is possible that relationships look different on different levels of aggregation (Hoffman & Stawski, 2009), and it would be false to conclude that relationships observed on the level of the individual necessarily hold on the group level (Freedman, 1999;Na et al., 2010).Despite this fact, it is possible that individual-level processes account for findings on a higher-level of aggregation.In our case, the individual-level process has already been documented in previous research.Thus, we are not inferring individuallevel processes based on our data but rather tested the predictions of a theoretical concept on different levels of aggregation.Of course, emotional resonance provides only one possible explanation for our findings.However, we did our best to rule out alternative explanations by controlling for plausible third variables in our model, such as socioeconomic status.
The long time span covered by our data comes with the advantage that previous elections can be controlled for, which makes a stronger case for the particular role of negative emotions in the 2016 election.Two out of four emotional language variables predicted vote choice in the 2016 election, while none of them predicted vote choice in 2012.This result is particularly surprising given that Twitter data was sampled in 2010 and therefore closer to the 2012 election.If anything, one would have expected that Twitter data should be a stronger predictor in the 2012 election.Further, the long interval between measurement of emotions and election outcomes suggests that effects sizes are likely to be underestimated due to increases in error variance (e.g., people change or move to other counties).
An open question that cannot be addressed by the present study is what caused the stable differences in anger and negative emotions that predicted election outcomes.Based on previous research, showing regional differences in traits across the U.S. (Rentfrow, 2010), we can only speculate that these emotions might reflect regional differences in trait anger and negative affectivity mainly driven by local culture.Another possible explanation would be that anger and negative emotions reflected stable regional differences in (political) discontent or life dissatisfaction present already in 2010.Research suggests that life satisfaction is relatively stable over longer period of times and differs not only between individuals but also between U.S. counties (Eid & Diener, 2004;Fujita & Diener, 2005;Lucas & Donnellan, 2007;Schwartz et al., 2013).As argued by Eichstaedt et al. (2015), "local communities create physical and social environments that influence the behaviors, stress experiences, and health of their residents (Diez Roux & Mair, 2009;Lochner, Kawachia, Brennan, & Bukac, 2003)" (p.166).A local like, for instance, the dismissal of a significant amount of employees by an enterprise might elicit anger or other negative emotions.Such events might continuously affect people's emotions, because of family members' longer lasting unemployment or because of empty buildings reminding of the event.The fact that emotions predicted the election outcome over and above socioeconomic status and demographic variables does not necessarily contradict the idea that the part of the stable variance in anger that predicted Trump votes was an expression of stable life dissatisfaction or political discontent.For instance, perceived or expected threats (e.g., loss of privileges relative to outgroups) or stress might cause negative emotions independent of the objective developments of people's actual life circumstances.As already mentioned, the data of the present study can unfortunately not speak to the question what causes regional differences in the expression of anger and negative emotions on social media.However, we and others could show that they might be useful to predict important county-level outcomes (Eichstaedt et al., 2015).

Strength and Limitations
The present study used a sample of 148 million tweets posted by U.S. Americans located across 1,347 different counties, objectively coded with regard to their emotional content, to predict the actual outcomes of the presidential election at the county level.This approach entails several strengths such as the behavioral nature of all of our data and the quasi-prospective study design of the study.Both address shortcomings of previous studies which used selfreports and a cross-sectional design (Finn & Glaser, 2010;Roseman et al., 1986).Further, in combination with the analytic approach the longitudinal nature of the data allows the claim that we predicted election results in three ways.First, we used a machine learning approach to crossvalidated the results of our multiple regression model, which means that the results of the analyses reflect an actual prediction of data that was not used to train the model.Second, we used Twitter data from 2010 to predict outcomes of the presidential elections six years later and, third, we controlled for previous election results to predicted change in counties' election results from 2012 to 2016.
Our study also has several limitations that we would like to discuss.First, our data is purely correlational which comes with two caveats: First, there might be third variables that can account for the effects of emotions.To address this issue we controlled for a number of plausible third variables that may be associated with the experience of anger or negative emotions and with its expression on social media (e.g., education).Previous studies have already linked some of these demographic and socioeconomic factors to the election results in 2016 (Brenan, 2018;Mutz, 2018;Rothwell, 2016;Schaffner et al., 2018) and U.S. Republican primaries (Stoetzer, Gerlich, & Koesters, 2017), which supports the validity of our data and speaks for the specific role of emotions over and above these variables.A second possible caveat of correlational data is multicollinearity.To address this problem, we additionally applied machine learning techniques that allow selecting a set of predictors based on their independent contribution to the prediction of an outcome.In these analyses, both negative emotions and anger were selected as predictors, suggesting that despite their intercorrelation, they independently explain variance in the election results.
Last, although the sample of tweets was randomly selected among the population of tweets available, it may still not be representative of the general population of the U.S. (Eichstaedt et al., 2015).Research shows that the Twitter population tends to live in urban areas and to have above-average levels of education (Mislove, Lehmann, Ahn, Onnela, & Rosenquist, 2011).Results may therefore overly reflect relationships for the urban and well-educated part of the population and, therefore, may not generalize to the U.S. population as a whole.

Conclusion
Using the unique possibilities of social media to capture spontaneously expressed emotions of citizens, the present research sheds some light on the role of emotions in the 2016 US presidential election.Results showed that counties' relative vote choice for Donald Trump can be significantly predicted by the level of anger and negative emotions expressed by their residents on social media.Further, language analyses of campaign speeches show that Trump expressed more anger and negative emotions in his presidential campaign than any other candidate in the 2012 and 2016 election.These two results are interesting independently as they offer a new perspective on who voted for Donald Trump and underline the important role of negative emotions in this particular election.Further, the phenomenon of emotional resonance offers one possible account explaining how these two findings could be related by suggesting that political messages are more successful if they match with voters predominant emotions (Roseman et al., 1986).Although, the present results cannot be regarded as direct test of this individual-level process, emotional resonance offers one possible explanation for the present findings and may as well play a role in future elections.Future studies should therefore aim to replicate emotional resonance effects on different levels of aggregation and aim to better understand the mechanisms and boundary conditions involved on the level of the group and the level of the individual.

Data Accessibility Statements
Data can be accessed on the Open Science Framework (OSF) via the following link: https://osf.io/ukn39/.

Note
1 We removed the words 'great again' in all speeches and tweets (here also 'greatagain' because of hashtags) to account for differences in positive emotions words that might appear due to Trump's campaign slogan.

Additional Files
The additional files for this article can be found as follows: • Fig S1 Note: *Significant at p < .004(df = 1,328).b cv = Regression coefficients of 10-fold cross-validated multiple regression model.ben = Regression coefficients of 10-fold cross-validated multiple regression model using the elastic net regression (λ = 0.00028, α = 0.55).

Table 1 :
Partial Correlations of Language Parameters and County-Level Results for Presidential Elections 2012 and 2016.

Table 2 :
Linear Regression Model Predicting County-Level Results for Donald Trump in the U.S. Presidential Election 2016.

Table 3 :
Descriptive Statistics and Cohen's ds for the Comparison of Language Variables in Pre-Election Tweets and Campaign Speeches Between Donald Trump and other Presidential Candidates in 2012 and 2016.
Averaged effect size for the comparisons between campaign speeches of Donald Trump with campaign speeches of Hillary Clinton, Mitt Romney, and Barack Obama.