V-Forecast: Predicting Adverse Regime Transitions (PART)

The PART project uses data from the Varieties of Democracy (V-Dem) Institute and other sources to estimate the risk of adverse regime transitions during the next two years (2019-2020).

We define an adverse regime transition as a year-to-year decrease in the Regimes of the World (RoW) index, which classifies political regimes as either a closed autocracy, electoral autocracy, electoral democracy, or liberal democracy. The PART project estimates the risk of moving down this scale within a two-year window.

For more details, please see our working paper. We welcome feedback.

Click on a country in the map, the plot, or select a country from the list below to view its estimated risk and select regime characteristics.

Country Characteristics:

Interpretation: The complexity of the machine learning models we use makes it difficult to identify which specific variables are driving predictions. Therefore, the time-series plot is for descriptive purposes only. It is meant to help identify trends, not causal relationships, across key V-Dem indices. For access to country-level time series across all V-Dem variables, please use V-Dem's Country Graph tool here
Blue bars indicate that an ART occurred within 2-year forecast window
Purple bars indicate that no ART occurred within 2-year forecast window
Gray bars indicate the estimated risk for current 2-year forecast window

Methodology (See working paper for more details)

It is important to note that these forecasts are probabilistic: a high estimated risk does not mean that an ART will occur with certainty; similarly, a low estimated risk does not mean that there is zero chance an ART will not occur.

Further, given that this is the first year of this project, we do not yet know how accurately this project can forecast future transitions in the real world as opposed to statistical simulations and out-of-sample tests. However, we are posting these forecasts in the interest of transparency.

Yearly Risk Estimates - We derive risk estimates for 2011-2017 using a simulation framework that mimics the process we use to produce our 2019-2020 forecasts. In short, we first train our models using all data from 1970 to 2009. We then use data from 2010 to produce estimated risk forecasts for 2011-12. We then retrain our models using all data from 1970 to 2010, use data from 2011 to produce estimates for 2012-12. We conduct this iterative model check procedure for all years, 2011 to 2017.

Data - To produce our estimated risk forecasts, we use V-Dem data version 9 along with UN GDP and population data, ethnic power relations data (Vogt et al. 2015), coup event data (Powell and Thyne 2011), and UCDP armed conflict data (Pettersson and Eck 2018), over 400 variables altogether. We lag all variables one year and derive the first differences for a number of variables. All of variables we use in our models are lagged one year. Thus, data for 2016 is used to estimate the risk of ARTs for 2017/18.

Unit of analysis - We use country-years as our unit of analysis and limit our temporal frame to 1970-2018. We reconcile the differences between the V-Dem country-year set and the Gleditsch and Ward country-year set. This leaves 169 countries for our 2019-2020 forecasts. The number of countries in our data per year ranges from 135 to 169. Our training and validation country-year set covers 7,754 country-year observations.

Effective positive rate - The yearly rate of adverse regime transitions of our dependent variable in any given year ranges from around 1.4 percent to 10 percent; 75 percent of our yearly positive rates are between 3.8 percent and 5.6 percent.

Models - We use three machine learning models: logit with elastic-net regularization, random forest, and gradient boosted forest. To help account for differences across these models, we use an unweighted model average ensemble. This is our preferred approach, as it helps smooth out our predicted risk estimates while improving accuracy. The estimates shown above are from this ensemble model.

Model Assessment - Our unweighted model average ensemble model reports an AUC-PR score of 0.46 in a 2x7 repeated cross-validation procedure (1970-2017) and an AUC-PR score of 0.39 in our set of yearly test forecasts (2011-2017). As a general benchmark of performance, an AUC-PR score that is higher than the observed frequency of events in the data is a signal that the model is an improvement over chance. With an observed frequency of ARTs at roughly 4 percent, our unweighted model average ensemble exceeds performance expectations.