Reproducing tables and results from Külpmann and Kuzmics (2022)

This vignette shows how to reproduce every table, graph and test from Külpmann and Kuzmics (2020).

Set up

The data as well as predictions for each theory mentioned in Külpmann and Kuzmics (2020) are included as additional data to the package “oottest”. Thus, you only need to install the package. Furthermore, we use the package xtables (Dahl et al. (2009)) to convert the R tables to latex tables which we use in the paper.

library("oottest")
library("xtable")

Accessing the data and predictions

Data and predictions can be accessed directly after installing the oottest package. Access them as:

  • data_two_action_games
  • data_three_action_games
  • predictions_two_action_games
  • predictions_three_action_games

Please see the documentation for each dataset to learn about formatting, usage and the source. To access the documentation for e.g., the data from the two action games, type:

?data_two_action_games

Tables and figures from the main paper

Vuong matrix (table 1 and 2)

Table 1 and 2 consist of 6 Vuong matrices, one for each of the different types of games:

  • Two-action games
    • Hawk-dove games
    • Matching pennies: asymmetric own-payoffs
    • Matching pennies: symmetric own-payoffs
  • Three-action games
    • Hawk-middle-dove games
    • Rock-Scissors-Paper: asymmetric own-payoffs
    • Rock-Scissors-Paper: symmetric own-payoffs

The following commands creates a Vuong matrix for each of them and saves them as vuong_table_GAME_NAME. (The ranges below are to select the treatments.)

vuong_table_hdg <- vuong_matrix(data_two_action_games[,1:10], predictions_two_action_games[,1:10,])
vuong_table_mp1 <- vuong_matrix(data_two_action_games[,11:15], predictions_two_action_games[,11:15,])
vuong_table_mp2 <- vuong_matrix(data_two_action_games[,16:20], predictions_two_action_games[,16:20,])
vuong_table_ac <- vuong_matrix(data_three_action_games[,1:10], predictions_three_action_games[,1:10,])
vuong_table_rsp1 <- vuong_matrix(data_three_action_games[,11:15], predictions_three_action_games[,11:15,])
vuong_table_rsp2 <- vuong_matrix(data_three_action_games[,16:20], predictions_three_action_games[,16:20,])

Now, we have six Vuong matrix, each looking similar to this:

Hawk-Dove Games
CH CH-RA LK NE NE-RA NI NI-RA QCH QCH-RA QLK QLK-RA QRE QRE-RA RND
CH NaN -1.77 3.05 7.98 -1.21 -0.10 1.40 0.82 -1.91 1.04 2.72 -0.92 0.25 3.77
CH-RA 1.77 NaN 5.61 6.36 0.47 2.40 5.07 3.36 -1.34 2.12 5.56 1.63 3.99 7.11
LK -3.05 -5.61 NaN 4.31 -3.71 -6.13 -5.03 -6.89 -5.63 -1.76 1.40 -4.35 -6.80 6.49
NE -7.98 -6.36 -4.31 NaN -5.86 -6.47 -4.67 -5.76 -6.27 -9.30 -2.72 -7.73 -5.19 -3.49
NE-RA 1.21 -0.47 3.71 5.86 NaN 1.43 3.17 2.05 -1.54 1.57 5.76 1.00 2.26 4.98
NI 0.10 -2.40 6.13 6.47 -1.43 NaN 2.81 4.32 -2.55 0.97 2.84 -1.27 0.59 6.81
NI-RA -1.40 -5.07 5.03 4.67 -3.17 -2.81 NaN -2.11 -5.41 -0.62 2.17 -2.30 -8.93 12.38
QCH -0.82 -3.36 6.89 5.76 -2.05 -4.32 2.11 NaN -3.46 0.07 2.50 -2.34 -0.81 7.62
QCH-RA 1.91 1.34 5.63 6.27 1.54 2.55 5.41 3.46 NaN 2.21 5.69 1.80 4.54 7.23
QLK -1.04 -2.12 1.76 9.30 -1.57 -0.97 0.62 -0.07 -2.21 NaN 2.37 -2.42 -0.33 2.57
QLK-RA -2.72 -5.56 -1.40 2.72 -5.76 -2.84 -2.17 -2.50 -5.69 -2.37 NaN -3.01 -2.92 -0.74
QRE 0.92 -1.63 4.35 7.73 -1.00 1.27 2.30 2.34 -1.80 2.42 3.01 NaN 0.86 5.05
QRE-RA -0.25 -3.99 6.80 5.19 -2.26 -0.59 8.93 0.81 -4.54 0.33 2.92 -0.86 NaN 11.20
RND -3.77 -7.11 -6.49 3.49 -4.98 -6.81 -12.38 -7.62 -7.23 -2.57 0.74 -5.05 -11.20 NaN

In the paper, we colored the matrices to allow for an easier overview. To create the latex used in the paper, one only has to run the function “color_vuong_table” on each table. It outputs latex code which can directly me used as a table in the document.

Vuong matrix: all games (table 3)

To create the Vuong matrix using the two and three-action games together, we can not use the vuong_matrix command directly, the structure of two- and three-action games are different.

This is done here. The (hide) output is latex code of a colored Vuong matrix as above.

num_theories <- dim(predictions_two_action_games)[3]
result <- matrix(, nrow = num_theories, ncol = num_theories)
for (i in 1:num_theories) {
  for (j in 1:num_theories) {
    llr_2 <- oottest:::get_llr(data_two_action_games, predictions_two_action_games[,,i], predictions_two_action_games[,,j])
    llr_3 <- oottest:::get_llr(data_three_action_games, predictions_three_action_games[,,i], predictions_three_action_games[,,j])
    variance_2 <- oottest:::get_variance_of_llr(data_two_action_games, predictions_two_action_games[,,i], predictions_two_action_games[,,j])
    variance_3 <- oottest:::get_variance_of_llr(data_three_action_games, predictions_three_action_games[,,i], predictions_three_action_games[,,j])
    result[i, j] <- (llr_2 + llr_3) / (variance_2 + variance_3)^(.5)
  }
}
colnames(result) <- colnames(predictions_two_action_games[1,,])
rownames(result) <- colnames(predictions_two_action_games[1,,])
oottest:::color_vuong_table(result)

Calculating likelihoods

The third and final graph in the paper is a comparison of log-likelihood differences for four different theories.

The graph itself is created in latex using Tikz (Tantau (2013)) which uses the data from R as inputs.

The data was created as follows:

Creating likelihood tables.

llh_table_2 <- oottest:::create_likelihood_table(data_two_action_games, predictions_two_action_games)
llh_table_3 <- oottest:::create_likelihood_table(data_three_action_games, predictions_three_action_games)
llh_table_all <- cbind(llh_table_2,llh_table_3)

Output:

Loglikelihoods: two action games
T01 T02 T03 T04 T05 T06 T07 T08 T09 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20
CH -101.89 -97.48 -91.01 -91.01 -77.43 -101.89 -102.01 -94.90 -103.95 -118.19 -101.89 -93.60 -84.55 -84.55 -77.43 -101.89 -98.71 -99.40 -97.46 -97.60
CH-RA -101.89 -97.65 -93.40 -91.01 -77.43 -94.25 -109.43 -96.24 -101.17 -99.41 -101.89 -93.60 -84.55 -84.55 -77.43 -101.89 -98.71 -99.40 -97.46 -97.60
LK -101.89 -99.94 -98.73 -98.73 -96.21 -104.98 -100.78 -99.45 -101.14 -103.78 -101.89 -99.21 -97.53 -97.53 -96.21 -101.89 -99.09 -99.69 -98.01 -98.13
NE -101.89 -97.73 -91.73 -99.23 -69.27 -101.89 -110.40 -123.37 -146.11 -166.82 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -92.87 -100.52 -89.57 -106.11
NE-RA -101.89 -98.03 -92.55 -90.83 -70.38 -101.90 -110.66 -97.06 -101.42 -99.40 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -95.97 -96.29 -86.72 -86.05
NI -101.89 -100.53 -97.85 -94.82 -76.96 -101.89 -100.53 -95.08 -101.81 -107.84 -101.89 -100.01 -96.08 -91.24 -75.17 -101.89 -101.85 -101.83 -101.69 -101.44
NI-RA -101.89 -100.80 -99.02 -97.42 -88.86 -98.96 -102.75 -100.64 -101.79 -101.15 -101.89 -100.35 -97.72 -95.26 -87.92 -101.89 -101.82 -101.79 -101.61 -101.43
QCH -101.89 -100.80 -98.64 -96.11 -82.15 -101.89 -100.75 -95.85 -101.14 -106.19 -101.89 -100.39 -97.25 -93.33 -80.91 -101.89 -101.86 -101.85 -101.74 -101.58
QCH-RA -101.89 -98.01 -92.92 -91.32 -72.85 -94.57 -109.52 -97.02 -101.40 -99.40 -101.89 -93.74 -82.24 -81.27 -70.25 -101.89 -96.58 -97.56 -93.41 -93.68
QLK -101.89 -99.02 -93.73 -90.67 -66.42 -101.89 -100.16 -99.10 -111.82 -121.28 -101.89 -97.75 -89.70 -83.40 -66.79 -101.89 -101.53 -101.32 -100.14 -99.14
QLK-RA -101.89 -103.76 -93.73 -93.65 -72.68 -106.42 -147.95 -98.76 -102.55 -112.66 -101.89 -93.61 -82.47 -82.41 -73.46 -101.89 -92.88 -96.83 -85.68 -86.48
QRE -101.89 -99.50 -95.28 -92.19 -72.14 -101.89 -100.10 -95.11 -104.33 -113.18 -101.89 -97.96 -90.24 -83.63 -65.59 -101.89 -100.98 -100.52 -97.58 -94.63
QRE-RA -101.89 -100.18 -97.52 -95.53 -84.32 -96.90 -103.56 -99.89 -101.72 -100.66 -101.89 -99.08 -94.42 -90.76 -79.92 -101.89 -100.45 -100.03 -96.87 -94.60
RND -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89 -101.89
Loglikelihoods: three action games
T21 T22 T23 T24 T25 T26 T27 T28 T29 T30 T31 T32 T33 T34 T35 T36 T37 T38 T39 T40
CH -182.37 -179.56 -177.11 -166.42 -174.78 -183.77 -197.40 -198.12 -192.14 -186.98 -182.37 -156.82 -152.52 -153.38 -136.17 -182.37 -187.23 -187.63 -185.82 -185.22
CH-RA -204.33 -179.40 -177.11 -166.85 -172.51 -152.52 -169.29 -167.37 -168.42 -152.52 -182.37 -156.82 -152.52 -153.38 -136.17 -182.37 -187.23 -187.63 -185.82 -185.22
LK -182.37 -244.57 -283.98 -264.06 -285.90 -173.16 -213.36 -207.06 -195.69 -173.16 -182.37 -236.09 -195.60 -212.36 -170.34 -182.37 -229.99 -228.78 -242.32 -254.24
NE -182.37 -188.15 -185.69 -176.20 -185.60 -181.44 -209.76 -235.91 -215.17 -186.47 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -174.00 -175.08 -196.50 -227.22
NE-RA -183.26 -183.70 -179.93 -174.06 -176.19 -143.14 -168.71 -171.20 -167.27 -140.68 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -176.47 -174.45 -178.24 -186.37
NI -182.37 -181.73 -179.76 -175.20 -175.70 -182.96 -184.99 -188.34 -185.26 -183.64 -182.37 -179.88 -176.95 -172.20 -152.28 -182.37 -182.38 -182.40 -182.41 -182.44
NI-RA -182.48 -182.20 -181.10 -179.24 -179.34 -177.08 -180.28 -180.86 -180.28 -176.61 -182.37 -180.35 -178.52 -176.27 -168.13 -182.37 -182.40 -182.42 -182.43 -182.45
QCH -182.37 -182.08 -181.20 -179.23 -179.21 -182.59 -183.29 -184.40 -183.30 -182.83 -182.37 -181.40 -180.22 -178.17 -169.25 -182.37 -182.37 -182.37 -182.38 -182.38
QCH-RA -182.41 -182.29 -181.83 -181.08 -181.09 -180.25 -181.53 -181.77 -181.54 -180.07 -182.37 -181.59 -180.86 -179.93 -176.60 -182.37 -182.37 -182.38 -182.38 -182.38
QLK -182.37 -195.47 -194.83 -171.94 -186.71 -227.55 -246.48 -252.28 -240.16 -259.68 -182.37 -156.11 -154.25 -157.96 -119.50 -182.37 -193.60 -197.52 -197.48 -203.35
QLK-RA -196.37 -189.94 -184.78 -168.04 -176.90 -152.69 -182.03 -168.60 -181.41 -153.03 -182.37 -155.13 -151.29 -156.93 -119.49 -182.37 -195.69 -200.08 -199.45 -201.30
QRE -177.81 -193.79 -257.43 -368.06 -457.30 -149.34 -225.60 -273.00 -283.93 -202.21 -182.37 -182.30 -182.58 -182.51 -183.17 -182.37 -185.30 -189.30 -193.90 -216.65
QRE-RA -178.11 -187.08 -210.73 -243.32 -261.23 -148.06 -176.48 -177.83 -183.69 -148.48 -182.37 -181.95 -182.12 -181.62 -180.89 -182.37 -184.48 -186.54 -187.21 -190.50
RND -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37 -182.37

Creating a csv file with the log-likelihood difference to the baseline (CH-RA) for each of the four theories: NA-RA,CH-RA (normalized to 1), QLK-RA, QCH-RA

theories <- c("NE-RA", "CH-RA", "QLK-RA", "QCH-RA")
baseline <- llh_table_all["CH-RA", ]
output <- c()
for (theory in theories) {
  lh_diff <- round(llh_table_all[theory,] - baseline, 12)
  treatment <- 1:40
  output <- cbind(treatment, lh_diff)
  write.csv(output, file= paste0(theory, ".csv"), row.names = FALSE, quote = FALSE)
}

Recreating tables from the Online Appendix

To recreate the tables from the Online Appendix use:

vuong_table_2 <- vuong_matrix(data_two_action_games, predictions_two_action_games)
vuong_table_3 <- vuong_matrix(data_three_action_games, predictions_three_action_games)

Log-likelihoods

Creating tables from the log-likelihoods from above (requires xtable):

xtable(llh_table_2, type = "latex")
xtable(llh_table_3, type = "latex")

Testing theories individually

The following code gives us the results of the tests for each theory, if it generates predictions different from the observed data:

chi_sq_hdg <- oottest:::get_all_chi_sq(data_two_action_games[,1:10], predictions_two_action_games[,1:10,])
chi_sq_mp1 <- oottest:::get_all_chi_sq(data_two_action_games[,11:15], predictions_two_action_games[,11:15,])
chi_sq_mp2 <- oottest:::get_all_chi_sq(data_two_action_games[,16:20], predictions_two_action_games[,16:20,])
chi_sq_2 <- oottest:::get_all_chi_sq(data_two_action_games, predictions_two_action_games)
chi_sq_ac <- oottest:::get_all_chi_sq(data_three_action_games[,1:10], predictions_three_action_games[,1:10,])
chi_sq_rsp1 <- oottest:::get_all_chi_sq(data_three_action_games[,11:15], predictions_three_action_games[,11:15,])
chi_sq_rsp2 <- oottest:::get_all_chi_sq(data_three_action_games[,16:20], predictions_three_action_games[,16:20,])
chi_sq_3 <- oottest:::get_all_chi_sq(data_three_action_games, predictions_three_action_games)

Here the results for the hawk-dove games:

Chi-squared test for the hawk-dove games
chi-sq p-value
CH 90.65 0e+00
CH-RA 52.48 1e-07
LK 134.97 0e+00
NE 482.47 0e+00
NE-RA 61.20 0e+00
NI 86.14 0e+00
NI-RA 111.10 0e+00
QCH 97.02 0e+00
QCH-RA 47.70 7e-07
QLK 109.56 0e+00
QLK-RA 237.13 0e+00
QRE 81.78 0e+00
QRE-RA 90.01 0e+00
RND 161.05 0e+00

Now, to create the latex tables as seen in the paper. These can be just copied into the tex document.

table_2 <- cbind(chi_sq_hdg, chi_sq_mp1, chi_sq_mp2, chi_sq_2)
table_3 <- cbind(chi_sq_ac, chi_sq_rsp1, chi_sq_rsp2, chi_sq_3)
xtable(table_2, type = "latex", digits=c(0,2,8,2,8,2,8,2,8))
xtable(table_3, type = "latex", digits=c(0,2,8,2,8,2,8,2,8))

Data and theory predictions

Predictions

Creating tex code of a table of all predictions:

pred_2 <- round(predictions_two_action_games, 3)
th_names <- names(predictions_two_action_games[1,1,])
output <- c()
for (theory in th_names) {
  output <- rbind(output, paste0("(", pred_2[1,,theory], ", ", pred_2[2,,theory], ")"))
}
colnames(output) <- names(predictions_two_action_games[1,,1])
rownames(output) <- th_names
xtable(output, type = "latex")

pred_3 <- round(predictions_three_action_games, 3)
output <- c()
for (theory in th_names) {
  output <- rbind(output, paste0("(", pred_3[1,,theory], ", ", pred_3[2,,theory], ", ", pred_3[3,,theory],  ")"))
}
colnames(output) <- names(predictions_two_action_games[1,,1])
rownames(output) <- th_names
xtable(output, type = "latex")

Data

Creating tex code to display the summary of the choice data of the experiments:

xtable(data_two_action_games, type = "latex", digits=0)
xtable(data_three_action_games, type = "latex", digits=0)

References

Dahl, David B, David Scott, Charles Roosen, Arni Magnusson, and Jonathan Swinton. 2009. Xtable: Export Tables to LaTeX or HTML.
Külpmann, Philipp, and Christoph Kuzmics. 2020. “Comparing Theories of One-Shot Play Out of Treatment.” https://dx.doi.org/10.2139/ssrn.3441675.
Tantau, Till. 2013. The TikZ and PGF Packages: Manual for Version 3.1.0. https://sourceforge.net/projects/pgf/.