batchtools.slurm.tmpl
session-info
test-cross-platform.R
Code was run on two platforms:
1. A 2020 Macbook Air (M1) running R 4.0.5, used for the California Prop99 example and aggregation and plotting of the results from the simulations
2. A cluster of x86_64 machines running R 4.0.2, used to run the simulations themselves 
To run code using the cluster:
i.   The notebook paper-simulations.Rmd was converted to a script by calling knitr::purl('paper-simulations.Rmd').
ii.  This script, and the slurm template file batchtools.slurm.tmpl, were put in the same folder on the cluster and the script is run.
iii. The resulting 'simulations/*.rds' files were then copied from the cluster to the vignettes directory and the notebook was run to generate plots and tables
Simply running the notebook on a computer like platform 1 should yield essentially the same results, but take a few days.

While we did not run all the simulations on platform 1, we ran some and checked them against what we got on platform 2 by running test-cross-platform.R. Results were essentially identical, with differences on the order of 1e-13 or smaller, except for the MC estimator: the MCPanel library uses the C++ std::default_random_engine RNG to choose folds for cross-validation, which does not guarantee identical results on different platforms. Among the 5500 simulations on which we compared results for the MC estimator on our two platforms, the absolute difference of point estimates had the following quantiles.
          0%          10%          20%          30%          40%          50%          60%          70%          80%          90%         100% 
0.000000e+00 3.505241e-07 6.974247e-06 1.939810e-05 4.741519e-05 9.357476e-05 1.737794e-04 3.053189e-04 6.039954e-04 1.520949e-03 5.482976e-02 
 
Below we give the output of sessionInfo() on these two platforms after running the code in paper-results.Rmd.

We installed glmnet (an MCPanel dependency), MCPanel, and synthdid from source as follows:

devtools::install_github('cran/glmnet',              ref='f4fc95ab49efaad9b6e1728a7c840bc6159501dc')
devtools::install_github('susanathey/MCPanel',       ref='6b2706fd7c35f3266048ceb22a7e9a61ae1774da')
devtools::install_github('synth-inference/synthdid', ref='2408a924e4dba02c116e935ded7ee1093de24b23')

We installed the other packages from CRAN as follows:

install.packages(c('doFuture', 'future.batchtools', 'rngtools'))
install.packages(c('dplyr', 'tidyr', 'tibble', 'ggplot2'))

======================================== Platform 1 ===============================

> sessionInfo()
R version 4.0.5 (2021-03-31)
Platform: aarch64-apple-darwin20.3.0 (64-bit)
Running under: macOS Big Sur 11.2.3

Matrix products: default
BLAS:   /opt/homebrew/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib
LAPACK: /opt/homebrew/Cellar/r/4.0.5/lib/R/lib/libRlapack.dylib

Random number generation:
 RNG:     L'Ecuyer-CMRG 
 Normal:  Inversion 
 Sample:  Rejection 
 
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] future.batchtools_0.10.0 doFuture_0.12.0          foreach_1.5.1           
[4] future_1.21.0            rngtools_1.5             MCPanel_0.0             
[7] synthdid_0.0.9          

loaded via a namespace (and not attached):
 [1] progress_1.2.2    shape_1.4.5       tidyselect_1.1.1  purrr_0.3.4      
 [5] listenv_0.8.0     splines_4.0.5     lattice_0.20-41   latex2exp_0.5.0  
 [9] colorspace_2.0-0  vctrs_0.3.8       generics_0.1.0    utf8_1.2.1       
[13] survival_3.2-10   rlang_0.4.10      pillar_1.6.0      withr_2.4.2      
[17] glue_1.4.2        rappdirs_0.3.3    lifecycle_1.0.0   stringr_1.4.0    
[21] munsell_0.5.0     gtable_0.3.0      mvtnorm_1.1-2     codetools_0.2-18 
