Rituals of Reasons

README for Replication

Authors
Affiliations

Elias Bouacida

University Paris 8

Renaud Foucart

Lancaster University

Published

October 7, 2024

Overview

This replication packages contains the data and the code for the paper Rituals of Reason: A Choice-Based Approach to the Acceptability of Lotteries in Allocation Problems (previously circulated as The Acceptability of Lotteries in Allocation Problems: A Choice-Based Approach). It uses Julia and R. Two main files run all the code to generate the data for the 3 figures and 14 tables in the paper. The replicator should expect the code to run for approximately 10 minutes.

Data Availability and Provenance Statements

The data has been collected during two experiment we ran online, on Amazon Mechanical Turk during the summer of 2021 and on Prolific in August 2023. The material related to the experiment is available in the Experiment folder. A README in the root of the folder explains in more details the different files.

Statement about Rights

License for Data

The data are licensed under a Creative Commons/CC-BY-NC license. See LICENSE for details.

Summary of Availability

Note: Data that cannot be made available are identifiers on Prolific and Amazon Mechanical Turk. These identifiers have been removed to preserve the anonymity of subjects.

Details on each Data Source

All the data is provided in .csv. Description of all the variables in all the data is provided in the file README_data.md in the Data folder.

Data.Name Data.Files Location Provided Citation
Pilot all_apps_wide_2020-04-30.csv; all_apps_wide_2020-05-14.csv FALSE
Pilot TimeSpent (accessed 2020-04-30).csv; Pilot1RawData.csv; Pilot1Comments.csv; TimeSpent (accessed 2020-05-14).csv; Pilot2RawData.csv; Pilot2Comments.csv Data/OriginalData/ TRUE Not used in the paper.
Experiment 1 all_apps_wide_2021-06-22.csv; all_apps_wide_2021-07-07.csv; all_apps_wide_2021-08-31.csv; FALSE
Experiment 1 Experiment1_ARawData.csv; Experiment1_BRawData.csv; Experiment1_CRawData.csv; Experiment1_CRawData.csv; Experiment1_AComments.csv; Experiment1_BComments.csv; Experiment1_CComments.csv; Feedback_2021-06-22.csv; Feedback_2021-07-07.csv; Feedback_2021-08-31.csv; PageTimes_2021-06-22.csv; PageTimes_2021-07-07.csv; PageTimes_2021-08-31.csv Data/OriginalData/ TRUE
Experiment 2 all_apps_wide_2023-08-15.csv; prolific_export_64c7dd8a74bb548c50b4e381.csv FALSE
Experiment 2 Experiment2RawData.csv; PageTimes_2023-08-15.csv Data/OriginalData/ TRUE
RPS Rock_Paper_Scissors_Raw_data.csv Data/OriginalData/ TRUE

Public use data collected by the authors

The data used to support the findings of this study have been deposited in the Paris 8 Data verse and Pure repository (DOI:10.57745/NZTMN9). The data were collected by the authors, and are available under a Creative Commons Non-commercial license.

Public use data sourced from elsewhere and provided

Data on Rock, Paper, Scissors in the field provided by Lasse Hassig, from Roshambo.me. Data can be asked directly from Roshambo.me. A copy of the data is provided as part of this archive.

Datafile: Rock_Paper_Scissors_Raw_data.csv

Preliminary code during the editorial process

Code for data cleaning and anonymization is provided as part of the replication package. It is available at https://gitlab.huma-num.fr/ebouacida/rituals-of-reasons-reproduction for review. It will be uploaded to the [JOURNAL REPOSITORY] once the paper has been conditionally accepted.

Dataset list

Data file Source Notes Provided
all_apps_wide_2020-04-30.csv Pilot Raw export from Pilot 1, contains mturk ID No
all_apps_wide_2020-05-14.csv Pilot Raw export from Pilot 2, contains mturk ID No
Pilot1RawData.csv Pilot Anonymized data from Pilot 1 Yes
Pilot2RawData.csv Pilot Anonymized data from Pilot 2 Yes
Pilot1Comments.csv Pilot Encoded comments from Pilot 1 Yes
Pilot2Comments.csv Pilot Encoded comments from Pilot 2 Yes
TimeSpent (accessed 2020-04-30).csv Pilot Time data data from Pilot 1. Yes
TimeSpent (accessed 2020-05-14).csv Pilot Time data data from Pilot 2. Yes
all_apps_wide_2021-06-22.csv Experiment 1 Raw export from Experiment 1, session 1, contains mturk ID No
all_apps_wide_2021-07-07.csv Experiment 1 Raw export from Experiment 1, session 2, contains mturk ID No
all_apps_wide_2021-08-31.csv Experiment 1 Raw export from Experiment 1, session 3, contains mturk ID No
Experiment1_ARawData.csv Experiment 1 Anonymized data from Experiment 1, session 1. Yes
Experiment1_BRawData.csv Experiment 1 Anonymized data from Experiment 1, session 2. Yes
Experiment1_CRawData.csv Experiment 1 Anonymized data from experiment 1, session 3. Yes
Experiment1_AComments.csv Experiment 1 Encoded comments from Experiment 1, session 1. Yes
Experiment1_BComments.csv Experiment 1 Encoded comments from Experiment 1, session 2. Yes
Experiment1_CComments.csv Experiment 1 Encoded comments from Experiment 1, session 3. Yes
Feedback_2021-06-22.csv Experiment 1 Custom data export from Experiment 1, session 1. Yes
Feedback_2021-07-07.csv Experiment 1 Custom data export from Experiment 1, session 2. Yes
Feedback_2021-08-31.csv Experiment 1 Custom data export from Experiment 1, session 3. Yes
PageTimes_2021-06-22.csv Experiment 1 Time data from Experiment 1, session 1. Yes
PageTimes_2021-07-07.csv Experiment 1 Time data from Experiment 1, session 2. Yes
PageTimes_2021-08-31.csv Experiment 1 Time data from Experiment 1, session 3. Yes
Experiment2RawData.csv Experiment 2 Data from Experiment 2. Yes
PageTimes_2023-08-15.csv Experiment 2 Time data from Experiment 2. Yes
prolific_export_64c7dd8a74bb548c50b4e381.csv Experiment 2 Demographic data for Experiment 2, contains also Prolific ID. No
Rock_Paper_Scissors_Raw_data.csv Roshambo Answers from players on the website <www.roshambo.me>. We thank Lasse Hassing for giving us access to the data. Yes
Experiment1_ACleanedData.csv Experiment 1 Cleaned data from Experiment 1, session 1, computed using the Experiment1_ACleaningData.jl. Yes
Experiment1_BCleanedData.csv Experiment 1 Cleaned data from Experiment 1, session 2, computed using the Experiment1_BCleaningData.jl Yes
Experiment1_CCleanedData.csv Experiment 1 Cleaned data from Experiment 1, session 3, computed using the Experiment1_CCleaningData.jl Yes
Experiment1CleanedData.csv Experiment 1 Cleaned data from Experiment 1 (merging of the three files above.). Yes
Experiment2CleanedData.csv Experiment 2 Cleaned data from Experiment 2, computed using the Experiment2CleaningData.jl yes

Computational requirements

Any laptop from 2020 onwards should be able to reproduce the results. The software and libraries used and their version are available in the renv.lock lock file for R libraries, and the MANIFEST.toml and PROJECT.toml for the Julia libraries.

Software Requirements

Following the .gitlab-ci.yaml steps will work. A rough very suboptimal way to do it is to follow the setup-environment.sh script at the root of this repository.

  • quarto 1.5.57
  • R 4.4.1
    • dplyr (1.1.4)
    • gt (0.11.0)
    • kableExtra (1.4.0)
    • knitr (1.48)
    • modelsummary (2.2.0)
    • readr (2.1.5)
    • renv (1.0.9)
    • rmarkdown (2.28)
    • scales (1.3.0)
    • testit (0.13)
    • tidyverse (2.0.0)
    • viridis (0.6.5)
    • The replicator should first install the package renv and then use the command renv::restore() to restore all packages from the lock file (while setting the working directory at the root tree of this replication package).
  • Julia 1.10.5
    • CSV (0.10.14)
    • DataFrames (1.7.0)
    • Dates (included in Julia install)
    • HypothesisTests (0.11.3)
    • IJulia (1.25.0)
    • Pipe (1.3.0)
    • PrettyTables (2.4.0)
    • Random (included in Julia install)
    • Statistics (1.10.0)
    • StatsBase (0.34.3)
    • The replicator should install Julia then run the command using Pkg; Pkg.activate("Code"); Pkg.instantiate() in the REPL.
  • Python 3.9 or 3.12
    • oTree 5.10 or 5.11
    • pycryptodome for Experiment 2

Controlled Randomness

Memory, Runtime, Storage Requirements

Summary

Approximate time needed to reproduce the analyses on a standard (2024) desktop machine:

Approximate storage space needed:

Details

The code was last run on three different machines:

  1. A Windows 11 Laptop with an 8-core 16-threads AMD Ryzen 5800HS machine with 16GB of RAM
  2. A Linux laptop with Ubuntu 24.04 with an 16-cores 32 thread Intel Core-11800H with 32GB of RAM
  3. A docker image provided by rocker:verse online.

Description of programs/code

  • Programs in Code/Experiment1Anonymization.jl and Code/Experiment2Anonymization.R were used to anonymize the data in Experiments 1 and 2 respectively. It should not be used for the reproduction of the results, as the original non-anonymized data is not available for diffusion.
  • Programs in Code/Experiment[1_X|2]Payments.jl (X takes value A, B or C) were used to pay subjects. It should not be used for the reproduction of the results, as the original non-anonymized data is not available for diffusion. Programs in Code/Setup.jl creates the includes that are common to all these codes. Programs in Code/Constants.jl regroups all the variables that are constant in these analyses. Programs in Code\PaymentsFunctions.jl and Code\CommonFunctions.jl create some functions that are common to multiple analyses.
  • Programs in Code/PAP_report.qmd will generate the pre-registered analysis report, with all the figures, tables and comments.
  • Programs in Code/PaperOutput.qmd will generate all the results of the paper.
  • Programs in Code/RoshamboData.qmd will generate the Roshambo comparisons.
  • Programs in my_email.py was used to send participants in Experiment 2 their results through their Prolific email.
  • Programs in the folder Code\Payments\ were used to send the payments to subjects, grand them “qualitifications” (an AMT characteristic) and create the webpages to show them their results.

License for Code

The code is licensed under a GPL v.3.0 license. See LICENSE for details.

Instructions to Replicators

  • Clone the repository, for instance using the command git clone git@gitlab.huma-num.fr:ebouacida/rituals-of-reasons-reproduction.git.
  • Install R and the R package renv, and then run the command renv::restore() in the console at the root of the cloned Git repository, which should be used as R working directory. It should restore all the libraries at the version which we used for the analysis.
  • Install Julia and run the command using Pkg; Pkg.activate("Code"); Pkg.instantiate() in the Julia REPL (assuming your working directory is the root folder) (or julia -e using Pkg; Pkg.activate("Code"); Pkg.instantiate() in the terminal in the root folder.)
  • Install Python and Jupyter (tested on Python 3.12)
  • Run the command quarto render at the root of the folder. It should do a bunch of things, in particular, run the following scripts:
    • Run Code/PAP_report.qmd to generate the pre-registered analysis report.
    • Run Code/PaperOutput.qmd to generate all the figures and tables from the paper.
    • Run Code/RoshamboData.qmd to generate the Rock, Paper, Scissors comparison to the general public.

Details

  • setup-environment.sh Create a script file that installs all the software and libraries needed for the replication. It runs in particular the following commands:
    • renv::restore(): will download and install all R libraries needed for replication.
      • If wishing to update the ado packages used by this archive, run command renv::update() in the console. However, this is not needed to successfully reproduce the manuscript tables and may actually broke reproduction.
    • using Pkg; Pkg.activate("Code"); Pkg.instantiate() will download and install the libraries needed for replication.
  • Code/PAP_report.qmd.
    • Self-contained run of the pre-registered analysis. It was last successfully run in October 2024.
  • Code/PaperOutput.qmd.
    • Output all figures and tables used in the paper. The HTML file produce may contain some numbers that appear in the paper. Tables and figures are numbered as in the paper. Sections in the analysis refer to corresponding sections in the paper. Programs were last run successfully in October 2024.
  • Code/RoshamboData.qmd
    • Output the figures related to the Rock, Paper, Scissors comparisons to the Roshambo data set. Programs were last run successfully in October 2024.

List of tables and programs

The provided code reproduces:

Figure/Table # Program Line Number (code chunk) Output file Note
Table 1 PaperOutput.qmd 178-203 (treatment) TableTreatments.tex
Table 2 PaperOutput.qmd 357-404 (tbl-regression-exp1) TableRegressionExp1.tex
Table 3 PaperOutput.qmd 569-599 (performance_correlation) TableCorrelation.tex
Table 4 PaperOutput.qmd 759-778 (conditional_choices) TableBeliefs.tex
Table 5 PaperOutput.qmd 927-942 (tbl-control-exp1) TableControl.tex
Table 6 PaperOutput.qmd 978-1006 (tbl-regression-exp2) TableRegressionExp2.tex
Table 7 PaperOutput.qmd 1016-1048 (performance-correlation-detailed) TableCorrelationDetailed.tex
Table 8 PaperOutput.qmd 1070-1087 (tbl-wta) TablePreferences.tex
Table 9 PaperOutput.qmd 1129-1149 (gender) TableGender.tex
Table 10 PaperOutput.qmd 1162-1180 (age) TableAge.tex
Table 11 PaperOutput.qmd 1192-1203 (ethnicity) TableEthnicity.tex
Table 12 PaperOutput.qmd 1216-1236 (country) TableCountry.tex
Table 13 PaperOutput.qmd 1345-1362 (tbl-sequences) TableLotterySequenceTotal.tex
Table 14 PaperOutput.qmd 1466-1481 (tbl-sequence2) TableLotterySequence.tex
Figure 1 PaperOutput.qmd 261-275 (graph_choice_amt) fig-exp1.pdf
Figure 2 PaperOutput.qmd 465-475 (graph_choice_prol) fig-both.pdf
Figure 3 PaperOutput.qmd 1100-1113 (fig-wta) fig-WTA.pdf

Acknowledgements

We thank Lancaster University for funding support for this experiment. We thank Lasse Hassig for providing us with the Roshambo.me data on Rock, Paper, Scissors. This README file was built based on the Template provided by the Social Science Data Editors https://social-science-data-editors.github.io/

References