Publications

System-, teacher-, and student-level interventions for improving participation in online learning at scale in high schools


​Abstract: Many school systems across the globe turned to online education during the COVID-19 pandemic. This context differs significantly from the prepandemic situation in which massive open online courses attracted large numbers of voluntary learners who struggled with completion. Students who are provided online courses by their high schools also have their behavior determined by actions of their teachers and school system. We conducted experiments to improve participation in online learning before, during, and right after the COVID-19 outbreak, with 1,151 schools covering more than 45,000 students in their final years of high school in Ecuador. These experiments tested light-touch interventions at scale, motivated by behavioral science, and were carried out at three levels: that of the system, teacher, and student. We find the largest impacts come from intervening at the system level. A cheap, online learning management system for centralized monitoring increased participation by 0.21 SD and subject knowledge by 0.13 SD relative to decentralized management. Centralized management is particularly effective for underperforming schools. Teacher-level nudges in the form of benchmarking emails, encouragement messages, and administrative reminders did not improve student participation. There was no significant impact of encouragement messages to students, or in having them plan and team-up with peers. Small financial incentives in the form of lottery prizes for finishing lessons did increase study time, but was less cost-effective, and had no significant impact on knowledge. The results show the difficulty in incentivizing online learning at scale, and a key role for central monitoring.
​
​[Download PDF]

​Resumen: ​Muchos sistemas educativos en el mundo recurrieron a la educación en línea durante la pandemia de COVID-19. Este contexto difiere significativamente de la situación prepandemia, donde los cursos en línea masivos atrajeron a un gran número de estudiantes voluntarios, quienes tenían problemas para completarlos. Los estudiantes a quienes se les provee de cursos en línea por parte de escuelas secundarias también tienen un comportamiento determinado por las acciones de sus profesores y el sistema escolar. Realizamos experimentos para mejorar la participación en aprendizaje en línea antes, durante e inmediatamente después del brote de COVID-19, con 1.151 instituciones, abarcando más de 45.000 estudiantes en los últimos años de secundaria o bachillerato en Ecuador. Estos experimentos testearon intervenciones ligeras a escala motivados por la ciencia del comportamiento y, fueron llevados a cabo en tres niveles: sistema, profesores y, estudiantes. Encontramos que los grandes efectos vienen de la intervención a un nivel de sistema. Un sistema barato de gestión del aprendizaje en línea para la monitorización centralizada incrementa la participación en 0.21 DE y conocimiento de las asignaturas en 0.13 DE relativo a la gestión descentralizada. La gestión descentralizada es particularmente efectiva para las instituciones educativas de bajo rendimiento. Al nivel de profesores, los incentivos en forma de correos electrónicos de evaluación comparativa, mensajes motivacionales y recordatorios administrativos no mejoraron la participación de los estudiantes. No hubo efectos significativos en los mensajes motivacionales para los estudiantes o la planificación y trabajo en grupo con compañeros. Pequeños incentivos económicos en forma de premios de lotería por la finalización de lecciones incrementaron el tiempo de estudio, pero fue menos costo-efectivo y no fue significativo en el impacto en los conocimientos. Los resultados mostraron la dificultad para incentivar el aprendizaje en línea a escala y el importante rol de la gestión centralizada.
​
​​[Descargar pdf in inglés]
Igor Asanov, Anastasiya-Mariya Asanov (Noha), Thomas Åstebro, Guido Buenstorf, Bruno Crépon, David McKenzie, Francisco Pablo Flores T., Mona Mensmann & Mathis Schulte (2023): "System-, Teacher-, and Student-level Interventions for Improving Participation in Online Learning at Scale in High Schools",  Proceedings of the National Academy of Science (PNAS), 2023 Vol. 120 No. 30 e2216686120; https://doi.org/10.1073/pnas.221668612

Bandit cascade: A test of observational learning in the bandit problem​

Abstract: I conduct an experimental investigation of observational (social) learning in a simple two- armed bandit framework where the models are based on Bayesian reasoning and non- Bayesian count heuristics providing different predictions. The agents can choose between two alternatives with different probabilities of providing a reward. They must make their choice in order to see the outcome and act in a sequence. They can base their decision on the choices of the predecessors and the outcomes of their own choice. The results of the experiment follow neither Bayesian Nash Equilibrium nor Naïve herding model (BRTNI): Subjects follow and cascade on choices that contain no information about the state of the world, and, therefore, sustain losses when learning from others. I also test the Quantal response equilibrium and the robustness of this theory.

​[Download PDF]

Igor Asanov, Bandit cascade: A test of observational learning in the bandit problem, Journal of Economic Behavior & Organization, Volume 189, 2021,
Pages 150-171, ISSN 0167-2681, https://doi.org/10.1016/j.jebo.2021.06.006.

Remote-learning, Time-Use, and Mental Health of Ecuadorian High-School Students during the COVID-19 Quarantine
with Francisco Flores  (University of Kassel) , David McKenzie (World Bank), Mona Mensmann (Warwick Business School), Mathis Schulte (HEC Paris)

Aprendizaje remoto, Uso de Tiempo, y Salud Mental de Estudiantes Ecuatorianos de secundaria durante la Cuarentena COVID-19
con Francisco Flores  (University of Kassel) , David McKenzie (World Bank), Mona Mensmann (Warwick Business School), Mathis Schulte (HEC Paris)
​Abstract: ​The COVID-19 pandemic has closed schools around the world, forcing school systems and students to quickly attempt remote learning. A rapid response phone survey of over 1,500 high school students aged 14 to 18 in Ecuador was conducted to learn how students spend their time during the period of quarantine, examine their access to remote learning, and measure their mental health status. The data show that 59 percent of students have both an internet connection at home and a computer or tablet, 74 percent are engaging in some online or telelearning, and 86 percent have done some schoolwork on the last weekday. Detailed time-use data show most students have established similar daily routines around education, although gender and wealth differences emerge in time spent working and on household tasks. Closure of schools and social isolation are the two main problems students say they face, and while the majority are mostly happy, 16 percent have mental health scores that indicate depression.

​
​
[Download working paper pdf in English]
Resumen: La pandemia de COVID-19 ha cerrado escuelas en todo el mundo, obligando a los sistemas escolares y a los estudiantes a intentar rápidamente el aprendizaje remoto. Realizamos una encuesta telefónica de respuesta rápida a más de 1,500 estudiantes de secundaria de 14 a 18 años de edad en Ecuador para conocer cómo pasan su tiempo durante el período de cuarentena, examinar su acceso al aprendizaje remoto y medir su estado de salud mental. Encontramos que el 59 por ciento de los estudiantes tiene conexión a Internet en casa y un computador o tableta, el 74 por ciento está participando en algún curso o aprendizaje en línea, y el 86 por ciento ha realizado algunas tareas escolares el último día de la semana. El detalle de datos sobre el uso del tiempo muestra que la mayoría de los estudiantes han establecido rutinas diarias similares en torno a la educación, aunque el género y la inequidad económica influyen en el tiempo que se dedica al trabajo escolar y a las tareas del hogar. El cierre de las escuelas y el aislamiento social son los dos problemas principales que los estudiantes dicen enfrentar, y si bien la mayoría está contenta, el 16 por ciento tiene puntajes de salud mental que indican depresión.
​
[​Descargar pdf in Español]
Asanov, I., Flores, F., Mckenzie, D., Mensmann, M., & Schulte, M. (2021). Remote-learning, Time-Use, and Mental Health of Ecuadorian High-School Students during the COVID-19 Quarantine. World Development, 138, 105225. https://doi.org/10.1016/j.worlddev.2020.105225​
[LINK free access till 11 December 2020 on full article]
Some Media Coverage Related to the Paper Above: 
​"'​We need to think more about ways to make distance education truly inclusive [...] Getting this data is important to think about how to balance the quality of face-to-face and virtual education in order to improve the quality of life of children and young people for the future.' head of the Evidence-Based Science and Innovation Policy Research Group, University of Kassel, Germany." translated from SciDev.net
​
[Click here for the article in Spanish  on scidev.net that cites the paper above] 
"'​Necesitamos pensar más en formas de hacer la educación a distancia realmente inclusiva […] Obtener estos datos es importante para pensar cómo balancear la calidad de la enseñanza presencial con la virtual y así poder mejorar la calidad de vida los niños y jóvenes para el futuro.' Igor Asanov, jefe del Grupo de Política Científica y de Innovación Basada en la Evidencia, Universidad de Kassel, Alemania."  SciDev.net 
​
[Haga clic aquí para ver el artículo en scidev.net que cita el documento anterior]. ​

Can Group Identity Explain Gender Gap in Recruitment Process? ​
with Maria Mavlikeeva (University of Kassel)

Abstract: ​Despite evidence that the gender gap in the labour market favours men, aggregate findings from correspondence studies show that women are more likely than men to be invited for a job interview. We hypothesize that the predominance of women among recruiters may explain this somewhat puzzling finding; recruiters may favour applicants of their own gender. We use the data from a large-scale correspondence study to test this hypothesis. As expected, we find that female applicants are more likely to receive callbacks for interview. We also see that in our sample the majority of contact persons responsible for the recruitment process are female. More importantly, we find that if recruiter and applicant are of the same gender, then the likelihood that the applicant will be invited for an interview increases. These findings reveal the gender favouritism at the selection stage in the labour market.

[Download PDF]

Asanov, I., & Mavlikeeva, M. (2023). Can group identity explain the gender gap in the recruitment process? Industrial Relations Journal, 54, 95–113. https://doi.org/10.1111/irj.12392

Short- and Long-run Effects of External Interventions on Trust. 
with Simone Vannuccini  (FSU-Jena) 

Abstract: We experimentally analyze the effects of external intervention such as subsidy and targeting on the investment decision during the intervention and after. We employ a multi-period version of the trust (investment) game (Berg et al., 1995) introducing either a monetary incentives for contribution or providing a suggestion about the level of investment. The results of the experiment indicate  that targeting is an effective instrument to promote trustful behavior, whereas subsidy policy is not effective both in short- and long-run.  Therefore  we suggest considering a targeting policy as one of the instruments that can foster trustful behavior. 
​
 [Download PDF]

Asanov, I. and Vannuccini, S., 2020. Short-and Long-run Effects of External Interventions on Trust. Review of Behavioral Economics, 7(2), pp.159-195.

Reporting errors and biases in published empirical findings: Evidence from innovation research
with Stephan Bruns (Hasselt University) et al.

Abstract: Errors and biases in published results compromise the reliability of empirical research, posing threats to the cumulative research process and to evidence-based decision making. We provide evidence on reporting errors and biases in innovation research. We find that 45% of the articles in our sample contain at least one result for which the provided statistical information is not consistent with reported significance levels. In 25% of the articles, at least one strong reporting error is diagnosed where a statistically non-significant finding becomes significant or vice versa using the common significance threshold of 0.1. The error rate at the test level is very small with 4.0% exhibiting any error and 1.4% showing strong errors. We also find systematically more marginally significant findings compared to marginally non-significant findings at the 0.05 and 0.1 thresholds of statistical significance. These discontinuities indicate the presence of reporting biases. Explorative analysis suggests that discontinuities are related to authors’ affiliations and to a lesser extent the article’s rank in the issue and the style of reporting.

​[Download PDF]

Bruns, S. B., Asanov, I., Bode, R., Dunger, M., et al. (2019). Reporting Errors and Biases in Published Empirical Findings: Evidence from Innovation Research. Research Policy, 48(9):103796.

Working Papers

Mental Health and Stress Level of Ukrainians Seeking Psychological Help Online 
with Anastasiya-Mariya Asanov Noha , Guido Buenstorf  (University of Kassel) 

We screen the mental well-being and psychological distress of 1165 refugees, migrants, internally displaced, and non-displaced people from Ukraine who seek psychological help online in Ukraine and across 24 countries of the European Union. We see that more than half of the respondents exhibit low levels of mental well-being and high psychological distress, with 81% being at risk of depression and 57% having severe psychological distress. Refugees and internally displaced people in our sample show a particularly high risk of depression and severe psychological distress. Nevertheless, the majority of Ukrainians seeking psychological help still work, study, or volunteer, and those who do have relatively alleviated mental levels of well-being. 
[Download PDF]

​Folktale Narratives and Economic Behavior
with Dominik P. Heinisch, Nhat Luong  (University of Kassel) 

​Narratives – stories – prevail in social life, however little is known about their relation to economic behavior and outcomes. We shed light on this issue by studying the association between folktales’ motifs and economic behavior across the world. First, we explore possible relations between the prevalence of narratives and the observed behavior in economic experiments by connecting the Berezkin (2015) collection of narratives with individual choices in experiments performed across the world. Second, we construct a motif distance index that approximates the cultural distance between countries. Third, we provide evidence that motif distance is associated with economic performance.
[Download PDF]

Is Self-employment a Career Trap? Large-Scale Field Experiment in the Labor market
with Maria Mavlikeeva (University of Kassel)

Abstract: ​We provide a large-scale correspondence experiment by sending more than 12 000 fictitious resumes in Russia to assess treatment of self-employed at the labor market.  Our study is the first correspondence experiment in Russia (Bertrand and Duflo, 2017).  Thus, to ensure validity of the experiment in the new context, we, also, assess discrimination against ethnic minorities. We find that the level of ethnic discrimination in Russia is comparable to the world average (Zschirnt and Ruedin, 2016). More importantly, we find discrimination against self-employed. However, we observe discrimination against self-employed at the low-skill market but not on the high-skill level market.  Those results go in lines with statistical form of discrimination of self-employed. 
 
This study is registered in the AEA RCT Registry and the unique identifying number is AEARCTR-0001308.

Confidence, Confidence Training and Entrepreneurial Behavior: Field Experiment.​
with Alla Levitskaia (Comrat State University)

Abstract: ​We provide a field experiment to study the effect of  an entrepreneurial program in Gagauzia region, Moldova. We randomly assign applicants for the entrepreneurship training to three educational tracks: (1) Business and confidence training; (2) Confidence training; (3) No training (control group). To understand underlying behavioral mechanisms of the intervention we elicit in the lab-in-field experiment participants'  behavioral characteristics. We find that confidence and ambiguity preference, but not risk preferences or self-efficacy,  associated with entrepreneurial behavior. We also find that a combination of business and confidence training affect an entrepreneurial related decision.​
[Download PDF]

The power of experiments: How big is your n?
with Christoph Bühren (Clausthal University of Technology),  Panagiota Zacharodimou (European Parliament)

Abstract: ​The replicability and credibility crisis in psychology and economics sparked the debate on underpowered experiments, publication biases, and p-hacking. Analyzing the number of independent observations of experiments published in Experimental Economics, Games and Economic Behavior, and the Journal of Economic Behavior and Organization, we observe that we did not learn much from this debate. The median experiment in our sample has too few independent observations and, thus, is underpowered. Moreover, we find indications for biases in reporting highly significant results. We investigate for which papers and experiments it is more likely to find reporting biases, and we suggest remedies that could help to overcome the replicability crisis.
[Download PDF]

 On the observational learning in the optimal stopping problem.
with Arno Riedl  (Maastricht University)

Abstract: We experimentally test the theoretical predictions about observational learning in case of the optimal stopping problem with publicly observable behavior but privately observable payoffs. The paper focuses on the two-agents two-arms bandit framework and follows the theoretical analysis of Rosenberg et al. (2007).  Rational behavior in the framework implies solving a non-trivial optimization task. That raises the question whether people act according to the theoretical predictions. We find evidence that subjects overreact on thier own outcomes and comply with the choices of others.
[Download PDF]

Work in Progress

Showing Life Opportunities: Increasing Opportunity-Driven Entrepreneurship and STEM Careers Through Online Courses In Schools.​
with Thomas Åstebro (HEC Paris), Guido Buenstorf ( University of Kassel); Bruno Crepón (ENSAE and École Polytechnique);  Diego d'Andria (JRC European Commission), Francisco Flores (University of Kassel), David McKenzie (World Bank); Mona Mensmann  (Warwick Business School); Mathis Schulte (HEC Paris)

​Abstract: ​We  provide an RCT to assess the effect of online courses in schools on intentions, career-related actions, and career choices to become an opportunity driven entrepreneur or choose a STEM type of education. We plan to work with to 15-17-year-old students (about 17,000 students) in 110 schools. We will randomly assign schools either to treatment (and receiving the entrepreneurship and science content online), or placebo-control (receiving a placebo treatment of online courses from standard curricula) groups. Within the treatment group, we randomize at the grade-level the type of entrepreneurship curricula, and then randomize the order of entrepreneurship and science courses to measure the short-term effects of each component and to mitigate order effects. In addition, we cross-randomize schools to a role model treatment of interviews with successful scientists and entrepreneurs. In addition, we provide information about career options. We then plan to follow subjects until (at least) age 19, to measure both short- and medium-term outcomes. 

This study is registered in the AEA RCT Registry and the unique identifying number is AEARCTR-0003593

Ethnic Employment Gap during the Covid-19 Outbreak: Great Equalizer or Divider?
with Maria Mavlikeeva (University of Kassel)
Many employees lose their jobs as the result of Covid-19 outbreak (ILO, 19 March 2020). Moreover, the number of job openings decreases, making it harder for applicants to compete. This challenging situation can be particularly hard for minority groups. We aim to assess the level of ethnic discrimination under the condition of the slack labor market caused by the Covid-19 outbreak. We replicate a correspondence study, conducted in 2017, sending resumes of applicants with the native and ethnic minority names in response to job openings in different occupations. We plan to analyse the ethnic employment gap under the tension at the labor market and heterogeneity of treatment of different ethnic groups across occupations in this conditions.

This study is registered in the AEA RCT Registry https://doi.org/10.1257/rct.5388-2.0. This study is registered in EEA list of COVID-19 projects and Economics Observatory (ECO),

[Download presentation of preliminary results]   

​[Download accompanying paper "Assessing the ethnic employment gap during the early stages of COVID-19" by Maria Mavlikeeva in COVID Economics]