Publications

System-, teacher-, and student-level interventions for improving participation in online learning at scale in high schools

Abstract: Many school systems across the globe turned to online education during the COVID-19 pandemic. This context differs significantly from the prepandemic situation in which massive open online courses attracted large numbers of voluntary learners who struggled with completion. Students who are provided online courses by their high schools also have their behavior determined by actions of their teachers and school system. We conducted experiments to improve participation in online learning before, during, and right after the COVID-19 outbreak, with 1,151 schools covering more than 45,000 students in their final years of high school in Ecuador. These experiments tested light-touch interventions at scale, motivated by behavioral science, and were carried out at three levels: that of the system, teacher, and student. We find the largest impacts come from intervening at the system level. A cheap, online learning management system for centralized monitoring increased participation by 0.21 SD and subject knowledge by 0.13 SD relative to decentralized management. Centralized management is particularly effective for underperforming schools. Teacher-level nudges in the form of benchmarking emails, encouragement messages, and administrative reminders did not improve student participation. There was no significant impact of encouragement messages to students, or in having them plan and team-up with peers. Small financial incentives in the form of lottery prizes for finishing lessons did increase study time, but was less cost-effective, and had no significant impact on knowledge. The results show the difficulty in incentivizing online learning at scale, and a key role for central monitoring.
​
​[Download PDF]
​Resumen (Abstract in Spanish): ​Muchos sistemas educativos en el mundo recurrieron a la educación en línea durante la pandemia de COVID-19. Este contexto difiere significativamente de la situación prepandemia, donde los cursos en línea masivos atrajeron a un gran número de estudiantes voluntarios, quienes tenían problemas para completarlos. Los estudiantes a quienes se les provee de cursos en línea por parte de escuelas secundarias también tienen un comportamiento determinado por las acciones de sus profesores y el sistema escolar. Realizamos experimentos para mejorar la participación en aprendizaje en línea antes, durante e inmediatamente después del brote de COVID-19, con 1.151 instituciones, abarcando más de 45.000 estudiantes en los últimos años de secundaria o bachillerato en Ecuador. Estos experimentos testearon intervenciones ligeras a escala motivados por la ciencia del comportamiento y, fueron llevados a cabo en tres niveles: sistema, profesores y, estudiantes. Encontramos que los grandes efectos vienen de la intervención a un nivel de sistema. Un sistema barato de gestión del aprendizaje en línea para la monitorización centralizada incrementa la participación en 0.21 DE y conocimiento de las asignaturas en 0.13 DE relativo a la gestión descentralizada. La gestión descentralizada es particularmente efectiva para las instituciones educativas de bajo rendimiento. Al nivel de profesores, los incentivos en forma de correos electrónicos de evaluación comparativa, mensajes motivacionales y recordatorios administrativos no mejoraron la participación de los estudiantes. No hubo efectos significativos en los mensajes motivacionales para los estudiantes o la planificación y trabajo en grupo con compañeros. Pequeños incentivos económicos en forma de premios de lotería por la finalización de lecciones incrementaron el tiempo de estudio, pero fue menos costo-efectivo y no fue significativo en el impacto en los conocimientos. Los resultados mostraron la dificultad para incentivar el aprendizaje en línea a escala y el importante rol de la gestión centralizada.
​
​​[Descargar pdf in inglés]
Igor Asanov, Anastasiya-Mariya Asanov (Noha), Thomas Åstebro, Guido Buenstorf, Bruno Crépon, David McKenzie, Francisco Pablo Flores T., Mona Mensmann & Mathis Schulte (2023): "System-, Teacher-, and Student-level Interventions for Improving Participation in Online Learning at Scale in High Schools",  Proceedings of the National Academy of Science (PNAS), 2023 Vol. 120 No. 30 e2216686120; https://doi.org/10.1073/pnas.221668612

Bandit cascade: A test of observational learning in the bandit problem​

Abstract: I conduct an experimental investigation of observational (social) learning in a simple two- armed bandit framework where the models are based on Bayesian reasoning and non- Bayesian count heuristics providing different predictions. The agents can choose between two alternatives with different probabilities of providing a reward. They must make their choice in order to see the outcome and act in a sequence. They can base their decision on the choices of the predecessors and the outcomes of their own choice. The results of the experiment follow neither Bayesian Nash Equilibrium nor Naïve herding model (BRTNI): Subjects follow and cascade on choices that contain no information about the state of the world, and, therefore, sustain losses when learning from others. I also test the Quantal response equilibrium and the robustness of this theory.

​[Download PDF]

Igor Asanov, Bandit cascade: A test of observational learning in the bandit problem, Journal of Economic Behavior & Organization, Volume 189, 2021,
Pages 150-171, ISSN 0167-2681, https://doi.org/10.1016/j.jebo.2021.06.006.

Remote-learning, Time-Use, and Mental Health of Ecuadorian High-School Students during the COVID-19 Quarantine
with Francisco Flores  (University of Kassel) , David McKenzie (World Bank), Mona Mensmann (Warwick Business School), Mathis Schulte (HEC Paris)

​Abstract: ​The COVID-19 pandemic has closed schools around the world, forcing school systems and students to quickly attempt remote learning. A rapid response phone survey of over 1,500 high school students aged 14 to 18 in Ecuador was conducted to learn how students spend their time during the period of quarantine, examine their access to remote learning, and measure their mental health status. The data show that 59 percent of students have both an internet connection at home and a computer or tablet, 74 percent are engaging in some online or telelearning, and 86 percent have done some schoolwork on the last weekday. Detailed time-use data show most students have established similar daily routines around education, although gender and wealth differences emerge in time spent working and on household tasks. Closure of schools and social isolation are the two main problems students say they face, and while the majority are mostly happy, 16 percent have mental health scores that indicate depression.

​
​
[Download working paper pdf in English]
Resumen (Abstract in Spanish): La pandemia de COVID-19 ha cerrado escuelas en todo el mundo, obligando a los sistemas escolares y a los estudiantes a intentar rápidamente el aprendizaje remoto. Realizamos una encuesta telefónica de respuesta rápida a más de 1,500 estudiantes de secundaria de 14 a 18 años de edad en Ecuador para conocer cómo pasan su tiempo durante el período de cuarentena, examinar su acceso al aprendizaje remoto y medir su estado de salud mental. Encontramos que el 59 por ciento de los estudiantes tiene conexión a Internet en casa y un computador o tableta, el 74 por ciento está participando en algún curso o aprendizaje en línea, y el 86 por ciento ha realizado algunas tareas escolares el último día de la semana. El detalle de datos sobre el uso del tiempo muestra que la mayoría de los estudiantes han establecido rutinas diarias similares en torno a la educación, aunque el género y la inequidad económica influyen en el tiempo que se dedica al trabajo escolar y a las tareas del hogar. El cierre de las escuelas y el aislamiento social son los dos problemas principales que los estudiantes dicen enfrentar, y si bien la mayoría está contenta, el 16 por ciento tiene puntajes de salud mental que indican depresión.
​
[​Descargar pdf in Español]
Asanov, I., Flores, F., Mckenzie, D., Mensmann, M., & Schulte, M. (2021). Remote-learning, Time-Use, and Mental Health of Ecuadorian High-School Students during the COVID-19 Quarantine. World Development, 138, 105225. https://doi.org/10.1016/j.worlddev.2020.105225​
[LINK free access till 11 December 2020 on full article]
Some Media Coverage Related to the Paper Above: 
​"'​We need to think more about ways to make distance education truly inclusive [...] Getting this data is important to think about how to balance the quality of face-to-face and virtual education in order to improve the quality of life of children and young people for the future.' head of the Evidence-Based Science and Innovation Policy Research Group, University of Kassel, Germany." translated from SciDev.net
​
[Click here for the article in Spanish  on scidev.net that cites the paper above] 
"'​Necesitamos pensar más en formas de hacer la educación a distancia realmente inclusiva […] Obtener estos datos es importante para pensar cómo balancear la calidad de la enseñanza presencial con la virtual y así poder mejorar la calidad de vida los niños y jóvenes para el futuro.' Igor Asanov, jefe del Grupo de Política Científica y de Innovación Basada en la Evidencia, Universidad de Kassel, Alemania."  SciDev.net 
​
[Haga clic aquí para ver el artículo en scidev.net que cita el documento anterior]. ​

A meta-analysis of lost-letter field experiments
with  Helena Schirmacher  (University of Göttingen) and  Christoph Bühren  (Ruhr-University Bochum)

Abstract: The lost-letter technique ( Milgram et al., 1965 ) has been used for almost 60 years to measure social attitudes and helping behavior in psychological, sociological, and economic research. We provide a meta-analysis of lost-letter experiments to summarize the evidence. We analyze 78 studies with an overall sample size of 53,504 letters from 18 countries on five continents. We find an average return rate of 50 percent across all countries. Our meta-analysis shows that the return rate is lower for political or deviant issues. Stamped letters are also more likely to be returned, but letters with money are not more likely to be returned. A high socio-economic environment increases the chances of the return. We conclude that in line with the lost-letter paradigm, the technique allows capturing citizens’ attitudes toward the issue communicated. However, citizens do not act selflessly but react differently depending on the type of incentives.
​
 [Download Here]
Asanov, I., Schirmacher, H. and Bühren, C., 2024. A meta-analysis of lost-letter field experiments. Journal of Economic Psychology, 105, p.102759. https://doi.org/10.1016/j.joep.2024.102759

Mental Health and Stress Level of Ukrainians Seeking Psychological Help Online​
with Anastasiya-Mariya Asanov (University of Kassel),  Guido Buenstorf (University of Kassel)

Анотація (Abstract in Ukranian): ​Через російське вторгнення в Україну 24 лютого 2022 року понад 8 мільйонів українців були змушені покинути свої домівки. Українці, які зазнали впливу збройного конфлікту та міграції, скоріше за все, матимуть низький рівень психічного здоров'я та будуть звертатись по допомогу. Ми надаємо рівномірну кількісну оцінку стану психічного здоров'я українців, які звертаються за допомогою незабаром після вторгнення та переселення. Ми досліджуємо психічне благополуччя та психологічний дистрес 1165 біженців, мігрантів, внутрішньо переміщених та непереміщених осіб з України, які звернулися за психологічною допомогою онлайн в Україні та 24 країнах Європейського Союзу. Ми опитували учасників дослідження в рамках нашої онлайн-програми для дорослих українців з 22 червня по 6 липня 2022 року. Ми представляємо описові результати щодо стану психічного здоров'я для загальної вибірки та підгруп. Більше половини опитаних демонструють низький рівень психічного благополуччя та високий психологічний дистрес, при цьому 81% мають ризик депресії; 57% мають важкий психологічний дистрес. Біженці та внутрішньо переміщені учасники демонструють особливо високий ризик депресії та важкого психологічного дистресу в нашій вибірці. Багато українців, які звертаються за психологічною допомогою, працюють, навчаються або є волонтерами, і ті, хто займається цими видами діяльності, мають відносно кращий психічний стан. Наші результати свідчать про значне психологічне навантаження на українців. Хоча наш аналіз зосереджений на українцях, які шукають допомоги, він показує, що переміщені особи незабаром після переселення потребують особливої уваги, і з ними можна швидко зв’язатись в різних країнах, незалежно від їхнього (офіційно оформленого) статусу переселенця.




[Завантажити PDF англійською]
Резюме (Abstract in Russian): Из-за российского вторжения в Украину 24 февраля 2022 года более 8 миллионов украинцев были вынуждены покинуть свои дома. Украинцы, которые подверглись воздействию вооруженного конфликта и миграции, скорее всего, будут иметь низкий уровень психического здоровья и искать помощи. Мы приводим единую количественную оценку состояния психического здоровья украинцев, обратившихся за помощью вскоре после вторжения и переселения. Мы провели эту оценку психического благополучия и психологических расстройств среди 1165 беженцев, мигрантов, внутренне перемещенных и неперемещенных лиц из Украины, которые обратились за психологической помощью онлайн в Украине и 24 странах Европейского Союза. Мы опросили участников исследования в рамках нашей онлайн-программы для совершеннолетних украинцев с 22 июня по 6 июля 2022 года. Мы приводим описательную статистику состояния психического здоровья для общей совокупности опрошенных, а также подгрупп. Более половины опрошенных имеют низкий уровень психического благополучия и высокий уровень психологического дистресса, в частности, 81% из них находится в группе риска по депрессии, а 57 % испытывают сильный психологический дистресс. Беженцы и внутренне перемещенные демонстрируют особенно высокий риск депрессии и тяжелого психологического дистресса в нашей выборке. Многие украинцы, обращающиеся за психологической помощью, работают, учатся или добровольно помогают другим, и те, кто занимается этими видами деятельности, имеют относительно улучшенное психическое состояние. Наши результаты свидетельствуют о значительной психологической нагрузке на украинцев. Хотя наш анализ сосредоточен на украинцах, ищущих помощи, он показывает, что перемещенные лица вскоре после переселения нуждаются в особом внимании, но с ними можно быстро связаться в разных странах, независимо от их (официально оформленного)  статуса переселенца.

[Скачать PDF на английском]
Asanov, A.M., Asanov, I. and Buenstorf, G., 2023. Mental health and stress level of Ukrainians seeking psychological help online. Heliyon.  9, (11). https://doi.org/10.1016/j.heliyon.2023.e21933

Can Group Identity Explain Gender Gap in Recruitment Process? ​
with Maria Mavlikeeva (University of Kassel)

Abstract: ​Despite evidence that the gender gap in the labour market favours men, aggregate findings from correspondence studies show that women are more likely than men to be invited for a job interview. We hypothesize that the predominance of women among recruiters may explain this somewhat puzzling finding; recruiters may favour applicants of their own gender. We use the data from a large-scale correspondence study to test this hypothesis. As expected, we find that female applicants are more likely to receive callbacks for interview. We also see that in our sample the majority of contact persons responsible for the recruitment process are female. More importantly, we find that if recruiter and applicant are of the same gender, then the likelihood that the applicant will be invited for an interview increases. These findings reveal the gender favouritism at the selection stage in the labour market.

[Download PDF]

Asanov, I., & Mavlikeeva, M. (2023). Can group identity explain the gender gap in the recruitment process? Industrial Relations Journal, 54, 95–113. https://doi.org/10.1111/irj.12392
One of the most downloaded during its first 12 months of publication in Industrial Relations Journal.

Patterns of dissertation dissemination: publication-based outcomes of doctoral theses in the social sciences
with Anastasiya-Mariya Asanov , Guido Buenstorf,  Valon Kadriu &  Pia Schoch  (University of Kassel)

Abstract: Dissemination of knowledge through the publication of findings is a cornerstone of the academic research system. Doctoral dissertations document the findings made by early-stage researchers during their doctoral studies. However, prior research suggests that dissertations may not be effective in disseminating these findings to the broader community of researchers. We study how knowledge documented in doctoral dissertations is disseminated. Specifically, we investigate which dissertation characteristics and institutional factors are related to the number of journal publications based on these dissertations and the number of citations that these publications receive. Our analysis uses a random sample of doctoral dissertations from German universities in economics, political science, and sociology. We find that “cumulative” dissertations—dissertations consisting of a number of separate articles—are turned into three times more publications which receive three times more citations than monographic dissertations. We also find explorative evidence that dissertations written in English and empirical dissertations have higher publication-based outcomes. We conclude that a policy allowing doctoral candidates to write their dissertations in a cumulative format provides them with an opportunity to share the results of their research through publications in peer-reviewed journals.

[Download PDF]

Asanov, A.M., Asanov, I., Buenstorf, G., Kadriu, V. and Schoch, P., 2024. Patterns of dissertation dissemination: publication-based outcomes of doctoral theses in the social sciences. Scientometrics, pp.1-17.

Short- and Long-run Effects of External Interventions on Trust. 
with Simone Vannuccini  (FSU-Jena) 

Abstract: We experimentally analyze the effects of external intervention such as subsidy and targeting on the investment decision during the intervention and after. We employ a multi-period version of the trust (investment) game (Berg et al., 1995) introducing either a monetary incentives for contribution or providing a suggestion about the level of investment. The results of the experiment indicate  that targeting is an effective instrument to promote trustful behavior, whereas subsidy policy is not effective both in short- and long-run.  Therefore  we suggest considering a targeting policy as one of the instruments that can foster trustful behavior. 
​
 [Download PDF]

Asanov, I. and Vannuccini, S., 2020. Short-and Long-run Effects of External Interventions on Trust. Review of Behavioral Economics, 7(2), pp.159-195.

Reporting errors and biases in published empirical findings: Evidence from innovation research
with Stephan Bruns (Hasselt University) et al.

Abstract: Errors and biases in published results compromise the reliability of empirical research, posing threats to the cumulative research process and to evidence-based decision making. We provide evidence on reporting errors and biases in innovation research. We find that 45% of the articles in our sample contain at least one result for which the provided statistical information is not consistent with reported significance levels. In 25% of the articles, at least one strong reporting error is diagnosed where a statistically non-significant finding becomes significant or vice versa using the common significance threshold of 0.1. The error rate at the test level is very small with 4.0% exhibiting any error and 1.4% showing strong errors. We also find systematically more marginally significant findings compared to marginally non-significant findings at the 0.05 and 0.1 thresholds of statistical significance. These discontinuities indicate the presence of reporting biases. Explorative analysis suggests that discontinuities are related to authors’ affiliations and to a lesser extent the article’s rank in the issue and the style of reporting.

​[Download PDF]

Bruns, S. B., Asanov, I., Bode, R., Dunger, M., et al. (2019). Reporting Errors and Biases in Published Empirical Findings: Evidence from Innovation Research. Research Policy, 48(9):103796.

Working Papers

​Folktale Narratives and Economic Behavior
with Dominik P. Heinisch, Nhat Luong  (University of Kassel) 

​Narratives – stories – prevail in social life, however little is known about their relation to economic behavior and outcomes. We shed light on this issue by studying the association between folktales’ motifs and economic behavior across the world. First, we explore possible relations between the prevalence of narratives and the observed behavior in economic experiments by connecting the Berezkin (2015) collection of narratives with individual choices in experiments performed across the world. Second, we construct a motif distance index that approximates the cultural distance between countries. Third, we provide evidence that motif distance is associated with economic performance.
[Download PDF]

Is Self-employment a Career Trap? Large-Scale Field Experiment in the Labor market
with Maria Mavlikeeva (University of Kassel)

Abstract: ​We provide a large-scale correspondence experiment by sending more than 12 000 fictitious resumes in Russia to assess treatment of self-employed at the labor market.  Our study is the first correspondence experiment in Russia (Bertrand and Duflo, 2017).  Thus, to ensure validity of the experiment in the new context, we, also, assess discrimination against ethnic minorities. We find that the level of ethnic discrimination in Russia is comparable to the world average (Zschirnt and Ruedin, 2016). More importantly, we find discrimination against self-employed. However, we observe discrimination against self-employed at the low-skill market but not on the high-skill level market.  Those results go in lines with statistical form of discrimination of self-employed. 
 
This study is registered in the AEA RCT Registry and the unique identifying number is AEARCTR-0001308.

Confidence, Confidence Training and Entrepreneurial Behavior: Field Experiment.​
with Alla Levitskaia (Comrat State University)

Abstract: ​We provide a field experiment to study the effect of  an entrepreneurial program in Gagauzia region, Moldova. We randomly assign applicants for the entrepreneurship training to three educational tracks: (1) Business and confidence training; (2) Confidence training; (3) No training (control group). To understand underlying behavioral mechanisms of the intervention we elicit in the lab-in-field experiment participants'  behavioral characteristics. We find that confidence and ambiguity preference, but not risk preferences or self-efficacy,  associated with entrepreneurial behavior. We also find that a combination of business and confidence training affect an entrepreneurial related decision.​
[Download PDF]

The power of experiments: How big is your n?
with Christoph Bühren (Clausthal University of Technology),  Panagiota Zacharodimou (European Parliament)

Abstract: ​The replicability and credibility crisis in psychology and economics sparked the debate on underpowered experiments, publication biases, and p-hacking. Analyzing the number of independent observations of experiments published in Experimental Economics, Games and Economic Behavior, and the Journal of Economic Behavior and Organization, we observe that we did not learn much from this debate. The median experiment in our sample has too few independent observations and, thus, is underpowered. Moreover, we find indications for biases in reporting highly significant results. We investigate for which papers and experiments it is more likely to find reporting biases, and we suggest remedies that could help to overcome the replicability crisis.
[Download PDF]

 On the observational learning in the optimal stopping problem.
with Arno Riedl  (Maastricht University)

Abstract: We experimentally test the theoretical predictions about observational learning in case of the optimal stopping problem with publicly observable behavior but privately observable payoffs. The paper focuses on the two-agents two-arms bandit framework and follows the theoretical analysis of Rosenberg et al. (2007).  Rational behavior in the framework implies solving a non-trivial optimization task. That raises the question whether people act according to the theoretical predictions. We find evidence that subjects overreact on thier own outcomes and comply with the choices of others.
[Download PDF]

Work in Progress

Showing Life Opportunities: Increasing Opportunity-Driven Entrepreneurship and STEM Careers Through Online Courses In Schools.​
with Thomas Åstebro (HEC Paris), Guido Buenstorf ( University of Kassel); Bruno Crepón (ENSAE and École Polytechnique);  Diego d'Andria (JRC European Commission), Francisco Flores (University of Kassel), David McKenzie (World Bank); Mona Mensmann  (Warwick Business School); Mathis Schulte (HEC Paris)

​Abstract: ​We  provide an RCT to assess the effect of online courses in schools on intentions, career-related actions, and career choices to become an opportunity driven entrepreneur or choose a STEM type of education. We plan to work with to 15-17-year-old students (about 17,000 students) in 110 schools. We will randomly assign schools either to treatment (and receiving the entrepreneurship and science content online), or placebo-control (receiving a placebo treatment of online courses from standard curricula) groups. Within the treatment group, we randomize at the grade-level the type of entrepreneurship curricula, and then randomize the order of entrepreneurship and science courses to measure the short-term effects of each component and to mitigate order effects. In addition, we cross-randomize schools to a role model treatment of interviews with successful scientists and entrepreneurs. In addition, we provide information about career options. We then plan to follow subjects until (at least) age 19, to measure both short- and medium-term outcomes. 

This study is registered in the AEA RCT Registry and the unique identifying number is AEARCTR-0003593

Ethnic Employment Gap during the Covid-19 Outbreak: Great Equalizer or Divider?
with Maria Mavlikeeva (University of Kassel)
Many employees lose their jobs as the result of Covid-19 outbreak (ILO, 19 March 2020). Moreover, the number of job openings decreases, making it harder for applicants to compete. This challenging situation can be particularly hard for minority groups. We aim to assess the level of ethnic discrimination under the condition of the slack labor market caused by the Covid-19 outbreak. We replicate a correspondence study, conducted in 2017, sending resumes of applicants with the native and ethnic minority names in response to job openings in different occupations. We plan to analyse the ethnic employment gap under the tension at the labor market and heterogeneity of treatment of different ethnic groups across occupations in this conditions.

This study is registered in the AEA RCT Registry https://doi.org/10.1257/rct.5388-2.0. This study is registered in EEA list of COVID-19 projects and Economics Observatory (ECO),

[Download presentation of preliminary results]   

​[Download accompanying paper "Assessing the ethnic employment gap during the early stages of COVID-19" by Maria Mavlikeeva in COVID Economics]