In this Blog we collect news, perspectives, case studies, and other materials shared by scholars that demonstrate the implementation of Open Science practices in all fields of research, help new members adopt new tools and resources, and disseminate Open Science culture.

If you have a story to tell (in Italian or English), contact us.

Failed replications, successful science

Michelangelo Vianello - Professor at The University of Padova, Italy
Open Science Story - 1 - Published on October 13th, 2021

My story with Open Science begins in 2012 when a colleague with whom I used to work with on other projects (Brian Nosek) invited me to join a Google Group discussion whose participants later became the Open Science Collaboration. We replicated 100 studies sampled from the 3 leading journals in psychology and found that we were unable to reproduce the original results approximately 60% of the times when the expected rate of not reproducible studies was roughly 30%.

I then led Many labs 1 and Many Labs 2 together with Rick Klein Fred Hasselman and Brian Nosek. It’s been an exciting and incredibly demanding experience. In these studies, we reproduced altogether 41 effects on more than 20.000 subjects all around the world. The reproducibility rate was collectively around 55%. Most interestingly, we found that reproducibility did not depend on where the data were collected. The likelihood of an effect being reproduced was very similar across labs. Instead, across effects, there was a huge variability.

Open Science requires extra effort

I started adopting Open Science practices in 2012 and never stopped. Now I regularly preregister my hypotheses when I run confirmatory studies and post data and scripts on the Open Science Framework. I started doing so because it is consistent with the idea of science that I have, but quickly realized that posting data and code publicly actually is a good investment since it facilitates post-publication activities. Open Science requires extra effort.

The first and most important barrier in embracing Open Science practices relates to traditions, culture, and expertise. These are things that have never been taught formally or informally.

The second barrier to change is due to the extra effort that is needed to implement Open Science practices. Our burden is already heavy, pressure to publish is ridiculously high. So, one really needs to be extremely motivated to follow open science practices.

It pays in the end

However, Open Science pays in the end, when people ask for further information regarding your studies, and you have everything archived and clearly explained. If the approach was not Open, it is likely that, in case you want to reply to requests you need to devote hours to collecting all relevant information, eventually back-engineer what you did 8 years ago, and then apologize for the delay, or even regretfully inform that some information cannot be found. If everything is open, you just need to send them a link! Paradise on earth.

Open Science also makes it much easier to talk with colleagues, less time spent over digging deep in the hard drive and so on. I don’t think it changes interactions with students. Just the content of what I teach. Personally, I feel prouder than before when I tell students that I use and promote an Open approach to science.

In my experience, it is not only the right way to do science but also a good personal investment.