Spanish universities and open science: a study of barriers and facilitators

1 Introduction

Open science constitutes a radical shift in the research process based on collaboration, openness and transparency at all stages of research (data collection, peer review, final publication, assessment, etc.). It emerged due to the widespread use of technology in research and the increasing importance of open access to articles, and is being driven by institutional support, especially from the European Commission and several funding agencies.

Open science is a generic or umbrella term that refers to various interrelated dimensions of a new working model, although there is no unanimity concerning what these dimensions are. Thus, the taxonomy of the FOSTER project (Facilitate Open Science Training for European Research) (Pontika et al., 2015) considers five elements (open access, open data, open reproducible research, open science policies and open science tools), while the Open Science Policy Platform (European Commission, 2018) refers to the eight pillars of open science: FAIR data, the future of scholarly communication, infrastructure (EOSC), rewards and incentives, next generation metrics, education and skills, research integrity and citizen science. The reviews on the concept of open science carried out by Fecher and Friesike (2014) and Vicente-Sáez and Martínez-Fuentes (2018), among others, the educational materials and documents prepared by the FOSTER+ project (2018) and the UNESCO document on recommendations to achieve open, inclusive and equitable science (UNESCO, 2021) offer an overview that provides a clearer understanding of open science.

Spanish legislation has been progressively adapting to this new framework in recent years. In 2011, Article 37 of the Law on Science, Technology and Innovation included the obligation to deposit articles resulting from publicly funded research projects. It was modified in 2022: “open access” was changed to “open science” and aspects related to research data management (“the public administration must promote initiatives to preserve and disseminate data in accordance with the FAIR model”) and the possibility of using open access articles in the assessment of research were included. Also in 2022, the Law on the University System (LOSU), which is currently going through parliament, provided for the “promotion of science and citizen science” in Article 12 and included open access and FAIR data, citizen science and research assessment. It also mentions open science in the preamble, devotes Article 56 to funding and presents open science indicators for additional funding by objectives.

The drive towards open science in the academic domain is also evident in the Commitment of Spanish universities to Implement Open Science, which was approved in 2019 by CRUE, in line with the indications of the European Commission’s Open Science Policy Platform (2018), the declaration of the European University Association (2017) and the roadmap of associations such as the League of European Research Universities(2018). Likewise, some universities such as the Universitat Oberta de Catalunya have approved comprehensive open science plans and many others have implemented policies on open access (Abadal et al., 2013) or research data sharing.

Universities play a key role in the adoption of open science by researchers and academics, although there are many barriers and reservations that are hindering progress in these measures (González-Teruel et al., 2022). The aim of this study was to shed light on the current situation, highlight these issues at Spanish universities and focus on the measures taken by universities to ensure that scientific publications can freely access research data and assessment of science.

2 Methodology

Between September and November 2021, a questionnaire was sent to the vice-rectors for research at Spanish universities with a view to determining the universities’ commitment to open science (especially open access to articles), how this has influenced the adoption of open science principles by the university community and their opinion on possible indicators for assessing research in the new scientific ecosystem.

The study population consisted of the 76 vice-rectors for research at the universities belonging to the Conference of Rectors of Spanish Universities (CRUE), 50 of which are public (65.8%) and 26 private (34.2 %). Together, their average annual productivity between 2000 and 2019 was 67,062 articles featured in the Web of Science (WoS) (IUNE, 2022).

The questionnaire was designed based on a review of the literature and the responses obtained in a previous qualitative study carried out between March and May 2020 (González-Teruel et al., 2022). It included 54 items within 13 questions divided into the following four sections:

• The university’s explicit alignment with open science.
• Facilitators for the promotion of open access publication.
• Assessment models and proposed new indicators.
• Measures taken towards a change in the assessment model.

The questionnaire was conducted in LimeSurvey. CRUE was responsible for contacting the vice-rectors and sending the questionnaires. The questionnaires were first sent in September 2021, and a reminder of the deadline (15 November 2021) was sent in October.

3 Results

A total of 28 vice-rectors responded (36.8 %). Their universities accounted for 47% of Spain’s scientific production featured in the Web of Science between 2010 and 2019 (IUNE, 2022).

3.1 Universities’ explicit alignment with open science, infrastructure and policies

Nineteen universities (79.1 %) said that they had a political figure with specific responsibility for open science. Of these, 11 said that this was the responsibility of a single vice-rectorate (nine for research, one for science policy and open science, and one not specified) and eight that the responsibility was shared between several vice-rectorates; six of these included the vice-rectorate for research.

Seventy-five percent of the universities confirmed that open science was included as a mission in their strategic plan or institutional reference documents, 53% said it was being translated into specific objectives, and only 17.9 % had established explicit criteria to assess their compliance with these objectives.

A total of 96.4 % of the universities analysed had an institutional repository, a figure similar to the rest of Europe, where 90 % of institutions have their own repository or participate in a shared repository (Morais et al., 2021).

Half the universities analysed were not part of the shared or individual repositories identified by the re3data Registry of Research Data Repositories (re3data.org, 2022), which collects metadata from repositories specialising in research data. The vice-rectors at 11 of these universities (78.8 %) stated that they were working on the creation or adaptation of existing repositories to share research data.

Twenty-three universities (82.1 %) declared that they had their own policy to promote open access; 14 said it was encouraged and nine said it was mandatory. In Europe, 54 % of institutions have an open science policy and half of these include all research data policy elements (Morais et al., 2021).

Forty-six percent of the universities said they used the deposit of documents in open access as a criterion for assessing research reports.

To develop infrastructure and procedures related to open science, 60.7 % of the vice-rectors stated that they had a technical support department and 96.4 % said they worked in coordination with the library service.

3.2 Facilitators for the promotion of open access publication and the deposit of documents and data

The greatest facilitator was coordination between vice-rectorates and library services (Table 1), whose research support departments are usually set up to manage repositories and support publication in open access journals (Iribarren et al., 2015). The relevance of this aspect is recognised by the Ligue des Bibliothèques Européennes de Recherche (2017) and the League of European Research Universities (2018), which consider the creation of services for open science to be one of their challenges. Moreover, its role is reflected in Article 63 of the first draft of the Law on the University System (LOSU), relating to the promotion of open science and citizen science; paragraph 7 states that “libraries and other university departments shall facilitate citizens’ access to digital and non-digital information resources and shall provide the training required to promote the dissemination of open science in the university community and society at large”.

Table 1. Barriers and facilitators for the promotion of open access publication and the deposit of documents and data

Other aspects considered to facilitate open science included a commitment to achieve the established objectives, the inclusion of these objectives in the institution’s strategic plan and the existence of open science mandates. However, these results reveal a gap between what is considered to facilitate open science and the reality, given the absence of specific open science objectives at half the universities and the virtual absence of criteria to assess their compliance with these objectives. This aspect is included in universities’ commitments to open science, as established by CRUE (2019), in relation to “analysing Spain’s situation concerning open access and constantly monitoring its evolution so that the information available is always up-to-date”. This reflects the disparity between Spain and the practices of European universities, 80 % of which monitor the number of articles in their repositories and 70 % of which monitor articles published in open access journals (Morais et al., 2021).

Although the age and disciplinarity of a university were not considered relevant to achieving open science objectives, one would expect younger universities to be more active. In fact, the universities most actively involved in international networks or alliances (e.g. The League of European Research Universities, the Young European Research Universities Network and the European University Association) have been advocating for open science for years. However, the presence of Spanish universities remains low, since only the Universitat de Barcelona belongs to the League of European Research Universities, only Universidad Carlos III de Madrid and the Universidad Autónoma de Madrid are part of the Young European Research Universities Network and just 31 universities form part of the European Universities Association.

The factors that were considered facilitators related to the financial support required to carry out the transition from the traditional publication and knowledge dissemination model to the model favoured by open science; a model that is not cheap and requires investment both to transform university journals into digital, open access journals and to support, among other things, the payment of fees for publishing articles (article processing charges, APCs). This lack of financial support has already been identified as a barrier (Pardo-Martínez; Cotte-Poveda, 2018).

The principles of Plan S, which is firmly committed to immediate open access for articles stemming from funded research, state that publication costs should be covered by funders, research institutes or universities, and not by individual researchers. In addition, point 5 establishes the following: “The Funders support the diversity of business models for open access journals and platforms. When open access publication fees are applied, they must be commensurate with the publication services delivered and the structure of such fees must be transparent to inform the market and facilitate the potential standardisation and capping of payments of fees”. In addition, the commitments to open science established by CRUE (2019) include that of “compiling and disclosing university spending associated with access to information resources and the publication of results, as well as analysing the potential cost overrun or savings for universities should they move from the current payment-based access system to the immediate open access system”.

To that end, some measures have already been taken, including the transformative agreements (Borrego, Anglada & Abadal, 2021) recently signed by CRUE and the Spanish National Research Council (CSIC) with several academic publishers for journal subscription fees and the option for researchers to publish articles in open access at no additional cost, under a single agreement with each publisher. These clear changes in the policies of funding agencies will lead to changes in several issues (such as how much it costs, what the cost covers and who is required to pay), which could promote open access in the long term. This is in line with Article 63 of LOSU, which states that “universities shall promote the transparency of subscription agreements with scientific publishers”.

Economic incentives for researchers to achieve open access objectives were also considered a facilitator. One respondent commented that they disagreed with this aspect, since they considered that researchers already receive incentives through research sexennials.

3.3 Assessment models and proposed new indicators

Transforming the current scientific ecosystem requires assessment policies in line with the new open science scenario, and its frame of reference is reflected in the European Commission’s report Towards a Reform of the Research Assessment System: Scoping Report (2021).The models proposed by the San Francisco Declaration on Research Assessment (DORA, 2013), the Leiden Manifesto (Hicks et al., 2015) and the reports of the European Commission (Wilsdon et al., 2017) recommend that the assessment of articles be based on criteria other than the impact factor currently used and that they consider, in addition to citations, indicators such as the number of downloads (article usage) and their presence in blogs and social networks (altmetrics) (Wilsdon et al., 2017), as well as qualitative measures.

The results show that the vice-rectors mostly disagreed with the use of impact factor for assessing articles, agreed with the benefits of qualitative assessments and opt for the use of article citations to assess academic research performance (Table 2). On the other hand, they presented a moderate position regarding the number of downloads to assess scientific articles and the introduction of criteria related to their social impact (presence in the media, public collaboration) to assess academic research performance. In addition, they disagreed with the presence of articles in general blogs and social networks and academic social networks such as Academia.edu and ResearchGate as criteria to assess articles and academic research performance; both of these criteria are included in altmetrics. Worth noting was the lack of agreement concerning the use of the number of accredited open data files with DOIs and the percentage of open articles as criteria for assessing academic research performance. Both indicators are directly related to the achievement of open science objectives.

Table 2. Assessment models and proposed new indicators (1)

These results could be influenced by several aspects. Firstly, altmetrics present methodological shortcomings such as volatility, problems associated with standardisation and data collection, and a lack of consistency between the results provided by the data collection tools used (Abadal, 2018). Secondly, the aspects assessed by the new indicators are based on dimensions that are relevant to the open science model, which are distinct from the “quality” of the content of articles. Thus, for example, if an article appears on social and academic networks, this reflects the researcher’s efforts to disseminate and raise the profile of their work, while the number of accredited open data files with DOIs and the percentage of open access articles are based on the accessibility and reuse of content. In this regard, it is interesting to note the contradiction between the vice-rectors’ responses and their belief that the use of indicators related to open access in the assessment of research reports or in calls for scholarships promote the objectives of open science (Table 2). It is hoped that the provisions of Article 63 of the first draft of LOSU encourage the inclusion of new criteria and indicators, aligned with open science, in the assessment process. Paragraph 8 of this article establishes that “national and regional quality agencies shall include open access to the scientific results of teaching and research staff in their assessment criteria and requirements”.

The main barriers to the adoption of new assessment criteria were the workload and the time involved in qualitative assessments and the rigid administrative structure of calls, which hinders adaptation to the new model. Additional barriers mentioned, albeit with less agreement, were the possibility of citations being falsified on social networks and the potential impact of the subjective component of qualitative assessments in terms of the resources needed to meet the economic cost of implementation.

The importance attached to research in the political agenda, institutional commitment to change and institutional alignment with DORA were considered facilitators. However, of the 155 signatories to DORA, only four are Spanish universities: the Universitat Oberta de Catalunya, Universidad Camilo José Cela, Universidad Miguel Hernández de Elche and the Universitat de Barcelona. The lowest level of agreement related to the question about quality articles being published in non-indexed journals, which reduces the possibility of locally relevant articles with social impact being published in journals with limited international dissemination.

Table 3. Assessment models and proposed new indicators (2)

The vice-rectors considered that all stakeholders in the academic and research system, including researchers, should be involved in the drive towards a new assessment model (Table 4).

Table 4. Assessment models and proposed new indicators (3)

3.4 Measures taken towards a change in the assessment model

Only four of the universities (14 %) had taken any measures to include new criteria in the assessment process. In two cases, these were related to the assessment of teaching staff based on open access articles or deposits (in one case linked to additional remunerations). One had implemented a more comprehensive programme in this regard, while the other, interestingly, indicated that it had introduced “an annual assessment based on articles published in a list of quality journals” as a new criterion. It is worth noting that there was a contradiction between these responses and those mentioned above, in which 46.4 % of the universities stated that they used the deposit of documents in open access as a criterion for assessing their research reports (Section 3.1).

With respect to the future, the vice-rectors at 10 universities stated that they planned to modify their criteria, although only four defined these. In two cases, they were related to open access indicators; in one, they were linked to the quality of each article; and only one proposed comprehensive criteria to assess career development and staff recruitment.

4 Conclusions

The results show that, although open science features on the political agenda, Spanish universities have not yet made a firm commitment to it. Thus, despite the fact that policies and infrastructure that facilitate open access and repositories for data sharing are being developed, the universities’ commitment has not translated into specific objectives, criteria for monitoring these objectives or incentives to achieve them.

With respect to assessment, the preferred criteria (qualitative assessment of the content of articles, use of citations and social impact) focus on the value of the article, but criteria that reflect researchers’ efforts to disseminate their work, promote openness and encourage the reuse of content, aspects that are more consistent with the open science approach to research, were less accepted. This reflects the fact that open science is still understood as a process linked to the research product, rather than the global reference model needed to make the global scientific ecosystem function.

Implementing open science requires an increase in the resources allocated to achieve this and clear instructions on how to proceed. It is hoped that European guidelines and the recent legislative changes in Spain will lead to a decisive step forward.

Bibliography

Abadal, Ernest (2018). "Las altmétricas: aportaciones para la evaluación de publicaciones científicas en ciencias humanas y sociales". PH: Boletín del Instituto Andaluz del Patrimonio Histórico, 136.

Borrego, Ángel; Anglada, Lluís; Abadal, Ernest (2021). "Transformative agreements: Do they pave the way to open access?". Learned Publishing, vol. 34, no. 2, p. 216–232. <https://doi.org/10.1002/leap.1347>.

Comisión Europea (2018). Open Science Policy Platform Recommendations. Brussels: European Commission. <https://ec.europa.eu/research/openscience/pdf/
integrated_advice_opspp_recommendations.pdf>.

— (2021). Towards a reform of the research assessment system: scoping report. Luxembourg: Publications Office of the European Union. <https://data.europa.eu/doi/10.2777/707440>.

Conferencia de Rectores de las Universidades Españolas (2019). Compromisos de las universidades ante la Open Science. <https://www.crue.org/wp-content/uploads/2020/02/2019.02.20-Compromisos-CRUE_OPENSCIENCE-VF.pdf>.

Declaración de San Francisco (2013). San Francisco Declaration on Research Assessment. <https://sfdora.org/>.

European University Association (2017). EUA Statement on Open Science to EU Institutions and National Governments. European University Association [EUA]. <https://eua.eu/resources/publications/412:eua-statement-on-open-science-to-eu-institutions-and-national-governments.html>.

Fecher, Benedikt; Friesike, Sascha (2014). "Open Science: One Term, Five Schools of Thought". En: Bartling, Sönke; Friesike, Sascha (ed.), Opening Science. DOI: 10.1007/978-3-319-00026-8_2.

FOSTER (2018). The future of science is open. <https://www.fosteropenscience.eu/>.

González-Teruel, Aurora; López-Borrull, Alexandre; Santos-Hermosa, Gema; Abad-García, Francisca; Ollé, Candela; Serrano-Vicente, Rocío (2022). "Drivers and barriers in the transition to open science: the perspective of stakeholders in the Spanish scientific community". El profesional de la información. <https://doi.org/10.3145/epi.2022.may.05>.

Hicks, Diana; Wouters, Paul; Waltman, Ludo; De Rijcke, Sara; Rafols, Ismael (2015). "Bibliometrics: the Leiden Manifesto for research metrics". Nature, vol. 520, no. 7.548, p. 429–431.

Iribarren-Maestro, Isabel; Grandal, Teresa; Alecha, María; Nieva, Ana; San-Julián, Teresa (2015). "Apoyando la investigación: nuevos roles en el servicio de bibliotecas de la Universidad de Navarra". El profesional de la información, vol. 24, n.o 2, p. 131.

IUNE (2022). Actividad investigadora de la universidad española (20 May 2022). <https://iune.es/>.

League of European Research Universities (2018). Open Science and its role in universities: a roadmap for cultural change. <https://www.leru.org/files/LERU-AP24-Open-Science-full-paper.pdf>.

Ligue des Bibliothèques Européennes de Recherche (2017). Research Libraries Powering Sustainable Knowledge in the Digital Age: LIBER Europe Strategy 2018–2022. <https://libereurope.eu/wp-content/uploads/2020/10/LIBER-Strategy-2018-2022.pdf>.

Morais, Rita; Saenen, Bregt; Garbuglia, Federica; Berghmans, Stephane; Gaillard, Vinciane (2021). From principles to practices: Open Science at Europe's universities. 2020–2021 EUA Open Science Survey results.

Pardo-Martínez, Clara Inés; Cotte-Poveda, Alexander (2018). "Knowledge and Perceptions of Open Science among Researchers—A Case Study for Colombia". Information,vol. 9, no. 11, p. 292. <https://doi.org/10.3390/info9110292>.

Pontika, Nancy; Knoth, Petr; Cancellieri, Matteo; Pearce, Samuel (2015). "Fostering Open Science to Research using a Taxonomy and an eLearning Portal". En: iKnow: 15th International Conference on Knowledge Technologies and Data Driven Business, 21–22 Oct 2015, Graz, Austria.

re3data.org (2022). Registry of Research Data Repositories. <https://www.re3data.org/>.

UNESCO (2021). Recomendación de la UNESCO sobre la Ciencia Abierta. París: UNESCO. 36 p. <https://unesdoc.unesco.org/ark:/48223/pf0000379949_spa>. [Consulta: 10/01/2022].

Vicente-Sáez, Ruben; Martínez-Fuentes, Clara (2018). "Open Science now: A systematic literature review for an integrated definition". Journal of business research, vol. 88, p. 428–436. <https://doi.org/10.1016/j.jbusres.2017.12.043>.

Wilsdon, James; Bar-Ilan, Judit; Frodeman, Robert; Lex, Elisabeth; Peters, Isabella; Wouters, Paul (2017). Next-generation metrics: Responsible metrics and evaluation for open science. Report of the European Commission Expert Group on Altmetrics. Brussels: European Commission. <https://scholarlypublications.universiteitleiden.nl/access/item%3A29456>.