Prevalence of national Open Science policies
L. Stoy 
Description
This indicator aims to capture the existence of country-level policies for Open Science. Since Open Science can be considered a catch-all term for different practices and objectives, as elaborated in the Recommendation on Open Science from the United Nations Educational, Scientific and Cultural Organization (UNESCO), this may include different elements or distinct policies such as Open Access to publications, Open Data, fair data, research assessment reform, Citizen Science etc. A
Usually, “national policy” will imply the policy being adopted by a country-level organisation with an important structuring role for the research and innovation sector. Edge cases could be federal countries in which some elements of decision-making and institutions relevant for research and innovations are devolved to regional or community level (e.g., Germany, Belgium, Switzerland). Practically, a policy could be adopted by a parliamentary decision, a ministry strategy, a government agency in charge of research and innovation, a national research council, an alliance of research organisations etc.
Measuring the proliferation of national Open Science policies is relevant for several reasons.
- First, knowing whether a specific policy exists and what it entails is crucial part of the impact analysis. A policy would take the place of the activity and intervention that, through specific outputs and outcomes, creates impact. In other words, it one of the first steps within Key Impact Pathways, in particular those of interest for national-level policy making.
- Second, there is an ongoing effort to monitor progress of Open Science policy implementation by proxy of national policies. This is evident, for example, in the UNESCO recommendation on Open Science, which asks member states to develop policies (Art. 17) and to invest in Open Science (Art. 18 and 19). In the European context, Open Science policies are monitored as part of the European Open Science Cloud (EOSC) initiative and the European Research Area (ERA). In the case of EOSC, there is a specific ambition to also measure impact indicators in the future (O’Neill 2022a).
Metrics
Existence of Open Science policy
Measuring the existence of a national Open Science policy can be conceptualised at different levels and degrees of complexity.
‘Basic’: Existence of a country-level policy document or strategy on Open Science in general (yes/no)
‘Intermediate’: Existence of specific country-level policies on specific areas such Open Access to publications, FAIR/Open Data, Citizen Science, etc. (yes/no)
‘Advanced’: Specific measures (regulatory aspects, compliance levels, financial measures, target groups, implementing authority etc.) defined as part Open Science policy (Open Access to publications, FAIR/Open Data etc.)
Limitations but also advantages stemming from these approaches depend on the specific level:
- In the ‘basic’ case, the measurement can be conducted with relative ease. However, the lack of granularity of specific measures might limit the types of analyses that can be done with the data.
- In the ‘intermediate’ case, granularity is provided through the specific areas, which allows more detailed analyses that the ‘basic’ case. Data collection can be expected to be done relatively simply by capturing which topics are covered in the policy.
- The ‘advanced’ case will provide more detailed information about area, nature, and scope of Open Science policies which in turn allows more complex analyses. Data collection will be more challenging due to the need to define specific dimensions and their measurements and the need for an in-depth analysis of the source documents. Unfortunately, there are no controlled vocabularies for this type of data collection.
Measurement
As outlined in the previous section, the exact measurement of the existence of a national policy for Open Science depends on the chosen approach.
In the basic case, the existence of a policy, which means the existence of a national (or perhaps, regional) policy and strategy supported by government actors and ideally stakeholders, can probably be captured with a simple binary variable with yes, there is a policy, and with no, there is no policy. The only challenge would be to provide a common definition of what type of document constitutes a policy.
The intermediate case would work on a similar binary logic (yes/no) with the only difference being that this measurement is conducted for each Open Science policy area.
Only in the advanced case, additional refinements and more complex measurements and codes will be required. For example, differences in policy requirements (compliance, target groups), could be coded on an ordinal or nominal scale.
In practice, various studies have attempted to combine different levels of the conceptual approach outlined above.
SPARC Europe has regularly commissioned reports on Open Science policies in Europe. The latest report included information on the “scope, data definition, mandates, exceptions, mentions of FAIR, DMPs, data citation, data availability statements, re-use, Intellectual Property Rights (IPR) and licensing, and costs” (Sveinsdottir, Davidson, and Proudman 2021) p10. For each element and several follow-up items, a binary or nominal coding was used:
- Definition of data: yes / not specified
- Mandates: unspecified / suggested or recommended / required
- Exceptions to data sharing: yes / unspecified
- Mentions of FAIR: yes / unspecified
- DMPs: recommended / required / other
- Expectation of data citation: yes / unspecified / other
- Data availability statements: yes / unspecified
- IPR: yes / unspecified
- Preferred licenses: yes / unspecified / other
- Costs for RDM: yes / unspecified / other
The information collected on behalf of SPARC Europe moreover contains textual or numeric information on additional structural information about the policies:
- Type of policy: statute, government ministry, funder policy
- Entry into force: year
- Sponsoring organisation
- Scope and coverage beyond data
- Link to Open Access: yes / no
- Soft or hard policy: soft / hard
- Coverage of skills and training: yes / no
- Monitoring and/or compliance: yes / no
Another case of standardized data collection on Open Science policies is the EOSC Observatory (https://eoscobservatory.eosc-portal.eu/home). The EOSC Observatory has been developed as part of the EOSC-Future project and aims to “support the EOSC community in tracking the implementation of EOSC and the policy makers in developing actionable policies”.
As part of the observatory, an EOSC Monitoring Framework has been developed which collects information, on a regular basis, together with representatives of EU member states and associated countries. The monitoring framework collects standardized information on 8 policy areas and the existence of country-level policies and accompanying investments in Open Science.
In the current framework, this matrix of 8 categories crossed with policies and practices areas results in 96 indicators, 48 each for policies and practices. Detailed items for each indicator are included in the “Survey on National Contributions to EOSC 2022” (O’Neill et al. 2023). Practically, the large majority of items are measured through binary yes-no questions. In the case of Open Access to publications, the following items are included in the survey:
- Existence of national policy on Open Access to publications: yes / no
- Is this policy mandatory: yes / no
- Existence of a policy on immediate Open Access to publications: yes / no
- Is this policy mandatory: yes / no
- Is there a specific policy on retention of IPR on publications: yes / no
- Is this policy mandatory: yes / no
For each item, the questionnaire also includes an item about the public availability of the policy (measured as yes / no variable) and an open-text field to enter a URL to the corresponding website.
Datasources
EOSC Observatory
As introduced above, the EOSC Observatory is a “policy intelligence tool” which contains information about Open Science policies in EU member states and associated countries. The EOSC Observatory is available as an online tool (http://eoscobservatory.eosc-portal.eu). Underlying data is made re-usable via a Zenodo community available at https://zenodo.org/communities/eoscobservatory/.
A specificity of the EOSC Observatory is that the data collected through the survey is directly answered and validated by member states and associated countries’ representatives in the EOSC Steering Board. As part of the EOSC monitoring activities, the EOSC Observatory also provides crucial information for the monitoring of progress in the context of the European Research Area.
While originally started with the EOSC-Future project (2021-2023), the EOSC Observatory has secured follow-up funding through the Horizon Europe work programme for the next years. This makes it a stable source of information about the evolution of Open Science policies in the European Union and associated countries.
A detailed list of items asked for each policy category is included in the questionnaire. The latest version is available on Zenodo (O’Neill et al. 2023).
In addition, data is made available openly. The latest version is shared in .tsv format and therefore easily reusable. This availability also means that data can be re-coded with relatively little effort to adapt for specific research questions. Moreover, the data contains links to relevant policy sources, which reduces the effort to collect this information by researchers re-using the data (O’Neill and Martziou 2022).
SPARC Europe
SPARC Europe has commissioned several studies on national Open Science policies in Europe. The study is regularly updated and is now available in its 7th version (Sveinsdottir, Davidson, and Proudman 2021).
The data includes information on variables such as
- Definition of data: yes / not specified
- Mandates: unspecified / suggested or recommended / required
- Exceptions to data sharing: yes / unspecified
- Mentions of FAIR: yes / unspecified
- DMPs: recommended / required / other
- Expectation of data citation: yes / unspecified / other
- Data availability statements: yes / unspecified
- IPR: yes / unspecified
- Preferred licenses: yes / unspecified / other
- Costs for RDM: yes / unspecified / other
- Type of policy: statute, government ministry, funder policy
- Entry into force: year
- Sponsoring organisation
- Scope and coverage beyond data:
- Link to Open Access: yes / no
- Soft or hard policy: soft / hard
- Coverage of skills and training: yes / no
- Monitoring and/or compliance: yes / no
The information is presented as narrative content of the study report, although several tables are included too. It is not available in spreadsheet format publicly. In addition, the reports contain country fiches which provide narrative data about national Open Science policies. In the seventh and latest version, the specific focus was on FAIR and Open Data. Previous versions focussed on other areas, such as Open Access to publications.
OpenAIRE
OpenAIRE is providing information on national Open Science policies via its country pages (https://www.openaire.eu/os-eu-countries). The country pages are maintained by the respective National Open Access desks (NOAD). The information is spilt into several categories:
- Overview
- Open Science Policy
- Infrastructure & EOSC
- Training & Support
- Statistics
- News
This information is unstructured and not presented in a standardized fashion to facilitate re-use.
Council for National Open Science Coordination (CoNOSC)
The Council for National Open Science Coordination (CoNOSC) is providing information about member countries’ national Open Science system and policies at https://conosc.org/os-policies/#page-content.
This information is unstructured and not presented in a standardized fashion to facilitate re-use.
National Points of Reference on Scientific Information
The European Expert Group on National Points of Reference on Scientific Information provides occasional updates about the state of Open Access and Open Science policies in EU member states and associated countries. Information is available via the EU expert group registry: https://ec.europa.eu/transparency/expert-groups-register/screen/expert-groups/consult?lang=en&groupID=3477
Reports of the group do contain qualitative and quantitative information, which is however not made available in standardized formats.
ROARmap
The ROARMAP database contains information about policies concerning In total 87 funder-only policies are registered.
Open Access to publications from several types of organisations worldwide (https://roarmap.eprints.org) . For national Open Science policies, most relevant might be the category of research funder, which includes national research councils, government agencies etc. The database contains more than 770 policies at the time of writing. Data is available in standardized formats and with a unique identifier. All entries are reviewed by the curators of ROARMAP.
Information collected about policies includes attributes such as adoption, effective, and revision date; content types specified by the mandate; Open Access conditions; link to research evaluation; rights retention; etc.
References
Reuse
Citation
@online{apartis2024,
author = {Apartis, S. and Catalano, G. and Consiglio, G. and Costas,
R. and Delugas, E. and Dulong de Rosnay, M. and Grypari, I. and
Karasz, I. and Klebel, Thomas and Kormann, E. and Manola, N. and
Papageorgiou, H. and Seminaroti, E. and Stavropoulos, P. and Stoy,
L. and Traag, V.A. and van Leeuwen, T. and Venturini, T. and
Vignetti, S. and Waltman, L. and Willemse, T.},
title = {Open {Science} {Impact} {Indicator} {Handbook}},
date = {2024},
url = {https://handbook.pathos-project.eu/sections/1_open_science/prevalence_national_open_science_policies.html},
doi = {10.5281/zenodo.14538442},
langid = {en}
}