Prevalence of open/FAIR data practices

Author

Affiliation

T. Willemse

Leiden University

History

Version	Revision date	Revision	Author
1.1	2023-07-20	Edited & revised	T. Willemse
1.0	2023-04-13	First draft	T. Willemse

Description

The Open Science discourse has been fuelled by the prospect of a more open and collaborative scientific effort that can accelerate scientific development and innovation. A big step in this direction is the development of Open Data practices that make it easier for scientist to share and reuse data for research. Often this agenda is pushed forward with the Findability, Accessibility, Interoperability and Reusability (FAIR) principles (Wilkinson et al., 2016). Taking these principles in mind serves to open up data practices in science and thereby improve scientific data practices, data reuse and reproducibility. Accordingly, it is important to get an indication of the prevalence of such practices in the scientific system to get an overview of the status of data sharing and Open Science in general.

The FAIR principles and Open Data management try to establish a data environment in which high quality data is easily accessible in the long term and where this data can be simply discovered, evaluated and reused (Wilkinson et al., 2016). Making data more findable could be achieved by using identifiers, adding rich metadata and registration in a searchable resource. Accessibility could be improved by data being retrieved by their identifier in a standardized format, as well as by keeping metadata accessible even if the data is no longer available. Interoperability could be enhanced by using applicable language and vocabularies along with qualified references to other data. Reusability of data can be increased by unambiguous and comprehensive storing and describing practices.

Different stakeholders, such as researchers, data publishers and funding agencies stand to benefit from these practices. More insight in the application and presence of FAIR data principles could be very relevant in their profession. Questions typically relate to how to improve the implementation and development of FAIR principles. In order to improve (FAIR) data sharing practices, it is important to first have an overview of the current practices. Hence, relevant questions are where, how and what FAIR data practices are used in an area of interest.

Metrics

Number of publications with shared data

The number of publications with shared data can serve as a quick measure to assess to what extent Open Data practices are used in the area of interest. It does however not take into account how and to what extent the FAIR principles were followed and the nature of the data itself. Due to these shortcomings, it can give a skewed representation in areas where poor quality or partly available data is documented as shared data. An additional challenge is to identify not only shared data that is shared through official repositories, but also data that is shared as supplementary material.

In similar fashion the percentage of publications with shared data in the area of interest can give a quick representation of how widespread the use of Open Data practices is. When looking specifically at the prevalence of Open Data practices this would be the preferred metric over the total number of publications with shared data. However, the percentage measure suffers the same shortcomings as mentioned before for the total number.

In addition, it is important to note that more targeted indications can be used than if a publication shares data or not. Alternatives like number of publications with data shared in a repository, data availability statements or including the level of fairness of the shared data can be used to reach more specific results.

Measurement.

The number of publications with shared data can be represented as a simple count measure of the sample of interest. The percentage can be calculated by dividing the total number of publications with shared data with the total number of units included.

Datasources

DataCite/Crossref

DataCite and Crossref are both organizations that provide services for identifying and citing research data. They maintain large databases of metadata about research articles and associated datasets, including information on Open Access data. Besides providing DOIs for datasets, DataCite and Crossref also maintain metadata about the datasets, including information on data availability, access restrictions, and licensing information. This metadata can be accessed programmatically through APIs provided by DataCite and Crossref, making it a valuable data source for researchers interested in Open Access data.

Existing methodologies

Extract dataset sharing based on Natural Language Processing

The Public Library of Science (PLOS) is a non-profit publisher of open-access journals. PLOS provides indicators on data repository use in PLOS articles as well as overall data repository use. PLOS uses a combination of manual curation and automated methods to generate information on Open Access data. This includes reviewing data availability statements provided by authors, checking data repositories for publicly accessible data associated with articles, and using natural language processing and machine learning algorithms to identify mentions of data availability in articles. PLOS also encourages authors to provide detailed data availability statements and to deposit their data in public repositories to facilitate Open Data access.

PLOS has developed the indicators on data sharing and data use through DataSeer. DataSeer provides a Natural Language Processing (NLP) and AI backed algorithm that can automatically link data sources to doi’s and check if these data sources are Open Source. Both the machine learning code and web app code are openly available.

PLOS also provides API’s to search its database. This page provides some example Solr queries, the specific queries will depend on the research question.

Level of FAIRness of data

Metrics on the level of FAIRness of data (sources) can support in establishing the prevalence of open/FAIR data practices. This metric attempts to show in a more nuanced manner where FAIR data practices are used and in some cases even to what extent they are used. Assessing whether or not a data source practices FAIR principles is not trivial with a quick glance, but there are initiatives that developed methodologies that assist to determine this for (a large number of) data sources.

Measurement.

Existing methodologies

Research Data Alliance

The Research Data Alliance developed a FAIR Data Maturity Model that can help to assess whether or not data adheres to the FAIR principles. This document is not meant to be a normative model, but provide guidelines for informed assessment.

The document includes a set of indicators for each of the four FAIR principles that can be used to assess whether or not the principles are met. Each indicator is described in detail and its relevance is annotated (essential, important or useful). The model recommends to evaluate the maturity of each indicator with the following set of maturity categories:

0 – not applicable

1 – not being considered yet

2 – under consideration or in planning phase

3 – in implementation phase

4 – fully implemented

By following this methodology, one could assess to what extent the FAIR data practices are adhered to and create comprehensive overviews, for instance by showing the scores in radar charts.

Data life cycle assessment

Determining the level of FAIR data practices can involve assessing how well data adheres to the FAIR principles at each stage of the data lifecycle, from creation to sharing and reuse (Jacob, 2019).

Identify the stages of the data lifecycle: The data lifecycle typically includes stages such as planning, collection, processing, analysis, curation, sharing, and reuse. Identify the stages that are relevant to the data to be assessed.

Evaluate adherence to FAIR principles at each stage: For each stage of the data lifecycle, evaluate the extent to which the data adheres to the FAIR principles. Use for instance the FAIR Data Maturity Model to score the adherence to the FAIR principles, assign a score for each principle and stage of the data lifecycle.

Determine the overall level of FAIR data practices: Once the scores for each principle and stage have been assigned, determine the overall level of FAIR data practices. This can be done by using a summary score that takes into account the scores for each principle and stage, or by assigning a level of FAIR data practices based on the average score across the principles and stages.

Availability of data statement

A data availability statement in a publication describes how the reader could get access to the data of the research. Having such a statement in place improves transparency on data availability and can thus be considered as an Open Data practice. However, having a data availability statement in place does not necessarily imply that the data is openly available or that it is more likely that the data can be shared (Gabelica et al., 2022). Nevertheless, a description of how to access an Open Data repository, how to make a request for data access or an explanation why some data cannot be shared due to ethical considerations are all examples of Open Data practices that make data reuse more accessible and transparent (Federer et al., 2018). The availability of a data statement can therefore be considered as an Open Data practice.

Measurement

Existing methodology

All PLOS journals require publications to include a data availability statement. Moreover, it is strongly recommended that procedures on how to access research data are described in the data availability statement and that the data is stored in a public repository. Other practices that comply with this recommendation are including a data file, data requests through an approving committee and providing contact information for a third party that owns the data (Federer et al., 2018). A detailed description of how to use PLOS data availability statements for quantitative research can be found in the (Colavizza et al., 2020) publication.

Known correlates

Some research suggests that openly sharing data is positively related to the citation rate of publications (Piwowar et al., 2007; Piwowar & Vision, 2013).

References

Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020). The citation advantage of linking publications to research data. PloS one, 15(4), e0230416.

Federer, L. M., Belter, C. W., Joubert, D. J., Livinski, A., Lu, Y. L., Snyders, L. N., & Thompson, H. (2018). Data sharing in PLOS ONE: an analysis of data availability statements. PloS one, 13(5), e0194768.

Gabelica, M., Bojčić, R., & Puljak, L. (2022). Many researchers were not compliant with their published data sharing statement: a mixed-methods study. Journal of Clinical Epidemiology, 150, 33-41.

Jacob, D. (2019, April). FAIR principles, an new opportunity to improve the data lifecycle. In ado2019: journée thématique sur les autorités de données.

Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007). Sharing Detailed Research Data Is Associated with Increased Citation Rate. PLoS ONE, 2(3), e308. https://doi.org/10.1371/journal.pone.0000308

Piwowar, H. A., & Vision, T. J. (2013). Data reuse and the Open Data citation advantage. PeerJ, 1, e175. https://doi.org/10.7717/peerj.175

Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), Article 1. https://doi.org/10.1038/sdata.2016.18