It is critical for researchers and grantees alike to adhere to rigor and transparency criteria to ensure their contributions to scientific research are sufficiently transparent so they can be replicated and eventually reproduced. SciScore evaluates scientific manuscripts for compliance with consensus granting agency and journal recommendations designed to address different aspects of rigor and transparency in the published literature (e.g., MDAR [Materials Design Analysis Reporting], ARRIVE [Animal Research: Reporting In Vivo Experiments], CONSORT [Consolidated Standards of Reporting Trials], RRID [Research Resource Identification] standards). SciScore has been implemented by several society publishers in different ways, with one allowing authors to run the tool as often as they wish, and another having editors verify manuscripts using the report provided by the tool. Results show that the use of the tool led to an increase in the average SciScore over time or via the revision process. The use of the tool also resulted in an increase in the number of manuscripts with RRIDs, a fairly easy transparency criterion to check. We conclude that the use of the tool is effective in improving some aspects of rigor of research articles.

Roelandse, Martijn, et al. "Assessing the Effectiveness of SciScore in Supporting the Reproducibility of Scientific Research." Science Editor 46.2: 46-52. View publication

Evaluated 1,578,964 articles for rigor criteria using SciScore, an automated tool
Developed a journal quality metric and studied rigor criteria reporting trends
No correlation between the Rigor and Transparency Index and the Impact Factor
The Rigor and Transparency Index may serve as a proxy for methodological quality View publication