Nyhetsinnsamleren til creativecommons.no

Plos: To Sign or Not to Sign: A Slice of Transparency in Peer Review

Leave a Comment / Articles /

Scientists depend on the proper evaluation of research that creates the foundation for future work, and the public expects curated scientific content to be trustworthy. All forms of peer review, whether for ethical, technical and sound science criteria or for additional novelty, significance and perceived impact help ensure rigor in scientific research.

There is, however, community and public skepticism regarding the quality, trustworthiness and authenticity of the review process, from the initial stage of evaluation before reviewer assignment to the final editorial decision. Making peer review more transparent, at any stage, has the potential to revitalize the process and restore trust in the system.

Efforts to increase transparency in peer review should address challenges that include:

Reshaping Peer Review

Change is already happening as the scientific community develops variations on themes of open and transparent, and as publishers provide more peer review offerings that range from community participation to open but anonymous, to fully open and signed reviews. While not yet functioning at scale, experiments incorporating more transparent ways to discuss and assess papers over the entire lifecycle of the research are inching their way into practice. This way, the publication of an article isn’t the single defining event in its life; it is just one chapter of its story. Many of the arguments in favor of increased transparency in peer review also hold true in the discussion of benefits of preprint submissions. According to researchers working with neuroimaging, as stated in their PLOS Biology Community Page, “preprints allow the wider community to give feedback to the authors about the manuscript and potentially improve it, which is beneficial for both the authors as well as the journal the paper will be submitted to. For example, the present paper received useful comments from three individuals in addition to the appointed peer reviewers.”

A Question of Signing

At PLOS, we’ve looked at one slice of transparency in peer review—signed reviews made available exclusively to the authors. In a research project that used a survey mechanism to collect experiences and opinions, our Publishing Operations team underwent an assessment of reviews from 2013-2016 in three of the PLOS journals (PLOS ONE, PLOS Medicine and PLOS Computational Biology).

“Our reviewer community is particularly engaged, and that’s what makes working at PLOS on this issue so exciting. Together we will be able to create solutions, both incremental and substantial, that bring constructive feedback to authors and transparency to the review process.” Helen Atkins, PLOS Director of Publishing Operations

Some of what we learned is that reviewers who were not in the habit of signing reviews simply had never been asked, or were not sure of the benefits. But it’s not as easy as just educating in these areas. These scientists also indicated that not signing allows them to be more honest and safe from retribution. We also discovered that signing can improve reviewer accountability and constructiveness, and help authors learn of a reviewer’s area of expertise. Authors who favored receiving signed reviews valued having this additional information as it provides potential for more open communication, moving research forward. The full results of this project, presented by Elizabeth Seiver, PLOS Researcher and Helen Atkins are part of the Editorial and Peer-Review Process Innovations Session on Tuesday, September 12 during the International Congress on Peer Review and Scientific Publication.

Simple but Substantive Practices

In the PLOS ONE article “Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals,” the focus is a general aspect of transparency in peer review—how academic journals (and/or publishers) present their peer-review process to the public. According to the authors of this study, a transparent peer review system “conveys to readers and potential contributors how the peer review is implemented and how articles are selected for publication.” The researchers found that “author’s ratings of peer-review transparency predicted their assessment of the quality of peer-review at that journal.”  Even small changes in practices and on publisher websites can help in this area. In addition, guidelines from the Committee on Publication Ethics, “Principles of Transparency and Best Practice in Scholarly Publishing,” stating that all journal content apart from editorial material should be subject to peer review from outside experts must be met.

Learning from Early Career Researchers

Many ECRs do not get appropriate training on how to prepare or review manuscripts. Improved transparency in peer review would not only satisfy mid-to senior-level scientists, but in providing some form of open dialogue or open accountability it enables these scientists to lead by example and provide mentorship to the next generation of reviewers.

Young and upcoming scientists have plenty of ideas when it comes to improving transparency in peer review. At PLOS, we received over 150 essays on how to revamp peer review from Early Career Researchers applying for our ECR Travel Award Program. These creative young scientists described what they consider to be characteristics of the optimal peer review process and how they might build this process either from scratch or using aspects of existing practice.

Their ideas, edited for brevity, include:

  • Invite reviewers to publish reviews of the article (should they wish to reveal their identity) as an accompanying commentary, for no additional fee. If reviewers know they could gain an additional publication for their efforts, this would motivate them to review more articles and respond in a constructive and timely fashion. Victoria Leong; orcid.org/0000-0003-0666-9445
  • [Provide] incentives for reviewing that encourages kind, open but fair responses; we would also be affecting a positive change in the culture of science; which will advance the science itself. Rebecca Gelding; orcid.org/0000-0003-4883-8075
  • Reviews should be open, archived and after publication, reviewers should be revealed. This aims to ensure two aspects of quality control: reviewers take more seriously their job since it will be public with their name tag on it; and reviewing records can be used when considering career development. Juan Rocha; orcid.org/0000-0003-2322-5459
  • Review record should also be one of the criteria judging and advancing a researcher’s professional development. Knowing that a reviewer’s identify would be revealed later and shared among peers, a reviewer would have more incentive to avoid giving low-quality comments. Xiao-Peng Song; orcid.org/0000-0002-5514-0321
  • A transparent, open review process may promote accountability among reviewers. A peer reviewer whose dated comments are published as supplementary material with the article has a greater incentive to conduct a thorough and timely review of the manuscript. These same published comments could also be accessed by other researchers who are struggling to address similar issues in their own studies. Sericea Stallings-Smith; orcid.org/0000-0002-4876-9965

Slices of the Transparency Pie

Solutions that will help peer review achieve its scholarly ideal are not untenable; the challenge lies in that they must satisfy a diverse researcher and stakeholder community. While some improvements are easier to implement than others, even small slices that expedite and enrich the process of assessment in fundamentally new ways contribute to advancing science and discovery for the broader scientific community.

Publishers have an opportunity to improve both speed and efficiencies: to improve review forms that may be cumbersome or insufficient to provide thoughtful and constructive feedback to authors and to provide training for reviewers and editors that mitigate potential bias. Additional possibilities include direct or facilitated mentoring of early career researchers to improve their understanding of the principles of peer review and how it is practiced within the scientific community. Imagine the impact of a global editorial and reviewer contributor community, appropriately trained, recognized and incentivized.

For more insights on peer review listen to the following PLOScast episodes and read the following posts on The PLOS Blogs Network.
PLOScasts
PLOS BLOGS

 

Image Credit:

Etsy; FuzzyButtFarm

Plos: To Sign or Not to Sign: A Slice of Transparency in Peer Review Read More »

Plos: Tensions in Scientific Culture Contribute to Reproducibility Challenges

Leave a Comment / Articles /

“Once a system is running a certain way it’s hard to change course.”—Richard Harris, award-winning science correspondent for National Public Radio

Scientists, publishers, journalists and the public talk of the problem of reproducibility in experimental science. There are committees, symposia, peer-reviewed articles, blogs and opinion pieces documenting the issue and exploring remedies to the challenge of reproducibility. NPR science correspondent Richard Harris takes this societal challenge one step further in his book with a title that in no uncertain terms calls attention to the root cause of the reproducibility challenge—Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope and Wastes Billions.

PLOS interviewed Harris about the lack of rigor in experimental research that he describes in his book, and from a discussion that covered materials standards, publishing fewer papers with greater confidence, text and data mining and the cultural shift required to improve the rigor in science, one unifying theme emerged—tension. Tension between evaluation and culture, hands-on training and coursework, between making work public and ensuring its reliability, and between responsible reporting and satisfying the needs of a news organization.

These lines of tension contribute to the current structural rigidity in the culture of science that, according to Harris, make scientific rigor a difficult challenge to address. As he delved into biomedical sciences in 2014 after nearly a decade reporting on climate change and the environment, Harris took a broad look at the state of the research enterprise and asked, what are the consequences of limited funds on the structural influences supporting the current state of academic biomedical research?

Evaluation and Culture

The biggest obstacles influencing rigor, according to Harris, are the “underlying cultural issues” that confront science. He cites the financial crunch, career pressures and the hyper competitiveness of science, particularly in biomedical sciences, as examples. These are, he thinks, also the hardest problems to solve. “Even if you poured huge amounts of new money into biomedical research this problem would not go away quickly. It’s changing a mindset.” Fundamentally the “incentives are misaligned,” says Harris, to reward numbers of papers and how many of them are published in high profile journals, rather than “careful work where one is highly confident in results.”

Hands-on Training and Coursework

On the point of experimental responsibility, Harris’ book follows a path of practical recommendations to improve scientific rigor, including:

  • Improve experimental design and provide methods training
  • Validate cell lines, antibodies, gene constructs
  • Apply appropriate statistical analysis
  • Disclose experimental and analytical methods

During his research for the book, Harris was quite surprised to find out how little formal training there is in experimental methodology, particularly in biomedical research. He did note, however, that NIH is now funding attempts to develop curriculum, following an unsuccessful search to find the best training courses in the country to replicate. There is “huge room for improvement” in this area, he says, but how to integrate this into a training program? Changing a system that’s already in place to accommodate that is difficult, especially without reward and recognition for faculty teaching those courses. “Should it start more robustly at the end of the undergrad career?” One potential solution discussed during the interview is for post-docs to teach these courses to incoming graduate students, as part of a summer or first-quarter orientation program.

Making Work Public and Ensuring Reliability

Steps can be taken to improve the situation. For example, researchers with grants from the National Institutes of Health are required to authenticate the cell lines used in their work. “There’s no simple solution to be imposed from top down, it needs to also work bottom up,” says Harris when considering what key additional recommendations the community might consider. “Young scientists are more open to sharing and that leads to transparency, that helps solve some of the issues.” It’s not a silver bullet but it improves things if people can put their data out there, he says. Increased sharing and transparency addresses a number of these issues, and “to the extent that the culture of the young scientist is open to that, that’s great. Although it’s hard for them to change the culture, but over time this can help.”

A “more nimble publication system might encourage scientists to publish confirmatory or negative results,” Harris states. When asked specifically about the role of preprints and alternative forms of science communication, he acknowledges that experiments in openness and transparency are interesting although it’s unclear how successful they will be. Will that additional literature be less reliable, he wonders? “It’s the job of the entire community, not simply the scientist who makes a claim, to figure out what’s right and what’s wrong,” he says, when discussing the viable options of prepublication sharing of work and use of blogs and commenting as forums for peer review. However, as a science journalist, he ponders: “Do we want more literature out there or do we want more strict checks and less literature? Personally, I would like to see less literature, have people spend more time thinking about what they’re doing and being sure they’re right.”

Responsible Reporting and News Flow

Science journalists can help the issue of scientific rigor, acknowledges Harris. “They should step back from doing the story of the day. It takes more time to think about things from a broader perspective, but that’s more important than ever.” Science journalists must find a balance between that and satisfying the needs of their news organizations. Even an award-winning science correspondent like Harris admits that he needs to think differently about how he does his job on a day to day basis. “Look at what’s published and fits into the broader trend and context of similar results, and what it means elsewhere,” he recommneds. These are results that should be the focus of science journalists.

Easing the Tug of War

The good news is that many institutions now evaluate candidates for jobs and promotions based on a set limit of publications, chosen by the author to best represent their contributions to science. Additional efforts to broaden evaluation within the constraints of the existing scientific culture include recognition and credit for reagent validation, peer review activities and training others on experimental responsibility. Dedicating more time to thinking and less time to drafting and revising manuscripts may not the working philosophy of many labs or the culture of science, but it’s an issue getting attention and was expressed as a concern by leaders of the organization Rescuing Biomedical Research.

Everyone has a stake in the current structure of the scientific enterprise, says Harris, from journals caring about impact factors [PLOS de-emphasizes journal impact factors] to deans making sure scientists draw in funding and overhead for their institutions.

“The biomedical research enterprise is driven by economics; economic systems are much harder to change and that should be something the overall enterprise should be thinking about—how to rethink that.”

Harris believes clinical medicine in the 1990s experienced similar cultural stresses, but ClinicalTrials.gov, a web-based resource from the National Library of Medicine that provides public access to information about clinical studies and the availability of experimental drugs, got people “past some of these issues without making them rethink their place in the scientific universe.” Hopefully academic biomedical research can do the same.

****************

Richard Harris, science correspondent for National Public Radio (NPR) received the 2010 Kavli Science Journalism Award from the American Association for the Advancement of Science (AAAS) for his reporting on BP’s Deepwater Horizon oil spill in the Gulf of Mexico. He also received the award in 1988 and 1995. He currently covers biomedical sciences with a focus on investigative stories and in 2014 completed an eight-part series examining the stresses on biomedical research in the US caused by fluctuating funding levels.

 

Tug of War Image Credit: falco; pixabay.com

 

Plos: Tensions in Scientific Culture Contribute to Reproducibility Challenges Read More »

Plos: A Publishing Milestone to Celebrate: 200,000 PLOS Research Articles and Counting

Leave a Comment / Articles /

In 2003, PLOS published its first research article and this month we’re proud to announce that we have now published more than 200,000 research articles across our seven Open Access journals. It has been an amazing journey to reach this milestone.

“In only 14 years of existence, PLOS has helped catalyze the rapid growth of the Open Access literature. The 200,000th article is a remarkable milestone for PLOS and for the scientific community that has supported the Open Access movement.”-Veronique Kiermer, PLOS Executive Editor

We’d like to take this opportunity to thank those who have helped rapidly create this large corpus of diverse, Open Access research:

PLOS was incorporated in 2001 as an Open Access advocacy organization and became a publisher to prove the value of Open Access, beginning with PLOS Biology in 2003 and PLOS Medicine in 2004 as open alternatives to prestigious subscription journals. In 2005, PLOS launched PLOS Genetics, PLOS Pathogens, and PLOS Computational Biology as proof of concept to show that research communities built around and across specific areas and disciplines could thrive with an Open Access model. In late December 2006, PLOS ONE spearheaded the innovative editorial approach focused on evaluation of research independent of perceived impact; this editorial approach has now been adopted by journals from nearly every scientific publisher. Since its founding, PLOS ONE has published more than 175,000 research articles based on three compelling features: advancing quality science for everyone, moving publication forward in all scientific disciplines, and favoring speed to publication over subjective assessment of significance.

In 2007, with the support of the Bill & Melinda Gates Foundation, PLOS launched a fourth community journal, PLOS Neglected Tropical Diseases (NTDs). Together with the journal’s leadership and global community, we now proudly celebrate the 10th Anniversary of PLOS NTDs and the more than 4,500 research articles that contribute to advances in the field.

Open Access continues to gain momentum but there is further work to be done. At PLOS, we are constantly looking for ways to use emerging technology and new ideas to open up scientific communication—to make it faster, more efficient, more connected and more useful. We look forward to the next 200,000 articles and continued involvement with the scientific community.

Plos: A Publishing Milestone to Celebrate: 200,000 PLOS Research Articles and Counting Read More »

GoOpen.no: Workshops using Storyweaver in Nepal and Ethiopia – get involved as translator now!

Leave a Comment / Articles /

Over the last three weeks, we at the Global Digital Library have conducted workshops in Nepal and Ethiopia, as a part of the initial phase of our project. These user tests are an important part of our work as they provide us with initial user feedback on prototypes and personas. For both workshops, we have made prototypes based on a great mix if content and tech from different open sources and OER projects including resources from Storyweaver by Pratham Books. 

Localization using Storyweaver

Localization and translation will be an important part of our work and as a point of reference, we have tested both our own tool for localization and a tool developed by Storyweaver.

We at the GDL project are in the early stages of developing our platform, but if you want to join the community of translators now, you can start using Storyweaver. Our friends at Storyweaver have developed a great website with stories and books that you can read or translate into you own language.

To prepare our workshops we made this tutorial that also can serve as the first practical introduction for anyone that wants to join our movement of translators, using the Storyweaver platform. Check out this 4-minute video to get you going!

GoOpen.no: Workshops using Storyweaver in Nepal and Ethiopia – get involved as translator now! Read More »

Plos: PLOS Supports Net Neutrality to Ensure Global Access to the Scientific Literature

Leave a Comment / Articles /

PLOS works to remove barriers to public access of scientific research. Typically, these barriers are considered in terms of copyrights and journal subscriptions, but unfettered access to network infrastructure also contributes to supporting readers’ access to scientific literature. In simple terms, unencumbered dissemination of scientific research depends on a fair Internet. The provision of a fair and open Internet lies in the hands of Internet Service Providers (ISPs) and government agencies that regulate these providers. PLOS supports today’s Day of Action in the United States, led by Battle for the Net, aimed at publicizing the issues surrounding Open Internet Rules and their critical role in maintaining Internet freedoms as currently in place.

In February 2015, the US Federal Communication Commission (FCC) reclassified broadband providers as common carriers rather than information providers under Title II of the Communications Act of 1934, the law that continues to regulate modern communication modes such as the Internet. This gave the FCC authority to ensure that established, large corporations including AT&T, Comcast and Verizon can’t block, slow (throttle) or otherwise interfere with Internet traffic. Innovations in online businesses and services, including those of PLOS, have thrived under Title II regulations. Importantly, these regulations in the US provide each and every user of the Internet a guarantee that ISPs and government regulators treat all data on the Internet the same, without discriminating against or charging differentially by user, content, website, platform, application, type of equipment or mode of communication (see https://en.wikipedia.org/wiki/Net_neutrality). These protections are known as net neutrality; scientists and those working to support the scientific endeavor rely on net neutrality for unprejudiced access to databases, the literature and information services.

Allowing ISPs to sort traffic based on content, sender and receiver opens the door for corporate and government censorship which would greatly hinder access to scientific information around the globe.

To protect against this type of restriction in information flow, the first EU-wide Net Neutrality rules were adopted in October 2015 with public guidelines released by the Body of European Regulators for Electronic Communications in late 2016.

In the US, the current FCC Commissioner wants to weaken these protections and this could have consequences for all scientists, not just those in the US: access to information around the world could become pay to play without these protections. Services provided by publishers such as PLOS and other providers could be restricted for all our users around the world, unless we pay for priority access to our content. This will affect any Internet traffic that routes through the US, from services relying on servers located in the US to requests that are routed through the US. For instance, most of the Domain Name Server (DNS) requests for South America, Central America and the Carribbean are routed by servers located in Florida.

Today, July 12, 2017, is a Day of Action in support of a fair and open Internet. Visitors to the PLOS.org homepage (www.plos.org) and active PLOS staff blogs (The Official PLOS Blog, EveryONE, PLOS Biologue, Speaking of Medicine, PLOS Channels and Collections, PLOS Tech and PLOS Podcasts) will see a message of explanation and letter of support for net neutrality in the form of a pop up window. This message will appear only today, once per site, per device. Visitors to these web pages can choose to either immediately close the pop up by clicking on the x in the upper right or fill in the four boxes to send a message to the FCC in support of its current Open Internet Rules and the efforts to dissuade FCC Chairman Ajit Varadaraj Pai from his plan to roll back these rules. Public comment continues for an additional 60 days following the Day of Action. Form letters or personalized comments may be sent directly from https://www.battleforthenet.com/.

PLOS is purposefully not placing this notice on any journal content or information pages, so that access to our content is not impeded in any way.

To learn more about net neutrality around the globe, visit https://www.thisisnetneutrality.org/; to learn more about the Day of Action visit https://www.battleforthenet.com/july12/.

Plos: PLOS Supports Net Neutrality to Ensure Global Access to the Scientific Literature Read More »

Plos: Immediate and Lasting Impact: Top Ten New Species of 2017

Leave a Comment / Articles /

The unique morphology of the spiny ant, uncovered with the use of an advanced form of 3D microscopy known as 3D X-ray microtomography, garnered this species a place this year among the Top 10 New Species of 2017. The Top 10 New Species list honors the legacy of Swedish botanist Carolus Linnaeus, known for his pioneering work on the hierarchical classification of plants and animals that developed beyond genus and species into modern taxonomy. The list is compiled by the International Institute for Species Exploration (IISE) working with a panel of international scientists as selection committee members.

SUNY-ESF President Quentin Wheeler. (Photos for SUNY-ESF by M.J. Okoniewski)

PLOS spoke with Quentin Wheeler, founding director of IISE and president of the College of Environmental Science and Forestry in Syracuse, New York, on assessing research impact, the purpose and influence of this list and the impact of climate change on species diversity.

Policies and Purpose

Research reporting these new species is published in a variety of journal types, from subscription to Open Access. When asked if there are considerations of access to research when the members of the international selection committee evaluate nominations, Wheeler states that “the selection committee is encouraged to focus on the organisms rather than where they were published.” This de-emphasis on journal choice is a hallmark of the IISE selection process; work that is posted on a preprint server, rather than published in a peer-reviewed journal, may also be eligible for consideration. “What matters is compliance with the international codes of (botanical and zoological) nomenclature which requires publication that can now include electronic publication. So long as the requirements of the code were met during the previous calendar year, a species is eligible for consideration,” he says.

In an era of continued global extinction of animals, plants and microbes, Wheeler hopes the Top 10 New Species list brings research outcomes to the public that help convey the acute relevance of climate change. “Our goal is to increase awareness of the loss of species in the biodiversity crisis and the important roles played by taxonomy and natural history museums in biodiversity exploration and conservation. The wide media attention gained by the Top 10 (and this year’s PLOS ONE article) has hopefully played a role toward that goal,” he says. While he is not aware of specific policies shaped directly by the list, the hope is that it “keeps the importance of species exploration in the minds of those making such policies.”

Valuing Work, Not Impact Factor

Over the past 10 years of generating this list, one notable detail is that selected work is published in journals with impact factors ranging from less than one to greater than 20. When asked what this says to him and the scientific community about the value and relevance of evaluating a work based on its own merit, rather than on the journal in which it appears, Wheeler is quite direct.

“As a scientist and scholar, I like to think that science is a meritocracy of ideas and that their value derives from the quality of the work and its impact rather than the impact factor of the journal overall. Taxonomy is a very special case that is not at all served well by impact factors as they are today calculated.”

He explains this concept more fully. “First, the best taxonomic work is comprehensive and comparative in the form of lengthy taxonomic revisions and monographs. Such long works are not accepted by most journals with high impact factors. Second, even the best taxonomic work is rarely cited because once species are known they are typically identified by field biologists through secondary literature (field guides, etc.). Even the secondary literature is often omitted from citations by ecologists and others, and the primary literature is only rarely cited outside of other taxonomy papers.” Due to the long-term nature of taxonomy work, “we routinely consult papers from 1758 forward,” says Wheeler. “Thus the true impact of the work is measured over generations which is quite different from most experimental fields where papers are outdated in just a few years.”

Last year two species on the list made their debuts in PLOS ONE: the description of a new Galapagos giant tortoise species and a new genus, species and subfamily of isopod crustacean. Together with this year’s winner and the four PLOS ONE articles describing top species in 2014, this cohort of articles has collectively received over 290,000 views, 3,500 shares and broad media coverage since publication, indicating their influence and interest for taxonomists and the public at large.

Connectivity and Credit

As with other scientific disciplines, innovation and modernization are a must in taxonomy, and Wheeler is in favor of such policy shifts. First, he recommends mandating a “central deposition of all nomenclatural acts, including descriptions of new species” as it now takes several years to track down all new species named in any given year. He also believes “the actual technical description of species should be open access, even in journals that are not. The descriptive material should be intellectual property of humankind and available to everyone.”

Further modernization, according to Wheeler, would establish electronic connectivity between published work and the central repository. “That central repository, likely IPNI for botany and ZooBank for zoology,” he says, “ought to be connected via hyperlink to every scientific name published. Were this done it would be a service to editors by assuring the correct scientific name is being used and spelled correctly. Also by virtue of that link we could electronically track the usage of scientific names and give “impact” measure of the use of the names as credit to the taxonomists.”

PLOS encourages scientists making these discoveries to publish the entire research article, not just the technical data, in fully Open Access journals and repositories to ensure the work has maximum visibility and reach. ORCID iDs can help in linking taxonomy descriptions, datasets, published work and grants to individual researchers for maximum credit and recognition. Those interested in learning more about the biodiversity crisis can watch a brief interview with Quentin Wheeler and those interested in learning more about new species and biodiversity can browse this selection of PLOS articles.

****************

Quentin Wheeler is founding director of the International Institute for Species Exploration and president of the College of Environmental Science and Forestry (ESF) in Syracuse, New York. He was previously vice president and dean of the College of Liberal Arts and Sciences at Arizona State University, chair of entomology and director of the Liberty Hyde Bailey Hortorium at Cornell University, Head of Entomology at the Natural History Museum in London and Director of the Division of Environmental Biology at the National Science Foundation.

 

Hero Image Credit: ESF

 

Plos: Immediate and Lasting Impact: Top Ten New Species of 2017 Read More »

Plos: Getting the Impact Factor Genie Back in the Box

Leave a Comment / Articles /

On occasion The Official PLOS Blog presents Thought Leadership interviews with scientists leading the way on issues integral to the transformation of science communication and advancement of Open Science. Previous interviewees include Bruce Alberts and Trevor Bedford. Here we present our conversation with Sandra Schmid from University of Texas Southwestern Medical Center.

 

Image Credit: University of Texas Southwestern Medical Center

Over the years, Sandra Schmid has gained a reputation for academic strength and leadership, most recently as Professor and Chairman of the cell biology department at University of Texas Southwestern (UTSW) Medical Center. She’s also gained a reputation for her honesty regarding varied issues, including the position of post-docs, “if it were a job, we’d pay you better and give you retirement benefits,” the training of faculty, “few of us as mentors, as Principal Investigators, were ever taught how to run a lab or how to mentor individuals” and how she participates in open discussion of research before publication “mostly over beers.”

Schmid has been particularly vocal about the misuse of journal impact factors (JIFs) as a way to evaluate researchers and, as she claims, “the unfortunate consequences to the scientific community of their misuse.” At UTSW, Schmid’s home institution, there has been no overt discussion among the leadership regarding JIFs and where faculty should choose to publish. There is no formalized preference for high impact journals. “In fact, we celebrated the founding of eLIFE [a journal which rejects the use of JIFs] and have faculty on the Editorial Board of the journal,” says Schmid. The JIF was “never intended to and indeed does not measure the quality or impact of the individual papers in a journal,” says Schmid. It was originally developed and commercialized by Eugene Garfield to help librarians decide on which journals they should spend their subscription dollars.

“Individuals and institutions are being spuriously judged – by other scientists, funders, governing bodies and administrators – based indirectly on JIF, rather than directly on the quality and impact of their work,” Schmid wrote in “Negative Consequences of the Misuse of Journal Impact Factors for Scientific Assessment” as part of the 8th Forum on the Internationalization of Sciences and Humanities.

Flawed Statistics

The JIF is a statistic calculated based on the average citations of a selection of papers in a given journal. One major problem with the JIF is that citations are highly skewed, with most articles receiving fewer citations. Since citation distributions are skewed, averages are meaningless. “Indeed,” writes Schmid, there are journals that “flaunt their JIF in marketing material to authors that would ironically not accept papers reporting such flawed statistics.” This skewed distribution was clearly demonstrated last year through a collaboration between multiple publishers, including Université de Montréal, Imperial College London, PLOS, eLife, EMBO Journal, The Royal Society, Nature and Science (see Measuring Up: Impact Factors Do Not Reflect Article Citation Rates). The analysis, posted on bioRxiv, showed that citation distributions of journals with clearly distinct impact factors greatly overlap—in other words that all journals publish many papers with similar lower numbers of citations, and few highly cited papers.

A Better Option: Citation Distributions

The authors of the bioRxiv analysis call for publishers to make publicly available the actual citation distributions of their journal’s articles, rather than rely on irrelevant and misleading JIFs. Since journals use many different techniques to artificially increase their impact factor, including publication of review articles (which are often more highly cited than the original research papers they review) and front matter, including commentary and mini-review articles (that generate citations but are not counted as “citable” content) comparison across journals is problematic. It is hoped that public disclosure of article citation distributions will lead to more granular comparisons and better informed decisions by authors on where to submit their work.

Then and Now

From the perspective of a senior investigator with a long-established career and history of publishing quality work at all tiers of influence, what has changed for Schmid when deciding where to publish is that in the past, “journals had different purposes and different scopes” and that was good. Before there was the JIF there was an understanding of what journal went with what type of data. “We sent our best biochemistry to Journal of Biological Chemistry; our best cell biology to Journal of Cell Biology. If we happened upon a new and potentially important discovery, even before we understood mechanism, we’d communicate it rapidly in Science and Nature because they were three figure papers.” Before the advent of supplemental materials, more meaty, in-depth studies were published in non-page limited, subject-specific journals.

When asked in the post-print era, how do researchers decide where to publish, Schmid replies, “That is the unfortunate part.” A lot of the decisions are being made by postdocs telling her about impact factors, although she cautions that “publishing in high impact factor journals doesn’t mean it’s high quality work.” Early career researchers are looking at numbers as a distinguisher between journals, says Schmid, so her efforts are focused on getting these scientists to think more broadly. Her response and recommendation? First and foremost is to choose the journal where the work will get in front of the audience that matters the most. Schmid is crystal clear when outlining her main considerations for deciding where to publish her work and the work from her lab:

  • Are the people who handle my paper able to identify qualified referees?
  • Are the editors going to understand the discussion and criticisms and be helpful in handling my papers; do they understand my field?
  • Do my peers read and respect the content in this journal?

Unintended Consequences

The real question for Schmid is how to get the “impact factor genie” as she calls it, “back in the box.” Why is this so important? Scientists and publishers often focus on the limitations of JIFs and the benefits of evaluating work at the article rather than journal level. However, there are more than just limitations to the JIF. According to Schmid there are very “specific and unintended consequences of the abuse of JIF as a tool for individual and institutional assessment.” Many of these, she notes, are direct; others are subtle, downstream ramifications:

  • Deferred communication of discoveries that might launch new fields as reviewers and editors demand more information per paper
  • Discouraged follow-up or augmentative studies to verify results due to over-interpretation of findings for the purpose of artificially inflating a work’s value
  • Misguided evaluation of graduate students, postdoctoral fellows and junior faculty by their individual papers rather than the combinatorial impact of their work in context
  • Wasted time and resources spent satisfying unnecessary demands of reviewers and editors in high-impact journals
  • Demoralized early career researchers forced to package an entire thesis or postdoctoral project into one comprehensive paper

A Better Option: Article-Level Metrics

Perhaps wanting to get that impact factor genie back in the box was more than a mistaken mixing of two idioms. The difficulty of reverting to a situation that formerly existed (putting the genie back in the bottle) combined with the repercussions of doing something that causes unexpected and unintended negative consequences (opening a Pandora’s box) does describe the situation the scientific community has with JIFs. Fortunately, this is not an impossible situation to remedy. Article-Level Metrics were developed by PLOS as a better means to assess research value in an electronically networked world. They are gaining acceptance across a broad swath of the scientific community, from scientists to funders and more, since they provide granularity, breadth and proximity (PLOS ALMs are updated daily to monthly, depending on source and age of the individual article). ALMs also allow different scholarly research outputs to be tracked, such as policy impact, datasets, software and code. Schmid also recommends simply using PubMed as a portal for assessing the influence of an article, stating, “from title to abstract to download is a good metric,” although not as complete as a suite of ALMs.

Leadership in Practice

In 2013 as Schmid took up the position of Chair of the Department of Cell Biology at UTSW, she offered an employer’s manifesto (published as a Science Careers column) on the approach her department would take in hiring new assistant professors. This manifesto promised “a better job of screening applicants—and to avoid inappropriate criteria such as journal impact factors.” The idea was to encourage applications from qualified candidates who “might feel sidelined because their paper has yet to be, or perhaps won’t be, published in a high-impact journal.” Schmid closed her column with an enthusiastic “Let’s run this experiment!” Four years later, she shared some of the results with PLOS. Using their Academic Jobs portal the entire faculty is engaged in viewing applicants and every candidate that has piqued the interest of even one faculty member is interviewed via Skype, removing the need for reaching a ‘consensus’ that might rely more on JIFs. Those few candidates whose programs are most likely to thrive in the department’s specific environment are invited to campus to visit. Since taking this approach “our new faculty are indeed thriving,” says Schmid.

This approach suggests that a reduction on emphasis of JIFs in favor of more constructive and meaningful measures of evaluation, both quantitative and qualitative, fosters an assessment program that is both fair and thoughtful. This is how science should be; if it works for people it can work for research outputs as well.

*************

Sandra Schmid is Cecil H. Green Distinguished Chair in Cellular and Molecular Biology, Professor and Chairman, Department of Cell Biology at the University of Texas Southwestern Medical Center. She was co-founding editor of Traffic, Editor-in-Chief of Molecular Biology of the Cell and president of the American Society for Cell Biology. Schmid was elected fellow of the American Association for the Advancement of Science and Vice-Chair of the European Molecular Biology Laboratory Scientific Advisory Committee.

Plos: Getting the Impact Factor Genie Back in the Box Read More »

Plos: PLOS Appoints Alison Mudditt Chief Executive Officer

Leave a Comment / Articles /

PLOS is pleased to announce the appointment of Alison Mudditt as its Chief Executive Officer, effective June 19, 2017.  For the past six years Mudditt served as Director of University of California Press (UC Press) where she ushered in new strategies to lead the company into the digital age, including the innovative journal and monograph Open Access programs Collabra and Luminos. Prior to UC Press, Mudditt was Executive Vice President at SAGE Publications, Inc., leading publishing programs across books, journals and digital platforms. Her 25 plus years in the publishing industry include leadership positions at Blackwell Publishers in Oxford, UK, and Taylor & Francis Inc., in Philadelphia, US. Mudditt received her Bachelor of Arts degree from the University of Bath and her Masters in Business Administration from The Open University.

“PLOS is truly privileged to bring on board a person of Alison’s caliber whose extensive experience in and deep knowledge of academic publishing will invigorate a renewed focus on our mission—transforming research communication to better serve authors, readers and the public,” said Gary Ward, Chairman of the Board at PLOS. “Her history and accomplishments as a leader, coalition builder and strategic thinker for organizations experiencing change is impressive and will prove very valuable to PLOS in the years ahead.”

“I am delighted to join a like-minded publisher in PLOS, which fully embraces the Open Access principles that drive its mission,” said Mudditt. “PLOS’ long-held reputation as a change agent in this industry provides a tremendously gratifying challenge as we continue to push the boundaries of what is possible in scientific publishing.”

Plos: PLOS Appoints Alison Mudditt Chief Executive Officer Read More »

Plos: Beyond Slogans: After the March for Science Has Passed

Leave a Comment / Articles /

“Science and democracy are logical allies, they both flourish with an open network of ideas, evidence and an uncompromising examination of results.” – Cindy Schaffer, former microbiologist with the Environmental Protection Agency

Clever slogans, such as “May the Facts Be with You” or “There Is No Planet B” flourished at the Earth Day March for Science as scientists and supporters of science around the globe chanting “Less Invasions More Equations” and “All Six American Nobel Prize Winners Last Year Were Immigrants” took to the streets demanding attention to the tangible impact of science on humans “Do You Have Polio? Thank A Scientist” and the environment “Ice Has No Agenda, It Just Melts.”

It is powerful for scientists and science-based organizations to show how important scientific facts are to everyday life, irrespective of political alignment, but what happens to this momentum after the collective advocacy effort has passed? What difference does it make, many have asked, and what role should each of us play, considering daily professional and personal commitments and demands on our attention and time. These are the issues faced by every social movement in search of long-lasting tangible impact.

Science Not Silence

The official slogan of March for Science, “Science Not Silence,” is a phrase that has potential to propel advocacy beyond the day of the march. “Because the results of scientific research benefit our everyday lives, we have taken for granted that science would be a vital, respected part of discussions about societal issues that impact health, the environment, technology and other science-based issues,” says Erika Shugart, Executive Director at the American Society for Cell Biology (ASCB).

“This is no longer the case. If science is not represented and advocated for, then it will be ignored or, even worse, replaced by discredited information,” she continues. “We can no longer be silent and assume science will be at the table, we must stand up for science.”

“My interpretation [of “Science Not Silence”] is that we need to use science and evidence-based scientific results to inform government policy and that we, as scientists, need to speak up about our research rather than staying silent,” says Elizabeth Blaber, Visiting Research Scientist at NASA Ames Research Center in Mountain View. “More often than not, scientists get caught up in their research, writing grants and publishing papers; we forget that our results can really make a difference to the general population. We can use our results to help inform policymakers about important decisions that they need to make about climate policy, research funding, health care policies and the next step for NASA’s human exploration endeavors,” she says.

Blaber cites the unique insight scientists working as government contractors have into government science and the bureaucracy of government agencies. “Science not silence means education to us, educating policymakers on the importance of each scientific study that is being conducted in and outside of government agencies and how these studies collectively make all of our lives better in an unimaginable number of ways.”

Beyond the March

How we harness the excitement and momentum of the march and translate that into action is not a single solution for each scientist or scientific discipline. “When you ask ‘what next?’ that’s when I run into difficulty,” says Dan Schaffer, former National Oceanic and Atmospheric Administration Weather Research Software Engineer. “We can march every week all day long but in the end, there are far more important and difficult decisions we have to make if we are to do something significant about the issue of climate change, for example.” For some scientists, this means bringing scientific evidence into daily habits (and convincing others to do the same). “Here in Prius driving, solar panel powered Boulder, Colorado, folks like to talk about what we’re doing about climate change.” Yet some of these same climate scientists “fly as far as Australia for climate change conferences,” he says. “One round trip from Denver to London is equivalent to driving one of those Prius 9,000 miles. That trip to Sydney? 16,000 miles.”

Individual behavioral changes can, collectively, make an impact. March for Science provides easy opportunities on their Week of Action page. For example, by clicking on ‘Science Connects Saturday’ (available everyday) you can send an email to your representatives just by filling in a few form fields. The relevant representative is determined automatically by your zip code entry. ‘Science Discovers Monday’ leads to suggestions for game night fun, science-style.

For some, follow up from the March for Science means being more open and emphatic, publicly, about what is evidence-based science and what is not. “As a former microbiologist at the Environmental Protection Agency, it disheartens me to hear that we have to prove that science matters,” says Cindy Schaffer. “The more active we, as scientists, can be in promoting real science,” she continues, “the better chance we will have for the false news to remain false in the general public’s mind.”

The message “Science Is Nonpartisan” took to the streets as a demonstration of “This Is What Democracy Looks Like.” These slogans call out for participation of the public in open, honest and constructive discussion. Says ASCB’s Shugart, “It is up to scientific societies and other organizations to help harness this energy to be a force for good in our communities.” Several organizations make it easy to participate, as we choose, in the democratization of science. Visit the PLOS Stand Up for Science page to learn how; email communications@plos.org if your professional association or society is taking action and wants to be listed on Stand Up for Science.

 

Image Credit: Bob Hemstock

Plos: Beyond Slogans: After the March for Science Has Passed Read More »

GoOpen.no: NDLA technology reused by 8000 websites worldwide

Leave a Comment / Articles /

Over the last couple of years, the NDLA team have been working to replace Flash based applications and interactive learning objects. NDLA also needed a tool to make it easy to create, share and reuse HTML5 content and applications. We started developing a new tool in public-private partnership with Joubel, a tech startup in Tromsø, in the northern part of Norway. This collaboration ended up as a project and product called H5P.

H5P is at the time of writing installed on over 8,000 websites. H5P is reused by many universities, large companies and smaller personal websites worldwide. It´s great to see this kind of reuse and in the long run, this will make the platform more sustainable, also for NDLA.

The team developing and designing H5P have been set up with the best product developers from NDLA and designers and developers from Joubel. This kind of public-private partnership is essential to NDLAs innovation process.

In H5P, all you need is a web browser and a website with an H5P plugin. H5P empowers creatives to create rich and interactive web experiences more efficiently.

H5P is a free and open source tool that helps you create HTML5 content in the browser of your choice and share it across all operating systems and browsers. Check out the list of different content types.

As H5P is open source there are no “strings attached”. Anyone can reuse both content and technology without asking Joubel or NDLA for permission. One of the universities that have reused H5P is Colorado.

How to use H5P?

H5P is a plugin for existing CMS and Learning Management Systems (LMS) systems like WordPress and Drupal. Just install the H5P and your system becomes able to create, share, and reuse great interactive content. For systems that don’t have an H5P plugin available yet it is possible to embed content using an iframe or using the Learning Tool Interoperability (LTI) standard. With the LTI and supporting APIs and specifications embedding an externally hosted H5P authoring tool is also possible.

The H5P format is open and the tools for creating H5P content are open source. This guarantees that creatives own their own content and are not locked into the fate and licensing regime of a specific tool.

 

GoOpen.no: NDLA technology reused by 8000 websites worldwide Read More »