Plos: Getting the Impact Factor Genie Back in the Box

On occasion The Official PLOS Blog presents Thought Leadership interviews with scientists leading the way on issues integral to the transformation of science communication and advancement of Open Science. Previous interviewees include Bruce Alberts and Trevor Bedford. Here we present our conversation with Sandra Schmid from University of Texas Southwestern Medical Center.

 

Image Credit: University of Texas Southwestern Medical Center

Over the years, Sandra Schmid has gained a reputation for academic strength and leadership, most recently as Professor and Chairman of the cell biology department at University of Texas Southwestern (UTSW) Medical Center. She’s also gained a reputation for her honesty regarding varied issues, including the position of post-docs, “if it were a job, we’d pay you better and give you retirement benefits,” the training of faculty, “few of us as mentors, as Principal Investigators, were ever taught how to run a lab or how to mentor individuals” and how she participates in open discussion of research before publication “mostly over beers.”

Schmid has been particularly vocal about the misuse of journal impact factors (JIFs) as a way to evaluate researchers and, as she claims, “the unfortunate consequences to the scientific community of their misuse.” At UTSW, Schmid’s home institution, there has been no overt discussion among the leadership regarding JIFs and where faculty should choose to publish. There is no formalized preference for high impact journals. “In fact, we celebrated the founding of eLIFE [a journal which rejects the use of JIFs] and have faculty on the Editorial Board of the journal,” says Schmid. The JIF was “never intended to and indeed does not measure the quality or impact of the individual papers in a journal,” says Schmid. It was originally developed and commercialized by Eugene Garfield to help librarians decide on which journals they should spend their subscription dollars.

“Individuals and institutions are being spuriously judged – by other scientists, funders, governing bodies and administrators – based indirectly on JIF, rather than directly on the quality and impact of their work,” Schmid wrote in “Negative Consequences of the Misuse of Journal Impact Factors for Scientific Assessment” as part of the 8th Forum on the Internationalization of Sciences and Humanities.

Flawed Statistics

The JIF is a statistic calculated based on the average citations of a selection of papers in a given journal. One major problem with the JIF is that citations are highly skewed, with most articles receiving fewer citations. Since citation distributions are skewed, averages are meaningless. “Indeed,” writes Schmid, there are journals that “flaunt their JIF in marketing material to authors that would ironically not accept papers reporting such flawed statistics.” This skewed distribution was clearly demonstrated last year through a collaboration between multiple publishers, including Université de Montréal, Imperial College London, PLOS, eLife, EMBO Journal, The Royal Society, Nature and Science (see Measuring Up: Impact Factors Do Not Reflect Article Citation Rates). The analysis, posted on bioRxiv, showed that citation distributions of journals with clearly distinct impact factors greatly overlap—in other words that all journals publish many papers with similar lower numbers of citations, and few highly cited papers.

A Better Option: Citation Distributions

The authors of the bioRxiv analysis call for publishers to make publicly available the actual citation distributions of their journal’s articles, rather than rely on irrelevant and misleading JIFs. Since journals use many different techniques to artificially increase their impact factor, including publication of review articles (which are often more highly cited than the original research papers they review) and front matter, including commentary and mini-review articles (that generate citations but are not counted as “citable” content) comparison across journals is problematic. It is hoped that public disclosure of article citation distributions will lead to more granular comparisons and better informed decisions by authors on where to submit their work.

Then and Now

From the perspective of a senior investigator with a long-established career and history of publishing quality work at all tiers of influence, what has changed for Schmid when deciding where to publish is that in the past, “journals had different purposes and different scopes” and that was good. Before there was the JIF there was an understanding of what journal went with what type of data. “We sent our best biochemistry to Journal of Biological Chemistry; our best cell biology to Journal of Cell Biology. If we happened upon a new and potentially important discovery, even before we understood mechanism, we’d communicate it rapidly in Science and Nature because they were three figure papers.” Before the advent of supplemental materials, more meaty, in-depth studies were published in non-page limited, subject-specific journals.

When asked in the post-print era, how do researchers decide where to publish, Schmid replies, “That is the unfortunate part.” A lot of the decisions are being made by postdocs telling her about impact factors, although she cautions that “publishing in high impact factor journals doesn’t mean it’s high quality work.” Early career researchers are looking at numbers as a distinguisher between journals, says Schmid, so her efforts are focused on getting these scientists to think more broadly. Her response and recommendation? First and foremost is to choose the journal where the work will get in front of the audience that matters the most. Schmid is crystal clear when outlining her main considerations for deciding where to publish her work and the work from her lab:

  • Are the people who handle my paper able to identify qualified referees?
  • Are the editors going to understand the discussion and criticisms and be helpful in handling my papers; do they understand my field?
  • Do my peers read and respect the content in this journal?

Unintended Consequences

The real question for Schmid is how to get the “impact factor genie” as she calls it, “back in the box.” Why is this so important? Scientists and publishers often focus on the limitations of JIFs and the benefits of evaluating work at the article rather than journal level. However, there are more than just limitations to the JIF. According to Schmid there are very “specific and unintended consequences of the abuse of JIF as a tool for individual and institutional assessment.” Many of these, she notes, are direct; others are subtle, downstream ramifications:

  • Deferred communication of discoveries that might launch new fields as reviewers and editors demand more information per paper
  • Discouraged follow-up or augmentative studies to verify results due to over-interpretation of findings for the purpose of artificially inflating a work’s value
  • Misguided evaluation of graduate students, postdoctoral fellows and junior faculty by their individual papers rather than the combinatorial impact of their work in context
  • Wasted time and resources spent satisfying unnecessary demands of reviewers and editors in high-impact journals
  • Demoralized early career researchers forced to package an entire thesis or postdoctoral project into one comprehensive paper

A Better Option: Article-Level Metrics

Perhaps wanting to get that impact factor genie back in the box was more than a mistaken mixing of two idioms. The difficulty of reverting to a situation that formerly existed (putting the genie back in the bottle) combined with the repercussions of doing something that causes unexpected and unintended negative consequences (opening a Pandora’s box) does describe the situation the scientific community has with JIFs. Fortunately, this is not an impossible situation to remedy. Article-Level Metrics were developed by PLOS as a better means to assess research value in an electronically networked world. They are gaining acceptance across a broad swath of the scientific community, from scientists to funders and more, since they provide granularity, breadth and proximity (PLOS ALMs are updated daily to monthly, depending on source and age of the individual article). ALMs also allow different scholarly research outputs to be tracked, such as policy impact, datasets, software and code. Schmid also recommends simply using PubMed as a portal for assessing the influence of an article, stating, “from title to abstract to download is a good metric,” although not as complete as a suite of ALMs.

Leadership in Practice

In 2013 as Schmid took up the position of Chair of the Department of Cell Biology at UTSW, she offered an employer’s manifesto (published as a Science Careers column) on the approach her department would take in hiring new assistant professors. This manifesto promised “a better job of screening applicants—and to avoid inappropriate criteria such as journal impact factors.” The idea was to encourage applications from qualified candidates who “might feel sidelined because their paper has yet to be, or perhaps won’t be, published in a high-impact journal.” Schmid closed her column with an enthusiastic “Let’s run this experiment!” Four years later, she shared some of the results with PLOS. Using their Academic Jobs portal the entire faculty is engaged in viewing applicants and every candidate that has piqued the interest of even one faculty member is interviewed via Skype, removing the need for reaching a ‘consensus’ that might rely more on JIFs. Those few candidates whose programs are most likely to thrive in the department’s specific environment are invited to campus to visit. Since taking this approach “our new faculty are indeed thriving,” says Schmid.

This approach suggests that a reduction on emphasis of JIFs in favor of more constructive and meaningful measures of evaluation, both quantitative and qualitative, fosters an assessment program that is both fair and thoughtful. This is how science should be; if it works for people it can work for research outputs as well.

*************

Sandra Schmid is Cecil H. Green Distinguished Chair in Cellular and Molecular Biology, Professor and Chairman, Department of Cell Biology at the University of Texas Southwestern Medical Center. She was co-founding editor of Traffic, Editor-in-Chief of Molecular Biology of the Cell and president of the American Society for Cell Biology. Schmid was elected fellow of the American Association for the Advancement of Science and Vice-Chair of the European Molecular Biology Laboratory Scientific Advisory Committee.

Plos: PLOS Appoints Alison Mudditt Chief Executive Officer

PLOS is pleased to announce the appointment of Alison Mudditt as its Chief Executive Officer, effective June 19, 2017.  For the past six years Mudditt served as Director of University of California Press (UC Press) where she ushered in new strategies to lead the company into the digital age, including the innovative journal and monograph Open Access programs Collabra and Luminos. Prior to UC Press, Mudditt was Executive Vice President at SAGE Publications, Inc., leading publishing programs across books, journals and digital platforms. Her 25 plus years in the publishing industry include leadership positions at Blackwell Publishers in Oxford, UK, and Taylor & Francis Inc., in Philadelphia, US. Mudditt received her Bachelor of Arts degree from the University of Bath and her Masters in Business Administration from The Open University.

“PLOS is truly privileged to bring on board a person of Alison’s caliber whose extensive experience in and deep knowledge of academic publishing will invigorate a renewed focus on our mission—transforming research communication to better serve authors, readers and the public,” said Gary Ward, Chairman of the Board at PLOS. “Her history and accomplishments as a leader, coalition builder and strategic thinker for organizations experiencing change is impressive and will prove very valuable to PLOS in the years ahead.”

“I am delighted to join a like-minded publisher in PLOS, which fully embraces the Open Access principles that drive its mission,” said Mudditt. “PLOS’ long-held reputation as a change agent in this industry provides a tremendously gratifying challenge as we continue to push the boundaries of what is possible in scientific publishing.”

Plos: Beyond Slogans: After the March for Science Has Passed

“Science and democracy are logical allies, they both flourish with an open network of ideas, evidence and an uncompromising examination of results.” – Cindy Schaffer, former microbiologist with the Environmental Protection Agency

Clever slogans, such as “May the Facts Be with You” or “There Is No Planet B” flourished at the Earth Day March for Science as scientists and supporters of science around the globe chanting “Less Invasions More Equations” and “All Six American Nobel Prize Winners Last Year Were Immigrants” took to the streets demanding attention to the tangible impact of science on humans “Do You Have Polio? Thank A Scientist” and the environment “Ice Has No Agenda, It Just Melts.”

It is powerful for scientists and science-based organizations to show how important scientific facts are to everyday life, irrespective of political alignment, but what happens to this momentum after the collective advocacy effort has passed? What difference does it make, many have asked, and what role should each of us play, considering daily professional and personal commitments and demands on our attention and time. These are the issues faced by every social movement in search of long-lasting tangible impact.

Science Not Silence

The official slogan of March for Science, “Science Not Silence,” is a phrase that has potential to propel advocacy beyond the day of the march. “Because the results of scientific research benefit our everyday lives, we have taken for granted that science would be a vital, respected part of discussions about societal issues that impact health, the environment, technology and other science-based issues,” says Erika Shugart, Executive Director at the American Society for Cell Biology (ASCB).

“This is no longer the case. If science is not represented and advocated for, then it will be ignored or, even worse, replaced by discredited information,” she continues. “We can no longer be silent and assume science will be at the table, we must stand up for science.”

“My interpretation [of “Science Not Silence”] is that we need to use science and evidence-based scientific results to inform government policy and that we, as scientists, need to speak up about our research rather than staying silent,” says Elizabeth Blaber, Visiting Research Scientist at NASA Ames Research Center in Mountain View. “More often than not, scientists get caught up in their research, writing grants and publishing papers; we forget that our results can really make a difference to the general population. We can use our results to help inform policymakers about important decisions that they need to make about climate policy, research funding, health care policies and the next step for NASA’s human exploration endeavors,” she says.

Blaber cites the unique insight scientists working as government contractors have into government science and the bureaucracy of government agencies. “Science not silence means education to us, educating policymakers on the importance of each scientific study that is being conducted in and outside of government agencies and how these studies collectively make all of our lives better in an unimaginable number of ways.”

Beyond the March

How we harness the excitement and momentum of the march and translate that into action is not a single solution for each scientist or scientific discipline. “When you ask ‘what next?’ that’s when I run into difficulty,” says Dan Schaffer, former National Oceanic and Atmospheric Administration Weather Research Software Engineer. “We can march every week all day long but in the end, there are far more important and difficult decisions we have to make if we are to do something significant about the issue of climate change, for example.” For some scientists, this means bringing scientific evidence into daily habits (and convincing others to do the same). “Here in Prius driving, solar panel powered Boulder, Colorado, folks like to talk about what we’re doing about climate change.” Yet some of these same climate scientists “fly as far as Australia for climate change conferences,” he says. “One round trip from Denver to London is equivalent to driving one of those Prius 9,000 miles. That trip to Sydney? 16,000 miles.”

Individual behavioral changes can, collectively, make an impact. March for Science provides easy opportunities on their Week of Action page. For example, by clicking on ‘Science Connects Saturday’ (available everyday) you can send an email to your representatives just by filling in a few form fields. The relevant representative is determined automatically by your zip code entry. ‘Science Discovers Monday’ leads to suggestions for game night fun, science-style.

For some, follow up from the March for Science means being more open and emphatic, publicly, about what is evidence-based science and what is not. “As a former microbiologist at the Environmental Protection Agency, it disheartens me to hear that we have to prove that science matters,” says Cindy Schaffer. “The more active we, as scientists, can be in promoting real science,” she continues, “the better chance we will have for the false news to remain false in the general public’s mind.”

The message “Science Is Nonpartisan” took to the streets as a demonstration of “This Is What Democracy Looks Like.” These slogans call out for participation of the public in open, honest and constructive discussion. Says ASCB’s Shugart, “It is up to scientific societies and other organizations to help harness this energy to be a force for good in our communities.” Several organizations make it easy to participate, as we choose, in the democratization of science. Visit the PLOS Stand Up for Science page to learn how; email communications@plos.org if your professional association or society is taking action and wants to be listed on Stand Up for Science.

 

Image Credit: Bob Hemstock

GoOpen.no: NDLA technology reused by 8000 websites worldwide

Over the last couple of years, the NDLA team have been working to replace Flash based applications and interactive learning objects. NDLA also needed a tool to make it easy to create, share and reuse HTML5 content and applications. We started developing a new tool in public-private partnership with Joubel, a tech startup in Tromsø, in the northern part of Norway. This collaboration ended up as a project and product called H5P.

H5P is at the time of writing installed on over 8,000 websites. H5P is reused by many universities, large companies and smaller personal websites worldwide. It´s great to see this kind of reuse and in the long run, this will make the platform more sustainable, also for NDLA.

The team developing and designing H5P have been set up with the best product developers from NDLA and designers and developers from Joubel. This kind of public-private partnership is essential to NDLAs innovation process.

In H5P, all you need is a web browser and a website with an H5P plugin. H5P empowers creatives to create rich and interactive web experiences more efficiently.

H5P is a free and open source tool that helps you create HTML5 content in the browser of your choice and share it across all operating systems and browsers. Check out the list of different content types.

As H5P is open source there are no “strings attached”. Anyone can reuse both content and technology without asking Joubel or NDLA for permission. One of the universities that have reused H5P is Colorado.

How to use H5P?

H5P is a plugin for existing CMS and Learning Management Systems (LMS) systems like WordPress and Drupal. Just install the H5P and your system becomes able to create, share, and reuse great interactive content. For systems that don’t have an H5P plugin available yet it is possible to embed content using an iframe or using the Learning Tool Interoperability (LTI) standard. With the LTI and supporting APIs and specifications embedding an externally hosted H5P authoring tool is also possible.

The H5P format is open and the tools for creating H5P content are open source. This guarantees that creatives own their own content and are not locked into the fate and licensing regime of a specific tool.

 

CC Danmark: State of the Commons 2016-rapporten ligger klar

Én gang om året udgiver Creative Commons rapporten ‘State of the Commons’, der tager pulsen på den globale delebevægelse og anvendelsen af Creative Commons licenserne på digitalt indhold: Fotos, tekst, film, musik og meget andet. 

Således også I år, hvor State of the Commons 2016, der dækker aktiviteter hele det forgangne år, netop er udkommet. Den viser, at 2016 uden sammenligning var det mest aktive i licensværktøjernes historie, og at nye mennesker, organisationer og virksomheder verden over tager CC-licenserne til sig i et tempo, som aldrig er set højere. Blandt de mest bemærkelsværdige fakta denne gang kan det således nævnes at det totale antal værker under CC-licens har passeret 1,2 milliarder, og ydermere at 65% af disse værker falder i open culture-kategorien, hvilket betyder de er licenseret under de to mest åbne licenser og dermed tillader både fri bearbejdning og kommerciel anvendelse.

Læs flere højdepunkter i denne blog post fra Creative Commons HQ og dyk ned i rapporten her: State of the Commons 2016. Hvis man vil læse flere perspektiveringer på rapporten, så er der rigtigt gode artikler hos eksempelvis OpenSource.com og TechDirt.

 

Automotive Industry Revs up for Sales

Buying a car is more than just an investment. It can be a life-changing event, a family milestone, or for some, a simple purchase that came from haggling with a salesman. How the Hispanic community views this big purchase is looked at with review of the Hispanic community’s buying power. Read More http://www.associationofmarketing.org/articles/view.php?article_id=5345 Views: 0