KEMBAR78
Science in the context of journals, Open, and the future | PDF
The State of science,
journals, peer-review,
thoughts on Open
Science, reproducibility,
and Science 2.0?
Dr. Benjamin Laken
University of Oslo
@benlaken
We are now on the brink of an achievable
aim: for all science literature to be online,
for all of the data to be online and for the
two to be interoperable.
Source XKCD
Essentially, the volume of published work is constantly
increasing.
A rough estimate of the rate of increase is …
Unsurprisingly, the number of PhDs
is also increasing
Nature 472, 276-279 (2011) | doi:10.1038/472276a
Trends in PhDs granted per year
for all disciplines,
values x103
The number of science doctorates
earned each year grew by nearly
40% between 1998 and 2008, to
some 34,000, in countries that are
members of the Organisation for
Economic Co-operation and
Development (OECD). The growth
shows no sign of slowing
Systems Research and Behavioral Science Syst. Res (2014)
Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324
(StatshereforUSAbiomedicalsciences)However, there is no matching rise in permanent positions
This has created ‘The Postdoc Pile-Up’
Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a
In essence a saturated Postdoc market. This means that postdocs who want to
stay in academia are usually destined to travel from one place to another with no
long-term prospects. (One of the reasons you may hear postdocs referring to
themselves as ‘hobos with a PhD’.)
In the context of the increasing pressures on individual
scientists to compete—largely through publications—enters
the problem of Academic publishers as for-profit entities
The first journal, Proceedings of the Royal Society, was
created by Henry Oldenburg (1619--1677), in 1665.
The idea was to drive science forward through rapid
communication and sharing to a broad audience.
However, things have gone slightly astray over the years…
The role of journals: have they changed from a force of
innovation to a hinderance?
The academic publishing business model explained by
Nature’s Publishing Director
Apple:
Exxon:
Profits (% of revenue) in 2011
Elsevier:
Springer:
John Wiley & Sons:
~25%
~8.5%
36% (£724m)
33.9% (£866m)
42% ($106m)
Are journals overcharging?
Some opinions on academic journals…
Cushing, 2014, https://www.techdirt.com/articles/20140605/05175627467/academic-publisher-fights-publication-paper-
criticizing-publishers-price-increases-profits.shtml
Elsveier a particularly bad example
Is a good solution to science communication simply
traditional journals but as open-access?
Open access does not mean free, often the authors still pay high prices
to publish their research.
The growth of Open Access (OA) journals in the last 10 years opened the
door for the creation of hundreds of new Publishing company and new
journals, who engage in behaviour often best described as ‘predatory’…
They claim their articles are peer-reviewed, however independent
research and many examples have repeatedly shown that the real barrier
to publication in their journals is simply the publication fee…
The Dark-side of the OA journal explosion
In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with
two figures) to International Journal of Advanced Computer Technology…
…they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal
requested a fee of $150.
Tom Spears submitted a completely incoherent paper featuring a made-
up study area (Nepean Desert)
Several famous examples of OA journal ‘stings’
Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.
More famous sting: John Bohannon
J Bohannon Science 2013;342:60-65
Abstract
Dozens of open-access journals targeted in an
elaborate Science sting accepted a spoof research
article, raising questions about peer-review
practices in much of the open-access world.
Peer review reviewed.Few journals did substantial review that identified the paper's flaws.
Another problem: the valuable reviews are usually wasted
It is not a question of if an article will be published, but where.
In this context, the expert reviews, community feedback, and
reproducibility are the hallmarks of quality, and a real factor in
the value of the research.
We need to develop systems which focus on these aspects of
the peer system, rather than see peer-review as the ‘behind
the scenes’ work.
Source: PhD comic
Explosion in OA journals over last decade
(and rapid growth of predatory OA journals)
Published science doubles every ~9 yrs
New PhD’s awarded annually increasing
(simultaneous decrease in TT and Permanent positions)
‘Publish or perish’, saturated PhD-level job market, and resulting
competitiveness has combined with journal profiteering
…and if a paper gets a hard review, rejection usually means a paper will be
submitted (unchanged) to new journal, until accepted
Many Journals = opaque review process, expensive, & behind the
scenes politics/motives
End result (*at least for me) = scepticism, cynicism, frustration, time loss,
high-blood pressure, and madness
Publication ≠ quality guarantee
Favour systems totally open (inc. review) and communal
New paradigm: publication not the end of a funding cycle but the
start of a collaborative communal conversation
Summary
My fantasy model of Science 2.0
Data, code, paper: all in one open repository (like ArXiV)
Version control system (like Github)
All researchers have unique ID profile (like RG/ORCID)
ID-tagged comments. Can be general, or specific:
specific comments tagged to sections, as a thread, including up-
vote system (like StackOverflow) on paper and comments.
Copying, and contributing (pull-requests) allowed
to create rapid-iterations and community refinement (like Github)
Vote-based score system, with scores contributing to different levels of
community privileges (like StackOverflow).
Score a measure of a researchers overall contributions (Like RG), > than
citation metrics [e.g. as bad papers often cited on purpose]
Needs shared super-computer-scale resources to re-run
experiments (Like Google Earth Engine)
(This model already exists for the open software we use every day. The model can be used for
research, and for lecture/course/textbook material too.)

Science in the context of journals, Open, and the future

  • 1.
    The State ofscience, journals, peer-review, thoughts on Open Science, reproducibility, and Science 2.0? Dr. Benjamin Laken University of Oslo @benlaken
  • 5.
    We are nowon the brink of an achievable aim: for all science literature to be online, for all of the data to be online and for the two to be interoperable.
  • 6.
  • 7.
    Essentially, the volumeof published work is constantly increasing. A rough estimate of the rate of increase is …
  • 8.
    Unsurprisingly, the numberof PhDs is also increasing Nature 472, 276-279 (2011) | doi:10.1038/472276a Trends in PhDs granted per year for all disciplines, values x103 The number of science doctorates earned each year grew by nearly 40% between 1998 and 2008, to some 34,000, in countries that are members of the Organisation for Economic Co-operation and Development (OECD). The growth shows no sign of slowing
  • 9.
    Systems Research andBehavioral Science Syst. Res (2014) Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324 (StatshereforUSAbiomedicalsciences)However, there is no matching rise in permanent positions
  • 10.
    This has created‘The Postdoc Pile-Up’ Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a In essence a saturated Postdoc market. This means that postdocs who want to stay in academia are usually destined to travel from one place to another with no long-term prospects. (One of the reasons you may hear postdocs referring to themselves as ‘hobos with a PhD’.)
  • 12.
    In the contextof the increasing pressures on individual scientists to compete—largely through publications—enters the problem of Academic publishers as for-profit entities The first journal, Proceedings of the Royal Society, was created by Henry Oldenburg (1619--1677), in 1665. The idea was to drive science forward through rapid communication and sharing to a broad audience. However, things have gone slightly astray over the years… The role of journals: have they changed from a force of innovation to a hinderance?
  • 13.
    The academic publishingbusiness model explained by Nature’s Publishing Director
  • 14.
    Apple: Exxon: Profits (% ofrevenue) in 2011 Elsevier: Springer: John Wiley & Sons: ~25% ~8.5% 36% (£724m) 33.9% (£866m) 42% ($106m) Are journals overcharging?
  • 15.
    Some opinions onacademic journals…
  • 16.
  • 17.
  • 18.
    Is a goodsolution to science communication simply traditional journals but as open-access? Open access does not mean free, often the authors still pay high prices to publish their research. The growth of Open Access (OA) journals in the last 10 years opened the door for the creation of hundreds of new Publishing company and new journals, who engage in behaviour often best described as ‘predatory’… They claim their articles are peer-reviewed, however independent research and many examples have repeatedly shown that the real barrier to publication in their journals is simply the publication fee…
  • 19.
    The Dark-side ofthe OA journal explosion In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with two figures) to International Journal of Advanced Computer Technology… …they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal requested a fee of $150.
  • 20.
    Tom Spears submitteda completely incoherent paper featuring a made- up study area (Nepean Desert) Several famous examples of OA journal ‘stings’ Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.
  • 21.
    More famous sting:John Bohannon J Bohannon Science 2013;342:60-65 Abstract Dozens of open-access journals targeted in an elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world. Peer review reviewed.Few journals did substantial review that identified the paper's flaws.
  • 22.
    Another problem: thevaluable reviews are usually wasted It is not a question of if an article will be published, but where. In this context, the expert reviews, community feedback, and reproducibility are the hallmarks of quality, and a real factor in the value of the research. We need to develop systems which focus on these aspects of the peer system, rather than see peer-review as the ‘behind the scenes’ work.
  • 23.
  • 24.
    Explosion in OAjournals over last decade (and rapid growth of predatory OA journals) Published science doubles every ~9 yrs New PhD’s awarded annually increasing (simultaneous decrease in TT and Permanent positions) ‘Publish or perish’, saturated PhD-level job market, and resulting competitiveness has combined with journal profiteering …and if a paper gets a hard review, rejection usually means a paper will be submitted (unchanged) to new journal, until accepted Many Journals = opaque review process, expensive, & behind the scenes politics/motives End result (*at least for me) = scepticism, cynicism, frustration, time loss, high-blood pressure, and madness Publication ≠ quality guarantee Favour systems totally open (inc. review) and communal New paradigm: publication not the end of a funding cycle but the start of a collaborative communal conversation Summary
  • 25.
    My fantasy modelof Science 2.0 Data, code, paper: all in one open repository (like ArXiV) Version control system (like Github) All researchers have unique ID profile (like RG/ORCID) ID-tagged comments. Can be general, or specific: specific comments tagged to sections, as a thread, including up- vote system (like StackOverflow) on paper and comments. Copying, and contributing (pull-requests) allowed to create rapid-iterations and community refinement (like Github) Vote-based score system, with scores contributing to different levels of community privileges (like StackOverflow). Score a measure of a researchers overall contributions (Like RG), > than citation metrics [e.g. as bad papers often cited on purpose] Needs shared super-computer-scale resources to re-run experiments (Like Google Earth Engine) (This model already exists for the open software we use every day. The model can be used for research, and for lecture/course/textbook material too.)