Thursday, June 1, 2017

WCRI 2017, Day 3

Day 2                                                                                                             Day 4
Day 3 of the World Conference on Research Integrity began with a plenary session on the role institutions play in research integrity. Bertil Anderson, the president of the Nanyang Technological University Singapore, spoke on  "Research and Research integrity - a key priority for a young and fast rising university." He reported in a refreshingly open manner about quite a number of cases of academic integrity his university was concerned with. In addition to much plagiarism, authorship disputes, self-plagiarism, there was outright fraud. He asked how much a university can do to investigate a case when something has happened that also hits the media? He presented four cases:
  1. NTU retracts NIE academic papers after malpractice investigations (The Straits Times), 2016
    A professor, hired in 2006, was contractually obligated to publish 10 papers in 3 years. An external whistleblower alerted the university to data fabrication that included an invented person and an invented company, eventually the police and the Ministry of Education were involved. The case led to 21 retractions. Such a clause is no longer in contracts.
  2. 3 Singapore-based scientists linked to research fraud (The Straits Times), 2016
    NTU professor fired for data falsification (New Press), 2016
    This case involved Western blot imagery manipulation and three institutions in Singapore, the USA and New Zealand. Two PhDs were revoked (and this has to be done by the president of the country, not the university president), nine papers were retracted and the professor dismissed for willful negligence. The national research organization is also seeking repayment. As a problematic side effect, students of this professor are now left without a supervisor, and are often not accepted on joint programmes elsewhere because of the tarnished reputation of laboratory. They are innocent persons who suffer.
  3. Adventures in Copyright Violation: The Curious Case of Utopian Constructions (Blog, Lincoln Cushing), 2012
    This was a case of images being mis-used. The owner of the copyright on the pictures stumbled over his pictures and wrote to the full professor at the Arts School who was using them. All he wanted was his name on the images and a link to his site. The professor refused, a lengthy investigation including external people ensued (that also uncovered additional problems) and ended with the professor being dismissed. 
  4. Fake Peer Reviews, the Latest Form of Scientific Fraud, Fool Journals (Chronicle of Higher Education, paywalled), 2012
    A scientist managed to hack into the Elsevier system for referee reports. He added sentences along the lines of "paper X needs to be referenced" in order to increase his own citation index. It turned out that there were 122 instances of such hackings. The scientist resigned, but NTU referred the case to the Singapore Police Force under the "Misuse of Computers" legislation, and the scientist has apparently left the county.
Anderson concluded by discussing the challenges in developing a culture of research integrity in a rapidly developing university in a competitive environment. There are aggrevating factors such as hierarchy, hetrogeneous faculty, tolerance of misconduct, and also the fact that investigations of research integrity cases need competencies outside the traditional university framework, such als lawyers. He emphasized that no university or institution can be immune to research fraud, and thus they need to have clear procedures defined.

The second speaker in the plenary session was Mai Har Sham, the associate vice president of research at The University of Hong Kong. She noted that just recently there was a meeting of the Asia-Pacific Research Integrity Network with 110 participants from 20 countries. She specified three areas in which the institution is called on to take action:
  1. Determination and commitment - provide policies, resources and infrastructure support
  2. RCR education, skills training, setting up a data management system andsupporting platforms
  3. Taking the initiative for quality assurance and risk managment
The third speaker was Jay Walsh, the vice president for research at Northwestern University, USA. He noted wryly that if you have never been quoted out of context, you haven't been quoted enough. How true. He then embarked on a short description of how we learn: We gather evidence and we form stories.We then develop a hypothesis about how the words in the stories work. We take data, distill it into information, coalesce that to knowledge, from which we develop wisdom.

But things can go wrong that warp the stories, the data, the information, the knowledge and/or the wisdom. Beyond inadequate methods, poor data, and poor practices, he referred to a paper that recently identified 235 forms of bias.

He feels that the path forward involves the training of researchers in the responsible conduct of research (RCR). Since it is hard to change the curriculum, the funders should make RCR training a requirement. Speaking of funders, he desires a single system for handling FFP  (falsification, fabrication & plagiarism) cases, as each funder has a different process. There need to be robust RCR courses, and professors could be given credit for teaching such courses. It is also vital to have a system that allows students and post-docs to come forward with problems without retribution, although this is so easy to say and so hard to do. The root causes of research integrity issues are wrong incentives. These need to be solved, otherwise we are just treating the symptoms.

I then chaired a session on Authorship. This seemed to me to be such a trivial topic, as the focus of publication is on communication from a group of authors with a collective of readers, so I find a ranking to be unnecessary. But there are many and various forms of author orderings and inclusions and perceptions of what they all mean. And where there are differences of opinion, there are fights, somtimes quite intense and protracted. It was interesting to see people investigating this from all sides, I'll just list them here, as I was busy dealing with the time and the questions during the session:
  • Authors without borders: Investigating international authorship norms
    among scientists & engineers
    Dana Plemmons (University of California, Riverside, U.S.A.)
  • Experiences of the handling of authorship issues among recent doctors
    in medicine in Sweden
    Gert Helgesson (Karolinska Institutet, Stockholm, Sweden)
  • A philosophical framework for a morally legitimate definition of
    scientific authorship
    Mohammed Hosseini (Utrecht University, Utrecht, The Netherlands)
  • The perceptions of researchers working in multidisciplinary teams on
    authorship and publication ethics
    Zubin Master (Albany Medical College, Albany, NY, U.S.A.
  • An investigation of researchers’ understanding and experience of
    scientific authorship in South Africa
    Lyn Horn (University of Cape Town, Cape Town, Republic of South Africa)
The final plenary session was about interventions that work. The session chair, Lex Bouter, remarked that the only thing that people have widely learned to use is text-matching software, all other investigations of interventions have either shown no effect, or an effect in the wrong direction.

The first speaker was Klaas Sijtsma, who was vice-dean at the Tilburg University School of Social and Behavioral Sciences under the deanship of Diederik Stapel when Stapel confessed to the rector that he had, indeed, committed extensive academic misconduct. Sijtsma was first named interim dean, then continued on as dean and will be stepping down in the coming school year. In his talk "Never Waste a Good Crisis: Towards Responsible Data Management," he spoke about this scandal that also touched the University of Amsterdam and the University of Groningen and involved dozens of articles and book chapters, and affected several PhD theses that were granted on the basis of analysis of fraudulent data.

They were lucky to have had a confession, so that Stapel's contract could be terminated, although there were still many committees needed to investigate all of the publications. There were a few criteria that were identified that permitted a culture to thrive in which the frauds were possible: Staple was unusual in that he preferred to work alone, he would not allow his PhD students to collect their own data, and he presented unlikely results to journals.

The University Tilburg is taking the following steps to foster a climate of integrity:
  • Each PhD student must have at least 2 supervisors.
  • Master theses and PhD theses are scanned for plagiarism (although they did this already).
  • An official formula is read aloud publicly when a doctorate is awarded. It refers to the young doctor's obligation to academia and society to act with integrity.
  • The university has a Code of Conduct.
  • Every staff member must sign an integrity code.
  • There is a now independent Integrity Officer and Research Committee.
Additionally, the School of Social and Behavioral Sciences took two actions:
    •    They intensified classes on research ethics and research integrity.
    •    The dean instituted a "Science Committee" in the Spring of 2012.

This, it seems, was one of the best ideas they had. This committee is tasked with auditing a small sample (about 20 out of 500) of the articles published by members of the school each year. Their task is to assess the quality of the data storage and to look closely at how well the research methods are described. The committee thus learns where there are problems in preserving data, and advises the school's management team and the researchers about data storage, completeness of data sets, honoring subjects' privacy, access to data, and making the data available to others. They are not out to "witch-hunt" for fraudsters, but just to eyeball the data. That, however, keeps the various research groups on their toes and thinking about these aspects of their data before they publish. In turn, this creates a better research atmosphere.

Sijtsma has often been asked, why he didn't design a universal data storage system and data management policy first? Well, it seems he understands that computer systems are often too complex, take a lot of time, are very expensive, and tend to encounter unpleasant technical surprises. It would have taken too long, grass would have grown over the scandal, and the sence or urgency would have disappeared. So he installed the committee first. They set up rules and regulations, announced annual random audits. Now the groups were motivated to come up with a data policy that suited their needs best.

The worked quite well! Some groups are better than others, people tend to only arrange their storage only when they are audited. When they leave the school, they lose commitment. No consistent data storage system, but was a deliberate choice, so much more to do.

Do these interventions work?  He reports that they do. They won't prevent new affairs, but they do encourage RCR and reduce QRP (Questionable Research Practices). He also noted in the Q&A that the university decided not to rescind the doctorates of the people using Stapel's fraudulent data, as they did not know that the data was false.

For more information on the Stapel scandal, see the report: Flawed science: The fraudulent research practices of social psychologist Diederik Stapel.

Patricia Valdez, the extramural Research Integrity Officer(RIO) of the National Institute of Health spoke on the NIH Perspective on Research Integrity.

Her focus was on the reproducibility crisis, as the NIH invests $30 billion of taxpayer money annually. They don't want to waste money trying to reproduce something that is erroneous. They are focusing on evaluating the rigor of the methodology and the transparency of the research in the hope that this will have an effect on reproducability.

She referred to a 2017 book by Richard Harris, Rigor Mortis: How sloppy science creates worthless cures, crushes hope and wastes billions (Basic Books). The take-home message from the book is: Teach students methods the first year, not facts!

Ian Freckelton closed the session speaking on Research Misconduct and the Law: Intervening to Name, Shame and Deter. He is a lawyer (Queen's Counsel at the Victorian Bar in Australia) and a professor for Law and Psychiatry at the University of Melbourne.
He published a book in 2016 called Scholarly Misconduct and the Law (Oxford University Press).

After reading to us the most important bit from Stapel's book about the fraud (so we don't have to read it), he raced us through criminal law, which is invoked to shame and deter, as it has been applied to research misconduct. Then he spoke of a number of cases (I've put in links to press articles or the Wikipedia for more detail):
He also spoke about another book by Tom Nichols, The Death of Expertise (2017), about when experts lie, and noted that there are many other cases of fraud that have not reached the courts. He closed with the observation that the law is a very slow, blunt instrument and that criminal prosecution is not the answer, but notes that research fraud is not victimless. A court decision would, however, vindicate whistleblowers and hopefully present a high deterrence factor.

We then were ferried by boat through the canals of Amsterdam and the Amstel River to our dinner. I'll try to get a short description of day 4 out by tomorrow!

No comments:

Post a Comment