Everything Hertz

We chat with Amy Orben, who applies "multiverse" methodology to combat and expose analytical flexibility in her research area of the impact of digital technologies on psychological wellbeing. We also discuss ReproducibiliTea, an early career researcher-led journal club initiative she co-founded, which helps young researchers create local open science groups. Here are some more details and links: * The tweet (https://twitter.com/_vdeni_/status/1126485212337143808) pointing our Dan's gramatical error in his usual introduction. THANKS DENIS * Is Twitter melting our brains? * The history of "new technology" panic * What's the next panic? * Moral entrepreneurs: profiting from moral panic * Specification curve analysis (https://socialsciences.nature.com/users/200472-amy-orben/posts/42763-beyond-cherry-picking): a way to run all theoretically defensible analysis options on a given dataset * Amy's Nature Human Behavior paper (https://www.nature.com/articles/s41562-018-0506-1) * Amy's PNAS paper (https://www.pnas.

By popular demand, Dan and James are kicking it old school and just shooting the breeze. They cover whether scientists should be on Twitter, if Fortnite is ruining our youth, book recommendations, and null oxytocin studies. Stuff they cover and links to obsure references * Should scientists be on twitter? * James runs a Twitter experiment * Scite has now gone live, listen to our episode (https://everythinghertz.com/80) on this platform * Our dreams of a live Hertz episode * Is Fortnite (https://en.wikipedia.org/wiki/Fortnite) killing our youth and the parallels with the “heavy metal” scare * Amy Orben’s screen time study (https://www.nature.com/articles/s41562-018-0506-1) * Multiverse analysis * Book recommendations: Kevin Mitchell’s "Innate" (https://www.amazon.com/Innate-How-Wiring-Brains-Shapes/dp/0691173885), Gareth Leng's "Heart of the brain" (https://mitpress.mit.edu/books/heart-brain) * Daryl dug a hole reference (https://www.youtube.com/watch?v=mDl1i3mpKwM), from the Aussie classic, "The Castle"

We answer a listener question on the possible negative consequences of the open science movement—are things moving too quickly? Links and things we discuss in the episode: * We have a new logo, if you haven't already noticed... * Contact us via our website form (https://everythinghertz.com/contact)! * Considering the potential downsides of open science * Here come dat boi meme explination (https://knowyourmeme.com/memes/dat-boi) * The dangers of open access by fiat * The role of commercial entities in open science * The “University of Oslo fancy Norway people-pay-taxes oil money bloody library (https://www.ub.uio.no/english/)” * Dropping the success rate of grants to increase the quality of evaluation * Reframing open science reform efforts to a mission of equity and fairness * We don’t know the process behind university sexual harassment/misconduct investigations * Does transparency even matter if people won’t follow up on problems? * James' prediction: If someone starts a journal that ONLY does Register

We answer our first audio question, on whether academia is too broken to fix, and a second question on whether we’ve ever worried about the possible repercussions of our public critiques and commentary on academia. Show details: Our first audio question is from Erin Williams (@DrErinWill), who asks whether academia is too broken to fix The letter to the editor that got rejected, despite the publication of the response to the letter Harassment in academia Have we ever been worried that someone might say, "I'd never hire those dudes" because of what we say? Other stuff that has happened to us as a result of the podcast Fahrenheit vs. Celsius Supply and demand for academic jobs The criticism that comes with putting yourself out there Links - @ReproRocks (https://twitter.com/ReproRocks): for those working in reproduction to share their work through twitter - The Steven Pinker book - The Better Angels of Our Nature: Why Violence Has Declined (https://www.amazon.com/Better-Angels-Our-Nature-Violence/dp/0143122010

We chat with Sean Rife, who the co-founder of scite.ai (https://scite.ai), a start-up that combines natural language processing with a network of experts to evaluate the veracity of scientific work. Here's what we cover and links for a few things we mention * What is scite.ai? * The Winnower (https://thewinnower.com) * Why is there no good (and free) plagiarism detector? * Grobid (https://grobid.readthedocs.io/en/latest/Introduction/) - A machine learning library for extracting, parsing and re-structuring PDFs * Meta-analysis can prop up flawed bodies of literature * The "Too meta (https://xkcd.com/1447/)" XKCD cartoon * What’s the end game for scite? * The 80,000 hours game (https://80000hours.org/articles/can-you-guess/) * Spooner (http://spooner.lyceum.ws), a utility that allows authors of scientific publications to make their work available to the general public (probably) without violating publishing agreements Other links - Sean on twitter (www.twitter.com/seanrife) - Dan on twitter (www.twitter.com/dsq

We chat with Henry Drysdale (University of Oxford), co-founder of the COMPare trials project (http://compare-trials.org), which compared clinical trial registrations with reported outcomes in five top medical journals and qualitatively analysed the responses to critical correspondence. Discussion points and links galore: The history behind the COMPare project The two papers that were published: a prospective cohort study (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3173-2) correcting and monitoring 58 misreported trials and a qualitative analysis (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3172-3) of researchers’ responses to critical correspondence Ben Goldacre's books (https://www.amazon.co.uk/l/B002C1VRBQ?_encoding=UTF8&redirectedFromKindleDbs=true&rfkd=1&shoppingPortalEnabled=true) What is outcome switching? What were some of the responses to query letters from the authors and journals? Misreporting trials (usually) doesn't lead to patient harm, but it ha

In this episde, we chat with Lisa DeBruine (University of Glasgow) about her experience with large-scale collaborative science and how her psychology department made the switch from SPSS to R. Discussion points and links galore: Deborah Apthorp's tweet on having to teach SPSS (https://twitter.com/deborahapthorp/status/1092599860212068352), "because that's what students know" People who are involved with teaching R for psychology at the University of Glasgow: @Eavanmac @dalejbarr @McAleerP @clelandwoods @PatersonHelena @emilynordmann Why the #psyTeachR started teaching R for reproducible science Data wrangling vs. statistical analysis The psyTeachR website (https://psyteachr.github.io) Danielle Navarro (https://djnavarro.net), and her R text book (https://learningstatisticswithr.com) that you should read Lisa's "faux" package (https://github.com/debruine/faux) for data simulation Sometimes you can't share data, simulations are a good way around this problem "synthpop" is the name of the package (https://cran.r

Dan and James discuss how to deal with the problem of scientists who start talking about topics outside their area of expertise. They also discuss what they would do differently if they would do their PhDs again Here's what they cover... The podcast will now be permanently archived on Open Science Framework (https://osf.io/zj7y3/) James did a talk at the Sound Education conference on podcasting for early career researchers. Here's the video (https://www.youtube.com/watch?v=26t6660_f-A) if you want to see him squirm uncomfortably in his chair for 20 minutes and/or hear his thoughts our approach to podcasting The temptation for academics to believe their own press and to have their thoughts reinforced by the praise they get Keeping a handle on what you know and don't know Nassim Nicholas Taleb (https://twitter.com/nntaleb) has FANS The "Pete Evans" effect, James' solution, that we should eat Pete Evans (https://medium.com/@jamesheathers/i-think-i-have-a-solution-i-m-going-to-eat-pete-evans-7e2da6f3967f)

Peer review is typically conducted behind closed doors. There's been a recent push to make open peer review standard, but what's often left out of these conversations are the potential downsides. To illustrate this, Dan and James discuss a recent instance of open peer review that led to considerable online debate. Here's what they cover... How should we navigate the open review of preprints? Gate keepers gonna gate keep, but is this better out in the open? Weaponising openness Some people don't realise that some data can’t be shared Should the reviewers of rejected papers follow them to the next journal? When bad papers that you reject pop up in another journal, unchanged Does the venue and timing of the open peer review matter? Signing your reviews Using publons to track your reviews Links - Brad Love’s blog post: http://bradlove.org/blog/open-review - Niko’s blog post: https://nikokriegeskorte.org/2019/01/09/whats-the-best-measure-of-representational-dissimilarity/ - Publons: https://publons.com - Dan on t

We’re joined by Daniele Marinazzo (University of Ghent) to chat about the recently launched overlay journal Neurons, Behavior, Data analysis and Theory (NBDT), for which he on the Editorial Board. An overlay journal is organised a set of manuscripts that is published and hosted by a seperate entity (in this case, the Arxiv server), a feature that dramatically reduces publication costs. We discuss the unique overlay model, how this can drive article fees to essentially zero, and what it takes to build a good community journal. Here’s what we cover: Why launch a new neuroscience journal and how is it different from currently established journals? The unique way that editor’s decide which papers to send out for review How does the journal operate when it’s open access and submissions only cost $10? How do you build a journal that your target community will recognise as a ‘good’ journal? The process of submitting a manuscript to NBDT Should journals allow or encourage authors to suggest potential reviewers for t

In this episode, Dan and James chat with microbiologist Elisabeth Bik about about the detection of problematic images in scientific papers, the state of microbiome research, and making the jump from academia to industry. More info on what they cover: How Elisabeth get into error detection of scientific images The process of detecting errors in images How groups of authors tend to publish multiple papers with problematic images The association between journal prestige and problematic images Providing monetary incentives for publications Making the jump from academia to industry The current state of microbiome research Links - Patreon: www.patreon.com/hertzpodcast - Elisabeth on Twitter:  www.twitter.com/microbiomdigest - Elisabeth online: https://microbiomedigest.com - The problematic image paper: https://mbio.asm.org/content/7/3/e00809-16.short - Pubpeer: https://pubpeer.com - Dan on twitter: www.twitter.com/dsquintana - James on twitter: www.twitter.com/jamesheathers - Everything Hertz on twitter: www.twi

Dan and James discuss what's missing from biobehavioral science course syllabi. Here's the episode lowdown: - A thank you to our new Patron supporters - The (supposed)CRISPR baby - SPSS vs. R: What should be used for instruction? - What would Dan and James include in a new syllabus? - Should students be taught scientific communication? - If we’re going to add new stuff to a syllabus, what gets removed? - Are courses too big these days? - Should students be taught how to set up a side hustle to apply their research skills outside of academia? Links - Patreon: https://www.patreon.com/hertzpodcast - CRISPR baby story: https://www.vox.com/science-and-health/2018/11/30/18119589/crispr-technology-he-jiankui - Dan on twitter: https://www.twitter.com/dsquintana - James on twitter: https://www.twitter.com/jamesheathers - Everything Hertz on twitter: https://www.twitter.com/hertzpodcast - Everything Hertz on Facebook: https://www.facebook.com/everythinghertzpodcast/ Music credits: Lee Rosevere freemusicarchive.org/musi

Dan and James discuss a new journal of "controversial ideas" that will allow authors to publish articles anonymously. They also launch their Patreon page, in which listeners can support the show and get bonus features. Here's the episode lowdown - James describes his first experience eating a “ding dong” - Why James and Dan do the show - What is Patreon? - The Journal of Controversial Ideas - The link between a vaccine batch and narcolepsy in Norway - Can you “claim” our anonymous article a few years into the future? - What’s the difference between anonymous blogging and anonymous journal articles? - The new Neurons, Behavior, Data analysis and Theory journal Links - Patreon: https://www.patreon.com/hertzpodcast - Narcolepsy paper: https://www.sciencedirect.com/science/article/pii/S1389945713001548 - Journal of Controversial ideas story: https://www.bbc.com/news/education-46146766 - PsyBrief twitter account: twitter.com/psybrief - NBDT journal: nbdt.scholasticahq.com - Dan on twitter: https://www.twitter.c

In this episode, we chat about whether it’s necessary to move for an academic job to demonstrate “independence”. Here's a rundown of the other stuff we cover: - James' appearance at the “Sound education” conference - Dan’s first day of school as a latino in a white neighbourhood - Our thoughts on the restrictive social media policy at the SfN (Society for Neuroscience)conference - Why and how Dan and James moved overseas from Australia - Do you really need to move overseas to demonstrate independence? - The two-body problem - Can you demonstrate independence with sole-author papers? Links SFN social media policy https://twitter.com/fedeadolfi/status/1058760331747581953 Dan on twitter https://www.twitter.com/dsquintana James on twitter https://www.twitter.com/jamesheathers Everything Hertz on twitter https://www.twitter.com/hertzpodcast Everything Hertz on Facebook https://www.facebook.com/everythinghertzpodcast/ Music credits: Lee Rosevere freemusicarchive.org/music/Lee_Rosevere/

Dan and James discuss the recent "grievance studies" hoax, whereby three people spent a year writing twenty-one fake manuscripts for submission to various cultural studies journals. They also discuss a new proposal to shift publication culture in which researchers pledge to publish exclusively in community-run journals but only when a pre-specified threshold of support for this commitment by the research community has been met. Here's an overview of the episode: - It’s fat bear week! - The new proposal to fix the stranglehold of commercial publishers in academia - Flipping journals to open access - The ‘grievance studies’ hoax - When James first came across the “dog rape” paper - What if you were to design the dog study properly? - Should we systematically try and hoax journals? - Astronomy already injects fake data, can we learn from this? - Should these new hoaxes all be associated with Sokal? Links - Brian Resnick’s fat bear week story: https://www.vox.com/science-and-health/2018/10/9/17955432/fat-bear-w

We’re joined by Brian Nosek (Centre for Open Science and University of Virginia) to chat about building technology to make open science easier to implement, and shifting the norms of science to make it more open. We also discuss his recent social sciences replication project in which researchers accurately predicted which studies would replicate. Here’s what we cover: - What is the Centre for Open Science? - How did Brian go from Psychology professor to the director of tech organisation? - How can researchers use the Open Science Framework (OSF)? - How does OSF remove friction for conducting open science? - Registered reports (now available at 131 journals!) - What factors converged to cause the emerging acceptance of open science? - The social sciences replication project - Can researchers anticipate which findings can replicate? - What happened when Brian and his team tried to submit their replication attempts of Science papers to Science? - The experience of reviewing registered reports Links: Centre for

This episode includes part two of a chat with Nathan Hall (McGill University), who is the person behind the ’Shit academics say’ account (@AcademicsSay), which pokes fun of all the weird stuff that academics say. Before getting to the discussion, James and Dan answer two listener questions on grants and data cleaning. Here’s what is covered in the episode: People talk about papers all the time, but the grant process is not discussed openly—why? Speaking to your funding body’s relevant program officer Assembling a team that complements your weaknesses Data carpentry and the tidyverse Outlier analysis Nathan Hall on big publishing Upending the publication system by getting journals to bid for papers Using peer review quality to judge the quality of journals Debunking learning styes Academics chasing after celebrity and hype The cost of chasing academic prestige Using twitter hashtags like #PhDChat and #ECRchat to learn more about the experiences of other people Links Data carpentry https://datacarpentry.org

We’re joined by Nathan Hall (McGill University) to chat about the role of humour in academia. Nathan is the person behind the ’Shit academics say’ Twitter account (@AcademicsSay), which pokes fun of all the weird stuff that academics say. Here’s what we cover: How Nathan got started with the account The story behind Nathan's 'Research Wahlberg' Twitter account (@ResearchMark) The risk of social media usage being perceived as “unprofessional” The amount of free labor that academics are pressured to do How alcohol is becoming an unspoken coping strategy in academia Academic guilt and glamorising overwork Why Nathan changed his mind about making Imposter Syndrome jokes Leaving tweets in your draft folder Links Nein Quarterly https://twitter.com/NeinQuarterly Shit my Dad says https://twitter.com/shitmydadsays Cern and comic sans https://www.theverge.com/2012/7/4/3136652/cern-scientists-comic-sans-higgs-boson Ate the onion https://www.reddit.com/r/AteTheOnion/ Shit Academics Say on twitter https://www.twitter.c

Dan and James answer listener questions on tips for starting your PhD and the role of statistics in exploratory research. Other stuff they cover: James new paper on people that voluntarily give themselves goosebumps Dan’s new podcast: Physiology and Behavior A preview of next weeks guest, Nathan Hall When things are taken out of context on Twitter What do you do when people are angry with you on the internet? Tips for people starting a PhD Can inferential statistics play a role in exploratory research? Why don’t journals publish peer review reports? Why is PsycNet so bad? Links James’ paper https://peerj.com/articles/5292/ Physiology and Behavior podcast from Dan https://shows.pippa.io/dsquintana The tweet we discuss https://twitter.com/andpru/status/1024005699737509888?s=21 Dan on twitter https://www.twitter.com/dsquintana James on twitter https://www.twitter.com/jamesheathers Everything Hertz on twitter https://www.twitter.com/hertzpodcast Everything Hertz on Facebook https://www.facebook.com/everythingh

Dan and James chat with Jon Brock (Cognitive scientist at Frankl) about the use of blockchain technology for open science. Here's what they cover: What is the blockchain? Why Jon made the jump from academia to Frankl A cryptocurrency for open science What do institutional review boards think about using blockchain for data collection and storage? Autism heterogeneity How will this approach scale to biological signals and genetics data? What’s something that Jon’s changed him mind about in regards to academia? Links Frankl https://frankl.io Five reasons Frankl has a token https://medium.com/franklopenscience/why-does-frankl-need-a-frankl-token-4129d718ab74 Bjoern Brembs blog post http://bjoern.brembs.net/2018/05/after-24-years-when-will-academic-culture-finally-shift/ An explainer on cryptographic hashes https://en.wikipedia.org/wiki/Cryptographichashfunction#Illustration Frankl in a nutshell https://medium.com/franklopenscience/frankl-in-a-nutshell-9b488c554dea Frankl for autism https://medium.com/franklope