The following question needs to be addressed and answered:
1. Canthe value and validity of research results be convincingly shown without P values? If so, how? If not, why not?
Full Terms & Conditions of access and use can be found at https://amstat.tandfonline.com/action/journalInformation?journalCode=utas20 The American Statistician ISSN: 0003-1305 (Print) 1537-2731 (Online) Journal homepage: https://amstat.tandfonline.com/loi/utas20 Moving to a World Beyond “p < 0.05” ronald="" l.="" wasserstein,="" allen="" l.="" schirm="" &="" nicole="" a.="" lazar="" to="" cite="" this="" article:="" ronald="" l.="" wasserstein,="" allen="" l.="" schirm="" &="" nicole="" a.="" lazar="" (2019)="" moving="" to="" a="" world="" beyond="">< 0.05”, the="" american="" statistician,="" 73:sup1,="" 1-19,="" doi:="" 10.1080/00031305.2019.1583913="" to="" link="" to="" this="" article:="" https://doi.org/10.1080/00031305.2019.1583913="" ©="" 2019="" the="" author(s).="" published="" by="" informa="" uk="" limited,="" trading="" as="" taylor="" &="" francis="" group.="" published="" online:="" 20="" mar="" 2019.="" submit="" your="" article="" to="" this="" journal="" article="" views:="" 76682="" view="" crossmark="" data="" citing="" articles:="" 8="" view="" citing="" articles="" https://amstat.tandfonline.com/action/journalinformation?journalcode="utas20" https://amstat.tandfonline.com/loi/utas20="" https://amstat.tandfonline.com/action/showcitformats?doi="10.1080/00031305.2019.1583913" https://doi.org/10.1080/00031305.2019.1583913="" https://amstat.tandfonline.com/action/authorsubmission?journalcode="utas20&show=instructions" https://amstat.tandfonline.com/action/authorsubmission?journalcode="utas20&show=instructions" http://crossmark.crossref.org/dialog/?doi="10.1080/00031305.2019.1583913&domain=pdf&date_stamp=2019-03-20" http://crossmark.crossref.org/dialog/?doi="10.1080/00031305.2019.1583913&domain=pdf&date_stamp=2019-03-20" https://amstat.tandfonline.com/doi/citedby/10.1080/00031305.2019.1583913#tabmodule="" https://amstat.tandfonline.com/doi/citedby/10.1080/00031305.2019.1583913#tabmodule="" the="" american="" statistician="" 2019,="" vol.="" 73,="" no.="" s1,="" 1–19:="" editorial="" https://doi.org/10.1080/00031305.2019.1583913="" editorial="" moving="" to="" a="" world="" beyond="" “p="">< 0.05”="" some="" of="" you="" exploring="" this="" special="" issue="" of="" the="" american="" statis-="" tician="" might="" be="" wondering="" if="" it’s="" a="" scolding="" from="" pedantic="" statis-="" ticians="" lecturing="" you="" about="" what="" not="" to="" do="" with="" p-values,="" without="" offering="" any="" real="" ideas="" of="" what="" to="" do="" about="" the="" very="" hard="" problem="" of="" separating="" signal="" from="" noise="" in="" data="" and="" making="" decisions="" under="" uncertainty.="" fear="" not.="" in="" this="" issue,="" thanks="" to="" 43="" innovative="" and="" thought-provoking="" papers="" from="" forward-looking="" statisti-="" cians,="" help="" is="" on="" the="" way.="" 1.="" “don’t”="" is="" not="" enough="" there’s="" not="" much="" we="" can="" say="" here="" about="" the="" perils="" of="" p-values="" and="" significance="" testing="" that="" hasn’t="" been="" said="" already="" for="" decades="" (ziliak="" and="" mccloskey="" 2008;="" hubbard="" 2016).="" if="" you’re="" just="" arriv-="" ing="" to="" the="" debate,="" here’s="" a="" sampling="" of="" what="" not="" to="" do:="" •="" don’t="" base="" your="" conclusions="" solely="" on="" whether="" an="" association="" or="" effect="" was="" found="" to="" be="" “statistically="" significant”="" (i.e.,="" the="" p-="" value="" passed="" some="" arbitrary="" threshold="" such="" as="" p="">< 0.05).="" •="" don’t="" believe="" that="" an="" association="" or="" effect="" exists="" just="" because="" it="" was="" statistically="" significant.="" •="" don’t="" believe="" that="" an="" association="" or="" effect="" is="" absent="" just="" because="" it="" was="" not="" statistically="" significant.="" •="" don’t="" believe="" that="" your="" p-value="" gives="" the="" probability="" that="" chance="" alone="" produced="" the="" observed="" association="" or="" effect="" or="" the="" probability="" that="" your="" test="" hypothesis="" is="" true.="" •="" don’t="" conclude="" anything="" about="" scientific="" or="" practical="" impor-="" tance="" based="" on="" statistical="" significance="" (or="" lack="" thereof).="" don’t.="" don’t.="" just…don’t.="" yes,="" we="" talk="" a="" lot="" about="" don’ts.="" the="" asa="" statement="" on="" p-values="" and="" statistical="" significance="" (wasserstein="" and="" lazar="" 2016)="" was="" developed="" primarily="" because="" after="" decades,="" warnings="" about="" the="" don’ts="" had="" gone="" mostly="" unheeded.="" the="" statement="" was="" about="" what="" not="" to="" do,="" because="" there="" is="" widespread="" agreement="" about="" the="" don’ts.="" knowing="" what="" not="" to="" do="" with="" p-values="" is="" indeed="" necessary,="" but="" it="" does="" not="" suffice.="" it="" is="" as="" though="" statisticians="" were="" asking="" users="" of="" statistics="" to="" tear="" out="" the="" beams="" and="" struts="" holding="" up="" the="" edifice="" of="" modern="" scientific="" research="" without="" offering="" solid="" construction="" materials="" to="" replace="" them.="" pointing="" out="" old,="" rotting="" timbers="" was="" a="" good="" start,="" but="" now="" we="" need="" more.="" recognizing="" this,="" in="" october="" 2017,="" the="" american="" statistical="" association="" (asa)="" held="" the="" symposium="" on="" statistical="" infer-="" ence,="" a="" two-day="" gathering="" that="" laid="" the="" foundations="" for="" this="" special="" issue="" of="" the="" american="" statistician.="" authors="" were="" explic-="" itly="" instructed="" to="" develop="" papers="" for="" the="" variety="" of="" audiences="" interested="" in="" these="" topics.="" if="" you="" use="" statistics="" in="" research,="" busi-="" ness,="" or="" policymaking="" but="" are="" not="" a="" statistician,="" these="" articles="" were="" indeed="" written="" with="" you="" in="" mind.="" and="" if="" you="" are="" a="" statistician,="" there="" is="" still="" much="" here="" for="" you="" as="" well.="" the="" papers="" in="" this="" issue="" propose="" many="" new="" ideas,="" ideas="" that="" in="" our="" determination="" as="" editors="" merited="" publication="" to="" enable="" broader="" consideration="" and="" debate.="" the="" ideas="" in="" this="" editorial="" are="" likewise="" open="" to="" debate.="" they="" are="" our="" own="" attempt="" to="" distill="" the="" wisdom="" of="" the="" many="" voices="" in="" this="" issue="" into="" an="" essence="" of="" good="" statistical="" practice="" as="" we="" currently="" see="" it:="" some="" do’s="" for="" teaching,="" doing="" research,="" and="" informing="" decisions.="" yet="" the="" voices="" in="" the="" 43="" papers="" in="" this="" issue="" do="" not="" sing="" as="" one.="" at="" times="" in="" this="" editorial="" and="" the="" papers="" you’ll="" hear="" deep="" dissonance,="" the="" echoes="" of="" “statistics="" wars”="" still="" simmering="" today="" (mayo="" 2018).="" at="" other="" times="" you’ll="" hear="" melodies="" wrapping="" in="" a="" rich="" counterpoint="" that="" may="" herald="" an="" increasingly="" harmonious="" new="" era="" of="" statistics.="" to="" us,="" these="" are="" all="" the="" sounds="" of="" statistical="" inference="" in="" the="" 21st="" century,="" the="" sounds="" of="" a="" world="" learning="" to="" venture="" beyond="" “p="">< 0.05.”="" this="" is="" a="" world="" where="" researchers="" are="" free="" to="" treat="" “p="0.051”" and="" “p="0.049”" as="" not="" being="" categorically="" different,="" where="" authors="" no="" longer="" find="" themselves="" constrained="" to="" selectively="" publish="" their="" results="" based="" on="" a="" single="" magic="" number.="" in="" this="" world,="" where="" studies="" with="" “p="">< 0.05”="" and="" studies="" with="" “p=""> 0.05” are not automatically in conflict, researchers will see their results more easily replicated—and, even when not, they will better understand why. As we venture down this path, we will begin to see fewer false alarms, fewer overlooked discoveries, and the development of more customized statistical strategies. Researchers will be free to communicate all their findings in all their glorious uncertainty, knowing their work is to be judged by the quality and effective communication of their science, and not by their p-values. As “statistical significance” is used less, statistical thinking will be used more. The ASA Statement on P-Values and Statistical Significance started moving us toward this world. As of the date of publi- cation of this special issue, the statement has been viewed over 294,000 times and cited over 1700 times—an average of about 11 citations per week since its release. Now we must go further. That’s what this special issue of The American Statistician sets out to do. To get to the do’s, though, we must begin with one more don’t. © 2019 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. https://doi.org/10.1080/00031305.2019.1583913 https://crossmark.crossref.org/dialog/?doi=10.1080/00031305.2019.1583913&domain=pdf&date_stamp=2019-03-14 http://creativecommons.org/licenses/by-nc-nd/4.0/ 2 EDITORIAL 2. Don’t Say “Statistically Significant” The ASA Statement on P-Values and Statistical Significance stopped just short of recommending that declarations of “statistical significance” be abandoned. We take that step here. We conclude, based on our review of the articles in this special issue and the broader literature, that it is time to stop using the term “statistically significant” entirely. Nor should variants such as “significantly different,” “p < 0.05,” and “nonsignificant” survive, whether expressed in words, by asterisks in a table, or in some other way. regardless of whether it was ever useful, a declaration of “statistical significance” has today become meaningless. made broadly known by fisher’s use of the phrase (1925), edgeworth’s (1885) original intention for statistical significance was simply as a tool to indicate when a result warrants further scrutiny. but that idea has been irretrievably lost. statistical significance was never meant to imply scientific importance, and the confusion of the two was decried soon after its widespread use (boring 1919). yet a full century later the confusion persists. and so the tool has become the tyrant. the problem is not simply use of the word “significant,” although the statistical and ordinary language meanings of the word are indeed now hope- lessly confused (ghose 2013); the term should be avoided for that reason alone. the problem is a larger one, however: using bright-line rules for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision making (asa statement, principle 3). a label of statistical significance adds nothing to what is already conveyed by the value of p; in fact, this dichotomization of p-values makes matters worse. for example, no p-value can reveal the plausibility, presence, truth, or importance of an association or effect. therefore, a label of statistical significance does not mean or imply that an association or effect is highly probable, real, true, or important. nor does a label of statistical nonsignificance lead to the associ- ation or effect being improbable, absent, false, or unimportant. yet the dichotomization into “significant” and “not significant” is taken as an imprimatur of authority on these characteristics. in a world without bright lines, on the other hand, it becomes untenable to assert dramatic differences in interpretation from inconsequential differences in estimates. as gelman and stern (2006) famously observed, the difference between “significant” and “not significant” is not itself statistically significant. furthermore, this false split into “worthy” and “unworthy” results leads to the selective reporting and publishing of results based on their statistical significance—the so-called “file drawer problem” (rosenthal 1979). and the dichotomized reporting problem extends beyond just publication, notes amrhein, trafimow, and greenland (2019): when authors use p-value thresholds to select which findings to discuss in their papers, “their conclusions and what is reported in subsequent news and reviews will be biased…such selective attention based on study outcomes will therefore not only distort the literature but will slant published descriptions of study results—biasing the summary descriptions reported to practicing professionals and the general public.” for the integrity of scientific publishing and research dissemination, therefore, whether a p-value passes any arbitrary threshold should not be considered at all when deciding which results to present or highlight. to be clear, the problem is not that of having only two labels. results should not be trichotomized, or indeed catego- rized into any number of groups, based on arbitrary p-value thresholds. similarly, we need to stop using confidence inter- vals as another means of dichotomizing (based, on whether a null value falls within the interval). and, to preclude a reap- pearance of this problem elsewhere, we must not begin arbi- trarily categorizing other statistical measures (such as bayes factors). despite the limitations of p-values (as noted in princi- ples 5 and 6 of the asa statement), however, we are not recommending that the calculation and use of continuous p-values be discontinued. where p-values are used, they should be reported as continuous quantities (e.g., p = 0.08). they should also be described in language stating what the value means in the scientific context. we believe that a reasonable prerequisite for reporting any p-value is the ability to interpret it appropriately. we say more about this in section 3.3. to move forward to a world beyond 0.05,”="" and="" “nonsignificant”="" survive,="" whether="" expressed="" in="" words,="" by="" asterisks="" in="" a="" table,="" or="" in="" some="" other="" way.="" regardless="" of="" whether="" it="" was="" ever="" useful,="" a="" declaration="" of="" “statistical="" significance”="" has="" today="" become="" meaningless.="" made="" broadly="" known="" by="" fisher’s="" use="" of="" the="" phrase="" (1925),="" edgeworth’s="" (1885)="" original="" intention="" for="" statistical="" significance="" was="" simply="" as="" a="" tool="" to="" indicate="" when="" a="" result="" warrants="" further="" scrutiny.="" but="" that="" idea="" has="" been="" irretrievably="" lost.="" statistical="" significance="" was="" never="" meant="" to="" imply="" scientific="" importance,="" and="" the="" confusion="" of="" the="" two="" was="" decried="" soon="" after="" its="" widespread="" use="" (boring="" 1919).="" yet="" a="" full="" century="" later="" the="" confusion="" persists.="" and="" so="" the="" tool="" has="" become="" the="" tyrant.="" the="" problem="" is="" not="" simply="" use="" of="" the="" word="" “significant,”="" although="" the="" statistical="" and="" ordinary="" language="" meanings="" of="" the="" word="" are="" indeed="" now="" hope-="" lessly="" confused="" (ghose="" 2013);="" the="" term="" should="" be="" avoided="" for="" that="" reason="" alone.="" the="" problem="" is="" a="" larger="" one,="" however:="" using="" bright-line="" rules="" for="" justifying="" scientific="" claims="" or="" conclusions="" can="" lead="" to="" erroneous="" beliefs="" and="" poor="" decision="" making="" (asa="" statement,="" principle="" 3).="" a="" label="" of="" statistical="" significance="" adds="" nothing="" to="" what="" is="" already="" conveyed="" by="" the="" value="" of="" p;="" in="" fact,="" this="" dichotomization="" of="" p-values="" makes="" matters="" worse.="" for="" example,="" no="" p-value="" can="" reveal="" the="" plausibility,="" presence,="" truth,="" or="" importance="" of="" an="" association="" or="" effect.="" therefore,="" a="" label="" of="" statistical="" significance="" does="" not="" mean="" or="" imply="" that="" an="" association="" or="" effect="" is="" highly="" probable,="" real,="" true,="" or="" important.="" nor="" does="" a="" label="" of="" statistical="" nonsignificance="" lead="" to="" the="" associ-="" ation="" or="" effect="" being="" improbable,="" absent,="" false,="" or="" unimportant.="" yet="" the="" dichotomization="" into="" “significant”="" and="" “not="" significant”="" is="" taken="" as="" an="" imprimatur="" of="" authority="" on="" these="" characteristics.="" in="" a="" world="" without="" bright="" lines,="" on="" the="" other="" hand,="" it="" becomes="" untenable="" to="" assert="" dramatic="" differences="" in="" interpretation="" from="" inconsequential="" differences="" in="" estimates.="" as="" gelman="" and="" stern="" (2006)="" famously="" observed,="" the="" difference="" between="" “significant”="" and="" “not="" significant”="" is="" not="" itself="" statistically="" significant.="" furthermore,="" this="" false="" split="" into="" “worthy”="" and="" “unworthy”="" results="" leads="" to="" the="" selective="" reporting="" and="" publishing="" of="" results="" based="" on="" their="" statistical="" significance—the="" so-called="" “file="" drawer="" problem”="" (rosenthal="" 1979).="" and="" the="" dichotomized="" reporting="" problem="" extends="" beyond="" just="" publication,="" notes="" amrhein,="" trafimow,="" and="" greenland="" (2019):="" when="" authors="" use="" p-value="" thresholds="" to="" select="" which="" findings="" to="" discuss="" in="" their="" papers,="" “their="" conclusions="" and="" what="" is="" reported="" in="" subsequent="" news="" and="" reviews="" will="" be="" biased…such="" selective="" attention="" based="" on="" study="" outcomes="" will="" therefore="" not="" only="" distort="" the="" literature="" but="" will="" slant="" published="" descriptions="" of="" study="" results—biasing="" the="" summary="" descriptions="" reported="" to="" practicing="" professionals="" and="" the="" general="" public.”="" for="" the="" integrity="" of="" scientific="" publishing="" and="" research="" dissemination,="" therefore,="" whether="" a="" p-value="" passes="" any="" arbitrary="" threshold="" should="" not="" be="" considered="" at="" all="" when="" deciding="" which="" results="" to="" present="" or="" highlight.="" to="" be="" clear,="" the="" problem="" is="" not="" that="" of="" having="" only="" two="" labels.="" results="" should="" not="" be="" trichotomized,="" or="" indeed="" catego-="" rized="" into="" any="" number="" of="" groups,="" based="" on="" arbitrary="" p-value="" thresholds.="" similarly,="" we="" need="" to="" stop="" using="" confidence="" inter-="" vals="" as="" another="" means="" of="" dichotomizing="" (based,="" on="" whether="" a="" null="" value="" falls="" within="" the="" interval).="" and,="" to="" preclude="" a="" reap-="" pearance="" of="" this="" problem="" elsewhere,="" we="" must="" not="" begin="" arbi-="" trarily="" categorizing="" other="" statistical="" measures="" (such="" as="" bayes="" factors).="" despite="" the="" limitations="" of="" p-values="" (as="" noted="" in="" princi-="" ples="" 5="" and="" 6="" of="" the="" asa="" statement),="" however,="" we="" are="" not="" recommending="" that="" the="" calculation="" and="" use="" of="" continuous="" p-values="" be="" discontinued.="" where="" p-values="" are="" used,="" they="" should="" be="" reported="" as="" continuous="" quantities="" (e.g.,="" p="0.08)." they="" should="" also="" be="" described="" in="" language="" stating="" what="" the="" value="" means="" in="" the="" scientific="" context.="" we="" believe="" that="" a="" reasonable="" prerequisite="" for="" reporting="" any="" p-value="" is="" the="" ability="" to="" interpret="" it="" appropriately.="" we="" say="" more="" about="" this="" in="" section="" 3.3.="" to="" move="" forward="" to="" a="" world="">