# title
Scientific Evidence Code System (SEVCO) -- Release 2.0

# description
The Scientific Evidence Code System (SEVCO) includes terms for describing scientific research, including study design, risk of bias, and statistical concepts. For additional derivation details for any term, see https://fevir.net/resources/CodeSystem/27270

For SEVCO Release 2.0:

Authors/Editors: Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, Mario Tristan, Paul Whaley, Kenneth Wilkins, Khalid Shahin, Erfan Shamsoddin, Michael Panzer, Asiyah Yu Lin, Ellen Jepson, Ilkka Kunnamo, Neeraj Ojha, Olga Vovk, Airton Tetelbom Stein, Bhagvan Kommadi, Carlos Alva-Diaz, Cauê Monaco, Homa Keshavarz, Javier Bracchiglione, Kevin O'Brien. Li Wang, Muhammad Afzal, Philippe Rocca-Serra, Saphia Mokrane, Sheyu Li, Xing Song, Yuan Gao

Additional Endorsers: Jesús López-Alcalde, Eric Harvey, Paola Rosati, Robin Ann Yurk, Alejandro Piscoya, Cauê Monaco, Cheow Peng Ooi, Nisha Mathew, Leo Orozco, Nelle Stocquart, Sorana D. Bolboaca, KM Saif-Ur-Rahman, A.G. Radhika, Andrew Beck, Louis Leff, Tatyana Shamliyan, Sunu Alice Cherian, Yasser Sami Amer, Sebastien Bailly, Eric Au, Paul Harris, Ahmad Sofi-Mahmudi, Elma Omeragic. Jennifer Hunter, Lara Kahaleh, Lenny Vasanthan, Mahnoor Ahmed, Morufu Olalekan Raimi, Sean Grant, Sumalatha Arunachala, Yaowaluk Ngoenwiwatkul

Additional Reviewers: Rebecca Baker, Arnav Agarwal, Kailei Nong

for SEVCO Release 1.1 -- see https://fevir.net/resources/CodeSystem/103845
for SEVCO Release 1.0 -- see https://fevir.net/resources/CodeSystem/52433

# concept[].definition
A plan specification for how and what kinds of data will be gathered as part of an investigation which may produce testable explanations, conclusions and predictions or test a hypothesis.

# concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Asiyah Lin, Mario Tristan, Neeraj Ojha

# concept[].property[].valueString
9/9 as of 4/26/2021: Eric Harvey, Bhagvan Kommadi, KM Saif-Ur-Rahman, Paola Rosati, Jesús López-Alcalde, Tatyana Shamliyan, Sorana D. Bolboaca, Asiyah Lin, Eric Au

# concept[].concept[].definition
A study design in which an independent variable (an exposure or intervention) is prospectively assigned or modified by the investigator to evaluate a response in the dependent variable (an effect or outcome).

# concept[].concept[].property[].valueString
Mario Tristan, Joanne Dehnbostel, Harold Lehmann, Khalid Shahin, Brian S. Alper

# concept[].concept[].property[].valueString
12/12 as of 5/31/2021: Eric Harvey, Bhagvan Kommadi, Brian Alper, Sebastien Bailly, Alejandro Piscoya, Harold Lehmann, KM Saif-Ur-Rahman, Paola Rosati, Sorana D. Bolboaca, Asiyah Lin, Leo Orozco, Erfan Shamsoddin

# concept[].concept[].property[].valueString
We acknowledge that interventional study design and interventional study  may not be exact synonyms of interventional research, but interventional research could be used to encompass both design and implementation of the design

# concept[].concept[].concept[].definition
An interventional study design in which an independent variable (an exposure or intervention) is prospectively assigned or modified by random chance to separate groups.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Mario Tristan, Kenneth Wilkins, Erfan Shamsoddin, Ellen Jepson

# concept[].concept[].concept[].property[].valueString
8/8 as of 7/19/2021: Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A randomized assignment in which each participant has the same prespecified likelihood of being assigned to a group as all other participants, independent of the assignment of any other participant.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Mario Tristan, Kenneth Wilkins, Erfan Shamsoddin, Ellen Jepson

# concept[].concept[].concept[].concept[].property[].valueString
8/8 as of 7/19/2021: Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A randomized assignment in which participants are stratified into groups based on prognostic variables and then randomized into balanced treatment groups

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
8/8 as of 7/19/2021: Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A randomized assignment in which a pre-specified number of subjects is assigned to a block containing the same pre-specified ratio of group assignments in random order.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 7/26/2021: Mario Tristan, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Leo Orozco, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2021-07-19 vote 7-1 on "A randomized assignment in which a pre-specified number of subjects is assigned to a block containing the same pre-specified number of balanced group assignments in random order" by Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
I think I'm hung up on the word "balanced". Does allocation in block design need to be balanced? Couldn't a block design allocate subjects to treatment arms in a 2:1, or other "unbalanced" ratio?

# concept[].concept[].concept[].concept[].definition
A randomized assignment in which a participant’s group assignment probability is adjusted based on any factor such that the likelihood of assignment is not the same for all participants.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Mario Tristan, Kenneth Wilkins, Erfan Shamsoddin, Ellen Jepson

# concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 8/9/2021: Erfan Shamsoddin, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Joanne Dehnbostel, Alejandro Piscoya

# concept[].concept[].concept[].concept[].property[].valueString
2021-07-19 vote 7-1 on "A randomized assignment in which a participant’s group assignment probability is adjusted based on any factor such that the likelihood of assignment is not the same for all participants." by Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte, 2021-07-26 vote 6-1 by Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].concept[].property[].valueString
I deem this kind of adaptation could determine conflict of interests or a new kind of bias. I disagree with adding an adaptive randomization as a new term 7-26-21 comment: Again, why and for what you wish to maintain this term? I think the term adaptive randomization risks a severe selection bias. In ethical terms, I deem there is no justification to proceed with such a methodology in clinical trials.

# concept[].concept[].concept[].definition
An interventional study design in which an independent variable (an exposure or intervention) is prospectively assigned or modified by methods other than random chance to separate groups.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Michael Panzer, Janice Tufte, Erfan Shamsoddin, Ellen Jepson, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
9/9 as of 8/9/2021: Erfan Shamsoddin, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Joanne Dehnbostel, Alejandro Piscoya

# concept[].concept[].concept[].property[].valueString
2021-07-19 vote 6-2 on "An interventional study design in which an independent variable (an exposure or intervention) is prospectively assigned or modified by methods other than random chance to separate groups." by Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana D. Bolboaca, Janice Tufte, 2021-07-26 vote 6-1 by Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].property[].valueString
In this case, if the patients choose which is the arm they want to be in it would beok to insert this term. I presumetherefore that if the choice is made bythe researchers they offer a clearjustification for it in the protocol As written, this category would include all quasi-randomized designs. If this is the intent, fine. If this was not the intent, perhaps we could change "..randomized.." to "..randomized or quasi-randomized.." 7-26-21 comment: We usually have started the definitions by saying "A xxx assignment that..." (see previous ones in this page). That is, we define the assigment. However, for"Non-Randomized Assignment" we start by saying "An interventional study design..." I propose to describe the "assignment" (avoid starting by defining the study design itself)

# concept[].concept[].concept[].concept[].definition
An interventional study design with a method of allocation that is not limited to random chance but is intended to produce similar baseline groups for experimentation.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Michael Panzer, Janice Tufte, Erfan Shamsoddin, Ellen Jepson, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 7/26/2021: Mario Tristan, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Leo Orozco, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2021-07-19 vote 6-2 on "An interventional study design with a method of allocation that is not limited to random chance but is intended to produce similar baseline groups for experimentation." by Alejandro Piscoya, Harold Lehmann, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Sorana d. Bolboaca, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
Which is the difference between thisquasi-randomized assignment and theadaptive randomization? It is unclearwhy we should insert these two terms inthe glossary I would specify in the definition thatquasi-randomisation is a non-randommethod of allocation

# concept[].concept[].concept[].concept[].property[].valueString
Quasi-random methods of allocation include allocation by alternate order of entry, date of birth, day of the week, month of the year, or medical record number

# concept[].concept[].concept[].definition
Interventional research in which one or more healthcare-related actions (i.e., a diagnostic, prognostic, therapeutic, preventive or screening method or intervention) is evaluated for effects on health-related biomedical or behavioral processes and/or outcomes.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].property[].valueString
2021-11-30 vote 7-1 by Alejandro Piscoya, Mario Tristan, Robin Ann Yurk, Muhammad Afzal, Paola Rosati, Paul Whaley, Janice Tufte, Jesus Lopez-Alcalde
2021-12-07 vote 4-1 by Mario Tristan, Robin Ann Yurk, Janice Tufte, Joanne Dehnbostel, CP Ooi

# concept[].concept[].concept[].property[].valueString
2021-11-30 comments: (We should include the classical definition for  Phase lV Field Trials of Health Interventions: A Toolbox. 3rd edition.
Smith PG, Morrow RH, Ross DA, editors.
Oxford (UK): OUP Oxford; 2015 Jun 1.https://www.ncbi.nlm.nih.gov/books/NBK305508/), Instead of "methods" I would use the term "interventions". I also miss the term "prognostic" as they are not diagnostic or screening. Besides, it would be important to highlight that the clinical trial is done in humans

2021-12-07 comment: A clinical trial is a type of research that studies new tests and treatments and evaluates their effects on human health outcomes. The medical intervention can be drugs, cells and other biological products, surgical procedures, radiological procedures, devices, behavioural treatments and preventive care.

# concept[].concept[].concept[].property[].valueString
Some definitions for "clinical trial" include human subject research for effects on human health outcomes. The term "human" was not added to this definition because a study design with animal subjects for effects on animal health outcomes to inform veterinary care would be considered a clinical trial. However, a study design with animal subjects to inform human health outcomes would not be considered a clinical trial.

# concept[].concept[].concept[].property[].valueString
NIH Clinical Trial Definition = A research study[1] in which one or more human subjects[2] are prospectively assigned[3] to one or more interventions[4] (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.[5] 

[4]An intervention is defined as a manipulation of the subject or subject’s environment for the purpose of modifying one or more health-related biomedical or behavioral processes and/or endpoints.  Examples include:  drugs/small molecules/compounds; biologics; devices; procedures (e.g., surgical techniques); delivery systems (e.g., telemedicine, face-to-face interviews); strategies to change health-related behavior (e.g., diet, cognitive therapy, exercise, development of new habits); treatment strategies; prevention strategies; and, diagnostic strategies.

from https://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-015.html

# concept[].concept[].concept[].property[].valueString
2021-12-14 vote 6-0 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Robin Ann Yurk, Janice Tufte, Paul Whaley, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
A clinical trial conducted under conditions of routine clinical practice.

# concept[].concept[].concept[].concept[].property[].valueString
NCIt:  Pragmatic Trial = A study designed to test the effectiveness of an intervention in a broad routine clinical practice. Term used to describe a clinical study designed to examine the benefits of a product under real world conditions.
UMLS: Works about randomized clinical trials that compare interventions in clinical settings and which look at a range of effectiveness outcomes and impacts.
CDISC Glossary: pragmatic trial = Term used to describe a clinical study designed to examine the benefits of a product under real world conditions.
EDDA: pragmatic clinical trial = Randomized clinical trials that compare interventions in clinical settings and which look at a range of effectiveness outcomes and impacts. [MeSH_2015]
SCO: pragmatic trial = A study designed to test the effectiveness of an intervention in a broad routine clinical practice.
"Pragmatic trials are designed to evaluate the effectiveness of interventions in real-life routine practice conditions, whereas explanatory trials aim to test whether an intervention works under optimal situations. The pragmatic trial, on the other hand, is designed to test interventions in the full spectrum of everyday clinical settings in order to maximize applicability and generalizability. The research question under investigation is whether an intervention actually works in real life." (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3181997/)

# concept[].concept[].concept[].concept[].property[].valueString
"Pragmatic trials are designed to evaluate the effectiveness of interventions in real-life routine practice conditions, whereas explanatory trials aim to test whether an intervention works under optimal situations. The pragmatic trial, on the other hand, is designed to test interventions in the full spectrum of everyday clinical settings in order to maximize applicability and generalizability. The research question under investigation is whether an intervention actually works in real life." (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3181997/)

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Janice Tufte, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2021-11-30 vote 5-1 by Alejandro Piscoya, Robin Ann Yurk, Muhammad Afzal, Paul Whaley, Janice Tufte, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].property[].valueString
2021-11-30 comments: (The definition in the current form is fine however the last part may be thought like;
where "everyday" means day-to-day clinical practice wherein the conditions are not modified for the conduct of the research.),  Suggested alternative: = A clinical trial designed to test the effects of an intervention under everyday conditions, where "everyday conditions" means clinical conditions are not modified for the conduct of the research

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-07 vote 5-0 by Mario Tristan, Robin Ann Yurk, Janice Tufte, CP Ooi, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].definition
A clinical trial that provides a means for obtaining an experimental drug or device for patients who are not adequately treated by existing therapy, who do not meet the eligibility criteria for enrollment, or who are otherwise unable to participate in another clinical study.

# concept[].concept[].concept[].concept[].property[].valueString
from CTO:

Expanded Access Study

Studies that provide a means for obtaining an experimental drug or device for patients who are not adequately treated by existing therapy, who do not meet the eligibility criteria for enrollment, or who are otherwise unable to participate in another clinical study. Expanded Access studies include individual-patient IND, treatment IND, compassionate use, emergency use or continued access. An investigational drug product (including biological product) available through expanded access for patients who do not qualify for enrollment in a clinical trial. Expanded Access includes all expanded access types under section 561 of the Federal Food, Drug, and Cosmetic Act: (1) for individual patients, including emergency use; (2) for intermediate-size patient populations; and (3) under a treatment IND or treatment protocol.


from NCIt:

Expanded Access Study

Studies that provide a means for obtaining an experimental drug or device for patients who are not adequately treated by existing therapy, who do not meet the eligibility criteria for enrollment, or who are otherwise unable to participate in another clinical study. Expanded Access studies include individual-patient IND, treatment IND, compassionate use, emergency use or continued access. 


also Compassionate Treatment (compassionate use trial, expanded access trial, pre-approval access)

Providing experimental therapies to very sick individuals even though they don't meet the critera for inclusion in a trial.

A way to provide an investigational therapy to a patient who is not eligible to receive that therapy in a clinical trial, but who has a serious or life-threatening illness for which other treatments are not available. Compassionate use trials allow patients to receive promising but not yet fully studied or approved cancer therapies when no other treatment option exists.

A potential pathway for a patient with an immediately life-threatening condition or serious disease or condition to gain access to an investigational medical product (drug, biologic, or medical device) for treatment outside of clinical trials when no comparable or satisfactory alternative therapy options are available. NOTE: The intent is treatment, as opposed to research. Individual, Intermediate-size, and Widespread Use Expanded Access, also Emergency IND, are all programs administered under FDA guidelines. Additionally, the US Right-to-Try Act, which is independent of FDA, expands access. [FDA Expanded Access: Information for Physicians]



from EDDA:

compassionate use trial (expanded access trial, compassionate treatment)

Providing experimental therapies to very sick individuals even though they don't meet the critera for inclusion in a trial. [NCI 2014_12E]

Providing an investigational therapy to a patient who is not eligible to receive that therapy in a clinical trial, but who has a serious or life-threatening illness for which other treatments are not available. Compassionate use trials allow patients to receive promising but not yet fully studied or approved therapies when no other treatment option exists. Also called expanded access trial. [MeSH 2014_2014_02_10]


shared as a comment: Expanded access is the use of an investigational new drug, biologics, and medical devices used to diagnose, monitor, or treat patients with serious diseases or conditions for which there are no comparable or satisfactory therapy options available outside of clinical trials. (USA FDA)

# concept[].concept[].concept[].concept[].property[].valueString
Expanded Access studies include individual-patient investigational new drug (IND), treatment IND, compassionate use, emergency use or continued access.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-15 vote 10-0 by Paul Whaley, Andrew Beck, Brian S. Alper, Paola Rosati, Robin Ann Yurk, Janice Tufte, Jesus Lopez-Alcalde, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-15 comment: Define IND acronym under comment for application under individual patient IND, treatment IND

# concept[].concept[].concept[].concept[].definition
A clinical trial to gather initial evidence in humans to support further investigation of an intervention.

# concept[].concept[].concept[].concept[].property[].valueString
https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-312/subpart-B/section-312.21 is the US Code of Federal Regulations Title 21 (Food and Drugs) Chapter I Subchapter D Part 312 Subpart B § 312.21 and includes:

§ 312.21 Phases of an investigation.

An IND may be submitted for one or more phases of an investigation. The clinical investigation of a previously untested drug is generally divided into three phases. Although in general the phases are conducted sequentially, they may overlap. These three phases of an investigation are a[sic] follows:

....

Phase 1. 

(1) Phase 1 includes the initial introduction of an investigational new drug into humans. Phase 1 studies are typically closely monitored and may be conducted in patients or normal volunteer subjects. These studies are designed to determine the metabolism and pharmacologic actions of the drug in humans, the side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. During Phase 1, sufficient information about the drug's pharmacokinetics and pharmacological effects should be obtained to permit the design of well-controlled, scientifically valid, Phase 2 studies. The total number of subjects and patients included in Phase 1 studies varies with the drug, but is generally in the range of 20 to 80. 

(2) Phase 1 studies also include studies of drug metabolism, structure-activity relationships, and mechanism of action in humans, as well as studies in which investigational drugs are used as research tools to explore biological phenomena or disease processes.



from CTO:

Phase I trial (phase I study, early-stage clinical trial, phase I protocol, phase I clinical trial, trial phase 1)
A clinical research protocol designed to test a new biomedical intervention in a small group of people for the first time. A Phase I trial can be to establish the toxicity of a new treatment with escalating intensity of the treatment administered and/or to determine the side effects of a new treatment for a particular indication in subjects. Includes initial studies to determine the metabolism and pharmacologic actions of drugs in humans, the side effects associated with increasing doses, and to gain early evidence of effectiveness; may include healthy participants and/or patients.

	
	
The initial introduction of an investigational new drug into humans. Phase 1 studies are typically closely monitored and may be conducted in patients or normal volunteer subjects. NOTE: These studies are designed to determine the metabolism and pharmacologic actions of the drug in humans, the side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. During Phase 1, sufficient information about the drug's pharmacokinetics and pharmacological effects should be obtained to permit the design of well-controlled, scientifically valid, Phase 2 studies. The total number of subjects and patients included in Phase I studies varies with the drug, but is generally in the range of 20 to 80. Phase 1 studies also include studies of drug metabolism, structure-activity relationships, and mechanism of action in humans, as well as studies in which investigational drugs are used as research tools to explore biological phenomena or disease processes. [After FDA CDER Handbook, ICH E8] (CDISC glossary)

The first step in testing a new treatment in humans. These studies test the best way to give a new treatment (for example, by mouth, intravenous infusion, or injection) and the best dose. The dose is usually increased a little at a time in order to find the highest dose that does not cause harmful side effects. Because little is known about the possible risks and benefits of the treatments being tested, phase I trials usually include only a small number of patients who have not been helped by other treatments.

The initial introduction of an investigational new drug into humans. Phase 1 studies are typically closely monitored and may be conducted in patients or normal volunteer subjects. NOTE: These studies are designed to determine the metabolism and pharmacologic actions of the drug in humans, the side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. During Phase 1, sufficient information about the drug's pharmacokinetics and pharmacological effects should be obtained to permit the design of well-controlled, scientifically valid Phase 2 studies. The total number of subjects and patients included in Phase 1 studies varies with the drug, but is generally in the range of 20 to 80. Phase 1 studies also include studies of drug metabolism, structure-activity relationships, and mechanism of action in humans, as well as studies in which investigational drugs are used as research tools to explore biological phenomena or disease processes. [after FDA CDER handbook, ICH E8]

from SCO:
	
phase I trial not independently defined

from NCIt:

same as CTO

from OCRe:
	
A Phase 1 trial includes initial studies to determine the metabolism and pharmacologic actions of drugs in humans, the side effects associated with increasing doses, and to gain early evidence of effectiveness; may include healthy participants and/or patients.

from EDDA:

A clinical research protocol designed to test a new biomedical intervention in a small group of people for the first time. A Phase I trial can be to establish the toxicity of a new treatment with escalating intensity of the treatment administered and/or to determine the side effects of a new treatment for a particular indication in subjects. [NCI 2014_12E] 

Studies performed to evaluate the safety of diagnostic, therapeutic, or prophylactic drugs, devices, or techniques in healthy subjects and to determine the safe dosage range (if appropriate). These tests also are used to determine pharmacologic and pharmacokinetic properties (toxicity, metabolism, absorption, elimination, and preferred route of administration). They involve a small number of persons and usually last about 1 year. This concept includes phase I studies conducted both in the U.S. and in other countries. [MeSH 2014_2014_02_10]

from INTERNATIONAL COUNCIL FOR HARMONISATION OF TECHNICAL REQUIREMENTS FOR PHARMACEUTICALS FOR HUMAN USE (ICH HARMONISED GUIDELINE) GENERAL CONSIDERATIONS FOR CLINICAL STUDIES E8(R1) https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2021_1006.pdf Adopted on 6 October 2021

4.3.1 Human Pharmacology
The protection of study participants should always be the first priority when designing early
clinical studies, especially for the initial administration of an investigational product to humans
(usually referred to as phase 1). These studies may be conducted in healthy volunteer
participants or in a selected population of patients who have the condition or the disease,
depending on drug properties and the objectives of the development programme.

These studies typically address one or a combination of the following aspects:

4.3.1.1 Estimation of Initial Safety and Tolerability
The initial and subsequent administration of a drug to humans is usually intended to determine
the tolerability of the dose range expected to be evaluated in later clinical studies and to
determine the nature of adverse reactions that can be expected. These studies typically include
both single and multiple dose administration.

4.3.1.2 Pharmacokinetics
Characterisation of a drug's absorption, distribution, metabolism, and excretion continues
throughout the development programme, but the preliminary characterisation is an essential
early goal. Pharmacokinetic studies are particularly important to assess the clearance of the
drug and to anticipate possible accumulation of parent drug or metabolites, interactions with
metabolic enzymes and transporters, and potential drug-drug interactions. Some
pharmacokinetic studies are commonly conducted in later phases to answer more specialised
questions. For orally administered drugs, the study of food effects on bioavailability is
important to inform the dosing instructions in relation to food. Obtaining pharmacokinetic
information in sub-populations with potentially different metabolism or excretion, such as
patients with renal or hepatic impairment, geriatric patients, children, and ethnic subgroups
should be considered (ICH E4 Dose-Response Studies, E7 Clinical Trials in Geriatric
Population, E11, and E5, respectively).

4.3.1.3 Pharmacodynamics & Early Measurement of Drug Activity
Depending on the drug and the endpoint of interest, pharmacodynamic studies and studies
relating drug levels to response (PK/PD studies) may be conducted in healthy volunteer
participants or in patients with the condition or disease. If there is an appropriate measure,
pharmacodynamic data can provide early estimates of activity and efficacy and may guide the
dosage and dose regimen in later studies.


from March 1998 https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-8-general-considerations-clinical-trials-step-5_en.pdf

3.1.3.1 Phase I (Most typical kind of study: Human Pharmacology)
Phase I starts with the initial administration of an investigational new drug into humans.
Although human pharmacology studies are typically identified with Phase I, they may also be
indicated at other points in the development sequence. Studies in this phase of development
usually have non-therapeutic objectives and may be conducted in healthy volunteer subjects
or certain types of patients, e.g. patients with mild hypertension. Drugs with significant
potential toxicity, e.g. cytotoxic drugs, are usually studied in patients. Studies in this phase
can be open, baseline controlled or may use randomisation and blinding, to improve the
validity of observations.
Studies conducted in Phase I typically involve one or a combination of the following aspects:
a) Estimation of Initial Safety and Tolerability
The initial and subsequent administration of an investigational new drug into humans is
usually intended to determine the tolerability of the dose range expected to be needed for later
clinical studies and to determine the nature of adverse reactions that can be expected. These
studies typically include both single and multiple dose administration.
b) Pharmacokinetics
Characterisation of a drug's absorption, distribution, metabolism, and excretion continues
throughout the development plan. Their preliminary characterisation is an important goal of
Phase I. Pharmacokinetics may be assessed via separate studies or as a part of efficacy, safety
and tolerance studies. Pharmacokinetic studies are particularly important to assess the
clearance of the drug and to anticipate possible accumulation of parent drug or metabolites
and potential drug-drug interactions. Some pharmacokinetic studies are commonly conducted
in later phases to answer more specialised questions. For many orally administered drugs,
especially modified release products, the study of food effects on bioavailability is important.
Obtaining pharmacokinetic information in sub-populations such as patients with impaired
elimination (renal or hepatic failure), the elderly, children, women and ethnic subgroups
should be considered. Drug-drug interaction studies are important for many drugs; these are
generally performed in phases beyond Phase I but studies in animals and in vitro studies of
metabolism and potential interactions may lead to doing such studies earlier.
c) Assessment of Pharmacodynamics
Depending on the drug and the endpoint studied, pharmacodynamic studies and studies
relating drug blood levels to response (PK/PD studies) may be conducted in healthy volunteer
subjects or in patients with the target disease. In patients, if there is an appropriate measure,
pharmacodynamic data can provide early estimates of activity and potential efficacy and may
guide the dosage and dose regimen in later studies. 
d) Early Measurement of Drug Activity
Preliminary studies of activity or potential therapeutic benefit may be conducted in Phase I as
a secondary objective. Such studies are generally performed in later phases but may be
appropriate when drug activity is readily measurable with a short duration of drug exposure in
patients at this early stage.

# concept[].concept[].concept[].concept[].property[].valueString
Phase 1 trials are often the first step in testing a new treatment in humans and may include safety assessment, measurement of metabolism and pharmacologic actions of a drug in humans, or the side effects associated with increasing doses. Phase 1 studies often include between 20 and 80 subjects, and often involve healthy subjects.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-04 vote 5-2 by Robin Ann Yurk, Harold Lehmann, janice tufte, Paola Rosati, C P Ooi,  Paul Whaley, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-04 comments: Perhaps adding the following may improve the clarity "It may include testing the best way to give a new treatment (for example, by mouth, infusion into a vein, or injection)".

 "providing the initial investigation" sounds a bit vague compared to the other trial phase definitions. Also, can a trial really "provide an investigation"? Maybe suggest changing to "in which xxx is investigated", where "xxx" is a tighter definition of what "the initial" is referring to.

2022-01-11 comment: I would suggest not adding how many subjects are typically involved, maybe state that these usually have very small sample sizes. Unfortunately, sample sizes have decreased over time. https://bmjopen.bmj.com/content/11/12/e053377

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-11 vote 7-0 by Harold Lehmann, Jesus Lopez-Alcalde, Mario Tristan, janice tufte, Paul Whaley,  Andrew Beck, Robin Ann Yurk

# concept[].concept[].concept[].concept[].concept[].display
exploratory investigational new drug study

# concept[].concept[].concept[].concept[].concept[].definition
A clinical trial that is conducted early in phase 1, involves very limited human exposure, and has no therapeutic or diagnostic intent (e.g., screening studies, microdose studies).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
the original source at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/exploratory-ind-studies
FDA GUIDANCE DOCUMENT
Exploratory IND Studies
Guidance for Industry, Investigators, and Reviewers
JANUARY 2006
investigational new drug (IND)

For the purposes of this guidance the phrase exploratory IND study is intended to describe a clinical trial that

is conducted early in phase 1,
involves very limited human exposure, and
has no therapeutic or diagnostic intent (e.g., screening studies, microdose studies). 

Such exploratory IND studies are conducted prior to the traditional dose escalation, safety, and tolerance studies that ordinarily initiate a clinical drug development program.  The duration of dosing in an exploratory IND study is expected to be limited (e.g., 7 days).


from CTO:

Early Phase I clinical trial (Phase 0 trial, Phase 0 clinical trial, Pre-Clinical Trial)

A clinical trial that is at an Early Phase i or Phase 0, which is designed to use an investigational agent that is available only in very limited quantities and which has never previously given to humans or for which there is extremely limited human experience. Phase 0 clinical trials are intended to enable researchers to understand the path of the drug in the body and its efficacy. Adverse event reporting in Phase 0 trials is expedited. [def-source: NCI] Exploratory trials, involving very limited human exposure, with no therapeutic or diagnostic intent (e.g., screening studies, microdose studies). (Formerly listed as "Phase 0") A clinical trial that is at Early Phase 1 or Phase 0

from SCO:
	
not included

from NCIt:

Preferred Name:  Exploratory Investigational New Drug Study

Definition:  A type of clinical trial that involves low dosage and short duration of drug exposure for a limited number of study participants with the intent of gathering preliminary data on the mechanism of action, pharmacodynamics, pharmacokinetics, or bioavailability of promising therapeutic candidate agents in human subjects.

CDISC-GLOSS Definition:  A clinical study that is conducted early in Phase 1; involves very limited human exposure and has no therapeutic or diagnostic intent (e.g., screening studies, microdose studies) [FDA Guidance for industry, investigators, and Reviewers: exploratory IND studies, January 2006] See also Phase 0.

First-in-Human Study = A type of phase 1 clinical trial in which the test product is administered to human beings for the first time. 

Phase 0 Trial = Pre-Clinical Trial = A clinical trial that uses an investigational agent that is available only in very limited quantities and which has never previously given to humans or for which there is extremely limited human experience. Phase 0 clinical trials are intended to enable researchers to understand the path of the drug in the body and its efficacy. Adverse event reporting in Phase 0 trials is expedited.

First-in-human trials, in a small number of subjects, that are conducted before Phase 1 trials and are intended to assess new candidate therapeutic and imaging agents. The study agent is administered at a low dose for a limited time, and there is no therapeutic or diagnostic intent. NOTE: FDA Guidance for Industry, Investigators, and Reviewers: Exploratory IND Studies, January 2006 classifies such studies as Phase 1. NOTE: A Phase 0 study might not include any drug delivery but may be an exploration of human material from a study (e.g., tissue samples or biomarker determinations). [Improving the Quality of Cancer Clinical Trials: Workshop summary-Proceedings of the National Cancer Policy Forum Workshop, improving the Quality of Cancer Clinical Trials (Washington, DC, Oct 2007)] (CDISC glossary)

First-in-human trials, in a small number of subjects, that are conducted before Phase 1 trials and are intended to assess new candidate therapeutic and imaging agents. The study agent is administered at a low dose for a limited time, and there is no therapeutic or diagnostic intent. NOTE: FDA Guidance for Industry, Investigators, and Reviewers: Exploratory IND Studies, January 2006 classifies such studies as Phase 1. NOTE: A Phase 0 study might not include any drug delivery but may be an exploration of human material from a study (e.g., tissue samples or biomarker determinations). [Improving the Quality of Cancer Clinical Trials: Workshop summary-Proceedings of the National Cancer Policy Forum Workshop, improving the Quality of Cancer Clinical Trials (Washington, DC, Oct 2007)]   )

from OCRe:
	
Phase 0 = A Phase 0 trial is an exploratory trial involving very limited human exposure, with no therapeutic or diagnostic intent (e.g., screening study, microdose study).

from EDDA:

pre-clinical trial = phase 0 trial = A clinical trial that uses an investigational agent that is available only in very limited quantities and which has never previously given to humans or for which there is extremely limited human experience. Phase 0 clinical trials are intended to enable researchers to understand the path of the drug in the body and its efficacy. Adverse event reporting in Phase 0 trials is expedited. [NCI 2014_12E]

# concept[].concept[].concept[].concept[].concept[].property[].valueString
According to the original FDA guidance, such exploratory IND studies are conducted prior to the traditional dose escalation, safety, and tolerance studies that ordinarily initiate a clinical drug development program.  The duration of dosing in an exploratory IND study is expected to be limited (e.g., 7 days).

A type of clinical trial that involves low dosage and short duration of drug exposure for a limited number of study participants with the intent of gathering preliminary data on the mechanism of action, pharmacodynamics, pharmacokinetics, or bioavailability of promising therapeutic candidate agents in human subjects.

Less official terms (phase 0 trial, pre-clinical trial) have been used to describe a clinical trial that uses an investigational agent that has never previously given to humans or for which there is extremely limited human experience. A Phase 0 study might not include any drug delivery but may be an exploration of human material from a study (e.g., tissue samples or biomarker determinations).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Janice Tufte, Olga Vovk

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-25 vote 8-1 by Harold Lehmann, Alejandro Piscoya, Janice Tufte, Paola Rosati, Robin Ann Yurk, Philippe Rocca-Serra, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-25 comments: I had to read a couple of times and check the hierarchy to appreciate this definition, but I agree.
For a later version of SEVCO, we probably should put citations ("original FDA guidance") into the documentation.
concern over the use of the term 'phase 1' in the definition and the presence of an Alternative term 'phase 0 study`.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-01 vote 5-0 by Paola Rosati, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
A clinical trial with a component meeting the definition of phase 1 trial and a component meeting the definition of phase 2 trial.

# concept[].concept[].concept[].concept[].property[].valueString
from CTO:

phase I/II trial (trial phase 1/2, trial phase 1-2)

Trials that are a combination of phases 1 and 2. A clinical research protocol designed to study the safety, dosage levels and response to new treatment. Phase I/II trials combine a Phase I and a Phase II trial of the same treatment into a single protocol.

A class of clinical study that combines elements characteristic of traditional Phase I and Phase II trials. See also Phase I, Phase II.

A trial to study the safety, dosage levels, and response to a new treatment.

from SCO:
	
phase I/II trial (trial phase 1/2, trial phase 1-2)

A clinical research protocol designed to study the safety, dosage levels and response to new treatment. Phase I/II trials combine a Phase I and a Phase II trial of the same treatment into a single protocol.

from NCIt:

same as CTO

from OCRe:
	
not included

from EDDA:

phase I/II trial (trial phase 1/2, trial phase 1-2)

A class of clinical study that combines elements characteristic of traditional Phase I and Phase II trials. See also Phase I, Phase II. [NCIT_14.08d] [Contributing_Source_CDISC]

A clinical research protocol designed to study the safety, dosage levels and response to new treatment. Phase I/II trials combine a Phase I and a Phase II trial of the same treatment into a single protocol. [NCIT_14.08d]

A trial to study the safety, dosage levels, and response to a new treatment. [NCIT_14.08d]

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-18 vote 3-2 by Harold Lehmann, Paul Harris, Robin Ann Yurk, raradhikaag@gmail.com, Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
A phase 1 trial is a clinical trial to gather initial evidence in humans to support further investigation of an intervention.
A phase 2 trial is a clinical trial to gather evidence of effectiveness and safety for an intervention in patients with the disease or condition under study, but not intended to provide an adequate basis for regulatory approval for clinical use.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-18 comments: Does it matter that the Term has Arabic numerals and the Definition, Roman?
Consider adding a comment for application to improve definition interpretation with individual term definitions for Phase I/Phase 2 trial
Not sure I quite understand what the "separate sets of design parameters with" phrase means here?

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-25 vote 9-0 by Harold Lehmann, Alejandro Piscoya, Janice Tufte, Paola Rosati, Robin Ann Yurk, Philippe Rocca-Serra, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].definition
A clinical trial to gather evidence of effectiveness and safety for an intervention in patients with the disease or condition under study, but not intended to provide an adequate basis for regulatory approval for clinical use.

# concept[].concept[].concept[].concept[].property[].valueString
https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-312/subpart-B/section-312.21 is the US Code of Federal Regulations Title 21 (Food and Drugs) Chapter I Subchapter D Part 312 Subpart B § 312.21 and includes:

§ 312.21 Phases of an investigation.

An IND may be submitted for one or more phases of an investigation. The clinical investigation of a previously untested drug is generally divided into three phases. Although in general the phases are conducted sequentially, they may overlap. These three phases of an investigation are a[sic] follows:

....

Phase 2.  Phase 2 includes the controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in patients with the disease or condition under study and to determine the common short-term side effects and risks associated with the drug. Phase 2 studies are typically well controlled, closely monitored, and conducted in a relatively small number of patients, usually involving no more than several hundred subjects.



from CTO:

Phase II trial
A clinical research protocol designed to study a biomedical or behavioral intervention in a larger group of people (several hundred), to evaluate the drug's effectiveness for a particular indication in patients with the disease or condition under study, and to determine the common short-term side effects and risks associated with the intervention. Includes controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in participants with the disease or condition under study and to determine the common short-term side effects and risks.

	
Phase 2. Controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in patients with the disease or condition under study and to determine the common short-term side effects and risks associated with the drug. NOTE: Phase 2 studies are typically well controlled, closely monitored, and conducted in a relatively small number of patients, usually involving no more than several hundred subjects. [After FDA CDER Handbook, ICH E8] (CDISC glossary)

A study to test whether a new treatment has an anticancer effect (for example, whether it shrinks a tumor or improves blood test results) and whether it works against a certain type of cancer.

Controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in patients with the disease or condition under study and to determine the common short-term side effects and risks associated with the drug. NOTE: Phase 2 studies are typically well controlled, closely monitored, and conducted in a relatively small number of patients, usually involving no more than several hundred subjects. [after FDA CDER handbook, ICH E8]

from SCO:
	
phase II trial not independently defined

from NCIt:

same as CTO

from OCRe:
	
A Phase 2 trial includes controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in patients with the disease or condition under study and to determine the common short-term side effects and risks.

from EDDA:
A clinical research protocol designed to study a biomedical or behavioral intervention in a larger group of people (several hundred), to evaluate the drug's effectiveness for a particular indication in patients with the disease or condition under study, and to determine the common short-term side effects and risks associated with the intervention. [NCI 2014_12E] Studies that are usually controlled to assess the effectiveness and dosage (if appropriate) of diagnostic, therapeutic, or prophylactic drugs, devices, or techniques. These studies are performed on several hundred volunteers, including a limited number of patients with the target disease or disorder, and last about two years. This concept includes phase II studies conducted in both the U.S. and in other countries. [MeSH 2014_2014_02_10]
A clinical research protocol designed to study a biomedical or behavioral intervention in a larger group of people (several hundred), to evaluate the drug's effectiveness for a particular indication in patients with the disease or condition under study, and to determine the common short-term side effects and risks associated with the intervention. [NCI 2014_12E]

Studies that are usually controlled to assess the effectiveness and dosage (if appropriate) of diagnostic, therapeutic, or prophylactic drugs, devices, or techniques. These studies are performed on several hundred volunteers, including a limited number of patients with the target disease or disorder, and last about two years. This concept includes phase II studies conducted in both the U.S. and in other countries. [MeSH 2014_2014_02_10]

from INTERNATIONAL COUNCIL FOR HARMONISATION OF TECHNICAL REQUIREMENTS FOR PHARMACEUTICALS FOR HUMAN USE (ICH HARMONISED GUIDELINE) GENERAL CONSIDERATIONS FOR CLINICAL STUDIES E8(R1) https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2021_1006.pdf Adopted on 6 October 2021

After initial clinical studies provide sufficient information on safety, clinical pharmacology and
dose, exploratory and confirmatory studies (usually referred to as phases 2 and 3, respectively)
are conducted to further evaluate both the safety and efficacy of the drug. 

Exploratory studies are designed to investigate safety and efficacy in a selected population of
patients for whom the drug is intended. Additionally, these studies aim to refine the effective
dose(s) and regimen, refine the definition of the targeted population, provide a more robust
safety profile for the drug, and include evaluation of potential study endpoints for subsequent
studies. Exploratory studies may provide information on the identification and determination
of factors that affect the treatment effect and, possibly combined with modelling and
simulation, serve to support the design of later confirmatory studies.

from March 1998 https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-8-general-considerations-clinical-trials-step-5_en.pdf

3.1.3.2 Phase II (Most typical kind of study: Therapeutic Exploratory)
Phase II is usually considered to start with the initiation of studies in which the primary
objective is to explore therapeutic efficacy in patients.
Initial therapeutic exploratory studies may use a variety of study designs, including
concurrent controls and comparisons with baseline status. Subsequent trials are usually
randomised and concurrently controlled to evaluate the efficacy of the drug and its safety for
a particular therapeutic indication. Studies in Phase II are typically conducted in a group of
patients who are selected by relatively narrow criteria, leading to a relatively homogeneous
population and are closely monitored.
An important goal for this phase is to determine the dose(s) and regimen for Phase III trials.
Early studies in this phase often utilise dose escalation designs (see ICH E4) to give an early
estimate of dose response and later studies may confirm the dose response relationship for the
indication in question by using recognised parallel dose-response designs (could also be
deferred to phase III). Confirmatory dose response studies may be conducted in Phase II or
left for Phase III. Doses used in Phase II are usually but not always less than the highest doses
used in Phase†I.
Additional objectives of clinical trials conducted in Phase II may include evaluation of
potential study endpoints, therapeutic regimens (including concomitant medications) and
target populations (e.g. mild versus severe disease) for further study in Phase II or III. These
objectives may be served by exploratory analyses, examining subsets of data and by including
multiple endpoints in trials.

# concept[].concept[].concept[].concept[].property[].valueString
Phase 2 trials are typically controlled clinical studies conducted to evaluate the effectiveness of the intervention for a particular indication and to determine the common short-term side effects and risks associated with the intervention. Phase 2 trials may have a goal of determining the dose(s) or regimen(s) for Phase 3 trials. Phase 2 studies usually include no more than several hundred subjects.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2021-01-04 vote 5-2 by Robin Ann Yurk, Harold Lehmann, janice tufte, Paola Rosati, C P Ooi, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
2021-01-04 comments: The first part of the definition is ok. In the second part, I would suggest to change with "An insufficient evidence for the intervention tested or the desired patients' number failure could occur thus impeding regulatory approval for clinical use"

Comment Suggestion to add to comment for term from extracted from notes-3.1.3.2:  An important goal for this phase is to determine the dose(s) and regimen for Phase III trials.

Early studies in this phase often utilize dose escalation designs (see ICH E4) to give an early estimate of dose response and later studies may confirm the dose response relationship for the indication in question by using recognized parallel dose-response designs (could also be deferred to phase III)

Minor change - the phrasing is a little awkward, suggest "gather evidence about the effectiveness and safety of an intervention in patients with the disease or condition under study, but not sufficient...". 

I am not sure the comment for application is fully consistent with the definitions (what about safety?).

2022-01-11 comment: I would suggest not adding how many subjects are typically involved, maybe state that these usually have small sample sizes. Unfortunately, sample sizes have decreased over time. https://bmjopen.bmj.com/content/11/12/e053377

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-11 vote 7-0 by Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, janice tufte, Paul Whaley, Andrew Beck, Robin Ann Yurk

# concept[].concept[].concept[].concept[].definition
A clinical trial with a component meeting the definition of phase 2 trial and a component meeting the definition of phase 3 trial.

# concept[].concept[].concept[].concept[].property[].valueString
from CTO:

phase II/III trial (trial phase 2/3, trial phase 2-3)

Trials that are a combination of phases 2 and 3. A type of clinical study that combines elements characteristic of traditional Phase II and Phase III trials.

A trial to study response to a new treatment and the effectiveness of the treatment compared with the standard treatment regimen.

A class of clinical study that combines elements characteristic of traditional Phase II and Phase III trials.

from SCO:
	
not included

from NCIt:

phase II/III trial (trial phase 2/3, trial phase 2-3)

A type of clinical study that combines elements characteristic of traditional Phase II and Phase III trials.

A trial to study response to a new treatment and the effectiveness of the treatment compared with the standard treatment regimen.

A class of clinical study that combines elements characteristic of traditional Phase II and Phase III trials.

from OCRe:
	
not included

from EDDA:

phase II/III trial (trial phase 2/3, trial phase 2-3)

A type of clinical study that combines elements characteristic of traditional Phase II and Phase III trials. [NCIT_14.08d]

A class of clinical study that combines elements characteristic of traditional Phase II and Phase III trials. [NCIT_14.08d] [Contributing_Source_CDISC]

A trial to study response to a new treatment and the effectiveness of the treatment compared with the standard treatment regimen. [NCIT_14.08d]


"Designs that combine phase II and III functions (ie, phase II/III designs) have separate sets of design parameters that correspond to their phase II and III components." -- Korn EL et al. Design Issues in Randomized Phase II/III Trials. J Clin Oncol 2012 https://ascopubs.org/doi/full/10.1200/JCO.2011.38.5732. https://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC3295562&blobtype=pdf

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-18 vote 2-3 by Harold Lehmann, Paul Harris, Robin Ann Yurk, raradhikaag@gmail.com, Paul Whaley
2022-01-25 vote 9-1 by Harold Lehmann, Alejandro Piscoya, Janice Tufte, Paola Rosati, Robin Ann Yurk, Philippe Rocca-Serra, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde, Paul Whaley
2022-02-01 vote 4-1 by Paola Rosati, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-18 comments: Phase 2/3 trials determine efficacy of a new biomedical intervention i.e. whether it works as intended in a larger group of study participants, and monitor adverse effects so that the intervention may be used safely.
Consider adding a comment for application to improve definition interpretation with individual term definitions for Phase 2/Phase 3 trial
Not sure I quite understand what the "separate sets of design parameters with" phrase means here?
2022-01-25 comment: As already pointed out, to me these definitions seem incongruent and lack specification of the outcomes used, namely core clinical outcomes relevant for patients. Are pahse 2 and phase 3 trials designed to gather evidence of 'effectiveness' and safety or 'efficacy' and monitor adverse effects of a new biomedical intervention? For what outcome? The three sentences proposed in the comment for application of this code seem overlapping the two terms (i.e. is it still efficacy the right term used for trials or is it effectiveness, commonly used for prospective observational studies?). I think it is important to justify why the two terms are used for clinical trial designs.
2022-02-01 comment: To me this definition has no clear meaning. As your are working and struggling so hard to define and clarify the scientific evidence code system, I wish to participate to the meeting to discuss with you this tricky definition. If you agree, please, let me know.

# concept[].concept[].concept[].concept[].property[].valueString
A phase 2 trial is a clinical trial to gather evidence of effectiveness and safety for an intervention in patients with the disease or condition under study, but not intended to provide an adequate basis for regulatory approval for clinical use.
A phase 3 trial is a clinical trial to gather the evidence of effectiveness and safety of an intervention, intended to provide an adequate basis for regulatory approval for clinical use.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-08 vote 7-0 by Paola Rosati, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper, Paul Whaley, Sunu Alice Cherian

# concept[].concept[].concept[].concept[].definition
A clinical trial to gather the evidence of effectiveness and safety of an intervention, intended to provide an adequate basis for regulatory approval for clinical use.

# concept[].concept[].concept[].concept[].property[].valueString
https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-312/subpart-B/section-312.21 is the US Code of Federal Regulations Title 21 (Food and Drugs) Chapter I Subchapter D Part 312 Subpart B § 312.21 and includes:

§ 312.21 Phases of an investigation.

An IND may be submitted for one or more phases of an investigation. The clinical investigation of a previously untested drug is generally divided into three phases. Although in general the phases are conducted sequentially, they may overlap. These three phases of an investigation are a[sic] follows:

....

Phase 3.  Phase 3 studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence suggesting effectiveness of the drug has been obtained, and are intended to gather the additional information about effectiveness and safety that is needed to evaluate the overall benefit-risk relationship of the drug and to provide an adequate basis for physician labeling. Phase 3 studies usually include from several hundred to several thousand subjects.



from CTO:

Phase III trial
Includes trials conducted after preliminary evidence suggesting effectiveness of the drug has been obtained, and are intended to gather additional information to evaluate the overall benefit-risk relationship of the drug. A clinical research protocol designed to investigate the efficacy of the biomedical or behavioral intervention in large groups of human subjects (from several hundred to several thousand), to confirm efficacy, to monitor adverse reactions to the new medication or treatment regimen with respect to long-term use and by comparing the intervention to other standard or experimental interventions as well as to a placebo.

	
Phase 3. Studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence suggesting effectiveness of the drug has been obtained, and are intended to gather the additional information about effectiveness and safety that is needed to confirm efficacy and evaluate the overall benefit-risk relationship of the drug and to provide an adequate basis for physician labeling. NOTE: Phase 3 studies usually include from several hundred to several thousand subjects. [After FDA CDER Handbook, ICH E8] (CDISC glossary)

A study to compare the results of people taking a new treatment with the results of people taking the standard treatment (for example, which group has better survival rates or fewer side effects). In most cases, studies move into phase III only after a treatment seems to work in phases I and II. Phase III trials may include hundreds of people.

Studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence suggesting effectiveness of the drug has been obtained and are intended to gather the additional information about effectiveness and safety that is needed to confirm efficacy and evaluate the overall benefit-risk relationship of the drug and to provide an adequate basis for physician labeling. NOTE: Phase 3 studies usually include from several hundred to several thousand subjects. [after FDA CDER handbook, ICH E8]

from SCO:
	
A clinical research protocol designed to investigate the efficacy of the biomedical or behavioral intervention in large groups of human subjects (from several hundred to several thousand), to confirm efficacy, to monitor adverse reactions to the new medication or treatment regimen with respect to long-term use and by comparing the intervention to other standard or experimental interventions as well as to a placebo.

from NCIt:
Phase III trial (Phase III Clinical Trial; Phase III Trial; phase 3; Trial Phase 3; PHASE III TRIAL; phase III trial; Phase III Trials; 3; Phase 3 Study; Clinical Trials, Phase III; Phase III Study; Phase III Protocol)
A clinical research protocol designed to investigate the efficacy of the biomedical or behavioral intervention in large groups of human subjects (from several hundred to several thousand), to confirm efficacy, to monitor adverse reactions to the new medication or treatment regimen with respect to long-term use and by comparing the intervention to other standard or experimental interventions as well as to a placebo.

from OCRe:
A Phase 3 trial includes expanded controlled and uncontrolled trials after preliminary evidence suggesting effectiveness of the drug has been obtained, and are intended to gather additional information to evaluate the overall benefit-risk relationship of the drug and provide an adequate basis for physician labeling.

from EDDA:
Comparative studies to verify the effectiveness of diagnostic, therapeutic, or prophylactic drugs, devices, or techniques determined in phase II studies. During these trials, patients are monitored closely by physicians to identify any adverse reactions from long-term use. These studies are performed on groups of patients large enough to identify clinically significant responses and usually last about three years. This concept includes phase III studies conducted in both the U.S. and in other countries. [MeSH 2014_2014_02_10] 
A clinical research protocol designed to investigate the efficacy of the biomedical or behavioral intervention in large groups of human subjects (from several hundred to several thousand), to confirm efficacy, to monitor adverse reactions to the new medication or treatment regimen with respect to long-term use and by comparing the intervention to other standard or experimental interventions as well as to a placebo. [NCI 2014_12E]

from INTERNATIONAL COUNCIL FOR HARMONISATION OF TECHNICAL REQUIREMENTS FOR PHARMACEUTICALS FOR HUMAN USE (ICH HARMONISED GUIDELINE) GENERAL CONSIDERATIONS FOR CLINICAL STUDIES E8(R1) https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2021_1006.pdf Adopted on 6 October 2021

After initial clinical studies provide sufficient information on safety, clinical pharmacology and
dose, exploratory and confirmatory studies (usually referred to as phases 2 and 3, respectively)
are conducted to further evaluate both the safety and efficacy of the drug. 

Confirmatory studies are designed to confirm the preliminary evidence accumulated in earlier clinical studies that a drug is safe and effective for use for the intended indication and recipient population. These studies are often intended to provide an adequate basis for marketing approval, and to support adequate instructions for use of the drug and official product information. They aim to evaluate the drug in participants with or at risk of the condition or disease who represent those who will receive the drug once approved. This may include investigating subgroups of patients with frequently occurring or potentially relevant comorbidities  (e.g., cardiovascular disease, diabetes, hepatic and renal impairment) to characterise the safe and effective use of the drug in patients with these conditions. 

Confirmatory studies may evaluate the efficacy and safety of more than one dose or the use of the drug in different stages of disease or in combination with one or more other drugs. If the intent is to administer a drug for a long period of time, then studies involving extended exposure to the drug should be conducted (ICH E1 Clinical Safety for Drugs used in Long-Term Treatment). Irrespective of the intended duration of administration, the duration of effect of the drug will also inform the duration of follow-up.

Study endpoints selected for confirmatory studies should be clinically relevant and reflect disease burden or be of adequate surrogacy for predicting disease burden or sequelae. 

from March 1998 https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-8-general-considerations-clinical-trials-step-5_en.pdf

3.1.3.3 Phase III (Most typical kind of study: Therapeutic Confirmatory)
Phase III usually is considered to begin with the initiation of studies in which the primary objective is to demonstrate, or confirm therapeutic benefit.
Studies in Phase III are designed to confirm the preliminary evidence accumulated in Phase II that a drug is safe and effective for use in the intended indication and recipient population. These studies are intended to provide an adequate basis for marketing approval. Studies in Phase III may also further explore the dose-response relationship, or explore the drug's use in wider populations, in different stages of disease, or in combination with another drug. For drugs intended to be administered for long periods, trials involving extended exposure to the drug are ordinarily conducted in Phase III, although they may be started in Phase II (see ICH E1). ICH E1 and ICH E7 describe the overall clinical safety database considerations for chronically administered drugs and drugs used in the elderly. These studies carried out in Phase III complete the information needed to support adequate instructions for use of the drug (official product information).

# concept[].concept[].concept[].concept[].property[].valueString
Phase 3 trials are typically conducted after preliminary evidence suggests effectiveness and usually have the primary objective to demonstrate or confirm therapeutic benefit compared to placebo or a standard treatment. Phase 3 studies usually include from several hundred to several thousand subjects. Study endpoints for phase 3 trials should be clinically relevant or of adequate surrogacy for predicting clinical effects.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Joanne Dehnbostel, Janice Tufte, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-21 vote 2-2 by Robin Ann Yurk, C P Ooi, Janice Tufte, Paul Whaley
2022-01-04 vote 5-2 by Robin Ann Yurk, Harold Lehmann, Janice Tufte, Paola Rosati, C P Ooi, Joanne Dehnbostel, Paul Whaley
2022-01-11 vote 6-1 by Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, janice tufte, Paul Whaley, Andrew Beck, Robin Ann Yurk

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-21 comments: Note: consider adding the following comments from comments from previous reviewers to improve interpretation.

3.1.3.3 “Phase III usually is considered to begin with the initiation of studies in which the primary objective is to demonstrate, or confirm therapeutic benefit.”

EDDA:  “Comparative studies to verify the effectiveness of diagnostic, therapeutic, or prophylactic drugs, devices, or techniques determined in phase II studies…. A clinical research protocol designed to investigate the efficacy of the biomedical or behavioral intervention in large groups of human subjects (from several hundred to several thousand), to confirm efficacy, to monitor adverse reactions to the new medication or treatment regimen with respect to long-term use and by comparing the intervention to other standard or experimental interventions as well as to a placebo”
..................
I think the pieces are there but the phrasing is difficult to parse.


2022-01-04 comments: Perhaps adding "compared with a standard treatment" may improve the clarity. 

Minor change - the phrasing is a little awkward, suggest "gather evidence about the effectiveness and safety of an intervention that is needed...". 

2022-01-11 comments: I would suggest not adding how many subjects are typically involved. Unfortunately, sample sizes have decreased over time. https://bmjopen.bmj.com/content/11/12/e053377

Minor change for consistency with other trial definitions: "A clinical trial to gather evidence of effectiveness and safety of an intervention, that is intended to provide an adequate basis for regulatory approval for clinical use."

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-18 vote 6-0 by Harold Lehmann, Paul Harris, Robin Ann Yurk, Paola Rosati, raradhikaag@gmail.com, Paul Whaley

# concept[].concept[].concept[].concept[].definition
A clinical trial to gather additional evidence of effectiveness and safety of an intervention for an already approved clinical use.

# concept[].concept[].concept[].concept[].property[].valueString
from CTO:

Phase IV Trial (Phase IV Study, Phase IV clinical trial, phase 4 study, phase 4 trial, trial phase 4)

Studies of FDA-approved drugs to delineate additional information including the drug's risks, benefits, and optimal use. A randomized, controlled trial that is designed to evaluate the long-term safety and efficacy of a drug for a given indication. Often they are designed to study side effects that may have become apparent after the phase III study was completed.

After a treatment has been approved and is being marketed, it is studied in a phase IV trial to evaluate side effects that were not apparent in the phase III trial. Thousands of people are involved in a phase IV trial.

Post approval studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval. NOTE: These studies could include, but would not be limited to, studying different doses or schedules of administration than were used in Phase 2 studies, use of the drug in other patient populations or other stages of the disease, or use of the drug over a longer period of time. [after FDA CDER handbook, ICH E8]

Phase 4. Postmarketing (Phase 4) studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval. NOTE: These studies could include, but would not be limited to, studying different doses or schedules of administration than were used in Phase 2 studies, use of the drug in other patient populations or other stages of the disease, or use of the drug over a longer period of time. [After FDA CDER Handbook, ICH E8] (CDISC glossary)

from SCO:
	
not included

from NCIt:

Phase IV Trial (Phase IV Study, Phase IV clinical trial, phase 4 study, phase 4 trial, trial phase 4)

A randomized, controlled trial that is designed to evaluate the long-term safety and efficacy of a drug for a given indication. Often they are designed to study side effects that may have become apparent after the phase III study was completed.

After a treatment has been approved and is being marketed, it is studied in a phase IV trial to evaluate side effects that were not apparent in the phase III trial. Thousands of people are involved in a phase IV trial.

Post approval studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval. NOTE: These studies could include, but would not be limited to, studying different doses or schedules of administration than were used in Phase 2 studies, use of the drug in other patient populations or other stages of the disease, or use of the drug over a longer period of time. [after FDA CDER handbook, ICH E8]

Phase 4. Postmarketing (Phase 4) studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval. NOTE: These studies could include, but would not be limited to, studying different doses or schedules of administration than were used in Phase 2 studies, use of the drug in other patient populations or other stages of the disease, or use of the drug over a longer period of time. [After FDA CDER Handbook, ICH E8] (CDISC glossary)

from OCRe:
	
A Phase 4 study monitors FDA-approved drug to delineate additional information including the drug's risks, benefits, and optimal use.

from EDDA:

A randomized, controlled trial that is designed to evaluate the long-term safety and efficacy of a drug for a given indication. Often they are designed to study side effects that may have become apparent after the phase III study was completed. [NCIT_14.08d]

After a treatment has been approved and is being marketed, it is studied in a phase IV trial to evaluate side effects that were not apparent in the phase III trial. Thousands of people are involved in a phase IV trial. [NCIT_14.08d]

Phase 4. Postmarketing (Phase 4) studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval. NOTE: These studies could include, but would not be limited to, studying different doses or schedules of administration than were used in Phase 2 studies, use of the drug in other patient populations or other stages of the disease, or use of the drug over a longer period of time. [After FDA CDER Handbook, ICH E8] [Contributing Source_CDISC] [NCIT_14.08d]

# concept[].concept[].concept[].concept[].property[].valueString
Post-marketing studies (phase IV trials) are often used to evaluate adverse effects that were not apparent in phase III trials, and may involve thousands of patients. Postmarketing (Phase 4) studies to delineate additional information about the drug's risks, benefits, and optimal use that may be requested by regulatory authorities in conjunction with marketing approval.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-15 vote 10-0 by Paul Whaley, Andrew Beck, Paola Rosati, Robin Ann Yurk, Janice Tufte, Jesus Lopez-Alcalde, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-15 comment: Maybe add hyphen between "already" and "approved"

# concept[].concept[].definition
A study design in which the independent variables (exposures or interventions) are not prospectively assigned or modified by the investigator.

# concept[].concept[].property[].valueString
Mario Tristan, Joanne Dehnbostel, Harold Lehmann, Khalid Shahin, Brian S. Alper

# concept[].concept[].property[].valueString
8/8 as of 6/7/2021: Asiyah Lin, KM Saif-Ur-Rahman, Harold Lehmann, Sebastien Bailly, Bhagvan Kommadi, Mario Tristan, Leo Orozco, Ahmad Sofi-Mahmudi

# concept[].concept[].property[].valueString
2021-05-17 vote 5-3 on "Observational research = In a prospective or retrospective study, an independent variable is measured but not manipulated by the investigator to evaluate a response or outcome (the dependent variable)." by Eric Harvey, Bhagvan Kommadi, Paola Rosati, KM Saif-Ur-Rahman, Ahmad Sofi-Mahmudi, Jesus Lopez-Alcalde, Sorana D. Bolboaca, Harold Lehmann, 2021-05-24 vote 8-3 on Observational research="A study design in which the variables (exposures, interventions, and outcomes) are not prospectively assigned or modified by the investigator." by Alejandro Piscoya, Philippe Rocca-Serra, KM Saif-Ur-Rahman, Eric Harvey, Harold Lehmann, Bhagvan Kommadi, Sorana D. Bolboaca, Jesús López-Alcalde, Paola Rosati, Tatyana Shamliyan, Brian Alper, , 2021-05-31 vote 11-1 on Observational research="A study design in which the independent variables (exposures or interventions) are not prospectively assigned or modified by the investigator." by Eric Harvey, Bhagvan Kommadi, Brian Alper, Sebastien Bailly, Alejandro Piscoya, Harold Lehmann, KM Saif-Ur-Rahman, Paola Rosati, Sorana D. Bolboaca, Asiyah Lin, Leo Orozco, Erfan Shamsoddin

# concept[].concept[].property[].valueString
I dislike the term "manipulated" in the definition -- suggest change to:  In a prospective or retrospective study, without any specific intervention assigned to participants, an investigator observes and measures an intervention or procedure (the independent variable) to assess or learn more about an effect or outcome (the dependent variable). "In a prospective or retrospective study, an independent variable (a predictor) is obeserved or measured by the investigator to evaluate a response or an outcome (the dependent variable)." I would delete "in a prospective or retrospective study" as it could be ambispective    5-24-2021 similar comment about the synonyms assigned the class (conflating plan/design) with the object realised by executing a plan I think that the outcomes are never assigned or modified by the investigator (they are measured). Thus, to be consistent with the definition of interve…suggest to remove "outcomes" from ( )   (is there a semantic difference between "are not" and "none is"?) I suggest to clarify the goal as drawing causal inferences from the observed association between exposure and outcomes  5-31-2021 comment The suggested definition is a non-interventional study definition. Not sure if a non-interventional is fully equivalent to observational studies

# concept[].concept[].property[].valueString
We acknowledge that observational study design and observational study may not be exact synonyms of observational research, but observational research could be used to encompass both design and implementation of the design. In the context of coding study design factors, observational research is commonly used to denote non-interventional research.

# concept[].concept[].concept[].definition
An observational study to identify adverse events related to the use of an approved clinical intervention.

# concept[].concept[].concept[].property[].valueString
from CTO:

not included

from SCO:
	
not included

from NCIt:

Postmarketing Surveillance

Programs to identify adverse events that did not appear during the drug approval process.

Ongoing safety monitoring of marketed drugs. See also Phase 4 studies, Phase 5 studies.

also Phase V Trial (phase 5, trial phase 5)

Postmarketing surveillance is sometimes referred to as Phase V. See outcomes research.

from OCRe:
	
not included

from EDDA:

postmarketing evaluation study (post-marketing product surveillance)

Surveillance of drugs, devices, appliances, etc., for efficacy or adverse effects, after they have been released for general sale. [MeSH 2014_2014_02_10]

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-02-15 vote 10-0 by Paul Whaley, Andrew Beck, Brian S. Alper, Paola Rosati, Janice Tufte, Jesus Lopez-Alcalde, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2022-02-15 comments: Alternative terms could be: Post-marketing evaluation study, 
(Do we need to connect the "approval" to an indication?

# concept[].concept[].definition
A study design in which two or more groups are compared.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Ellen Jepson, Kenneth Wilkins, Mario Tristan

# concept[].concept[].property[].valueString
9/9 as of 8/9/2021: Erfan Shamsoddin, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Joanne Dehnbostel, Alejandro Piscoya

# concept[].concept[].concept[].definition
A comparative study design in which the groups are compared concurrently and participants are expected to remain in the groups being compared for the entire duration of participation in the study.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Ellen Jepson, Kenneth Wilkins, Mario Tristan, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
9/9 as of 8/9/2021: Erfan Shamsoddin, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Joanne Dehnbostel, Alejandro Piscoya

# concept[].concept[].concept[].definition
A comparative study design in which participants receive two or more alternative exposures during separate periods of time.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].property[].valueString
8/9 as of 8/9/2021: voting on "A comparative study design in which participants receive two or more alternative exposures during separate periods of time." by Erfan Shamsoddin, Paola Rosati, Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Joanne Dehnbostel, Alejandro Piscoya

# concept[].concept[].concept[].property[].valueString
8/9/21 comment: It's not clear from this definition that each group of participants receives the same 2 or more exposures, but not in the same time sequence

# concept[].concept[].concept[].concept[].definition
A crossover cohort design in which two or more cohorts have different orders of exposures.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].concept[].definition
A crossover cohort design in which all participants are in a single cohort with the same order of exposures.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].definition
A comparative study design in which the groups being compared are defined by outcome presence (case) or absence (control).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].definition
A comparative study design in which individual participants in different groups being compared are paired or matched into sets based on selected attributes for within-set analysis.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].concept[].definition
A matched study design in which related or non-related family members are compared.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].definition
A family study design in which twin siblings are compared.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].concept[].definition
A comparative study design in which participants are allocated to exposures (interventions) by their membership in groups (called clusters) rather than by individualized assignments.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].definition
A study design with no comparisons between groups with different exposures and no comparisons between groups with different outcomes.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Michael Panzer

# concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].definition
A non-comparative study design in which two or more participants are evaluated in a single group (or cohort).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].concept[].definition
A non-comparative study design in which a single participant is evaluated.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Michael Panzer

# concept[].concept[].concept[].property[].valueString
7/7 as of 8/23/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Leo Orozco, Alejandro Piscoya

# concept[].concept[].definition
A study design in which the unit of observation is a population or community.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Harold Lehmann

# concept[].concept[].property[].valueString
The term ‘population-based study’ is generally used for an observational comparative study design in which populations are compared.

# concept[].concept[].property[].valueString
5/5 as of 10/18/2021: Cheow Peng Ooi, Janice Tufte, Robin Ann Yurk, Eric Harvey, Joanne Dehnbostel

# concept[].concept[].concept[].definition
A study design in which the unit of observation is a population or community defined by social relationships or physical surroundings.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
5/5 as of 10/18/2021: Cheow Peng Ooi, Janice Tufte, Robin Ann Yurk, Eric Harvey, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
6 to 1 in 2021-09-20 vote with 7 participants (Ecological design = A comparative study design in which populations are compared. An ecologic study is a non individual-human study in which the unit of observation is a population or community.) - Robin Ann Yurk, Janice Tufte, Eric Harvey, Jesus Lopez-Alcalde, Mario Tristan, Sorana D Bolboaca, Paola Rosati, 8 to 1 vote on 2021-09-27 with 9 participants (Ecological design [Population-based design, Ecologic study, Population study] = A comparative study design in which populations are compared. An ecologic study is a non-individual study in which the unit of observation is a population or community.) - Jesus Lopez-Alcalde, Asiyah Lin, Eric Harvey, Bhagvan Kommadi, Alejandro Piscoya, Robin Ann Yurk, Mario Tristan, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2021-09-20 comment:  I miss here the explicit declaration that ecological studies are observational. A cluster trial can randomise communities and is not an ecological study. Besides, and I may be worng, but an ecological study may include non-humans, for example, ecological study of air contamination levels in Spain compared to Italy. 2021-09-27 comment: The differences of ecologic studies and other population based studies are not reflected. consider adding "Variables in an ecologic analysis may be aggregate measures, environmental measures, or global measures."

# concept[].definition
A specification of a sequence of actions for a component or part of a study design.

# concept[].property[].valueString
Study design is defined as a plan specification for how and what kinds of data will be gathered as part of an investigation which may produce testable explanations, conclusions and predictions or test a hypothesis.

# concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel

# concept[].property[].valueString
2022-03-22 vote 5-0 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Robin Ann Yurk

# concept[].concept[].definition
A study design process in which data is collected at a single point in time.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].property[].valueString
The word "feature" was added to the definition on March 7, 2022 to match the change in hierarchical terms.

# concept[].concept[].definition
A study design process in which data is collected at two or more points in time.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].property[].valueString
The word "feature" was added to the definition on March 7, 2022 to match the change in hierarchical terms.

# concept[].concept[].concept[].definition
A longitudinal data collection which includes a set of time-ordered observations.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].concept[].concept[].definition
A time series design which includes comparisons of observations before and after an event or exposure.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
8/8 as of 9/12/2021: Eric Harvey, Bhagvan Kommadi, Brian S. Alper, Cheow Peng Ooi, Janice Tufte, Paola Rosati, Robin Ann Yurk, MD, MPH, Mario Tristan

# concept[].concept[].definition
A study design process in which the data are recorded and collected during the study for the purpose of the same study.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Paul Whaley, Mario Tristan

# concept[].concept[].property[].valueString
2022-02-22 vote 7-1 by Paola Rosati, Sunu Alice Cherian, Joanne Dehnbostel, Sumalatha A, Rebecca Baker, Robin Ann Yurk, Janice Tufte, Harold Lehmann 
2022-03-01 vote 3-3 by Joanne Dehnbostel, Robin Ann Yurk, Paul Whaley, Nisha Mathew, Paola Rosati, Sunu Alice Cherian
2022-03-15 vote 5-1 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 5-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Robin Ann Yurk, nelle.stocquart

# concept[].concept[].property[].valueString
2022-02-22 comments: Definition: Data recorded and collected during the study.
For parallelism with "secondary data collection," perhaps write, "for the purpose of the current study."
2022-03-01 comments:
A data collection technique in which the data are collected and recorded during the study for the purpose of the same study.
For the term definition---I would edit so it reads...A study design in which the data are collected and recorded to answer a new research question.
Data collection is not study design, it can called as a technique A data collection technique in which data is recorded and collected during the study for the purpose of the same study.
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
"in which" sounds strange for a "feature." ("Color is a feature in which..." does not sound right.) Perhaps a...feature regarding how data are recorded..."?
2022-03-22 comment: Suggest modify definition or create a comment for application so it reads:  A Study design method in which the data are collected for original research to answer new research questions.

# concept[].concept[].property[].valueString
The study design process includes the source and method for data collection. When the data are collected for original research to answer the original research questions, this is called primary data collection.

# concept[].concept[].property[].valueString
2022-03-29 vote 6-0 by Paul Whaley, Robin Ann Yurk, Mario Tristan, Jesus Lopez-Alcalde, Brian S. Alper, Cauê Monaco

# concept[].concept[].definition
A study design process in which the study data are obtained from a source of data collected during a routine process in the natural environment rather than using a process designed or controlled by the researcher.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Paul Whaley, Mario Tristan

# concept[].concept[].property[].valueString
2022-02-22 vote 7-1 by Paola Rosati, Sunu Alice Cherian, Joanne Dehnbostel, Sumalatha A, Rebecca Baker, Robin Ann Yurk, Janice Tufte, Harold Lehmann
2022-03-15 vote 5-1 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-2 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Robin Ann Yurk, nelle.stocquart
2022-03-29 vote 4-1 by Paul Whaley, Robin Ann Yurk, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco
2022-04-05 vote 6-0 by Cauê Monaco, Harold Lehmann, Mario Tristan, Robin Ann Yurk, Jesus Lopez-Alcalde, Nisha Mathew
THEN THE TERM CHANGED to Real World Data Collection
2022-04-19 vote 3-1 by Cauê Monaco, Robin Ann Yurk, Jesus Lopez-Alcalde, Harold Lehmann

# concept[].concept[].property[].valueString
2022-02-22 comment: Definition: Data gathered from studies, surveys, experiments that have been done by other people for other studies
2022-03-15 comment: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
2022-03-22 comments: The proposed definition only details the source of the data but not the data collection itself. I propose: "A study design process in which the data are collected from data collected for a purpose other than the current study".
Suggest modify definition or create a comment for application so it reads.  A study design method in which the previously collected data is used to answer new and additional research questions.  Some example of the types of studies are retrospective study etc..
2022-03-29 comment: In the comment for application--suggestion.  Delete phrase   When data are collected.  I would combined sentence When data are used in the form of analysis and interpretation from original research to answer additional research questions separate from the original research.  

2022-04-12 comments: For Term definition:  Suggest revising definition to A study design process in which the study data are obtained from data collected for recording data for business purposes.  
Comment for application:  Add this statement, There are different categories of research such as business research, marketing research, insurance research etc.
"data are obtained from data collected" may be changed to "data are obtained from a source for data collection"

2022-04-19 comment: Suggest edit the term definition.  The Alternative term and comment for application are fine.  There are different kinds of research business research that can be classified as real world data.   The term definition should read....A study design in which the study data processes are obtained from a natural environment rather than controlled research.

# concept[].concept[].property[].valueString
Real world data collection occurs when the study uses data obtained from a source that was not created for research as a primary purpose.  A study can involve both primary data collection (with some data collected by a process created for the purpose of the study investigation) and real world data collection (with some data collected from a process created for a routine business or operational purpose). If a study involves both primary data collection and real world data collection, both terms can be applied.

# concept[].concept[].property[].valueString
2022-05-06 vote 7-0 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte

# concept[].concept[].concept[].display
real world data collection from healthcare records

# concept[].concept[].concept[].definition
Real world data collection from data obtained routinely for a purpose of recording healthcare delivery in a record controlled by a healthcare professional.

# concept[].concept[].concept[].designation[].value
RWD collection from clinical care records

# concept[].concept[].concept[].designation[].value
real world data collection from medical records

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Ilkka Kunnamo

# concept[].concept[].concept[].property[].valueString
2022-03-15 vote 4-2 by Mario Tristan, Paul Whaley, Nisha Mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart
2022-03-29 vote 3-1 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco
2022-04-05 vote 6-1 by Cauê Monaco, Paola Rosati, Harold Lehmann, Mario Tristan, Robin Ann Yurk, Jesus Lopez-Alcalde, Nisha Mathew

# concept[].concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
would proposed, "...for the purpose..." (as in primary data collection)
Seems like we should add that the original data is then used for a secondary research purpose in the definition, not only explain in Alternative terms
2022-03-22 comment: ídem: Proposal: "A study design process in which the data are collected from data collected for a purpose of recording healthcare delivery in a record controlled by a healthcare professional."
2022-03-29 comment: "medical records" and "health records" seem to be much more widely used expressions than "healthcare delivery records"
2022-04-05 comments: Suggest make a comment or distinction in the term definition that the primary data collected is categorized as real world data for the purpose of delivering professional healthcare services.  The data set can be used for secondary data collection.

# concept[].concept[].concept[].property[].valueString
This term is used when the original data collection (primary data collection) is done for the purpose of delivering professional healthcare services.  The secondary use of this data (sometimes called 'real world data') for research is then called secondary data collection.

# concept[].concept[].concept[].property[].valueString
2022-04-19 vote 5-0 by Cauê Monaco, Jesus Lopez-Alcalde, Harold Lehmann, Robin Ann Yurk, Muhammad Afzal

# concept[].concept[].concept[].display
real world data collection from personal health records

# concept[].concept[].concept[].definition
Real world data collection from data obtained routinely for a purpose of recording data related to personal health in a record controlled by the person, guardian, or caretaker.

# concept[].concept[].concept[].designation[].value
real world data collection from personal care records

# concept[].concept[].concept[].designation[].value
RWD collection from personal medical records

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Ilkka Kunnamo

# concept[].concept[].concept[].property[].valueString
2022-03-15 vote 4-2 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart
2022-03-29 vote 5-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Brian S. Alper
THEN TERM CHANGED 2022-04-05

# concept[].concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
Might suggest "the purpose," again
add in the definition that the original ddata is then used for a seconday purpose
2022-03-22 comment: dem: Proposal: "A study design process in which the data are collected from data collected for a purpose of recording data related to personal health in a record controlled by the person, guardian, or caretaker."

# concept[].concept[].concept[].property[].valueString
2022-04-19 vote 5-0 by Cauê Monaco, Jesus Lopez-Alcalde, Harold Lehmann, Brian S. Alper, Muhammad Afzal

# concept[].concept[].concept[].display
real world data collection from healthcare financing records

# concept[].concept[].concept[].definition
Real world data collection from data obtained routinely for a purpose of recording healthcare financing.

# concept[].concept[].concept[].designation[].value
RWD collection from healthcare financing records

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-03-15 vote 4-2 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart
2022-03-29 vote 5-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Brian S. Alper
THEN TERM CHANGED 2022-04-05

# concept[].concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
"the purpose"
add original financial data is then used for secondary analysis etc
2022-03-22 comment: Ídem. "A study design process in which the data are collected from data collected for a purpose of recording healthcare financing"

# concept[].concept[].concept[].property[].valueString
2022-04-19 vote 5-0 by Cauê Monaco, Jesus Lopez-Alcalde, Harold Lehmann, Brian S. Alper, Muhammad Afzal

# concept[].concept[].concept[].display
real world data collection from testing procedures

# concept[].concept[].concept[].definition
Real world data collection from data obtained routinely for a purpose of testing, such as diagnostic testing or screening examination.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-03-15 vote 4-2 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart
2022-03-29 vote 5-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Brian S. Alper
THEN TERM CHANGED 2022-04-05

# concept[].concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
"the purpose"
and  then used for secondary research purposes
2022-03-22 comment: Ídem. "A study design process in which the data are collected from data collected for a purpose of testing, such as diagnostic testing or screening examination"

# concept[].concept[].concept[].property[].valueString
2022-04-19 vote 5-0 by Cauê Monaco, Jesus Lopez-Alcalde, Harold Lehmann, Brian S. Alper, Muhammad Afzal

# concept[].concept[].concept[].concept[].display
real world data collection from monitoring procedures

# concept[].concept[].concept[].concept[].definition
Real world data collection from data obtained routinely for a purpose of repeated testing.

# concept[].concept[].concept[].concept[].designation[].value
RWD collection from monitoring procedures

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-03-15 vote 5-1 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart
2022-03-29 vote 5-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Brian S. Alper
THEN TERM CHANGED 2022-04-05

# concept[].concept[].concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
"the purpose"
2022-03-22 comment: Ídem. "A study design process in which the data are collected from data collected for a purpose of repeated testing."

# concept[].concept[].concept[].concept[].property[].valueString
2022-04-19 vote 5-0 by Cauê Monaco, Jesus Lopez-Alcalde, Harold Lehmann, Brian S. Alper, Muhammad Afzal

# concept[].concept[].display
secondary data collection from prior research

# concept[].concept[].definition
A study design process in which the data are collected from data obtained during a different study than the current study.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan

# concept[].concept[].property[].valueString
2022-03-15 vote 5-1 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart

# concept[].concept[].property[].valueString
2022-03-15 comments: ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
2022-03-22 comments: Ídem. "A study design process in which the data are collected from data collected during a different study than the current study"
When does this recording happen?

# concept[].concept[].property[].valueString
2022-03-29 vote 5-0 by Mario Tristan, Paul Whaley, Cauê Monaco, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].display
secondary data collection from a registry

# concept[].concept[].definition
A study design process in which the data are collected from a system organized to obtain and maintain uniform data for discovery and analysis, and this system is organized prior to the current study.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Ilkka Kunnamo

# concept[].concept[].property[].valueString
"For the purposes of this guide, a patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes" -- in https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/registries-guide-3rd-edition_research.pdf

# concept[].concept[].property[].valueString
2022-03-15 vote 3-2 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann
2022-03-22 vote 4-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart

# concept[].concept[].property[].valueString
2022-03-15 comments: the term discovery is not suitable.  Can we have some other term?
ok, if 'study design feature' is understood as 'part specified by study design plan/protocol', but sounds like a process
2022-03-22 comments: Ídem. "A study design process in which the data are collected from data collected in a system organized to obtain and maintain uniform data for discovery and analysis"
The definition needs to be more, When did this happen? Before the study starts?

# concept[].concept[].property[].valueString
2022-03-29 vote 5-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Brian S. Alper

# concept[].concept[].definition
A study design process in which data are collected from two or more geographic locations.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Mario Tristan

# concept[].concept[].property[].valueString
2022-05-06 vote 6-1 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte

# concept[].concept[].property[].valueString
2022-04-26 comment: As stated, this term has too much overlap with "Multicentric" Why do we need this term?

# concept[].concept[].property[].valueString
For studies conducted across multiple contexts (administrative or logistical) that are distinct from geographic locations, potentially introducing greater variability beyond multisite data collection, use the term Multicentric.

# concept[].concept[].property[].valueString
2022-05-10 vote 7-0 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte

# concept[].concept[].definition
A study design process in which data are analyzed with mathematical or statistical methods and formulas.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].property[].valueString
2022-05-17 vote 8-1 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde
2022-05-24 vote 5-1 by Robin Ann Yurk, nelle.stocquart@kce.fgov.be, Eric M Harvey, Mario Tristan, Harold Lehmann, Jesus Lopez-Alcalde

# concept[].concept[].property[].valueString
2022-05-17 comments: Suggest including examples of quantitative analysis so as to improve your definition as there are many categories of quantitative methods:  ie survey methods, logistic regression,...etc
Quantitative and qualitative have categorical results I believe
2022-05-24 comment: An analytic approach using statistical methods and formulas to report the data for interpretation
2022-05-26 comment: I would leave the description of a qualitative analysis out of the comment for application

# concept[].concept[].property[].valueString
The distinction of quantitative vs. qualitative analysis refers to whether mathematical processing is involved, whether or not the analysis includes numerical variables. Processing a categorical variable (e.g. values of happy, sad, or jealous as a response to "How are you feeling?") to produce numerical results (e.g. 30% happy, 50% sad, 20% surprised) would be classified as a Quantitative analysis.

# concept[].concept[].property[].valueString
2022-06-07 vote 5-0 by Brian S. Alper, Paola Rosati, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].definition
A study design process in which data are analyzed, without primary reliance on mathematical or statistical techniques, by coding and organizing data to provide interpretation or understanding of experiences or hypotheses.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].property[].valueString
2022-05-17 vote 5-4 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde
2022-05-24 vote 4-1 by Robin Ann Yurk, Eric M Harvey, Mario Tristan, Harold Lehmann, Jesus Lopez-Alcalde

# concept[].concept[].property[].valueString
2022-05-17 comments: What about ordinal data such as low/medium/high? I think I would view that as qualitative.
Suggest revise definition to include examples of the analysis methods as Alternative terms or comment for applications:  ie focus groups.  There are many new software tools which apply quantitative methods to qualitative studies.
Quantitative and qualitative have categorical results I believe
Disagree - (Sorry, maybe you already know my comment here ;>). From my experience, qualitative analysis produces more than descriptive or categorical results, and uses a range of essential complex methodologies for producing unattainable results from trials. Some methods are inductive, others are deductive, or a mix of both. This modify the results achievable. For example, phenomenology from categorical codes produces new understanding of people's lived experiences (deemed robust, even from a small but convenient sample of people), whereas grounded theory, from descriptive and categorical data results, discovers or creates novel theories, crucial for subsequent research scrutiny, even for a trial. 
I would suggest to define qualitative analysis differently = A study design process in which data, analysed and coded to produce descriptive and categorical results, lead to new understanding of people's lived experiences or new theories, unattainable from quantitative studies, essential for future trials.
In my opinion, descriptive numerical results come from quantitative analysis also (for example, incidence of SARS-COV2 per 100.000 habitants). I am not an expert in qualitative research but I guess it tackles phenomenons which can be observed but not measured.

2022-05-24 comment: Qualitative analysis provide a description or summary to understand exploratory experiences and patterns, themes in the data which can provide the framework for additional data interpretation through other analysis such as quantitative analysis.  An example of a qualitative method is focus groups.  Technology exists such as natural language processing or other software to report the analysis.

2022-05-26 comment: I would leave the description for a quantitative analysis out of the definition.  I would also delete the example of feelings as this can be quantified through satisfaction research which is a quantitative analysis.  I would give an example of focus groups or nature language processing.  The method involves identifying themes in narrative text.

# concept[].concept[].property[].valueString
The distinction of quantitative vs. qualitative analysis refers to whether mathematical processing is involved, whether or not the analysis includes numerical variables. Processing a categorical variable (e.g. values of happy, sad, or jealous as a response to "How are you feeling?") to produce numerical results (e.g. 30% happy, 50% sad, 20% surprised) would be classified as a Quantitative analysis. Processing the transcripts of interviews to categorize phrases and report themes identified across interviews would be classified as a Qualitative analysis. Qualitative analysis techniques may include phenomenology development from categorical codes, and may result in discovery or creation of theories that are unattainable through quantitative analysis.

# concept[].concept[].property[].valueString
2022-06-07 vote 5-0 by Brian S. Alper, Paola Rosati, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].definition
A study design process in which study participants are not informed of their intervention assignment.

# concept[].concept[].property[].valueString
Masking of study participants involves actions to conceal information that could lead to their awareness of their intervention assignment, such as provision of placebo or simulated interventions that mimic the target interventions.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte

# concept[].concept[].property[].valueString
2022-08-23 vote 6-0 by Mario Tristan, Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Eric Harvey, Robin Ann Yurk

# concept[].concept[].definition
A study design process in which the people administering the intervention are not informed of the intervention assignment.

# concept[].concept[].property[].valueString
Masking of intervention providers involves actions to conceal information that could lead to their awareness of the intervention assigned to individual study participants, such as provision of placebo interventions that mimic the target interventions.

The terms 'double-blinding' and 'triple-blinding' are not clearly and consistently defined terms but typically suggest blinding of intervention providers.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Kenneth Wilkins

# concept[].concept[].property[].valueString
2022-08-23 vote 6-0 by Mario Tristan, Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Eric Harvey, Robin Ann Yurk

# concept[].concept[].definition
A study design process in which the people determining the outcome are not informed of the intervention assignment.

# concept[].concept[].property[].valueString
Masking of outcome assessors involves actions to conceal information that could lead to their awareness of the intervention assigned to individual study participants to minimize the influence of such awareness on the determination of outcome measurement values.

The terms 'triple-blinding' and 'quadruple-blinding' are not clearly and consistently defined terms but may suggest blinding of outcome assessors.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Kenneth Wilkins

# concept[].concept[].property[].valueString
2022-08-23 vote 5-0 by Mario Tristan, Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Eric Harvey

# concept[].concept[].definition
A study design process in which the people managing or processing the data and statistical analysis are not informed of the intervention assignment.

# concept[].concept[].property[].valueString
The term 'data analysts' is meant to include any person who works with the data at any point between data collection and the reporting of analyzed results.

Masking of data analysts involves actions to conceal information that could lead to their awareness of the intervention assigned to individual study participants, such as noninformative labeling used to represent the study groups.

The terms 'triple-blinding' and 'quadruple-blinding' are not clearly and consistently defined terms but may suggest blinding of data analysts.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Janice Tufte, Kenneth Wilkins

# concept[].concept[].property[].valueString
2022-08-23 vote 5-0 by Mario Tristan, Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Eric Harvey

# concept[].concept[].definition
A study design process in which all parties influencing study enrollment and allocation to study groups are unaware of the group assignment for the study participant at the time of enrollment and allocation.

# concept[].concept[].property[].valueString
Allocation concealment occurs before and during the enrollment process and refers to limiting awareness of assignment during the process of recruitment and assignment to groups. Other blinding and masking terms refer to limiting awareness of the assignment during and after enrollment.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].property[].valueString
2022-08-30 vote 8-0 by Janice Tufte, nisha mathew,: Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Muhammad Afzal, Cauê Monaco, Eric Harvey

# concept[].definition
An aspect or characteristic of a study design.

# concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Janice Tufte

# concept[].property[].valueString
Study design is defined as a plan specification for how and what kinds of data will be gathered as part of an investigation which may produce testable explanations, conclusions and predictions or test a hypothesis.

# concept[].property[].valueString
2022-03-15 comments: I agree with the definition but feel this is meta-vocabulary that helps us talk about elements of study design that are not part of the code system itself. So I don't know if it should be included in the code system as a code, or if we should be considering some other means for defining these terms (e.g. in documentation or guidance about SEVCO).
not a fan of the synonym "study design factor" as it could cause confusion with 'Study Factor", Independent Variable. How different Study Design is from Study Protocol? "Study design planned process"  could cover the following subtypes
For the comment for application include ...as a technical plan specification....
2022-03-22 comment: The definition of "Study design" seems to exclude the "statistical analysis". Am I right?

# concept[].property[].valueString
2022-03-15 vote 7-0 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte, Robin Ann Yurk (but then the definition changed with the creation of Study Design Process)
2022-03-22 vote 5-1 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, nelle.stocquart, Robin Ann Yurk

# concept[].property[].valueString
2022-03-29 vote 8-0 by Paul Whaley, Mario Tristan, Jesus Lopez-Alcalde, Cauê Monaco, Joanne Dehnbostel, Philippe Rocca-Serra, Robin Ann Yurk, nelle.stocquart

# concept[].concept[].definition
A study design feature in which two or more institutions are responsible for the conduct of the study.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].property[].valueString
This term may be used for studies conducted across multiple contexts (administrative or logistical) that are distinct from geographic locations, potentially introducing greater variability beyond multisite data collection.

# concept[].concept[].property[].valueString
2022-03-15 vote 5-1 by Mario Tristan, Paul Whaley, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Robin Ann Yurk
2022-04-26 vote 3-0 by Eric M Harvey, Robin Ann Yurk, Mario Tristan

# concept[].concept[].property[].valueString
2022-03-15 comments: Suggest add to multiple contexts (reserach)

a multicenter study is_a study. 'multicentric' would be a subtype of study_design_feature.  
a concern here is that the current definition conflates 2 entities: a study and a characteristic of that study. 
At the end of the day, it depends on how the modeling will be made, e.g <study> <has_some_study_design_feature> <type of study_design_feature>

Or should it be "Multicenter data collection"  ?

# concept[].concept[].property[].valueString
2022-05-06 vote 6-0 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann

# concept[].concept[].definition
A study design feature in which one or more outcomes are reported directly from the patient without interpretation by a clinician or researcher.

# concept[].concept[].property[].valueString
Examples of patient-reported outcomes include symptoms, pain, quality of life, satisfaction with care, adherence to treatment, and perceived value of treatment. Data collection methods including surveys and interviews may obtain patient-reported outcomes. Reports derived from wearable devices would not typically include patient-reported outcomes. Such data may be coded with 'Real world data collection from monitoring procedures' (SEVCO:01046).

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].property[].valueString
The U.S. Food and Drug Administration (FDA) defines a patient-reported outcome (PRO) as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else [1].” -- from https://dcricollab.dcri.duke.edu/sites/NIHKR/KR/PRO%20Resource%20Chapter.pdf

# concept[].concept[].property[].valueString
2022-04-26 vote 3-0 by Eric M Harvey, Robin Ann Yurk, Mario Tristan

# concept[].concept[].property[].valueString
2022-04-26 comment: Suggest adding to the comment for application:  data methods to collect Patient Reported Outcomes such as survey data.
2022-05-06 comments: Perhaps direct the reader to "Patient generated health data" or whatever else is the SEVCO term for "wearables" or other data sources (e.g., bluetooth scale).
Suggest adding to the comment for application:  data methods to collect Patient Reported Outcomes such as survey data.
2022-06-07 preferred term changed from "Patient-reported outcome" to "Includes patient-reported outcome" to maintain consistency with sibling concepts

# concept[].concept[].property[].valueString
2022-05-06 vote 7-0 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte

# concept[].concept[].definition
A study design feature in which one or more measures are outcomes that patients directly care about, i.e. outcomes that are directly related to patients' experience of their life.

# concept[].concept[].property[].valueString
In healthcare research, outcomes are effects on patients or populations, including changes to health status, behavior, or knowledge as well as patient satisfaction and quality of life. A patient-centered outcome qualifies the type of outcome as that which patients directly care about, i.e. outcomes that are directly related to patients' experience of their life. Examples of patient-centered outcomes include mortality, morbidity, symptoms, and quality of life. Some use 'clinical outcome' as synonymous with 'patient-centered outcome' while some use 'clinical outcome' to represent outcomes that would assessed as part of healthcare practice.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Mario Tristan, Khalid Shahin

# concept[].concept[].property[].valueString
2022-05-06 vote 5-2 by Mario Tristan, Robin Ann Yurk, Eric M Harvey, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte
2022-05-17 vote 7-2 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].property[].valueString
2022-04-26 comment: Suggest Adding to comment for application:  Population Statistics such as mortality, morbidity.   Development of clinical outcomes is based on using a framework such as the Donabedian model:  Structure, Process, Outcomes where outcomes have some relationship to structural or process measures in clinical care.....
2022-05-06 comment:  I understand the goal of "quantity or quality of life," but I think it's too abstract--and limiting ("Quantity of life" is limited to life expectancy). I haven't reviewed other definitions, but the flavor is, "outcomes that patients care about." ("Function" is left off the list of "examples", albeit there is a large overlap with "morbidity," "symptoms," and "quality of life.") (See the Comments for Application for Surrogate Outcome!)
2022-05-17 comments: The definition seems to be in the comment: "A clinical outcome qualifies the type of outcome as that which patients directly care about." The definition as proposed doesn't really make sense to me.
The definition, Alternative terms and comment for application are correct.  However, it is more specific to patient reported outcomes.  Clinical outcomes are more broad and also includes:  physiologic measures, condition specific measures.....etc. Clinical outcomes can be structural, process or outcomes in the donabedian framework and or combined as composite outcomes.
While patient centered outcomes are typically considered clinical outcomes, they also indicate the observed outcomes  by the clinician but not so much by the patient. 

2022-05-26 comment: Suggest revise term definition so it is more inclusive or all healthcare or clinical outcomes, such as mortality, morbidity, physiologic measures, symptoms, experiences.  The term is not a study design but a measure.

For example:
A healthcare measure which captures results from healthcare populations, settings structures, processes, and patients directly related to their care with healthcare settings, people, providers, and interventions.

Insert other Alternative terms:  Morbidity, Mortality, Symptoms, Experience of Care, Health Status, Quality of life.  Suggest delete Patient Oriented Outcome, Patient Important Outcome, Patient Relevant Outcome, Patient Centered OUtcome, Includes clinical outcomes,

# concept[].concept[].property[].valueString
2022-06-07 vote 5-0 by Brian S. Alper, Paola Rosati, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].definition
A study design feature in which one or more measures are outcomes that relate to a health or illness condition but are not outcomes which patients directly care about.

# concept[].concept[].property[].valueString
In healthcare research, outcomes are effects on patients or populations, including changes to health status, behavior, or knowledge as well as patient satisfaction and quality of life. A patient-centered outcome qualifies the type of outcome as that which patients directly care about. Examples of patient-centered outcomes include mortality, morbidity, symptoms, and quality of life. A disease-oriented outcome qualifies the type of outcome as that which patients do not directly care about. Examples of disease-oriented outcomes include laboratory test measurements, imaging study findings, and calculated risk estimates. In this context, disease-oriented outcomes may be used as surrogate outcomes or proxy outcomes for ultimate effects on patient-centered outcomes, but do not provide direct evidence of effects on patient-centered outcomes.

# concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Khalid Shahin

# concept[].concept[].property[].valueString
2022-05-06 vote 4-2 by Mario Tristan, Robin Ann Yurk, nisha mathew, Paola Rosati, Harold Lehmann, Janice Tufte
2022-05-17 vote 6-2 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde

# concept[].concept[].property[].valueString
2022-05-06 comments: I would refer to whatever "clinical outcome" is defined as: "...indirect measures of clinical outcomes". The Comment for Application is redefining Clinical Outcome. I would spend that space pointing out that some surrogates are predictive (e.g., cholesterol levels, for MIs)  and others are after the fact (e.g., sales of orange juice for treating the flu).
Suggestion--look at the wikipedia definition, then explore other mapping definitions.  The current term definition and comment for application need improvement.

"In clinical trials, a surrogate endpoint is a measure of effect of a specific treatment that may correlate with a real clinical endpoint but does not necessarily have a guaranteed relationship. The National Institutes of Health defines surrogate endpoint as "a biomarker intended to substitute for a clinical endpoint".  wikipedia...

2022-05-17 comments: Maybe edit to "An indirect measure of quantity or quality of life, presumed or believed to have an effect on clinical outcomes."
I would focus on revising and define surrogate first and then include a broad definition, not just specific to clinical outcomes.

2022-05-26 comment: Surrogate Outcome is a proxy measure for capturing the outcome of interest.

Alternative Terms:  delete disease oriented and surrogate outcome measure.  Suggest add:  Proxy Outcome Measure.

Comment for application:  Delete first 3 sentences.
Edit the last sentence so it reads:   A surrogate outcome is a measure which captures an approximate measure.  Examples of surrogate outcomes includes survey measures rating scales for a child by the parent or teacher.  Geriatric rating scales from paid or professional caregivers for a seriously ill or geriatric patient are other examples.

# concept[].concept[].property[].valueString
2022-06-07 vote 5-0 by Brian S. Alper, Paola Rosati, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].definition
A study design feature in which one or more outcomes are actions or behaviors of a healthcare professional or care team.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Khalid Shahin, Kenneth Wilkins

# concept[].concept[].property[].valueString
2022-05-17 vote 7-1 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde
2022-05-17 vote 8-2 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde, nelle.stocquart@kce.fgov.be, Mario Tristan
2022-05-31 vote 8-2 by Paul Whaley, Harold Lehmann, Robin Ann Yurk, Paola Rosati, Eric Harvey, raradhikaag@gmail.com, Cauê Monaco, Jesus Lopez-Alcalde, nelle.stocquart@kce.fgov.be, Mario Tristan

# concept[].concept[].property[].valueString
2022-05-17 comment: Process Measure is included in the donabedian framework of structure, process, outcomes.  Do you want to define just for healthcare process measure versus keep the definition broad to include such as a series of steps or tasks providing a measurement pathway for any industry and the examples in healthcare processes are....
2022-05-24 comments: repeat 2022-05-17 comment plus:
you need to provide more info, it is not clear as such

2022-05-31 comments:
you need to provide more info, it is not clear as such
Add comment for application with examples:  A process measures captures the steps to care such as 
Lab test orders, Referrals....The literature defines a process measure in the donabedian framework of structure, process, outcomes.

# concept[].concept[].property[].valueString
A process outcome measure is a measure of change in actions or behaviors conducted in the process of healthcare delivery or clinical care, such as obtaining laboratory tests or referrals for follow-up care.

# concept[].concept[].property[].valueString
2022-06-07 vote 5-0 by Brian S. Alper, Paola Rosati, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].definition
A study design feature specifying the intent of the study.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].property[].valueString
2022-06-21 vote 5-0 by Janice Tufte, Louis Leff, Mario Tristan, Eric M Harvey, Muhammad Afzal

# concept[].concept[].property[].valueString
2022-06-21 comment: Another alternate term could be "Study Objective"

# concept[].concept[].concept[].definition
A study goal to assess the efficiency, effectiveness, and impact of a given program, process, person or piece of equipment.

# concept[].concept[].concept[].property[].valueString
Medical Subject Heading (MESH): this heading is used as a Publication Type; for original report of the conduct or results of a specific evaluation study; a different heading EVALUATION STUDIES AS TOPIC is used for general design, methodology, economics, etc. of evaluation studies
Scope Note
Works consisting of studies determining the effectiveness or utility of processes, personnel, and equipment. https://meshb.nlm.nih.gov/record/ui?ui=D023362

# concept[].concept[].concept[].property[].valueString
Intended to include all forms of evaluation study. (Child concepts for program, process, personnel and equipment evaluations may be added later.)

# concept[].concept[].concept[].property[].valueString
2022-07-12 vote 6-0 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Paola Rosati, Harold Lehmann, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].definition
A study goal with the intent to generate a predictive algorithm.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2022-07-19 vote 8-0 by Paola Rosati, Cauê Monaco, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A study goal with the intent to determine the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

# concept[].concept[].concept[].property[].valueString
https://meshb.nlm.nih.gov/record/ui?ui=D023361
MeSH Heading:  Validation Study
Annotation:
This heading is used as a Publication Type for original report of the conduct or results of a specific validation study. A different heading VALIDATION STUDIES AS TOPIC is used for general design, methodology, economics, etc. of validation studies. CATALOGER: Do not use
Scope Note:
Works consisting of research using processes by which the reliability and relevance of a procedure for a specific purpose are established.
Entry Term(s):
Validation Studies

# concept[].concept[].concept[].property[].valueString
Procedures that may be assessed in validation studies include predictive algorithms, measurement instruments, and educational materials. Internal validation is tested in populations from the source used for derivation of the procedure. External validation is tested in populations that differ from the source used for derivation of the procedure.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-07-19 vote 8-1 by Paola Rosati, Cauê Monaco, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-07-19 comment: "Relevance" is a value judgment that is not the hallmark of a validation study. (It requires elicitation of this judgment from experts or potential users.)
Accuracy, while difficult to measure, is certainly a validation aspiration (goal). Thus, validation of instruments assesses their sensitivity and specificity (measures of "accuracy").
Perhaps a broader goal is "performance", which would include accuracy but also applicability across sites or other external contexts.
Also, typo: "*from* the source used..."

# concept[].concept[].concept[].property[].valueString
2022-07-26 vote 6-0 by Jesus Lopez-Alcalde, Harold Lehmann, Paola Rosati, Eric Harvey, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].definition
A study design feature in which the study intent is to compare two or more interventions or exposures.

# concept[].concept[].concept[].property[].valueString
MeSH term "Equivalence Trial" https://www.ncbi.nlm.nih.gov/mesh/2023172 Trial that aims to show a new treatment is no better and no worse than the standard treatment.

Year introduced: 2018

Do not include MeSH terms found below this term in the MeSH hierarchy.

Tree Number(s): V03.175.250.500.500.125

MeSH Unique ID: D000073843

Entry Terms:

Non-Inferiority Trial
Noninferiority Trial
Superiority Trial
Equivalence Clinical Trial

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2022-06-21 vote 5-0 by Janice Tufte, Louis Leff, Mario Tristan, Eric M Harvey, Muhammad Afzal

# concept[].concept[].concept[].concept[].definition
A study design feature in which the study intent is to compare two or more interventions with respect to benefits and/or harms.

# concept[].concept[].concept[].concept[].property[].valueString
In 2009, the Institute of Medicine committee defined comparative effectiveness research (CER) as: "Comparative effectiveness research is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels."

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
Defining comparative effectiveness research (CER) was the first order of business for the Institute of Medicine Committee on Initial Priorities for CER. The Institute of Medicine committee approached the task of defining CER by identifying the common theme in the 6 extant definitions. The definition follows: "Comparative effectiveness research is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels." https://pubmed.ncbi.nlm.nih.gov/20473202/

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-21 vote 5-0 by Janice Tufte, Louis Leff, Mario Tristan, Eric M Harvey, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].definition
A study design feature in which the study intent is to compare two or more interventions with respect to effectiveness in ideal conditions.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-28 vote 8-0 by Jesus Lopez-Alcalde, Philippe Rocca-Serra, Harold Lehmann, Muhammad Afzal, Janice Tufte, Louis Leff, Mario Tristan, Eric M Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-28 comment: what does "in ideal conditions" really mean? is it necessary ?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Efficacy is defined as effectiveness in ideal conditions. In this context, an efficacy goal is a type of effectiveness goal. Efficacy is used to distinguish the context from effectiveness in 'real-world' settings.

# concept[].concept[].concept[].concept[].concept[].definition
A study design feature in which the study intent is to compare two or more interventions with respect to harms.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-28 vote 8-0 by Jesus Lopez-Alcalde, Philippe Rocca-Serra, Harold Lehmann, Muhammad Afzal, Janice Tufte, Louis Leff, Mario Tristan, Eric M Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-28 comment: no need to be "in ideal conditions" ? see related comment on. "comparative efficacy goal" class textual definition

# concept[].concept[].concept[].concept[].definition
A study goal with the intent to compare two or more interventions or exposures and determine that any difference in effects is within a prespecified range representing absence of a meaningful difference.

# concept[].concept[].concept[].concept[].property[].valueString
MeSH term "Equivalence Trial" https://www.ncbi.nlm.nih.gov/mesh/2023172 Trial that aims to show a new treatment is no better and no worse than the standard treatment.

Year introduced: 2018

Do not include MeSH terms found below this term in the MeSH hierarchy.

Tree Number(s): V03.175.250.500.500.125

MeSH Unique ID: D000073843

Entry Terms:

Non-Inferiority Trial
Noninferiority Trial
Superiority Trial
Equivalence Clinical Trial

# concept[].concept[].concept[].concept[].property[].valueString
An Equivalence Goal is only applicable with a Comparative study design. 
The prespecified range representing absence of a meaningful difference may be defined with an equivalence margin.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 vote 4-2 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Paola Rosati, Janice Tufte, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 comments: harmonize the definition to match the pattern used for the other terms. e.g. "evaluation goal" is a study goal in which the objective is to assess the efficience, effectivement and impact of a given process, process, person or piece of equipment'

so Equivalence Goal is a study goal in which the study intent is to compare two or more interventions or exposures and determine that any difference in effects is within a prespecified range representing absence of a meaningful difference

I think this definition is unclear. Is the equivalence goal an aim of a study? Which kind of study? My understanding is: Given a prespecified range (of results?) showing an absence of a meaningful (for which kind of subjects/previous research?) difference between two interventions/exposures, the equivalence goal assesses that there is no difference in effects. Is this the meaning of this definition? Which kind of study could give a valid result in terms of equivalence? An RCT?

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-19 vote 8-0 by Paola Rosati, Cauê Monaco, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A study goal with the intent to compare two or more interventions or exposures and determine that any difference in effects is below a prespecified value representing a threshold between a meaningful difference and absence of a meaningful difference.

# concept[].concept[].concept[].concept[].property[].valueString
MeSH term "Equivalence Trial" https://www.ncbi.nlm.nih.gov/mesh/2023172 Trial that aims to show a new treatment is no better and no worse than the standard treatment.

Year introduced: 2018

Do not include MeSH terms found below this term in the MeSH hierarchy.

Tree Number(s): V03.175.250.500.500.125

MeSH Unique ID: D000073843

Entry Terms:

Non-Inferiority Trial
Noninferiority Trial
Superiority Trial
Equivalence Clinical Trial

# concept[].concept[].concept[].concept[].property[].valueString
A Non-inferiority Goal is only applicable with a Comparative study design. 
The threshold between a meaningful difference and absence of a meaningful difference may be called a non-inferiority margin.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 vote 3-2 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Paola Rosati, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 comments: so Non-Inferiorty Goal is a study goal in which....
I have the same doubts already given for the equivalence goal

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-19 vote 8-0 by Paola Rosati, Cauê Monaco, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A study goal with the intent to compare two or more interventions or exposures and detect a difference in effects.

# concept[].concept[].concept[].concept[].property[].valueString
MeSH term "Equivalence Trial" https://www.ncbi.nlm.nih.gov/mesh/2023172 Trial that aims to show a new treatment is no better and no worse than the standard treatment.

Year introduced: 2018

Do not include MeSH terms found below this term in the MeSH hierarchy.

Tree Number(s): V03.175.250.500.500.125

MeSH Unique ID: D000073843

Entry Terms:

Non-Inferiority Trial
Noninferiority Trial
Superiority Trial
Equivalence Clinical Trial

# concept[].concept[].concept[].concept[].property[].valueString
A Superiority Goal is only applicable with a Comparative study design. 

A superiority study goal may be exploratory (to detect a difference) or confirmatory (to establish that a difference exists with a degree of certainty).

A superiority goal is not the opposite of a non-inferiority goal. A superiority goal uses a threshold of zero difference while an inferiority goal uses a threshold of a meaningful difference.

Some superiority comparisons are conducted following determination of non-inferiority.

Placebo-controlled trials are typically superiority studies.

Superiority, as commonly used, is 'statistical superiority,' with null used as the threshold of effect. An approach representing 'clinical superiority' would use the non-inferiority margin as the threshold of effect.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 vote 4-1 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Paola Rosati, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-12 comment: so Superiority Goal is a study goal in which...
2022-07-19 comment: alter definition to "...and detect *meaningful* difference in effects" (in order to be consistent with Equivalence and Non-inferiority Study Goals')

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-19 vote 9-0 by Paola Rosati, Cauê Monaco, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey, Janice Tufte

# concept[].concept[].definition
A study design feature describing the intended relative proportion of assignment across groups.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].property[].valueString
The allocation ratio may be expressed as Treatment:Control, e.g., 2:1, or, in the case of two treatment arms and one control, e.g. 2:2:1.

# concept[].concept[].property[].valueString
2023-04-10 vote 5-0 by Harold Lehmann, Joanne Dehnbostel, Eric Harvey, Janice Tufte, Jesus Lopez-Alcalde

# concept[].definition
A systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]).

# concept[].property[].valueString
Brian S. Alper, Philippe Rocca-Serra, Joanne Dehnbostel, Mario Tristan, Harold Lehmann; Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].property[].valueString
8/8 as of 2021-02-26: , Harold Lehmann, Khalid Shahin, Eric Harvey, Jesús López-Alcalde, Joanne Dehnbostel, Muhammad Afzal, Paola Rosati, Eric Au, 5/5 for second sentence as of 8/30/2021: Eric Harvey, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Janice Tufte

# concept[].concept[].definition
A bias resulting from methods used to select subjects or data, factors that influence initial study participation, or differences between the study sample and the population of interest

# concept[].concept[].property[].valueString
Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Asiyah Yu Lin, Joanne Dehnbostel

# concept[].concept[].property[].valueString
8/8 as of 3/5/2021 Eric Au, Alejandro Piscoya, Mario Tristan, Brian Alper, Zbys Fedorowicz, Bhagvan Kommadi, Eric Harvey, Muhammad Afzal

# concept[].concept[].property[].valueString
Selection bias can occur before the study starts (inherent in the study protocol) or after the study starts (during study execution).

# concept[].concept[].concept[].definition
A selection bias resulting from methods used to select participating subjects, factors that influence initial study participation, or differences between the study participants and the population of interest

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Asiyah Yu Lin, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
10/10 as of 3/22/2021 Harold Lehmann, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Ahmad Sofi-Mahmudi, Tatyana Shamliyan, Muhammad Afzal, Paola Rosati, Joanne Dehnbostel, Marc Duteau

# concept[].concept[].concept[].property[].valueString
2021-03-08 vote 7-2 on "A selection bias where key characteristics of the participants differ systematically from the population of interest." by Harold Lehmann, Philippe Rocca-Serra, Joanne Dehnbostel, 2021-03-19 vote 10-1 on "A bias resulting from methods used to select participating subjects, factors that influence initial study participation, or differences between the study participants and the population of interest" by Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Asiyah Yu Lin

# concept[].concept[].concept[].concept[].definition
A selection bias resulting from inclusion and exclusion criteria used to select participating subjects that could result in differences between the study participants and the population of interest.

# concept[].concept[].concept[].concept[].designation[].value
selection bias due to inappropriate selection criteria

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Asiyah Yu Lin, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 3/22/2021 Harold Lehmann, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Ahmad Sofi-Mahmudi, Tatyana Shamliyan, Muhammad Afzal, Paola Rosati, Joanne Dehnbostel, Marc Duteau

# concept[].concept[].concept[].concept[].property[].valueString
2021-03-19 vote 9-2 on "A bias resulting from inclusion and exclusion criteria used to select participating subjects that could make the included participants unrepresentative of the population of interest." by Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Asiyah Yu Lin

# concept[].concept[].concept[].concept[].definition
A selection bias resulting from the sampling frame, sampling procedure, or methods used to recruit participating subjects that could result in differences between the study participants and the population of interest.

# concept[].concept[].concept[].concept[].designation[].value
selection bias due to inappropriate sampling strategy

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 3/22/2021 Harold Lehmann, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Ahmad Sofi-Mahmudi, Tatyana Shamliyan, Muhammad Afzal, Paola Rosati, Joanne Dehnbostel, Marc Duteau

# concept[].concept[].concept[].concept[].property[].valueString
2021-03-19 vote 9-2 on "A bias resulting from the sample frame, sampling procedure, or methods used to recruit participating subjects that could make the included participants unrepresentative of the population of interest." by Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].display
inappropriate data source for participant selection

# concept[].concept[].concept[].concept[].concept[].definition
Participant selection bias due to inappropriate data source for sampling frame.

# concept[].concept[].concept[].concept[].concept[].designation[].value
participant selection bias due to inappropriate data source for sampling frame

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin, Muhammad Afzal, Bhagvan Kommadi

# concept[].concept[].concept[].concept[].concept[].property[].valueString
6/6 as of 4/12/2021: KM Saif-Ur-Rahman, Bhagvan Kommadi, Joanne Dehnbostel, Paola Rosati, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].definition
A selection bias due to differences between the included participants and the population of interest that distorts the research results (estimation of effect, association, or inference), limiting external validity or applicability.

# concept[].concept[].concept[].concept[].designation[].value
selection bias due to non-representative sample

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 3/22/2021 Harold Lehmann, Eric Harvey, Bhagvan Kommadi, Jesus Lopez-Alcalde, Ahmad Sofi-Mahmudi, Tatyana Shamliyan, Muhammad Afzal, Paola Rosati, Joanne Dehnbostel, Marc Duteau

# concept[].concept[].concept[].concept[].property[].valueString
2021-03-19 vote 10-1 on "Differences between the included participants and the population of interest that distorts the research results (estimation of effect, association, or inference), limiting external validity or applicability." by Brian S. Alper, Tatyana Shamliyan, Bhagvan Kommadi, Muhammad Afzal, Khalid Shahin, Harold Lehmann, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].display
inadequate enrollment of eligible subjects

# concept[].concept[].concept[].concept[].concept[].definition
A selection bias in which insufficient enrollment of eligible subjects results in differences (recognized or unrecognized) between the included participants and the population of interest that distorts the research results.

# concept[].concept[].concept[].concept[].concept[].designation[].value
selection bias due to inadequate enrollment

# concept[].concept[].concept[].concept[].concept[].designation[].value
non-representative sample due to inadequate enrollment

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Philippe Rocca-Serra, Marc Duteau, Khalid Shahin, Asiyah Yu Lin, Muhammad Afzal, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
11/11 as of 3/29/2021: Alejandro Piscoya, Eric Harvey, Bhagvan Kommadi, Ahmad Sofi-Mahmudi, Eric Au, Joanne Dehnbostel, Marc Duteau, Brian S. Alper, Jesús López-Alcalde, Tatyana Shamliyan, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-03-26 vote 8-2 on "Inadequate enrollment = A selection bias due to a rate of study entry among eligible subjects that is not sufficient for the included sample to be considered representative of the population of interest." by Harold Lehmann, Tatyana Shamliyan, Muhammad Afzal, Eric Au, Paola Rosati, Mario Tristan, Alejandro Piscoya, Bhagvan Kommadi, Jesús López-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
non-representative sample due to timing or duration of exposure

# concept[].concept[].concept[].concept[].concept[].definition
A selection bias in which the timing or duration of exposure influences the outcome, and the timing or duration of exposure in the sample does not represent that of the population of interest. This selection bias may occur when the selection for study participation is not coincident with the initiation of the exposure or intervention under investigation.

# concept[].concept[].concept[].concept[].concept[].designation[].value
mismatch in start of intervention and start of follow-up

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 4/9/2021: Alejandro Piscoya, KM Saif-Ur-Rahman, Bhagvan Kommadi, Eric Harvey, Joanne Dehnbostel, Paola Rosati, Harold Lehmann, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].concept[].concept[].definition
A non-representative sample due to exclusion of susceptible participants who have already had an outcome due to prior exposure. For example, the inclusion of prevalent users of a medication misrepresents the initial adverse effects rate by excluding persons who do not tolerate the medication.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
non-representative sample due to depletion of susceptibles

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 4/9/2021: Alejandro Piscoya, KM Saif-Ur-Rahman, Bhagvan Kommadi, Eric Harvey, Joanne Dehnbostel, Paola Rosati, Harold Lehmann, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].definition
A selection bias in which factors observed after study entry, baseline, or start of follow-up influence enrollment

# concept[].concept[].concept[].concept[].designation[].value
participant selection bias due to post-baseline factors

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Philippe Rocca-Serra, Marc Duteau, Khalid Shahin, Asiyah Yu Lin, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 4/5/2021: Alejandro Piscoya, KM Saif-Ur-Rahman, Bhagvan Kommadi, Eric Harvey, Joanne Dehnbostel, Mario Tristan, Harold Lehmann, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].concept[].display
participant selection bias due to early study termination

# concept[].concept[].concept[].concept[].concept[].definition
A selection bias due to premature closure of study enrollment.

# concept[].concept[].concept[].concept[].concept[].designation[].value
early termination bias affecting enrollment

# concept[].concept[].concept[].concept[].concept[].property[].valueString
'Early termination bias affecting enrollment' is a type of 'Post-baseline factors influence enrollment selection' which is defined as 'A selection bias in which factors observed after study entry, baseline, or start of follow-up influence enrollment.'
To express bias related to making the decision to terminate a study, use 'Early Study Termination Bias'.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Paul Whaley, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-25 vote 7-1 by Muhammad Afzal, Paul Whaley, Mario Tristan, Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-25 comment: Recommend simplifying the term and then add your test to the term definition.
For example edit term to Early Study Termination Bias.  Term definition should read.  Selection Bias due to premature closing of a study enrollment for the participants....

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-04-08 vote 12-0 by Muhammad Afzal, Paul Whaley, Mario Tristan, Joanne Dehnbostel, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, nelle.stocquart, nisha mathew, Harold Lehmann, Cauê Monaco

# concept[].concept[].concept[].concept[].display
factor associated with exposure influences enrollment selection

# concept[].concept[].concept[].concept[].definition
A selection bias in which a factor associated with the exposure under investigation influences study enrollment

# concept[].concept[].concept[].concept[].designation[].value
participant selection bias due to factor associated wiith exposure

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Khalid Shahin, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 4/5/2021: Alejandro Piscoya, KM Saif-Ur-Rahman, Bhagvan Kommadi, Eric Harvey, Joanne Dehnbostel, Mario Tristan, Harold Lehmann, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].concept[].display
factor associated with outcome influences enrollment selection

# concept[].concept[].concept[].concept[].definition
A selection bias in which a factor associated with the outcome under investigation influences study enrollment

# concept[].concept[].concept[].concept[].designation[].value
participant selection bias due to factor associated wiith outcome

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Khalid Shahin, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
9/9 as of 4/5/2021: Alejandro Piscoya, KM Saif-Ur-Rahman, Bhagvan Kommadi, Eric Harvey, Joanne Dehnbostel, Mario Tristan, Harold Lehmann, Jesús López-Alcalde, Tatyana Shamliyan

# concept[].concept[].concept[].definition
A selection bias resulting from factors that influence study selection, from methods used to include or exclude studies for evidence synthesis, or from differences between the study sample and the population of interest

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin, Muhammad Afzal, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
6/6 as of 4/26/2021: Eric Harvey, Bhagvan Kommadi, Harold Lehmann, Mario Tristan, Jesús López-Alcalde, Tatyana Shamliyan
2024-04-19 vote 8-0 by Cauê Monaco, Janice Tufte, Homa Keshavarz, Sheyu Li, Khalid Shahin, Lenny Vasanthan, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A study selection bias specific to the inclusion and exclusion criteria.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
If the study eligibility criteria (inclusion and exclusion criteria for study selection) result in a dataset that is non-representative of the population of interest, then the criteria introduce systematic error. 

A study selection bias is a selection bias resulting from factors that influence study selection, from methods used to include or exclude studies for evidence synthesis, or from differences between the study sample and the population of interest.

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-02 vote 5-0 by Brian S. Alper, Harold Lehmann, Xing Song, Cauê Monaco, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
study eligibility criteria not prespecified

# concept[].concept[].concept[].concept[].concept[].definition
A bias in study eligibility criteria in which the criteria are not stated before the study selection process occurs.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 1.1 Did the review adhere to pre-defined objectives and eligibility criteria?
A systematic review should begin with a clearly focused question or objective which is reflected in the criteria used for deciding whether studies are eligible for inclusion. Details that should be specified a priori in a review protocol vary according to review type, but should generally include the study designs, study participants, and types of interventions/exposures that are eligible. If outcomes or outcome domains are to form part of the eligibility criteria, this should be stated clearly. Any exclusions should also be pre-specified. Where a protocol providing this information is available, the answer to this question would be “Yes”. Where no protocol is available but information about pre-defined objectives and detailed eligibility criteria are supplied, and there is good reason to believe that these were specified in advance and adhered to throughout the
review, assessors can consider answer this question “Probably Yes”. Any post hoc changes to the eligibility criteria or outcomes must keep faith with the objectives of the review, and be properly justified and documented. In the absence of a pre-published protocol, where information about pre-defined objectives and eligibility criteria are only available post hoc in the review publication, unless there is some reason to believe that these details were specified in advance and adhered to from the start of the review, this question should be answered “Probably No”. Where all or some of these details are missing, this question should be answered “No”.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Failure to specify the study eligibility criteria before evaluating studies for selection can lead to a situation in which the data discovered during the study selection process influences the criteria for selection in a way that introduces systematic error.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-16 vote 6-0 by Javier Bracchiglione, Lenny Vasanthan, Xing Son, Brian S. Alper, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
study eligibility criteria not appropriate for review question

# concept[].concept[].concept[].concept[].concept[].definition
A bias in study eligibility criteria due to a mismatch between the inclusion and exclusion criteria and the research question.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Homa Keshavarz, Xing Song

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 1.2 Were the eligibility criteria appropriate for the review question?
The eligibility criteria should stem from the review question and should provide sufficient detail to enable judgement about whether the studies that are included are appropriate to the question. The information required is likely to vary by topic. For example, in order to judge appropriateness, the assessor might need a clear description of the population in terms of the age range and diagnosis of the study participants, the setting in which the study was conducted, the dose of a drug, or the frequency of exposure. To answer this question the assessor is likely to require some content knowledge.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The mismatch between the inclusion and exclusion criteria and the research question may relate to differences in the population, exposures, or outcomes studies from those of interest.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-16 vote 5-1 by Javier Bracchiglione, Lenny Vasanthan, Xing Song, Brian S. Alper, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-16 comment: I would simplify the definition to: "A bias in study eligibility criteria derived from a mismatch between the inclusion and exclusion criteria, and the research question, that could result in an inappropriate selection of studies"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-08 vote 5-0 :Lenny Vasanthan, Harold Lehmann, Janice Tufte, Javier Bracchiglione, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].definition
A bias in study eligibility criteria due to unclear specification.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 1.3 Were eligibility criteria unambiguous?
Specific information about the characteristics of eligible studies must be provided, as far as possible avoiding any ambiguities about the types of study, population, interventions, comparators and outcomes. Criteria should be sufficiently detailed that the review could be replicated using the criteria specified. A number of important details are commonly missing from the eligibility criteria in systematic reviews. For example, details about the diagnosis of study participants. Diagnosis might be made using a number of different methods, some of which might be more valid or accurate than others. Review authors should have decided in advance which diagnostic methods are appropriate to their review question in order to avoid introducing potential biases during the review process. Similarly, specific details about interventions/exposures and comparators must be provided, including characteristics such as medication dose, frequency of administration, concurrent treatments, and so on. The assessor is likely to require some content knowledge to answer this question, but where specific queries remain about the stated eligibility criteria, “No” or “Probably No” judgements can usually be made.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Eligibility criteria that are not sufficiently described to enable reproduction of study selection can introduce systematic error.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-16 vote 5-0 by Lenny Vasanthan, Xing Son, Brian S. Alper, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
study eligibility criteria limits for study characteristics not appropriate

# concept[].concept[].concept[].concept[].concept[].definition
A bias in study eligibility criteria that is specific to restrictions based on characteristics of the study design, conduct, or findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Xing Song, Joanne Dehnbostel, Kenneth Wilkins, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate?
Any restrictions applied on the basis of study characteristics must be clearly described and a sound rationale provided. These details will enable assessors to judge whether such restrictions were appropriate. Examples might be the study design, the date the study was published, the size of the study, some measure of study quality, and available outcomes measures. This question is different from the one above which refers to whether the eligibility criteria are appropriate to the review question. Where sufficient information is available, and the assessor is reasonably satisfied that the restrictions are appropriate, this question can be answered “Yes or “Probably Yes”. Where restrictions around study characteristics are not justified and there is insufficient information to judge whether these restrictions are appropriate, this question should be answered “Probably No” or “No”. Where eligibility criteria are sufficiently detailed, and no restrictions around study characteristics are explicitly reported, it can be assumed that none were imposed, and the question should be answered “Yes”.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Any restrictions applied on the basis of study characteristics should not introduce a bias in study eligibility criteria. Examples of such restrictions may include criteria based on study size, study design, study quality, or date when the study was conducted.

In the ROBIS tool used for risk of bias assessment of systematic reviews, there is a question (1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate?) that is different from the one which refers to whether the eligibility criteria are appropriate to the review question. Therefore, a separate term is available in SEVCO.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-1 by Homa Keshavarz, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Xing Song, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-23 comment: As it is, study characteristics could refer to clinical characteristics (e.g. age), which could indeed be appropriate. I would state that it refers to methodological characteristics.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-08 vote 5-0 Lenny Vasanhan, Harold Lehmann, Eric Harvey, Janice Tufte, Homa Kashavarz

# concept[].concept[].concept[].concept[].concept[].display
study eligibility criteria limits for study report characteristics not appropriate

# concept[].concept[].concept[].concept[].concept[].definition
A bias in study eligibility criteria that is specific to restrictions on the status, structure, language, or accessibility of the study report.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 1.5 Were any restrictions in eligibility criteria based on sources of information appropriate?
Any restrictions applied on the basis of sources of information must be clearly described and a sound rationale provided. These details will enable assessors to judge whether such restrictions were appropriate. Examples might be the publication status or format, language, and availability of data. This question is different from the question in domain 2 which is about restricting searches. Where eligibility criteria are sufficiently detailed, but no restrictions based on sources of information are explicitly reported, it must be assumed that none were imposed, and the question should be answered “Yes”.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Examples of study report characteristics include publication status (including preprints and unpublished data), format, language, and availability of data, as well as the date of publication.
The ROBIS tool used for assessing risk of bias of systematic reviews includes a signaling question '1.5 Were any restrictions in eligibility criteria based on sources of information appropriate?'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-16 comment: (in comments for application, there is a dot after the parenthesis that needs to be removed)

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A bias in search strategy or study eligibility criteria that results from restrictions regarding the language of the study report.

# concept[].concept[].concept[].concept[].property[].valueString
Limiting the study reports included in a systematic review by language may result in an incomplete view of the truly available evidence.

The terms [search strategy limits for study report characteristics not appropriate](https://fevir.net/resources/CodeSystem/27270#SEVCO:00266) and [study eligibility criteria limits for study report characteristics not appropriate](https://fevir.net/resources/CodeSystem/27270#SEVCO:00277) are used to describe types of study selection bias that results from restrictions regarding the study report, distinguishing different steps in the search and selection process.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
05-03-2024 
"what is interesting about this term is that language in terms of terms changes and actually can begin to mean something other than the original intent- later original term might mean something else down the road"

# concept[].concept[].concept[].concept[].property[].valueString
05-03-2024 5-0 by Homa Keshavarz, Eric Harvey, Janice Tufte, Lenny Vasanthan, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A bias in search strategy or study eligibility criteria that results from restrictions regarding the geographic origin of the research.

# concept[].concept[].concept[].concept[].property[].valueString
The role of geographic bias in knowledge diffusion: a systematic review and narrative synthesis (https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-019-0088-0) includes "geographic bias...are biased by the geographic origin of the research" and "geographic bias, such as the role of institutional affiliation, country of origin..." and "geographic bias, i.e., local, regional, national, or international"

# concept[].concept[].concept[].concept[].property[].valueString
Limiting the study reports included in a systematic review by country of origin may result in an incomplete view of the truly available evidence.

The geographic origin of the research may refer to the location of the research participants, the investigators, or their associated organizations.

 The terms [search strategy limits for study report characteristics not appropriate](https://fevir.net/resources/CodeSystem/27270#SEVCO:00266) and [study eligibility criteria limits for study report characteristics not appropriate](https://fevir.net/resources/CodeSystem/27270#SEVCO:00277) are used to describe types of study selection bias that results from restrictions regarding the study report, distinguishing different steps in the search and selection process.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
05-03-2024 5-0 by Homa Keshavarz, Eric Harvey, Janice Tufte, Lenny Vasanthan, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A study selection bias in which the publicly available studies are not representative of all conducted studies.

# concept[].concept[].concept[].concept[].designation[].value
study selection bias due to selective reporting bias

# concept[].concept[].concept[].concept[].designation[].value
study selection bias due to non-reporting bias

# concept[].concept[].concept[].concept[].designation[].value
study selection bias due to reporting bias

# concept[].concept[].concept[].concept[].property[].valueString
Publication bias arises from the failure to identify all studies that have been conducted, either published (i.e., publicly available) or unpublished. The term 'studies' means evidence or research results in any form where such studies would meet the study eligibility criteria without consideration of criteria regarding the form of publication. The phrase 'publicly available studies' means the studies are available to the broad academic community and the public through established distribution channels in any form, including forms with restricted access.  Established distribution channels include peer-reviewed journals, books, conference proceedings, dissertations, reports by governmental or research organizations, preprints, and study registries.

Publication bias often leads to an overestimate in the effect in favor of the study hypothesis, because studies with statistically significant positive results are more likely to be publicly available.

The terms [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) and [selective reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00024) are used to describe biases in study reports, i.e., reporting bias. To avoid confusion between biases in study reports and biases in study selection, when either 'reporting bias' or 'selective reporting bias' are used as alternative for 'publication bias', the term is appended to 'study selection bias due to' as shown below:
	• study selection bias due to reporting bias
	• study selection bias due to selective reporting bias

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 vote 7-1 by Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Cauê Monaco, Eric Harvey, Jennifer Hunter
2024-04-12 vote 6-1 by Sheyu Li, Jennifer Hunter, Janice Tufte, Eric Harvey, Homa Keshavarz, Lenny Vasanthan, Harold Lehmann
2024-04-19 vote 8-1 by Janice Tufte, Homa Keshavarz, Sheyu Li, Khalid Shahin, Lenny Vasanthan, Harold Lehmann, Eric Harvey, Jennifer Hunter, Cauê Monaco
2024-04-26 vote 4-1 by Homa Keshavarz, Sean Grant, Lenny Vasanthan, Jennifer Hunter, Eric Harvey
2024-05-03 vote 6-1 by Harold Lehmann, Lenny Vasanthan, Jennifer Hunter, janice Tufte, Eric Harvey, Homa Keshavarz, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 comment:
The current definition does not reflect the information of 'publication', which is the core of the term. I understand that it is not good to use the same word in the definition and the term itself. Nevertheless, the word available could be vague - studies could be available in registration website only but not published. For different authors, the 'availability' of the studies are different. 

A suggested definition:
A study selection bias in which the published studies are not representative of the conducted studies.	
alternatively: 
A study selection bias in which the studies available in the literature database are not representative of all conducted studies.

2024-04-12 comment:
Publication bias arises from the failure to identify all studies that have been conducted, either published (i.e., publicly available) or unpublished. Typically, publication bias leads to an overestimate in the effect in favour of the study hypothesis. This is because studies with statistically significant positive results are more likely to be publicly available or published in English-language journals.

2024-04-19 comment:
Consider adding some alternate terms used by Cochrane: non-reporting bias; bias due to missing results. https://training.cochrane.org/handbook/current/chapter-07#section-7-1

2024-04-26 comments:
1) Suggest revising second paragraph slightly to "Publication bias often (but not always) leads to..."
2N) The last paragraph/sentence in application is long and difficult to read. Here is a suggestion:
The terms reporting bias and selective reporting bias are used to describe biases in study reports. To avoid ambiguous use as alternative terms for publication bias, the terms are appended to 'study selection bias due to'.

2024-05-03 comments:
The last paragraph is still very difficult to read, and I understand what we are trying to communicate. What about this suggestion?
The terms reporting bias and selective reporting bias are used to describe biases in study reports. To avoid confusion between biases in study reports and biases in study selection, when either 'reporting bias' or 'selective reporting bias' are used as alternative for 'publication bias', the term is appended to 'study selection bias due to'.

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-10 vote 10-0 by Saphia Mokrane, Sheyu Li, Harold Lehmann, Brian S. Alper, Homa Keshavarz, Lenny Vasanthan, Cauê Monaco, Jennifer Hunter, Eric Harvey, Khalid Shahin

# concept[].concept[].concept[].concept[].definition
A study selection bias specific to the strategy used to identify potentially eligible studies.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Homa Keshavarz, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-23 comment: Although I am not 100% sure if this qualifies as "study selection" bias

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-23 vote 6-0 by Khalid Shahin, Homa Keshavarz, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].definition
A bias in search strategy in which the electronic sources are not sufficient to find the studies available in electronic sources.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports?
The assessor needs to judge what constitutes an appropriate range of databases. This will vary according to review topic. It is anticipated that at a minimum a MEDLINE and EMBASE search would be conducted. Searches of material published as conference reports should also be considered along with a search of research registers. Guidance on the appropriate range of databases can be found in SR guidance such as the Cochrane Handbook,5 or from the Centre for Reviews and Dissemination (CRD) website (http://www.york.ac.uk/inst/crd/finding_studies_systematic_reviews.htm)

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The set of databases (electronic sources) expected to include the studies of interest will vary with the review topic.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Caue Monaco, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Expert working group agreement 2024-03-01 5-0 vote by Eric Harvey, Harold Lehmann, Khalid Shahin, Javier Bracchiglione, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].definition
A bias in search strategy in which the sources other than electronic database sources are not sufficient to find the studies available.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 2.2 Were methods additional to database searching used to identify relevant reports?
Additional methods such as citation searches, contacting experts, reference checking, handsearching etc. should have been performed.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The set of sources other than databases (electronic sources) expected to include the studies of interest will vary with the review topic.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Caue Monaco, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-08 "I wonder about globally and developing nations where CHW workers are collecting data perhaps not electronically" , "Similar to other comments global data might be collected oin paper- qualitative - observational studies"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-08 6-0 Eric Harvey, Harold Lehmann, Javier Bracchiglione, Janice Tufte, Homa Kashavarz, Lenny Vasanthan

# concept[].concept[].concept[].concept[].concept[].definition
A bias in search strategy in which the search terms and combinations of search terms are not sufficient to find the studies available.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible?
A full search strategy showing all the search terms used, in sufficient detail to replicate the search, is required to be able to fully judge this question. If only limited details are provided, such as a list of search terms with no indication of how these are combined, assessors may be able to make a “Probably Yes” or “Probably No” judgment. Assessors should consider whether the search strategy included an appropriate range of terms for the topic, whether a combination of controlled terms (such as Medical Subject Headings (MeSH) for Medline) and words in the title and abstract were used, and whether any filters applied were appropriate. For example, for DTA reviews the use of filters has been shown to miss relevant studies and so this question should be answered as No for a strategy that includes such filters. Guidance on the critical appraisal of search strategies can be found in the PRESS Evidence-Based Checklist (http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/7402).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The search terms and combinations of search terms expected to include the studies of interest will vary with the review topic.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Caue Monaco, Homa Keshavarz

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-08 vote 5-0 Eric Harvey, Harold Lehmann, Javier Bracchiglione, Homa Kasharvarz, Lenny Vasanthan

# concept[].concept[].concept[].concept[].concept[].display
search strategy limits for study report characteristics not appropriate

# concept[].concept[].concept[].concept[].concept[].definition
A bias resulting from search strategy criteria implemented due to practical considerations for implementation, including properties of a report or its accessibility.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 2.4 Were restrictions based on date, publication format, or language appropriate?
If no restrictions were applied to the search strategy then this question should be answered as Yes. This is different from the question in domain 1 (1.5) which is about restriction to selection criteria. Information is required on all three components of this question (i.e. date, publication format and language) to be able to fully judge this item. Restriction of papers based on language (e.g. restriction to English language articles) or publication format (e.g. restriction to full text published studies) is rarely (if ever) appropriate, and so if any such restrictions were applied then this question should usually be answered as “No”. Restrictions on date may be appropriate but should be supported by a clearly described rationale for this question to be answered as “Yes”. For example, if a medication or test was not available before a certain date then it is reasonable to only start searches from the date at which the medication or test first became available.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A bias in search strategy that is specific to restrictions on the date, status, structure, language, or accessibility of the study report. Accessibility refers to where a resource is available and how one gains access to it.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Homa Keshavarz, Khalid Shahin, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-03-15 vote 5-0 by Xing Song, Lenny Vasanthan, Harold Lehmann, Homa Keshavarz, Eric Harvey

# concept[].concept[].concept[].concept[].display
misapplication of study eligibility criteria

# concept[].concept[].concept[].concept[].definition
A study selection bias due to inappropriate implementation of the study inclusion and exclusion criteria.

# concept[].concept[].concept[].concept[].designation[].value
nonadherence to study eligibility criteria

# concept[].concept[].concept[].concept[].property[].valueString
Sheyu Li, Ken Wilkins, Muhammad Afzal, Homa Keshavarz, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 vote 7-1 by Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Javier Bracchiglione, Jennifer Hunter, Homa Keshavarz, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 comment:
The title/name needs work. Maybe something like: Bias due to non-adherence to eligibility criteria?

# concept[].concept[].concept[].concept[].property[].valueString
2024-04-12 vote 6-0 by Harold Lehmann, Sheyu Li, Jennifer Hunter, Janice Tufte, Homa Keshavarz, Eric Harvey

# concept[].concept[].concept[].concept[].display
bias related to selection of the studies for synthesis

# concept[].concept[].concept[].concept[].definition
A study selection bias due to inappropriate choice of studies included in the synthesis.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 4.1 Did the synthesis include all studies that it should?

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Airton Stein, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2025-05-17 comment: (There could be a link to "Selection bias", making the point that this term is the equivalent for synthesis studies.)

2024-05-24 comment: Hmm. Maybe change, "typical use" to "typical risk"?

2024-06-07 comments re: "synthesis missing eligible studies" = "A synthesis bias in which eligible studies were not included in the evidence synthesis."
1N) The term itself is new to me: is there a more established term?
re: "synthesis missing eligible studies" = "A synthesis bias in which eligible studies were not included in the evidence synthesis."
2N) It is confusing regarding synthesis. Evidence synthesis includes quantitative synthesis (meta-analysis typically) and qualitative synthesis. The comments may mean that a study is included in a qualitative synthesis but not a quantitative synthesis. It can be true, but why? Typically it can be some synthestic gap, e.g., the unavailability of zero events in a meta-analysis. 
The definition can be revised as A synthesis bias in which eligible studies were not included in the some parts of evidence syntheses.

2024-06-14 comment re: "bias related to selection of the data for synthesis" = "A synthesis bias due to inappropriate choice of data included in the synthesis before the synthesis is applied."
1N) I understand the concept though I am not clear how this is a "bias"? "Inappropriate" choice just sounds like poor study execution rather than a bias.


2024-08-09 comment re: "bias related to selection of the studies for synthesis" = "A synthesis bias due to inappropriate choice of studies included in the synthesis before the synthesis is applied."
1N)
What is the difference between this bias and selection bias?   Is it is related to some technical barriers such as zero even issue? The synthesis have to opt out the zero event trials because of the restriction of the statistical methods.


2024-08-16 comment re: "bias related to selection of the studies for synthesis" = "A study selection bias due to inappropriate choice of studies included in the synthesis."
1N)
I fail to see how the comments for application differ from bias related to the selection of the data for synthesis. There seems to be considerable overlap.

Referring to a "specific meta-analysis" may be the issue, as this requires the selection of specific results from the included studies. Additionally, a meta-analysis is only one example of how results can be synthesized, and it does not account for other data (e.g., qualitative). Perhaps something like the following might work: "A potential use of this term is when studies were included in the overall review, yet none of the study results or findings (quantitative or qualitative) were analyzed or synthesized.

I'm unsure that "and appropriate for synthesis" is correct. For instance, an included study may have measured the outcome of interest but not reported the results in a way that can be used in the planned meta-analysis (e.g., only the p value is reported, or no SDM/SE is reported). Even though the results cannot be used (i.e., are not appropriate) for the meta-analysis, there is still a risk of distorted results which constitutes bias. Perhaps I am mistaken, and this bias only refers to errors of judgement by the reviewers.


2024-08-23 comment re: "bias related to selection of the studies for synthesis" = "A study selection bias due to inappropriate choice of studies included in the synthesis."
1Y)
I Could suggest a petit change in 'comment for application'..."If the selected for synthesis do not match the available studies, there is a risk of distorted results which constitutes bias."

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-23 vote 9-0 by Carlos Alva-Diaz, Elma OMERAGIC, Lenny Vasanthan, Harold Lehmann, Philippe Rocca-Serra, Eric Harvey, Sean Grant, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].concept[].concept[].property[].valueString
A potential use of this term is when studies meeting the review criteria were available but not included in the review. Another potential use of this term is when studies were included in the overall review but were not included in a specific meta-analysis or specific synthesis. If the studies selected for synthesis do not match the available studies, there is a risk of distorted results which constitutes bias.
The term 'bias related to selection of the studies for synthesis' matches the ROBIS signaling question 4.1 'Did the synthesis include all studies that it should?'
If a study was selected for the overall review but the data was not extracted from the study for a specific analysis or synthesis, then use [bias related to selection of the data for synthesis](https://fevir.net/resources/CodeSystem/27270#SEVCO:00352)

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-17 vote 5-0 by Saphia Mokrane, Lenny Vasanthan, Sheyu Li, Eric Harvey, Harold Lehmann
2024-05-24 vote 7-0 by Homa Keshavarz, Sheyu Li, Eric Harvey, Lenny Vasanthan, Harold Lehmann, Janice Tufte, Saphia Mokrane
2024-06-07 vote 6-2 by Sean Grant, Saphia Mokrane, Sheyu Li, Lenny Vasanthan, Harold Lehmann, Eric Harvey, Carlos Alva-Diaz, Kailei Nong
2024-06-14 vote 7-1 by Yaowaluk Ngoenwiwatkul, Homa Keshavarz, Sean Grant, Sheyu Li, Eric Harvey, Lenny Vasanthan, Harold Lehmann, Janice Tufte
2024-06-21 vote 8-0 by Cauê Monaco, Lenny Vasanthan, Homa Keshavarz, Yaowaluk Ngoenwiwatkul, Harold Lehmann, Sean Grant, Eric Harvey, Carlos Alva-Diaz BUT THEN THE TERM CHANGED from 'data' to 'studies'
2024-08-09 vote 7-1 by Brian S. Alper, Harold Lehmann, Homa Keshavarz, Sheyu Li, Sean Grant, Eric Harvey, Lenny Vasanthan, Airton Tetelbom Stein
2024-08-16 vote 5-1 by Cauê Monaco, Bhagvan Kommadi, Jennifer Hunter, Eric Harvey, Harold Lehmann, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].definition
A study selection bias due to an inadequate process for screening and/or evaluating potentially eligible studies.

# concept[].concept[].concept[].concept[].property[].valueString
from ROBIS 2.5 Were efforts made to minimise errors in selection of studies?
Both the process of screening titles and abstracts and of assessing full text studies for inclusion are covered by this question. Information on both are required to be able to fully judge this item. For an answer of “Yes”, titles and abstracts should be screened independently by at least two reviewers and full text inclusion assessment should involve at least two reviewers (either independently or with one performing the assessment and the second checking the decision).

# concept[].concept[].concept[].concept[].property[].valueString
An adequate process for screening and evaluating potentially eligible studies should generally include at least two independent reviewers for any steps that involve subjective judgment. Any step involving subjective judgment may introduce systematic distortions into the research findings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 vote 6-1 by Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Eric Harvey, Jennifer Hunter
2024-04-12 vote 6-2 by Cauê Monaco, Sheyu Li, Jennifer Hunter, Janice Tufte, Eric Harvey, Homa Keshavarz, Lenny Vasanthan, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-29 comments:
1) Although the comment for application may be questionable for some specific cases.
2N) The title/name needs more work. Perhaps something like: Study screening bias? Or just: Screening bias?

2024-04-12 comments:
1N) The term 'bias' refers to systematic error but back-to-back check by different reviewers reduce only random error. I do not think the definition and comment are in line. For my own experience, there is little room for bias during the study selection. 
2N) Suggest replace "and" with "or", as I am assuming that screening refers to T&A screening and evaluating refers to full text inclusion assessment. The bias could arise from one step only.
Consider adding some alternative terms e.g., bias in study selection, bias in selection of studies (that maps to ROBIS), screening bias, and/or study screening bias.
3) Re Comment for Application: There are other strategies, besides 2 independent readers, but I suppose the word, "generally," addresses my concern.

# concept[].concept[].concept[].concept[].property[].valueString
2024-04-19 vote 9-0 by Cauê Monaco, Sheyu Li, Jennifer Hunter, Janice Tufte, Eric Harvey, Homa Keshavarz, Lenny Vasanthan, Harold Lehmann, Khalid Shahin

# concept[].concept[].definition
A situation in which the effect or association between an exposure and outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Philippe Rocca-Serra, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi

# concept[].concept[].property[].valueString
2023-07-14 vote 5-0 by Paul Whaley, Harold Lehmann, Cauê Monaco, Jesus Lopez-Alcalde, Paola Rosati

# concept[].concept[].property[].valueString
2021-05-07 vote 4-2 on "Comparator Bias = A bias resulting from differences (other than in variables directly involved in the analysis) between the groups being compared." by KM Saif-Ur-Rahman, Harold Lehmann, Alejandro Piscoya, Paola Rosati, Tatyana Shamliyan, Bhagvan Kommadi
2021-05-10 vote 11-1 on "Confounding Covariate Bias = A bias resulting from differences in covariates (variables other than the exposure and outcome) between the groups being compared." by Eric Harvey, KM Saif-Ur-Rahman, Janice Tufte, Bhagvan Kommadi, Paola Rosati,  Alejandro Piscoya, Harold Lehmann, Ahmad Sofi-Mahmudi, Eric Au, Jesus Lopez-Alcalde, Tatyana Shamliyan, Joanne Dehnbostel
AGREEMENT VOTE 8/8 as of 5/17/2021: Tatyana Shamliyan, Janice Tufte, Mario Tristan, Bhagvan Kommadi, Jesús López-Alcalde, Isaac Fwemba, Eric Harvey, Paola Rosati

On 2023-06-16 the Steering Group corrected a technical error in the definition (between A or B ... corrected to ... between A and B), and added a Comment for Application, so re-opened the term for vote.

# concept[].concept[].property[].valueString
A bias resulting from differences (other than in variables directly involved in the analysis) between the groups being compared. ---led to --- Which differences do you mean between the groups?  This definition seems unclear.  Defining a Comparator bias means to addresss some possible specific explanation. Or it is preferable to delete this bias. The definition is for selection bias resulting from nonrandom allocation of participants to interventions. Random allocation of trial participants to interentions would reduce this bias. Comprator seletion would not.  A bias resulting from differences in covariates (variables other than the exposure and outcome) between the groups being compared -- led to I agree with the definition but I suggest detailing that the covariate is associated to the outcome

# concept[].concept[].property[].valueString
Association of any two variables includes direct associations and indirect associations through each of the variables having direct associations with a third variable.

# concept[].concept[].concept[].definition
A confounding covariate bias resulting from methods for assignment of the independent variable by the investigator to evaluate a response or outcome.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 5/17/2021: Tatyana Shamliyan, Janice Tufte, Mario Tristan, Bhagvan Kommadi, Jesús López-Alcalde, Isaac Fwemba, Eric Harvey, Paola Rosati

# concept[].concept[].concept[].property[].valueString
2021-05-07 vote 5-1 on "Comparator Selection Bias = A comparator bias resulting from methods for selection of or allocation to groups for comparative analysis that have the potential to introduce differences (other than in variables directly involved in the analysis) between the groups being compared." by KM Saif-Ur-Rahman, Harold Lehmann, Alejandro Piscoya, Paola Rosati, Tatyana Shamliyan, Bhagvan Kommadi, 2021-05-10 vote 11-1 on "Allocation Bias = A confounding covariate bias resulting from methods for assignment of exposures in an interventional study." by Eric Harvey, KM Saif-Ur-Rahman, Janice Tufte, Bhagvan Kommadi, Paola Rosati,  Alejandro Piscoya, Harold Lehmann, Ahmad Sofi-Mahmudi, Eric Au, Jesus Lopez-Alcalde, Tatyana Shamliyan, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
A comparator bias resulting from methods for selection of or allocation to groups for comparative analysis that have the potential to introduce differences (other than in variables directly involved in the analysis) between the groups being compared. -- led to--- Selection of comparators would not reduce differences between compared groups.  A confounding covariate bias resulting from methods for assignment of exposures in an interventional study. --led to-- In my opinion, in an interventional study the investigator assigns the intervention, not the exposures.  The differences in the covariates results from the methods for the assignment of the intervention. For example not concealed allocation.

# concept[].concept[].concept[].concept[].definition
An allocation bias resulting from awareness of the assigned intervention before study enrolment and intervention assignment

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].definition
An allocation bias resulting from a process of assigning participants or subjects to different groups or conditions which is not random.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
A confounding covariate bias is defined as a situation in which the effect or association between an exposure and outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

Allocation bias is defined as a confounding covariate bias resulting from *methods for assignment* of the independent variable by the investigator to evaluate a response or outcome.

Methods for assignment that are not random may introduce confounding with measured or unmeasured variables.

Non-random methods of generation of an allocation sequence may introduce a confounding covariate bias through associations with one ore more non-random variables related to sequence generation. A non-random allocation sequence may be described as a predictable sequence in mathematical terms.

The SEVCO term [Quasi-Randomized assignment](https://fevir.net/resources/CodeSystem/27270#SEVCO:01004) is defined as an interventional study design with a method of allocation that is not limited to random chance but is intended to produce similar baseline groups for experimentation. Although Quasi-Randomized assignment is "intended to produce similar baseline groups" the term is classified as a type of [Non-randomized assignment](https://fevir.net/resources/CodeSystem/27270#SEVCO:01005). Examples of non-random methods (which may be called 'partially randomized' or 'quasi-random') include every other participant, day of the week, even/odd identification number, birth date, etc.

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-12 vote 4-1 by Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Jesus Lopez-Alcalde, Harold Lehmann
2023-05-26 vote 5-1 by Harold Lehmann, Jesus Lopez-Alcalde, Sunu Alice Cherian, Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel
2023-06-09 vote 4-1 by Eric Harvey, Cauê Monaco, Paul Whaley, Sunu Alice Cherian, Harold Lehmann
2023-06-16 vote 3-2 by Eric Harvey, Paul Whaley, Sunu Alice Cherian, Harold Lehmann, Paola Rosati
2023-07-14 vote 4-1 by Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann, Cauê Monaco, Paola Rosati

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-12 comment: Does this term definition actually define "inadequate" = potentially predictable sequence ? seems like an innapropriate allocation bias perhaps
2023-05-26 comment: Defintion: Methods of allocating study participants to treatment comparison groups that are not random, but are intended to produce similar groups. 
Alternative terms: Quasi random allocation
2023-06-09 comment: The comment for application describes "unrecognised associations", but the definition talks about "potentially predictable", which implies exploiting a recognised association to break blinding. I am not sure it can be both of these.
2023-06-16 comments:
Type of bias that arises in research studies when the process of assigning participants or subjects to different groups or conditions is not random.
I think I remember my original concern now - in the definition, the problem is not that the sequence is predictable, it is that the sequence is associated with another variable, thus introducing this other variable as a confounder. Unless it is about the investigator being able to break blinding, in which case the concept of the sequence being predictable is important.

2023-07-14 comment: I think non-random methods are those clearly non-random, such as allocation by provider's preferences. However, quasi-random methods are those that apply a method that attempts to be random but that it isn't. Example: day of the week.

2023-07-28 comment: For consistency, should we call it, "Confounding Bias due to non-randomized allocation"?

# concept[].concept[].concept[].concept[].property[].valueString
2023-07-28 vote 5-0 by Brian S. Alper, Paul Whaley, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].definition
A confounding covariate bias resulting from methods used to select participating subjects, or factors that influence study participation, for the comparator group.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 5/17/2021: Tatyana Shamliyan, Janice Tufte, Mario Tristan, Bhagvan Kommadi, Jesús López-Alcalde, Isaac Fwemba, Eric Harvey, Paola Rosati

# concept[].concept[].concept[].property[].valueString
This situation is more commonly related to observational research.

# concept[].concept[].concept[].definition
A confounding covariate bias in which the unequal distribution of a potentially distorting variable is recognized.

# concept[].concept[].concept[].designation[].value
recognized difference with potential for confounding

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].concept[].property[].valueString
2021-05-07 vote 5-1 on "Recognized Difference with Potential for Confounding = A comparator bias resulting from known differences (other than in variables directly involved in the analysis) between the groups being compared." by KM Saif-Ur-Rahman, Harold Lehmann, Alejandro Piscoya, Paola Rosati, Tatyana Shamliyan, Bhagvan Kommadi, , 2021-05-24 vote 6-1 on "A confounding covariate bias in which the unequal distribution of a potentially distorting variable is recognized." by Harold Lehmann, Eric Harvey, KM Saif-Ur-Rahman, Bhagvan Kommadi, janice tufte, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].property[].valueString
A comparator bias resulting from known differences (other than in variables directly involved in the analysis) between the groups being compared. -- led to-- This defintion seems tricky. If you find any diference between groups that can go astray with analysis you simply address the potention for confounding explicitly in the discussion session of yoru protocol/paper The potnetial for confounding needs to be consideriend in the protocol, and specifically addresssed int eh post-analysis to avoid any further bias. The term comparator bias is misleading since differnece between groups would not be reduced by selecting different comparators.   If this is recognized and adjusted for, is it still a bias? Seems that we need to address this circumstance.

# concept[].concept[].concept[].property[].valueString
The potentially distorting variable is a covariate, and not the exposure or the outcome. Even if adjusted for in the analysis, a risk of bias can be present.

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is the time at which the outcome is measured or observed.

# concept[].concept[].concept[].property[].valueString
A confounding covariate bias is defined as a situation in which the effect or association between an exposure and outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

The time at which the outcome is measured or observed may be absolute (e.g. a specific date) or relative (e.g. 3 months after study enrollment).

To understand "confounding by time of observation" consider the following example:

An observational study is comparing patients with asthma taking Superdrug and patients with asthma not taking Superdrug.  The outcome of interest is mortality. The patients taking Superdrug are observed for their full duration of exposure to Superdrug. For comparison, the control group not receiving Superdrug is measured during a 1-year calendar period.

For the mortality outcome comparing Superdrug vs. no Superdrug, the time of observation for the control group is consistently 1 year but for the Superdrug group the time of observation varies for each patient. This comparison is confounded by the time of observation.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Paul Whaley

# concept[].concept[].concept[].property[].valueString
2023-06-09 vote 3-1 by Eric Harvey, Cauê Monaco, Paul Whaley, Sunu Alice Cherian
2023-06-16 vote 4-1 by Paola Rosati, Eric Harvey, Paul Whaley, Sunu Alice Cherian, Harold Lehmann
2023-07-14 vote 7-0 by Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin, Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann, Cauê Monaco
THEN REOPENED
2023-08-04 due to comment that suggests removing parenthetical from definition

# concept[].concept[].concept[].property[].valueString
2023-06-09 comment: The comment for application is not sufficiently informative. I am also not sure I understand what the definition means - what is the importance of recognition of unequal distribution of follow-up time?
2023-06-16 comments: A confounding that occurs when the relationship between an exposure or intervention and an outcome is confounded by the time at which the outcome is measured or observed. 
Alternate terms: time-varying confounding
Comment for application: This occurs when both the exposure and the outcome change over time, and there are other time-dependent factors that influence the outcome

The Comment for Application seems to be repeating the definition of the parent term. I though we usually add details specific to the current term.

# concept[].concept[].concept[].property[].valueString
2023-10-06 vote 5-0 by Jesus Lopez-Alcalde, Eric Harvey, Paul Whaley, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is the length of time that the participant has had the condition of interest at study enrollment.

# concept[].concept[].concept[].property[].valueString
A Confounding Covariate Bias is defined as a situation in which the effect or association between an exposure or outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

A lead time bias is often manifest as a distortion overestimating the apparent time surviving with a disease caused by bringing forward the time of its diagnosis (https://catalogofbias.org/biases/lead-time-bias/).

Lead time bias is a type of bias that occurs in medical screening or diagnostic tests when the early detection of a disease or condition artificially appears to improve survival or prognosis, even if it does not actually provide a true benefit in terms of overall health outcomes.

Lead time refers to the amount of time between the detection of a disease through early screening or diagnostic testing and the time when the disease would have been clinically detected without screening.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].property[].valueString
Lead time bias
A distortion overestimating the apparent time surviving with a disease caused by bringing forward the time of its diagnosis
https://catalogofbias.org/biases/lead-time-bias/

# concept[].concept[].concept[].property[].valueString
2023-06-09 vote 3-1 by Eric Harvey, Cauê Monaco, Paul Whaley, Sunu Alice Cherian
2023-06-16 vote 3-2 by Eric Harvey, Paul Whaley, Sunu Alice Cherian, Harold Lehmann, Paola Rosati
2023-07-14 vote 6-0 by Muhammad Afzal, Joanne Dehnbostel, Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann, Cauê Monaco
THEN REOPENED
2023-08-04 due to comment that suggests removing parenthetical from definition

# concept[].concept[].concept[].property[].valueString
2023-06-09 comment: I am not sure I can successfully parse the syntax of the definition. While I think I understand what is meant, I feel it could be phrased more clearly.
2023-06-16 comments: Lead time bias is a type of bias that occurs in medical screening or diagnostic tests when the early detection of a disease or condition artificially appears to improve survival or prognosis, even if it does not actually provide a true benefit in terms of overall health outcomes

Comment for application: Lead time refers to the amount of time between the detection of a disease through early screening or diagnostic testing and the time when the disease would have been clinically detected without screening. 

This definition seems difficult to understand: does it convey that lead time bias is related to the potentially distorting variable of the length of time chosen in the study in which some participants could have confounding differences between their diagnosis of the condition of interest and the time of enrolment?
I have some problem in understanding, sorry.

# concept[].concept[].concept[].property[].valueString
2023-08-18 vote 5-0 by Paul Whaley,  Eric Harvey, Mario Tristan, Cauê Monaco, Harold Lehmann

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is associated with deviations from the intended intervention.

# concept[].concept[].concept[].property[].valueString
A confounding covariate bias is defined as a situation in which the effect or association between an exposure or outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

For 'Confounding influencing adherence to intervention', the association of the distorting variable and the exposure is specific to deviations from the intended exposure (intended intervention). Deviations from the intended intervention may include deviations from the intervention protocol or lack of adherence. Lack of adherence includes imperfect compliance, cessation of intervention, crossovers to the comparator intervention and switches to another active intervention.

The term 'Confounding influencing adherence to intervention' is distinct from 'Performance Bias' (including 'Nonadherence of participants' or 'Imbalance in deviations from intended interventions') in that an additional variable (the distorting variable or confounding covariate) is acting as a confounder, while the 'Performance Bias' may occur with or without any differences in a third variable.

# concept[].concept[].concept[].property[].valueString
trigger question from ROBINS-I: 1.3. Were intervention discontinuations or switches likely to be related to factors that are prognostic for the outcome?

# concept[].concept[].concept[].property[].valueString
2023-06-02 comment from steering group: need to see the background to ROBINS-I to understand context for this term
2023-07-28 comment: I think the definition is good but the comment for application should specifically address this term and not just duplicate the definition of confounding covariate bias.
2023-08-11 comment: Is this the same as compliance bias, or compliance bias  ("https://catalogofbias.org/biases/compliance-bias/") is a subtype of this? If "compliance bias" is a synonim, should be added as such. If not, should be added as a separate term

# concept[].concept[].concept[].property[].valueString
Brian Alper, Joanne Dehnbostel, Harold Lehmann, Paul Whaley, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2023-07-28 vote 3-1 by Eric Harvey, Harold Lehmann, Jesus Lopez-Alcalde, Paul Whaley
2023-08-04 vote 5-0 by Joanne Dehnbostel, Philippe Rocca-Serra, Eric Harvey, Harold Lehmann, Mario Tristan BUT comment suggests removing parenthetical from definition
2023-08-11 vote 3-1 by Mario Tristan, Cauê Monaco, Eric Harvey, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-09-29 vote 5-0 by Joanne Dehnbostel, Harold Lehmann, Paul Whaley, Eric Harvey, Mario Tristan

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is the reason for receiving an exposure.

# concept[].concept[].concept[].property[].valueString
Confounding by indication
A distortion that modifies an association between an exposure and an outcome, caused by the presence of an indication for the exposure that is the true cause of the outcome.
from https://catalogofbias.org/biases/confounding-by-indication/

# concept[].concept[].concept[].property[].valueString
A Confounding Covariate Bias is defined as a situation in which the effect or association between an exposure or outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

The term 'indication' is derived from the medical community, in which the reason that an intervention is provided is called the indication. A reason for not providing an intervention may be called a 'contraindication' rather than 'indication to not provide'.

For example, people exposed to chemotherapy have higher mortality. This observation can easily be confounded by people exposed to chemotherapy having a higher rate of cancer (as the reason for receiving the chemotherapy).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-08-18 vote 5-0 by Paul Whaley,  Eric Harvey, Mario Tristan, Cauê Monaco, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-05-12 vote 5-0 by Muhammad Afzal, Brian S. Alper, Joanne Dehnbostel, Jesus Lopez-Alcalde, Harold Lehmann 
BUT THEN TERM CHANGED WITH HIERARCHY CHANGE on 2023-06-30
2023-07-14 vote 2-1 by Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann
2023-07-28 vote 3-1 by Paul Whaley, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann
2023-08-04 vote 5-0 by Joanne Dehnbostel, Philippe Rocca-Serra, Eric Harvey, Harold Lehmann, Mario Tristan BUT comment suggests removing parenthetical from definition

# concept[].concept[].concept[].property[].valueString
2023-05-12 comment: For Comment for Application, I thought we usually put the definition of the parent term first, and the comments about this child. So I would arrange the current 1, 2, 3 paragraphs as 2, 1, 3.

And I think what is now the first paragraph should start with, "A confounding different bias..."

2023-07-01 comment: I would add to the definition "or lack of". Thus: "A confounding covariate bias in which the confounder (distorting variable) is the reason for (or for lack of) an intended exposure.

2023-07-14 comment: I think the definition is good but the comment for application should specifically address this term in more detail than providing a definition for "indication". It is a complex concept and I am not sure I understand what is happening with this bias.

2023-07-28 comment: I still feel that an example of how the reason for receiving an exposure can end up being a confounder would be helpful.

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is the reason for not receiving an exposure.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Caue Monaco

# concept[].concept[].concept[].property[].valueString
A Confounding Covariate Bias is defined as a situation in which the effect or association between an exposure or outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

The term 'indication' is derived from the medical community, in which the reason that an intervention is provided is called the indication. A reason for not providing an intervention may be called a 'contraindication' rather than 'indication to not provide'.

For example, people with cancer exposed to surgery for curative resection have lower mortality than other people with cancer. This observation can easily be confounded by people exposed to surgery for curative resection having a lower rate of metastatic cancer (which is a contraindication to such a surgery).

# concept[].concept[].concept[].property[].valueString
2023-07-28 vote 3-1 by Paul Whaley, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann
2023-08-04 vote 5-0 by Joanne Dehnbostel, Philippe Rocca-Serra, Eric Harvey, Harold Lehmann, Mario Tristan BUT comment suggests removing parenthetical from definition

# concept[].concept[].concept[].property[].valueString
2023-07-28 comment: I still feel that an example of how the reason for receiving an exposure can end up being a confounder would be helpful.

# concept[].concept[].concept[].property[].valueString
2023-08-18 vote 5-0 by Paul Whaley,  Eric Harvey, Mario Tristan, Cauê Monaco, Joanne Dehnbostel

# concept[].concept[].concept[].display
time-varying confounding affected by past exposure

# concept[].concept[].concept[].definition
A confounding covariate bias in which the distorting variable is itself influenced by the exposure.

# concept[].concept[].concept[].property[].valueString
Confounding Covariate Bias is defined as a situation in which the effect or association between an exposure and outcome is distorted by another variable. For confounding covariate bias to occur the distorting variable must be (1) associated with the exposure and the outcome, (2) not in the causal pathway between exposure and outcome, and (3) unequally distributed between the groups being compared.

To distinguish "confounding by time of observation" from "time-varying confounding affected by past exposure" consider the following example:

An observational study is comparing patients with asthma taking Superdrug and patients with asthma not taking Superdrug.  The outcome of interest is mortality, both for association with the dose of Superdrug and compared to not receiving Superdrug. For comparison, the control group not receiving Superdrug is measured during a 1-year calendar period.

For the mortality outcome comparing Superdrug vs. no Superdrug, the time of observation for the control group is consistently 1 year but for the Superdrug group the time of observation varies for each patient. This comparison is confounded by the time of observation.

For the mortality outcome comparing high-dose vs. low-dose Superdrug, the confounding variable of asthma exacerbation rate is complicated in several ways. First, the asthma exacerbation rate is associated with the outcome (mortality) independent from the effects of Superdrug. Second, the asthma exacerbation rate may influence the exposure (the dose of Superdrug which is increased if frequent asthma exacerbations) and the exposure (higher dose of Superdrug) may influence the confounder (reducing the asthma exacerbation rate). This comparison of high-dose vs. low-dose Superdrug for effects on mortality is distorted by time-varying confounding affected by past exposure.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Harold Lehmann, Caue Monaco

# concept[].concept[].concept[].property[].valueString
2023-09-01 comment (with No vote): This term seems unnecessary. Describes a bias rarely seen.

# concept[].concept[].concept[].property[].valueString
2023-10-06 vote 5-0 by Jesus Lopez-Alcalde, Eric Harvey, Paul Whaley, Harold Lehmann, Mario Tristan

# concept[].concept[].definition
A bias resulting from differences between the received exposure and the intended exposure.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Philippe Rocca-Serra

# concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].property[].valueString
2021-05-24 vote 5-2 on "A bias resulting from differences between the received exposure and the intended exposure. Such differences could be the administration of additional interventions that are inconsistent with the study protocol, or non-adherence by the interventionalists or study participants to their assigned intervention. " by Harold Lehmann, Eric Harvey, KM Saif-Ur-Rahman, Bhagvan Kommadi, janice tufte, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].property[].valueString
Definition of performance bias should be modified, Performance bias should involve the blinding at participant level and implementer level in definition.I would add that the differences must be present between the study arms In a RCT with an active control (for example drug A vs drug B) both study arms may have had low adherence but if these deviations from the protocol occurred homogeneously accross arms the effect estimate may not be distorted (biased). As a reviewer, I would not penalise this estimate due to high risk of performance bias. So, concerning the definition, I would propose "A bias resulting from differences accross the study arms between the [...]"

# concept[].concept[].property[].valueString
Such differences could be the administration of additional interventions that are inconsistent with the study protocol, or non-adherence by the interventionalists or study participants to their assigned intervention. Such differences may occur based on assignment to intervention or may occur due to adherence to intervention.

# concept[].concept[].concept[].definition
A performance bias due to awareness of the allocated intervention by participants

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/14/2021: Eric Harvey, Eric Au, Bhagvan Kommadi, Ahmad Sofi-Mahmudi, Erfan Shamsoddin, Janice Tufte, Joanne Dehnbostel, Leo Orozco,

# concept[].concept[].concept[].property[].valueString
2021-06-07 vote 7-1 on "Inadequate blinding of participants  = A performance bias due to awareness of the allocated intervention by participants" by KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati, 2021-06-11 vote 9-1 on same

# concept[].concept[].concept[].property[].valueString
Need to distinguish blinding of intervention from blinding of allocation Inadequate blinding of participants does not always imply bias. Besides, it can also imply detection bias in patient reported outcomes

# concept[].concept[].concept[].property[].valueString
Inadequate blinding of participants is applied when there is awareness of assigned intervention AFTER intervention assignment. If there is awareness BEFORE study enrolment and intervention assignment, this would be Inadequate allocation concealment.
The term "Inadequate blinding of participants" is used to denote the TYPE of bias. Separate terms for the RATING of risk of bias are used to report the likelihood of the presence and influence of the type of bias.

# concept[].concept[].concept[].display
inadequate blinding of intervention deliverers

# concept[].concept[].concept[].definition
A performance bias due to awareness of the allocated intervention by individuals providing or delivering the intervention

# concept[].concept[].concept[].designation[].value
inadequate masking of intervention deliverers

# concept[].concept[].concept[].designation[].value
lack of blinding of intervention deliverers

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/14/2021: Eric Harvey, Eric Au, Bhagvan Kommadi, Ahmad Sofi-Mahmudi, Erfan Shamsoddin, Janice Tufte, Joanne Dehnbostel, Leo Orozco,

# concept[].concept[].concept[].property[].valueString
2021-06-07 vote 7-1 on "Inadequate blinding of participants  = A performance bias due to awareness of the allocated intervention by individuals providing or delivering the intervention" by KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati, 2021-06-11 vote 9-1 on same

# concept[].concept[].concept[].property[].valueString
Need to distinguish blinding of intervention from blinding of allocation; Should we use the term interventionalist or interventionist? Inadequate blinding of intervention deliverers does not always imply Performance bias

# concept[].concept[].concept[].property[].valueString
Inadequate blinding of intervention deliverers is applied when there is awareness of assigned intervention AFTER intervention assignment. If there is awareness BEFORE study enrolment and intervention assignment, this would be Inadequate allocation concealment.
The term noted here is used to denote the TYPE of bias. Separate terms for the RATING of risk of bias are used to report the likelihood of the presence and influence of the type of bias.

# concept[].concept[].concept[].display
deviation from study intervention protocol

# concept[].concept[].concept[].definition
A performance bias in which the intervention received differs from the intervention specified in the study protocol

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].concept[].definition
A performance bias in which the intervention or exposure received differs from the from the usual practice or expected care

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].concept[].definition
A performance bias in which the intervention deliverers do not completely adhere to the expected intervention

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].concept[].definition
A performance bias in which the participants do not completely adhere to the expected intervention or exposure

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].concept[].display
imbalance in deviations from intended intervention

# concept[].concept[].concept[].definition
A performance bias in which the degree of performance bias is unequally distributed between groups being compared

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Erfan Shamsoddin, Bhagvan Kommadi

# concept[].concept[].concept[].property[].valueString
8/8 as of 6/7/2021: KM Saif-Ur-Rahman, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].definition
A bias due to absence of expected participation or data collection after selection for study inclusion.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Philippe Rocca-Serra, Mario Tristan, Janice Tufte, Erfan Shamsoddin

# concept[].concept[].property[].valueString
13/13 as of 6/18/2021: Eric Au, Harold Lehmann, Erfan Shamsoddin, Ahmad Sofi-Mahmudi, Mario Tristan, Eric Harvey, Sebastien Bailly, Bhagvan Kommadi, Leo Orozco, Alejandro Piscoya, Jesús López-Alcalde, Janice Tufte, Paola Rosati

# concept[].concept[].property[].valueString
6/14/2021-06-14 vote 7-1 on "Attrition Bias = A bias due to absence of expected participation or data collection after study enrollment." by, Eric Harvey, Eric Au, Bhagvan Kommadi, Ahmad Sofi-Mahmudi, Erfan Shamsoddin, Janice Tufte, Joanne Dehnbostel, Leo Orozco,

# concept[].concept[].property[].valueString
The phrase "after study enrolment" might be confusing. Does enrolment apply to retrospective observational studies?

# concept[].concept[].concept[].display
attrition bias due to participant attrition

# concept[].concept[].concept[].definition
A bias due to absence of expected participation due to participant dropout, withdrawal or non-participation after selection for study inclusion.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-04-28 comment: I would not detail that 20%: it is misleading and not evidence-based

# concept[].concept[].concept[].property[].valueString
2023-05-12 vote 6-0 by Muhammad Afzal, Brian S. Alper, Joanne Dehnbostel , Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A bias due to data loss or absence of data collection from participants after selection for study inclusion.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-04-28 comment: I would not detail that 20%: it is misleading and not evidence-based

# concept[].concept[].concept[].property[].valueString
2023-05-12 vote 6-0 by Muhammad Afzal, Brian S. Alper, Joanne Dehnbostel , Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].display
attrition bias due to missing outcome data

# concept[].concept[].concept[].concept[].definition
An attrition bias due to missing data specific to the dependent variable.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-19 5-1 Muhammad Afzal, Janice Tufte, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann, Joanne Dehnbostel
2023-06-09 vote 3-1 by Eric Harvey, Cauê Monaco, Paul Whaley, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
The information in the parentheses, "(or data on an independent variable)," is unclear in its intended meaning. To improve clarity, we could revise the definition.
2023-06-09 comment: The definition is too difficult to parse, and probably too similar to the preferred term. The comment for application is also very difficult to read.

# concept[].concept[].concept[].concept[].property[].valueString
In a situation of repeated measures outcomes, attrition bias due to missing outcome data can occur if one or more measurements are missing.

# concept[].concept[].concept[].concept[].property[].valueString
2023-06-16 vote 5-0 by Joanne Dehnbostel, Eric Harvey, Paul Whaley, Sunu Alice Cherian, Harold Lehmann

# concept[].concept[].concept[].concept[].display
attrition bias due to missing exposure data

# concept[].concept[].concept[].concept[].definition
An attrition bias due to missing data specific to the independent variable(s) of primary interest, such as exposure or intervention.

# concept[].concept[].concept[].concept[].designation[].value
attrition bias due to missing intervention data

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin, Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-19 5-1 Muhammad Afzal, Janice Tufte, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
The information in the parentheses, "(or data on an independent variable)," is unclear in its intended meaning. To improve clarity, we could revise the definition.
2023-06-09 comment: This needs a comment for application, but the definition is clearer than for "attrition bias due to missing outcome data".

# concept[].concept[].concept[].concept[].property[].valueString
2023-06-09 vote 6-0 by Cauê Monaco, Eric Harvey, Paul Whaley, Harold Lehmann, Jesus Lopez-Alcalde, Sunu Alice Cherian

# concept[].concept[].concept[].concept[].property[].valueString
If coding a bias related to the classification of exposure, misclassification of exposure may be coded as Exposure Detection Bias, but if the data is excluded from analysis it may then be coded as Attrition bias due to missing exposure data.

# concept[].concept[].concept[].concept[].display
attrition bias due to missing modifier data

# concept[].concept[].concept[].concept[].definition
An attrition bias due to missing data specific to a confounder or effect modifier

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
The term modifier is intended to be broad, including variables used for modeling interactions, stratification factors to account for effect modification, or other variables such as mediators that need to be accounted for when modeling the relationship between the outcome and exposure.

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-12 vote 5-0 by Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].display
attrition bias due to missing data about attrition

# concept[].concept[].concept[].concept[].definition
An attrition bias due to missing data specific to the extent of or reasons for missing data.

# concept[].concept[].concept[].concept[].property[].valueString
Attrition bias due to missing data is defined as a bias due to data loss or absence of data collection from participants after selection for study inclusion. Data about the amount of missing data and data about the reasons for missing data are types of data that can also be missing. For example, in a time-to-event study, the reason a participant is censored might be missing and missing such data may interfere with distinguishing informative from non-informative censoring.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-06-16 vote 5-0 by Joanne Dehnbostel, Eric Harvey, Paul Whaley, Sunu Alice Cherian, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2023-06-16 comments: Time-to-event should be hyphenated
Funnily enough, this came up straight after our call in relation to another bias project I am working on, so I would consider this addition useful!

# concept[].concept[].concept[].definition
An attrition bias in which the degree of missing data is unequally distributed between groups being compared.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-05-12 vote 5-0 by Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
An attrition bias in which the reason for absence of data collection is a low response rate to data collection surveys.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-05-12 vote 5-0 by Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].definition
A bias due to distortions in any process involved in the determination of the recorded values for a variable.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Philippe Rocca-Serra, Mario Tristan, Harold Lehmann, Janice Tufte, Muhammad Afzal; Paul Whaley

# concept[].concept[].property[].valueString
6/14/2021-06-14 vote 7-1 on "Detection Bias = A bias due to distortions in how variable values (data) are determined (measured, classified or ascertained)." by, Eric Harvey, Eric Au, Bhagvan Kommadi, Ahmad Sofi-Mahmudi, Erfan Shamsoddin, Janice Tufte, Joanne Dehnbostel, Leo Orozco

AGREEMENT REACHED 10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper --- for DEFINITION OF: A bias due to distortions in how variable values (data) are determined.  COMMENT FOR APPLICATION: Determination may include ascertainment or assessment (classification or measurement).

2022-10-14 vote 3-1 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paul Harris

2022-01-21 vote 6-1 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paul Harris, Paul Whaley, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].property[].valueString
We need to state that this bias relates to the "outcome" -- The ROB-1 says the term "outcome assessment" as an alternative for detection bias. The ROBINS-1 says that "Non-differential misclassification is unrelated to the outcome and will usually bias the estimated effect of intervention towards the null". Still though, this leads to inadvertent deviations in the outcome assessment. I would suggest to at least state that this bias relates to outcome assessment. I remember Joanne saying that we will add a few "child concepts" later on and if that is the case here, then it is fine. Nevertheless, the RoB2 suggests not to use these terms to prevent "confusion" and does not actually agree with these sub-classifications (the fist page of the introduction section). Alternative terms according to (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5130591/): "Observer Bias", "Ascertainment Bias", or "Assessment Bias"  8/27/21 comment: Detection bias is no included in the list the more problematic Cochrane ROB1 however (Jørgensen et al. Systematic Reviews (2016) describe alll the domains of ROB1 "frequently implemented in a non-recommended way" The description in general is clear.

2022-10-14 comments: Do we need "Outcome Detection Bias" in addition to "Detection Bias"?
Blinding or masking may be used to reduce the risk of distorted outcome measurement(s).

2022-01-21 comment: I am not sure whether to vote yes or no: I understand the definition because I have been following our discussions and it is consistent with the bias model we have developed, but I worry that this definition may not be consistently understood or applied by a user of SEVCO - I feel there is too much unspoken metaphysical baggage that is coherent and correct but not useful.

2022-01-28 comment: Not perfect but good enough to live with. Could maybe improve on ascertainment component of the comment for application.

# concept[].concept[].property[].valueString
Detection of the value of the variable comprises three processes involved in the determination of the recorded values for the variable: ascertainment (providing the opportunity for assessment), assessment (measurement and/or classification), and documentation (recording of data values for analysis).

# concept[].concept[].property[].valueString
2022-01-28 vote 9-0 by Mario Tristan, Janice Tufte, Robin Ann Yurk, Brian S. Alper, C P Ooi, Harold Lehmann,  Paola Rosati, Jesus Lopez-Alcalde, Paul Whaley

# concept[].concept[].concept[].definition
A detection bias due to distortions in how an outcome is determined.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Philippe Rocca-Serra, Mario Tristan, Janice Tufte, Harold Lehmann, Erfan Shamsoddin, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].concept[].display
cognitive interpretive bias for outcome determination

# concept[].concept[].concept[].concept[].definition
An outcome detection bias due to the subjective nature of human interpretation.

# concept[].concept[].concept[].concept[].designation[].value
perception bias for outcome determination

# concept[].concept[].concept[].concept[].designation[].value
subjective interpretive bias for outcome determination

# concept[].concept[].concept[].concept[].designation[].value
subjectivity bias for outcome determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper, 7/7 for Alternative terms on 9/24/21: Janice/Brian/Eric/Paola/Jesus/Bhagvan/Mario

# concept[].concept[].concept[].concept[].property[].valueString
8/27/21 comment: This bias is difficult to manage and avoid it.

# concept[].concept[].concept[].concept[].concept[].display
bias due to lack of masking for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to awareness of the participant's status with respect to the exposure of interest.

# concept[].concept[].concept[].concept[].concept[].designation[].value
bias due to lack of blinding during outcome assessment

# concept[].concept[].concept[].concept[].concept[].designation[].value
bias due to lack of masking during outcome assessment

# concept[].concept[].concept[].concept[].concept[].designation[].value
awareness bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Paul Whaley, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
8/27/2021 vote 9-1 on "Lack of blinding during outcome assessment = A cognitive interpretive bias for outcome determination due to the outcome assessor’s awareness of the participant's status with respect to the exposure of interest." by, Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper
earlier term approved 5/5 as of 8/30/2021: Eric Harvey, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-08-27 comment: In my opinion "lack of blinding during outcome assessment" does not always imply bias for outcome determination (for example, for hard outcomes, such as analytic parameters, or all-cause mortality)
2022-03-18 comment: I would consider editing the term definition to ...lack of blinding.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Lack of blinding or masking is not automatically a bias, but if awareness of exposure status systematically distorts the outcome determination then a 'Bias due to lack of masking for outcome determination' exists.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-18 vote 5-0 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Nisha Mathew, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to subjective interpretations in the process of observing and recording information.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 8/30/2021: Eric Harvey, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].property[].valueString
8/27/2021 vote 9-1 on "Observer bias for outcome determination = A cognitive interpretive bias for outcome determination due to subjective interpretations in the process of observing and recording information." by, Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-08-27 comment: This situation seems to be covered by "Lack of blinding for outcome determination" and "Outcome ascertainment bias". I would suggest deleting this term to remove the overlap.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Multiple types of bias can overlap. Observer bias is different than lack of blinding with respect to the exposure. Observer bias is about the influence of the observer's interpretation of what they are observing, whether or not the observer is aware of the participant's exposure.

# concept[].concept[].concept[].concept[].concept[].concept[].display
confirmation bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An observer bias for outcome determination due to previous opinions or knowledge of a subject’s prior exposures or assessments.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
diagnostic suspicion bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
previous opinion bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 9/17/2021: Eric Harvey, Paola Rosati, Alejandro Piscoya, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to differences in accuracy or completeness of recall of past events or experiences.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].display
apprehension bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to a study participant's responding or behaving differently when aware of being observed.

# concept[].concept[].concept[].concept[].concept[].designation[].value
hawthorne effect for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 8/30/2021: Eric Harvey, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].property[].valueString
8/27/2021 vote 8-2 on "Apprehension bias for outcome determination = A cognitive interpretive bias for outcome determination due to study participants’ awareness of being observed resulting in different responses or behaviors." by, Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-08-27 comments: A cognitive interpretive bias for outcome determination due to study participants’ awareness of being observed and resulting in different responses or behaviors. (just a slight rewording - the existing wording doesn't read well to me) This definition seems to refer to performance bias. The key is that [...] results in different responses or behaviours concerning the outcome determination.

# concept[].concept[].concept[].concept[].concept[].display
hypothetical assessment bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to a difference between an individual’s report of an imagined or hypothetical response from their actual response. The response may be a behavior or valuation.

# concept[].concept[].concept[].concept[].concept[].designation[].value
subjunctivity bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 9/24/21: , Janice Tufte, Brian S. Alper, Eric Harvey, Paola Rosati, Jesus Lopez-Alcalde, Bhagvan Kommadi, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to a misinterpretation of observations that resemble the outcome.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 9/24/21: , Janice Tufte, Brian S. Alper, Eric Harvey, Paola Rosati, Jesus Lopez-Alcalde, Bhagvan Kommadi, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].display
unacceptability bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for outcome determination due to distortions in response, response values, or recording of responses resulting from perception of the social unacceptability of an outcome.

# concept[].concept[].concept[].concept[].concept[].designation[].value
unacceptable disease bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Ken Wilkins, Lisa Schilling

# concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 10/1/21: , Joanne Dehnbostel, Brian S. Alper, Eric Harvey, Alejandro Piscoya, Bhagvan Kommadi,

# concept[].concept[].concept[].concept[].definition
An outcome detection bias due to distortions in how the data are collected.

# concept[].concept[].concept[].concept[].designation[].value
ascertainment bias for outcome determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].display
nonrepresentative observation period for outcome of interest

# concept[].concept[].concept[].concept[].concept[].definition
An outcome ascertainment bias due to differences in the period used for observation of the outcome and the period for the outcome of interest.

# concept[].concept[].concept[].concept[].concept[].designation[].value
misaligned follow up period for outcome of interest

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 5-2 on "Inappropriate follow up period for outcome of interest = An outcome ascertainment bias due to differences in the time period used for observation of the outcome and the true time period for outcome occurrence." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Misaligned follow up period for outcome of interest = An outcome ascertainment bias due to differences in the time period used for observation of the outcome and the true time period for outcome occurrence." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I wonder if we need to address interim analysis validity. What about adding to inappropriate 'unreliable'?; Change word Inappropriate to Different
2021-10-25 comments: It is unclear what do you mean with 'and the true time period for outcome occurrence', On the other hand, I propose using 'period' instead of 'time period'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/29/2021 vote 6-0 by Cheow Peng Ooi, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].definition
An outcome ascertainment bias due to differences in the context in which the outcome is observed and the intended context for the outcome of interest.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Unreliable method for outcome ascertainment = An outcome ascertainment bias due to methods of data collection that result in inconsistent data values." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Undependable method for outcome of interest = An outcome ascertainment bias due methods of data collection that result in inconsistent or incorrect data values." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey
10/29/2021 vote 5-1 by Cheow Peng Ooi, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, Eric Harvey, Janice Tufte
11/22/2021 vote 6-1
2021-12-03 vote for priort term 7-0 by Philippe Rocca-Serra, Janice Tufte, Mario Tristan, Harold Lehmann, Paul Whaley, Joanne Dehnbostel, C  Ooi

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: the word Unreliable is misleading as more applicable to measurement error than bias
2021-10-25 comments: I do not fully understand the difference between the second and the third definitions
2021-10-29 comments: Suggest Incorrect or inconsistent method.
2021-11-22 comments: The term 'inconsistent' may be more appropriate  -- steering group discussion to move the "Comment for application" property higher on the page and see if this comment will resolve the concern

# concept[].concept[].concept[].concept[].concept[].property[].valueString
This term is used when the context used for outcome ascertainment is incorrect, insensitive, or nonspecific.  If the context (whether representative or not) is applied inconsistently, then use the term "Inconsistency in outcome ascertainment"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-18 vote 5-0 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Nisha Mathew, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].definition
An outcome ascertainment bias due to differences within or between groups in how the data are collected.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Imbalance in application of outcome ascertainment = An outcome ascertainment bias due to differences within or between groups in how the data are collected." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Imbalance in application of outcome ascertainment = An outcome ascertainment bias due to differences within or between groups in how the data are collected." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey
10/29/2021 vote on prior term 6-0 by Cheow Peng Ooi, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: Imbalance is misleading as more applicable to measurement error?
2021-10-25 comment: Suggestion, replace imbalance with Variation or Heterogeneity
2021-10-29 comment: Alternative Terms:  Variation or Heterogeneity --> converted 2021-10-29 to suggested addition of Alternative term "Variation in application of outcome ascertainment" by Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Harold Lehmann, Mario Tristan, Bhagvan Kommadi

# concept[].concept[].concept[].concept[].concept[].property[].valueString
This term is used when the context (whether representative or not) is applied inconsistently. If the context used for outcome ascertainment is incorrect, insensitive, or nonspecific, then use the term "Nonrepresentative context for outcome ascertainment"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-18 vote 5-0 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Nisha Mathew, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
An outcome detection bias due to distortions in how the observed outcomes are measured.

# concept[].concept[].concept[].concept[].designation[].value
measurement bias for outcome determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal; Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
PRIOR AGREEMENT 10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper WITH DEFINTION: An outcome detection bias due to distortions in how the data are measured.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-11 comment: Outcome Measurement Bias has a similar term definition as Outcome Classification Bias.  May need to add an additional comment for application from T&O discussion.

# concept[].concept[].concept[].concept[].property[].valueString
If one is addressing a bias in the instruments or processes used to measure the observed outcome, use Outcome Measurement Bias. If one is addressing how the measured outcome is categorized, use Outcome Classification Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paul Harris, Paul Whaley, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].display
inappropriate method for outcome measurement

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to use of an incorrect method or protocol.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Outcome measurement method inappropriate = An outcome measurement bias due to use of an incorrect method or protocol." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 2-2 on "Outcome measurement method inappropriate = An outcome measurement bias due to use of an incorrect method or protocol." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would change word inappropriate to different as the bias is from difference in comparison not flaws or errors in scientific methods.
2021-10-25 comments: suggest replace with incorrect method; Should not be 'Inappropriate outcome measurement method' (instead of placing the adjective at the end?)
2022-03-11 Preferred term revised (and Alternative term added) to match corresponding changes in Exposure Detection Bias)

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7 of 7 on 2021-11-05: Bhagvan Kommadi, Paola Rosati, Paul Whaley, Janice Tufte, Alejandro Piscoya, Robin Ann Yurk, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
insensitive measure bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to use of a method that does not reliably detect the outcome when the outcome is present.

# concept[].concept[].concept[].concept[].concept[].designation[].value
inadequate sensitivity for outcome measure

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Use of an inadequately sensitive outcome measure is likely to result in false negative findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Insensitive measure bias for outcome determination = An outcome measurement bias due to use of a method that does not reliably detect the outcome when the outcome is present." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Insensitive measure bias for outcome determination =An outcome measurement bias due to use of a method that does not reliably detect the outcome when the outcome is present." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: Change word Insensitive to Sensitivity measure bias as double negative in phrase
2021-10-25 comment: False Negative measure Bias or Unreliable measure bias

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7 of 7 on 2021-11-05: Bhagvan Kommadi, Paola Rosati, Paul Whaley, Janice Tufte, Alejandro Piscoya, Robin Ann Yurk, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
nonspecific measure bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to use of a method that falsely detects the outcome when the outcome is absent.

# concept[].concept[].concept[].concept[].concept[].designation[].value
inadequate specificity for outcome measure

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Use of an inadequately specific outcome measure is likely to result in false positive findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Nonspecific measure bias for outcome determination = An outcome measurement bias due to use of a method that falsely detects the outcome when the outcome is absent." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Nonspecific measure bias for outcome determination = An outcome measurement bias due to use of a method that falsely detects the outcome when the outcome is absent." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would change to Specificity measurement bias. Remove word falsely from the definition as it implies problems with scientific methods
2021-10-25 comment: Suggest use False Positive Measure Biac

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7 of 7 on 2021-11-05: Bhagvan Kommadi, Paola Rosati, Paul Whaley, Janice Tufte, Alejandro Piscoya, Robin Ann Yurk, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
inappropriate application of method for outcome measurement

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to inappropriate application of the method or protocol.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Outcome measurement conduct inappropriate = An outcome measurement bias due to incorrect application of the method or protocol." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 2-2 on "Outcome measurement conduct inappropriate = An outcome measurement bias due to incorrect application of the method or protocol." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey
2021-11-05 vote 6-1 on "Inappropriate outcome measurement conduct = An outcome measurement bias due to incorrect application of the method or protocol." by Bhagvan Kommadi, Paola Rosati, Paul Whaley, Janice Tufte, Alejandro Piscoya, Robin Ann Yurk, Eric Harvey
2022-03-18 vote 4-1 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Nisha Mathew, Brian S. Alper
2022-03-25 vote 7-1 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Philippe Rocca-Serra, Brian S. Alper, Jesus Lopez-Alcalde, Paul Whaley, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would eliminate this definition - as suggests flawed study design..
2021-10-25 comments: replace inappropriate with incorrect; Should not be 'Inappropriate outcome measurement conduct' (instead of placing the adjective at the end?)
2021-11-05 comment: There is enormous overlap with this term and "Inappropriate outcome measurement method", so this one should be eliminated
As of 2021-11-05 this term is not being prepared for vote. The current ROB tools do not distinguish the inappropriate conduct (used in QUADAS-2) from inadequate method (used in most other ROB tools) in the same tool, so the demand for this term is uncertain and thus not applied for version 1 of the Code System.
On 2022-03-11 we revised this term to match corresponding changes that passed for Exposure Detection Bias.
2022-03-18 comment: Suggest edit Alternative term from conduct to process
2022-03-25 comment: Recommend edit term definition so it reads:  Outcome Measurement method Bias.
Suggest reviewing your complete taxonomy of terms and identify similarities or duplicate terms and potentially integrating terms by keeping as primary term versus adding to alternate term for prior vote with similar term definition or statements.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An inappropriate application of the method or protocol suggests error is introduced by the process of measurement, as distinct from the method or protocol used for measurement (which would be an Inappropriate method for outcome measurement).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-04-08 vote 11-1 (no rationale provided for the negative vote) by Muhammad Afzal, Paul Whaley, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, nelle.stocquart, nisha mathew, Harold Lehmann, Cauê Monaco

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences within groups in how the observed outcomes are measured.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley, Robin Ann Yurk, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Imbalance in application of outcome measurement = An outcome measurement bias due to differences within or between groups in how the data are measured." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Imbalance in application of outcome measurement = An outcome measurement bias due to differences within or between groups in how the data are measured." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey
2021-11-05 vote 6-1 on "Inconsistency in application of outcome measurement = An outcome measurement bias due to differences within or between groups in how the data are measured." by Bhagvan Kommadi, Paola Rosati, Paul Whaley, Janice Tufte, Alejandro Piscoya, Robin Ann Yurk, Eric Harvey
2021-11-22 vote 3-2 on "Inconsistency in application of outcome measurement" = "An outcome measurement bias due to differences within or between groups in how the data are measured."
2021-12-10 vote 5-1 by Joanne Dehnbostel, Janice Tufte, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, Paul Whaley
PRIOR AGREEMENT 2021-12-17 vote 6-0 by Robin Ann Yurk, Janice Tufte, Paul Whaley, Mario Tristan, C P Ooi, Jesus Lopez-Alcalde FOR DEFINITION: An outcome measurement bias due to differences within groups in how the data are measured. AND COMMENT FOR APPLICATION: "How the data are measured" may refer to the methods applied for data measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would eliminate this definition
2021-10-25 comment: Replace Imbalance with Heterogeneity
2021-11-05 comment:  This is a specific type of "Inappropriate outcome measurement method" so this term should be moved into that position or eliminated (are we really going to describe all of the inappropriate methods?) [[discussed in COKA WG and noted that ROB2 has separate questions 4.1 and 4.2 for these terms so we need to support that]
2021-11-22 comments: "The wording 'inconsistent method of outcome measurement' may better reflect the definition" and "May be pedantic, but is it data that are measured, or the outcome as a variable (that results in data)? I also wonder if we mean differences within groups - some variation would be expected, but what matters is if the variation results in systematic error in measuring the variable between groups. If we feel that e.g. a study design where two different ways of measuring outcome were implemented within groups, but this did not lead to bias across the exposure and control arms, then I would vote yes (pending clarification of "data")."
2021-12-10 comment: It seems to not quite be correctly written. The two choices for definition are differently phrased ("application of methods" / "methods applied")even though I think they are supposed to refer to across groups or within groups, but both refer to within groups, so I am not sure how to interpret this.
2022-01-21 comment: As a comment: Is this term redundant, if the two child terms are the complete set of options for inconsistency in outcome measurement?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
"How the observed outcomes are measured" may refer to the methods applied for measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 6-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paul Whaley, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].concept[].display
inconsistency in instruments used for outcome measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences within groups in the instruments used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati, Jesus Lopez-Alcalde
2022-01-07 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, Paola Rosati, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 comment: To me there is no semantic difference between this definition and the other subordinate term for inconsistency in outcome measurement (application of methods ? methods applied)
2022-01-07 comment: I would approve this, except I am still not sure that one can measure data ("facts and statistics collected together for reference or analysis"). One can collect data, or measure a variable, but I don't think one can collect data.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Instruments used for measurement may include devices, surveys, and technologies. The concepts of "instruments used for measurement" is distinct from "process used for measurement" which may include protocols, techniques, and variations in context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Andrew Beck, Harold Lehmann, Robin Ann Yurk, Paul Whaley, Janice Tufte, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].concept[].display
inconsistency in processes used for outcome measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences within groups in the processes by which the instruments are used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati, Jesus Lopez-Alcalde
2022-01-07 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, Paola Rosati, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 comment: To me there is no semantic difference between this definition and the other subordinate term for inconsistency in outcome measurement (application of methods ? methods applied)
2022-01-07 comment: I would approve this, except I am still not sure that one can measure data ("facts and statistics collected together for reference or analysis"). One can collect data, or measure a variable, but I don't think one can collect data.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The processes used for measurement may include protocols, techniques, and variations in context. The concept of "processes used for measurement" is distinct from "instruments used for measurement" which may include devices, surveys, and technologies.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Andrew Beck, Harold Lehmann, Robin Ann Yurk, Paul Whaley, Janice Tufte, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences between groups in how the observed outcomes are measured.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Robin Ann Yurk, Janice Tufte, Harold Lehmann, Mario Tristan, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
"How the observed outcomes are measured" may refer to the methods applied for measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-10 vote 5-0 by Janice Tufte, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, Paul Whaley but steering group decided to make changes consistent with changes to Inconsistency in outcome measurement.
PRIOR AGREEMENT 2021-12-17 vote 5-0 by Robin Ann Yurk, Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati FOR DEFINITION: An outcome measurement bias due to differences between groups in how the data are measured. WITH COMMENT FOR APPLICATION: "How the data are measured" may refer to the methods applied for data measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-10 comment: Referring back to my comment on the inconsistency in method, I realise I hadn't read it quite right. In both cases, they maybe aren't quite as easy to parse as would be ideal but I can't think of a better definition. Maybe a use note to refer to how the terms are similar and clarify when one vs. the other should be used?
2022-01-21 comments: The term definition and comment is the same for Inconsistency in outcome measurement bias.  Suggest combining the two terms by listing one as an Alternative term. (yellow highlighting in messaging applied to show the differences in the terms)
As a comment: Is this term redundant, if the two child terms are the complete set of options for inconsistency in outcome measurement?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 6-0 by Harold Lehmann, Robin Ann Yurk, Paul Whaley, Janice Tufte, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].concept[].display
imbalance in instruments used for outcome measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences between groups in the instruments used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 vote 4-1 by Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati, Jesus Lopez-Alcalde
2022-01-07 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, Paola Rosati, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 comment: To me there is no semantic difference between this definition and the other subordinate term for inconsistency in outcome measurement (application of methods ? methods applied)
2022-01-07 comment: I would approve this, except I am still not sure that one can measure data ("facts and statistics collected together for reference or analysis"). One can collect data, or measure a variable, but I don't think one can collect data.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Instruments used for measurement may include devices, surveys, and technologies. The concepts of "instruments used for measurement" is distinct from "process used for measurement" which may include protocols, techniques, and variations in context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Paul Whaley, Janice Tufte, Alejandro Piscoya, Philippe Rocca-Serra, Andrew Beck

# concept[].concept[].concept[].concept[].concept[].concept[].display
imbalance in processes used for outcome measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome measurement bias due to differences between groups in the processes by which the instruments are used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 vote 4-1 by Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati, Robin Ann Yurk
2022-01-07 vote 5-1 by Robin Ann Yurk, Janice Tufte, Paul Whaley, Paola Rosati, Harold Lehmann, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-17 comment: To me there is no semantic difference between this definition and the other subordinate term for inconsistency in outcome measurement (application of methods ? methods applied)
2022-01-07 comment: I would approve this, except I am still not sure that one can measure data ("facts and statistics collected together for reference or analysis"). One can collect data, or measure a variable, but I don't think one can collect data.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The processes used for measurement may include protocols, techniques, and variations in context. The concept of "processes used for measurement" is distinct from "instruments used for measurement" which may include devices, surveys, and technologies.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Paul Whaley, Janice Tufte, Alejandro Piscoya, Philippe Rocca-Serra, Andrew Beck

# concept[].concept[].concept[].concept[].definition
An outcome detection bias due to distortions in how the observed outcomes are classified.

# concept[].concept[].concept[].concept[].designation[].value
misclassification bias for outcome determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].property[].valueString
PRIOR AGREEMENT 10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper, 7/7 for renaming on 9/24/21: Janice/Brian/Eric/Paola/Jesus/Bhagvan/Mario  FOR DEFINITION: An outcome detection bias due to distortions in how the data are classified.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-11 comment: Outcome Classification Bias has a similar term definition as Outcome Measurement Bias.  May need to add an additional comment for application from T&O discussion.

# concept[].concept[].concept[].concept[].property[].valueString
If one is addressing a bias in the instruments or processes used to measure the observed outcome, use Outcome Measurement Bias. If one is addressing how the measured outcome is categorized, use Outcome Classification Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-21 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, Paul Harris, Paul Whaley, Alejandro Piscoya, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].definition
An outcome classification bias resulting from the definition or threshold used for outcome classification.

# concept[].concept[].concept[].concept[].concept[].designation[].value
definition bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 9/17/2021: Eric Harvey, Paola Rosati, Alejandro Piscoya, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An outcome classification system bias suggests an internal validity problem in which  the definition or threshold used for outcome classification does not represent the outcome of interest. If considering an external validity problem, the "Wrong question bias" (term not yet defined) may be used. An outcome classification system bias is present when there are differences between the outcome of interest and the definition or threshold used for outcome classification.

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome classification system bias due to a mismatch between the outcome of interest and the definition or threshold used for outcome measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Nonrepresentative definition for outcome classification = An outcome classification system bias due to a definition or threshold that does not represent the outcome of interest." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
2021-11-29 vote 6-1 on "Nonrepresentative definition for outcome classification" = "An outcome classification system bias due to a definition or threshold that does not represent the outcome of interest." by Harold Lehmann, Paul Whaley, Janice Tufte, C P Ooi, Joanne Dehnbostel, Philippe Rocca-Serra, Robin Ann Yurk

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would eliminate this definition
2021-11-29 comments: ("represent in its entirety" instead? A definition could *partially* represent the outcome of interest, so perhaps we want to make clear that this bias is invoked only for something that is more than "partial"?) "Represent" feels ambiguous, would it be useful to clarify what is meant here? Is it that it includes outcomes in addition to that of interest, and/or excludes outcomes that are of interest? Maybe that doesn't make things clearer.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-10 vote 5-0 by Paul Whaley, Janice Tufte, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].display
surrogate marker bias for outcome classification

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Surrogate marker bias for outcome classification = A nonrepresentative definition for outcome classification due to use of a factor associated with the outcome rather than a direct observation of the outcome." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
2021-11-29 vote 3-3 on "Surrogate marker bias for outcome classification" = "A nonrepresentative definition for outcome classification due to use of a proxy for the outcome rather than a direct observation of the outcome."

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would edit the definition: An outcome classification system bias due to use of a definition that is proxy rather than direct observation of the outcome. {{Definition changed as result of this comment}}
2021-11-29 comments: The 10/15 comments stated that the definition should start with "An outcome classification system bias...."; but this definition does not. A little pickier, I might say, "result from use of a definition" rather than "due to". The latter sounds like the bias will always occur; the former, that there is a bias as a result, in this instance.
I'm not sure I fully understand this definition. A surrogate would generally be used in place of an outcome that cannot readily be observed in a research setting. I am not sure how this can be a classification error (the surrogate is what the surrogate is). I can, however, see how it could be an error in inference (assuming that because the exposure affects the surrogate, then the exposure also affects the outcome of actual interest). Is this a helpful way of thinking about this, or would it just be over-complicating matters?
This suggested definition is more appropriate: An outcome classification system bias due to use of a definition that is proxy rather than direct observation of the outcome

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-12-10 vote 5-0 by Paul Whaley, Janice Tufte, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome classification system bias due to use of a definition that is proxy for the outcome rather than direct observation of the outcome.

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
substitution bias for outcome classification

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An outcome classification system bias due to defining the outcome after interacting with the study data.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
definition for outcome classification not prespecified

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
no prespecified definition for outcome classification

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
10/15/2021 vote 6-1 on "Definition not prespecified for outcome classification = An outcome classification system bias due to absence of a predetermined definition." by Eric Harvey, Paola Rosati, Alejandro Piscoya, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte, Brian S. Alper
10/25/21 vote 3-1 on "Definition not prespecified for outcome classification = An outcome classification system bias due to absence of a predetermined definition." by Paola Rosati, Robin Ann Yurk, Jesus Lopez-Alcalde, Eric Harvey
2021-12-03 vote 5-2 by Harold Lehmann, Paul Whaley, Janice Tufte, C P Ooi, Joanne Dehnbostel, Philippe Rocca-Serra, Robin Ann Yurk
2021-12-10 vote 2-2 by Paul Whaley, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2021-10-15 comments: I would eliminate this definition as suggest flawed study design
2021-10-25 comment: I would phrase 'Not prespecified definition for outcome classification'
2021-12-03 comments: It feels uninformative to define "not prespecified" as "not predetermined". I wonder if "predetermined" can be clarified - presumably, the issue here is that the outcome is defined post-hoc, after data collection, so that outcome ends up being defined around the data rather than specified in advance of conduct of the research.  //   Rephrasing to this 'No prespecified definition for outcome classification' may be clearer and easier to understand. 
2021-12-10 comments: Consider removing term.  As methods are permitted to be revised for a variety of reasons with new definitions but would be described in methods or a revised protocol.  If truly post-hoc after a data set is closed then there are different issues for discussion.  ///   Suggest changing "due to determination of the outcome definition" to "due to outcome being defined"

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-01-07 vote 9-0 by Robin Ann Yurk, Janice Tufte, Paul Whaley, C P Ooi, Paola Rosati, Jesus Lopez-Alcalde, Harold Lehmann, Joanne Dehnbostel, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].definition
An outcome classification bias resulting from the application of the method used for outcome classification.

# concept[].concept[].concept[].concept[].concept[].designation[].value
classification process bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
7/7 as of 9/24/21: , Janice Tufte, Brian S. Alper, Eric Harvey, Paola Rosati, Jesus Lopez-Alcalde, Bhagvan Kommadi, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
4-1 vote as of 9/17/2021 regarding Outcome Classification Process Bias (SEVCO:00062) (Classification process bias for outcome determination) [Draft Term] = An outcome misclassification bias resulting from the application of the method used for outcome classification.: Eric Harvey, Paola Rosati, Alejandro Piscoya, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].property[].valueString
comment: "This might be related to outcome classification bias (child relationship)"

# concept[].concept[].concept[].concept[].concept[].display
incorporation bias for outcome determination

# concept[].concept[].concept[].concept[].concept[].definition
An outcome classification bias due to the inclusion of the exposure under investigation in the method or process used for outcome classification.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
5/5 as of 9/17/2021: Eric Harvey, Paola Rosati, Alejandro Piscoya, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].concept[].concept[].property[].valueString
In predictive model research, incorporation bias for outcome determination occurs if the predictor (explanatory variable) is included in the outcome definition.

# concept[].concept[].concept[].definition
A detection bias due to distortions in how an exposure of interest is determined.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Philippe Rocca-Serra, Mario Tristan, Janice Tufte, Harold Lehmann, Erfan Shamsoddin, Muhammad Afzal, Kenneth Wilkin

# concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].display
cognitive interpretive bias for exposure determination

# concept[].concept[].concept[].concept[].definition
An exposure detection bias due to the subjective nature of human interpretation.

# concept[].concept[].concept[].concept[].designation[].value
perception bias for exposure determination

# concept[].concept[].concept[].concept[].designation[].value
subjective interpretive bias for exposure  determination

# concept[].concept[].concept[].concept[].designation[].value
subjectivity bias for exposure determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. The human interpretation can be that of the observer or participant.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: Consistency of phrasing with other definitions ("bias due to distortions in..."), need comment for application.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 6-0 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].display
bias due to lack of masking for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to awareness of the participant's status with respect to the outcome of interest or other relevant exposures.

# concept[].concept[].concept[].concept[].concept[].designation[].value
bias due to lack of blinding during exposure assessment

# concept[].concept[].concept[].concept[].concept[].designation[].value
bias due to lack of masking during exposure assessment

# concept[].concept[].concept[].concept[].concept[].designation[].value
awareness bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. Lack of blinding is not automatically a bias, but if awareness of some data systematically distorts the exposure determination then a 'Bias due to lack of masking for exposure determination' exists.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-1 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper
2022-02-11 vote 8-1 by Mario Tristan, Paul Whaley, Sunu Alice Cherian, Janice Tufte, Robin Ann Yurk, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer, Jesus Lopez-Alcalde
2022-02-18 vote 10-3 by Rebecca Baker, Brian S. Alper, Mario Tristan, Paul Whaley, Sunu Alice Cherian, Janice Tufte, Robin Ann Yurk, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer, Jesus Lopez-Alcalde, Joanne Dehnbostel,Sumalatha A

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: Is it just awareness of the participant's status with respect solely to the outcome of interest? I could imagine being aware of e.g. socioeconomic status rather than outcome, and this potentially having an influence on exposure assessment. Blinding I think is supposed to be to as many characteristics of the participant as possible.
2022-02-11 comment: This definition assumes that the "Lack of blinding for exposure determination" always associates bias, which may not be the case. For example, if we want to assess the role of sex as a prognostic factor for ICU admission, the participant may not be blinded but this does not cause bias in his/her prognostic factor determination (sex)
2022-02-18 comments: As "lack of blinding" is contributing to but not the bias itself, perhaps rename to Awareness bias for exposure determination
This definition assumes that the "Lack of blinding for exposure determination" always associates bias, which may not be the case. For example, if we want to assess the role of sex as a prognostic factor for ICU admission, the participant may not be blinded but this does not cause bias in his/her prognostic factor determination (sex)
Not much difference between existing and new terminology 
2022-02-25 comment: Suggest removing Lack of blinding during exposure assessment from Alternative term and just list the other 3 Alternative terms.  The comment is based on your comment for application description.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 8-0 by Robin Ann Yurk, Sunu Alice Cherian, Paola Rosati, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, nisha mathew, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].display
observer bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to subjective interpretations in the process of observing and recording information.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
Multiple types of bias can overlap. Observer bias is different than lack of blinding with respect to the outcome. Observer bias is about the influence of the observer's interpretation of what they are observing, whether or not the observer is aware of the participant's outcome.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 6-0 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].concept[].display
confirmation bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An observer bias for exposure determination due to previous opinions or knowledge of a subject’s prior exposures or assessments.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
diagnostic suspicion bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
previous opinion bias for exposure  determination

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to differences in accuracy or completeness of recall of past events or experiences.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].display
apprehension bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to a study participant's responding or behaving differently when aware of being observed.

# concept[].concept[].concept[].concept[].concept[].designation[].value
hawthorne effect for exposure determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: What about using Hawthorne Effect for term definition and Apprehension Bias for Alternative term

# concept[].concept[].concept[].concept[].concept[].display
hypothetical assessment bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to a difference between an individual’s report of an imagined or hypothetical response from their actual response.

# concept[].concept[].concept[].concept[].concept[].designation[].value
subjunctivity bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins, Mario Tristan

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. The response may be a behavior or valuation. An individual's response to "What would you do?" or "What would you have done?" (an imagined or hypothetical response) may be different than the individual's response to "What did you do?" or observation of the individual's behavior (a reporting of an actual response). This bias is relevant for preference studies.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 4-1 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comments: Is there a spelling error in Subjunctivity?
A minor issue - would the sentence "The response may be a behavior or valuation." be better placed in the comment for application (otherwise, would vote yes)
2022-02-11 comment: I would add a comment for application for the word hypothetical

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 9-0 by Mario Tristan, Paul Whaley, Sunu Alice Cherian, Robin Ann Yurk, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to a misinterpretation of observations that resemble the exposure.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. Other terms (Exposure Ascertainment Bias, Exposure Measurement Bias, Exposure Classification Bias) may be used to describe the process in Exposure Detection in which the bias occurs. The term 'Mimicry bias for exposure determination' is used to represent the type of cognitive interpretive bias occurring in this process.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 comments: Suggest insert Alternative term:  Duplicate
I'm not quite sure this is clear enough, though I don't have any concrete suggestions for improvement. It might be that I am not familiar enough with the issue in question to interpret the definition. Reading around this a bit, it resembles a misclassification type bias (for a given set of observations, the observer takes X to be cause when the true cause is Y). Given our model for bias (see our flow diagram), might it be better defined in those terms? -- RESOLVED IN GROUP DISCUSSION

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 11-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Paul Whaley, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Joanne Dehnbostel, Sumalatha A

# concept[].concept[].concept[].concept[].concept[].display
unacceptability bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias for exposure determination due to distortions in response, response values, or recording of responses resulting from perception of the social unacceptability of an exposure.

# concept[].concept[].concept[].concept[].concept[].designation[].value
unacceptable disease bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Janice Tufte, Joanne Dehnbostel, Ken Wilkins, Lisa Schilling

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Paul Whaley

# concept[].concept[].concept[].concept[].definition
An exposure detection bias due to distortions in how the data are collected.

# concept[].concept[].concept[].concept[].designation[].value
ascertainment bias for exposure determination

# concept[].concept[].concept[].concept[].designation[].value
data collection bias for exposure determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: Suggest modify Alternative term to Data Collection Bias

# concept[].concept[].concept[].concept[].concept[].display
nonrepresentative observation period for exposure of interest

# concept[].concept[].concept[].concept[].concept[].definition
An exposure ascertainment bias due to differences in the time period used for observation of the exposure and the intended time period for the exposure of interest.

# concept[].concept[].concept[].concept[].concept[].designation[].value
misaligned follow up period for exposure of interest

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 4-1 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: I think this is about right but it could perhaps be tidied up a bit, e.g. using "time period" in both instances of "period"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 9-0 by Mario Tristan, Paul Whaley, Sunu Alice Cherian, Robin Ann Yurk, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].definition
An exposure ascertainment bias due to differences in the context in which the exposure is observed and the intended context for the exposure of interest.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
This term is used when the context used for exposure ascertainment is incorrect, insensitive, or nonspecific.  If the context (whether representative or not) is applied inconsistently, then use the term "Inconsistency in exposure ascertainment"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 10-1 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Joanne Dehnbostel, Sumalatha A

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 comments: Comment for application.  I would delete sentence:  If the method (whether dependable or undependable) is applied inconsistently then use the term inconsistency in application of exposure of ascertainment.  
I'm not sure if "undependable" is the word we really want to use. Also, (1) no method for exposure ascertainment will give a strictly "correct" result, (2) inconsistency can result in random error and imprecision, not necessarily bias, (3) we are presumably worried about consistency over- or under-reading of a measurement method compared to some (possibly hypothetical) gold standard? Overall, it feels like there is more to discuss here.
2022-02-25 comment: I would delete or edit the current Alternative term and replace with insensitive, or nonspecific context for exposure ascertainment.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 8-0 by Robin Ann Yurk, Sunu Alice Cherian, Paola Rosati, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, nisha mathew, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].definition
An exposure ascertainment bias due to differences within or between groups in how the data are collected.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
This term is used when the context (whether representative or not) is applied inconsistently. If the context used for exposure ascertainment is incorrect, insensitive, or nonspecific, then use the term "Nonrepresentative context for exposure ascertainment"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 8-1 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Joanne Dehnbostel, Sumalatha A

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 comments: I would add comment for application from previous term. 
 If the method (whether dependable or undependable) is applied inconsistently then use the term inconsistency in application of exposure of ascertainment.  
I don't really understand the term "Inconsistency in application of exposure ascertainment" - I am not clear what the nouns and verbs actually are here, nor what they refer to. I have been involved in the discussion of the underlying bias model and I still don't grasp the meaning here.
2022-02-25 comment: I would remove Alternative term.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 8-0 by Robin Ann Yurk, Sunu Alice Cherian, Paola Rosati, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, nisha mathew, Paul Whaley

# concept[].concept[].concept[].concept[].definition
An exposure detection bias due to distortions in how the observed exposures are measured.

# concept[].concept[].concept[].concept[].designation[].value
measurement bias for exposure determination

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
If one is addressing a bias in the instruments or processes used to measure the observed exposure, use Exposure Measurement Bias. If one is addressing how the measured exposure is categorized, use Exposure Classification Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Jesus Lopez-Alcalde, Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].display
inappropriate method for exposure measurement

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to use of an incorrect method or protocol.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 11-2 by Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Jesus Lopez-Alcalde, Sumalatha A, Joanne Dehnbostel, Paola Rosati, nisha mathew

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 comments: I would list measurement methods as examples under comment for application, such as pharma, survey...
I am not sure of the difference between this bias and "Undependable method for exposure ascertainment" bias. It also seems to me that "inappropriate" is a subjective term so I am not sure how it should be applied. [Side note: in the ballot, it might be useful to have terms arranged as they are in the SEVCO hierarchy, as this might be causing some of the confusion I am experiencing.]
The previous term convey almost similar meaning

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-11 vote 5-0 by Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, nisha mathew, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].display
insensitive measure bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to use of a method that does not reliably detect the exposure when the exposure is present.

# concept[].concept[].concept[].concept[].concept[].designation[].value
inadequate sensitivity for exposure measure

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. Use of an inadequately sensitive exposure measure is likely to result in false negative findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 comments: Suggest use term as Sensitivity Measure bias for exposure determination and insensitive measure bias for exposure determination for alternate term.
"Sensitivity" is not, in my experience, viewed exclusively in terms of measurement. Some experimental models cannot show the exposure (or outcome) because they are incapable of it, however it is measured in situ. For example, if the exposure was measured via presence of a metabolite, but the participant was not able to produce the metabolite, then the experiment would be insensitive regardless of measurement method. I am not sure this affects us here, but does it suggest a need for us to handle sensitivity in a comprehensive fashion? (Perhaps also specificity?)

As a side note, defining sensitivity well could be important for progress on risk of bias assessment methods used by EPA, who currently have assessment of "sensitivity" as a separate issue entirely outside of risk of bias assessment. 
NEGATIVE VOTE CHANGED TO POSITIVE DURING DISCUSSION 2022-02-25

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 13-0 by Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Jesus Lopez-Alcalde, Sumalatha A, Joanne Dehnbostel, Paola Rosati, nisha mathew

# concept[].concept[].concept[].concept[].concept[].display
nonspecific measure bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to use of a method that falsely detects the exposure when the exposure is absent.

# concept[].concept[].concept[].concept[].concept[].designation[].value
inadequate specificity for exposure measure

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. Use of an inadequately specific exposure measure is likely to result in false positive findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 vote 5-0 by Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-04 comment: Suggest use Specificity measure bias for exposure determination and non-specific measure bias for exposure determination for Alternative term.

# concept[].concept[].concept[].concept[].concept[].display
inappropriate application of method for exposure measurement

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to inappropriate application of the method or protocol.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. An inappropriate application of the method or protocol suggests error is introduced by the process of measurement, as distinct from the method or protocol used for measurement (which would be an Inappropriate method for exposure measurement).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 12-1 by Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Jesus Lopez-Alcalde, Sumalatha A, Joanne Dehnbostel, Paola Rosati, nisha mathew

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 comments: Add alternate term:  Incorrect application of exposure measurement bias.
I think this is OK, but the term should be rewritten so it is easier to read and understand what it means (the syntax is awkward, as it could be read as one adjective and three nouns)
2022-03-11 comment: In documenting this, and the "inappropriate method for exposure measurement", I think it would be helpful to document what we mean by e.g. "method" vs. "application of method". I feel these are meta-terms like "study design feature" that are part of the scaffolding of SEVCO, but not part of SEVCO itself.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-11 vote 5-0 by Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, nisha mathew, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences within groups in how the observed exposures are measured.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. "How the observed exposures are measured" may refer to the methods applied for measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 9-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Paul Whaley, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].concept[].display
inconsistency in instruments used for exposure measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences within groups in the instruments for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
Instruments used for measurement may include devices, surveys, and technologies. The concepts of "instruments used for measurement" is distinct from "process used for measurement" which may include protocols, techniques, and variations in context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].concept[].display
inconsistency in processes used for exposure measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences within groups in the processes by which the instruments are used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
The processes used for measurement may include protocols, techniques, and variations in context. The concept of "processes used for measurement" is distinct from "instruments used for measurement" which may include devices, surveys, and technologies.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences between groups in how the observed exposures are measured.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Paul Whaley, Robin Ann Yurk, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. "How the observed exposures are measured" may refer to the methods applied for data measurement or the application of those methods.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].concept[].display
imbalance in instruments used for exposure measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences between groups in the instruments used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
Instruments used for measurement may include devices, surveys, and technologies. The concepts of "instruments used for measurement" is distinct from "process used for measurement" which may include protocols, techniques, and variations in context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].concept[].display
imbalance in processes used for exposure measurement

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure measurement bias due to differences between groups in the processes by which the instruments are used for measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
The processes used for measurement may include protocols, techniques, and variations in context. The concept of "processes used for measurement" is distinct from "instruments used for measurement" which may include devices, surveys, and technologies.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Mario Tristan, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].definition
An exposure detection bias due to distortions in how the observed exposures are classified.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
If one is addressing a bias in the instruments or processes used to measure the observed exposure, use Exposure Measurement Bias. If one is addressing how the measured exposure is categorized, use Exposure Classification Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].concept[].definition
An exposure classification bias resulting from the definition or threshold used for exposure classification.

# concept[].concept[].concept[].concept[].concept[].designation[].value
definition bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].designation[].value
threshold bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.
An exposure definition bias suggests an internal validity problem in which the definition or threshold used for exposure classification does not represent the exposure of interest. If considering an external validity problem, the "Wrong question bias" (term not yet defined) may be used. An exposure definition bias is present when there are differences between the exposure of interest and the definition or threshold used for exposure classification.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 comments: Suggest Alternative term:  threshold bias for exposure determination.  
Suggest remove sentence on external validity problem....
In the comments, "term not yet identified", should be flagged for later replacement.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer [but comment discussion led to new term]

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 5-0 by Joanne Dehnbostel, Sumalatha A, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure definition bias due to a mismatch between the exposure of interest and the definition or threshold used for exposure measurement.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 comment: Should there be a hyphen between "classification" and "system"? (Is it a system(s) bias or a classification-system bias?) (I think this question applies to several definitions)

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer [but definition changed to match change to parent term]

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 6-0 by Joanne Dehnbostel, Alejandro Piscoya, Sumalatha A, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].display
surrogate marker bias for exposure classification

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 comment: Suggest add Alternative term:  proxy bias for exposure classification system.

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer [but definition changed to match change to parent term]

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 5-0 by Joanne Dehnbostel, Sumalatha A, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure definition bias due to use of a definition that is proxy for the exposure rather than direct observation of the exposure.

# concept[].concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
substitution bias for exposure classification

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An exposure definition bias due to definition of the exposure after interacting with the study data.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
definition for exposure classification not prespecified

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
no prespecified definition for exposure classification

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer [but definition changed to match change to parent term]

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 5-0 by Joanne Dehnbostel, Sumalatha A, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].display
classification process bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
An exposure classification bias resulting from the application of the method used for exposure classification.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. 
A classification process bias for exposure determination suggests error is introduced by the process of classification, as distinct from the definition or threshold used (which would be an Exposure Definition Bias).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 comments: I would provide an example such as survey severity classification example of a method.
(Inconsistent capitalization)

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-11 vote 8-0 by Jesus Lopez-Alcalde, Mario Tristan, Robin Ann Yurk, Janice Tufte, Sunu Alice Cherian, Alejandro Piscoya, Harold Lehmann, Yasser Sami Amer [but term changed to parallel changes to Exposure Definition Bias]

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 vote 6-0 by Joanne Dehnbostel, Sumalatha A, Robin Ann Yurk, Janice Tufte, Harold Lehmann, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].display
incorporation bias for exposure determination

# concept[].concept[].concept[].concept[].concept[].definition
An exposure classification bias due to the inclusion of the outcome or other relevant exposures under investigation in the method or process used for exposure classification.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Mario Tristan, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The exposure of interest can be an intervention or a prognostic factor, depending on the research context. If the statistical analysis assumes independence of two variables, but one variable incorporates the other variable in its definition, the assumption will be false and the result will be distorted.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-25 vote 11-2 by Paul Whaley, Mario Tristan, Robin Ann Yurk, Janice Tufte, Yasser Sami Amer, Harold Lehmann, Alejandro Piscoya, Sunu Alice Cherian, Jesus Lopez-Alcalde, Sumalatha A, Joanne Dehnbostel, Paola Rosati, nisha mathew

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-02-18 comments: Needs an Alternative term or new term definition. I.e. Inclusion Bias for exposure definition for the term.  Alternative term;  eligibility bias for exposure determination
Definitely needs a comment for application, I can't picture what this means!

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-03-11 vote 5-0 by Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, nisha mathew, Paul Whaley

# concept[].concept[].concept[].definition
A detection bias due to distortions in how the data for a potential confounder are determined.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].display
detection bias related to the reference standard

# concept[].concept[].concept[].definition
A detection bias due  to distortions in how the reference standard result is determined.

# concept[].concept[].concept[].designation[].value
bias for reference standard result determination

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
10/10 as of 8/27/2021: Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].display
detection bias related to the index test

# concept[].concept[].concept[].definition
A detection bias due to distortions in how the index test result is determined.

# concept[].concept[].concept[].designation[].value
bias for index test result determination

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
5/5 as of 8/30/2021: Eric Harvey, Harold Lehmann, Mario Tristan, Bhagvan Kommadi, Janice Tufte,

# concept[].concept[].concept[].property[].valueString
8/27/2021 vote 8-1 on "Detection Bias related to the index test (Bias for index text result determination) = A detection bias due to distortions in how the index text result is determined." by, Eric Au, Eric Harvey, Harold Lehmann, Alejandro Piscoya, Mario Tristan, Bhagvan Kommadi, Leo Orozco, Janice Tufte, Jesús López-Alcalde, Brian S. Alper

# concept[].concept[].concept[].property[].valueString
2021-08-27 comment: I think the word "text" should be "test" in the Alternative term and definition. Please consider broadening this term and definition to include distortions in how the index event is determined

# concept[].concept[].concept[].definition
A detection bias due to differences between measured values and recorded values.

# concept[].concept[].concept[].property[].valueString
Data Entry Bias may include distorted results due to errors in transcription, translation, or transposition between the measured value and the recorded value, or between a recorded value and a subsequent recording of the value.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Janice Tufte, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2022-08-26 vote 7-0 by nisha mathew, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, Cauê Monaco, Eric Harvey

# concept[].concept[].concept[].display
inappropriate time interval between predictor assessment and outcome determination

# concept[].concept[].concept[].definition
A detection bias involving the time interval between the observation of the predictor and outcome, where the interval used by the study differs from the interval assumed by the predictive model.

# concept[].concept[].concept[].property[].valueString
Nonrepresentative observation period for outcome of interest is defined as an outcome ascertainment bias due to differences in the period used for observation of the outcome and the period for the outcome of interest.

Nonrepresentative observation period for exposure of interest is defined as an exposure ascertainment bias due to differences in the time period used for observation of the exposure and the intended time period for the exposure of interest.

In the context of predictive modeling, the time interval between the exposure (predictor) and the outcome should be representative of the time interval of interest.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2023-10-06 vote 3-1 by Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey, Paul Whaley

# concept[].concept[].concept[].property[].valueString
2023-10-06 comment: Two problems: (1) I am not sure how the definition equates to the term - in the term, it is about inappropriate time interval, but in the definition it is about the time interval not being that which is intended and representative of application of model. (2) I don't understand what is meant by the phrase "the intended time interval between the predictor and outcome that is representative of the application of the predictive model" - there are too many concepts all at once here, I think?

# concept[].concept[].concept[].property[].valueString
2023-10-20 vote 5-0 by Muhammad Afzal, Eric Harvey, Harold Lehmann, Louis Leff, Joanne Dehnbostel

# concept[].concept[].definition
A bias related to the analytic process applied to the data.

# concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Philippe Rocca-Serra, Mhuammad Afzal, Kenneth Wilkins

# concept[].concept[].property[].valueString
6/6 as of 8/15/2021: Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Mhuammad Afzal, Eric Harvey

# concept[].concept[].concept[].display
bias related to selection of the analysis

# concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of analysis methods before the analysis is applied.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Philippe Rocca-Serra, Mhuammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
6/6 as of 8/15/2021: Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Mhuammad Afzal, Eric Harvey

# concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].property[].valueString
ROBIS 4.2 Were all pre-defined analyses reported or departures explained?

# concept[].concept[].concept[].concept[].display
bias related to selection of the data for analysis

# concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of data included in the analysis before the analysis is applied.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 6-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Janice Tufte, Mario Tristan, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].display
bias due to post-baseline factors influencing selection of the data for analysis

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the data analysis based on participant characteristics observed after study enrollment.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
ROBINS-I 2.1. Was selection of participants into the study (or into the analysis) based on participant characteristics observed after the start of intervention?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 6-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Janice Tufte, Mario Tristan, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the data analysis in which data are not completely analyzed according to the original assignment to comparison groups in an interventional study.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 4-1 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati
2022-05-20 vote 9-1 by Joanne Dehnbostel, nelle.stocquart@kce.fgov.be, Eric M Harvey, Jesus Lopez-Alcalde, Paul Whaley, Robin Ann Yurk, Harold Lehmann, raradhikaag@gmail.com, Mario Tristan, Paola Rosati
2022-05-27 vote 4-1 by Robin Ann Yurk, Mario Tristan, Jesus Lopez-Alcalde, Eric M Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 comment: Instead of defining "Inadequate intention-to-treat analysis" why not defining waht "intention-to-treat analysis" is?
2022-05-20 comment: Suggest change term name to Intention to Treat Analysis and remove word inadequate from the term as this term includes the limitation of the analysis in the definition.
2022-05-27 comment: Missing Data Analysis:  examples are imputation of data according to rules. 
Purpose:  To provide additional validity that the data are not biased from the missing data. {{2022-05-27 discussion suggests this can be handled by the SEVCO:00307 term [Inappropriate handling of missing data] which is classified as a 'Bias in processing of data'}}

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An intention-to-treat analysis may be defined as analysis of all randomized subjects according to their assigned intervention rather than according to the intervention actually received. There is considerable variation in reported studies with respect to the use of the term 'intention-to-treat analysis' and 'modified intention-to-treat analysis' but if the risk of bias assessment suggests an insufficient accounting for all participants as intended then one may report 'Inadequate intention-to-treat analysis'.
In non-randomized studies, this term may be used to denote missing or inadequate analysis according to the intended treatment, e.g prescribed medication vs. taken medication.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-03 vote 6-0 by Joanne Dehnbostel, Mario Tristan, Eric M Harvey, Harold Lehmann, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].display
missing or inadequate per-protocol analysis

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the data analysis in which data are not completely analyzed according to the study protocol.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel, Paul Whaley, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 4-1 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati
2022-05-20 vote 7-3 by Joanne Dehnbostel, nelle.stocquart@kce.fgov.be, Eric M Harvey, Jesus Lopez-Alcalde, Paul Whaley, Robin Ann Yurk, Harold Lehmann, raradhikaag@gmail.com, Mario Tristan, Paola Rosati
2022-05-27 vote 4-1 by Robin Ann Yurk, Mario Tristan, Jesus Lopez-Alcalde, Eric M Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-20 comments: I do not fully agree with this definition. I propose following the Cochrane Handbook: 
Naïve ‘per-protocol’ analysis: analysis restricted to individuals who adhered to their assigned interventions.

Moreover, there is another analysis that is often biased: 

‘As-treated’ analysis: analysis in which participants are analysed according to the intervention they actually received, even if their randomized allocation was to a different treatment group

I would present these as different analyses (not as synonims)

https://training.cochrane.org/handbook/current/chapter-08
------
I think I see what the definition is saying but it is rather hard to parse.
re: "Inadequate per-protocol analysis" = "A bias related to selection of the data analysis in which data are not completely analyzed according to the assignment to comparison groups according to the interventions received."

Suggest edit term so it reads per protocol analysis and remove the word inadequate. This type of analysis includes the bias in the term already

2022-05-13 comment: Instead of defining "Inadequate per-protocol analysis" why not defining what "per-protocol anlysis" is?

2022-05-27 comment: Missing Data Analysis:  examples are imputation of data according to rules. 
Purpose:  To provide additional validity that the data are not biased from the missing data. {{2022-05-27 discussion suggests this can be handled by the SEVCO:00307 term [Inappropriate handling of missing data] which is classified as a 'Bias in processing of data'}}

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A per-protocol analysis may be defined as analysis of participants according to adherence to the assigned intervention (the 'treatment protocol') and/or according to adherence to the data collection protocol. Adherence may refer to adherence by the study participants or study personnel.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-03 vote 6-0 by Joanne Dehnbostel, Mario Tristan, Eric M Harvey, Harold Lehmann, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].display
missing or inadequate as-treated analysis

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the data analysis in which data are not completely analyzed according to the interventions actually received.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An as-treated analysis may be defined as analysis of subjects according to the intervention actually received rather than their assigned intervention.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-20 comments (from precursor term of Inadequate per-protocol analysis): I do not fully agree with this definition. I propose following the Cochrane Handbook: 
Naïve ‘per-protocol’ analysis: analysis restricted to individuals who adhered to their assigned interventions.

Moreover, there is another analysis that is often biased: 

‘As-treated’ analysis: analysis in which participants are analysed according to the intervention they actually received, even if their randomized allocation was to a different treatment group

I would present these as different analyses (not as synonims)

https://training.cochrane.org/handbook/current/chapter-08
------
I think I see what the definition is saying but it is rather hard to parse.
re: "Inadequate per-protocol analysis" = "A bias related to selection of the data analysis in which data are not completely analyzed according to the assignment to comparison groups according to the interventions received."

Suggest edit term so it reads per protocol analysis and remove the word inadequate. This type of analysis includes the bias in the term already

2022-05-27 comment: Missing Data Analysis:  examples are imputation of data according to rules. 
Purpose:  To provide additional validity that the data are not biased from the missing data. {{2022-05-27 discussion suggests this can be handled by the SEVCO:00307 term [Inappropriate handling of missing data] which is classified as a 'Bias in processing of data'}}

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-03 vote 6-0 by Joanne Dehnbostel, Mario Tristan, Eric M Harvey, Harold Lehmann, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].display
bias related to selection of the variables for analysis

# concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of variables included in the analysis before the analysis is applied.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 5-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].display
bias related to selection of the variables for adjustment for confounding

# concept[].concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of the variables for adjustment for confounding before the analysis is applied.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 5-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].property[].valueString
This term was determined to also match 'Post-intervention confounding different (draft) Code: SEVCO:00283' which was originally derived from the trigger question from ROBINS-I: 1.6. Did the authors control for any post-intervention variables that could have been affected by the intervention?   Detailed analysis found this to be more about improper control of 'confounding variables' that were not truly confounding variables.

# concept[].concept[].concept[].concept[].concept[].concept[].display
bias controlling for time-varying confounding

# concept[].concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the variables for adjustment for confounding in which the confounding is time-dependent.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Mario Tristan, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 6-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the variables for adjustment for confounding by adherence.

# concept[].concept[].concept[].concept[].concept[].concept[].designation[].value
bias controlling for confounding by adherence

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
An analysis selection after the analysis is applied would be considered a Selective Analysis Reporting Bias.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 6-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].display
predictors included in outcome definition

# concept[].concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of the variables for estimation of association in which one variable is incorporated in the definition of the other variable.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Predictors are also called covariates, risk indicators, prognostic factors, determinants, index test results, or independent variables (https://www.acpjournals.org/doi/10.7326/M18-1377).

If a predictor in the model forms part of the definition or assessment of the outcome that the model predicts, the association between predictor and outcome will likely be overestimated, and estimates of model performance will be optimistic; in diagnostic research, this problem is generally called incorporation bias. (https://www.acpjournals.org/doi/10.7326/M18-1377)

When this type of analysis bias is applied to predictive model analyses (in which the predictor is the exposure of interest), this type of bias is equivalent to "Incorporation bias for outcome determination" [SEVCO:00063]

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-29 vote 5-0 by Janice Tufte, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-29 comment: should "incorporation bias" be added as 'Alternative term' ?

# concept[].concept[].concept[].concept[].concept[].display
bias related to selection of predictors based on univariable analysis

# concept[].concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of the predictor variables for estimation of association in which predictors are selected based on statistically significant univariable associations (without adjustment for other predictors).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Predictors are also called covariates, risk indicators, prognostic factors, determinants, index test results, or independent variables (https://www.acpjournals.org/doi/10.7326/M18-1377).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
PROBAST (https://www.acpjournals.org/doi/10.7326/M18-1377).:
4.5 Was selection of predictors based on univariable analysis avoided? (Model development studies only)

A data set will often have many features that could be used as candidate predictors, and in many studies researchers want to reduce the number of predictors during model development to produce a simpler model.

In a univariable analysis, individual predictors are tested for their association with the outcome. Researchers often select the predictors with a statistically significant univariable association (for example, P < 0.05) for inclusion in the development of a final prediction model. This method can lead to incorrect predictor selection because predictors are chosen on the basis of their statistical significance as a single predictor rather than in context with other predictors (49, 50, 191). Bias occurs when univariable modeling results in omission of variables from the model, because some predictors are important only after adjustment for other predictors, known from previous research to be important, did not reach statistical significance in the particular development set (for example, due to small sample size). Also, predictors may be selected on the basis of a spurious (accidental) association with the outcome in the development set.

A better approach to decide on omitting, combining, or including candidate predictors in multivariable modeling is to use nonstatistical methods—that is, methods without any statistical univariable pretesting of the associations between candidate predictors and outcome. Better methods include those based on existing knowledge of previously established predictors in combination with the reliability, consistency, applicability, availability, and costs of predictor measurement relevant to the targeted setting. Well-established predictors and those with clinical credibility should be included and retained in a prediction model regardless of any statistical significance (49, 50, 192). Alternatively, some statistical methods that are not based on prior statistical tests between predictor and outcome can be used to reduce the number of modeled predictors (for example, principal components analysis).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-29 vote 5-0 by Janice Tufte, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].display
bias related to selection of the analytic framework

# concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate choice of the analytic framework before the analysis is applied.

# concept[].concept[].concept[].concept[].designation[].value
bias related to selection of the statistical framework

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Mario Tristan, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
An analytic framework is the model, scaffolding, or organizational representation of concepts used in analyzing the data. The concepts included in an analytic framework may involve data, variables, formulas, assumptions, and adjustments.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-13 vote 6-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Paola Rosati, Janice Tufte [[but then the term changed in webmeeting 2022-05-13]]
2022-05-20 vote 4-2 by Joanne Dehnbostel, Eric M Harvey, Mario Tristan, Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-20 comments:
seems to be entirely too much overlap with the "inappropriate analytic framework" term

I like this term and definition but I am not sure it is adequately differentiated from "inappropriate analytical framework". I think the term needs changing in some way.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-27 vote 5-0 by Mario Tristan, Jesus Lopez-Alcalde, Eric M Harvey, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the analytic framework in which the analytic model does not match the dataset characteristics or does not match the intention of the analysis.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A bias related to selection of the analytic framework is defined as an analysis bias due to inappropriate choice of the analytic framework before the analysis is applied.

An inappropriate statistical model may include one in which there is a mismatch between the realities of the data and the assumptions required for the analytic model. Complexities in the data may include univariate concerns (e.g. skewness or outliers) and multivariate concerns (e.g. curvilinearity, co-linearity, or latent associations between variables).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-20 vote 5-1 by Joanne Dehnbostel, Eric M Harvey, Mario Tristan, Jesus Lopez-Alcalde, Paul Whaley, Harold Lehmann

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-20 comment:
I like this term and definition but I am not sure it is adequately differentiated from "Bias related to selection of the analytic framework". I think the term needs changing in some way.
2022-09-30 Steering Group change to Comment to application: comment added to this term instead of creating a new term for 'Inappropriate handling of complexities in the data'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-05-27 vote 5-0 by Mario Tristan, Jesus Lopez-Alcalde, Eric M Harvey, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An inappropriate statistical model due to inappropriate accounting for ranges of potential observation in which data observation is not possible.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
PROBAST 4.6 Were complexities in the data (e.g. censoring, competing risks, sampling of controls) accounted for appropriately?

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
An inappropriate statistical model is a bias related to selection of the analytic framework in which the analytic model does not match the dataset characteristics or does not match the intention of the analysis.

The "ranges of potential observation" may include periods of time (temporal ranges within which observation may occur), or ranges of detection with a measurement instrument  (ranges of values that could be observed).

The concept of ranges of potential observation in which data observation is "not possible" may include impossibility due to physical realities (such as timing after competing risks or measurement instruments with limited ranges of detection) or impossibility due to administrative decisions (such as the observation period defined by the study protocol).

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-10-20 vote 7-0 by Philippe Rocca-Serra, Harold Lehmann, Joanne Dehnbostel, Mario Tristan, Paul Whaley, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
bias due to selection of the statistical significance threshold

# concept[].concept[].concept[].concept[].concept[].definition
An analysis bias resulting from selection of an inappropriate threshold for statistical significance.

# concept[].concept[].concept[].concept[].concept[].designation[].value
bias related to selection of the threshold for statistical significance

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The statistical significance threshold is part of the analytic framework. A bias related to selection of the analytic framework is defined as an analysis bias due to inappropriate choice of the analytic framework before the analysis is applied.

In frequentist analysis, statistical significance is the rejection of the null hypothesis based on the p value. In Bayesian analysis, statistical significance is the acceptance of the hypothesis based on the posterior probability.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 vote 5-1 by Brian S. Alper, Robin Ann Yurk, Paola Rosati, Mario Tristan, Harold Lehmann, Eric M Harvey
2022-06-17 vote 4-1 by Paul Whaley, Muhammad Afzal, Eric M Harvey, Jesus Lopez-Alcalde, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 comment: Consider editing the term definition to just Statistical significance threshold.  For the Alternative term remove word bias.  For the comment for application remove the first sentence about bias.
2022-06-17 comments: I think I get it, but it is a bit tortured and I wonder if a normal user would interpret it correctly or understand it?
I am not sure we can rephrase the concept name making it more compact like "Statistical significance threshold selection bias"

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-24 vote 5-0 by Mario Tristan, Harold Lehmann, Eric Harvey, Janice Tufte, Louis Leff

# concept[].concept[].concept[].concept[].concept[].concept[].display
bias related to multiple comparison adjustment

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An analysis bias resulting from selection of a threshold for statistical significance which does not appropriately account for the effect of multiple comparisons on the statistical probability related to the result.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
This bias may cause inappropriate rejection of the null hypothesis due to an unmodified threshold for significance in the face of multiple comparisons. This bias may also occur when adjustment for multiple comparisons is inappropriately applied and leads to failure to reject the null hypothesis.

A bias due to selection of the statistical significance threshold is defined as an analysis bias resulting from selection of an inappropriate threshold for statistical significance.

In frequentist analysis, statistical significance is the rejection of the null hypothesis based on the p value. In Bayesian analysis, statistical significance is the acceptance of the hypothesis based on the posterior probability.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 vote 3-2 by Brian S. Alper, Robin Ann Yurk, Paola Rosati, Mario Tristan, Harold Lehmann
2022-06-17 vote 4-1 by Paul Whaley, Muhammad Afzal, Eric M Harvey, Jesus Lopez-Alcalde, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 comments: The measure does not have a statistical probability, the finding or result has a statistical probability. Change definition to "A statistical significance threshold selection bias in which the threshold for statistical significance does not account for the effect of multiple comparisons on the statistical probability related to the result."
Is this a bias or just an incomplete analysis due to data requirements needed to compute the multiple comparison adjustment.
2022-06-17  comment: Looking at the significance threshold bias terms, the other two refer to selection of the analytic framework, but this one does not. Is there a reason for that?

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-24 vote 5-0 by Muhammad Afzal, Mario Tristan, Harold Lehmann, Eric Harvey, Louis Leff

# concept[].concept[].concept[].concept[].concept[].concept[].display
mismatch of significance threshold and purpose

# concept[].concept[].concept[].concept[].concept[].concept[].definition
An analysis bias resulting from selection of a threshold for statistical significance which is inappropriate due to a mismatch between (1) how the statistical probability related to the result is determined and (2) the purpose for categorizing the result as statistically significant.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
A threshold used for variable selection in regression analysis is often more liberal than a threshold used in hypothesis testing. Similarly a situation regarding safety may tolerate a higher chance of false positive findings so significance threshold may be higher. Some factors to consider include sample size, power of the test, and expected losses from Type I and Type II errors.

In frequentist analysis, statistical significance is the rejection of the null hypothesis based on the p value. In Bayesian analysis, statistical significance is the acceptance of the hypothesis based on the posterior probability.

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Mario Tristan, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
How to Choose the Level of Significance: A Pedagogical Note -- The level of significance should be chosen with careful consideration of the key factors such as the sample size, power of the test, and expected losses from Type I and II errors. While the conventional levels may still serve as practical benchmarks, they should not be adopted mindlessly and mechanically for every application.  
(https://mpra.ub.uni-muenchen.de/66373/1/MPRA_paper_66373.pdf)

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 vote 2-2 by Brian S. Alper, Robin Ann Yurk, Mario Tristan, Harold Lehmann
2022-06-17 vote 4-1 by Paul Whaley, Muhammad Afzal, Eric M Harvey, Jesus Lopez-Alcalde, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-10 comments: A mismatch can occur even if the purpose was taken into account. As the term name "Mismatch of significance threshold and purpose" is a match for the definition of the parent term (Statistical significance threshold selection bias) there is a question of whether this term is needed.
Receiver operator curves are traditionally a statistic used to represent the continuum of cut point for the threshold value.  The Sensitivity and Specificity can be calculated to evaluate the validity of the threshold cut point.
2022-06-17 comment: Add "Bias related to..." at beginning for consistency with others. What work is "selection of the analytic framework" doing in this definition?

# concept[].concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-24 vote 5-0 by Muhammad Afzal, Mario Tristan, Harold Lehmann, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the analytic framework in which an outcome variable includes an observation period during which the outcome could not have occurred.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Consider a study in which a sample is followed from 2000 to 2010. Mortality during this time period is the outcome, and receipt of Superdrug is the exposure. 
--If 20 people received Superdrug in 2009 and 5 of them died in the subsequent year, the mortality with Superdrug is 25%. 
--If 20 people never received Superdrug and 1 died each year so by 2010 the mortality without Superdrug is 50%. 

Interpreting this result as Superdrug having a 50% relative risk reduction for mortality would be biased (distorted) by not accounting for the 9 years of time (immortal time) that the Superdrug recipients must have survived to be able to receive Superdrug in 2009.

If the outcome variable were defined as mortality 2009-2010, there would be no bias and the result would be a 150% relative risk increase.

If the outcome variable were defined as mortality 2000-2010, there is an immortal time bias (the Superdrug recipients could not have died before receiving Superdrug).

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel, Paul Whaley, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Catalog of bias:   A distortion that modifies an association between an exposure and an outcome, caused when a cohort study is designed so that follow-up includes a period of time where participants in the exposed group cannot experience the outcome and are essentially 'immortal'.

in https://academic.oup.com/aje/article/167/4/492/233064 :
Immortal time refers to a span of time in the observation or follow-up period of a cohort during which the outcome under study could not have occurred (13, 14). It usually occurs with the passing of time before a subject initiates a given exposure. While a subject is not truly immortal during this time span, the subject necessarily had to remain event free until start of exposure to be classified as exposed. An incorrect consideration of this unexposed time period in the design or analysis will lead to immortal time bias.

in JAMA https://jamanetwork.com/journals/jama/article-abstract/2776315
Such studies may be subject to immortal time bias, meaning that, during the period of observation, there is some interval during which the outcome event cannot occur

in https://watermark.silverchair.com/dyab157.pdf
In particular, incorrect handling of follow-up times in terms of exposure status in the analysis of such studies may introduce immortal time bias (ITB) in favour of the exposed group.2,3 Immortal time refers to a period of time in which, by design, participants in the exposed group cannot experience the outcome. This often  happens in pharmacoepidemiologic studies in which treatment is prescribed at variable times (with delay) after disease diagnosis. The bias occurs when the exposed group is considered to be exposed during their entire follow-up time (even during periods in which they are theoretically unexposed) or their unexposed follow-up times are discarded.2,3

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-15 vote 5-1 by Mario Tristan, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann, Janice Tufte

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-15 comment: Why the need to specify "in a larger sample" in the second sentence, since there is no assumption about size of the sample in the first assertion?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-22 vote 7-0 by Mario Tristan, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Jesus Lopez-Alcalde, Janice Tuft, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].definition
A bias related to selection of the analytic framework in which the sample size invalidates the assumptions of the analytic framework.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
An example of 'Inadequate sample size' is a finding of no effect with inadequate power to detect an effect. Another example of 'Inadequate sample size' is use of a parametric analysis with low numbers, which invalidates the assumptions for use of a parametric analysis.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Janice Tufte, Joanne Dehnbostel, Mario Tristan, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-07-22 vote 7-0 by Mario Tristan, Paul Whaley, Philippe Rocca-Serra, Harold Lehmann, Jesus Lopez-Alcalde, Janice Tuft, Eric Harvey

# concept[].concept[].concept[].display
bias related to execution of the analysis

# concept[].concept[].concept[].definition
An analysis bias due to inappropriate decisions pertaining to preparation of data for analysis and/or conduct of the analysis.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Paul Whaley, Yuan Gao

# concept[].concept[].concept[].property[].valueString
"Bias related to selection of the analysis" is used when the wrong analysis is done (the analysis is planned wrongly). "Bias in processing of data" is used when the analysis is done wrong (the analysis is executed wrongly).

# concept[].concept[].concept[].property[].valueString
2022-11-04 vote 5-0 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Janice Tufte, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
An analysis bias due to absence of a component of the analytic process.

# concept[].concept[].concept[].concept[].property[].valueString
Missing components may include addressing missing data, addressing potential confounders, checking model assumptions, or robustness checks for model misspecification.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 vote 4-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey
2022-08-19 vote 5-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey, Philippe Rocca-Serra
2022-08-25 vote 8-1 by nisha mathew, Jesus Lopez-Alcalde, Cauê Monaco, Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 comment: Ambiguous as to whether the data is incomplete or the analytic process incomplete. Also seems to be ambiguous as to whether the analysis is of a selected subset of the existing data (thus relating to selection bias?), or of data that is not representative of the totality of theoretically available data (thus relating to external validity?).

2022-08-19 comment: tension between bias and process. Shouldn't it be "incomplete analysis related bias"? omission seems to indicate a wilful act. "absence" may be more neutral when considering a 'canonical / state of the art / standardised ' protocol.

"An analysis bias due to absence of a component deemed necessary in a state-of- art (possibly regulator-approved ) analytic process."

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-09 vote 6-0 by Philippe Rocca-Serra, Harold Lehmann, Jesus Lopez-Alcalde, Khalid Shahin, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].display
inappropriate handling of uninterpretable data

# concept[].concept[].concept[].concept[].definition
An analysis bias due to omission of uninterpretable values, or their replacement with inappropriate values.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Brian S. Alper, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 vote 4-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 comment: I'm not sure I would understand the definition if I had not read the term, suggest rephrasing - "omission of accommodation for" is perhaps the problem part.

# concept[].concept[].concept[].concept[].property[].valueString
Inappropriate values may include use of non-representative imputation treating uninterpretable data like missing data. 

In evaluation of diagnostic tests, omission of or inappropriate classification of test results would be Inappropriate handling of uninterpretable data.

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-16 vote 5-0 by Mario Tristan, Janice Tufte, Eric Harvey, Yaowaluk Ngoenwiwatkul, nisha mathew

# concept[].concept[].concept[].concept[].definition
An analysis bias due to use of non-representative values in place of missing data.

# concept[].concept[].concept[].concept[].property[].valueString
Handling of missing data may address data missing at levels of single observations or groupings by encounter, participant, site, or subpopulation.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Brian S. Alper, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 vote 5-0 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].definition
An analysis bias due to processing a variable in an incorrect role or with an incorrect datatype.

# concept[].concept[].concept[].concept[].property[].valueString
Typical variable roles are population, exposure, confounder, and outcome.

A variable datatype may be numerical (continuous or discrete) or categorical (ordinal or nominal).

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Brian S. Alper, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-16 vote 5-0 by Mario Tristan, Janice Tufte, Eric Harvey, Yaowaluk Ngoenwiwatkul, nisha mathew

# concept[].concept[].concept[].concept[].definition
An analysis bias due to inappropriate application of adjustment techniques for correction of bias in the selection of participants for analysis.

# concept[].concept[].concept[].concept[].property[].valueString
Bias in the selection of participants for analysis could occur due to Participant Selection Bias (SEVCO:00003) or participant-level Bias related to selection of the data for analysis (SEVCO:00376).

"It is in principle possible to correct for selection biases, for example by using inverse probability weights to create a pseudo-population in which the selection bias has been removed, or by modelling the distributions of the missing participants or follow up times and outcome events and including them using missing data methodology." (Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. doi: 10.1136/bmj.i4919. PMID: 27733354; PMCID: PMC5062054.  Supplementary Table A.)

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
"It is in principle possible to correct for selection biases, for example by using
inverse probability weights to create a pseudo-population in which the
selection bias has been removed, or by modelling the distributions of the
missing participants or follow up times and outcome events and including
them using missing data methodology." (Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. doi: 10.1136/bmj.i4919. PMID: 27733354; PMCID: PMC5062054.  Supplementary Table A.)

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-30 vote 5-0 by Jesus Lopez-Alcalde, Harold Lehmann, Janice Tufte, Eric Harve, Morufu Olalekan Raimi

# concept[].concept[].concept[].concept[].definition
An analysis bias due to differences between recorded data and data used for analysis.

# concept[].concept[].concept[].concept[].property[].valueString
Data Transition Bias may include distorted results due to errors in transcription, translation, erroneous mapping, or transposition between the recorded data (values, labels, and other metadata) and the data used for analysis. Data Transition Bias may occur due to any problem encountered during the Extraction, Transformation, and Loading (ETL) process in data exchange.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2022-11-04 vote 5-0 by Philippe Rocca-Serra, Jesus Lopez-Alcalde, Janice Tufte, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].display
inappropriate handling of missing confounder data

# concept[].concept[].concept[].concept[].definition
An analysis bias due to use of non-representative values in place of missing data for variables in the role of confounder.

# concept[].concept[].concept[].concept[].property[].valueString
Handling of missing confounder data may address data missing at levels of single observations or groupings by encounter, participant, site, or subpopulation. Inappropriate handling of missing confounder data can result in misleading adjusted analyses.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-16 vote 5-0 by Mario Tristan, Janice Tufte, Eric Harvey, Yaowaluk Ngoenwiwatkul, nisha mathew

# concept[].concept[].concept[].concept[].definition
An analysis bias due to miscalculations in the processing of the data.

# concept[].concept[].concept[].concept[].property[].valueString
This bias is intended to cover a broad range of errors in curating the data and performing the calculations specified or implied by the analytic plan, including but not limited to: memory allocation and other environmental specifications, data ingestion pipeline, statistical package choice and vetting, and syntax, semantics and logic of coding. this bias can be applied to both manual or computer based computation.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte, Harold Lehmann, Khalid Shahin, Muhammad Afzal, Neeraj Ojha

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 vote 4-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey
2022-08-19 vote 4-2 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 comment: Not sure about including data entry errors among errors in software code - the latter is a computational error, the former is not. Also, the definition does not specify computational processing.
2022-08-19 comment: the class label is ambiguous: is it "computation error caused bias" or it is 'contradictions caused bias? The latter term does not add clarity.
Also, only data entry errors resulting from computational errors would fall under this type of bias, but not direct entry of values.

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-09 vote 6-0 by Philippe Rocca-Serra, Harold Lehmann, Jesus Lopez-Alcalde, Khalid Shahin, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].display
reported analysis not following pre-specified analysis plan

# concept[].concept[].concept[].definition
An analysis bias in which the reported analysis does not match the pre-specified analysis plan.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2023-03-10 vote 5-0 by Joanne Dehnbostel, Mario Tristan, Harold Lehmann, Eric Harvey, Jesus Lopez-Alcalde

# concept[].concept[].concept[].definition
An analysis bias in which an estimation of association between two variables is distorted by controlling for a third variable affected by both variables of interest (or factors causing the variables of interest).

# concept[].concept[].concept[].property[].valueString
A structural classification of bias distinguishes between biases resulting from conditioning on common effects (“selection bias”) --- A Structural Approach to Selection Bias, https://journals.lww.com/epidem/Fulltext/2004/09000/A_Structural_Approach_to_Selection_Bias.20.aspx

Collider bias occurs when an exposure and outcome (or factors causing these) each influence a common third variable and that variable or collider is controlled for by design or analysis. In contrast, confounding occurs when an exposure and outcome have a shared common cause that is not controlled for. -- JAMA 2022 Mar 14 https://jamanetwork.com/journals/jama/fullarticle/2790247

https://catalogofbias.org/biases/collider-bias/ Collider bias = A distortion that modifies an association between an exposure and outcome, caused by attempts to control for a common effect of the exposure and outcome

# concept[].concept[].concept[].property[].valueString
Collider bias occurs when an exposure and outcome (or factors causing these) each influence a common third variable and that variable or collider is controlled for by design or analysis. In contrast, confounding occurs when an exposure and outcome have a shared common cause that is not controlled for. (JAMA 2022 Mar 14 https://jamanetwork.com/journals/jama/fullarticle/2790247)

The "third variable" affected by both variables of interest can also be a "third variable" affected by an "intermediary variable" which is affected by both variables of interest.

An analysis bias is defined as a bias related to the analytic process applied to the data.

A bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].property[].valueString
2022-07-01 vote 3-2 by Harold Lehmann, Paul Whaley, Jesus Lopez-Alcalde, Eric Harvey, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
2022-07-01 comments: Is this the same as a confounding variable? If not, please differentiate.

the first comment seems a bit confusing: does collider bias occurs when the study design controls for a variable which is influenced by both the exposure and the outcome?

I'm not sure this is correct. My understanding is that collision comes into play when effect modifiers are treated as confounders (and possibly when confounders are treated as modifiers? I don't know if it is symmetric). This reads as though it is an analysis unadjusted for confounders, with the factor causing both the cause and effect variables.

Confounding: A < B > C and A > C
Modification: A > B > C and A > C
Collision: Conditioning on B under modification rather than confounding.

# concept[].concept[].concept[].property[].valueString
2022-07-08 vote 5-0 by Jesus Lopez-Alcalde, Eric Harvey, Paul Whaley, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].definition
An analysis bias related to analysis of data before the complete dataset is available.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2022-12-23 vote 6-0 by Joanne Dehnbostel, Harold Lehmann, Yuan Gao, Jesus Lopez-Alcalde, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
An analysis bias involving use of data analyses that are not pre-specified and fully disclosed, to select analyses with desirable results.

# concept[].concept[].concept[].property[].valueString
Types of data analysis that lead to data-dredging bias include but are not limited to repeated subgroup analyses, repeated adjusted analyses, repeated analyses with different analytic models, and repeated analyses across many outcomes for many variations of defining outcomes, any of which can be done to select ("cherry-pick") the analyses that provide a desired result.  The desired result may be statistically significant findings or other specific results.

The terms "p-hacking" and "Fishing expedition" are commonly used terms to describe data-dredging practices that lead to bias and are often used to imply bias.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Janice Tufte, Paul Whaley, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
from Catalog of Bias (https://catalogofbias.org/biases/data-dredging-bias/): Data-dredging bias = A distortion that arises from presenting the results of unplanned statistical tests as if they were a fully prespecified course of analyses.

from BMJ Evidence-Based Medicine (https://ebm.bmj.com/content/27/4/209): Background: what is data dredging bias?
Data-dredging bias encompasses a number of more specific questionable practices (eg, fishing, p-hacking) all of which involve probing data using unplanned analyses and then reporting salient results without accurately describing the processes by which the results were generated.

from Wikipedia (https://en.wikipedia.org/wiki/Data_dredging): Data dredging (also known as data snooping or p-hacking) is the misuse of data analysis to find patterns in data that can be presented as statistically significant, thus dramatically increasing and understating the risk of false positives.

# concept[].concept[].concept[].property[].valueString
2022-12-09 comment:  Ioannidis, J. P. A. (2019) P values linked to null hypothesis significance testing (NHST) is the most widely (mis)used method of statistical inference. Empirical data suggest that across the biomedical literature (1990–2015), when abstracts use P values 96% of them have P values of 0.05 or less. The same percentage (96%) applies for full-text articles. 
2022-12-16 comments: Delete comma in definition (before "that"). 
p-hacking and fishing expedition aren't synonyms but data processes leading to bias. "p-hacking induced bias" maybe

# concept[].concept[].concept[].property[].valueString
2022-12-09 votes 4-0 by Yuan Gao, Mario Tristan, Eric Harvey, Harold Lehmann
2022-12-16 votes 6-1 by Philippe Rocca-Serra, Janice Tufte, Yuan Gao, Jesus Lopez-Alcalde, Mario Tristan, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-12-23 vote 6-0 by Joanne Dehnbostel, Harold Lehmann, Yuan Gao, Jesus Lopez-Alcalde, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
An analysis bias due to inadequate approach to determine the implications of modeling assumptions, missing data, or distorted data for the interpretation of research results.

# concept[].concept[].concept[].property[].valueString
ROBIS 4.5 Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses?

# concept[].concept[].concept[].property[].valueString
Sensitivity analysis is the process of accounting for the implications of missing or distorted data or modeling assumptions. The purpose of sensitivity analysis is to assess the robustness of findings given plausible variations in the context.

Methods of sensitivity analysis to account for missing data include but are not limited to best-case scenario, worst-case scenario, and last-observation-carried-forward. Methods of sensitivity analysis to account for distorted data include but are not limited to intention-to-treat analysis, per-protocol analysis, and completer analysis.   Methods of sensitivity analysis to account for modeling assumptions include but are not limited to variations in prior probabilities and changes in the statistical model.

The targets and types of sensitivity analyses needed depend on the research question, the research design, the data, modeling assumptions (both verifiable and unverifiable from the data), and the context of the results interpretation. The adequacy of sensitivity analysis is assessed on the basis of the targets and types of sensitivity analyses reported.

The term 'inadequate sensitivity analysis' matches the ROBIS signaling question 4.5 ''Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses?'

A funnel plot may be used to detect missing data due to publication bias. Although funnel plot asymmetry has been equated with publication bias, the funnel plot displays a tendency for the intervention effects estimated in smaller studies to differ from those estimated in larger studies, and such small-study effects may be due to reasons other than publication bias. (Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629-634.) Consider also [inadequate accounting for heterogeneity](https://fevir.net/resources/CodeSystem/27270#SEVCO:00347).

An inadequate sensitivity analysis does not result in a bias in the effect estimates, but may result in a bias in the interpretations derived from the effect estimates.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Saphia Mokrane, Homa Keshavarz, Joanne Dehnbostel, Harold Lehmann, Airton Stein

# concept[].concept[].concept[].property[].valueString
2024-07-05 vote 4-1 by C P Ooi, Sheyu Li, Lenny Vasanthan, Eric Harvey, Harold Lehmann
2024-07-12 vote 6-1 by Paul Whaley, Cauê Monaco, Sheyu Li, Philippe Rocca-Serra, Lenny Vasanthan, Saphia Mokrane, Eric Harvey
2024-07-19 vote 5-1 by Carlos Alva-Diaz, Sheyu Li, Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].property[].valueString
2024-07-05 comments re: "inadequate sensitivity analysis" = "A synthesis bias due to inadequate approach to determine the magnitude or implications of missing or distorted data."
1N) My understanding of the main purpose of sensitivity analysis can be testing the robustness of the findings and determining  its impact factors. 
2) (The paragraph on funnel plot sounds more textbooky/prescriptive information than we have generally provided, but I'm not against including it.)

2024-07-12 comment re: "inadequate sensitivity analysis" = "An analysis bias due to inadequate approach to determine the implications of missing data, distorted data, or modeling assumptions."
1N) Need explanation regarding the term 'inadequate', which reflects clinical impact of this inadequate. 

2024-07-19 comments re: "inadequate sensitivity analysis" = "An analysis bias due to inadequate approach to determine the implications of missing data, distorted data, or modeling assumptions."
1N)
The sensitivity analyses achieves multiple targets including but not limited to the quoted ones. But all try to make sure the robustness of the findings given heterogeneous research approaches. For the definition, it is good to quote the definition of sensitivity analysis, especially its purpose. 

Also, inadequate sensitivity analysis is rather a bias, but a failure to reach to purpose of sensitivity analysis. 
2Y)
I am not suggesting changes, but I had to ask chatGPT today about sensitivity analysis. This was its response:

Sensitivity analysis is a technique used in the context of analyzing real-world data to assess how the results of a study or model are affected by changes in the input parameters or assumptions. It helps to understand the robustness of the findings and identify which variables have the most significant impact on the outcomes. Sensitivity analysis is particularly useful in fields such as economics, healthcare, environmental science, and engineering, where models often rely on uncertain or variable data inputs.

Key Aspects of Sensitivity Analysis
Purpose:

To evaluate the stability and reliability of the results.
To identify critical variables that significantly influence the outcomes.
To assess the impact of uncertainty in input parameters on the conclusions.
Applications:

In healthcare, sensitivity analysis can be used to determine how different clinical assumptions affect health outcomes or cost-effectiveness in medical studies.
In environmental science, it can help assess the effect of varying environmental parameters on pollution models.
In economics, it evaluates how changes in economic indicators impact financial models or forecasts.
Types of Sensitivity Analysis:

Deterministic Sensitivity Analysis: This involves systematically changing one parameter at a time while keeping others constant to observe the effect on the outcome.
Probabilistic Sensitivity Analysis: This involves changing multiple parameters simultaneously, often using statistical distributions to model uncertainty and variability in the inputs.
Scenario Analysis: Examining different possible future scenarios by varying several parameters together to understand potential outcomes under different conditions.
Local Sensitivity Analysis: Focuses on small changes around a baseline value of the parameters to assess local impact.
Global Sensitivity Analysis: Assesses the impact of varying all parameters over their entire range to understand the overall influence on the model.
re: "inadequate sensitivity analysis" = "An analysis bias due to inadequate approach to determine the implications of missing data, distorted data, or modeling assumptions."

# concept[].concept[].concept[].property[].valueString
2024-07-26 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].concept[].display
final model not corresponding to multivariable analysis

# concept[].concept[].concept[].definition
An analysis bias in which the predictors and coefficients in the final model do not match the predictors and coefficients reported in the multivariable analysis.

# concept[].concept[].concept[].property[].valueString
from PROBAST:
4.9 Do predictors and their assigned weights in the final model correspond to the results from the reported multivariable analysis? (Model development studies only)

Predictors and coefficients of the final developed model, including intercept or baseline components, should be fully reported to allow others to correctly apply the model to other individuals. Mismatch between the presented final model and the reported results from the multivariable analysis (such as the intercept and predictor coefficients) is frequent. A review of prediction models in cancer in 2010 found that only 13 of 38 final prediction model equations (34%) used the same predictors and coefficients as the final presented multivariable analyses, 8 used the same predictors but different coefficients, 11 used neither the same coefficients nor the same predictors, and 6 used an unclear method to derive the final prediction model from the presented results of the multivariable analysis (121).

Bias can arise when the presented final model and the results reported from the multivariable analysis do not match. One way this can occur is when nonsignificant predictors are dropped from a larger model to arrive at a final presented model but the predictor coefficients from the larger model are used to define the final model, which are no longer correct. When predictors are dropped from a larger model, it is important to reestimate all predictor coefficients of the smaller model because the latter has become the final model. These newly estimated predictor coefficients are likely different even if nonsignificant or irrelevant predictors from the larger model are dropped.

When a study reports a final model in which both predictors and regression coefficients correspond to the reported results of the multivariable regression analysis or model, this question should be answered as Y. If the final model is based only on a selection of predictors from the reported multivariable regression analysis without refitting the smaller model, it should be answered as N or PN. When no information is given on the multivariable modeling from which predictors and regression coefficients are derived, it should be answered as NI.

This signaling question is not about detecting improper methods of selecting predictors for the final model; such methods are addressed in signaling question 4.5.

# concept[].concept[].concept[].property[].valueString
This type of bias is applicable to model development studies and model selection within other study designs.

# concept[].concept[].concept[].property[].valueString
2023-12-08 vote 5-0 by Brian S. Alper, Harold Lehmann, Javier Bracchiglione, Yasser Sami Amer, Eric Harvey

# concept[].concept[].concept[].display
cognitive interpretive bias affecting analysis

# concept[].concept[].concept[].definition
A bias related to the analytic process due to the subjective nature of human interpretation.

# concept[].concept[].concept[].property[].valueString
The Cognitive Interpretive Bias affecting analysis can be mitigated by masking the analyst as to the assignments for the groups, and by specification of the analysis prior to data availability.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-11-18 vote 6-0 by Mahnoor Ahmed, Yuan Gao, Harold Lehmann, Jesus Lopez-Alcalde, Paul Whaley, Eric Harvey

# concept[].concept[].concept[].concept[].display
cognitive interpretive bias affecting analysis selection

# concept[].concept[].concept[].concept[].definition
A bias related to selection of the analysis due to the subjective nature of human interpretation.

# concept[].concept[].concept[].concept[].property[].valueString
Bias related to selection of the analysis is defined as an analysis bias due to inappropriate choice of analysis methods before the analysis is applied.

The Cognitive Interpretive Bias affecting analysis selection can be mitigated by masking the analyst as to the assignments for the groups, and by specification of the analysis prior to data availability.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-29 vote 5-0 by Janice Tufte, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].display
availability bias affecting analysis selection

# concept[].concept[].concept[].concept[].concept[].definition
A Cognitive Interpretive Bias due to the use of information which is most readily available, rather than information which is most representative, affecting analysis selection.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Janice Tufte, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Catalogue of Bias:
Availability bias
A distortion that arises from the use of information which is most readily available, rather than that which is necessarily most representative.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 vote 4-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey
2022-08-19 vote 5-1 by Paul Whaley, Mario Tristan, Harold Lehmann, Janice Tufte, Eric Harvey, Philippe Rocca-Serra

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-08-12 comment: Clarify as to whether this is exclusively about cognitive availability? Seems ambiguous in current phrasing. Would suggest comment for application to make clear specific circumstances in which this applies.

2022-08-19 comment: The definition is ambiguous about whether limits on access to the information is cognitive (e.g. familiarity) or otherwise. Also, the definition specifies "information" when the thing being selected is a technique for analysing information.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Selection of inappropriate data or variables for analysis is an availability bias when the appropriate data or variables are not readily available to the analyst and therefore the appropriate analysis is not selected.

Selection of an inappropriate analysis due to familiarity with the analytic techniques is an availability bias when the appropriate technique is unfamiliar and therefore not selected. 

The term "Availability bias affecting analysis selection" is about selection of the analysis and not about missing data.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-08-26 vote 7-0 by nisha mathew, Philippe Rocca-Serra, Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, Cauê Monaco, Eric Harvey

# concept[].concept[].concept[].concept[].display
cognitive interpretive bias affecting execution of the analysis

# concept[].concept[].concept[].concept[].definition
A bias in processing of data due to the subjective nature of human interpretation.

# concept[].concept[].concept[].concept[].property[].valueString
Bias in processing of data is defined as an analysis bias due to inappropriate decisions pertaining to preparation of data for analysis and/or conduct of the analysis.

This bias may be mitigated by the partial masking or blinding of the individuals conducting the analysis.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte, Paul Whaley, Yuan Gao, Harold Lehmann, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
2022-12-02 vote 6-0 by Mario Tristan,  Yuan Gao, Mahnoor Ahmed, Muhammad Afzal, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias affecting execution of the analysis due to the analyst's awareness of the participants' status with respect to the variables defining the comparison groups.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Janice Tufte, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-12-02 vote 5-0 by Mario Tristan, Mahnoor Ahmed, Muhammad Afzal, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-12-02 comment: Should it be participants' statuses --- EWG discussion notes that "status" can be used for the plural

# concept[].concept[].concept[].definition
An analysis bias in which the weights used in model construction do not align with the target of estimation or estimand.

# concept[].concept[].concept[].property[].valueString
This bias often occurs with the omission of sampling weights in a model or in the process of trying to mitigate misrepresentation of a population due to sampling.
One example is use of an unweighted model with National Health and Nutrition Examination Survey (NHANES) data.
This bias occurs when attempting to reweight imbalanced classes in a model to make them representative of the source population, when weights drive estimation away from the target.

# concept[].concept[].concept[].property[].valueString
2023-10-13 vote 6-0 by Louis Leff, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey, Paul Whaley, Janice Tufte

# concept[].concept[].concept[].definition
A bias in the conduct of an analysis combining two or more studies or datasets.

# concept[].concept[].concept[].property[].valueString
A synthesis bias results from methods used to select, manipulate or interpret data for evidence synthesis.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2024-05-17 vote 5-0 by Saphia Mokrane, Lenny Vasanthan, Sheyu Li, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].display
bias related to selection of the analytic framework for synthesis

# concept[].concept[].concept[].concept[].definition
A synthesis bias resulting from an analytic approach that is not suitable for the included studies.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 4.3 Was the synthesis appropriate given the nature and similarity in the research questions, study designs and outcomes across included studies?

# concept[].concept[].concept[].concept[].property[].valueString
The term 'synthesis bias related to selection of the analytic framework' used for a systematic review or synthesis is equivalent to the term  [bias related to selection of the analytic framework](#SEVCO:00378) used for a single study. An analytic framework is the model, scaffolding, or organizational representation of concepts used in analyzing the data. The concepts included in an analytic framework may involve data, variables, formulas, assumptions, and adjustments.
If the analytic framework selected for synthesis does not match the data and research question, there is a risk of distorted results which constitutes bias.
The term 'bias related to selection of the analytic framework for synthesis' matches the ROBIS signaling question 4.3 'Was the synthesis appropriate given the nature and similarity in the research questions, study designs and outcomes across included studies?'

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-06-07 vote 5-1 by Sean Grant, Sheyu Li, Lenny Vasanthan, Harold Lehmann, Eric Harvey, Carlos Alva-Diaz
2024-06-14 vote 8-1 by Carlos Alva-Diaz, Yaowaluk Ngoenwiwatkul, Homa Keshavarz, Sean Grant, Sheyu Li, Eric Harvey, Lenny Vasanthan, Harold Lehmann, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2024-06-07 comment:  The term itself is new to me: is there a more established term?

2024-06-14  comments re: "bias related to selection of the analytic framework for synthesis" = "A synthesis bias resulting from an analytic approach that is not suitable for the included studies."
1N) As above, I understand the concept though I am not clear how this is a "bias"? "Not suitable" also sounds like poor study execution rather than a bias.
2) Have we been consistent in other Comments for application in matching the signaling questions?

# concept[].concept[].concept[].concept[].property[].valueString
2024-06-21 vote 8-0 by Cauê Monaco, Lenny Vasanthan, Homa Keshavarz, Yaowaluk Ngoenwiwatkul, Harold Lehmann, Sean Grant, Eric Harvey, Carlos Alva-Diaz

# concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate approach to determine the magnitude, cause, or implications of variation among studies.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 4.4 Was between-study variation (heterogeneity) minimal or addressed in the synthesis?

# concept[].concept[].concept[].concept[].property[].valueString
Adequate accounting for variation among studies includes measuring the variation among studies, determining if substantial variation is systematic or random, and addressing the implications of substantial variation if present.
The term 'inadequate accounting for heterogeneity' matches the ROBIS signaling question 4.4 'Was between-study variation (heterogeneity) minimal or addressed in the synthesis?'

[Bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00001) is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]). [Synthesis bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00026) is defined as a bias in the conduct of an analysis combining two or more studies or datasets. An inadequate approach to the accounting for heterogeneity in the conduct of an evidence synthesis can introduce or obscure a systematic distortion in research results.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Homa Keshavarz, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-06-21 vote 5-2 by Cauê Monaco, Lenny Vasanthan, Homa Keshavarz, Yaowaluk Ngoenwiwatkul, Harold Lehmann, Sean Grant, Eric Harvey
2024-06-28 vote 6-1 by Harold Lehmann, Lenny Vasanthan, Homa Keshavarz, Sean Grant, Philippe Rocca-Serra, Eric Harvey, Sheyu Li

# concept[].concept[].concept[].concept[].property[].valueString
2024-06-21 comments re: "bias related to accounting for heterogeneity" = "A synthesis bias due to inadequate accounting for variation among studies."
1N) Missing text: "The term " matches..."
2N) Not clear how this is a "bias" rather than a low-quality review?

2024-06-28 comment re: "inadequate accounting for heterogeneity" = "A synthesis bias due to inadequate approach to determine the magnitude, cause, or implications of variation among studies."
This again does not sound like a "bias" to me. Perhaps it contributes to a bias, though it sounds more like poor execution of an evidence synthesis rather than a "bias"

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-05 vote 6-0 by Cauê Monaco, C P Ooi, Sheyu Li, Lenny Vasanthan, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].display
inadequate accounting for bias in constituent studies

# concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate approach to determine the risks of bias in the studies selected for synthesis or the implications of those risks.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria?
ROBIS 3.5 Were efforts made to minimise error in risk of bias assessment?
ROBIS 4.6 Were biases in primary studies minimal or addressed in the synthesis?

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Homa Keshavarz, Airton Stein

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-26 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].concept[].display
inadequate criteria for methodologic quality assessment

# concept[].concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate criteria to determine the risks of bias in the studies selected for synthesis.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The term 'inadequate criteria for methodologic quality assessment' matches the ROBIS signaling question 3.4 'Was risk of bias (or methodological quality) formally assessed using appropriate criteria?'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Homa Keshavarz, Airton Stein

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-07-26 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].concept[].display
inadequate process for methodologic quality assessment

# concept[].concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate process to determine the risks of bias in the studies selected for synthesis.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.5 Were efforts made to minimise error in risk of bias assessment?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The term 'inadequate process for methodologic quality assessment' matches the ROBIS signaling question 3.5 'Were efforts made to minimise error in risk of bias assessment?'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Homa Keshavarz, Airton Stein

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-07-26 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].concept[].display
inadequate adjusting for bias in constituent studies

# concept[].concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate approach to determine the implications of bias in the studies selected for synthesis.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The term 'inadequate adjusting for bias in constituent studies' matches the ROBIS signaling question 4.6 'Were biases in primary studies minimal or addressed in the synthesis?'

# concept[].concept[].concept[].concept[].concept[].property[].valueString
ROBIS 4.6 Were biases in primary studies minimal or addressed in the synthesis?

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Homa Keshavarz, Airton Stein

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2024-07-26 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate process to select and abstract the data from the included studies or datasets.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.1 Were efforts made to minimise error in data collection?

# concept[].concept[].concept[].concept[].property[].valueString
The term 'inadequate process for data extraction' matches the ROBIS signaling question 3.1 'Were efforts made to minimise error in data collection?'

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Airton Stein, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-02 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Airton Tetelbom Stein, Eric Harvey

# concept[].concept[].concept[].concept[].display
inadequate study characteristics available for results interpretation

# concept[].concept[].concept[].concept[].definition
A synthesis bias due to inadequate availability of information regarding design, data collection, analysis or reporting for the study results or datasets collected for synthesis.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.2 Were sufficient study characteristics available for both review authors and readers to be able to interpret the results?

# concept[].concept[].concept[].concept[].property[].valueString
The term 'inadequate study characteristics available for results interpretation' matches the ROBIS signaling question 3.2 'Were sufficient study characteristics available for both review authors and readers to be able to interpret the results?'

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Airton Stein, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-02 vote 5-0 by Harold Lehmann, Homa Keshavarz, Lenny Vasanthan, Airton Tetelbom Stein, Eric Harvey

# concept[].concept[].concept[].concept[].display
bias related to selection of the data for synthesis

# concept[].concept[].concept[].concept[].definition
A synthesis bias due to inappropriate choice of study results included in the synthesis.

# concept[].concept[].concept[].concept[].property[].valueString
ROBIS 3.3 Were all relevant study results collected for use in the synthesis?

# concept[].concept[].concept[].concept[].property[].valueString
Selection of study results may occur before or after data extraction and transformation. The process of extracting results from a study involves identifying, collecting, and recording the findings (quantitative or qualitative) that are relevant for synthesis. The process of transformation may include unit of measure changes, other data harmonization, or statistical transformations so the data can be integrated into a meta-analysis.
Another reason for not including a relevant study result is the failure to contact the study investigators for missing data.
If the data selected for synthesis does not match the data that is available, there is a risk of distorted results which constitutes bias.
The term 'bias related to selection of the data for synthesis' matches the ROBIS signaling question 3.3 'Were all relevant study results collected for use in the synthesis?'

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Airton Stein

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-09 vote 6-1 by Harold Lehmann, Homa Keshavarz, Sheyu Li, Sean Grant, Eric Harvey, Lenny Vasanthan, Airton Tetelbom Stein
2024-08-16 vote 5-1 by Cauê Monaco, Bhagvan Kommadi, Jennifer Hunter, Eric Harvey, Harold Lehmann, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-09 comment re: "bias related to selection of the data for synthesis" = "A synthesis bias due to inappropriate choice of study results included in the synthesis before the synthesis is applied."
1N)
Once again, what is the difference between this bias and selection bias?  Similar comments with above. 


2024-08-16 comment re: "bias related to selection of the data for synthesis" = "A synthesis bias due to inappropriate choice of study results included in the synthesis."
1N)
I'm unsure that "and appropriate for synthesis" is correct. For instance, an included study may have measured the outcome of interest but not reported the results in a way that can be used in the planned meta-analysis (e.g., only the p value is reported, or no SDM/SE is reported). Even though the results cannot be used (i.e., are not appropriate) for the meta-analysis, there is still a risk of distorted results which constitutes bias. Perhaps I am mistaken, and this bias only refers to errors of judgement by the reviewers.

Do we need to mention some examples relevant to selection bias arising after data extraction? e.g., failing to undertake appropriate statistical transformations so that the reported results can be used in a meta-analysis.

Another reason for not including a relevant study result is the failure to contact the study investigators for missing data. Does this need to be mentioned here or is it covered under a different term? If this term is supposed to map to  ROBIS 3.3., then this is another example of not collecting relevant results.

2024-08-23 comment re: "bias related to selection of the data for synthesis" = "A synthesis bias due to inappropriate choice of study results included in the synthesis."
1Y)
It might be good to mention conditions when the study selection happens after data extraction and transformation.

One question - if anyone is working on an update of a review and if the previous author team havent selected the appropriate data in their included studies and the current team have published an update without updating the study results of the previously included studies, how do we handle that here?

# concept[].concept[].concept[].concept[].property[].valueString
2024-08-23 vote 9-0 by Carlos Alva-Diaz, Elma OMERAGIC, Lenny Vasanthan, Harold Lehmann, Philippe Rocca-Serra, Eric Harvey, Sean Grant, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].concept[].display
inappropriate evaluation of predictive model performance measures

# concept[].concept[].concept[].definition
An analysis bias in which the method for analysis of a performance measure (such as calibration or discrimination) is not adequate or suitable for the predictive model.

# concept[].concept[].concept[].property[].valueString
According to PROBAST explanation, to fully gauge the predictive performance of a model, reviewers must assess both model calibration and discrimination (such as the c-index) addressing the entire range of the model-predicted probabilities. (https://www.acpjournals.org/doi/10.7326/M18-1377)

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2023-10-06 vote 5-0 by Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey, Paul Whaley, Janice Tufte

# concept[].concept[].concept[].concept[].display
inappropriate evaluation of calibration of predictive model

# concept[].concept[].concept[].concept[].definition
An analysis bias in which the method for analysis of calibration is not adequate or suitable for the predictive model.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-06 vote 5-0 by Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey, Paul Whaley, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-06 comment: Is the bias because an analyst prefers one model over another when there might be a more appropriate one ( perhaps the analyst is not familiar with?)

# concept[].concept[].concept[].concept[].display
inappropriate evaluation of discrimination of predictive model

# concept[].concept[].concept[].concept[].definition
An analysis bias in which the method for analysis of discrimination is not adequate or suitable for the predictive model.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-13 vote 5-0 by Louis Leff, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey, Paul Whaley

# concept[].concept[].concept[].concept[].definition
An analysis bias, specific to predictive model development studies, in which strategies to mitigate overfitting are not adequately applied.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
Predictive model performance measures (calibration and discrimination) may be misinterpreted if there are no strategies to mitigate overfitting.  This applies to development studies without external validation studies. Strategies to mitigate overfitting may include penalization/regularization, k-fold cross validation, train-test/validation split, etc.

From the PROBAST explanation (https://www.acpjournals.org/doi/10.7326/M18-1377):  "quantifying the predictive performance of a model on the same data from which the model was developed (apparent performance) tends to give optimistic estimates of performance due to overfitting—that is, the model is too much adapted to the development data set. This optimism is higher when any of the following are present: too few outcome events in total, too few outcome events relative to the number of candidate predictors (small EPV), dichotomization of continuous predictors, use of predictor selection strategies based on univariable analyses, or use of traditional stepwise predictor selection strategies (for example, forward or backward selection) in multivariable analysis in small data sets (small EPV)"

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-06 vote 3-1 by Jesus Lopez-Alcalde, Eric Harvey, Paul Whaley, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-06 comments: I am not sure about having a preferred term that actually consists of two terms - overfit and optimism. Is one a synonym of the other?
Optimism- being too over optimistic and fitting things into the model that really were not defined early on? (adding inappropriate data that can skew the outcomes?)

# concept[].concept[].concept[].concept[].property[].valueString
2023-10-20 vote 5-0 by Muhammad Afzal, Eric Harvey, Harold Lehmann, Louis Leff, Joanne Dehnbostel

# concept[].concept[].definition
A bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].property[].valueString
CoB: Reporting biases = A systematic distortion that arises from the selective disclosure or withholding of information by parties involved in the design, conduct, analysis, or dissemination of a study or research findings (https://catalogofbias.org/biases/reporting-biases/) also notes: The Dictionary of Epidemiology defines reporting bias as the “selective revelation or suppression of information (e.g., about past medical history, smoking, sexual experiences) or of study results.”
The Cochrane Handbook states it arises “when the dissemination of research findings is influenced by the nature and direction of results.”
The James Lind Library states “biased reporting of research occurs when the direction or statistical significance of results influence whether and how research is reported.”
QUIPS: The Statistical Analysis and Reporting domain addresses the appropriateness of the study’s statistical analysis and completeness of reporting. It helps the assessor judge whether results are likely to be spurious or biased because of analysis or reporting. To make this judgment, the assessor considers the data presented to determine the adequacy of the analytic strategy and model-building process and investigates concerns about selective reporting. Selective reporting is an important issue in prognostic factor reviews because studies commonly report only factors positively associated with outcomes. A study would be considered to have low risk of bias if the statistical analysis is appropriate for the data, statistical assumptions are satisfied, and all primary outcomes are reported.
ROB2 = This domain addresses bias that arises because the reported result is selected (based on its direction, magnitude or statistical significance) from among multiple intervention effect estimates that were calculated by the trial investigators. We call this bias in selection of the reported result. Consideration of risk of bias requires distinction between:
• An outcome domain. This is a state or endpoint of interest, irrespective of how it is measured (e.g. severity
of depression);
• An outcome measurement. This is a specific way in which an outcome domain is measured (e.g. measurement of depression using the Hamilton rating scale 6 weeks after starting intervention); and 
• An outcome analysis. This is a specific result obtained by analysing one or more outcome measurements (e.g. the difference in mean change in Hamilton rating scale scores from baseline to 6 weeks between experimental and comparator groups). This domain does not address bias due to selective non-reporting (or incomplete reporting) of outcome domains that were measured and analysed by the trial investigators (115). For example, deaths of trial participants may be recorded by the trialists, but the reports of the trial might contain no mortality data, or state only that the intervention effect estimate for mortality was not statistically significant. Such bias puts the result of a synthesis at risk because results are omitted based on their direction, magnitude or statistical significance. It should therefore be addressed at the review level, as part of an integrated assessment of the risk of reporting bias (116).
ROBINS-I = Bias in selection of the reported result

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Yuan Gao

# concept[].concept[].property[].valueString
Bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]).

# concept[].concept[].property[].valueString
2022-10-21 vote 7-0 by Philippe Rocca-Serra, Harold Lehmann, Joanne Dehnbostel, Mario Tristan, Brian Alper, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].definition
A reporting bias due to inappropriate selection of the results or research findings that are reported.

# concept[].concept[].concept[].property[].valueString
MASTER-31. There was no discernible data dredging or selective reporting of the outcomes

# concept[].concept[].concept[].property[].valueString
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Yuan Gao, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2023-01-06 vote 5-0 by Harold Lehmann, Yuan Gao, Janice Tufte, Eric Harvey, Mario Tristan

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of which outcomes are reported within results or research findings.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Paul Whaley, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-13 vote 5-0 by Harold Lehmann, Joanne Dehnbostel, Paul Whaley, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of which outcome measures are reported for an outcome.

# concept[].concept[].concept[].concept[].concept[].designation[].value
selective outcome measure reporting within outcome domain

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Selective outcome measure reporting may be considered a type of selective outcome reporting in which the measurement method for determination of the outcome is interpreted as a distinct outcome.
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Paul Whaley

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2023-01-13 vote 5-0 by Harold Lehmann, Joanne Dehnbostel, Paul Whaley, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of subsets of groups of participants for which results or research findings are reported.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.
Selective subgroup reporting relates to choice of attributes of participants within cohorts, for example reporting limited to male patients.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Paul Whaley, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-27 vote 7-0 by Janice Tufte, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Yuan Gao, Paul Whaley, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of comparison groups for which results or research findings are reported.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.
Selective comparison reporting relates to choice of cohort definitions, for example an intention-to-treat analysis (as-randomized analysis) vs. an as-treated analysis.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Paul Whaley, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-20 vote 2-1 by Yuan Gao, Paul Whaley, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-20 comment: I don't see enough of a connection between the term (selective comparison) and the definition, which does not seem to talk about comparisons.

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-27 vote 6-0 by Janice Tufte, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Yuan Gao, Eric Harvey

# concept[].concept[].concept[].concept[].display
selective analysis reporting from repeated analyses at multiple times

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of which analyses are reported for an outcome that was analyzed at multiple points in time in a longitudinal study.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-27 vote 6-0 by Janice Tufte, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Yuan Gao, Eric Harvey

# concept[].concept[].concept[].concept[].display
selective analysis reporting from multiple analytic models

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of which analyses are reported for an outcome that was analyzed in multiple ways.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.
Adjustment reporting bias, or selective reporting of adjusted estimates, is a type of selective analysis reporting from multiple analytic models.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-02-10 vote 6-0 by Cauê Monaco, Paul Whaley, Janice Tufte, Brian S. Alper, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A selective reporting bias due to inappropriate selection of which thresholds (used for definitions of the variables) are reported.

# concept[].concept[].concept[].concept[].property[].valueString
A selective reporting bias is a reporting bias due to inappropriate selection of the results or research findings that are reported.
A [reporting bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00023) is a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-02-10 vote 5-1 by Cauê Monaco, Paul Whaley, Janice Tufte, Brian S. Alper, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2023-02-10 comment: I'm not clear how the definition relates specifically to reporting bias.

# concept[].concept[].concept[].concept[].property[].valueString
2023-02-24 vote 7-0 by Harold Lehmann, Yasser Sami Amer, Mario Tristan, Paul Whaley, Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].display
cognitive interpretive bias in reporting

# concept[].concept[].concept[].definition
A distortion in the representation of study results or research findings due to the subjective nature of human interpretation.

# concept[].concept[].concept[].designation[].value
cognitive interpretive bias for reporting

# concept[].concept[].concept[].property[].valueString
CoB: Spin bias = The intentional or unintentional distorted interpretation of research results, unjustifiably suggesting favourable or unfavourable findings that can result in misleading conclusions (https://catalogofbias.org/biases/spin-bias/)

# concept[].concept[].concept[].property[].valueString
Reporting bias is defined as a bias due to distortions in the selection of or representation of information in study results or research findings.
Cognitive interpretive bias in reporting is about interpretation of the results rather than the choice of which results are presented (which would be Selective Reporting Bias).
Cognitive interpretive biases in reporting include selective theory reporting, confirmation bias, bias of rhetoric, novelty bias, popularity bias, and positive results bias.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Harold Lehmann, Janice Tufte, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-02-10 vote 4-1 by Cauê Monaco, Paul Whaley, Jesus Lopez-Alcalde, Janice Tuft, Eric Harvey

# concept[].concept[].concept[].property[].valueString
2023-02-10 comment: I think the definition is sound but the comment for application should be extended to make it clearer that this is about interpretation of the results rather than the choice of which results are presented.

# concept[].concept[].concept[].property[].valueString
2023-02-24 vote 7-0 by Harold Lehmann, Yasser Sami Amer, Mario Tristan, Paul Whaley, Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey

# concept[].concept[].concept[].concept[].display
interpretation of results not addressing potential for bias

# concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias in reporting whereby the reported interpretation of results does not adequately address potential for bias.

# concept[].concept[].concept[].concept[].designation[].value
interpretation of results not addressing risk of bias

# concept[].concept[].concept[].concept[].property[].valueString
Reporting bias is defined as a bias due to distortions in the selection of or representation of information in study results or research findings.
Cognitive interpretive bias in reporting is defined as a distortion in the representation of study results or research findings due to the subjective nature of human interpretation.
Interpretation of results not addressing potential for bias occurs when there is an absence of risk of bias assessment or incomplete inclusion of a risk of bias assessment in the interpretation of findings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-03-03 vote 6-0 by A.G. Radhika, Cauê Monac, Janice Tufte, Harold Lehmann, Yasser Sami Amer, Eric Harvey

# concept[].concept[].concept[].concept[].display
results emphasized based on statistical significance

# concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias in reporting whereby results with statistical significance are given exaggerated attention.

# concept[].concept[].concept[].concept[].property[].valueString
This bias may occur in several ways. Results may be interpreted as "positive" or "conclusive" if below the significance threshold and "negative" or "inconclusive" if above the significance threshold without proper interpretation of the meaning of the significance threshold.  Results may be selectively emphasized in overall summarization of the results based on whether or not they are under the significance threshold. Results may be interpreted based on statistical significance instead of clinical significance, or results may misrepresent statistical significance and clinical significance as synonymous.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-04-07 vote 5-0 by Paul Whaley, Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2023-04-07 comments:
I support this term as written, although I would suggest that we consider adding that assessment of statistical significance without assessing clinical significance often  leads to this bias.
I might suggest adding to Comment for application: "Another mis-interpretation is when statistical significance confused with clinical significance."

# concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias in reporting due to the influence of an individual’s ideas, beliefs or hypotheses.

# concept[].concept[].concept[].concept[].property[].valueString
Reporting bias is defined as a bias due to distortions in the selection of or representation of information in study results or research findings.
Cognitive interpretive bias in reporting is defined as a distortion in the representation of study results or research findings due to the subjective nature of human interpretation.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-03-10 vote 8-0 by Joanne Dehnbostel, Mario Tristan, Harold Lehmann, Jesus Lopez-Alcalde, A.G. Rradhika, Janice Tufte, Eric Harvey, Cauê Monaco

# concept[].concept[].concept[].concept[].definition
A cognitive interpretive bias in reporting due to a mismatch between what the observed data represent and the results that were reported.

# concept[].concept[].concept[].concept[].property[].valueString
Reporting bias is defined as a bias due to distortions in the selection of or representation of information in study results or research findings.
Cognitive interpretive bias in reporting is defined as a distortion in the representation of study results or research findings due to the subjective nature of human interpretation.
In the assessment of systematic reviews, this type of bias can be phrased as "Relevance of studies to research question not appropriately considered".

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Paul Whaley, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2023-4-14 by Janice Tufte, Eric Harvey, Harold Lehmann, Joanne Dehnbostel, Jesus Lopez-Alcalde

# concept[].concept[].concept[].definition
A reporting bias due to publication or reporting of results or research findings that change in subsequent reports.

# concept[].concept[].concept[].property[].valueString
One form of Early dissemination bias is the reporting of results in preprints or early versions during the peer review and publication process not matching the subsequent reports.
Another form of Early dissemination bias is the reporting of interim results (even if fully peer reviewed) when a study is ongoing and more data will be analyzed for the final results.
This bias may result from failure to disclose that the results are preliminary or subject to change.
This definition is not meant to indicate that preprints are inherently biased.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal, Paul Whaley

# concept[].concept[].concept[].property[].valueString
2023-04-07 vote 3-1 by Eric Harvey, Harold Lehmann, Paul Whaley, Jesus Lopez-Alcalde

# concept[].concept[].concept[].property[].valueString
2023-04-14 comments:
Should we make clear in Comment for Application that preprints represent *potential* bias, because preprinting does not prima facie mean bias?
It seems to me that the bias falls where the results do ot carefully convey that they are preliminary or early AND not to be read as final results -maybe could be word smithed Do you mean someone is reporting without full disclosure
2023-04-07 comments:
I would suggest "One form of potential Premature...", since prima facie, premature reporting does not *have* to be biased.
I feel that "reporting bias" has the same issue of being semantically loaded as "publication bias" - the problem is premature dissemination of results, via reporting them, publishing them, putting them in a press release, etc. So maybe "premature dissemination bias" could be considered as the preferred term? And then we could even consider "early dissemination bias" as that feels more objective than "premature", now that it is phrased this way.

# concept[].concept[].concept[].property[].valueString
2023-04-14 by Janice Tufte, Eric Harvey, Harold Lehmann, Jesus Lopez-Alcalde, Joanne Dehnbostel

# concept[].concept[].concept[].definition
A reporting bias resulting from intentional misrepresentation of any part of the study.

# concept[].concept[].concept[].property[].valueString
Examples include plagiarism, unjustified authorship, data manipulation, and intentional misrepresentation of figures and charts. Applying this code is a serious allegation of wrongdoing.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2023-04-21 vote 5-0 by Brian S. Alper, Janice Tufte, Harold Lehmann, Cauê Monaco, Eric Harvey

# concept[].concept[].concept[].definition
A reporting bias in which the interpretation of results is not adequately supported by the data collected.

# concept[].concept[].concept[].property[].valueString
Reporting bias is defined as a bias due to distortions in the selection of or representation of information in study results or research findings.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Caue Monaco, Li Wang, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2023-12-22 vote 5-0 Eric Harvey, Caue Monaco, Janice Tufte, Paul Whaley, Joanne Dehnbostel

# concept[].concept[].concept[].definition
A reporting bias in which included citations are limited to those that represent only some of the perspectives.

# concept[].concept[].concept[].property[].valueString
Catalogue of Bias (https://catalogofbias.org/biases/one-sided-reference-bias/) One-sided reference bias
When authors restrict their references to only those works that support their position.
One-sided reference bias occurs when a study author cites only publications that demonstrate one side of the picture of available evidence. This bias may arise when researchers cite publications that support their preconceptions or hypotheses, ignoring evidence that does not support their view. This can happen in any study report, but a particular problem arises when this occurs in literature reviews, which are supposed to represent a comprehensive collection of all relevant information, along with description and appraisal of quality and content. The result can be a misrepresentation of the current totality of evidence and can lead to spurious claims or needless additional research.
Catalogue of Bias Collaboration, Spencer EA, Brassey J, Heneghan C. One-sided reference bias. In: Catalogue of Bias 2017 https://www.catalogofbias.org/biases/one-sided-reference-bias

# concept[].concept[].concept[].property[].valueString
Joanne Dehnbostel, Brian S. Alper, Kenneth Wilkins, Sheyu Li, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2024-05-17 comment: "Perspective" needs to be clarified in the Comment for application. For instance, "perspective" could mean patient vs provider vs society OR in favor of drug A vs in favor of drug B. [In general, considering that there's always discussion about a term, there should be some Comment for application.]

2024-05-24 comment: Can we consider rephrasing the definition to -  A study selection bias in which included studies are limited to those that represent "only " one perspective

# concept[].concept[].concept[].property[].valueString
2024-05-17 vote 4-1 by Saphia Mokrane, Lenny Vasanthan, Sheyu Li, Eric Harvey, Harold Lehmann
2024-05-24 vote 7-0 by Homa Keshavarz, Sheyu Li, Eric Harvey, Lenny Vasanthan, Harold Lehmann, Janice Tufte, Saphia Mokrane

# concept[].concept[].concept[].property[].valueString
The term "perspective" covers a variety of contexts, such as a side of an argument or point of view.

# concept[].concept[].concept[].property[].valueString
2024-05-31 vote 5-0 by Saphia Mokrane, Lenny Vasanthan, Sheyu Li, Eric Harvey, Homa Keshavarz

# concept[].concept[].concept[].definition
A reporting bias due to insufficient reporting of methods to determine the validity of the results.

# concept[].concept[].concept[].designation[].value
inadequate reporting to assess analytic strategy

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-03-17 vote on "Inadequate Reporting Bias" 2-1 by Eric Harvey, Jesus Lopez-Alcalde, Janice Tufte
2023-03-17 comment on "Inadequate Reporting Bias": Inadequate reporting of methods is covered by another term. Recommend changing this term to "inadequate reporting of results" or deleting this term if terms covering "reporting results biases" have already been established.

# concept[].concept[].concept[].property[].valueString
2023-03-31 vote 5-0 by Harold Lehmann, Eric Harvey, Janice Tufte, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].definition
A reporting bias due to insufficient reporting of reasons for withdrawals of participants after study enrollment.

# concept[].concept[].concept[].designation[].value
inadequate explanation of participant attrition

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2023-03-31 vote 5-0 by Harold Lehmann, Eric Harvey, Janice Tufte, Paola Rosati, Jesus Lopez-Alcalde

# concept[].concept[].concept[].property[].valueString
2023-03-31 comment: Somewhere in this entry should be a link to the "withdrawal" SEVCO term. Or terms.

# concept[].concept[].definition
A bias specific to the design, conduct, analysis or reporting of qualitative research.

# concept[].concept[].property[].valueString
MMAT = “Qualitative research is an approach for exploring and understanding the meaning individuals or groups ascribe to a social or human problem” (Creswell, 2013b, p. 3).

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Caue Monaco, Li Wang

# concept[].concept[].property[].valueString
Qualitative research is a research approach that studies subjective aspects of social phenomenon, human behavior, and human perception. Qualitative research may encompass any non-quantitative method of analysis. Qualitative research often explores the meaning individuals or groups assign to concepts.

Bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]). In qualitative research, the actuality may include multiple meanings that individuals or groups assign to concepts, and there is no quantitative estimand.

# concept[].concept[].property[].valueString
2023-12-08 vote 5-1 by Cauê Monaco, Harold Lehmann, Javier Bracchiglione, Janice Tufte, Yasser Sami Amer, Eric Harvey

# concept[].concept[].property[].valueString
2023-12-08 comments:  Are the initial letters of each word really meant to be capitalized? Or they are this way by error?
As SEVCO states, bias relates to differences between the reported results and the actuality (the truth, the estimand). Qualitative research usually does not adopt a positivist approach, therefore, it does not assume there is necessarily one truth to be found. I think at some point (maybe in comments for application) this should be described.

2023-12-15 comment: Qualitative research covers more than, "subjective aspects of social phenomenon and human behavior". For instance, I'm not sure this Comment covers usability or pain perception. How about, "human perception"?

# concept[].concept[].property[].valueString
2023-12-22 Vote 5-0 Caue Monaco, Eric Harvey, Janice Tufte, Paul Whaley, Joanne Dehnbostel

# concept[].concept[].concept[].definition
A qualitative research bias in which the qualitative approach used in a study is not appropriate for the research question and problem.

# concept[].concept[].concept[].property[].valueString
Bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]). In qualitative research, the actuality may include multiple meanings that individuals or groups assign to concepts, and there is no quantitative estimand.

The qualitative approach used in a study should be appropriate for the research question and problem.

Common qualitative research approaches include (this list is not exhaustive): 

Ethnography - The aim of the study is to describe and interpret the shared cultural behavior of a group of individuals.
Phenomenology - The study focuses on the subjective experiences and interpretations of a phenomenon encountered by individuals.
Narrative research - The study analyzes life experiences of an individual or a group.
Grounded theory - Generation of theory from data in the process of conducting research (data collection occurs first).
Case study - In-depth exploration and/or explanation of issues intrinsic to a particular case. A case can be anything from a decision-making process, to a person, an organization, or a country.
Qualitative description - There is no specific methodology, but a qualitative data collection and analysis, e.g., in-depth interviews or focus groups, and hybrid thematic analysis (inductive and deductive). 

Reference - Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, Gagnon M-P, Griffiths F, Nicolau B, O’Cathain A, Rousseau M-C, Vedel I. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada. Accessed at: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf

# concept[].concept[].concept[].property[].valueString
2023-12-15 comment: (Ethnography could include grounded theory, so these are not a great pair.)

2024-01-05 comment:
I think there are some major problems with the definition and comment for application:
- First, the definition is almost the same as the term, using basically the same words (so, it adds no new knowledge to the reader).
- Second, the comment for application provides a more detailed approach to a definition than the definition itself ("The qualitative approach used in a study should be appropriate for the research question and problem").
- Third, among the common qualitative research approaches included, there are some in which the term "bias" may not be so applicable. Hierarchically, this term comes from "bias", which is defined as "A systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation])". So, we need an assumption of "actuality" for bias to exist. 
Some of the qualitative approaches described do not have that assumption. For example, phenomenology focuses on "subjective experiences", and grounded theory "generates a theory", which is not an "actuality" by itself. I think this comment is applicable to all of the terms related to qualitative bias.

# concept[].concept[].concept[].property[].valueString
2024-01-05 vote 4-1 by Javier Bracchiglione, Joanne Dehnbostel, Janice Tufte, Eric Harvey, Cauê Monaco

# concept[].concept[].concept[].property[].valueString
2024-01-19 vote 5-0 by Harold Lehmann, Brian S. Alper, Homa Keshavarz, Javier Bracchiglione, Eric Harvey

# concept[].concept[].concept[].display
bias in qualitative data collection methods

# concept[].concept[].concept[].definition
A qualitative research bias in which the data sources, the methods of data collection, and the forms of data are not adequate or appropriate to address the research question.

# concept[].concept[].concept[].property[].valueString
Bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]). In qualitative research, the actuality may include multiple meanings that individuals or groups assign to concepts, and there is no quantitative estimand.

The data sources (e.g., archives, documents), the methods of data collection (e.g., in depth interviews, group interviews, and/or observations), and the forms of the data (e.g., tape recording, video material, diary, photo, and/or field notes) should be adequate and appropriate to address the research question. The term 'bias in qualitative data collection methods' may be supplemental to other terms for types of detection bias or types of selection bias.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Caue Monaco, Li Wang

# concept[].concept[].concept[].property[].valueString
2024-01-05 vote 3-1 by Javier Bracchiglione, Janice Tufte, Eric Harvey, Cauê Monaco

# concept[].concept[].concept[].property[].valueString
2024-01-05 comment: see 'bias in qualitative research design'

# concept[].concept[].concept[].property[].valueString
2024-01-19 vote 5-0 by Harold Lehmann, Brian S. Alper, Homa Keshavarz, Javier Bracchiglione, Eric Harvey

# concept[].concept[].concept[].definition
A qualitative research bias in which the analysis approach is not appropriate for the research question and qualitative approach.

# concept[].concept[].concept[].property[].valueString
Bias is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]). In qualitative research, the actuality may include multiple meanings that individuals or groups assign to concepts, and there is no quantitative estimand.

The analysis approach should be appropriate for the research question and qualitative approach (design). The term 'bias in qualitative analysis' may be supplemental to other terms for types of analysis bias.
When interpretation is an integral part of qualitative analysis, bias in the interpretive analysis should use the term 'bias in qualitative analysis' rather than 'cognitive interpretive bias in reporting'.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Li Wang, Caue Monaco, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-12-15 comments: I suppose a "guard rail" would be, "Were the methods described adequately, that they could be reproduced", which is different from the bias itself

I am not sure why sometimes terms include the word "inadequate" and sometimes "inappropriate"

2024-01-05 comment: see 'bias in qualitative research design'

# concept[].concept[].concept[].property[].valueString
2024-01-05 vote 3-1 by Javier Bracchiglione, Janice Tufte, Eric Harvey, Cauê Monaco

# concept[].concept[].concept[].property[].valueString
2024-01-19 vote 5-0 by Harold Lehmann, Brian S. Alper, Homa Keshavarz, Javier Bracchiglione, Eric Harvey

# concept[].concept[].concept[].display
incoherence among qualitative data, analysis, and interpretation

# concept[].concept[].concept[].definition
A qualitative research bias in which there is any mismatch among hypothesis, data collected, data analysis, and results interpretation.

# concept[].concept[].concept[].property[].valueString
The term mismatch applies to an inappropriate or wrong or inadequate relationship.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Khalid Shahin, Xing Song

# concept[].concept[].concept[].property[].valueString
2024-01-19 vote 3-1 by Harold Lehmann, Homa Keshavarz, Javier Bracchiglione, Eric Harvey

# concept[].concept[].concept[].property[].valueString
2024-01-19 comment: I think the definition should not start with "There are...". Maybe it should be more straightforward: "Mismatch among (...)"
2024-02-02 comment: Does the phrase "in the study report" add value to the definition or add an unnecessary clause?
2024-02-16 comment: The definition sounds coherent, but, as it is, I would argue that other previous concepts (e.g. "study eligibility criteria not appropriate for review question") could also be interpreted by this term.

# concept[].concept[].concept[].property[].valueString
2024-03-08 vote 5-0 Homa Keshavarz, Eric Harvey, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan

# concept[].concept[].definition
A bias specific to the alignment of design, conduct, analysis or reporting of qualitative research and quantitative research within the same research project.

# concept[].concept[].property[].valueString
MMAT: Mixed methods (MM) research involves combining qualitative (QUAL) and quantitative (QUAN) methods. In this tool, to be considered MM, studies have to meet the following criteria (Creswell and Plano Clark, 2017): (a) at least one QUAL method and one QUAN method are combined; (b) each method is used rigorously in accordance to the generally accepted criteria in the area (or tradition) of research invoked; and (c) the combination of the methods is carried out at the minimum through a MM design (defined a priori, or emerging) and the integration of the QUAL and QUAN phases, results, and data

# concept[].concept[].property[].valueString
Mixed methods research is a research approach that combines both qualitative and quantitative research methods within a single study or research project. This methodology aims to provide a more comprehensive understanding of a research problem by integrating the strengths of both qualitative and quantitative research.
Examples of mixed methods research include combining surveys with in-depth interviews, using quantitative data to identify patterns and trends followed by qualitative data to explore the underlying reasons and meanings, or incorporating qualitative findings to help interpret and validate quantitative results.
Overall, mixed methods research provides a more holistic understanding of a research question by acknowledging and leveraging the strengths of both qualitative and quantitative approaches.

# concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Javier Bracchiglione, Lenny Vasanthan, Eric Harvey

# concept[].concept[].property[].valueString
2024-01-19 vote 3-1 by Harold Lehmann, Homa Keshavarz, Javier Bracchiglione, Eric Harvey

# concept[].concept[].property[].valueString
2024-01-19 comment: I would say a bias is not applied to the "coordination" (this sounds more like an administrative issue)

# concept[].concept[].concept[].definition
A mixed methods research bias in which the mixed methods approach used in a study is not appropriate for the research question and problem.

# concept[].concept[].concept[].designation[].value
inadequate rationale for mixed methods design

# concept[].concept[].concept[].property[].valueString
This signaling question in the Mixed Methods Assessment Tool (MMAT) is 5.1. Is there an adequate rationale for using a mixed methods design to address the research question? 

Common mixed methods designs include:

Convergent design
The QUAL and QUAN components are usually (but not necessarily) concomitant. The purpose is to examine the same phenomenon by interpreting QUAL and QUAN results (bringing data analysis together at the interpretation stage), or by integrating QUAL and QUAN datasets (e.g., data on same cases), or by transforming data (e.g., quantization of qualitative data).

Sequential explanatory design
Results of the phase 1 - QUAN component inform the phase 2 - QUAL component. The purpose is to explain QUAN results using QUAL findings. E.g., the QUAN results guide the selection of QUAL data sources and data collection, and the QUAL findings contribute to the interpretation of QUAN results.

Sequential exploratory design
Results of the phase 1 - QUAL component inform the phase 2 - QUAN component. The purpose is to explore, develop and test an instrument (or taxonomy), or a conceptual framework (or theoretical model). E.g., the QUAL findings inform the QUAN data collection, and the QUAN results allow a statistical generalization of the QUAL findings.

Key references: Creswell et al. (2011); Creswell and Plano Clark, (2017); O'Cathain (2010)

Reference - Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, Gagnon M-P, Griffiths F, Nicolau B, O’Cathain A, Rousseau M-C, Vedel I. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada. Accessed at: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Brian S. Alper, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].display
ineffective integration of qualitative and quantitative study components

# concept[].concept[].concept[].definition
A mixed methods research bias in which the qualitative research and quantitative research components are not adequately combined.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Brian S. Alper, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].display
inappropriate interpretation of integration of qualitative and quantitative findings

# concept[].concept[].concept[].definition
A mixed methods research bias in which the process of combining the results of the constituent analyses is flawed.

# concept[].concept[].concept[].property[].valueString
This criterion is related to meta-inference, which is defined as the overall interpretations derived from integrating qualitative and
quantitative findings (Teddlie and Tashakkori, 2009). Meta-inference occurs during the interpretation of the findings from the
integration of the qualitative and quantitative components, and shows the added value of conducting a mixed methods study
rather than having two separate studies. (Pluye et al 2018)

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Joanne Dehnbostel, Caue Monaco, Muhammad Afzal, Homa Keshavarz, Brian S. Alper, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
Pluye, P., Bengoechea, E.G., Granikov, V., Kaur, N., & Tang, D.L. (2018). A World of Possibilities in Mixed Methods: Review of the Combinations of Strategies Used to Integrate Qualitative and Quantitative Phases, Results and Data. INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES. https://www.semanticscholar.org/paper/A-World-of-Possibilities-in-Mixed-Methods%3A-Review-Pluye-Bengoechea/21f292d0cb5cc07a982b240b17ff077fe4646632

Teddlie, C. and Tashakkori, A. (2009) Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Sage, London.

# concept[].concept[].concept[].property[].valueString
2024-01-19 vote 3-0 by Harold Lehmann, Eric Harvey, Homa Keshavarz BUT then the definition was changed in conference

# concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Brian S. Alper, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].display
inadequate handling of inconsistency between qualitative and quantitative findings

# concept[].concept[].concept[].definition
A mixed methods research bias in which discrepancies in the results from the qualitative and quantitative components are not adequately addressed.

# concept[].concept[].concept[].property[].valueString
When integrating the findings from the qualitative and quantitative components, divergences and inconsistencies (also called conflicts, contradictions, discordances, discrepancies, and dissonances) can be found. It is not sufficient to only report the divergences; they need to be explained. Different strategies to address the divergences have been suggested such as reconciliation, initiation, bracketing and exclusion (Pluye et al., 2009b). (Reference - Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, Gagnon M-P, Griffiths F, Nicolau B, O’Cathain A, Rousseau M-C, Vedel I. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada. Accessed at: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf)

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2024-02-23 vote 5-0 by Homa Keshavarz, Harold Lehmann, Xing Song, Lenny Vasanthan, Eric Harvey

# concept[].concept[].definition
A bias in the design, conduct or reporting of studies or analyses intended to evaluate the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

# concept[].concept[].property[].valueString
PROBAST = ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Prediction models are sometimes described as risk prediction models, predictive models, prediction indices or rules, or risk scores.

# concept[].concept[].property[].valueString
Bias in validation assessment is often used for predictive model research and diagnostic research where optimal research design includes derivation studies and external validation studies.

A 'validation study' has a validation goal where validation goal {SEVCO:01098} is defined as a study goal with the intent to determine the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

Procedures that may be assessed in validation studies include predictive algorithms, measurement instruments, and educational materials. Internal validation is tested in populations from the source used for derivation of the procedure. External validation is tested in populations that differ from the source used for derivation of the procedure.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].property[].valueString
2023-10-27 vote 5-0 by Brian S. Alper, Eric Harvey, Yasser Sami Amer, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].definition
A bias in validation assessment using a sample source that differs from those used in the derivation of the procedure.

# concept[].concept[].concept[].property[].valueString
Validation assessment is often used for predictive model research and diagnostic research where optimal research design includes derivation studies and external validation studies.

A 'validation study' has a validation goal where validation goal {SEVCO:01098} is defined as a study goal with the intent to determine the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

Procedures that may be assessed in validation studies include predictive algorithms, measurement instruments, and educational materials. Internal validation is tested in populations from the source used for derivation of the procedure. External validation is tested in populations that differ from the source used for derivation of the procedure.

Bias in validation assessment is defined as a bias in the design, conduct or reporting of studies or analyses intended to evaluate the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

Bias in external validation assessment may be used for absence of any external validation assessment or inadequacy in external validation assessment.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-11-26 vote 5-0 by Harold Lehmann, Muhammad Afzal, Janice Tufte, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A bias in validation assessment specific to a validation assessment that uses the same sample source that was used in the derivation of the procedure.

# concept[].concept[].concept[].property[].valueString
Validation assessment is often used for predictive model research and diagnostic research where optimal research design includes derivation studies and external validation studies.

A 'validation study' has a validation goal where validation goal {SEVCO:01098} is defined as a study goal with the intent to determine the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

Procedures that may be assessed in validation studies include predictive algorithms, measurement instruments, and educational materials. Internal validation is tested in populations from the source used for derivation of the procedure. 

Model derivation is often based on a portion of data available from a sample source, and internal validation is performed using the same sample data but a different set of data. 

Whereas external validation is tested in populations that differ from the source used for derivation of the procedure, internal validation is tested in the same population.

Bias in validation assessment is defined as a bias in the design, conduct or reporting of studies or analyses intended to evaluate the reliability and/or performance of a procedure for a specific predictive, classification, measurement, or communication purpose.

Bias in internal validation assessment may be used for absence of any internal validation assessment or inadequacy in internal validation assessment. A common cause of bias in internal validation assessment is validation using the same data that was used for derivation.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-12-01 vote 5-0 by Xing Song, Javier Bracchiglione, Harold Lehmann, Eric Harvey, Caue Monaco

# concept[].concept[].definition
A bias due to the decision to end the study earlier than planned.

# concept[].concept[].property[].valueString
Child terms (types of Early Study Termination Bias) may be used to report the reasons for bias in the decision to end the study earlier than planned. Bias resulting from the early study termination may be described with other terms in the code system.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Mario Tristan, Khalid Shahin, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].property[].valueString
2022-04-01 vote 4-1 by Jesus Lopez-Alcalde, Harold Lehmann, Cauê Monaco, Robin Ann Yurk, Mario Tristan

# concept[].concept[].property[].valueString
2022-04-01 comment: Term Definition:  Simplify so it reads.
A bias in the reported results due to early termination of a study resulting in incomplete data collection.

# concept[].concept[].property[].valueString
2022-04-08 vote 6-0 by nelle.stocquart, nisha mathew, Mario Tristan, Robin Ann Yurk, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].display
early study termination bias due to competing interests

# concept[].concept[].concept[].definition
An early study termination bias due to the decision to end the study being influenced by financial, commercial, legal, political, social, professional, or intellectual  interests.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Mario Tristan, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2022-04-01 vote 6-0 by Brian S. Alper, Joanne Dehnbostel, Jesus Lopez-Alcalde, Harold Lehmann, Cauê Monaco, Mario Tristan

# concept[].concept[].concept[].display
early study termination bias due to unplanned use of interim analysis

# concept[].concept[].concept[].definition
An early study termination bias due to awareness of study results without following a preplanned protocol for how interim results will influence the decision to terminate the study.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Mario Tristan, Khalid Shahin, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2022-04-01 vote 5-0 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Harold Lehmann, Cauê Monaco, Mario Tristan

# concept[].concept[].concept[].display
early study termination bias due to inappropriate statistical stopping rule

# concept[].concept[].concept[].definition
An early study termination bias due to use of an inappropriate model or threshold in the analysis used for determination to end the study.

# concept[].concept[].concept[].property[].valueString
An example of an inappropriate statistical stopping rule is one that does not account for multiple analyses (i.e. does not use a lower p value threshold) for a conclusion of benefit warranting early termination of the study.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Mario Tristan, Khalid Shahin, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2022-04-01 vote 6-0 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Harold Lehmann, Cauê Monaco, Mario Tristan, Robin Ann Yurk

# concept[].concept[].concept[].display
early study termination bias due to external factors

# concept[].concept[].concept[].definition
An early study termination bias due to a decision to end the study based on factors other than the results of interim analysis.

# concept[].concept[].concept[].property[].valueString
Examples of external factors may include cessation of funding, and safety or efficacy results reported by other studies.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Mario Tristan, Khalid Shahin, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2022-04-01 vote 6-0 by Joanne Dehnbostel, Jesus Lopez-Alcalde, Harold Lehmann, Cauê Monaco, Mario Tristan, Robin Ann Yurk

# concept[].concept[].definition
A bias due to the decision to end the study later than planned.

# concept[].concept[].property[].valueString
When the study is allowed to continue longer than a pre-planned stopping point, then there is a risk of bias in that the results from the study may differ from the results that would occur according to the study plan. When the study termination is pre-planned to vary by events or results of interim analyses (e.g., in an adaptive design), this is not a 'late study termination bias'.

# concept[].concept[].property[].valueString
This term was added after evaluation of [A comprehensive item bank of internal validity issues of relevance to in vitro toxicology studies](https://www.tandfonline.com/doi/full/10.1080/2833373X.2024.2418045) which included a concept of 'Timing of study termination bias' defined as a bias due to the decision to end the study earlier or later than planned, and modified from the original SEVCO term “[early study termination bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00370),” defined as “a bias due to the decision to end the study earlier than planned”

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Airton Stein, Joanne Dehnbostel, Homa Keshavarz, Khalid Shahin, Cauê Monaco

# concept[].concept[].property[].valueString
2024-12-06 vote 7-1 by Homa Keshavarz, Sheyu Li, Cauê Monaco, Javier Bracchiglione, Lara Kahaleh, Bhagvan Kommadi, Saphia Mokrane, Airton Tetelbom Stein

# concept[].concept[].property[].valueString
2024-12-06 comment re: "late study termination bias" = "A bias due to the decision to end the study later than planned."
1No)
This bias highly relies on the reason of the late termination, which warrants discussion. It is also helpful to differentiate two situations, i.e., post-trial follow up (analysis based on the follow up data after the termination of the trial) and delayed termination based on the adaptive design (it is not a real delay).

# concept[].concept[].property[].valueString
2024-12-13 vote 5-0 by Cauê Monaco, Saphia Mokrane, Bhagvan Kommadi, Paul Whaley, Airton Tetelbom Stein

# concept[].definition
A risk factor for bias in which persons influencing research design, conduct, analysis or reporting have motivations that could compromise their impartiality.

# concept[].property[].valueString
MASTER-28. Conflict of interests were declared and absent

# concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Javier Bracchiglione, Janice Tufte, Muhammad Afzal, Caue Monaco

# concept[].property[].valueString
2023-11-10 vote 4-1 by Brian S. Alper, Harold Lehmann, Janice Tufte, Eric Harvey, Javier Bracchiglione

# concept[].property[].valueString
2023-11-10 comment: I do not think the term should be limited to goals and motivations this seems judgmental and manipulative. COI can be based on intellectual property and or current research work along the same subject where a researcher or partner is too involved with a project or paper on the same subject

# concept[].property[].valueString
Motivations may be explicit or implicit. Motivations may be unconscious or unrecognized. Conflict of interest is sometimes phrased "potential conflict of interest" or "perceived conflict of interest".

# concept[].property[].valueString
2023-12-01 vote 6-0 by Cauê Monaco, Xing Song, Javier Bracchiglione, Harold Lehmann, Janice Tufte, Eric Harvey

# concept[].concept[].definition
A risk factor for bias in which persons influencing research design, conduct, analysis or reporting have financial motivations that could compromise their impartiality.

# concept[].concept[].property[].valueString
Motivations may be explicit or implicit. Motivations may be unconscious or unrecognized. The financial motivations may be direct (e.g. salary or consulting fees) or indirect (e.g. stock interests or spousal financial interests). Conflict of interest is sometimes phrased "potential conflict of interest" or "perceived conflict of interest".

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Javier Bracchiglione, Janice Tufte, Muhammad Afzal, Caue Monaco

# concept[].concept[].property[].valueString
2023-12-01 vote 6-0 by Cauê Monaco, Xing Song, Javier Bracchiglione, Harold Lehmann, Janice Tufte, Eric Harvey

# concept[].concept[].property[].valueString
2023-12-01 comment: I agree with the definition of the term, but I think it will be better to further explicit what "financial" means in the comments for application (e.g. salary, stocks, paid assistance to congress)

# concept[].concept[].definition
A risk factor for bias in which persons influencing research design, conduct, analysis or reporting have non-financial motivations that could compromise their impartiality.

# concept[].concept[].property[].valueString
Motivations may be explicit or implicit. Motivations may be unconscious or unrecognized. The non-financial motivations may be related to social, political, professional, ideological, or other factors. Conflict of interest is sometimes phrased "potential conflict of interest" or "perceived conflict of interest".

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Javier Bracchiglione, Janice Tufte, Muhammad Afzal, Caue Monaco

# concept[].concept[].property[].valueString
2023-12-01 vote 6-0 by Cauê Monaco, Xing Song, Javier Bracchiglione, Harold Lehmann, Janice Tufte, Eric Harvey

# concept[].concept[].property[].valueString
2023-12-01 comment: I agree with the definition of the term, but I think it will be better to further explicit what "non-financial" means in the comments for application (e.g. intellectual)

# concept[].definition
The result of a qualitative assessment of the likelihood and potential impact of systematic distortion in research results.

# concept[].property[].valueString
[Bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00001) is defined as a systematic distortion in research results (estimation of effect, association, or inference). Distortions in research results means differences between the reported results (findings, conclusions, effect estimates) and the actuality (the truth, the estimand [the quantity targeted for estimation]).

All terms defined here are qualitative classifications of the risk of bias. Although a risk of bias can conceptually be quantified, there is no common agreed method on how to quantify a risk of bias.

# concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Carlos Alva-Diaz, Homa Keshavarz

# concept[].property[].valueString
2024-09-20 vote 6-0 by Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].property[].valueString
2024-08-30 vote 7-0 by Saphia Mokrane, Sheyu Li, Lenny Vasanthan, Eric Harvey, Janice Tufte, Homa Keshavarz, Airton Tetelbom Stein BUT THEN DEFINITION CHANGED to add "The result of " based on feedback on child terms.

# concept[].concept[].definition
The result of a qualitative assessment that there is a low likelihood and potential impact of systematic distortion in research results.

# concept[].concept[].property[].valueString
A 'low risk of bias' rating denotes a judgment that there are no serious concerns for bias. The 'potential impact of systematic distortion in research results' may include the impact on the clinical interpretation or conclusion of the study findings.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
In the Cochrane Handbook (https://training.cochrane.org/handbook/current/chapter-08#section-8-7):
Once the signalling questions are answered, the next step is to reach a risk-of-bias judgement, and assign one of three levels to each domain:

Low risk of bias;
Some concerns; or
High risk of bias.

# concept[].concept[].property[].valueString
2024-09-06 vote 5-1 by Carlos Alva-Diaz, Cauê Monaco, Sheyu Li, janice tufte, Eric Harvey, Javier Bracchiglione

# concept[].concept[].property[].valueString
2024-09-06 comment re: "low risk of bias" = "A qualitative assessment that there is a low likelihood and potential impact of systematic distortion in research results."
1N)
A qualitative judgement that the the systematic distortion is unlikely to impact the clinical interpretation or conclusion of the study findings. 

The rating of low, moderate, and high are subjective judgement or the result of an assessment based on a particular context. It is thus not a asessment per se. As a clinician, I would prefer to the term ending up with a clinical interpretation or decision. But other conclusion can be possible even for clinical research.

# concept[].concept[].property[].valueString
2024-09-20 vote 8-0 by Carlos Alva-Diaz, Saphia Mokrane, Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan, Eric Harvey, Airton Tetelbom Stein

# concept[].concept[].definition
The result of a qualitative assessment that there is some likelihood and potential impact of systematic distortion in research results, and this likelihood and potential impact is between low and high.

# concept[].concept[].property[].valueString
In the Cochrane Handbook (https://training.cochrane.org/handbook/current/chapter-08#section-8-7):
Once the signalling questions are answered, the next step is to reach a risk-of-bias judgement, and assign one of three levels to each domain:

Low risk of bias;
Some concerns; or
High risk of bias.

# concept[].concept[].property[].valueString
The 'potential impact of systematic distortion in research results' may include the impact on the clinical interpretation or conclusion of the study findings.

The term 'unclear risk of bias' is not explicitly defined because it could be used to represent 'moderate risk of bias' or [undetermined risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00192).

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Harold Lehmann, Carlos Alva-Diaz

# concept[].concept[].property[].valueString
2024-09-06 vote 5-2 by Airton Tetelbom Stein, Carlos Alva-Diaz, Cauê Monaco, Sheyu Li, janice tufte, Eric Harvey, Javier Bracchiglione

# concept[].concept[].property[].valueString
2024-09-06 comments re: "moderate risk of bias" = "A qualitative assessment that there is some likelihood and potential impact of systematic distortion in research results, but this likelihood and potential impact is neither low nor high."
1N)
I think that the term' undetermined risk of bias' is opposed to any other positive assertions of concern (low, moderate or high risk of bias). 
2N)
Same with above. But it is a bit confusing for stating as 'neither low nor high', which leads people have to refer to the definition of low to high.

# concept[].concept[].property[].valueString
2024-09-20 vote 6-0 by Carlos Alva-Diaz, Saphia Mokrane, Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan

# concept[].concept[].definition
The result of a qualitative assessment that there is a high likelihood and potential impact of systematic distortion in research results.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
A 'high risk of bias' rating denotes a judgment that there are serious concerns for bias. The 'potential impact of systematic distortion in research results' may include the impact on the clinical interpretation or conclusion of the study findings.

# concept[].concept[].property[].valueString
In the Cochrane Handbook (https://training.cochrane.org/handbook/current/chapter-08#section-8-7):
Once the signalling questions are answered, the next step is to reach a risk-of-bias judgement, and assign one of three levels to each domain:

Low risk of bias;
Some concerns; or
High risk of bias.

# concept[].concept[].property[].valueString
2024-09-06 vote 6-1 by Airton Tetelbom Stein, Carlos Alva-Diaz, Cauê Monaco, Sheyu Li, janice tufte, Eric Harvey, Javier Bracchiglione

# concept[].concept[].property[].valueString
2024-09-06 comment re: "high risk of bias" = "A qualitative assessment that there is a high likelihood and potential impact of systematic distortion in research results."
1N)
A qualitative judgement that the the systematic distortion is unlikely to impact the clinical interpretation or conclusion of the study findings. 

The rating of low, moderate, and high are subjective judgement or the result of an assessment based on a particular context. It is thus not a asessment per se. As a clinician, I would prefer to the term ending up with a clinical interpretation or decision. But other conclusion can be possible even for clinical research.

# concept[].concept[].property[].valueString
2024-09-20 vote 6-0 by Carlos Alva-Diaz, Saphia Mokrane, Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan

# concept[].concept[].definition
The result of a qualitative assessment that there is such a high likelihood and potential impact of systematic distortion in research results that the findings are not valid.

# concept[].concept[].property[].valueString
The colloquial term 'fatal flaw' is sometimes used to report a critical risk of bias, signifying no further use of the study being assessed.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Airton Stein, Muhammad Afzal

# concept[].concept[].property[].valueString
2024-09-20 vote 7-0 by Carlos Alva-Diaz, Saphia Mokrane, Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan, Eric Harvey

# concept[].concept[].definition
There is insufficient information to make a qualitative assessment regarding the likelihood and potential impact of systematic distortion in research results.

# concept[].concept[].property[].valueString
Unlike the terms [low risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00186), [moderate risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00187), and [high risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00188), the term 'undetermined risk of bias' does not express a positive assertion of concern. 

The term 'unclear risk of bias' is not explicitly defined because it could be used to represent 'undetermined risk of bias' or [moderate risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00187). Some guidance has suggested to use 'unclear risk of bias' to mean 'undetermined risk of bias' but others have interpreted it to mean [moderate risk of bias](https://fevir.net/resources/CodeSystem/27270#SEVCO:00187).

The 'potential impact of systematic distortion in research results' may include the impact on the clinical interpretation or conclusion of the study findings.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Harold Lehmann, Carlos Alva-Diaz

# concept[].concept[].property[].valueString
2024-09-06 vote 5-2 by Airton Tetelbom Stein, Carlos Alva-Diaz, Cauê Monaco, Sheyu Li, janice tufte, Eric Harvey, Javier Bracchiglione

# concept[].concept[].property[].valueString
2024-09-06 comments re: "undetermined risk of bias" = "A qualitative assessment that there is insufficient information to make a qualitative assessment regarding the likelihood and potential impact of systematic distortion in research results."
1N)
The term' undetermined risk of bias' is opposed to any other positive assertions of concern (low, moderate or high risk of bias). 

I think that a better explication is

"As opposed to the term low, moderate or high risk of bias, the term 'undetermined risk of bias' does not express a positive assertion of concern."

2N)
Same with above. 
3Y)
is this a cop out though possibly?
4Y)
I would suggest "unclear risk of bias" as an alternative term.


2024-09-20 comment re: "undetermined risk of bias" = "There is insufficient information to make a qualitative assessment regarding the likelihood and potential impact of systematic distortion in research results."
Yes
As I understand, at least the Cochrane RoB1 tool guidance specified that "unclear risk of bias" referred to undetermined and not moderate. Its use, however, was often seen as moderate risk of bias, but this was not the guidance. I don't know if you are thinking of other tools that also specify "unclear risk of bias" as a category.

# concept[].concept[].property[].valueString
2024-09-20 vote 6-0 by Carlos Alva-Diaz, Saphia Mokrane, Javier Bracchiglione, C P Ooi, Bhagvan Kommadi, Lenny Vasanthan

# concept[].definition
The result of a qualitative assessment of the likelihood of a situation or event.

# concept[].property[].valueString
For approaches to assessment of the risk of bias (which includes bias presence and potential impact) in which multiple factors are separately considered, the 'rating of factor presence' may be applied to each of the factors considered.

# concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Carlos Alva-Diaz, Homa Keshavarz

# concept[].property[].valueString
2024-10-04 vote 5-0 by Joanne Dehnbostel, Saphia Mokrane, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey

# concept[].property[].valueString
2024-09-27 comment re: "rating of presence of cause of bias" = "The result of a qualitative assessment of the likelihood of a situation leading to systematic distortion in research results."
1Yes)
Possible alternatives: "rating of presence of biasing factor" "judgment regarding presence of factor introducing possible bias" "presence or absence of possible biasing factor"

2 No) Rating of presence of cause of bias just doesn't read well. Rating of factor presence was better, rating of bias presence would work. Likelihood of bias presence would also work.

# concept[].property[].valueString
2024-09-27 vote 5-1 by Joanne Dehnbostel, Brian S. Alper, Carlos Alva-Diaz, Homa Keshavarz, Arnav Agarwal, Airton Tetelbom Stein

# concept[].concept[].definition
The result of a qualitative assessment that a situation exists or an event has occurred.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-10-04 vote 5-0 by Joanne Dehnbostel, Saphia Mokrane, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey

# concept[].concept[].definition
The result of a qualitative assessment that a situation probably exists or an event has probably occurred.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-10-04 vote 5-0 by Joanne Dehnbostel, Saphia Mokrane, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey
Yes

# concept[].concept[].definition
The result of a qualitative assessment that a situation probably does not exist or an event has probably not occurred.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-10-04 vote 5-0 by Joanne Dehnbostel, Saphia Mokrane, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey

# concept[].concept[].definition
The result of a qualitative assessment that a situation does not exist or an event has not occurred.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-10-04 vote 5-0 by Joanne Dehnbostel, Saphia Mokrane, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey

# concept[].concept[].definition
There is insufficient information to make a qualitative assessment regarding whether a situation exists or an event has occurred.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Homa Keshavarz, Harold Lehmann

# concept[].concept[].property[].valueString
2024-10-11 vote 7-2 by Javier Bracchiglione, Saphia Mokrane, Harold Lehmann, Arnav Agarwal, Lenny Vasanthan, Joanne Dehnbostel, Airton Tetelbom Stein, Homa Keshavarz, Eric Harvey

# concept[].concept[].property[].valueString
2024-10-11 comments re: "factor presence or absence unclear" = "There is insufficient information to make a qualitative assessment regarding whether a situation exists or an event has occurred."
1No)
To be consistent with the previous terms under "rating of factor presence", I would suggest rephrasing the term, as for example : "factor unclearly present of absent" 
Suggestion for the definition : "The result of a qualitative assessment of the presence or the absence of a situation or an event due to insufficient information to make the assessment." 
2No)
"Situation" is especially vague; "event" mildly suggestive. Sounds like we need a Comment for application---either here or in the parent term---that puts these terms into perspective. E.g., "Assessment of presence of a factor depends on the definition of the factor. Assessing such presence depends on whether a situation

# concept[].concept[].property[].valueString
The presence or absence of a factor may be undetermined because a qualitative assessment was attempted and concluded that there was insufficient information to rate the factor presence, or because a qualitative assessment was not done.

# concept[].concept[].property[].valueString
2024-10-18 vote 5-0 by Saphia Mokrane, Homa Keshavarz, Airton Tetelbom Stein, Lenny Vasanthan, Eric Harvey

# concept[].definition
The result of a qualitative assessment of the direction of influence on research results.

# concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel

# concept[].property[].valueString
The meaning of direction of influence on research results changes with the type of estimation. For an estimation of comparative effect, the direction may be favoring a side being compared or towards no effect. For an estimation of prevalence, the direction may be overestimation or underestimation.
Bias is defined as a systematic distortion of research results.

# concept[].property[].valueString
2024-11-08 vote 6-0 by Saphia Mokrane, Homa Keshavarz, Lara Kahaleh, Bhagvan Kommadi, Airton Tetelbom Stein, Eric Harvey

# concept[].property[].valueString
https://training.cochrane.org/handbook/current/chapter-08

RoB 2 includes optional judgements of the direction of the bias for each domain and overall. For some domains, the bias is most easily thought of as being towards or away from the null. For example, high levels of switching of participants from their assigned intervention to the other intervention may have the effect of reducing the observed difference between the groups, leading to the estimated effect of adhering to intervention (see Section 8.2.2) being biased towards the null. For other domains, the bias is likely to favour one of the interventions being compared, implying an increase or decrease in the effect estimate depending on which intervention is favoured. Examples include manipulation of the randomization process, awareness of interventions received influencing the outcome assessment and selective reporting of results. If review authors do not have a clear rationale for judging the likely direction of the bias, they should not guess it and can leave this response blank.

Cite this chapter as: Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial [last updated October 2019]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. Available from www.training.cochrane.org/handbook.

https://onlinelibrary-wiley-com.ezproxy3.library.arizona.edu/doi/10.1111/acem.12255

# concept[].concept[].display
risk of bias favoring experimental exposure

# concept[].concept[].definition
The result of a qualitative assessment that a bias, if present, would influence the effect estimate in a direction that suggests greater benefit or less harm from the experimental exposure compared to the comparator exposure.

# concept[].concept[].property[].valueString
The experimental exposure is the exposure of interest, whether the study is interventional or observational, the exposure of interest is an intervention or an exposure, or the outcome of interest is desirable or undesirable. The desirability of the outcome of interest determines whether an increase or decrease in the value of the effect estimate is considered beneficial or harmful.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Airton Stein

# concept[].concept[].property[].valueString
2024-11-22 vote 6-0 by Saphia Mokrane, Sheyu Li, Homa Keshavarz, Eric Harvey, Lara Kahaleh, Airton Tetelbom Stein

# concept[].concept[].display
risk of bias favoring comparator exposure

# concept[].concept[].definition
The result of a qualitative assessment that a bias, if present, would influence the effect estimate in a direction that suggests greater benefit or less harm from the comparator exposure compared to the experimental exposure.

# concept[].concept[].property[].valueString
The experimental exposure is the exposure of interest, whether the study is interventional or observational, the exposure of interest is an intervention or an exposure, or the outcome of interest is desirable or undesirable. The desirability of the outcome of interest determines whether an increase or decrease in the value of the effect estimate is considered beneficial or harmful.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Airton Stein

# concept[].concept[].property[].valueString
2024-11-22 vote 5-0 by Sheyu Li, Homa Keshavarz, Eric Harvey, Lara Kahaleh, Airton Tetelbom Stein

# concept[].concept[].display
risk of bias towards the null hypothesis

# concept[].concept[].definition
The result of a qualitative assessment that a bias, if present, would increase the likelihood of failing to reject the null hypothesis.

# concept[].concept[].property[].valueString
"there are many exceptions to the heuristic for which bias towards the null cannot be assumed." Yland JJ, Wesselink AK, Lash TL, Fox MP. Misconceptions About the Direction of Bias From Nondifferential Misclassification. Am J Epidemiol. 2022 Jul 23;191(8):1485-1495. doi: 10.1093/aje/kwac035. Erratum in: Am J Epidemiol. 2022 Nov 19;191(12):2123. doi: 10.1093/aje/kwac129. PMID: 35231925; PMCID: PMC9989338. https://pmc.ncbi.nlm.nih.gov/articles/PMC9989338/

# concept[].concept[].property[].valueString
A risk of bias towards the null hypothesis increases the likelihood of a type 2 error (mistaken failure to reject the null hypothesis when the null hypothesis is false).

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Airton Stein, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-11-22 vote 6-0 by Saphia Mokrane, Sheyu Li, Homa Keshavarz, Eric Harvey, Lara Kahaleh, Airton Tetelbom Stein

# concept[].concept[].display
risk of bias away from the null hypothesis

# concept[].concept[].definition
The result of a qualitative assessment that a bias, if present, would increase the likelihood of rejecting the null hypothesis.

# concept[].concept[].designation[].value
risk of bias against the null hypothesis

# concept[].concept[].property[].valueString
A risk of bias away from the null hypothesis increases the likelihood of a type 1 error (mistaken rejection of the null hypothesis when the null hypothesis is true).

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Airton Stein, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-11-22 vote 6-0 by Saphia Mokrane, Sheyu Li, Homa Keshavarz, Eric Harvey, Lara Kahaleh, Airton Tetelbom Stein

# concept[].concept[].definition
The result of a qualitative assessment that a bias, if present, cannot be classified as to whether the bias would lead to an increase or a decrease in the estimate.

# concept[].concept[].property[].valueString
2024-11-22 vote 5-0 by Saphia Mokrane, Sheyu Li, Homa Keshavarz, Lara Kahaleh, Airton Tetelbom Stein

# concept[].display
rating of potential influence on research results

# concept[].definition
The result of a qualitative assessment of, if a cause of bias were present, the potential impact on the research results.

# concept[].property[].valueString
For approaches to assessment of the risk of bias (which includes bias presence and potential impact) in which multiple factors are separately considered, the 'rating of potential influence' may be applied to each of the factors considered.

# concept[].property[].valueString
Brian S. Alper, Airton Stein, Joanne Dehnbostel

# concept[].property[].valueString
2024-10-25 vote 7-0 by Saphia Mokrane, Harold Lehmann, Lenny Vasanthan, Bhagvan Kommadi, Homa Keshavarz, Eric Harvey, Airton Tetelbom Stein

# concept[].property[].valueString
2024-10-18 comment re: "rating of potential influence" = "The result of a qualitative assessment of, if a cause of bias were present, the potential impact on the research results."
1Yes)
the 'rating of potential influence'' => the 'rating of potential influence'

# concept[].property[].valueString
2024-10-18 vote 5-0 by Saphia Mokrane, Homa Keshavarz, Airton Tetelbom Stein, Lenny Vasanthan, Eric Harvey BUT THEN THE TERM CHANGED

# concept[].concept[].display
high potential to influence research results

# concept[].concept[].definition
High certainty that a cause of bias, if present, has an impact on the research results.

# concept[].concept[].property[].valueString
The rating of potential influence on research results is a qualitative assessment; there is no agreed-upon quantitative scale of certainty. The rating of 'high potential to influence research results' matches an answer of Yes (from an option list of Yes/Likely Yes/Likely No/No) in the ROB2 risk of bias assessment tool.

# concept[].concept[].property[].valueString
2024-11-01 vote 5-0 by Harold Lehmann, Saphia Mokrane, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein BUT THEN COMMENT ADDED

# concept[].concept[].property[].valueString
2024-11-01 comment:
1Yes)
Consider a Comment for application that says something like, "There is no agreed-upon quantitative scale on which to base this qualitative assessment."

# concept[].concept[].property[].valueString
2024-11-08 vote 6-0 by Saphia Mokrane, Homa Keshavarz, Lara Kahaleh, Bhagvan Kommadi, Airton Tetelbom Stein, Eric Harvey

# concept[].concept[].display
some potential to influence research results

# concept[].concept[].definition
Low to moderate certainty that a cause of bias, if present, has an impact on the research results.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin

# concept[].concept[].property[].valueString
2024-11-01 comment:
1Yes)
Consider a Comment for application that says something like, "There is no agreed-upon quantitative scale on which to base this qualitative assessment."


2024-11-08 comment re: "some potential to influence research results" = "Low to moderate certainty that a cause of bias, if present, has an impact on the research results."
1Yes)
instead of some, it can be worded as significant potential


2024-11-15  comment re: "significant potential to influence research results" = "Low to moderate certainty that a cause of bias, if present, has an impact on the research results."
1No)
Significant can be interpreted as substantial or important -- that does not match the described use of 'Likely Yes' as distinct from 'Yes' in response to whether there is a potential to impact research results.

# concept[].concept[].property[].valueString
2024-11-01 vote 5-0 by Harold Lehmann, Saphia Mokrane, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein BUT THEN COMMENT ADDED 
2024-11-08 vote 6-0 by Saphia Mokrane, Homa Keshavarz, Lara Kahaleh, Bhagvan Kommadi, Airton Tetelbom Stein, Eric Harvey BUT THEN TERM CHANGED DUE TO COMMENT
2024-11-15 vote 6-1 by Brian S. Alper, Cauê Monaco, Airton Tetelbom Stein, Homa Keshavarz, Lenny Vasanthan, Yaowaluk Ngoenwiwatkul, Eric Harvey

# concept[].concept[].property[].valueString
The rating of potential influence on research results is a qualitative assessment; there is no agreed-upon quantitative scale of certainty. The rating of 'some potential to influence research results' matches an answer of Likely Yes (from an option list of Yes/Likely Yes/Likely No/No) in the ROB2 risk of bias assessment tool.

# concept[].concept[].property[].valueString
2024-11-22 vote 6-0 by Saphia Mokrane, Sheyu Li, Homa Keshavarz, Eric Harvey, Airton Tetelbom Stein, Lara Kahaleh

# concept[].concept[].display
limited potential to influence research results

# concept[].concept[].definition
Low to moderate certainty that a cause of bias, if present, does NOT have an impact on the research results.

# concept[].concept[].property[].valueString
The rating of potential influence on research results is a qualitative assessment; there is no agreed-upon quantitative scale of certainty. The rating of 'limited potential to influence research results' matches an answer of Likely No (from an option list of Yes/Likely Yes/Likely No/No) in the ROB2 risk of bias assessment tool.

# concept[].concept[].property[].valueString
2024-11-01 vote 4-1 by Harold Lehmann, Saphia Mokrane, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].property[].valueString
2014-11-01 comment re: "Low to moderate certainty that a cause of bias, if present, has little to no impact on the research results.":
1No)
Definition is too similar to 'some potential'

# concept[].concept[].property[].valueString
2024-11-08 vote 5-0 by 2024-11-08 vote 6-0 by Saphia Mokrane, Homa Keshavarz, Lara Kahaleh, Bhagvan Kommadi, Airton Tetelbom Stein, Eric Harvey

# concept[].concept[].display
no potential to influence research results

# concept[].concept[].definition
High certainty that a cause of bias, if present, has NO impact on the research results.

# concept[].concept[].property[].valueString
The rating of potential influence on research results is a qualitative assessment; there is no agreed-upon quantitative scale of certainty. The rating of 'no potential to influence research results' matches an answer of No (from an option list of Yes/Likely Yes/Likely No/No) in the ROB2 risk of bias assessment tool.

# concept[].concept[].property[].valueString
2024-11-01 comment re:  "no potential to influence research results" = "High certainty that a cause of bias, if present, has no impact on the research results."
1No)
Weird that both "High potential" and "No potential" start with "High certainty..." The negation in the definition here is too buried

# concept[].concept[].property[].valueString
2024-11-01 vote 4-1 by Harold Lehmann, Saphia Mokrane, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].property[].valueString
2024-11-08 vote 6-0 by Saphia Mokrane, Homa Keshavarz, Lara Kahaleh, Bhagvan Kommadi, Airton Tetelbom Stein, Eric Harvey

# concept[].definition
An information content entity that is a formalization of relationships between variables and value specification.

# concept[].property[].valueString
The 'statistic' does not include the numerical value for which the statistic is used--that would be the statistic value, and the 'statistic' does not include the model characteristics.

# concept[].property[].valueString
Brian S. Alper, Philippe Rocca-Serra, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].property[].valueString
revision 6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca; original approval 6/6 as of 9/27/2021: Harold Lehmann, Bhagvan Kommadi, Louis Leff, Janice Tufte, Joanne Dehnbostel, Mario Tristan

# concept[].concept[].definition
A statistic that represents the distance of a value from zero.

# concept[].concept[].property[].valueString
The | symbol is used around the value to denote the absolute value, e.g. |x|, such that if x = -3, then |x| = 3.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].property[].valueString
2024-04-15 vote 5-0 by Harold Lehmann, Janice Tufte, Eric Harvey, Homa Keshavarz, Lenny Vasanthan

# concept[].concept[].definition
A statistic that represents the number of instances or occurrences of something.

# concept[].concept[].property[].valueString
A count can only be denoted by non-negative integer values.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Kenneth Wilkins

# concept[].concept[].property[].valueString
6/6 as of 9/27/2021: Harold Lehmann, Bhagvan Kommadi, Louis Leff, Janice Tufte, Joanne Dehnbostel, Mario Tristan

# concept[].concept[].definition
A statistic that represents the result of adding all the values in a collection of values.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel,Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].concept[].property[].valueString
6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca

# concept[].concept[].definition
A statistic that represents the largest non-null value in a collection of values that can be ordered by magnitude.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel,Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].concept[].property[].valueString
6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca

# concept[].concept[].definition
A statistic that represents the smallest non-null value in a collection of values that can be ordered by magnitude.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel,Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].concept[].property[].valueString
6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca

# concept[].concept[].definition
A statistic that represents the largest value that could occur.

# concept[].concept[].property[].valueString
This term may be used to denote the upper limit of a scale or score.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel,Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].concept[].property[].valueString
6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca

# concept[].concept[].definition
A statistic that represents the smallest value that could occur.

# concept[].concept[].property[].valueString
This term may be used to denote the lower limit of a scale or score.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel,Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Khalid Shahin

# concept[].concept[].property[].valueString
6/6 as of 10/4/2021: Alejandro Piscoya, Bhagvan Kommadi, Brian S. Alper, Janice Tufte, Louis Leff, Sorana D. Bolboaca

# concept[].concept[].definition
A statistic that represents a central value for a set of data.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte,  Kenneth Wilkins, Philippe Rocca-Serra

# concept[].concept[].property[].valueString
6/6 as of 10/27/2021: Janice Tufte, Louis Leff, Vignesh Subbian, Robin Ann Yurk, Harold Lehmann, Muhammad Afzal, Pentti Nieminen

# concept[].concept[].concept[].definition
A measure of central tendency calculated as the sum of a set of values divided by the number of values in the set.

# concept[].concept[].concept[].property[].valueString
A=sum[Ai] / n where i ranges from 1 to n and Ai represents the value of individual observations.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte,  Kenneth Wilkins, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
6/6 as of 10/27/2021: Janice Tufte, Louis Leff, Vignesh Subbian, Robin Ann Yurk, Harold Lehmann, Muhammad Afzal, Pentti Nieminen

# concept[].concept[].concept[].concept[].definition
A mean of values in which each value is the subtraction of one quantity from another.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin, Janice Tufte, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 vote 6-1 by Louis Leff, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte, C P Ooi

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 comment: 'Difference in means' may be more appropriate. 'Mean value from one population subtract the mean value of another population' may be clearer reflecting the definition

# concept[].concept[].concept[].concept[].property[].valueString
The primary use of this term is in analyzing within-individual differences.

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-15 vote 5-0 by Robin Ann Yurk, Harold Lehmann, Janice Tufte, Paola Rosati, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
A mean of values in which each value is the duration of time between the start of observation and the occurrence of an event.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Yuan Gao

# concept[].concept[].concept[].concept[].property[].valueString
2022-10-19 vote 5-0 by Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
A measure of central tendency calculated as the nth root of the product of all of the observations in a data set (n being the number of all observations).

# concept[].concept[].concept[].property[].valueString
For n observations with values x1, x2, … xn, the product of all the values P = x1 * x2 … xn [also expressed as P = (x1)(x2)...(xn)].  The nth root of the product = (P)^(1/n).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte,  Kenneth Wilkins, Philippe Rocca-Serra

# concept[].concept[].concept[].property[].valueString
6/6 as of 10/27/2021: Janice Tufte, Louis Leff, Vignesh Subbian, Robin Ann Yurk, Harold Lehmann, Muhammad Afzal, Pentti Nieminen

# concept[].concept[].concept[].definition
A measure of central tendency equal to the middle value (or mean of the two middle values) of a set of ordered data.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].property[].valueString
The median value is equal to the middle value of a set of ordered data with an odd number of values. The median value is calculated as the mean of the two middle values of a set of ordered data with an even number of values. The median is sometimes called the second quartile or fiftieth percentile.

# concept[].concept[].concept[].property[].valueString
6-1 on 2021-11-01 by Louis Leff, Vignesh Subbian, Pentti Nieminen, Bhagvan Kommadi, Janice Tufte, Sorana D. Bolboaca, Robin Ann Yurk
2021-12-01 vote 5-1 by Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2021-11-01 comment: the definition is appropriate. Suggest use Alternative terms: center value, statistical median or middle value. I don't recommend using fiftieth percentile or second quartile
2021-12-01 comment: I would change definition to: A measure of central tendency equal to the middle value of a set of ordered data with an odd number of values. It could be calculated also as  the mean of the two middle values of a set of ordered data with an even number of values. ((Perhaps simpler as: A measure of central tendency equal to the middle value of a set of ordered data. In a set of ordered data with an even number of values, the middle value is calculated as the mean of the two middle values.))

# concept[].concept[].concept[].property[].valueString
2021-12-15 vote 6-0 by Robin Ann Yurk, Muhammad Afzal, Harold Lehmann, Janice Tufte, Paola Rosati, Khalid Shahin

# concept[].concept[].concept[].concept[].definition
A median of values in which each value is the duration of time between the start of observation and the occurrence of an event.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Yuan Gao

# concept[].concept[].concept[].concept[].property[].valueString
2022-10-19 vote 5-0 by Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
A measure of central tendency that is the most frequently occurring value in a data set. If no value is repeated, there is no mode. If more than one value occurs with the same greatest frequency, each of these values is a mode.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].property[].valueString
7/7 on 2021-11-01 by Louis Leff, Vignesh Subbian, Pentti Nieminen, Bhagvan Kommadi, Janice Tufte, Sorana D. Bolboaca, Robin Ann Yurk

# concept[].concept[].concept[].definition
A measure of central tendency calculated by dividing the total number of observations by the sum of the reciprocals of each observed value.

# concept[].concept[].concept[].property[].valueString
STATO:
The harmonic mean is a kind of mean which is calculated by dividing the total number of observations by the reciprocal of each number in a series. Harmonic Mean = N/(1/a1+1/a2+1/a3+1/a4+.......+1/aN) where a(i)= Individual score and N = Sample size (Number of scores)

# concept[].concept[].concept[].property[].valueString
Harmonic Mean = N/(1/a1+1/a2+1/a3+1/a4+...+1/aN) where a(i)= Individual observed value and N = Sample size (Number of observations)

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2022-10-19 vote 6-0 by Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Mario Tristan, Eric Harvey, Yuan Gao

# concept[].concept[].definition
A statistic that represents the value for which the number of data points at or below it constitutes a specific portion of the total number of data points.

# concept[].concept[].property[].valueString
STATO-a quantile is a data item which corresponds to specific elements x in the range of a variate X. the k-th n-tile P_k is that value of x, say x_k, which corresponds to a cumulative frequency of Nk/n (Kenney and Keeping 1962). If n=4, the quantity is called a quartile, and if n=100, it is called a percentile.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].property[].valueString
Quantile is a type of statistic but not used without specification of the portion it represents. Typically, the specification of the portion it represents includes both the number of equal portions (e.g., percentile for 100 equal portions, or quartile for 4 equal portions) and selection of one of these portions (e.g., 25th percentile or first quartile).

For common uses in communicating statistic values, more specific types of quantiles (such as percentile, decile, or quartile) would be used instead of the term *quantile*.

# concept[].concept[].property[].valueString
2024-04-15 vote 2-1 by Lenny Vasanthan, Harold Lehmann, Eric Harvey
2024-04-22 vote 5-1 by Lenny Vasanthan, Homa Keshavarz, Eric Harvey, Harold Lehmann, Sheyu Li, Khalid Shahin

# concept[].concept[].property[].valueString
2024-04-15 comment: 
Can we add information about quartiles also here. 
eg. Quartile is a type of quantile that divides the distribution into four equal halves (Q1, Q2, Q3, Q4)

2024-04-22 comment:
It is confusing in the comments whether the quantile refers to 25% percentile, 50% percentile and 75% percentile. What is the difference between quantile and percentile?

# concept[].concept[].property[].valueString
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].concept[].definition
A quantile in which the specific portion of the number of data points is expressed as a percentage.

# concept[].concept[].concept[].property[].valueString
STATO-a percentile is a quantile which splits data into sections accrued of 1% of data, so the first percentile delineates 1% of the data, the second quartile delineates 2% of the data and the 99th percentile, 99 % of the data

# concept[].concept[].concept[].property[].valueString
Quantile is defined as a statistic that represents the value for which the number of data points at or below it constitutes a specific portion of the total number of data points.

Percentile is a type of statistic but not used to define a statistic value without specification of the portion it represents. For example, one may report a fortieth percentile (40%ile) but one does not report a percentile without specification of which percentile. 40% of the data is at or below the 40%ile.

Most statistic values can be reported in FHIR Evidence Resources with a statisticType element including the SEVCO term as a CodeableConcept. To report a specific percentile (such as the fortieth percentile), one may use the attributeEstimate element containing a type element with the SEVCO term for percentile as a CodeableConcept and a level element with the corresponding value (such as 40).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2024-04-22 comment:
It is unclear if it is OK to say 50% percentile, or it is recommended to say 50% percentile.

# concept[].concept[].concept[].property[].valueString
2024-04-22 vote 5-1 by Lenny Vasanthan, Homa Keshavarz, Eric Harvey, Harold Lehmann, Sheyu Li, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].concept[].definition
A quantile in which the specific portion of the number of data points is expressed as a number of tenths.

# concept[].concept[].concept[].property[].valueString
STATO-a decile is a quantile where n=10 and which splits data into sections accrued of 10% of data, so the first decile delineates 10% of the data, the second decile delineates 20% of the data and the nineth decile, 90 % of the data

# concept[].concept[].concept[].property[].valueString
Quantile is defined as a statistic that represents the value for which the number of data points at or below it constitutes a specific portion of the total number of data points.

Decile is a type of statistic but not used to define a statistic value without specification of the portion it represents. For example, one may report a fourth decile but one does not report a decile without specification of which decile. 40% of the data is at or below the fourth decile.

Most statistic values can be reported in FHIR Evidence Resources with a statisticType element including the SEVCO term as a CodeableConcept. To report a specific decile (such as the fourth decile), one may use the attributeEstimate element containing a type element with the SEVCO term for decile as a CodeableConcept and a level element with the corresponding value (such as 4).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2024-04-22 vote 6-0 by Lenny Vasanthan, Homa Keshavarz, Eric Harvey, Harold Lehmann, Sheyu Li, Khalid Shahin
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].concept[].definition
A quantile in which the specific portion of the number of data points is expressed as a number of fourths.

# concept[].concept[].concept[].property[].valueString
STATO-a quartile is a quantile which splits data into sections accrued of 25% of data, so the first quartile delineates 25% of the data, the second quartile delineates 50% of the data and the third quartile, 75% of the data

# concept[].concept[].concept[].property[].valueString
Quantile is defined as a statistic that represents the value for which the number of data points at or below it constitutes a specific portion of the total number of data points.

Quartile is a type of statistic but not used to define a statistic value without specification of the portion it represents. For example, one may report a third quartile but one does not report a quartile without specification of which quartile. 75% of the data is at or below the third quartile. The second quartile is also called the median.

To report the first quartile, use the SEVCO term for [first quartile](https://fevir.net/resources/CodeSystem/27270#TBD:first-quartile). To report the second quartile, use the SEVCO term for [median](https://fevir.net/resources/CodeSystem/27270#STATO:0000574). To report the third quartile, use the SEVCO term for [third quartile](https://fevir.net/resources/CodeSystem/27270#TBD:third-quartile). To report the fourth quartile, use the SEVCO term for maximum observed value.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2024-04-22 vote 6-0 by Lenny Vasanthan, Homa Keshavarz, Eric Harvey, Harold Lehmann, Sheyu Li, Khalid Shahin
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].concept[].concept[].definition
A quantile for which the number of data points at or below it constitutes a 25% of the total number of data points.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].concept[].concept[].definition
A quantile for which the number of data points at or below it constitutes a 75% of the total number of data points.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2024-04-29 vote 6-0 by Lenny Vasanthan, Brian S. Alper, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant

# concept[].concept[].definition
A statistic that is a subtraction of one quantity from another.

# concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Muhammad Afzal, Khalid Shahin, Philippe Rocca-Serra, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].property[].valueString
2021-12-01 vote 5-0 by Philippe Rocca-Serra, Paola Rosati, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].definition
A statistic that is a subtraction of one quantity from another, with no modification of the resulting value.

# concept[].concept[].concept[].property[].valueString
As a type of statistic, "Absolute Difference" is the actual difference between two quantities and can be positive or negative depending on the order of subtraction. The term "Absolute Difference" should not be confused with the mathematical term 'absolute value' which is a numerical value without a negative sign.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2021-12-01 vote 5-0 by Philippe Rocca-Serra, Paola Rosati, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A statistic that is a subtraction of one count from another.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Muhammad Afzal, Khalid Shahin, Philippe Rocca-Serra, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 vote 5-1 by Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte
2021-12-08 vote 6-1 by Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 comment: Suggest include as an Alternative term under difference and remove this term as unclear on distinction as a separate term.
2021-12-08 comment: Suggest removing this term and adding as an Alternative term to Difference {(atlernative term and Comment for application added in response}}

# concept[].concept[].concept[].concept[].property[].valueString
The term Count Difference is used to specify the Absolute Difference is with respect to a count or number of items (such as number of events, platelet counts, sample size e.g. number of people in the group) to distinguish from differences in other types of statistics (mean difference, median difference, risk difference, etc.)

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-15 vote 6-0 by Robin Ann Yurk, Janice Tufte, Paola Rosati, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].definition
A statistic that is a subtraction of one mean from another.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Muhammad Afzal, Khalid Shahin, Philippe Rocca-Serra, Kenneth Wilkins, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 vote 6-0 by Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte
2021-12-01 Steering group added comment for application and decided not to send out for vote again.

# concept[].concept[].concept[].concept[].property[].valueString
The primary use of this term is in analyzing between-group differences.

# concept[].concept[].concept[].concept[].definition
A statistic that is a subtraction of one median from another.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin, Janice Tufte, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2021-12-01 vote 6-0 by Philippe Rocca-Serra, Paola Rosati, Robin Ann Yurk, Joanne Dehnbostel, Jesus Lopez-Alcalde, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A measure of association that is the subtraction of the risk of an event in one group from the risk of the same event in another group.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: risk difference = The risk difference is the difference between the observed risks (proportions of individuals with the outcome of interest) in the two groups.
The risk difference is straightforward to interpret: it describes the actual difference in the observed risk of events between experimental and control interventions.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel, Harold Lehmann, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 5-0 by Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].definition
A statistic that is a subtraction of one difference from another.

# concept[].concept[].concept[].concept[].property[].valueString
The term 'Difference-in-differences' may be used to assess the incremental benefit or harm of an intervention or exposure, where the effect of the exposure is measured as a difference (for example, pre-post testing comparison of values before and after the exposure) in two groups being compared.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 6-0 by Robin Ann Yurk, Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 comment: do you want to add to comment for application, pre-post testing or as an Alternative term?

# concept[].concept[].concept[].definition
A statistic that is a difference between 1 and a ratio of the two quantities being compared.

# concept[].concept[].concept[].property[].valueString
Relative Difference = 1 - ( a / b ).   Because 1 - ( a / b ) is not equal to 1 - ( b / a ), Relative Difference may be expressed as "Relative Difference with respect to b" when referring to 1 - ( a / b ).

The relative difference can also be defined as a statistic that is a ratio of the absolute difference (of the two quantities being compared) to the reference value (one of the quantities being compared). Relative Difference = ( b - a ) / ( b ) where b is the reference value and this may also be called "Relative Difference with respect to b"

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
Example of a relative difference (relative to placebo) that is not a relative mean difference or a relative risk difference:

Relative median difference (%) = [(active median - placebo median) / placebo median] x 100. 
This can be transformed to:
Relative median difference = (active median / placebo median) - 1.

# concept[].concept[].concept[].property[].valueString
2022-06-15 vote 2-2 by Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte
2022-06-22 vote 4-2 by Mario Tristan, Eric M Harvey, Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-06-15 comments: Suggest combining this set of terms(relative difference, relative mean difference and relative risk difference  and summarizing the comment for applications so it is one term.

To me this definition is unclear...sorry, what it means? Is it weird to a have a ratio of a difference to a reference value? Sorry, but I am unable to understand this definition.

relative and absolute difference seems confusing to me

# concept[].concept[].concept[].property[].valueString
2022-06-29 vote 5-0 by Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A statistic that is a difference between 1 and a ratio of the two mean values being compared.

# concept[].concept[].concept[].concept[].property[].valueString
Relative Mean Difference = 1 - ( a / b ) where a and b are mean values.

The relative mean difference can also be defined as a statistic that is a ratio of the difference in means to the reference mean value. Relative Mean Difference = ( b - a ) / ( b ) where b is the reference mean value and a is another mean value.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-15 vote 3-1 by Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte
2022-06-22 vote 5-1 by Mario Tristan, Eric M Harvey, Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-15 comment: Suggest combining this set of terms(relative difference, relative mean difference and relative risk difference  and summarizing the comment for applications so it is one term.  
2022-06-29 comment: Relative Mean Difference is_a kind of Relative difference where the quantities being compared are two means, one of which is or acts as reference mean value  (additional comment: define 'reference mean value' if it refers to something more specific

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-29 vote 6-0 by Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Janice Tufte, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A statistic that is a difference between 1 and a ratio of the two risk values being compared.

# concept[].concept[].concept[].concept[].property[].valueString
Relative Risk Difference = 1 - ( a / b ) where a and b are risk values.

The relative risk difference can also be defined as a statistic that is a ratio of the risk difference to the risk used as a reference. Relative Risk Difference = ( b - a ) / ( b ) where b is the reference risk value and a is another risk value.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-15 vote 3-1 by Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte
2022-06-22 vote 5-1 by Mario Tristan, Eric M Harvey, Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-15 comment: Suggest combining this set of terms(relative difference, relative mean difference and relative risk difference  and summarizing the comment for applications so it is one term.

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-22 vote 6-0 by Mario Tristan, Eric M Harvey, Jesus Lopez-Alcalde, Robin Ann Yurk, Paola Rosati, Janice Tufte

# concept[].concept[].concept[].definition
A statistic that is a difference between two means, divided by a statistical measure of dispersion.

# concept[].concept[].concept[].property[].valueString
STATO: standardized mean difference (Cohen's d statistic, SMD) = standardized mean difference is data item computed by forming the difference between two means, divided by an estimate of the within-group standard deviation.
It is used to provide an estimatation of the effect size between two treatments when the predictor (independent variable) is categorical and the response(dependent) variable is continuous

# concept[].concept[].concept[].property[].valueString
In English, "standardized" is often used to express relative comparison to any reference value. However, in SEVCO, "standardized" is used to express relative comparison to a statistical measure of dispersion.
In SEVCO, the term Standardized Mean Difference is a description of the concept without an explicit type of statistical measure of dispersion. If the statistical measure of dispersion is specified, then a type (child term) of Standardized Mean Difference is preferred.

For example, in Cohen's d statistic, the statistical measure of dispersion is specified as the square root of an average of the variances of the two groups being compared.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2022-07-06 vote 3-1 by Paola Rosati, Harold Lehmann, Robin Ann Yurk, Eric Harvey

# concept[].concept[].concept[].property[].valueString
2022-07-06 comment: Consider listing Cohen's D statistic as an Alternative term or selecting one of the two terms to be the term to evaluate as the definition are similar.

# concept[].concept[].concept[].property[].valueString
2022-07-20 vote 6-0 by Janice Tufte, Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A standardized mean difference which is calculated as a difference between two means, divided by a square root of an average of the variances of the two groups.

# concept[].concept[].concept[].concept[].property[].valueString
A standardized mean difference is a statistic that is a difference between two means, divided by a statistical measure of dispersion. In SEVCO, the term Standardized Mean Difference is a description of the concept without an explicit type of statistical measure of dispersion. If the statistical measure of dispersion is specified, then a type (child term) of Standardized Mean Difference is preferred.

In Cohen's d statistic, the statistical measure of dispersion is specified as the square root of an average of the variances of the two groups being compared. The variances of the two groups are based on within-group standard deviations.

For sample sizes < 50, a correction factor is used.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: standardized mean difference (Cohen's d statistic, SMD) = standardized mean difference is data item computed by forming the difference between two means, divided by an estimate of the within-group standard deviation.
It is used to provide an estimatation of the effect size between two treatments when the predictor (independent variable) is categorical and the response(dependent) variable is continuous

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-06 vote 3-1 by Paola Rosati, Harold Lehmann, Robin Ann Yurk, Eric Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-06  comment: Consider listing SMD statistic as an Alternative term or selecting one of the two terms to be the term to evaluate as the definition are similar and the other term to be the Alternative term.

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-20 vote 5-0 by Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A standardized mean difference which is calculated as a difference between two means, divided by the standard error of the difference between the two means.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: strictly standardized mean difference (SSMS) is a standardized mean difference which corresponds to the ratio of mean to the standard deviation of the difference between two groups. SSMD directly measures the magnitude of difference between two groups. SSMD is widely used in High Content Screen for hit selection and quality control. When the data is preprocessed using log-transformation as normally done in HTS experiments, SSMD is the mean of log fold change divided by the standard deviation of log fold change with respect to a negative reference. In other words, SSMD is the average fold change (on the log scale) penalized by the variability of fold change (on the log scale). For quality control, one index for the quality of an HTS assay is the magnitude of difference between a positive control and a negative reference in an assay plate. For hit selection, the size of effects of a compound (i.e., a small molecule or an siRNA) is represented by the magnitude of difference between the compound and a negative reference. SSMD directly measures the magnitude of difference between two groups. Therefore, SSMD can be used for both quality control and hit selection in HTS experiments.

# concept[].concept[].concept[].concept[].property[].valueString
A standardized mean difference is a statistic that is a difference between two means, divided by a statistical measure of dispersion. In SEVCO, the term Standardized Mean Difference is a description of the concept without an explicit type of statistical measure of dispersion. If the statistical measure of dispersion is specified, then a type (child term) of Standardized Mean Difference is preferred.

In Strictly standardized mean difference, the statistical measure of dispersion is specified as the standard error of the difference between means [SEVCO TBD:0000063].

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-20 vote 6-0 by Janice Tufte, Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A standardized mean difference which is calculated as a difference between two means, divided by the pooled standard deviation.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Hedges's g = Hedges's g is an estimator of effect size, which is similar to Cohen's d and is a measure based on a standardized difference. However, the denominator, corresponding to a pooled standard deviation, is computed differently from Cohen's d coefficient, by applying a correction factor (which involves a Gamma function).

# concept[].concept[].concept[].concept[].property[].valueString
A standardized mean difference is a statistic that is a difference between two means, divided by a statistical measure of dispersion. In SEVCO, the term Standardized Mean Difference is a description of the concept without an explicit type of statistical measure of dispersion. If the statistical measure of dispersion is specified, then a type (child term) of Standardized Mean Difference is preferred.

In Hedges’s g, the statistical measure of dispersion is specified as the pooled standard deviation. There is a correction factor for small sample sizes.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-20 vote 5-0 by Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A standardized mean difference which is calculated as a difference between two means (of an experimental group and a control group), divided by the standard deviation of the control group.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Glass's delta is an estimator of effect size which is similar to Cohen's d but where the denominator corresponds only to the standard deviation of the control group (or second group). It is considered less bias than the Cohen's d for estimating effect sizes based on means and distances between means.

# concept[].concept[].concept[].concept[].property[].valueString
A standardized mean difference is a statistic that is a difference between two means, divided by a statistical measure of dispersion. In SEVCO, the term Standardized Mean Difference is a description of the concept without an explicit type of statistical measure of dispersion. If the statistical measure of dispersion is specified, then a type (child term) of Standardized Mean Difference is preferred.

In Glass's delta, the statistical measure of dispersion is specified as the standard deviation of the control group. There is a correction factor for small sample sizes.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-07-20 vote 5-0 by Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].definition
A statistic that is a quotient of one and a difference.

# concept[].concept[].property[].valueString
A difference is a statistic that is a subtraction of one quantity from another.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].property[].valueString
2022-07-20 vote 5-0 by Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].definition
A statistic that represents the number of units that needs to be treated to prevent one additional undesired outcome. The Number Needed to Treat is calculated as the reciprocal of a treatment effect estimate, where the effect estimate is expressed as a risk difference.

# concept[].concept[].concept[].property[].valueString
The Number Needed to Treat (NNT) value is often rounded up to the next highest whole integer.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
Centre for Evidence-Based Medicine
Number Needed to Treat (NNT)
The Number Needed to Treat (NNT) is the number of patients you need to treat to prevent one additional bad outcome (death, stroke, etc.).
https://www.cebm.ox.ac.uk/resources/ebm-tools/number-needed-to-treat-nnt

# concept[].concept[].concept[].property[].valueString
2022-07-20 vote 7-0 by Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].definition
A statistic that represents the number of units that needs to be tested to identify one additional case. The Number Needed to Screen to Detect is calculated as the reciprocal of a difference in rate of detected cases with and without screening.

# concept[].concept[].concept[].property[].valueString
The Number Needed to Screen (NNS) value is often rounded up to the next highest whole integer.  The Number Needed to Screen to Detect is distinct from the Number Needed to Screen to Prevent as the formulas to calculate are different, even though both may be abbreviated as NNS.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-08-03 vote 5-0 by Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey, Philippe Rocca-Serra, Harold Lehmann

# concept[].concept[].concept[].definition
A statistic that represents the number of units that needs to be tested to prevent one additional adverse outcome, assuming that positive testing will lead to preventive intervention. The Number Needed to Screen to Prevent is calculated as the Number Needed to Treat divided by the prevalence.

# concept[].concept[].concept[].property[].valueString
The Number Needed to Screen (NNS) value is often rounded up to the next highest whole integer.  The Number Needed to Screen to Detect is distinct from the Number Needed to Screen to Prevent as the formulas to calculate are different, even though both may be abbreviated as NNS.

The formula may be adjusted for test performance characteristics (e.g. dividing by the sensitivity) or assumptions regarding acceptance or adherence of interventions.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
BMJ 1998;317: 307 Number needed to screen: development of a statistic for disease screening
Number needed to screen is defined as the number of people that need to be screened for a given duration to prevent one death or adverse event.
Number needed to screen was then calculated by dividing the number needed to treat for treating risk factors by the prevalence of disease that was unrecognised or untreated. 
https://www.bmj.com/content/317/7154/307.long

# concept[].concept[].concept[].property[].valueString
2022-08-03 vote 5-0 by Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey, Philippe Rocca-Serra, Harold Lehmann

# concept[].concept[].concept[].definition
A statistic that represents the number of units that, if treated or exposed to the intervention, to lead to one additional undesired outcome. The Number Needed to Harm is calculated as the reciprocal of a treatment effect estimate, where the effect estimate is expressed as a risk difference.

# concept[].concept[].concept[].property[].valueString
The Number Needed to Harm (NNH) value is often rounded down to the next lowest whole integer.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte

# concept[].concept[].concept[].property[].valueString
Centre for Evidence-Based Medicine
Number needed to treat (NNT): The number of patients who need to be treated to prevent one bad outcome. It is the inverse of the ARR: NNT=1/ARR. Numbers needed to harm (NNH)-the number of patients who, if they received the experimental treatment, would lead to one additional person being harmed compared with patients who receive the control treatment; calculated as 1/ARI.
https://www.cebm.ox.ac.uk/resources/ebm-tools/glossary

# concept[].concept[].concept[].property[].valueString
2022-07-20 vote 7-0 by Cauê Monaco, Janice Tufte, Philippe Rocca-Serra, Mario Tristan, Jesus Lopez-Alcalde, Harold Lehmann, Eric Harvey

# concept[].concept[].definition
A statistic that is a quotient of two quantities.

# concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Muhammad Afzal

# concept[].concept[].property[].valueString
STATO: A ratio is a data item which is formed with two numbers r and s is written r/s, where r is the numerator and s is the denominator. The ratio of r to s is equivalent to the quotient r/s.
NCIt:  The quotient of one quantity divided by another, with the same units of measurement.
UMLS: Quotient of quantities of the same kind for different components within the same system.
OECD: A ratio is a number that expresses the relative size of two other numbers.
OCRe: A ratio is a quotient of quantities of the same kind for different components within the same system.
SCO: A ratio is a relationship between two numbers of the same kind expressed arithmetically as a dimensionless quotient of the two which explicitly indicates how many times the first number contains the second.
Quotient of quantities of the same kind for different components within the same system.

# concept[].concept[].property[].valueString
Although some definitions for Ratio include "with the same units of measurement" and some definitions for Ratio include "a dimensionless quotient", not all definitions have these concepts, and there are ratios with units of measurement that are different for numerator and denominator such as event rate, body mass index, and cost-effectiveness ratio.

# concept[].concept[].property[].valueString
2021-12-22 vote 3-1 by Robin Ann Yurk, Harold Lehmann, Janice Tufte, Jesus Lopez-Alcalde
2021-12-29 vote 3-1 by Robin Ann Yurk, Harold Lehmann, Janice Tufte, C P Ooi

# concept[].concept[].property[].valueString
2021-12-22 comment: I suggest adding "with the same measurement units"
2021-12-29 comment: I agree with the term definition.  However, the comment could be improved and I would not include BMI as an example as an index may not necessarily be a ratio but a more complex statistic or calculation
2022-01-05 comment: Comment,  I would remove body mass index from the comment section as an example as an index is a unique statistical defnition.

# concept[].concept[].property[].valueString
2022-01-05 vote 6-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, C P Ooi, Louis Leff, Jesus Lopez-Alcalde

# concept[].concept[].concept[].definition
A ratio that is multiplied by 100, and has the same units of measurement in the numerator and the denominator.

# concept[].concept[].concept[].property[].valueString
When a percentage is a fraction of hundred or proportion per hundred, then the percentage is the proportion multiplied by 100. However, a percentage can be greater than 100% so the definition is a ratio that is multiplied by 100.
Proportion is SEVCO code of TBD:0000018, Ratio is SEVCO code of STATO:0000184

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Janice Tufte, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
NCIt-A fraction or ratio with 100 understood as the denominator. Alt definition One hundred times the quotient of one quantity divided by another, with the same units of measurement.
OECD-A percentage is a special type of proportion where the ratio is multiplied by a constant, 100, so that the ratio is expressed per 100.
SCO-A fraction or ratio with 100 understood as the denominator.
UMLS-A unit for expressing a number as a fraction of hundred (on the basis of a rate or proportion per hundred)-NCI

# concept[].concept[].concept[].property[].valueString
2022-01-05 vote 5-1 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, C P Ooi, Louis Leff, Jesus Lopez-Alcalde

# concept[].concept[].concept[].property[].valueString
2022-01-05 comment: Instead of "A ratio" I would propose "A proportion that is multiplied by 100, [...]"

# concept[].concept[].concept[].property[].valueString
2022-01-07 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, C P Ooi, Louis Leff, Jesus Lopez-Alcalde, Mario Tristan

# concept[].concept[].concept[].concept[].definition
A percentage in which the numerator represents the absolute value of one minus the difference between the true value and the observed value, and the denominator represents the true value.

# concept[].concept[].concept[].concept[].property[].valueString
from https://www.sciencedirect.com/topics/engineering/measurement-accuracy
Measurement Accuracy
Measurement accuracy is defined as the closeness of agreement between a measured quantity value and a true quantity value of a measurand (i.e., the quantity intended to be measured) (ISO-JCGM 200, 2008), and is often limited by calibration errors.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-24 vote 5-0 by Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, Cauê Monaco, Eric Harvey

# concept[].concept[].concept[].definition
A ratio in which the numerator represents a part, fraction or share of the amount represented by the denominator.

# concept[].concept[].concept[].property[].valueString
STATO: observed risk [as a data item STATO_0000423] = the proportion of individuals in a population with the outcome of interest

NCIt: A part, fraction, share, or number considered in relation to the whole amount or number.

OECD Definition:
A proportion is a special type of ratio in which the denominator includes the numerator.

An example is the proportion of deaths that occurred to males which would be deaths to males divided by deaths to males plus deaths to females (i.e. the total population).

OCRe: A proportion is a measure of the frequency of some phenomenon of interest within an average population

# concept[].concept[].concept[].property[].valueString
The value of a proportion must be between 0 and 1 (inclusive). Proportions may represent the frequency of some phenomenon of interest within a population, or may represent a subset of a whole.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2021-12-29 vote 3-1 by Robin Ann Yurk, Harold Lehmann, Janice Tufte, C P Ooi

# concept[].concept[].concept[].property[].valueString
2021-12-29 comment: I agree with the term definition.  However, for the comment, I would edit to include the OCRe defintion:  A proportion is a measure of the frequency of some phenomenon of interest within a population.

# concept[].concept[].concept[].property[].valueString
2022-01-07 vote 7-0 by Harold Lehmann, Robin Ann Yurk, Janice Tufte, C P Ooi, Louis Leff, Jesus Lopez-Alcalde, Mario Tristan

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents new events.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Incidence is the ratio of the number of new cases of a disease divided by the number of persons at risk for the disease.

NCIt The relative frequency of occurrence of something.

OBCS A data item that refers to the number of new events that have occurred in a specific time interval divided by the population at risk at the beginning of the time interval. The result gives the likelihood of developing an event in that time interval.

UMLS The number of new cases of a given disease during a given period in a specified population. It also is used for the rate at which new events occur in a defined population. It is differentiated from PREVALENCE, which refers to all cases in the population at a given time. (MSH)
The relative frequency of occurrence of something. (NCI)
The number of new cases of a disease diagnosed each year. (NCI)

CDC: https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section2.html
Incidence refers to the occurrence of new cases of disease or injury in a population over a specified period of time. Although some epidemiologists use incidence to mean the number of new cases in a community, others use incidence to mean the number of new cases per unit of population.

Two types of incidence are commonly used — incidence proportion and incidence rate.

Incidence proportion or risk

Synonyms for incidence proportion

Attack rate
Risk
Probability of developing disease
Cumulative incidence


Definition of incidence proportion

Incidence proportion is the proportion of an initially disease-free population that develops disease, becomes injured, or dies during a specified (usually limited) period of time. Synonyms include attack rate, risk, probability of getting disease, and cumulative incidence. Incidence proportion is a proportion because the persons in the numerator, those who develop disease, are all included in the denominator (the entire population).

# concept[].concept[].concept[].concept[].property[].valueString
Outside of the Scientific Evidence Code System (SEVCO), there is substantial inconsistency in the terms and definitions used for incidence and related concepts.

Within SEVCO, Incidence is a proportion in which the numerator represents new events. The denominator may represent the entire population or may represent that population at risk (i.e., those without prior events). 

Disease incidence is the ratio of the number of new cases of a disease divided by the number of persons at risk for the disease. Incidence is a proportion because the persons in the numerator, those who develop disease, are all included in the denominator (the entire population).

When a time period or a duration of time is used to define the period of time in which the incidence is measured, the statistic type is Incidence. Examples include 1-year incidence, in-hospital incidence, and cumulative incidence.

When time is considered as a variable in the formalization of the statistic, such as incidence per unit of time, then the statistic type is Incidence Rate (SEVCO code of TBD:0000024)

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-05 vote 2-2 by Robin Ann Yurk, janice tufte, Jesus Lopez-Alcalde, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-05 comments: I propose "The number of new occurrences of an event (for example, infection) in a population at risk over a particular period of time.

Doesn't denominator need to include a time component? Even if not, the time component should be referenced in the Comment. Also, do note that, in public health, the numerator *attempts* to be a subset of the denominator, but that relationship cannot be assured. (E.g., fertility incidence may be number of births (vital statistics) with denominator of number of women of child bearing age (census).

2022-06-15 Expert Working Group/Steering Committee removed 'Risk' as Alternative term as we created a separate term for 'Risk' (TBD:0000185)

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-12 vote 6-0 by Harold Lehman, Mario Tristan, janice tufte, Andrew Beck, Robin Ann Yurk, Paul Harris

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents all events of interest (for example, both new and preexisting cases of a disease) in the population, which is represented by the denominator.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: prevalence is a ratio formed by the number of subjects diagnosed with a disease divided by the total population size.

Period prevalence:
The ratio (for a given time period) of the number of occurrences of a disease or event to the number of units at risk in the population.
a prevalence rate that occurs at a specific period of time


Point prevalence:
NCIt The ratio (for a given time period) of the number of occurrences of a disease or event to the number of units at risk in the population.
OBCS a prevalence rate that occurs at a specific point of time

UMLS:
The total number of cases of a given disease in a specified population at a designated time. It is differentiated from INCIDENCE, which refers to the number of new cases in the population at a given time. (MSH)
The ratio (for a given time period) of the number of occurrences of a disease or event to the number of units at risk in the population. (NCI)
Proportion of the people having a certain disease or condition in a given population (CHV)

CDC https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section2.html
Point prevalence = Number of current cases (new and preexisting) at a specified point in time / Population at the same specified point in time
Period prevalence = Number of current cases (new and preexisting) over a specified period of time / Average or mid-interval population

Definition of prevalence
Prevalence, sometimes referred to as prevalence rate, is the proportion of persons in a population who have a particular disease or attribute at a specified point in time or over a specified period of time. Prevalence differs from incidence in that prevalence includes all cases, both new and preexisting, in the population at the specified time, whereas incidence is limited to new cases only.

Point prevalence refers to the prevalence measured at a particular point in time. It is the proportion of persons with a particular disease or attribute on a particular date.

Period prevalence refers to prevalence measured over an interval of time. It is the proportion of persons with a particular disease or attribute at any time during the interval.

# concept[].concept[].concept[].concept[].property[].valueString
Prevalence is the proportion of persons in a population who have a particular disease or attribute at a specified point in time or over a specified period of time.
Prevalence differs from incidence in that prevalence includes all cases, both new and preexisting, in the population at the specified time, whereas incidence is limited to new cases only.
In Bayesian calculations, the prevalence value is often used as the pre-test probability or prior probability value, but these probability values are not always based on or derived from the prevalence value.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel, Janice Tufte, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-12 vote 6-1 by Robin Ann Yurk, janice tufte, Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, Andrew Beck, Paul Harris
2022-01-19 vote 2-1 by Harold Lehmann, Robin Ann Yurk, Alejandro Piscoya
2022-01-26 vote 7-1 by Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, Paola Rosati, Mario Tristan, Robin Ann Yurk, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-12 comments: I propose "A proportion in which the numerator represents all events (new and preexisting)." I think it is very important to detail "new and preexisting"
2022-01-03 comment: The comment here is better than for incidence (why not copy this comment into "Incidence," and edit?). But it still feels like the denominator should be called out in the definition.
2022-01-19 comment:  I would edit the term definition to include ...as part of a denominator of a broader population.
2022-01-26 comments: (1) suggestion: alter the definition to: A proportion in which the numerator represents all events of interest (e.g. both new and preexisting cases of a disease) in the population, which is represented by the denominator.
(2) I would delete this sentence from the comment for application. " Prevalence is a proportion because the persons in the numerator, those who develop or have disease, are all included in the denominator (the entire population)"
(3) Probability should be a type of Proportion but distinct from Prevalence. Probability relates to the likelihood of something, but in that sense incidence and prevalence are both probabilities.  If Prevalence and Probability were considered synonyms then one would still not call it the same as "Pre-test" or "Prior" probability.  The term pre-test probability could be a type of (child of) probability.

2022-02-02 comment: I would remove the statement ..In Bayesian calculations, as the pre-test probability is a formula with new variables.

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-02 vote 5-0 by Paola Rosati, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the detected items within the denominator that represents all items with the targeted attribute.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: true positive rate (recall, sensitivity) = sensitivity is a measurement datum qualifying a binary classification test and is computed by substracting the false negative rate to the integral numeral 1
NCIt diagnostic sensitivity The probability that a test will produce a true positive result when used on effected subjects as compared to a reference or "gold standard". The sensitivity of a test can be determined by calculating: number of true positive results divided by the sum of true positive results plus number of false negative results.
OBCS-
a data item that measures the proportion of actual positives which are correctly identified as such (e.g. the percentage of sick people who are correctly identified as having the condition).
OCRe
An index of performance of a discriminant test calculated as the percentage of correct positives in all true positives
STATO
sensitivity is a measurement datum qualifying a binary classification test and is computed by subtracting the false negative rate to the integral numeral 1
NICE glossary-Sensitivity of a test-How well a test detects what it is testing for. It is the proportion of people with the disease or condition that are correctly identified by the study test. For example, a test with a sensitivity of 96% will, on average, correctly identify 96 people in every 100 who truly have the condition, but incorrectly identify as not having the condition 4 people in every 100 who truly have it. It is different from positive predictive value.

MeSH scope note-sensitivity and specificity-Scope Note
Binary classification measures to assess test results. Sensitivity or recall rate is the proportion of true positives. Specificity is the probability of correctly determining the absence of a condition. (From Last, Dictionary of Epidemiology, 2d ed)

# concept[].concept[].concept[].concept[].property[].valueString
In a population of people with and without a disease, and a test which is positive (suggesting the disease) or negative (suggesting not having the disease), the sensitivity is the proportion of true positives (all people with the disease who test positive) within all people with the disease (true positives plus false negatives). Sn = TP / (TP + FN).
In information retrieval, recall is the proportion of items correctly retrieved within all relevant items.
True positive rate (TPR) is listed as an Alternative term because of common usage, but TPR is not a Rate as defined in SEVCO.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-19 vote 3-1 by Paul Harris, Harold Lehmann, Robin Ann Yurk, Alejandro Piscoya

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-19 comment: I would remove recall from Alternative terms and comment for applications, as it is a specialized informatics measures and list it as a separate term. (EWG discussion: This comment is not persuasive. If the same statistic type (formula) has different names in different contexts we still want one common code for the concept. This consolidation of terms is the purpose of a standardized terminology or controlled vocabulary where we are controlling the code for the concept, not the name for common use.)

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-26 vote 10-0 by Paul Harris, Harold Lehmann, Robin Ann Yurk, Alejandro Piscoya, Janice Tufte, Philippe Rocca-Serra, Paola Rosati, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the non-detected items within the denominator that represents all items without the targeted attribute.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: true negative rate (specificity) = specificity is a measurement datum qualifying a binary classification test and is computed by substracting the false positive rate to the integral numeral 1


NCIt The probability that a test will produce a true negative result when used on non-effected subjects as compared to a reference or "gold standard". The specificity of a test can be determined by calculating: number of true negative results divided by the sum of true negative results plus number of false positive results.

OBCS a data item that refers to the proportion of negatives in a binary classification test which are correctly identified

OCRe An index of performance of a discriminant test calculated as the percentage of negatives in all true negatives

NICE glossary-Specificity (of a test)
How well a test correctly identifies people who do not have what it is testing for. It is the proportion of people without the disease or condition that are correctly identified by the study test. For example, a test with a specificity of 96% will, on average, correctly identify 96 people in every 100 who truly do not have the condition, but incorrectly identify as having the condition 4 people in every 100 who truly do not have it. It is different from negative predictive value.

MeSH scope note-sensitivity and specificity-Scope Note
Binary classification measures to assess test results. Sensitivity or recall rate is the proportion of true positives. Specificity is the probability of correctly determining the absence of a condition. (From Last, Dictionary of Epidemiology, 2d ed)

# concept[].concept[].concept[].concept[].property[].valueString
In a population of people with and without a disease, and a test which is positive (suggesting the disease) or negative (suggesting not having the disease), the specificity is the proportion of true negatives (all people without the disease who test negative) within all people without the disease (true negatives plus false positives). Sp = TN / (TN + FP).
True Negative Rate (TNR) is listed as an Alternative term because of common usage, but TNR is not a Rate as defined in SEVCO.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-19 vote 5-0 by Paul Harris, Harold Lehmann, Robin Ann Yurk, Alejandro Piscoya, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the correctly detected items within the denominator that represents all items detected.

# concept[].concept[].concept[].concept[].designation[].value
precision (as used in information retrieval)

# concept[].concept[].concept[].concept[].property[].valueString
NCIt The probability that an individual is affected with the condition when a positive test result is observed. Predictive values should only be calculated from cohort studies or studies that legitimately reflect the number of people in the population who have the condition of interest at that time since predictive values are inherently dependent upon the prevalence. PPVDT can be determined by calculating: number of true positive results divided by the sum of true positive results plus number of false positive results.

# concept[].concept[].concept[].concept[].property[].valueString
In a population of people with and without a disease, and a test which is positive (suggesting the disease) or negative (suggesting not having the disease), the positive predictive value is the proportion of true positives (all people with the disease who test positive) within all the people with a positive test (true positives plus false positives). PPV = TP / (TP + FP).
In information retrieval, 'precision' is the proportion of items correctly retrieved within all retrieved items.
In Bayesian calculations, the 'Positive Predictive Value' is equivalent to the 'post-test probability' or 'posterior probability' following a positive test.

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-26 vote 7-1 by Robin Ann Yurk, Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, Paola Rosati, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-26 comments: (1) I would remove precision from Alternative terms
(2) minor change=quote terms of interest: In information retrieval, `precision` is the proportion of items correctly retrieved within all retrieved items.
The terms `post-test probability` and `posterior probability` are used in Bayesian calculations.
(3) Post-test probability is not fully synonymous with positive predictrive value.  A negative predictive value is also the "post-test" probability of a true negative if the test has a negative result.  And a test with a continuous rather than binary result could have a post-test probability that is neither positive nor negative predictive value.  Post-test probability (and posterior probability) should become a child of probability.
2022-02-02 comment: I would remove the alternate term Precision and the comment for application for precision.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Phillippe Rocca-Serra, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-02-02 vote 5-0 by Paola Rosati, Mario Tristan, Robin Ann Yurk, Janice Tufte, Brian S. Alper

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the correctly non-detected items within the denominator that represents all items not detected.

# concept[].concept[].concept[].concept[].property[].valueString
NCIt The probability that an individual is not affected with the condition when a negative test result is observed. This measure of accuracy should only be used if the data on the prevalence of condition of interest in given population is available. NPVDT can be determined by calculating: number of true negative results divided by the sum of true negative results plus number of false negative results.

# concept[].concept[].concept[].concept[].property[].valueString
In a population of people with and without a disease, and a test which is positive (suggesting the disease) or negative (suggesting not having the disease), the negative predictive value is the proportion of true negatives (all people without the disease who test negative) within all the people with a negative test (true negatives plus false negatives). NPV = TN / (TN + FN).

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Ken Wilkins, Phillippe Rocca-Serra, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-01-26 vote 8-0 by Robin Ann Yurk, Janice Tufte, Harold Lehmann, Philippe Rocca-Serra, Paola Rosati, Mario Tristan, Brian S. Alper, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the correctly detected items within the denominator that represents all items tested.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
"Diagnostic yield was defined as the number of participants with positive findings for advanced neoplasia relative to all participants" in https://pubs.rsna.org/doi/10.1148/radiol.12112486
Other 'definitions' found include synonymous use with sensitivity, and 'diagnostic yield' describing the statistic array of TP, FP, TN, and FN data.

https://medical-dictionary.thefreedictionary.com/diagnostic+yield
Diagnostic yield The likelihood that a test or procedure will provide the information needed to establish a diagnosis.

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-10 vote 5-0 by Philippe Rocca-Serra, Mario Tristan, Harold Lehmann, Eric Harvey, Paola Rosati

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-10 comment:  I would simply suggest to simplify the definition to:

A proportion obtained by dividing the number of correctly detected items (numerator) by the number of all items tested (denominator)

# concept[].concept[].concept[].concept[].definition
A proportion in which the numerator represents the cases in which an event or characteristic occurs and the denominator represents all possible cases.

# concept[].concept[].concept[].concept[].property[].valueString
In the English language, 'risk' may be used synonymously with 'hazard', 'chance', 'likelihood', 'relative likelihood', 'probability' and many other terms. In SEVCO the term 'risk' is explicitly defined for how it is used in other terms such as 'Risk Ratio' and 'Relative Risk Difference'  The statistical definition of 'risk' does not have a negative or undesirable connotation.

Risk may be conditioned on many factors.  In such cases the statistic type is Risk and the statistic may be reported as a conditional risk (for example, predicted risk).

When a time period or a duration of time is used to define the period of time in which the risk is measured, the statistic type is Risk. Examples include 1-year risk, in-hospital risk, and cumulative risk.

In frequentist statistics, the risk is a ratio of the number of events to the number of possible cases. In subjective Bayesian statistics, the risk is a proportion as a whole that represents degree of belief, where 0 represents certainty that an event will not occur and 1 represents certainty that the event will occur.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-22 vote 4-1 by Eric Harvey, Janice Tufte, Mario Tristan, Muhammad Afza, Eric M Harvey

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-22 comment: It is confusing to consider "conditional probability" as an Alternative term for "risk." 

Generally; 
"Conditional probability" refers to a probability whose value is dependent upon the occurrence of some process/event. In contrast, "risk" refers to the probability that an event will occur. 

Mathematically;
"Conditional probability" is a measure of the probability of an event occurring, given that another event has already occurred. Let us have two events, A and B, and we want to know P(A) given P(B); notationally, P(A|B). Here the word 'given' defines a subset of the population of events because it applies condition on B. For example, if we care about the incidence of COVID-19 in men only, we might want to know P(COVID-19 | male). This means that first, pick out all the males, and second, figure out the probability they will get COVID-19. More formally, what P(A|B) says is: pick out the events to which both P(A) and P(B) apply and consider them as part of the subset of events to which only P(B) applies: hence P(A/B) = P(A and B)/P(B).
In simple words, what we are doing with P(A|B) = P(A and B) | P(B) is selecting out the same subset of the event population in both the
the numerator and the denominator: in this case, only men.
While "risk" by definition involves no condition. Taking the same example, we can say, "what is the risk of COVID-19?" here, we refer to the whole population; however, we can apply can make a condition over it like "what is the risk of COVID-19 in males?" This risk may be taken as "conditional risk," and it could be taken as an Alternative term to conditional probability.  
Conclusion: Let us define two terms, "risk" and "conditional risk," as a subset of "risk." Then "conditional probability" shall be taken as an Alternative term to "conditional risk."

One more important point about the current definition of "Risk," i.e., 
Risk = A proportion in which the numerator represents the probability that an event or characteristic occurs and the denominator represents the probability that the event or characteristic occurs or does not occur.
If we write symbolically, it will look like this; P(A)/P(A or B), where A indicates "positive," which is the occurrence of something, and B indicates "negative," which is the non-occurrence of the same. We can write it formally as P(A) / P(AUB). In set theory, when there is "OR," in other words, "Union" infer the True value when either of them is True. It means A is true, or B is true, or both are true; we will get the true result. Interestingly, occurrence and non-occurrence are mutually exclusive, so two situations arise.
I) when the event occurs: P(A) / P(AUB)  --> P(A)/P(A) = 1
II) when the event does not occur: P(A) / P(AUB)  --> P(A)/P(B)  = Odds
Therefore the definition needs to be revised for the correct meaning of the denominator. I believe the denominator refers to the whole population where some people are at risk and some are not, while the numerator refers to only those at risk.

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-29 vote 6-0 by Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Janice Tufte, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].definition
A ratio in which the numerator represents the probability that an event will occur and the denominator represents the probability that an event will not occur.

# concept[].concept[].concept[].property[].valueString
OCRe:
Odds is a quotient in which the relative likelihood that an event will occur is divided by the the relative likelihood that it won't. In probability theory and statistics, where the variable "p" is the probability in favor of the event, and the probability against the event is 1-p, "the odds" of the event are the quotient of the two, or p / (1-p)

# concept[].concept[].concept[].property[].valueString
'Odds' and 'Odds ratio' are different terms. 'Odds' is a ratio of probabilities. 'Odds ratio' is a ratio of two different odds.

Odds are calculated as p / (1-p) where p is the probability of event occurrence.  When p = 0, the odds = 0. When p = 1, the odds may be expressed as not calculable or as "odds against = 0".

Odds may be expressed as p:(1-p). Odds may be expressed as p:q where q = 1-p. Odds may be expressed as a:b where a and b are multiples of p and (1-p). Examples of different expressions of the same odds include 3:2, 3/2, 0.6:0.4, 0.6/0.4, and 1.5.

Odds may be expressed as "odds for" or "odds in favor" (e.g. 1:5 for a "3" on a 6-sided die) or "odds against" (e.g. 5:1 against a "3" on a 6-sided die).

The term "betting odds" used in gambling that involves financial amounts in the formulation is not an "Odds" in the definition of the Scientific Evidence Code System.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Khalid Shahin, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2022-02-16 vote 4-2 by Janice Tufte, Paola Rosati, Eric Moyer,  Harold Lehmann, Robin Ann Yurk, Jesus Lopez-Alcalde
2022-02-23 vote 5-2 by nisha mathew, Harold Lehman, Paola Rosati, Sunu Alice Cherian, Robin Ann Yurk,  Joanne Dehnbostel, Sumalatha A
2022-03-09 vote 3-1 by Robin Ann Yurk, Janice Tufte, Eric Moyer, nisha mathew
2022-03-16 vote 8-1 by Robin Ann Yurk, Janice Tufte, Eric Moyer, nisha mathew, Harold Lehmann, Philippe Rocca-Serra, Louis Leff, Paola Rosati, Mario Tristan

# concept[].concept[].concept[].property[].valueString
2022-02-16 comments: The term definition and comment for application are clear and well written.  It would help to have a discussion on the Parent and Child relationships for this term as right now you only have Statistic, Ratio, Odds.  In statistics for the scientific code system is Statistic, Ratio, Odds Ratio a better sequence and put the Odds under comment for application.  

This term needs two Alternative terms: "Odds For" and "Odds in Favor." The definition needs to deal with the cases p=1 and p=0. (I can think of 3 questions regarding these cases. (1) Are they defined? (2) Is p=1 the same as 8. (3) Does 3:0=1:0?)

We should mention that this term does not include gambling odds. (As I understand it, gambling odds are the ratio of stake to winnings with several representations and frequently have a "rounding" factor to ensure a profit for the bookmaker).

Another issue is whether to represent "Odds Against" in the vocabulary. It could come up when annotating an immutable pre-existing source that gives odds as odds against; for example, an NLP system that scans published works to output labels for sections of the text.

A term related to "Odds" missing from the parent branch, "Ratio," is "Log Odds."

(Not unique to this term, but I noticed it here) The children of "Statistic" should inherit the application comment from "Statistic" about distinguishing between the statistic and statistic value. That way, a reader will not need to read the whole tree to know that 1.5 is not "Odds"; it is "Odds statistic value." (However, I do not see a place for "Odds statistic value" in the tree.)

Finally, the repetition of "Odds may be expressed as" is awkward.

2022-02-23 comments: "Odds may be expressed as p:(1-p). Odds may be expressed as p:q where q = 1-p. " Sounds redundant.
Alternative terms: Probablity likelihood, chance ---{{Group meeting decided that 'probability' and 'likelihood' are terms we may consider adding to the SEVCO but they are not Alternative terms for odds, 'chance' is considered a lay term and not a specific statistical term for the code system}}
Odds is a computational function such as addition, subtraction, multiplication.  Odds Ratio may be better term for the term definition.  This comment is based on your term definition and comment for application.  

2022-03-09 comment: Edit the term definition:  A ratio of probabilities in which the numerator represents the probability of the number of times an event will occur and the denominator represents the probability of the number of time an event will not occur.  (Steering group 2022-03-09 considers the suggested change does not add clarification or improved understanding.)

2022-03-16 comment: I would delete likelihood from the term definition as in statistics it introduces a different formula such as likelihood ratio.
My suggestion is to simplify to a ration in which the numerator represents the number of times an event will occur and the denominator represents the number of times an event will not occur. (Steering group 2022-03-16 again considers the suggested change to include "number of times" not persuasive, but changed "relative likelihood" to "probability" in the definition to avoid the potential confusion with likelihood ratio.)

# concept[].concept[].concept[].property[].valueString
2022-03-22 vote 5-0 by Muhammad Afzal, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Robin Ann Yurk

# concept[].concept[].concept[].definition
A ratio in which the numerator represents any quantity and the denominator represents an interval of time.

# concept[].concept[].concept[].property[].valueString
When the numerator represents a count, the rate is an Event Rate.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
NCIt
Rate = A measurement of degree, speed, or frequency relative to time.

OBCS
rate= A quality of a single process inhering in a bearer by virtue of the bearer's occurrence per unit time.

OCRe
Rate = A rate is a quantity per unit of time.

# concept[].concept[].concept[].property[].valueString
2022-03-30 vote 4-2 by Cauê Monaco, Muhammad Afzal, Jesus Lopez-Alcalde, Philippe Rocca-Serra, Mario Tristan, Robin Ann Yurk
2022-04-06 vote 3-2 by Mario Tristan, Robin Ann Yurk, Paola Rosati, Jesus Lopez-Alcalde, nisha mathew

# concept[].concept[].concept[].property[].valueString
2022-03-30 comments: A rate does not necessarily represent time. "In math, a rate is a special ratio in which the two terms are in different units"
Edit term definition:  A proportion represented by a rate of an event count for another quantified measure.
 My comments are more focused on in the term definition and comment you have incomplete definitions or comments as you only describe in the term definition the denominator and in the comment you only describe the numerator.
Improvement Suggestion:  By definition, a rate would have both a numerator and denominator so it is important for your to include in a definition both numerator and denominator.

The term definition should read:  A ratio in which the numerator represents an event count and the denominator represents the total sum of the events considered as a count and non count.

The underlying concept for Rate is that the Denominator is a measure of time.   So we need a definition where the numerator is X and the denominator is a measure time.
 
Our approach to definitions has been:
 
Ratio = A statistic that is a quotient of two quantities.  [[By definition any statistic that is a ratio has a numerator and a denominator.   Any statistic that has a numerator and a denominator is a Ratio, and may be given a more specific term when it is a type of Ratio.]]   The Ratio definition inherits the Statistic definition so we do not re-define statistic.
 
Rate = A ratio in which the denominator represents a duration of time.     This means that when we constrain the definition of ratio to limit to statistics where the denominator represents a duration of time, then the type of Ratio is a Rate.   There is a logic to this approach to setting a definition, but your comment shows that it feel lacking because it does not mention the numerator.  There is no constraint or modification being applied to the numerator.
 
Perhaps we can try “Rate = A ratio in which the numerator represents any quantity and the denominator represents a duration of time.”
 
Would that help clarify this item?

2022-04-06 comments: I would insert in the term definition, the numerator represents a quantity defined as a unit which is a smaller part of the denominator divided by the total sum of units in the denominator.

the concepts,  "frequency of events" and  "over a specified period of time"  are not reflected in this definition 

2022-04-27 comment: Edit term definition:  A proportion represented by a rate of an event count or another quantified measure divided by the total sum of units.  {{Discussion by Expert Working Group: The proposed definition describes a Proportion, but a Rate is NOT a Proportion.}}

# concept[].concept[].concept[].property[].valueString
2022-05-12 vote 9-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A rate in which the number of new events per total at risk is divided by an interval of time.

# concept[].concept[].concept[].concept[].property[].valueString
Incidence is defined as a proportion in which the numerator represents new events and the denominator represents the total at risk for events. Rate is defined as a ratio in which the numerator represents any quantity and the denominator represents an interval of time. The interval of time used for the denominator may be data-dependent when the duration of observation varies across the observations.

In the method for calculating incidence rate (described at https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section2.html), the numerator is the "Number of new cases of disease or injury during the specified period" and the denominator is the "Time each person was observed, totaled for all persons"

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
NCIt 
Incidence Rate = The frequency of new occurrences of an event during a specified time period.

CDC: https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section2.html
Incidence refers to the occurrence of new cases of disease or injury in a population over a specified period of time. Although some epidemiologists use incidence to mean the number of new cases in a community, others use incidence to mean the number of new cases per unit of population.

Two types of incidence are commonly used — incidence proportion and incidence rate.

Synonyms for incidence rate

Person-time rate


Definition of incidence rate

Incidence rate or person-time rate is a measure of incidence that incorporates time directly into the denominator. A person-time rate is generally calculated from a long-term cohort follow-up study, wherein enrollees are followed over time and the occurrence of new cases of disease is documented. Typically, each person is observed from an established starting time until one of four “end points” is reached: onset of disease, death, migration out of the study (“lost to follow-up”), or the end of the study. Similar to the incidence proportion, the numerator of the incidence rate is the number of new cases identified during the period of observation. However, the denominator differs. The denominator is the sum of the time each person was observed, totaled for all persons. This denominator represents the total time the population was at risk of and being watched for disease. Thus, the incidence rate is the ratio of the number of cases to the total time the population is at risk of disease.


Alternative terms for incidence rate (incidence density, average hazard) noted at https://www.sjsu.edu/faculty/gerstman/eks/formula_sheet.pdf

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-11 vote 7-1 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Janice Tufte, Harold Lehmann, Paola Rosati, Robin Ann Yurk

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-11 comment: Suggest improving current term definition with the definition in comment for application.  The Alternative terms I am not sure fit here--you may want to add more detail for the alternate terms to the comment for application.
2022-05-25 comment: The definition defines the ideal ("at risk"); very often, however, incidence rates are calculated more grossly. While they are semantically wrong, they are quantitatively correct. Classic: birth incidence. The proper denominator would be fertile women, but *could * be calculated "per woman" or even "per capita".

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-25 vote 6-0 by Jesus Lopez-Alcalde, Brian S. Alper, Joanne Dehnbostel, Eric M Harvey, Mario Tristan, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A conditional instantaneous rate in which the numerator represents an incidence conditioned on survival to a specified time, and the denominator represents a time interval with a duration approaching zero.

# concept[].concept[].concept[].concept[].property[].valueString
In the definition of Hazard Rate, the term "survival" is not literally about life and death but is used to represent existence without experiencing the event. "Hazard" as a statistical term is not specific to "bad" or "dangerous" events.
A hazard rate is expressed as a unitless numerator per unit of time, occurring at a specified time, and conditioned on survival to that time. 
A hazard rate is mathematically the negative derivative of the log of the survival function. The survival function is the probability of surviving past a specified point in time, expressed as Pr{ T >= t }. 
A hazard rate is also mathematically defined as lim(dt -> 0) [ Pr{ ( t <= T < t + dt ) | ( T >= t ) } / dt ].

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
A Dictionary of Epidemiology (5 ed.) by Miquel Porta
Hazard rate = A theoretical measure of the probability of occurrence of an event per unit time at risk; e.g., death or new disease, at a point in time, t, defined mathematically as the limit, as ?t approaches zero, of the probability that an individual well at time t will experience the event by t + ?t, divided by ?t.

formula expressed at https://data.princeton.edu/wws509/notes/c7s1

# concept[].concept[].concept[].concept[].property[].valueString
2022-04-06 vote 4-3 by Mario Tristan, Robin Ann Yurk, Cauê Monaco, Harold Lehmann, Paola Rosati, Jesus Lopez-Alcalde, nisha mathew

# concept[].concept[].concept[].concept[].property[].valueString
2022-04-06 comments: An instantaneous rate in which the numerator represents an incidence and the denominator represents a time interval conditioned on survival to a specified time with a duration approaching zero

A hazard is any danger or peril. It does not necessarily represent a survival/death relationship.

I would add a vote choice:  No Comment-Specialized Term or Not Applicable or some other choice as this is specialized formula.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-12 vote 8-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew

# concept[].concept[].concept[].concept[].definition
The number of occurrences per unit of time.

# concept[].concept[].concept[].concept[].property[].valueString
An event rate is a ratio in which the numerator represents a count and the denominator represents an interval of time.
When the numerator represents a count: 
--If the denominator includes an interval of time, the type of ratio is an Event Rate. 
--If the denominator includes a count without an interval of time, the type of ratio is an Event Frequency. 
--If the denominator includes a count and an interval of time, the type of ratio is an Event Frequency Rate. 
--If the denominator includes an interval of space, the type of ratio is a Number Density

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-04-27 comment: The term definition and comment for application are comprehensive.  However, for your comment for application I would only use the following...
An event rate is a ratio in which the numerator represents a count and the denominator represents an interval of time.
When the numerator represents a count: 
--If the denominator includes an interval of time, the type of ratio is an Event Rate. 
{{Expert Working Group discussion: the comment providing instructions for choosing among 4 related and confusing terms is considered useful for guidance, and purposefully mentions other terms that may be more appropriate.}}

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-12 vote 8-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew

# concept[].concept[].concept[].concept[].definition
A ratio in which the numerator represents an event frequency and the denominator represents an interval of time.

# concept[].concept[].concept[].concept[].property[].valueString
When the numerator represents a count: 
--If the denominator includes an interval of time, the type of ratio is an Event Rate. 
--If the denominator includes a count without an interval of time, the type of ratio is an Event Frequency. 
--If the denominator includes a count and an interval of time, the type of ratio is an Event Frequency Rate. 
--If the denominator includes an interval of space, the type of ratio is a Number Density

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-12 vote 9-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Janice Tufte

# concept[].concept[].concept[].definition
A ratio in which the numerator represents a count and the denominator represents a count (without involving an interval of time).

# concept[].concept[].concept[].property[].valueString
When the numerator represents a count: 
--If the denominator includes an interval of time, the type of ratio is an Event Rate. 
--If the denominator includes a count without an interval of time, the type of ratio is an Event Frequency. 
--If the denominator includes a count and an interval of time, the type of ratio is an Event Frequency Rate. 
--If the denominator includes an interval of space, the type of ratio is a Number Density

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-05-12 vote 9-0 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Janice Tufte

# concept[].concept[].concept[].definition
A ratio in which the numerator represents any quantity and the denominator represents an interval of space (distance, area, or volume).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-05-08 vote 7-0 by Mario Tristan, Janice Tufte, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Eric M Harvey

# concept[].concept[].concept[].property[].valueString
2022-05-08 comment: Examples would be nice, since "linear density" is not a traditional measure

# concept[].concept[].concept[].concept[].definition
A ratio in which the numerator represents a count and the denominator represents an interval of space (distance, area, or volume).

# concept[].concept[].concept[].concept[].property[].valueString
When the numerator represents a count: 
--If the denominator includes an interval of time, the type of ratio is an Event Rate. 
--If the denominator includes a count without an interval of time, the type of ratio is an Event Frequency. 
--If the denominator includes a count and an interval of time, the type of ratio is an Event Frequency Rate. 
--If the denominator includes an interval of space, the type of ratio is a Number Density

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-12 vote 8-1 by Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-12 comment: I wonder if we could define better "Number Density" as Density ratio {{2022-05-18 discussion found 2 instances of the term 'Number density' matching our definition, and the term 'density ratio' defines a density divided by a density which does not match this concept.}}

# concept[].concept[].concept[].concept[].property[].valueString
Wikipedia https://en.wikipedia.org/wiki/Number_density 
The number density (symbol: n or ?N) is an intensive quantity used to describe the degree of concentration of countable objects (particles, molecules, phonons, cells, galaxies, etc.) in physical space: three-dimensional volumetric number density, two-dimensional areal number density, or one-dimensional linear number density. Population density is an example of areal number density.

IUPAC Gold Book https://goldbook.iupac.org/terms/view/N04262
number density, n
Number of particles divided by the volume they occupy.

# concept[].concept[].concept[].concept[].property[].valueString
2022-05-27 vote 10-0 by Khalid Shahin, Joanne Dehnbostel, Eric Harvey, raradhikaag@gmail.com, Jesus Lopez-Alcalde, Mario Tristan, Harold Lehmann, Muhammad Afzal, nisha mathew, Janice Tufte

# concept[].concept[].concept[].definition
A ratio in which the numerator is a measure of the solute and the denominator is a measure of the solvent.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-05-08 vote 6-0 by Mario Tristan, Harold Lehmann, Paola Rosati, Muhammad Afzal, nisha mathew, Eric M Harvey

# concept[].concept[].definition
A statistic that quantitatively represents a relationship between two or more variables.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal, Neeraj Ojha

# concept[].concept[].property[].valueString
2022-02-24 comment: There are measures of association between more than two variables, for example, an estimator of interaction information. So, this should be "two or more variables" (or just "variables"). 

Also, I don't like the term "represents", I'd prefer to say "A statistic that quantifies a relationship between variables."

# concept[].concept[].property[].valueString
2022-03-16 vote 7-0 by Mario Tristan, Paola Rosati, Louis Leff, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte

# concept[].concept[].concept[].definition
A measure of association expressed as a ratio.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel, Janice Tufte, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2022-02-24 comment: I think we should replace "represents" with "quantifies" and remove the restriction to two variables.

"A statistic that is a ratio and quantifies a relationship between variables."

Second, I think you want a more restrictive definition than a statistic that is a ratio. For example, the uncertainty coefficient, I(X;Y)/H(Y), is a ratio and a measure of association, but I don't think you'd consider it a ratio-based measure of association (maybe you would, in which case this is OK).

You should also consider whether monotonic transformations of ratios count as ratio-based measures. It is common for people to take logarithms of ratios.

I'm not sure what the utility is of this category. When does someone need it? Could we just put all its children Measure of Association?

# concept[].concept[].concept[].property[].valueString
This categorical (parent) term can be used for a statistic that is a ratio, quantifies a relationship between two variables, and is not found in the child terms.

# concept[].concept[].concept[].property[].valueString
2022-12-28 vote 7-0 by Janice Tufte, Mario Tristan, Joanne Dehnbostel, Harold Lehman, Yuan Gao, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the hazard rate of an event in one group to the hazard rate of the same event in another group.

# concept[].concept[].concept[].concept[].property[].valueString
NCIt:
Hazard ratio = A measure of how often a particular event happens in one group compared to how often it happens in another group, over time. In cancer research, hazard ratios are often used in clinical trials to measure survival at any point in time in a group of patients who have been given a specific treatment compared to a control group given another treatment or a placebo. A hazard ratio of one means that there is no difference in survival between the two groups. A hazard ratio of greater than one or less than one means that survival was better in one of the groups.

https://www.statisticshowto.com/hazard-ratio/
The hazard ratio is a comparison between the probability of events in a treatment group, compared to the probability of events in a control group.

Hazard Ratio in Clinical Trials (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC478551/)
The hazard ratio is an estimate of the ratio of the hazard rate in the treated versus the control group. The hazard rate is the probability that if the event in question has not already occurred, it will occur in the next time interval, divided by the length of that interval. The time interval is made very short, so that in effect the hazard rate represents an instantaneous rate.

The Hazards of Hazard Ratios (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653612/)
The hazard ratio (HR) is the main, and often the only, effect measure reported in many epidemiologic studies. For dichotomous, non–time-varying exposures, the HR is defined as the hazard in the exposed groups divided by the hazard in the unexposed groups. For all practical purposes, hazards can be thought of as incidence rates and thus the HR can be roughly interpreted as the incidence rate ratio. The HR is commonly and conveniently estimated via a Cox proportional hazards model, which can include potential confounders as covariates.

# concept[].concept[].concept[].concept[].property[].valueString
Hazard rate (SEVCO TBD:0000025) is defined as: A conditional instantaneous rate in which the numerator represents an incidence conditioned on survival to a specified time, and the denominator represents a time interval with a duration approaching zero.

The groups being compared are often the exposed group versus the unexposed group, but hazard ratio can also be applied to comparisons of one exposure relative to another exposure.

A hazard ratio of one means there is no difference between two groups in terms of their hazard rates, based on whether or not they were exposed to a certain substance or factor, or how they responded to two interventions being compared. A hazard ratio of greater than one implies an association of greater risk, and a hazard ratio of less than one implies an association of lower risk.

The hazard ratio can be calculated from studies in which the proportion of exposed participants who had the event is known, the proportion of unexposed participants who had the event is known, and the timing of events for each participant is known or estimable, such as a cohort study or clinical trial.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 5-0 by Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of two incidence rates.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
Incidence Rate (SEVCO TBD:0000024) is defined as: A rate in which the number of new events per total at risk is divided by an interval of time.

The incidence rates may refer to the same event comparing two different groups, or the same group comparing two different events.

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 5-0 by Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].definition
An incidence rate ratio in which the numerator is the incidence rate in a group and the denominator is the incidence rate for a reference population.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
The incidence rate used for the denominator may be an expected incidence rate for a reference population. The reference population may refer to a general population of the geographic area from which the cohort was selected.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 5-0 by Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 comment: ... and the denominator is the incidence rate or expected incidence rate for a reference population.

Comment for application: The reference population may refer to a general population of the geographic area from which the cohort was selected.

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of two odds.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
STATO: odds ratio (OR) = Odds ratio is a ratio that measures effect size, that is the strength of association between 2 dichotomous variables, one describing an exposure and one describing an outcome.
It represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure ( the probability of the event occuring divided by the probability of an event not occurring). The odds ratio is a ratio of describing the strength of association or non-independence between two binary data values by forming the ratio of the odds for the first group and the odds for the second group. Odds ratio are used when one wants to compare the odds of something occurring to two different groups.

UMLS: The ratio of two odds. The exposure-odds ratio for case control data is the ratio of the odds in favor of exposure among cases to the odds in favor of exposure among noncases. The disease-odds ratio for a cohort or cross section is the ratio of the odds in favor of disease among the exposed to the odds in favor of disease among the unexposed. The prevalence-odds ratio refers to an odds ratio derived cross-sectionally from studies of prevalent cases. (MSH)
A measure of the odds of an event happening in one group compared to the odds of the same event happening in another group. In cancer research, odds ratios are most often used in case-control (backward looking) studies to find out if being exposed to a certain substance or other factor increases the risk of cancer. For example, researchers may study a group of individuals with cancer (cases) and another group without cancer (controls) to see how many people in each group were exposed to a certain substance or factor. They calculate the odds of exposure in both groups and then compare the odds. An odds ratio of one means that both groups had the same odds of exposure and, therefore, the exposure probably does not increase the risk of cancer. An odds ratio of greater than one means that the exposure may increase the risk of cancer, and an odds ratio of less than one means that the exposure may reduce the risk of cancer. (NCI)
The ratio of the odds of an event occurring in one group to the odds of it occurring in another group, or to a sample-based estimate of that ratio. (NCI)

NICE: Compares the odds (probability) of something happening in 1 group with the odds of it happening in another. An odds ratio of 1 shows that the odds of the event happening (for example, a person developing a disease or a treatment working) is the same for both groups. An odds ratio of greater than 1 means that the event is more likely in the first group than the second. An odds ratio of less than 1 means that the event is less likely in the first group than in the second group.

# concept[].concept[].concept[].concept[].property[].valueString
2022-03-16 vote 8-0 by Mario Tristan, Paola Rosati, Louis Leff, nisha mathew, Philippe Rocca-Serra, Harold Lehmann, Janice Tufte, Eric Moyer

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of two prevalences.

# concept[].concept[].concept[].concept[].property[].valueString
Prevalence (SEVCO STATO:0000412) is defined as: A proportion in which the numerator represents all events of interest (for example, both new and preexisting cases of a disease) in the population, which is represented by the denominator.

The Prevalence Ratio indicates the magnitude of the prevalence of an event/outcome in one group of subjects/individuals (with characteristics/attribute) relative to another group (with different characteristics/attributes), such as the prevalence of the disease among the exposed persons to the prevalence of the disease among the unexposed persons.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal, Janice Tufte

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 5-0 by Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 comment: Comment for application: The prevalence Ratio indicates how large is the prevalence of an event/outcome in one group of subjects/individuals (with characteristics/attribute) relative to another group (without the characteristics/attributes), such as the prevalence of the disease among the exposed persons to the prevalence of the disease among the unexposed persons.

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the risk of an event in one group to the risk of the same event in another group.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: relative risk (risk ratio) = Relative risk is a measurement datum which denotes the risk of an 'event' relative to an 'exposure'. Relative risk is calculated by forming the ratio of the probability of the event occurring in the exposed group versus the probability of this event occurring in the non-exposed group.

NCIt	Relative Risk	A measure of the risk of a certain event happening in one group compared to the risk of the same event happening in another group. In cancer research, risk ratios are used in prospective (forward looking) studies, such as cohort studies and clinical trials. A risk ratio of one means there is no difference between two groups in terms of their risk of cancer, based on whether or not they were exposed to a certain substance or factor, or how they responded to two treatments being compared. A risk ratio of greater than one or of less than one usually means that being exposed to a certain substance or factor either increases (risk ratio greater than one) or decreases (risk ratio less than one) the risk of cancer, or that the treatments being compared do not have the same effects

OBCS	relative risk  A data item that equals the incidence in exposed individuals divided by the incidence in unexposed individuals. The relative risk can be calculated from studies in which the proportion of patients exposed and unexposed to a risk is known, such as a cohort study.

CDC https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section5.html:
A risk ratio (RR), also called relative risk, compares the risk of a health event (disease, injury, risk factor, or death) among one group with the risk among another group. It does so by dividing the risk (incidence proportion, attack rate) in group 1 by the risk (incidence proportion, attack rate) in group 2. The two groups are typically differentiated by such demographic factors as sex (e.g., males versus females) or by exposure to a suspected risk factor (e.g., did or did not eat potato salad). Often, the group of primary interest is labeled the exposed group, and the comparison group is labeled the unexposed group.

# concept[].concept[].concept[].concept[].property[].valueString
The groups being compared are often the exposed group versus the unexposed group, but risk ratio can also be applied to comparisons of one exposure relative to another exposure.

A risk ratio of one means there is no difference between two groups in terms of their risk, based on whether or not they were exposed to a certain substance or factor, or how they responded to two interventions being compared. A risk ratio of greater than one implies an association of greater risk, and a risk ratio of less than one implies an association of lower risk.

The risk ratio can be calculated from studies in which the proportion of exposed participants who had the event is known and the proportion of unexposed participants who had the event is known, such as a cohort study or clinical trial.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-06-08 vote 6-0 by Robin Ann Yurk, Harold Lehmann, Mario Tristan, Eric M Harvey, Muhammad Afzal, Paola Rosati

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the probability of the test giving a positive result when testing an affected subject and the probability of the test giving a positive result when a subject is not affected.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: positive likelihood ratio (likelihood ratio for positive results) = the likelihood ratio of positive results is a ratio which is form by dividing the sensitivity value of a test by the difference between 1 and specificity of the test. This can be expressed also as dividing the probability of the test giving a positive result when testing an affected subject versus the probability of the test giving a positive result when a subject is not affected.

AHRQ https://effectivehealthcare.ahrq.gov/products/test-performance-metrics/appendixes:
The positive and negative likelihood ratios (LR+ and LR-, respectively) quantify the change in the certainty of the “diagnosis” conferred by test results. More specifically, the likelihood ratios transform the pretest odds to the posttest odds of a given (positive or negative) diagnosis:

posttest odds = pretest odds x LR

For a positive result with the medical test, the positive likelihood ratio would be used in the above relationship; for a negative result with the medical test portable monitor, the negative likelihood ratio would be used.

If a given medical test has very good ability to predict the “true disease status,” its positive likelihood ratio will be high (i.e., will greatly increase the odds of a positive diagnosis) and its negative likelihood ratio will be low (i.e., will diminish substantially the likelihood of the positive diagnosis). A completely non-informative portable monitor would have likelihood ratios equal to 1 (i.e., does not transform the pre-test odds substantially in the equation above). Typically, a positive likelihood ratio of 10 or more and a negative likelihood ratio of 0.1 or less are considered to represent informative tests.3 We note that other, more lenient boundaries for LR+ and LR- can be used3 and that the choice of the boundaries is a subjective decision. It is interesting to note that studies with high LR+ and low LR- can be readily identified in the square sensitivity/100 percent-specificity plot, as shown in the Appendix Figure above.

# concept[].concept[].concept[].concept[].property[].valueString
The probability of the test giving a positive result when testing an affected subject is also called the sensitivity [SEVCO term STATO:0000233] or true positive rate. The probability of the test giving a positive result when a subject is not affected is called the false positive rate and is calculated as 1 minus the specificity [SEVCO term STATO:0000134]. The Likelihood Ratio Positive (LR+) is calculated as Sensitivity / (1 - Specificity).

The Likelihood Ratio Positive may also be calculated as the posterior probability (positive predictive value) divided by the prior probability (prevalence).

When the test result is a specific value on a continuous scale, the Likelihood Ratio Positive is the ratio of the likelihood of the test giving the specific value when testing an affected subject and the likelihood of the test giving the specific value when a subject is not affected. 

In the context of a probability distribution function, e.g. normal distribution, the x axis is the value and y axis is the likelihood.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-10 vote 5-0 by Robin Ann Yurk, Mario Tristan, Harold Lehmann, Eric Harvey, Paola Rosati

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the probability of the test giving a negative result when testing an affected subject and the probability of the test giving a negative result when a subject is not affected.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: negative likelihood ratio (likelihood ratio for negative results) = the likelihood ratio of negative results is a ratio which is formed  by dividing the difference between 1 and sensitivity of the test by the specificity value of a test.. This can be expressed also as dividing the probability of a person who has the disease testing negative by the probability of a person who does not have the disease testing negative.

AHRQ https://effectivehealthcare.ahrq.gov/products/test-performance-metrics/appendixes:
The positive and negative likelihood ratios (LR+ and LR-, respectively) quantify the change in the certainty of the “diagnosis” conferred by test results. More specifically, the likelihood ratios transform the pretest odds to the posttest odds of a given (positive or negative) diagnosis:

posttest odds = pretest odds x LR

For a positive result with the medical test, the positive likelihood ratio would be used in the above relationship; for a negative result with the medical test portable monitor, the negative likelihood ratio would be used.

If a given medical test has very good ability to predict the “true disease status,” its positive likelihood ratio will be high (i.e., will greatly increase the odds of a positive diagnosis) and its negative likelihood ratio will be low (i.e., will diminish substantially the likelihood of the positive diagnosis). A completely non-informative portable monitor would have likelihood ratios equal to 1 (i.e., does not transform the pre-test odds substantially in the equation above). Typically, a positive likelihood ratio of 10 or more and a negative likelihood ratio of 0.1 or less are considered to represent informative tests.3 We note that other, more lenient boundaries for LR+ and LR- can be used3 and that the choice of the boundaries is a subjective decision. It is interesting to note that studies with high LR+ and low LR- can be readily identified in the square sensitivity/100 percent-specificity plot, as shown in the Appendix Figure above.

# concept[].concept[].concept[].concept[].property[].valueString
The probability of the test giving a negative result when testing an affected subject is also called the false negative rate and is calculated as 1 minus the sensitivity [SEVCO term STATO:0000233]. The probability of the test giving a negative result when a subject is not affected is called the specificity [SEVCO term STATO:0000134] or true negative rate. The Likelihood Ratio Negative (LR-) is calculated as (1 - Sensitivity ) / Specificity.

The Likelihood Ratio Negative may also be calculated as the posterior probability (1 - negative predictive value) divided by the prior probability (prevalence).

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-10 vote 5-0 by Robin Ann Yurk, Mario Tristan, Harold Lehmann, Eric Harvey, Paola Rosati

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the number of correct results to the total number tested.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: "accuracy (Rand accuracy, Rand index) = in the context of binary classification, accuracy is defined as the proportion of true results (both true positives and true negatives) to the total number of cases examined (the sum of true positive, true negative, false positive and false negative).

It can be understood as a measure of the proximity of measurement results to the true value."

# concept[].concept[].concept[].concept[].property[].valueString
Where results are reported as positive or negative, correct results are reported as true, and incorrect results are reported as false, the diagnostic accuracy is calculated as ( True Positives + True Negatives ) / ( True Positives + True Negatives + False Positives + False Negatives ).
For continuous values, Measurement Accuracy (SEVCO term: TBD:MeasAccu) would be used instead of Diagnostic Accuracy.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-24 vote 6-0 by Jesus Lopez-Alcalde, Harold Lehmann, Mario Tristan, Cauê Monaco, Eric Harvey, Janice Tufte

# concept[].concept[].concept[].concept[].definition
A measure of association that is the ratio of the odds of a positive test in those with disease relative to the odds of a positive test in those without disease.

# concept[].concept[].concept[].concept[].property[].valueString
AHRQ https://effectivehealthcare.ahrq.gov/products/test-performance-metrics/appendixes: 
The diagnostic odds ratio (DOR) describes the odds of a positive test in those with disease relative to the odds of a positive test in those without disease.4 It can be computed in terms of sensitivity and specificity as well as in terms of positive and negative likelihood ratios (DOR = LR+/LR-). Thus this single measure includes information about both sensitivity and specificity and tends to be reasonably constant despite diagnostic threshold. However, it is impossible to use diagnostic odds ratios to weigh sensitivity and specificity separately, and to distinguish between tests with high sensitivity and low specificity and tests with low sensitivity and high specificity.

Another disadvantage is that it is difficult for clinicians to understand and apply, limiting its clinical value. This is partly because they are not often exposed to diagnostic odds ratios. A diagnostic odds ratio is similar to an odds ratio that measures strength of association in an observational study or effect size in a trial. However, contrary to the typical effect size magnitudes of such odds ratios (often between 0.5 and 2), diagnostic odds ratios can attain much larger values (often greater than 100).

# concept[].concept[].concept[].concept[].property[].valueString
The Diagnostic Odds Ratio may be calculated as the Likelihood Ratio Positive divided by the Likelihood Ratio Negative. The Diagnostic Odds Ratio is an overall measure of the discriminatory power of a test and does not distinguish between the power to detect (rule in) or exclude (rule out).

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2022-08-31 vote 6-0 by Janice Tufte, nisha mathew, Muhammad Afza, Harold Lehmann, Philippe Rocca-Serra, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A measure of association, ranging from -1 to 1, that measures the strength and direction of the linear relationship between two binary variables.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Matthews correlation coefficient (MCC) = Matthews Correlation Coefficient (or MCC) is a correlation coefficient which is a measure of the quality of binary (two-class) classifications, introduced by biochemist Brian W. Matthews in 1975.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
For a 2×2 contingency table

| Variable 1 Value 1 | Variable 1 Value 2
Variable 2 Value 1 | A | B
Variable 2 Value 2 | C | D

where A, B, C, and D represent the observation frequencies (the cell count), the formula for the phi coefficient ($\Phi$) is: $$\Phi = \frac{AD - BC}{\sqrt{(A+B)(C+D)(A+C)(B+D)}}$$

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-25 vote 6-0 by Mario Tristan, Jesus Lopez-Alcalde, Joanne Dehnbostel,  Harold Lehmann, Yuan Gao, Eric Harvey

# concept[].concept[].concept[].definition
A measure of association of two variables representing measurements of the same attribute of an entity.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
The term 'Measure of Agreement' is primarily used as a class for types of measure of agreement listed in the hierarchy but may be used as the code for a measure of agreement that is not listed.

# concept[].concept[].concept[].property[].valueString
2022-12-21 vote 5-0 by Mario Tristan, Philippe Rocca-Serra, Eric Harvey, Janice Tuft, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A measure of agreement among categorical assessments, corrected for chance agreement.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
OBCS kappa statistic = a generic term for several similar measures of agreement used with categorical data; typically used in assessing the degree to which two or more raters, examining the same data, agree on assigning data to categories

# concept[].concept[].concept[].concept[].property[].valueString
In the literature, the same eponymic term (e.g., 'Cohen's kappa') is used with different formulas. In SEVCO, we define each term with a single formula, and recommend annotators to choose the SEVCO term based on the formula.

This is a widely used term to measure inter-rater reliability. Refer to measures of association to see other terms: for example, intra-class correlation coefficient (ICC).

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-14 (After deleting one "yes" vote at the request of the voter) vote 6-0 by Nisha Mathew, Philippe Rocca-Serra, Harold Lehmann, Eric Harvey, Jesus Lopez-Alcalde, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2022-09-14 Comment  "I recommend adding ....is a measure of interrater reliability or is this an Interrater reliability testing an alternate term."

# concept[].concept[].concept[].concept[].concept[].definition
A Kappa statistic in which the expected agreement by chance is based on an assumption that all possible categories for assignment are equally likely.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A Kappa statistic is a measure of agreement among categorical assessments, corrected for chance agreement.

In the simple chance-corrected agreement coefficient, the expected chance agreement is modeled as the inverse of the number of categories (1/q) where q is the number of possible categories for assignment.

The simple chance-corrected agreement coefficient is calculated as ( p[a] - 1/q ) / ( 1 - 1/q ) where p[a] is the observed percent agreement and q is the number of possible categories for assignment.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5965565
Gwet KL. Testing the Difference of Correlated Agreement Coefficients for Statistical Significance. Educ Psychol Meas. 2016 Aug;76(4):609-637. doi: 10.1177/0013164415596420. Epub 2015 Jul 28. PMID: 29795880; PMCID: PMC5965565.

Brennan and Prediger (1981) proposed a simple chance-corrected agreement coefficient, which generalizes to multiple raters and multiple categories, the G-index previously proposed by Holley and Guilford (1964) for two raters and two categories. What is known as the Holley–Guilford G-index was previously proposed independently by various authors under different names. Among them are Guttman (1945), Bennett, Alpert, and Goldstein (1954), and Maxwell (1977). For an interrater reliability experiment involving r raters who classify n subjects into one of q possible categories, the Brennan-Prediger coefficient is given by

?[BP] = ( p[a] - 1/q ) / ( 1 - 1/q ),
where the percent agreement p[a] is defined by Equation (3 -- see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5965565/#disp-formula3-0013164415596420), and the percent chance agreement is a constant representing the inverse of the number of categories.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-09-14 vote 6-0 by Nisha Mathew, Philippe Rocca-Serra, Harold Lehmann, Eric Harvey, Jesus Lopez-Alcalde, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].definition
A Kappa statistic in which the expected agreement by chance is based on an assumption that the likelihood of each category for assignment is based on the proportion observed, and the number of raters is 2.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
OBCS Cohen's kappa measurement (inter-rater agreement, inter-annotator agreement; inter-rater agreement, inter-annotator agreement) = a statistical measure of agreement for categorical data; a measure of inter-rater agreement or inter-annotator agreement

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A Kappa statistic is a measure of agreement among categorical assessments, corrected for chance agreement.

In Cohen's kappa, the expected chance agreement is modeled as the summation of the differences, between the square of the expected probability of the category and the quotient of its variance divided by 2 (the number of raters), for each category.

Cohen's kappa is calculated as ( p[a] - p[e] ) / ( 1 - p[e] ) where p[a] is the observed percent agreement and p[e] is the expected chance agreement.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-09-14 vote 6-0 by Nisha Mathew, Philippe Rocca-Serra, Harold Lehmann, Eric Harvey, Jesus Lopez-Alcalde, Khalid Shahin

# concept[].concept[].concept[].concept[].concept[].display
modified Cohen’s kappa for more than 2 raters

# concept[].concept[].concept[].concept[].concept[].definition
A Kappa statistic in which the expected agreement by chance is based on an assumption that the likelihood of each category for assignment is based on the proportion observed, and the number of raters is more than 2.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
OBCS Cohen's kappa measurement (inter-rater agreement, inter-annotator agreement; inter-rater agreement, inter-annotator agreement) = a statistical measure of agreement for categorical data; a measure of inter-rater agreement or inter-annotator agreement

# concept[].concept[].concept[].concept[].concept[].property[].valueString
A Kappa statistic is a measure of agreement among categorical assessments, corrected for chance agreement.

In the modified Cohen's kappa for more than 2 raters, the expected chance agreement is modeled as the summation of the differences, between the square of the expected probability of the category and the quotient of its variance divided by the number of raters, for each category.

The modified Cohen's kappa for more than 2 raters is calculated as ( p[a] - p[e] ) / ( 1 - p[e] ) where p[a] is the observed percent agreement and p[e] is the expected chance agreement.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-09-14 vote 5-0 by Nisha Mathew, Philippe Rocca-Serra, Harold Lehmann, Eric Harvey, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].concept[].definition
A Kappa statistic where the expected agreement between two raters is expressed in terms of the square of arithmetic means of marginal proportions of each assessment category.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Scott's pi is a kappa statistic for two raters that assumes the likelihood of each category for assignment is based on the same distribution of rater responses, leading to the use of squared arithmetic means of the marginal proportion of each assessment category as its estimate of "chance agreement."

Pr(expected) is calculated using squared "joint proportions" which are squared arithmetic means of the marginal proportions of each assessment category, in contrast to Cohen's Kappa which uses squared geometric means.

Scott's pi = ( p[a] - p[e] ) / ( 1 - p[e] ) where p[a] is the observed percent agreement and p[e] is the expected chance agreement expressed as the squared joint proportions of the marginal sums.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-09-21 comment: I think there should be a formula included in the comment for application as with all the other Kappa terms
2022-09-28 adjustment: Steering Group changed the first sentence of Comment for application to better represent the assumption.

# concept[].concept[].concept[].concept[].concept[].property[].valueString
2022-10-19 vote 6-0 by Joanne Dehnbostel, Muhammad Afzal, Jesus Lopez-Alcalde, Mario Tristan, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A ratio of the number of incorrect results to the total number tested.

# concept[].concept[].concept[].concept[].property[].valueString
Where results are reported as positive or negative, incorrect results are reported as false, and correct results are reported as true, the misclassification rate is calculated as ( False Positives + False Negatives ) / ( True Positives + True Negatives + False Positives + False Negatives ).

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2022-10-19 vote 5-0 by Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A ratio representing the harmonic mean of recall and precision.

# concept[].concept[].concept[].concept[].property[].valueString
OCRe F measure [not used due to inaccuracy in the definition]

# concept[].concept[].concept[].concept[].property[].valueString
The F1-score is used as a measure of quality for classification algorithms and information retrieval strategies, where 1 represents the best precision and recall and 0 represents the worst precision and recall.

A harmonic mean of a set of quantities is the reciprocal of the arithmetic mean of the reciprocals of each quantity.  The F score is thus calculated as 1 / (the arithmetic mean of the reciprocals), or:

F = 1 / ( ( (1/recall) + (1/precision) ) / 2 )

F = 2*( (precision*recall) / (precision+recall) )

Recall is sensitivity STATO:0000233
Precision (PPV) is SEVCO TBD:0000022

[[F-beta will be defined elsewhere in the code system.]]

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
2022-10-19 vote 6-0 by Joanne Dehnbostel, Muhammad Afzal, Jesus Lopez-Alcalde, Mario Tristan, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].definition
A measure of association between ordinal or continuous variables.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Yuan Gao, Kenneth Wilkins, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2022-10-26 vote 6-1 by Yuan Gao, Philippe Rocca-Serra, Eric Harvey, Paola Rosati, Harold Lehmann, Joanne Dehnbostel, Janice Tufte

# concept[].concept[].concept[].property[].valueString
2022-10-26 comments: there are measures of correlation which characterise non-linear relation between 2 variables . so I was wondering if there was a need to specify "measure of linear correlation" , where a subclass would be 'correlation coefficient). The type 'measure of correlation' becoming a parent class for the 'measure of non-linear correlation'
Should we say, "A value of 0 means no linear association, a value of +1 mean perfect positive linear (a positive slope) association, and a value of -1 means perfect negative association (a negative slope)."

# concept[].concept[].concept[].property[].valueString
A value of 0 means no association. A positive value means a positive association (as one variable increases, the other variable increases). A negative value means a negative association (as one variable increases, the other variable decreases). 
For correlation coefficients, the possible values range from +1 (perfect positive association) to -1 (perfect negative association).

# concept[].concept[].concept[].property[].valueString
2022-11-16 vote 5-0 by Brian S. Alper, Philippe Rocca-Serra, Harold Lehman, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A measure of correlation that is not normalized by the variances of the variables.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: "covariance = The covariance is a measurement data item about the strength of correlation between a set (2 or more) of random variables.
The covariance is obtained by forming:
cov(X,Y)=E([X-E(X)][Y-E(Y)] where E(X), E(Y) is the expected value (mean) of variable X and Y respectively.

covariance is symmetric so cov(X,Y)=cov(Y,X).

The covariance is usefull when looking at the variance of the sum of the 2 random variables since:
var(X+Y) = var(X) +var(Y) +2cov(X,Y)

The covariance cov(x,y) is used to obtain the coefficient of correlation cor(x,y) by normalizing (dividing) cov(x,y) but the product of the standard deviations of x and y."

# concept[].concept[].concept[].concept[].property[].valueString
A measure of correlation is a measure of association between ordinal or continuous variables.

Covariance is used in the calculation of other measures of correlation. Covariance can only be calculated for interval or continuous variables.

Because the covariance is not normalized by the variances of the variables, the magnitude of the covariance is not informative without consideration of the magnitude of the respective variances. Covariance is informative regarding whether both variables vary in the same direction (positive covariance) or in the opposite direction (negative covariance).

Covariance for a sample is calculated as the mean of the products of deviations from the sample mean for the variables.
Cov(X,Y) = S (($x_i – \overline{x}$) ($y_i – \overline{y}$)) / (n-1) where $x_i$ is one the observed values of X, $\overline{x}$ is the sample mean of X,  $y_i$ is one the observed values of Y, and $\overline{y}$ is the sample mean of Y.

Covariance as the population-level quantity is given by the expected value of the product of deviations from the mean for the variables.
Cov(X, Y) = E [ (X - µ) (Y - ?) ] where µ = E(X) and ? = E(Y)

Covariance is a continuous value with a range of negative infinity to positive infinity.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Kenneth Wilkins, Yuan Gao, Khalid Shahin, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2022-11-23 vote 5-0 by Mario Tristan, Yuan Gao, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A measure of correlation, ranging from -1 to 1, that measures the strength and direction of the linear relationship between values of two continuous variables.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: "Pearson's correlation coefficient (Pearson product-moment correlation coefficient; Pearson's r; r statistics) = The Pearson's correlation coefficient is a correlation coefficient which evaluates two continuous variables for association strength in a data sample. It assumes that both variables are normally distributed and linearity exists. 
The coefficient is calculated by dividing their covariance with the product of their individual standard deviations. It is a normalized measurement of how the two are linearly related."

# concept[].concept[].concept[].concept[].property[].valueString
A measure of correlation is a measure of association between ordinal or continuous variables. Pearson correlation coefficient is designed to be used between continuous variables.
Pearson correlation coefficient for a sample ($r$) is calculated as $r = \dfrac{\widehat{cov}(x,y)}{s_x*s_y}$ where $ \widehat{cov}(x,y)$ is the estimated covariance, and $s_x$ and $s_y$ are the sample standard deviations.
Pearson correlation coefficient for a population ($\rho$) is defined as $\rho= \dfrac{cov(X,Y)}{\sigma_X*\sigma_Y}$ where cov(X,Y) is covariance of X and Y and $\sigma_X$ and $\sigma_Y$ are the population standard deviations.

Assumptions for computing Pearson's correlation coefficient include a linear relationship between 2 continuous variables and each of the variables approximates a normal distribution.

Covariance is [defined in SEVCO](https://fevir.net/resources/CodeSystem/27270#STATO:0000301).

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Muhammad Afzal, Yuan Gao, Khalid Shahin, Joanne Dehnbostel, Brian S. Alper, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2022-12-07 vote 5-0 by Muhammad Afzal, Mario Tristan, Eric Harvey, Yuan Gao, Mahnoor Ahmed

# concept[].concept[].concept[].concept[].definition
A measure of correlation, ranging from -1 to 1, that measures the strength and direction of the relationship between ranks by value of two ordinal or continuous variables, and is calculated as the Pearson correlation coefficient between the rank values.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: "Spearman's rank correlation coefficient (Spearman's rho) = Spearman's rank correlation coefficient is a correlation coefficient which is a nonparametric measure of statistical dependence between two ranked variables. It assesses how well the relationship between two variables can be described using a monotonic function. If there are no repeated data values, a perfect Spearman correlation of +1 or -1 occurs when each of the variables is a perfect monotone function of the other.
Spearman's coefficient may be used when the conditions for computing Pearson's correlation are not met (e.g linearity, normality of the 2 continuous variables) but may require a ranking transformation of the variables"

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Muhammad Afzal, Yuan Gao, Joanne Dehnbostel, Brian S. Alper, Harold Lehmann, Noor Ahmed

# concept[].concept[].concept[].concept[].property[].valueString
A measure of correlation is a measure of association between ordinal or continuous variables. Spearman rank-order correlation coefficient is designed to be used between ordinal and/or continuous variables.

The Spearman rank-order correlation coefficient can identify monotonic (i.e. consistently non-increasing or consistently non-decreasing) relationships, whether the relationships are linear or non-linear.

The Spearman rank-order correlation coefficient between two variables is equal to the [Pearson correlation coefficient](https://fevir.net/resources/CodeSystem/27270#STATO:0000280) between the rank values of those two variables.

The Spearman rank-order correlation coefficient is the nonparametric counterpart to the Pearson correlation coefficient and may be used when the assumptions for computing Pearson's correlation coefficient (include a linear relationship between 2 continuous variables and each of the variables approximates a normal distribution) are not met. The Spearman rank-order correlation coefficient is appropriate when either variable has outliers, is ordinal, or is not normally distributed; when the variances of the two variables are unequal; or when the apparent relationship between the variables is non-linear. The assumptions for computing Spearman rank-order correlation coefficient include a monotonic relationship between 2 continuous or ordinal variables.

# concept[].concept[].concept[].concept[].property[].valueString
2022-12-07 comment: The fundamental difference between the two correlation coefficients is that the Pearson coefficient works with a linear relationship between the two variables whereas the Spearman Coefficient works with monotonic relationships as well.

# concept[].concept[].concept[].concept[].property[].valueString
2022-12-14 vote 5-0 by Jesus Lopez-Alcalde, Yuan Gao, Mario Tristan, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A measure of correlation, ranging from -1 to 1, that measures the strength and direction of the relationship between ranks by value of two ordinal or continuous variables, and is calculated based on the difference in the number of concordant and discordant pairs of rankings divided by the number of all possible pairs of rankings.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Kendall's correlation coefficient (Kendall's tau (t) coefficient; Kendall rank correlation coefficient) = Kendall's correlation coefficient is a correlation coefficient between 2 ordinal variables (natively or following a ranking procedure) and may be used when the conditions for computing Pearson's correlation are not met (e.g linearity, normality of the 2 continuous variables)

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
A measure of correlation is a measure of association between ordinal or continuous variables. Kendall's correlation coefficient is designed to be used between ordinal variables (or continuous variables converted to ordinal variables).

The Kendall's correlation coefficient can identify monotonic (i.e. consistently non-increasing or consistently non-decreasing) relationships, whether the relationships are linear or non-linear.

The Kendall's correlation coefficient between two variables is calculated by determining the concordance or discordance of each pair of ranked values (whether or not two raters are concordant in one value being ranked equal or higher to the other value), and then dividing the difference between the number of concordant values ($n_c$) and the number of discordant values ($n_d$) by the number of pairs of ranked values ($\frac{1}{2}n(n-1)$).

$$\tau =  \dfrac{n_c - n_d}{\frac{1}{2}n(n-1)}$$


The Kendall's correlation coefficient is a nonparametric statistic and may be used when the assumptions for computing Pearson's correlation coefficient (include a linear relationship between 2 continuous variables and each of the variables approximates a normal distribution) are not met. The Kendall's correlation coefficient is appropriate when either variable has outliers, is ordinal, or is not normally distributed; when the variances of the two variables are unequal; or when the apparent relationship between the variables is non-linear. The assumptions for computing Kendall's correlation coefficient include a monotonic relationship between 2 ordinal variables.

# concept[].concept[].concept[].concept[].property[].valueString
2022-12-21 vote 5-0 by Joanne Dehnbostel, Mario Trista, Philippe Rocca-Serra, Eric Harvey, Harold Lehmann
2023-01-04 definition change by Steering Committee

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-25 vote 6-0 by Mario Tristan, Jesus Lopez-Alcalde, Joanne Dehnbostel,  Harold Lehmann, Yuan Gao, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A measure of correlation, ranging from -1 to 1, that measures the strength and direction of the relationship between ranks by value of two ordinal or continuous variables, and is calculated based on the difference in the number of concordant and discordant pairs of rankings divided by the total number of pairs of rankings, where ties are not counted among the pairs of rankings.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2023-01-25 vote 5-0 by Mario Tristan, Jesus Lopez-Alcalde, Joanne Dehnbostel,  Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].definition
A measure of association that is used as the coefficient of an independent variable in a regression model, of the dependent variable, which is linear in its parameters.

# concept[].concept[].concept[].property[].valueString
STATO: regression coefficient = a regression coefficient is a data item generated by a type of data transformation called a regression, which aims to model a response variable by expression the predictor variables as part of a function where variable terms are modified by a number. A regression coefficient is one such number.

# concept[].concept[].concept[].property[].valueString
A value of zero means no association. The sign (positive or negative) reflects the direction of association.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-02-07 vote 5-0 by Cauê Monaco, Harold Lehmann, Mario Tristan, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A measure of association between a variable representing known or true values and a variable representing measured or predicted values.

# concept[].concept[].concept[].property[].valueString
Calibration is often used for measurement devices. The known or true values may be called the reference standard.
Calibration is also used for predictive models and other contexts comparing computed or expected probabilities with empirical frequencies.
A measure of calibration indicates the degree to which the computed values are underestimated or overestimated.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Harold Lehmann, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2023-12-18 vote 6-0 by Janice Tufte, Eric Harvey, Caue Monaco, Philippe Rocca-Serra, Harold Lehmann, Jesus Lopez-Alcalde

# concept[].concept[].concept[].concept[].definition
A measure of calibration that is the average of a function of the difference between the expected values and the observed values.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Muhammad Afzal, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
For predictive modeling of non-continuous variables, the mean calibration is a measure of calibration that is the average of a function of the difference between the expected probabilities and the observed frequencies.
The expected values may be computed (as in predictive models) or may be derived from reference data (as typical for a measurement device).
When the function is the square of the difference and the variables are binary (0 or 1), the measure is called the Brier score.

# concept[].concept[].concept[].concept[].property[].valueString
2024-01-22 vote 5-0 by Harold Lehmann, Homa Keshavarz, Eric Harvey, Janice Tufte, Cauê Monaco

# concept[].concept[].concept[].concept[].definition
A measure of calibration that is the difference between the mean expected value and the mean observed value.

# concept[].concept[].concept[].concept[].property[].valueString
For calibration of binary outcome variables (0 or 1), the calibration intercept is computed from a statistical model where the log odds of the predicted probabilities is a linear function of the empirical frequencies.
For calibration of count outcome variables (0, 1, 2, 3, ...), the calibration intercept may be computed from a statistical model where the log of the predicted mean counts is a linear function of the empirical frequencies determined by unique combinations of the covariates.
There are other types of outcome variables for which the calibration-in-the-large measure may be obtained.
The notion of calibration in the large is that the intercept is a gross assessment of whether the average prediction matches the average outcome, however, this interpretation is exquisitely sensitive to the choice of referent factors within the prediction model.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Ken Wilkins, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-05 vote 4-0 by Xing Song, Eric Harvey, Harold Lehmann, Homa Keshavarz

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-05 comment: The definition looks good to me. Only comment is that the first comment about "calibration of binary variable" made it is sounds like this concept is only for binary outcomes, while I think calibration is a generic measure for all generalized linear models.

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 vote 5-0 by Lenny Vasanthan, Xing Song, Eric Harvey, Harold Lehmann, Homa Keshavarz

# concept[].concept[].concept[].concept[].definition
A measure of calibration that is the rate of change in the appropriately transformed value per unit change of the correspondingly transformed predicted value.

# concept[].concept[].concept[].concept[].property[].valueString
For calibration of binary outcome variables (0 or 1), the calibration slope is computed from a statistical model where the log odds of the predicted probabilities is a linear function of the empirical frequencies (logistical regression). The transformation is log odds (logit, the link function for a generalized linear model for the expected value of the outcome).
 
For calibration of count outcome variables (0, 1, 2, 3, ...), the calibration slope may be computed from a statistical model where the log of the predicted mean counts is a linear function of the empirical frequencies determined by unique combinations of the covariates. The transformation is log, the link function, of the counts.
 
There are other types of outcome variables for which the calibration slope may be obtained.
 
Slopes further away from 1.0 indicate, at upper and lower values, over- or under-confidence in the prediction.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Ken Wilkins, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-03-11 vote 6-0 by Eric Harvey, Xing Song, Lenny Vasanthan, Elma OMERAGIC, Homa Keshavarz, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 comment: Again, the first comment about "calibration of binary variable" made it is sounds like this concept is only for binary outcomes. Logistic regression could be used as an example for this concept.

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 vote 5-0 by Lenny Vasanthan, Xing Song, Eric Harvey, Harold Lehmann, Homa Keshavarz BUT THEN term definition changed in response to the comment

# concept[].concept[].definition
A statistic that represents the variation or spread among data values in a dataset or data distribution.

# concept[].concept[].property[].valueString
STATO: measure of variation (measure of dispersion) = measure of variation or statistical dispersion is a data item which describes how much a theoritical distribution or dataset is spread.

NCIt: "Statistical dispersion-	
The variation between data values in a sample."

UMLS: "Dispersion (C0332624)
Definition: The variation between data values in a sample.
Semantic Types: Spatial Concept"

# concept[].concept[].property[].valueString
This categorical (parent) term can be used for a statistic that is a measure of dispersion and is not found in the child terms.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Yuan Gao, Joanne Dehnbostel

# concept[].concept[].property[].valueString
2023-02-07 vote 5-0 by Cauê Monaco, Harold Lehmann, Janice Tufte, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A measure of dispersion calculated as the difference between the maximum observed value and the minimum observed value.

# concept[].concept[].concept[].property[].valueString
STATO: range = the range is a measure of variation which describes the difference between the lowest score and the highest score in a set of numbers (a data set)

# concept[].concept[].concept[].property[].valueString
A measure of dispersion is a statistic that represents the variation or spread among data values in a dataset or data distribution.
The maximum observed value is a statistic that represents the largest non-null value in a collection of values that can be ordered by magnitude.
The minimum observed value is a statistic that represents the smallest non-null value in a collection of values that can be ordered by magnitude.
A range (as a statistic) is represented as a single value (the difference between maximum and minimum observed values) while, in common language, the term range is often expressed with two values (from the minimum to maximum values, or from the lower limit to the higher limit).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-02-20 vote 5-0 by Philippe Rocca-Serra, Janice Tufte, Harold Lehmann, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
A measure of dispersion calculated as the difference between the 75th percentile and the 25th percentile.

# concept[].concept[].concept[].property[].valueString
STATO: "inter quartile range = The interquartile range is a data item which corresponds to the difference between the upper quartile (3rd quartile) and lower quartile (1st quartile).
The interquartile range contains the second quartile or median.
The interquartile range is a data item providing a measure of data dispersion"

# concept[].concept[].concept[].property[].valueString
A measure of dispersion is a statistic that represents the variation or spread among data values in a dataset or data distribution.

The 75th percentile is the median of the portion of the dataset or distribution with values greater than the median value.

The 25th percentile is the median of the portion of the dataset or distribution with values lesser than the median value.

An interquartile range (as a statistic) is represented as a single value (the difference between 75th and 25th percentiles) while, in common language, the term interquartile  range is often expressed with two values (the 25th percentile and the 75th percentile).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-02-20 vote 5-0 by Philippe Rocca-Serra, Janice Tufte, Harold Lehmann, Mario Tristan, Eric Harvey

# concept[].concept[].concept[].definition
A measure of dispersion that represents the average of the distances from the mean of the dataset to each data point in the dataset.

# concept[].concept[].concept[].property[].valueString
STATO: standard deviation (s) = The standard deviation of a random variable, statistical population, data set, or probability distribution is a measure of variation which correspond to the average distance from the mean of the data set to any given point of that dataset. It also corresponds to the square root of its variance.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
Standard deviation for sample is a standard deviation in which the dataset is a sample. Standard deviation for population, when used as a statistical model parameter, is not a standard deviation as a type of statistic.

# concept[].concept[].concept[].property[].valueString
2023-05-15 vote 6-0 by Muhammad Afzal, Brian S. Alper, Jesus Lopez-Alcalde, Janice Tufte, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].concept[].definition
A standard deviation that is the square root of the quotient of the summation across data points of the square of the distance from each data point to the sample mean, and the degrees of freedom (where the degrees of freedom is sample size minus one).

# concept[].concept[].concept[].concept[].property[].valueString
Standard deviation is defined as a measure of dispersion that represents the average of the distances from the mean of the dataset to each data point in the dataset.
Standard deviation for sample is a standard deviation in which the dataset is a sample.
The formula for the standard deviation for sample ($s$) is:
$$s = \sqrt \frac{\sum\\{\substack{n\\i=1}} (x_i - \overline{x})^2}{n - 1}$$
where $n$ is the sample size (the number of independent observations, indexed by $i$), $x$ is observed value, and $\overline{x}$ is the sample mean.
The formula to calculate degrees of freedom depends on the model. For the degrees of freedom for a sample standard deviation, given the sample mean, it is n-1, because the nth observation is no longer independent, given the n-1 other observations and the sample mean.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2023-05-15 vote 5-0 by Muhammad Afzal, Brian S. Alper, Jesus Lopez-Alcalde, Eric Harvey, Harold Lehmann

# concept[].concept[].concept[].definition
A measure of dispersion that represents the square of the standard deviation.

# concept[].concept[].concept[].property[].valueString
STATO: variance (s2) = variance is a data item about a random variable or probability distribution. it is equivalent to the square of the standard deviation.  It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean (expected value).The variance is the second moment of a distribution.

# concept[].concept[].concept[].property[].valueString
Standard deviation is defined as a measure of dispersion that represents the average of the distances from the mean of the dataset to each data point in the dataset.

Variance for sample is a variance in which the dataset is a sample. Variance for population, when used as a probability distribution parameter, is not a variance as a type of statistic.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2023-11-20 vote 5-0 by Brian S. Alper, Jesus Lopez-Alcalde, Harold Lehmann, Muhammad Afzal, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A variance that is the quotient of the summation across data points of the square of the distance from each data point to the sample mean, and the degrees of freedom (where the degrees of freedom is sample size minus one).

# concept[].concept[].concept[].concept[].property[].valueString
Variance is defined as a measure of dispersion that represents the square of the standard deviation. Standard deviation is defined as a measure of dispersion that represents the average of the distances from the mean of the dataset to each data point in the dataset.

Variance for sample is a variance in which the dataset is a sample.
The formula for the variance for sample ($s^2$) is:
$$s^2 = \frac{\sum\\{\substack{n\\i=1}} (x_i - \overline{x})^2}{n - 1}$$
where $n$ is the sample size (the number of independent observations, indexed by $i$), $x$ is observed value, and $\overline{x}$ is the sample mean.
The formula to calculate degrees of freedom depends on the model. For the degrees of freedom for a sample variance, given the sample mean, it is n-1, because the nth observation is no longer independent, given the n-1 other observations and the sample mean.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2023-11-20 vote 5-0 by Brian S. Alper, Jesus Lopez-Alcalde, Harold Lehmann, Muhammad Afzal, Eric Harvey

# concept[].concept[].concept[].definition
A measure of dispersion that is half the relative mean absolute difference between all pairs of observed values.

# concept[].concept[].concept[].property[].valueString
The Gini index is typically used as a measure of inequality for income, wealth, or resource distribution.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2023-11-20 vote 5-0 by Brian S. Alper, Jesus Lopez-Alcalde, Harold Lehmann, Muhammad Afzal, Eric Harvey BUT comment of "between all pairs of observed values?" led to recognition of incorrect definition

# concept[].concept[].concept[].property[].valueString
2023-12-04 vote 5-0 by Yasser Sami Amer, Xing Song, Eric Harvey, Harold Lehmann, Brian S. Alper

# concept[].concept[].concept[].definition
A measure of dispersion applied to estimates across hypothetical repeated random samples.

# concept[].concept[].concept[].property[].valueString
STATO: It is a measure of how precise is an estimate of the statistical parameter is. Standard error is the estimated standard deviation of an estimate. It measures the uncertainty associated with the estimate. Compared with the standard deviations of the underlying distribution, which are usually unknown, standard errors can be calculated from observed data.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper, Xing Song, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
A standard error is used to quantify the uncertainty around a statistical estimate due to random sampling error.

# concept[].concept[].concept[].property[].valueString
2023-12-18 vote 7-0 by Xing Song, Muhammad Afzal, Yasser Sami Amer, Eric Harvey, Harold Lehmann, Janice Tufte, Caue Monaco

# concept[].concept[].concept[].concept[].definition
A measure of dispersion applied to means across hypothetical repeated random samples.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: The standard error of the mean (SEM) is data item denoting the standard deviation of the sample-mean's estimate of a population mean. It is calculated by dividing the sample standard deviation (i.e., the sample-based estimate of the standard deviation of the population) by the square root of n , the size (number of observations) of the sample.

# concept[].concept[].concept[].concept[].property[].valueString
A standard error is used to quantify the uncertainty around a statistical estimate due to random sampling error.
The standard error of the mean is calculated by dividing the sample standard deviation (STATO:0000237) by the square root of n, the size (number of observations) of the sample.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Muhammad Afzal, Xing Song, Kenneth Wilkins, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-01-22 vote 5-0 by Homa Keshavarz, Eric Harvey, Cauê Monaco, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].definition
A measure of dispersion applied to proportions across hypothetical repeated random samples.

# concept[].concept[].concept[].concept[].property[].valueString
A standard error is used to quantify the uncertainty around a statistical estimate due to random sampling error.

The formula for the standard error of the sample proportion ($SE(\hat{p})$) is:
$$SE(\hat{p}) = \sqrt \frac{\hat{p}(1-\hat{p})} {n}$$
where $\hat{p}$ is the sample proportion and $n$ is the size (number of observations) of the sample.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Xing Song, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].concept[].property[].valueString
2024-01-22 vote 5-0 by Homa Keshavarz, Eric Harvey, Brian S. Alper, Harold Lehmann, Yasser Sami Amer

# concept[].concept[].concept[].concept[].display
standard error of the difference between independent means

# concept[].concept[].concept[].concept[].definition
A measure of dispersion applied to differences between means of independent groups across hypothetical repeated random samples.

# concept[].concept[].concept[].concept[].property[].valueString
A standard error is used to quantify the uncertainty around a statistical estimate due to random sampling error.

In cases where the samples are assumed to have unequal population variances for X, the formula for the standard error of the sample difference between means $(SE_{unequal}(\overline{x}_{1} - \overline{x}_{2}))$ is:
$$SE_{unequal}(\overline{x}_{1} - \overline{x}_{2}) = \sqrt{\frac{s^2_1}{n_1}+\frac{s^2_2}{n_2}}$$
where $\overline{x}_{1}$ and $\overline{x}_{2}$ are the sample means, $s^2_1$ and $s^2_2$ are the sample standard deviations, and $n_1$ and $n_2$ are the sizes (number of observations) of the samples.

In cases where the samples are assumed to have the same (equal) population variance for X, the formula for the standard error of the sample difference between means $(SE_{equal}(\overline{x}_{1} - \overline{x}_{2}))$ is:
$$SE_{equal}(\overline{x}_{1} - \overline{x}_{2}) = \sqrt{\frac{n_1 s^2_1 + n_2 s^2_2}{n_1 + n_2 - 2}}$$
where $\overline{x}_{1}$ and $\overline{x}_{2}$ are the sample means, $s^2_1$ and $s^2_2$ are the sample standard deviations, and $n_1$ and $n_2$ are the sizes (number of observations) of the samples. In cases where the samples are assumed to have the same (equal) population variance for X, the standard error of the sample difference between means is also called the pooled standard deviation.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 vote 5-0 by Lenny Vasanthan, Xing Song, Eric Harvey, Harold Lehmann, Homa Keshavarz

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 comment: We may consider including the term of "pooled standard deviation" to describe the SE_equal(x1-x2), SE when assuming equal variance.

# concept[].concept[].concept[].concept[].display
standard error of the difference between independent proportions

# concept[].concept[].concept[].concept[].definition
A measure of dispersion applied to differences between proportions arising from independent groups across hypothetical repeated random samples.

# concept[].concept[].concept[].concept[].property[].valueString
A standard error is used to quantify the uncertainty around a statistical estimate due to random sampling error.

The formula for the standard error of the sample difference between proportions $(SE(\hat{p}_1 - \hat{p}_2))$ is:
$$SE(\hat{p}_1 - \hat{p}_2) =  \sqrt {\frac{\hat{p}_1(1-\hat{p}_1)} {n_1} + \frac{\hat{p}_2(1-\hat{p}_2)} {n_2}}$$
where $\hat{p}_1$ and $\hat{p}_2$ are the sample proportions and $n_1$ and $n_2$ are the sizes (number of observations) of the samples.

# concept[].concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Brian S. Alper

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 vote 5-0 by Lenny Vasanthan, Xing Song, Eric Harvey, Harold Lehmann, Homa Keshavarz

# concept[].concept[].concept[].concept[].property[].valueString
2024-02-12 comment: In hypothesis testing for p1 = p2, I think the typical way of calculating standard error of proportion difference, is using the formula with pooled proportion, should we also include that in the comment?

# concept[].concept[].concept[].definition
The range in which the value of the parameter of interest is likely to reside, typically within a posterior probability distribution.

# concept[].concept[].concept[].property[].valueString
STATO: In Bayesian statistics context, a credible interval is an interval of a posterior distribution which is such that the density at any point inside the interval is greater than the density at any point outside and that the area under the curve for that interval is equal to a prespecified probability level. For any probability level there is generally only one such interval, which is also often known as the highest posterior density region. Unlike the usual confidence interval associated with frequentist inference, here the intervals specify the range within which parameters lie with a certain probability. The Bayesian counterparts of the confidence interval used in Frequentists Statistics.

UMLS: "Interval (C1272706)
Definition: The period of time or the distance separating two instances, events, or occurrences.
Semantic Types: Temporal Concept"

OBCS: A quantitative confidence value that is used in Bayesian analysis to describe the range in which a posterior probability estimate is likely to reside.

OECD: calculated interval-The interval containing possible values for a suppressed cell in a table, given the table structure and the values published.

SCO: interval-An interval is a set of real numbers that includes all numbers between any two numbers in the set.

# concept[].concept[].concept[].property[].valueString
The credible interval is used in Bayesian analysis and plays an analogous role to the confidence interval in frequentist statistics.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Kenneth Wilkins, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2023-11-27 vote 5-0 by Xing Song, Yasser Sami Amer, Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
The estimated range of values that encompasses the point estimate and quantifies the uncertainty about that estimate in terms of a prespecified level of coverage, expected to include the true value between upper and lower bounds, across hypothetically repeated random samples, with all assumptions regarding the sampling distribution across random samples having been fully met.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Ken Wilkins, Joanne Dehnbostel, Robin Ann Yurk, Janice Tufte

# concept[].concept[].concept[].property[].valueString
STATO: A confidence interval is a data item which defines an range of values in which a measurement or trial falls corresponding to a given probability. also confidence interval calculation is a data transformation which determines a confidence interval for a given statistical parameter

NCIt: A range of values for a parameter that may contain the parameter and the degree of confidence that it is in fact there. A measure of the precision of an estimated value. The interval represents the range of values, consistent with the data, that is believed to encompass the "true" value with high probability (usually 95%). The confidence interval is expressed in the same units as the estimate. Wider intervals indicate lower precision; narrow intervals, greater precision. [CONSORT Statement]

OBCS: A quantitative confidence value that refers to an interval give values within which there is a high probability (95 percent by convention) that the true population value can be found. The calculation of a confidence interval considers the standard deviation of the data and the number of observations. Thus, a confidence interval narrows as the number of observations increases, or its variance (dispersion) decreases.

CDISC Glossary: A measure of the precision of an estimated value. The interval represents the range of values, consistent with the data, that is believed to encompass the "true" value with high probability (usually 95%). The confidence interval is expressed in the same units as the estimate. Wider intervals indicate lower precision; narrow intervals, greater precision. [CONSORT Statement]

NICE: "Confidence interval
A way of expressing how certain we are about the findings from a study, using statistics. It gives a range of results that is likely to include the 'true' value for the population. A wide confidence interval (CI) indicates a lack of certainty about the true effect of the test or treatment - often because a small group of patients has been studied. A narrow CI indicates a more precise estimate (for example, if a large number of patients have been studied).

The CI is usually stated as '95% CI', which means that the range of values has a 95 in a 100 chance of including the 'true' value. For example, a study may state that 'based on our sample findings, we are 95% certain that the 'true' population blood pressure is not higher than 150 and not lower than 110'. In such a case the 95% CI would be 110 to 150."

OECD: A confidence interval is an interval which has a known and controlled probability (generally 95% or 99%) to contain the true value.

"Rothman textbook: confidence interval, which provides a range of values for the association, under the hypothesis that only random variation has created discrepancies between the true value of the association under study and the value observed in the data (Altman et al., 2000; see Chapters 13 through 16) Altman DG, Machin D, Bryant TN, Gardner MJ, eds. Statistics with confidence, 2nd ed. London: BMJ Books, 2000
"

# concept[].concept[].concept[].property[].valueString
The prespecified level of coverage is commonly 0.95 or 95%. 

Confidence cannot be directly interpreted as a probability. This is in contrast to credibility for credible intervals. Confidence only conveys uncertainty indirectly by reflecting a long term relative frequency across hypothetically repeated sample estimates.

Width of a confidence interval can convey precision. This precision can be increased by increasing the sample size in most cases assuming variability in sample is only due to random sample-to-sample variation.

# concept[].concept[].concept[].property[].valueString
2023-11-27 vote 5-0 by Xing Song, Yasser Sami Amer, Harold Lehmann, Jesus Lopez-Alcalde, Eric Harvey

# concept[].concept[].concept[].definition
A statistic that represents the variation or spread among values in the set of estimates across studies.

# concept[].concept[].concept[].property[].valueString
STATO: a measure of heterogeneity in meta-analysis is a data item which aims to describe the variation in study outcomes between studies.
Cochrane Handbook 10.10.4 (August 2023)
Analysing data and undertaking meta-analyses 
Variability in the intervention effects being evaluated in the different studies is known as statistical heterogeneity, and is a consequence of clinical or methodological diversity, or both, among the studies. Statistical heterogeneity manifests itself in the observed intervention effects being more different from each other than one would expect due to random error (chance) alone. We will follow convention and refer to statistical heterogeneity simply as heterogeneity.
https://training.cochrane.org/handbook/current/chapter-10#section-10-10-4
There are types of heterogeneity (or diversity) other than statistical heterogeneity which are important factors in whether or not evidence can be pooled. Clinical heterogeneity may refer to variations in the population, intervention, comparator, or outcome. Methodological heterogeneity may refer to variations in study design.

# concept[].concept[].concept[].property[].valueString
There are several types of heterogeneity (or diversity) which are important factors which determine whether or not evidence should be pooled. Qualitative descriptors of explainable sources of heterogeneity include clinical heterogeneity and methodological heterogeneity. Clinical heterogeneity may refer to variations in the population, intervention, comparator, or outcome. Methodological heterogeneity may refer to variations in study design. Statistical heterogeneity, which is a quantitative measure of heterogeneity, whether explained or not, is described here.

A measure of dispersion is defined as a statistic that represents the variation or spread among data values in a dataset or data distribution. In the context of a meta-analysis, a measure of heterogeneity is a measure of dispersion in which the dataset is the set of estimates across studies.

# concept[].concept[].concept[].property[].valueString
Brian Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2024-03-11 vote 5-0 by Eric Harvey, Xing Song, Lenny Vasanthan, Harold Lehmann, Homa Keshavarz
2024-05-06 vote 5-0 by Harold Lehmann, Cauê Monaco, Homa Keshavarz, Eric Harvey, Lenny Vasanthan

# concept[].concept[].concept[].property[].valueString
2024-04-29 Comment for application revised during the Statistic Terminology Working Group meeting

# concept[].concept[].concept[].concept[].definition
A measure of heterogeneity, based on the chi-square statistic, for reporting an analytic finding regarding whether two or more multinomial distributions are equal, accounting for chance variability.

# concept[].concept[].concept[].concept[].designation[].value
chi-square test statistic for heterogeneity

# concept[].concept[].concept[].concept[].designation[].value
chi-square test statistic for homogeneity

# concept[].concept[].concept[].concept[].property[].valueString
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies.
Chi square for homogeneity assesses whether observed differences in results are compatible with chance alone.
A chi square for homogeneity is a hypothesis testing measure that is testing the hypothesis of heterogeneity. A chi square for homogeneity is distinct from a chi square for independence (also called Pearson's chi square). A chi square test for homogeneity is based on testing whether the distributions across two or more populations are the same.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Harold Lehmann, Muhammad Afzal, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Cochran's Q test is a statistical test used for unreplicated randomized block design experiments with a binary response variable and paired data. In the analysis of two-way randomized block designs where the response variable can take only two possible outcomes (coded as 0 and 1), Cochran's Q test is a non-parametric statistical test to verify whether k treatments have identical effects.

from Cochrane Handbook https://training.cochrane.org/handbook/current/chapter-10
More formally, a statistical test for heterogeneity is available. This Chi2 (χ2, or chi-squared) test is included in the forest plots in Cochrane Reviews. It assesses whether observed differences in results are compatible with chance alone. A low P value (or a large Chi2 statistic relative to its degree of freedom) provides evidence of heterogeneity of intervention effects (variation in effect estimates beyond chance).
Care must be taken in the interpretation of the Chi2 test, since it has low power in the (common) situation of a meta-analysis when studies have small sample size or are few in number. This means that while a statistically significant result may indicate a problem with heterogeneity, a non-significant result must not be taken as evidence of no heterogeneity. This is also why a P value of 0.10, rather than the conventional level of 0.05, is sometimes used to determine statistical significance. A further problem with the test, which seldom occurs in Cochrane Reviews, is that when there are many studies in a meta-analysis, the test has high power to detect a small amount of heterogeneity that may be clinically unimportant.

Hoaglin DC. Misunderstandings about Q and 'Cochran's Q test' in meta-analysis. Stat Med. 2016 Feb 20;35(4):485-95. doi: 10.1002/sim.6632. Epub 2015 Aug 24. PMID: 263037 and discussions

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-13 vote 3-1 by Eric Harvey, Harold Lehmann, Saphia Mokrane, Sheyu Li
2024-05-06 vote 5-2 by Cauê Monaco, Homa Keshavarz, Eric Harvey, Lenny Vasanthan, Harold Lehmann, Sean Grant, Sheyu Li

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-13 comments
Cochran's Q may be the most accepted name. I would thus suggest changing the term. Actually, Cochran's Q is Chi square contributed and theoretically there can be other Chi square distributed in testing heterogeneity or homogeneity (although current Cochran's Q may be the only one). 

2024-04-29 comments:
1) Add to "comment for application" the following from the Cochrane Handbook: "Chi square for homogeneity assesses whether observed differences in results are compatible with chance alone."

2) Is it chi square or Chi square? 

Is it Chi square for homogeneity or Chin square for heterogeneity? 

X2 test is a very common statistic. To avoid confusion, should we add some detailed statistical explanation regarding the difference between this x2 test and other x2 tests?

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-28 vote 7-0 by Carlos Alva-Diaz, Homa Keshavarz, Sheyu Li, Harold Lehmann, Saphia Mokrane, Eric Harvey, Lenny Vasanthan

# concept[].concept[].concept[].concept[].definition
A measure of heterogeneity that estimates the proportion of variability across studies that is in excess of the expected variability due to chance.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: I-squared = The quantity called I2, describes the percentage of total variation across studies that is due to heterogeneity rather than chance. I2 can be readily calculated from basic results obtained from a typical meta-analysis as I2 = 100%×(Q - df)/Q, where Q is Cochran's heterogeneity statistic and df the degrees of freedom. Negative values of I2 are put equal to zero so that I2 lies between 0% and 100%. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity. Unlike Cochran's Q, it does not inherently depend upon the number of studies considered. A confidence interval for I² is constructed using either i) the iterative non-central chi-squared distribution method of Hedges and Piggott (2001); or ii) the test-based method of Higgins and Thompson (2002). The non-central chi-square method is currently the method of choice (Higgins, personal communication, 2006) – it is computed if the 'exact' option is selected. (STATO:0000420)

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological methods, 6(3), 203–217.https://pubmed.ncbi.nlm.nih.gov/11570228/

Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186 https://pubmed.ncbi.nlm.nih.gov/12111919/

# concept[].concept[].concept[].concept[].property[].valueString
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. $I^2$ = 100%×(Q - df)/Q, where Q is Cochran's heterogeneity statistic and df is the degrees of freedom. $I^2$ is strictly non-negative and lies between 0 and 100%. (If Q is less than df and the calculation would result in a negative value, then $I^2$ is defined as zero.) There are competing methods of calculating the confidence interval such as Hedges and Piggott (2001) and Higgins and Thompson (2002).

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-20 comment:
"df, the degrees of freedom." Note punctuation needs.
"(If calculation results in a negative value, Q is less than df, 
𝐼2 is defined as zero.)" Is that supposed or be "if Q is less than df" or, "and Q is less than df" or "because"?

2024-05-28 comment:
Is the most commonly used name I square or I squared?  [[[so added as an alternative term]]]

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-28 vote 7-0 by Lenny Vasanthan, Carlos Alva-Diaz, Saphia Mokrane, Homa Keshavarz, Sheyu Li, Brian S. Alper, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A measure of heterogeneity that estimates the variance of the distribution of true effect sizes.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Tau-squared is an estimate of the between-study variance in a random-effects meta-analysis. The square root of this number (i.e. tau) is the estimated standard deviation of underlying effects across studies. (STATO:0000421)

# concept[].concept[].concept[].concept[].property[].valueString
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. The tau squared estimates the between-study variance in a random-effects meta-analysis or hierarchical multilevel model meta-analysis.

# concept[].concept[].concept[].concept[].property[].valueString
2024-05-28 vote 6-0 by Lenny Vasanthan, Carlos Alva-Diaz, Saphia Mokrane, Homa Keshavarz, Sheyu Li, Eric Harvey

# concept[].concept[].definition
A statistic that summarizes the variation of a quantity of interest across a domain interval of interest.

# concept[].concept[].property[].valueString
STATO: area under curve is a measurement datum which corresponds to the surface define by the x-axis and bound by the line graph represented in a 2 dimensional plot resulting from an integration or integrative calculus. The interpretation of this measurement datum depends on the variables plotted in the graph

# concept[].concept[].property[].valueString
As examples, in classification tasks, the quantity of interest is the true positive rate and the domain interval of interest is the false positive rate; in pharmacodynamic studies, the quantity of interest is the concentration of a drug and the domain interval of interest is time; and in assessment of lung barotrauma in intensive care, the quantity of interest is pressure and the domain interval of interest is time. 

In general, the average quantity is calculated as the area under the curve divided by the range of the domain interval of interest.

# concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Joanne Dehnbostel, Muhammad Afzal, Khalid Shahin

# concept[].concept[].property[].valueString
2024-03-11 vote 5-0 by Eric Harvey, Xing Song, Lenny Vasanthan, Harold Lehmann, Homa Keshavarz

# concept[].concept[].concept[].definition
An area under the curve where the curve is the true positive rate and the range of interest is the false positive rate.

# concept[].concept[].concept[].designation[].value
area under the receiver operating characteristic curve

# concept[].concept[].concept[].property[].valueString
One of the earliest descriptions of this concept is found in "The area above the ordinal dominance graph and the area below the receiver operating characteristic graph" (https://doi.org/10.1016/0022-2496(75)90001-2)

# concept[].concept[].concept[].property[].valueString
ROC stands for Receiver Operating Characteristic. The area under the ROC curve is used to assess the performance of a classifier used to distinguish between two or more groups. Another term for true positive rate is sensitivity and another term for false positive rate is 1-specificity. 

The c-statistic is the area under the ROC curve calculated with the full range of possible values for true positive rate and false positive rate. Another interpretation of the c-statistic is similar without explicitly referencing the ROC curve: "The C statistic is the probability that, given 2 individuals (one who experiences the outcome of interest and the other who does not or who experiences it later), the model will yield a higher risk for the first patient than for the second. It is a measure of concordance (hence, the name “C statistic”) between model-based risk estimates and observed events. C statistics measure the ability of a model to rank patients from high to low risk but do not assess the ability of a model to assign accurate probabilities of an event occurring (that is measured by the model’s calibration). C statistics generally range from 0.5 (random concordance) to 1 (perfect concordance)." (JAMA. 2015;314(10):1063-1064. doi:10.1001/jama.2015.11082)

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Kenneth Wilkins, Muhammad Afzal, Joanne Dehnbostel, Khalid Shahin

# concept[].concept[].concept[].property[].valueString
2024-04-22 vote 6-0 by Cauê Monaco, Harold Lehmann, Eric Harvey, Homa Keshavarz, Lenny Vasanthan, Sheyu Li

# concept[].concept[].concept[].property[].valueString
area under the ROC curve was approved 2024-03-18 vote 6-0 by Cauê Monaco, Homa Keshavarz, Elma OMERAGIC, Xing Song, Lenny Vasanthan, Eric Harvey BUT then ...
c-statistic had 2024-04-01 vote 4-2 by Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Lenny Vasanthan, Eric Harvey, Homa Keshavarz

# concept[].concept[].concept[].property[].valueString
2024-03-18 comment: Can we consider having the statement on "ROC stands for Receiver Operating Characteristic" in the first line followed by the area under the ROC curve?

 2024-04-01 comments for c-statistic:
1) isn't this a synonym for AUROC ? (with the assumption that AUROC refers to the totality of the space located below the curve.
2) Is it c-statistic or C statistic (capitaled)? 
Quote: The C statistic is the probability that, given 2 individuals (one who experiences the outcome of interest and the other who does not or who experiences it later), the model will yield a higher risk for the first patient than for the second. It is a measure of concordance (hence, the name “C statistic”) between model-based risk estimates and observed events. C statistics measure the ability of a model to rank patients from high to low risk but do not assess the ability of a model to assign accurate probabilities of an event occurring (that is measured by the model’s calibration). C statistics generally range from 0.5 (random concordance) to 1 (perfect concordance).I would use discrimination rather than distinguish for the purpose of C-statistic. (JAMA. 2015;314(10):1063-1064. doi:10.1001/jama.2015.11082)
I am also worried about the definition is highly relied on the definition of ROC. A link of ROC may be better for users. Discrimination rather than distinguish could be a better word to describe C statistic. 
C statistic also refers to concordance statistic.

# concept[].concept[].concept[].definition
An area under the curve where the curve is the true positive rate and the range of interest is a specified portion of the range of possible values for the false positive rate and/or range of possible values for the true positive rate.

# concept[].concept[].concept[].property[].valueString
Area under the ROC curve is defined as an area under the curve where the curve is the true positive rate and the range of interest is the false positive rate. ROC stands for Receiver Operating Characteristic. The area under the ROC curve is used to assess the performance of a classifier used to distinguish between two or more groups. Another term for true positive rate is sensitivity and another term for false positive rate is 1-specificity.

# concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Muhammad Afzal, Harold Lehmann

# concept[].concept[].concept[].property[].valueString
2024-04-01 vote 4-2 by Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Lenny Vasanthan, Eric Harvey, Homa Keshavarz
2024-04-15 vote 3-1 by Harold Lehmann, Eric Harvey, Lenny Vasanthan, Homa Keshavarz

# concept[].concept[].concept[].property[].valueString
2024-04-01 comments:
1) i assume 'area under the curve' to equate the entire area located under the curve, so 'a part' can not be a subtype of the whole.
2) Similar concerns with the definition of C statistic. 

2024-04-15 comment:
I think we have to add, to the definition, "and/or range of possible values for the true positive rate" as well. My apologies, but I didn't catch this during our dicussions.

# concept[].concept[].concept[].property[].valueString
2024-04-22 vote 5-0 by Lenny Vasanthan, Homa Keshavarz, Eric Harvey, Harold Lehmann, Sheyu Li

# concept[].concept[].concept[].definition
An area under the curve where the curve is the precision and the domain of interest is the recall.

# concept[].concept[].concept[].property[].valueString
In information retrieval, recall is a synonym for sensitivity and precision is a synonym for positive predictive value.

# concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Harold Lehmann, Brian S. Alper

# concept[].concept[].concept[].property[].valueString
2024-04-01 vote 6-0 by Homa Keshavarz, Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].definition
An area under the curve where the curve is the repeated measures of a variable over time and the domain of interest is time.

# concept[].concept[].concept[].property[].valueString
The area under the value-by-time curve is used for pharmacokinetics, pharmacodynamics, and physiological monitoring.

# concept[].concept[].concept[].property[].valueString
2024-04-01 comment: Is it value-time curve?

# concept[].concept[].concept[].property[].valueString
2024-04-01 vote 5-1 by Homa Keshavarz, Philippe Rocca-Serra, Sheyu Li, Harold Lehmann, Lenny Vasanthan, Eric Harvey

# concept[].concept[].concept[].property[].valueString
2024-04-29 vote 7-0 by Lenny Vasanthan, Harold Lehmann, Eric Harvey, Homa Keshavarz, Sean Grant, Philippe Rocca-Serra, Sheyu Li

# concept[].concept[].definition
A statistic that represents the boundary at which something changes.

# concept[].concept[].property[].valueString
The thing that changes at the threshold value may be relevant for function, application, classification, or detection.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Kenneth Wilkins, Khalid Shahin

# concept[].concept[].property[].valueString
5/5 as of 10/11/2021: Janice Tufte, Joanne Dehnbostel, Louis Leff, Vignesh Subbian, Robin Ann Yurk

# concept[].concept[].definition
A statistic that represents the number of independent values used to calculate a statistical estimate. The number of degrees of freedom ν is equal to the number of independent units of information given the model.

# concept[].concept[].property[].valueString
The formula to calculate degrees of freedom will depend on the model. For example, the degrees of freedom for a sample standard deviation, given the sample mean, is N-1, because the Nth observation is no longer independent, given the N-1 other observations and the sample mean.

# concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Janice Tufte, Bhagvan Kommadi, Kenneth Wilkins, Khalid Shahin, Harold Lehmann

# concept[].concept[].property[].valueString
6/6 as of 10/27/2021: Janice Tufte, Louis Leff, Vignesh Subbian, Robin Ann Yurk, Harold Lehmann, Muhammad Afzal, Pentti Nieminen

# concept[].concept[].property[].valueString
Include * in P = x1 * x2...to clarify this is a product.

# concept[].concept[].definition
A statistic that represents the relative support for competing hypotheses, based on the observed data under an assumed modeling framework.

# concept[].concept[].property[].valueString
A hypothesis testing measure may be used within the frequentist framework (Neyman-Pearson framework) or the Bayesian framework. Within the frequentist framework, the criterion for rejecting the null hypothesis is typically expressed as a [p-value](https://fevir.net/resources/CodeSystem/27270#TBD:0000076) that is less than an [alpha setting](https://fevir.net/resources/CodeSystem/27270#TBD:0000081). Within the Bayesian framework, the approach for rejecting the null hypothesis is typically based on a Bayes factor.

# concept[].concept[].property[].valueString
Brian S. Alper, Kenneth Wilkins, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].property[].valueString
2024-06-10 vote 4-1 by Harold Lehmann, Lenny Vasanthan, Sean Grant, Eric Harvey, Sheyu Li
2024-06-17 vote 7-0 by Harold Lehmann, Sean Grant, Carlos Alva-Diaz, Lenny Vasanthan, Eric Harvey, Yaowaluk Ngoenwiwatkul, Homa Keshavarz BUT ALTERNATIVE TERMS AND COMMENT MODIFIED
2024-06-24 vote 7-1 by Sean Grant, Philippe Rocca-Serra, Sheyu Li, Saphia Mokrane, Cauê Monaco, Eric Harvey, Lenny Vasanthan, Homa Keshavarz

# concept[].concept[].property[].valueString
2024-06-10 comments re: "hypothesis testing measure" = "A statistic that represents the result of evaluating the congruence between a hypothesis and the statistics derived from the observed data."
1) I guess we'll add "hypothesis" to our emerging glossary.
2N) Would Bayesians say that they are "testing" a hypothesis?
3) The definition looks fine but the comment seems confusing and unnecessary. 


2024-06-17 comment re: "hypothesis testing measure" = "A statistic that represents the result of evaluating the congruence between a hypothesis and the statistics derived from the observed data."
"Hypothesis testing statistic" as an alternative term?

2024-06-24 comment re: "hypothesis testing measure" = "A statistic that represents the result of evaluating the congruence between a hypothesis and the statistics derived from the observed data."
The definition seems wordy. Why not: A statistic that represents the strength (or value?) of a statistical test (or: between the hypothesis and the observation).

# concept[].concept[].property[].valueString
2024-07-08 vote 6-0 by Saphia Mokrane, C P Ooi, Harold Lehmann, Eric Harvey, Lenny Vasanthan, Homa Keshavarz

# concept[].concept[].concept[].definition
A hypothesis testing measure that represents the probability of obtaining a result at least as far from the value actually obtained as the value expected, assuming the null hypothesis is true.

# concept[].concept[].concept[].property[].valueString
STATO: A quantitative confidence value that represents the probability of obtaining a result at least as extreme as that actually obtained, assuming that the actual value was the result of chance alone. (OBI:0000175)

"You may summarize this comparison using a Bayesian p-value (Gelman et al., 1996, 2004), the predictive probability that a statistic is equal to or more extreme than that observed under the assumptions of the model." -- https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials

# concept[].concept[].concept[].property[].valueString
[Hypothesis testing measure](https://fevir.net/resources/CodeSystem/27270#TBD:0000073) is defined as a statistic that represents the result of evaluating the congruence between a hypothesis and the statistics derived from the observed data.

Within the frequentist framework, a p-value is typically a [p-value for two-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-two-sided) and the criterion for rejecting the null hypothesis is typically expressed as a p-value that is less than an [alpha setting](https://fevir.net/resources/CodeSystem/27270#TBD:0000081) divided by 2. In some cases, the p-value is a [p-value for one-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-one-sided), and the criterion for rejecting the null hypothesis is typically expressed as a p-value that is less than an [alpha setting](https://fevir.net/resources/CodeSystem/27270#TBD:0000081). A p-value is preferably coded more precisely as a [p-value for two-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-two-sided) or a [p-value for one-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-one-sided) rather than using the code for p-value without specification. The code for p-value (without specification) may be used in contexts where the p-value reported is ambiguous.

Within the Bayesian framework, use the [posterior predictive p-value](https://fevir.net/resources/CodeSystem/27270#TBD:Bayesianp).

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].concept[].property[].valueString
2024-06-17 vote 7-1 by Harold Lehmann, Sean Grant, Khalid Shahin, Carlos Alva-Diaz, Lenny Vasanthan, Eric Harvey, Yaowaluk Ngoenwiwatkul, Homa Keshavarz
2024-07-01 vote 8-1 by Harold Lehmann, Sean Grant, Philippe Rocca-Serra, Sheyu Li, Saphia Mokrane, Cauê Monaco, Eric Harvey, Lenny Vasanthan, Homa Keshavarz
2024-07-08 vote 3-0 by Saphia Mokrane, Eric Harvey, C P Ooi BUT THEN THE DEFINITION CHANGED
2024-07-15 vote 4-2 by Homa Keshavarz, Cauê Monaco, Sheyu Li, Eric Harvey, Philippe Rocca-Serra, Lenny Vasanthan

# concept[].concept[].concept[].property[].valueString
2024-06-17 comments re: "p value" = "A hypothesis testing measure that represents the probability of obtaining a result at least as extreme as that actually obtained, assuming that the actual value was the result of chance alone."
1) "Within the frequentist framework..."
2N) The threshold is the alpha-level, not the p-value
3) I personally prefer p-value, but I'm fine with "p value"

2024-07-01 comments re: "p-value" = "A hypothesis testing measure that represents the probability of obtaining a result at least as extreme as that actually obtained, assuming the null hypothesis is true."
1) suggestion: add a sentence to also cover Bayesian framework
2N) The hypothesis can be null or not null (non-inferior test for example). 


2024-07-15 comments re: "p-value" = "A hypothesis testing measure that represents the probability of obtaining a result at least as far from the value actually obtained as the value expected, assuming the null hypothesis is true."
1N)
with the distinction with posterior predictive p-value, one-sided pvalue and two-sided pvalue, is pvalue still needed?
when should the term be used? if used to annotate data, is it imprecise as one would not know if both hypotheses have been tested. The introduction of more specific terms may render this type moot.
Also in the 'comment for application', if pvalue is parent term, the statement 'typically expressed as a p-value lesss than an alpha setting ' would not be true for 'two-sided pvalue'.
2N)
I am wondering if it might be helpful to add a statement on how 'p' value can help to identify if the desired effect is due to 'chance' or if there is an actual difference

# concept[].concept[].concept[].property[].valueString
2024-07-22 vote 7-0 by Saphia Mokrane, Carlos Alva-Diaz, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A p-value which represents the probability of obtaining a result at least as far, in one direction, from the value actually obtained as the value expected, assuming the null hypothesis is true. A p-value for one-sided test interprets 'at least as far from' with only one of two directions, either the direction of 'greater than' or the direction of 'less than'.

# concept[].concept[].concept[].concept[].property[].valueString
[P-value](https://fevir.net/resources/CodeSystem/27270#TBD:0000076) is defined as a hypothesis testing measure that represents the probability of obtaining a result at least as far from the value actually obtained as the value expected, assuming the null hypothesis is true. For p-value for one-sided test, the criterion for rejecting the null hypothesis is typically expressed as a p-value that is less than an [alpha setting](https://fevir.net/resources/CodeSystem/27270#TBD:0000081).

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Harold Lehmann, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-08 vote 8-1 by C P Ooi, Harold Lehmann, Sean Grant, Philippe Rocca-Serra, Sheyu Li, Saphia Mokrane, Eric Harvey, Lenny Vasanthan, Homa Keshavarz
2024-07-15 vote 5-1 by Homa Keshavarz, Cauê Monaco, Sheyu Li, Eric Harvey, Philippe Rocca-Serra, Lenny Vasanthan

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-08 comment re: "p-value for one-sided test" = "A p-value which represents the probability of obtaining a result at least as extreme, in one direction, as that actually obtained."
1N) The word 'one direction' did not explain 'one sided'. 

2024-07-15 comment re: "p-value for one-sided test" = "A p-value which represents the probability of obtaining a result at least as far, in one direction, from the value actually obtained as the value expected, assuming the null hypothesis is true."
1N) from an end user perspective, i feel that the definition should included the last statement found in the 'comment for application' = " A p-value for two-sided test interprets 'at least as far from' with both the direction of 'greater than' and the direction of 'less than'. For hypothesis test interpretation, the one-tailed p-value is compared to the alpha setting"

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-22 vote 6-0 by Carlos Alva-Diaz, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].concept[].definition
A p-value which represents the probability of obtaining a result at least as far, in either direction, from the value actually obtained as the value expected, assuming the null hypothesis is true. A p-value for two-sided test interprets 'at least as far from' with both the direction of 'greater than' and the direction of 'less than'.

# concept[].concept[].concept[].concept[].property[].valueString
[P-value](https://fevir.net/resources/CodeSystem/27270#TBD:0000076) is defined as a hypothesis testing measure that represents the probability of obtaining a result at least as  far from the value actually obtained as the value expected, assuming the null hypothesis is true. For hypothesis test interpretation, the two-tailed p-value is compared to the [alpha setting](https://fevir.net/resources/CodeSystem/27270#TBD:0000081) divided by 2.

# concept[].concept[].concept[].concept[].property[].valueString
Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal, Harold Lehmann, Kenneth Wilkins

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-15 comment re: "p value for two-sided test" = "A p-value which represents the probability of obtaining a result at least as far, in either direction, from the value actually obtained as the value expected, assuming the null hypothesis is true."
1N) from an end user perspective, i feel that the definition should included the last statement found in the 'comment for application' = " A p-value for two-sided test interprets 'at least as far from' with both the direction of 'greater than' and the direction of 'less than'. For hypothesis test interpretation, the two-tailed p-value is compared to the alpha setting divided by 2"

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-15 vote 5-1 by Homa Keshavarz, Cauê Monaco, Sheyu Li, Eric Harvey, Philippe Rocca-Serra, Lenny Vasanthan

# concept[].concept[].concept[].concept[].property[].valueString
2024-07-22 vote 6-0 by Carlos Alva-Diaz, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Harold Lehmann, Eric Harvey

# concept[].concept[].concept[].definition
A hypothesis testing measure that is assumed to follow a chi-square distribution.

# concept[].concept[].concept[].property[].valueString
STATO: Chi-squared statistic is a statistic computed from observations and used to produce a p-value in statistical test when compared to a Chi-Squared distribution. (STATO:0000030)

# concept[].concept[].concept[].property[].valueString
A hypothesis testing measure is defined as a statistic that represents the relative support for competing hypotheses, based on the observed data under an assumed modeling framework.

# concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Harold Lehmann, Brian S. Alper, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2024-10-28 vote 6-0 by Saphia Mokrane, Eric Harvey, Harold Lehmann, Homa Keshavarz, Airton Tetelbom Stein, Lenny Vasanthan

# concept[].concept[].concept[].concept[].definition
A chi-square statistic used to test whether two categorical or nominal variables are associated.

# concept[].concept[].concept[].concept[].property[].valueString
STATO: Chi-squared statistic is a statistic computed from observations and used to produce a p-value in statistical test when compared to a Chi-Squared distribution. (STATO:0000030)

# concept[].concept[].concept[].concept[].property[].valueString
Types of chi-square statistic for independence include the Pearson's chi-square statistic for independence and the Yate's corrected chi-square statistic. The chi-square statistic for independence is a chi-square statistic used for testing for an association.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Harold Lehmann, Brian S. Alper, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-04 vote 6-0 by Lara Kahaleh, Bhagvan Kommadi, Harold Lehmann, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].property[].valueString
2024-10-28 comment re: "chi-square statistic for independence" = "A chi-square statistic used to determine whether two categorical or nominal variables are likely to be related."
1Yes)
Can we consider adding in "chi-squared statistic for association" along with this.

# concept[].concept[].concept[].concept[].property[].valueString
2024-10-28 vote 6-0 by Saphia Mokrane, Eric Harvey, Harold Lehmann, Homa Keshavarz, Airton Tetelbom Stein, Lenny Vasanthan BUT THEN COMMENT CHANGED DEFINITION AND COMMENT FOR APPLICATION

# concept[].concept[].concept[].concept[].display
Cochran-Armitage chi-square statistic for trend

# concept[].concept[].concept[].concept[].definition
A chi-square statistic used to test whether a dichotomous variable and an ordinal variable are related.

# concept[].concept[].concept[].concept[].designation[].value
Cochran-Armitage chi-squared statistic for trend

# concept[].concept[].concept[].concept[].property[].valueString
There are types of chi-square statistic for trend other than the Cochran-Armitage statistic.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-04 vote 6-0 by Lara Kahaleh, Bhagvan Kommadi, Harold Lehmann, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].definition
A chi-square statistic used to test whether observed data from two or more groups follow the same distribution.

# concept[].concept[].concept[].concept[].property[].valueString
In meta-analysis, a chi-square statistic for homogeneity is often used to determine the appropriateness of combining the statistics to represent a common population.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel, Harold Lehmann

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-04 vote 5-1 by Lara Kahaleh, Bhagvan Kommadi, Harold Lehmann, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-04 comment re: "chi-square statistic for homogeneity" = "A chi-square statistic used to test whether observed data from two or more groups follow the same distribution."
1No)
I recall that there used to be considerable controversy about the value of statistical tests applied to baseline characteristics, the recommendation being to avoid such tests. If this remains the case, shouldn't we note that here?

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-11 vote 7-0 by Lenny Vasanthan, Yaowaluk Ngoenwiwatkul, Cauê Monaco, Saphia Mokrane, Eric Harvey, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].concept[].concept[].display
chi-square statistic for goodness of fit

# concept[].concept[].concept[].concept[].definition
A chi-square statistic used to test whether observed data follows a specified distribution.

# concept[].concept[].concept[].concept[].designation[].value
chi-squared statistic for goodness of fit

# concept[].concept[].concept[].concept[].property[].valueString
There are types of chi-square statistic for goodness of fit with named tests based on the specified distribution, such as the Hosmer–Lemeshow test for a given logistic regression model and the Shapiro–Wilk test for a normal (Gaussian) distribution.

# concept[].concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Brian S. Alper, Joanne Dehnbostel

# concept[].concept[].concept[].concept[].property[].valueString
2024-11-04 vote 6-0 by Lara Kahaleh, Bhagvan Kommadi, Harold Lehmann, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein

# concept[].concept[].concept[].definition
A hypothesis testing measure that is a z-score where the specified mean is based on the null hypothesis and the standard deviation is based on the observed data.

# concept[].concept[].concept[].property[].valueString
STATO: A z-score (also known as z-value, standard score, or normal score) is a measure of the divergence of an individual experimental result from the most probable result, the mean. Z is expressed in terms of the number of standard deviations from the mean value. (STATO:0000104)   Z-statistic is a statistic computed from observations and used to produce a p-value when compared to a Standard Normal Distribution in a statistical test called the Z-test. (STATO:0000376)

# concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Joanne Dehnbostel, Harold Lehmann, Brian S. Alper

# concept[].concept[].concept[].property[].valueString
A [z-score](#STATO:0000104) is defined as a statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value.
A z-statistic is a z-score of a sample in the conduct of a Z-test.

# concept[].concept[].concept[].property[].valueString
2024-08-12 vote 7-0 by Harold Lehmann, Homa Keshavarz, Eric Harvey, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Sean Grant

# concept[].concept[].concept[].definition
A hypothesis testing measure that is a t-score where the specified mean is based on the null hypothesis, the standard error is based on the observed data, and the degrees of freedom is based on the sample size.

# concept[].concept[].concept[].property[].valueString
STATO: t-statistic is a statistic computed from observations and used to produce a p-value in statistical test when compared to a Student's t distribution. (STATO:0000176)

# concept[].concept[].concept[].property[].valueString
A t-score is defined as a statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value.
A t-statistic is a t-score of a sample in the conduct of a t-test.

# concept[].concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].concept[].property[].valueString
2024-10-14 vote 5-0 by Lenny Vasanthan, Harold Lehmann, Eric Harvey, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].concept[].definition
A hypothesis testing measure that is a ratio of the probability of the observed data under one hypothesis divided by the probability of the same observed data under a different hypothesis.

# concept[].concept[].concept[].property[].valueString
STATO: Bayes factor is a ratio between 2 probabilities of observing data according 2 distinct models. It is used in Bayes model selection to evaluate which model best explains the data. if K<0, the model used in the denominator term is supported, if K>1, the model used in the numerator term is supported. The Bayes factor is about the plausibility of 2 different models

NCI Code         CDISC Submission Value      CDISC Synonym          NCI Preferred Term    CDISC Definition

C142403          Bayesian approaches             _                                           Bayesian Approach        

Approaches to data analysis that provide a posterior probability distribution for some parameter (e.g., treatment effect), derived from the observed data and a prior probability distribution for the parameter. The posterior distribution is then used as the basis for statistical inference. [ICH E9 Glossary]

C142404          Bayesian statistics                   _                                           Bayesian Statistics        

Statistical approach named for Thomas Bayes (1701-1761) that has among its features giving a subjective interpretation to probability, accepting the idea that it is possible to talk about the probability of hypotheses being true and of parameters having particular values.

# concept[].concept[].concept[].property[].valueString
The Bayes factor numerator and denominator take into account the prior probabilities of each hypothesis. The Bayes factor represents the relative plausibility between competing hypotheses, taking the prior probabilities into account.

# concept[].concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel

# concept[].concept[].concept[].property[].valueString
2024-11-11 vote 7-0 by Lenny Vasanthan, Yaowaluk Ngoenwiwatkul, Cauê Monaco, Saphia Mokrane, Eric Harvey, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].concept[].definition
A hypothesis testing measure that is the probability that a statistic value derived from a posterior predictive distribution under the assumptions of the model is equal to or more extreme than the observed posterior value of the statistic.

# concept[].concept[].concept[].property[].valueString
[Hypothesis testing measure](https://fevir.net/resources/CodeSystem/27270#TBD:0000073) is defined as a statistic that represents the result of evaluating the congruence between a hypothesis and the statistics derived from the observed data.

A posterior predictive p-value includes assumptions of prior probability of the hypothesis or parameters in the model.

The term 'posterior predictive p-value' is used within the Bayesian framework. Within the frequentist framework, use [p-value for two-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-two-sided) or [p-value for one-sided test](https://fevir.net/resources/CodeSystem/27270#TBD:p-value-one-sided).

Note that in the Bayesian case, one generally focuses on the posterior probability distribution of the parameter of interest, not on this posterior probability of a statistic.

# concept[].concept[].concept[].property[].valueString
"You may summarize this comparison using a Bayesian p-value (Gelman et al., 1996, 2004), the predictive probability that a statistic is equal to or more extreme than that observed under the assumptions of the model." -- https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials

# concept[].concept[].concept[].property[].valueString
Kenneth Wilkins, Harold Lehmann, Philippe Rocca-Sera, Brian S. Alper

# concept[].concept[].concept[].property[].valueString
2024-07-08 vote 3-1 by C P Ooi, Sheyu Li, Eric Harvey, Harold Lehmann
2024-07-15 vote 5-1 by Homa Keshavarz, Cauê Monaco, Sheyu Li, Eric Harvey, Philippe Rocca-Serra, Lenny Vasanthan
2024-07-22 vote 6-0 by Carlos Alva-Diaz, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Harold Lehmann, Eric Harvey (Comment led us to reopen this term for voting)

# concept[].concept[].concept[].property[].valueString
2024-07-08 comment re: "posterior predictive p-value" = "A hypothesis testing measure that is the predictive probability that a statistic is equal to or more extreme than that observed under the assumptions of the model."
1N) I think "Bayesian" should be the primary name, and something about priors and data should be included in the comment for application, since this is an atypical measure. (Sorry not having pointed this out during the session.)

2024-07-15 comment re: "posterior predictive p-value" = "A hypothesis testing measure that is the predictive probability that a statistic is equal to or more extreme than that observed under the assumptions of the model."
1N) add "Within the frequentist framework, use 'one-sided or two sided pvalue" (or pvalue if kept) to the 'comment for application' for consistency and reciprocity

2024-07-22 comment re: "posterior predictive p-value" = "A hypothesis testing measure that is the probability that a statistic value derived from a posterior predictive distribution under the assumptions of the model is equal to or more extreme than the observed posterior value of the statistic."
1Y) Because most "posteriors" in Bayesian statistics are about the parameter we really care about, let me suggest saying something like: "Note that in the Bayesian case, one generally focuses on the posterior probability distribution of the parameter of interest, not on this posterior probability of a statistic."  And we will need a SEVCO term for "posterior probability," to go under Bayes factor (not a child).

# concept[].concept[].concept[].property[].valueString
2024-07-29 vote 5-0 by Lenny Vasanthan, Airton Tetelbom Stein, Eric Harvey, Harold Lehmann, Homa Keshavarz

# concept[].concept[].definition
A statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value.

# concept[].concept[].property[].valueString
STATO: A z-score (also known as z-value, standard score, or normal score) is a measure of the divergence of an individual experimental result from the most probable result, the mean. Z is expressed in terms of the number of standard deviations from the mean value. (STATO:0000104)

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].property[].valueString
2024-08-12 vote 7-1 by Harold Lehmann, Homa Keshavarz, Eric Harvey, Lenny Vasanthan, Sheyu Li, Elma OMERAGIC, Airton Tetelbom Stein, Sean Grant
2024-08-19 vote 6-1 by Philippe Rocca-Serra, Sean Grant, Airton Tetelbom Stein, Homa Keshavarz, Cauê Monaco, Eric Harvey, Brian S. Alper

# concept[].concept[].property[].valueString
2024-08-12
Comment for application maybe should include: "A z-score can be calculated for a statistic for a sample z-statistic or may be calculated for an individual, using a reference mean and standard deviation, as in an IQ score."

2024-08-19 comment re: "z-score" = "A statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value."
1N)
Does 'calculated for a statistic for a sample z-statistic' mean that you calculate the z-score from the z-statistic and they are 2 different values, or does it mean that z-statistic is the term for z-score when it is calculated for a sample?  This ambiguity can be avoided by making the comment for application 2 separate sentences, one clearly describing "for a sample" and the other clearly describing "for an individual observation" -- note individual needs to be an adjective and not a noun in the second sentence.  An individual person can be the population for a sample of observations about the individual.
re: "z-score" = "A statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value."

2024-08-26 comment re: "z-score" = "A statistic that represents a measure of the divergence of an individual experimental result from a specified mean, expressed in terms of the number of standard deviations from the mean value."
1Y)
Only a petit change in comment for application..."A z-score can also be calculated for a statistic for a sample, such as a z-statistic, which is a hypothesis testing measure. The z-score for the sample is based on the null hypothesis mean, and the standard deviation is based on the observed data."

# concept[].concept[].property[].valueString
A z-score can be calculated for an individual observation, using a reference mean and standard deviation, such as in an IQ score or a weight-for-age-and-sex value in a pediatric growth chart.
A z-score can also be calculated for a statistic for a sample, such as a [z-statistic](https://fevir.net/resources/CodeSystem/27270#STATO:0000376), which is a hypothesis testing measure. The z-score for the sample is based on the null hypothesis mean, and the standard deviation is based on the observed data.

# concept[].concept[].property[].valueString
2024-08-26 vote 7-0 by Carlos Alva-Diaz, Elma OMERAGIC, Lenny Vasanthan, Eric Harvey, Homa Keshavarz, Airton Tetelbom Stein, Harold Lehmann

# concept[].concept[].definition
A statistic that represents a measure of the divergence of an individual experimental result from a specified mean, taking sample size into account and expressed in terms of the number of standard errors from the mean value.

# concept[].concept[].property[].valueString
A t-score can be calculated for an individual observation, using a reference mean and standard error, such as in a bone mineral density value with a reference mean and standard error for young adult men.
A t-score can also be calculated for a statistic for a sample, such as a t-statistic, which is a hypothesis testing measure. The t-score for the sample is based on the null hypothesis mean, and the standard error is based on the observed data.
A t-score is used preferentially to a z-score when the sample size is low (e.g. < 30 for a unimodal symmetric distribution), the population standard deviation is unknown, or when the sample distribution cannot be assumed to follow a normal distribution.

# concept[].concept[].property[].valueString
Brian S. Alper, Harold Lehmann, Joanne Dehnbostel, Muhammad Afzal, Kenneth Wilkins

# concept[].concept[].property[].valueString
2024-10-14 vote 5-0 by Lenny Vasanthan, Harold Lehmann, Eric Harvey, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].definition
A statistic that represents the likelihood of a parameter of interest, before accounting for the observed data from the study.

# concept[].concept[].property[].valueString
This term is core to Bayesian statistics, where the focus is on estimation of the parameter of interest (unlike frequentist statistics, where the focus is on the likelihood of producing an estimate given a specific value of the parameter).
The prior probability may be constructed in several ways: from prior research, from raw data in a database, from expert opinion, or specified as "non-informative" (meaning pure ignorance).
The likelihood of the parameter is usually represented as a distribution over all possible values of the parameter.

# concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].property[].valueString
2024-08-12  comments:
1N)
I would suggest including Bayersian analysis as a context in the definition but not only in the comments. 
2Y)
Just a comment for us: Note that this may be the first time that "statistic" is used NOT to mean a function of the observed data, but is still covered by our SEVCO definition for "statistic."


2024-08-19 comment re: "prior probability" = "A statistic that represents the uncertainty of a parameter of interest, before accounting for the observed data from the study."
1N)
Uncertainty and likelihood are not synonymous.  The word 'uncertainty' does in the proposed definition of prior probability does not match the parallel definition of posterior probability (using 'likelihood') and does not match the comment in posterior probability which states 'The prior probability is the likelihood of a parameter of interest'

# concept[].concept[].property[].valueString
2024-08-12 vote 6-1 by Harold Lehmann, Homa Keshavarz, Eric Harvey, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Sean Grant
2024-08-19 vote 6-1 by Philippe Rocca-Serra, Sean Grant, Airton Tetelbom Stein, Homa Keshavarz, Cauê Monaco, Eric Harvey, Brian S. Alper

# concept[].concept[].property[].valueString
NCI Code         CDISC Submission Value      CDISC Synonym          NCI Preferred Term    CDISC Definition

C142403          Bayesian approaches             _                                           Bayesian Approach        

Approaches to data analysis that provide a posterior probability distribution for some parameter (e.g., treatment effect), derived from the observed data and a prior probability distribution for the parameter. The posterior distribution is then used as the basis for statistical inference. [ICH E9 Glossary]

C142404          Bayesian statistics                   _                                           Bayesian Statistics        

Statistical approach named for Thomas Bayes (1701-1761) that has among its features giving a subjective interpretation to probability, accepting the idea that it is possible to talk about the probability of hypotheses being true and of parameters having particular values.

# concept[].concept[].property[].valueString
2024-11-11 vote 7-0 by Lenny Vasanthan, Yaowaluk Ngoenwiwatkul, Cauê Monaco, Saphia Mokrane, Eric Harvey, Airton Tetelbom Stein, Homa Keshavarz

# concept[].concept[].definition
A statistic that represents the likelihood of a parameter of interest, after updating its prior probability with the observed data from the study.

# concept[].concept[].property[].valueString
This term is core to Bayesian statistics, where the focus is on estimation of the parameter of interest (unlike frequentist statistics, where the focus is on the likelihood of producing an estimate given a specific value of the parameter). The [prior probability](#TBD:priorprobability) is the likelihood of a parameter of interest from before the study. For calculating the posterior probability of a parameter, the  [prior probability](#TBD:priorprobability) is updated using Bayes' Theorem' and the likelihood function.

# concept[].concept[].property[].valueString
Harold Lehmann, Brian S. Alper, Joanne Dehnbostel, Muhammad Afzal

# concept[].concept[].property[].valueString
2024-08-12 vote 6-1 by Harold Lehmann, Homa Keshavarz, Eric Harvey, Lenny Vasanthan, Sheyu Li, Airton Tetelbom Stein, Sean Grant

# concept[].concept[].property[].valueString
2024-08-12
1N)
I would suggest including Bayersian analysis as a context in the definition but not only in the comments. 
2Y)
I would amend the last sentence of the Comment for application as, "For calculating the posterior probability of a parameter, the prior probabilty is updated using Bayes' Theorem' "

# concept[].concept[].property[].valueString
2024-08-19 vote 7-0 by Philippe Rocca-Serra, Brian S. Alper, Sean Grant, Airton Tetelbom Stein, Homa Keshavarz, Cauê Monaco, Eric Harvey