January 10, 2024 | Author: Anonymous | Category: N/A
Download ISO-TC194 - N1254 - Draft ISO-NP 18969 - Clinical Evaluation of Medical Devices...
ISO/TC 194 N 1254 ISO/TC 194 "Biological and clinical evaluation of medical devices" Secretariat: DIN Committee manager: Minkwitz Susann Mrs Dr.
draft ISO/NP 18969 - Clinical evaluation of medical devices Document type Ballot / Reference document
Related content
Document date
Ballot: ISO/NP 18969 (restricted access) 2022-08-19
Expected action VOTE by 2022-11-15
ISO #####-#:####(X) ISO TC 194/ WG 4
Date: YYYY-MM-DD
Clinical evaluation of medical devices
NP proposal Warning for WDs and CDs
This document is not an ISO International Standard. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an International Standard. Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation.
© ISO #### – All rights reserved
ISO #####-#:####(X)
© ISO 20XX
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of the requester. ISO copyright office CP 401 • Ch. de Blandonnet 8 CH-1214 Vernier, Geneva Phone: +41 22 749 01 11 Email:
[email protected] Website: www.iso.org
Published in Switzerland
ii
© ISO #### – All rights reserved
ISO #####-#:####(X)
Contents Foreword................................................................................................................................................................. v Introduction .......................................................................................................................................................... vi 1
Scope................................................................................................................................................................. 1
2
Normative references ................................................................................................................................. 1
3
Terms and definitions ................................................................................................................................ 1
4
Symbols and abbreviated terms ............................................................................................................. 9
5 General requirements for clinical evaluation ................................................................................... 9 5.1 Purpose of the clinical evaluation.......................................................................................................... 9 5.2 Clinical evaluation process .................................................................................................................... 10 5.3 Competencies required for the clinical evaluation ...................................................................... 12 5.4 Clinical evaluation plan .......................................................................................................................... 13 5.4.1 Purpose of the clinical evaluation plan ...................................................................................... 13 5.4.2 Clinical evaluation plan inputs ...................................................................................................... 13 5.5 Clinical evaluation plan outputs .......................................................................................................... 14 6 Determination of the state of the art (SOTA) and benchmarks ............................................... 15 6.1 Purpose of the SOTA analysis ............................................................................................................... 15 6.2 Process for establishing the SOTA ...................................................................................................... 16 6.2.1 Overall process description ............................................................................................................ 16 6.2.2 Scope of the SOTA review ................................................................................................................ 16 6.3 Data collection for evaluation of the SOTA ...................................................................................... 17 6.3.1 Sources of evidence for establishing the SOTA ........................................................................ 17 6.3.2 Limitations of potential SOTA data sources .............................................................................. 17 6.4 SOTA data selection.................................................................................................................................. 18 6.4.1 General ................................................................................................................................................... 18 6.4.2 SOTA relevance criteria.................................................................................................................... 18 6.4.3 SOTA quality criteria ......................................................................................................................... 19 6.4.4 SOTA data analysis ............................................................................................................................. 19 6.4.5 SOTA conclusions................................................................................................................................ 19 7 7.1 7.2 7.3 7.4 7.5
Equivalence ................................................................................................................................................. 19 General principles of equivalence for medical devices............................................................... 19 Criteria for clinical equivalence .......................................................................................................... 19 Criteria for technical equivalence....................................................................................................... 19 Criteria for biological equivalence ..................................................................................................... 20 Justification and documentation of equivalence ........................................................................... 20
8 Data collection ........................................................................................................................................... 20 8.1 Sources of data ........................................................................................................................................... 20 8.1.1 Clinical investigation data ............................................................................................................... 21 8.1.2 Literature search data ...................................................................................................................... 21 8.1.3 PMCF Studies ........................................................................................................................................ 22 8.1.4 Registry data ........................................................................................................................................ 22 8.1.5 Other forms of real-world evidence............................................................................................. 22 9 Data appraisal ............................................................................................................................................ 22 9.1 Literature data ........................................................................................................................................... 22 9.2 Registry data ............................................................................................................................................... 24 10 Data analysis ............................................................................................................................................... 25 10.1 Purpose of data analysis .................................................................................................................. 25 © ISO #### – All rights reserved
iii
ISO #####-#:####(X)
10.2
Data analysis methods.......................................................................................................................25
11 Clinical evaluation report (CER) ..........................................................................................................25 Annex A (informative) Annex title e.g. Example of a figure and a table .......................................27 A.1 Suggested template for a clinical evaluation plan .........................................................................27 A.2 Suggested content for a clinical evaluation report .......................................................................28 A.2.1 Executive Summary ............................................................................................................................28 A.2.2 Scope and objectives ..........................................................................................................................28 A.2.3 Clinical context and state of the art ..............................................................................................28 A.2.4 Device under evaluation ...................................................................................................................28 A.2.5 Clinical evidence summary and appraisal .................................................................................28 A.2.6 Analysis of clinical evidence............................................................................................................28 A.2.7 Conclusions ...........................................................................................................................................28 A.3 Flowcharts for consideration:...............................................................................................................31 A.4 Data sources ................................................................................................................................................34 A.5 Literature search methods.....................................................................................................................35 A.6 Data appraisal.............................................................................................................................................36 A.6.1 Data appraisal criteria for determination of the SOTA .........................................................36 A.6.2 Data appraisal criteria for the device under evaluation.......................................................36 A.7 Data analysis ...............................................................................................................................................37 A.8 Equivalence..................................................................................................................................................38 Bibliography ........................................................................................................................................................39
iv
© ISO #### – All rights reserved
ISO #####-#:####(X)
Foreword ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any patent rights identified during the development of the document will be in the Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 194 Biological and clinical evaluation of medical devices. This is the first edition of this standard.
A list of all parts in the ISO ##### series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html.
© ISO #### – All rights reserved
v
ISO #####-#:####(X)
Introduction This document provides guidance and requirements for the clinical evaluation of medical devices as a part of an overall risk management process. Clinical evaluation provides for the verification and validation of the safety, performance and benefit-risk of a medical device, using clinical data obtained from the use of the medical device. It is the process of: — gathering, through systematic literature review and post-market surveillance, and — where appropriate, generating through clinical investigation,
sufficient clinical evidence to confirm that a medical device meets specified requirements for safety, performance, and benefit-risk in its intended use over its intended lifetime. Safety and clinical performance requirements are specified in relation to the state of the art for other available treatment options for the same patient population and indication or intended use, and can be subject to national or regional regulations.
The requirements contained in this document provide manufacturers with a systematic framework within which clinical experience with medical devices forms the basis for judgments on safety, performance and benefit-risk. The primary aims of this document are to provide assurance of the performance and clinical benefit of medical devices and to protect humans from risks arising from their use. The clinical evaluation process thus ensures that medical devices are associated with a positive benefit/risk conclusion.
vi
© ISO #### – All rights reserved
ISO #####-#:####(X)
Clinical evaluation of medical devices 1 Scope This document specifies terminology, principles and a process for the clinical evaluation of medical devices. The process described in this document aims to assist manufacturers of medical devices to estimate the clinical risks associated with a medical device and evaluate the acceptability of those risks in the light of the clinical benefits achieved when the device is used as intended. The requirements of this document are applicable throughout the life cycle of a medical device. The process described in this document applies to the assessment of risks and benefits from clinical data obtained from the use of medical devices in humans. This document specifies general requirements intended to
— verify the safety of medical devices when used in accordance with their instructions for use; — verify that the clinical performance or effectiveness of a medical device meet the claims of the manufacturer in relation to its intended use; — verify that there is sufficient clinical evidence to demonstrate the achievement of a positive benefit/risk balance when a medical device is used in the intended patient population in accordance with its intended use; — ensure the scientific conduct of a clinical evaluation and the credibility of conclusions drawn on the safety and performance of a medical device; — define the responsibilities of the manufacturer and those conducting or contributing to a clinical evaluation; and — assist manufacturers, clinicians, regulatory authorities and other bodies involved in the conformity assessment of medical devices.
Note 1 This standard can be used for regulatory purposes.
Note 2 This document does not apply to in vitro diagnostic medical devices. However, there may be situations, dependent on the device and national or regional requirements, where sections and/or requirements of this document might be applicable.
2 Normative references The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO 14971, Medical devices — Application of risk management to medical devices
ISO 14155, Clinical investigation of medical devices for human subjects — Good clinical practice
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
© ISO #### – All rights reserved
1
ISO #####-#:####(X)
— ISO Online browsing platform: available at https://www.iso.org/obp — IEC Electropedia: available at https://www.electropedia.org/ 3.1 adverse device effect ADE adverse event related to the use of a medical device
Note 1 to entry: This definition includes adverse events resulting from insufficient or inadequate instructions for use, deployment, implantation, installation, or operation, or any malfunction of the medical device. Note 2 to entry: This definition includes any event resulting from use error or from intentional misuse of the medical device.
[SOURCE: …]
3.2 adverse event AE untoward medical occurrence, unintended disease or injury, or untoward clinical signs (including abnormal laboratory findings) in subjects, users or other persons, whether or not related to the medical device and whether anticipated or unanticipated. Note 1 to entry: This definition includes events related to the medical device or the comparator.
Note 2 to entry: This definition includes events related to the procedures involved.
Note 3 to entry: For users or other persons, this definition is restricted to events related to the use of medical devices.
[SOURCE: …]
3.3 benchmark point of reference that serves as a basis for evaluation or comparison
Note 1 to entry: Benchmarks are used to determine acceptable outcomes for safety, performance and clinical benefit, based on outcomes achievable with other state of the art / standard of care treatment options intended for the same treatment indication and patient population.
Note 2 to entry: Benchmarks for specific clinical outcomes are measurable and preferably quantitative, but may be expressed as a range with confidence intervals
[SOURCE: …]
3.4 bias intentional or unintentional adjustment in the design, conduct of a scientific study, or the collection, analysis and evaluation of data, that may affect the conclusions drawn Note 1 to entry: Bias can occur at any phase of research, e.g. during trial design, data collection, data analysis or publication
[SOURCE: …]
3.5 claim assertions made by the manufacturer pertaining to the features and / or benefits of a device, for example in the information for use, in the literature or in promotional material [SOURCE: …] 2
© ISO #### – All rights reserved
ISO #####-#:####(X)
3.6 clinical benefit positive impact of a device on the health of an individual, expressed in terms of a meaningful, measurable, patient-relevant clinical outcome(s), including outcome(s) related to diagnosis, or a positive impact on patient management or public health. Note 1 to entry: clinical benefits may be either direct or indirect; for example devices which assist other medical devices in achieving their intended purpose, without having a direct therapeutic or diagnostic function themselves.
[SOURCE: …]
3.7 Clinical data text of the definition information related to safety, performance or clinical benefit generated from the clinical use of a device. Note 1 to entry: Clinical data are sourced from:
— clinical investigation(s) of the device concerned; or
— clinical investigation(s) or other studies reported in the scientific literature, of an equivalent device; or
— published and/or unpublished reports on other clinical experience of either the device in question or an equivalent device; or
— postmarket data which can be collated and interpreted to reach conclusions regarding, safety, performance, or clinical benefit.
[SOURCE: …]
3.8 clinical development plan document that states the rationale, objectives, and outline plans for the stages of clinical development of a medical device. Note 1 to entry: Clinical development stages are described in Annex I of ISO 14155:2020.
[SOURCE: …]
3.9 clinical evaluation methodologically sound ongoing procedure to collect, appraise and analyse clinical evidence to determine whether a medical device meets specified requirements for safety, performance and benefitrisk in its intended use over its intended lifetime. [SOURCE: …]
3.10 clinical evaluation plan CEP document that states the rationale, objectives and clinical data requirements for a clinical evaluation [SOURCE: …]
3.11 clinical evidence clinical data and clinical evaluation report pertaining to a medical device [SOURCE: …]
© ISO #### – All rights reserved
3
ISO #####-#:####(X)
3.12 clinical investigation clinical trial, clinical study systematic investigation in one or more human subjects, undertaken to assess the clinical performance, effectiveness or safety of a medical device [SOURCE: …]
3.13 clinical investigation plan CIP, protocol document that states the rationale, objectives, design and pre-specified analysis, methodology, organization, monitoring, conduct and record-keeping of a clinical investigation [SOURCE: …]
3.14 clinical outcome parameter result of a clinical intervention that can be used to determine whether or not specified requirements for safety, performance or effectiveness have been achieved [SOURCE: …]
3.15 clinical performance behaviour of a medical device and response of the patient(s) to that medical device in relation to its intended use, when used in accordance with the manufacturer’s instructions and clinical standard of care. Note 1 to entry: Clinical performance can be defined under national regulations.
[SOURCE: …]
3.16 clinical safety freedom from unacceptable risk to patients, clinicians or other end users
Note 1 to entry: There can be no absolute safety: some risk will remain, which is defined as residual risk. Therefore, a medical device can only be relatively safe. Safety is achieved by reducing risk to an acceptable level. Acceptable risk is determined by the search for an optimal balance between the ideal of absolute safety, the risk control achievable for the medical device, and factors such as intended clinical benefit, outcomes achievable and the risks and benefits of other available therapeutic options for the same intended patient population and treatment indication (or, for devices without a direct clinical benefit, other options for the same intended purpose).
[SOURCE: …]
3.17 comparator medical device, therapy (e.g. active treatment, normal clinical practice), placebo or no treatment, against which the subject medical device can be compared [SOURCE: …]
3.18 effectiveness clinical effectiveness measure of how beneficial a test or treatment is under usual or everyday conditions, compared with doing nothing or opting for another type of care
Note to entry 1: "effectiveness" is not synonymous with "efficacy", which is a measure of how beneficial a test, treatment or public health intervention is under ideal conditions, and is normally correlated to a dose-response curve for a medicinal substance.
4
© ISO #### – All rights reserved
ISO #####-#:####(X)
Note to entry 2: effectiveness can be measured in terms of achievement of a clinically significant intended result in a defined portion of the target population when the medical device is used within its intended purpose and according to its instructions for use, as determined by documented scientific evidence
[SOURCE: …]
3.19 effect size size of treatment effect The observed association between interventions and outcomes, or a statistic to summarise the strength of the observed association [SOURCE: …]
3.20 endpoint principal indicator(s) used for assessing the clinical performance, effectiveness, benefit or safety in a clinical investigation [SOURCE: …]
3.21 equivalent device device for which equivalence to the subject device can be demonstrated [SOURCE: …]
3.22 indication indication for use clinical condition to be diagnosed, prevented, monitored, treated, alleviated, compensated for, replaced, modified or controlled by the medical device Note 1 to entry: indication is distinguished from ‘intended purpose/intended use’, which describes the effect of a device. All devices have an intended purpose/intended use, but not all devices have an indication (e.g. medical devices with an intended purpose of disinfection or sterilisation of devices).
[SOURCE: …]
3.23 instructions for use information for use information provided by the manufacturer to inform the user of a device's intended purpose and proper use and of any precautions to be taken [SOURCE: …]
3.24 intended purpose intended use use for which a device is intended according to the data supplied by the manufacturer in the instructions for use; the intended purpose includes, if applicable, the indications for use, including specific treatment indications, patient populations, contraindications and duration of use [SOURCE: …]
3.25 medical device instrument, apparatus, implement, machine, appliance, implant, reagent for in vitro use, software, material or other similar or related article, intended by the manufacturer to be used, alone or in combination, for human beings, for one or more of the specific purpose(s) of: © ISO #### – All rights reserved
5
ISO #####-#:####(X)
—
diagnosis, prevention, monitoring, treatment or alleviation of disease;
—
investigation, replacement, modification, or support of the anatomy or of a physiological process;
—
diagnosis, monitoring, treatment, alleviation of or compensation for an injury;
—
supporting or sustaining life;
—
disinfection of medical devices;
—
control of conception;
providing information by means of in vitro examination of specimens derived from the human body; and does not achieve its primary intended action by pharmacological, immunological or metabolic means, in or on the human body, but which may be assisted in its intended function by such means —
Note 1 to entry: Products which may be considered to be medical devices in some jurisdictions but not in others include: −
− − −
[SOURCE: …]
disinfection substances;
aids for persons with disabilities;
devices incorporating animal and/or human tissues;
devices for in vitro fertilization or assisted reproduction technologies.
3.26 meta-analysis quantitative statistical analysis of separate but similar experiments or studies in order to reach a single estimate with respect to an outcome measure across these studies, and to test the pooled data for statistical significance Note 1 to entry: meta-analyses should include an evaluation of the combinability of the included studies. Factors affecting combinability of studies in clinical research include study design, similarity of patient population and treatment indications, similarity of treatment evaluated, equivalence of outcomes evaluated and follow up, similarity of data analysis and statistical techniques employed, and statistical validity of the results of each individual study
Note 2 to entry: frequent sources of bias in meta-analyses include reporting bias, search bias and inclusion bias. These can have a significant negative impact on the validity of the conclusions of a meta-analysis, and should be considered as part of the overall evaluation.
[SOURCE: …]
3.27 post market clinical follow up PMCF continuous proactive process to collect and evaluate clinical data from the clinical use of medical devices in a given regulatory region, that have been legally placed on the market in that region Note 1 to entry: “post market” refers to the market in which the clinical data is being collected. For example, if a device has regulatory clearance in the US but not the EU, studies or other data collection undertaken in the US are considered “post market” and those undertaken in the EU are considered “pre-market”
6
© ISO #### – All rights reserved
ISO #####-#:####(X)
3.28 post market clinical follow up study PMCF study study carried out in a given regulatory region following marketing approval in that region intended to answer specific questions relating to clinical safety (i.e. residual risks) or performance of a medical device when used in accordance with its approved labelling
Note 1 to entry: These can examine issues such as long-term performance, the appearance of clinical events (such as delayed hypersensitivity reactions or thrombosis), events specific to defined patient populations, or the performance of the medical device in a more representative population of providers and patients.
3.29 post market surveillance systematic process to collect and analyse experience gained from medical devices in a given regulatory region that have been placed on the market in that region 3.30 registry device registry, product registry organised system that uses observational study methods to collect defined clinical data under normal conditions of use relating to one or more devices to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical or policy purpose(s). Note 1 to entry: A registry evaluates meaningful outcomes and comprehensively covers the population defined by exposure to particular device(s) at a reasonably generalizable scale (e.g. international, national, regional, and health system)
3.31 real world data RWD data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. Note 1 to entry: RWD can come from a number of sources, for example: — — — — —
Electronic health records (EHRs); Claims and billing activities; Product and disease registries; Patient-generated data including in home-use settings; Data gathered from other sources that can inform on health status, such as mobile devices.
3.32 real world evidence RWE clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD Note 1 to entry: RWE can be generated by different study designs or analyses, including but not limited to, randomized trials, including large simple trials, pragmatic trials, and observational studies (prospective and/or retrospective).
3.33 residual risk risk remaining after risk control measures have been implemented
© ISO #### – All rights reserved
7
ISO #####-#:####(X)
3.34 risk control process in which decisions are made and measures implemented by which risks are reduced to, or maintained within, specified levels Note 1 to entry: The hierarchy of risk control measures is, in order of application: — Inherent safety by design; — Protective measures in the device or its manufacture; — Information for safety, such as warnings and contraindications.
3.35 serious adverse event SAE adverse event that led to any of the following: a.
b.
death,
serious deterioration in the health of the subject, users or other persons as defined by one or more of the following: 1)
a life-threatening illness or injury, or
2)
a permanent impairment of a body structure or a body function including chronic diseases, or
4)
medical or surgical intervention to prevent life-threatening illness or injury or permanent impairment to a body structure or a body function,
3)
5)
in-patient or prolonged hospitalization, or
foetal distress, foetal death or a congenital abnormality or birth defect including physical or mental impairment
Note 1 to entry: Planned hospitalization for a pre-existing condition, or a procedure required by the CIP, without serious deterioration in health, is not considered a serious adverse event.
3.36 similar device set of devices having the same or similar intended purposes or a commonality of technology allowing them to be classified in a generic manner not reflecting specific characteristics 3.37 state of the art SOTA developed stage of technical capability at a given time as regards products, processes and services, based on the relevant consolidated findings of science, technology and experience
Note 1 to entry: The state of the art embodies what is currently and generally accepted as good practice in technology and medicine. The state of the art does not necessarily imply the most technologically advanced solution. The state of the art described here is sometimes referred to as the “generally acknowledged state of the art”.
Note 2 to entry: in the context of the clinical evaluation of medical devices, ‘state of the art’ may be considered synonymous with ‘standard of care’
3.38 subject device medical device for which the clinical evaluation has been or will be undertaken
8
© ISO #### – All rights reserved
ISO #####-#:####(X)
3.39 sufficient clinical evidence amount and quality of clinical evidence to guarantee the scientific validity of the conclusions.
4 Symbols and abbreviated terms
The abbreviated terms given in Table 1 are used in this document.
Table 1 — Methodology abbreviations (To be populated as standard develops)
Abbreviated term
Long term
CDP
Clinical development plan
CER
Clinical evaluation report
CEP
Clinical evaluation plan
DUE
IMDRF PMCF PMS
Device under evaluation
International medical device regulators’ forum Postmarket clinical follow up Postmarket surveillance
SOTA
State of the art
5 General requirements for clinical evaluation 5.1 Purpose of the clinical evaluation The purpose of the clinical evaluation is to provide assurance of the clinical effectiveness of medical devices and to protect humans from risks arising from their use, thereby ensuring devices are not placed on the market unless an acceptable benefit/risk conclusion can be demonstrated. Clinical evaluation is an essential part of the risk management process and comprises the analysis and assessment of clinical risks and benefits. Clinical evaluation documentation, including plans and reports, shall therefore be an input to the risk management process, and cross-referenced in the risk management file.
The objective of the clinical evaluation is to confirm that the subject device meets specified requirements for safety, performance and benefit/risk ratio over its intended lifetime. Requirements shall be specified in relation to other available treatment options for the same patient population and treatment indication (or, where a device does not have an indication, intended purpose).
The clinical evaluation shall aim to ensure that sufficient clinical evidence is presented to address all of the following, where relevant: — all device variants and combinations;
Note depending on novelty, intended purpose and residual risk, clinical evidence for some variants or combinations may be based on extrapolation of conclusions for other devices within the scope of the clinical evaluation. IN these cases a rationale and justification for such extrapolations is provided within the CER, to ensure that the basis upon which conclusions on clinical safety, performance, effectiveness, and benefit is unambiguous
— all intended purposes, including specifics of intended patient populations, and treatment indications; — intended user and conditions of use; — device lifetime in use, including potential medium to long term impacts of transient procedures (where the device has a specific impact on the procedure); © ISO #### – All rights reserved
9
ISO #####-#:####(X)
— risks and benefits evident from clinical data; — risks identified through design risk analysis and review of the state of the art; — clinical aspects of compatibility with accessories and related instrumentation.
The clinical evaluation shall include an assessment of all available evidence relevant to the clinical safety, performance, effectiveness and benefit of the device against specified clinical outcome parameters and benchmarks, and reach conclusions on the safety, performance and benefit/risk balance of the subject device. Benchmarks shall be established based on outcomes or performances achievable with alternative treatment options (the “state of the art”). The clinical evaluation shall provide a basis for the determination of additional clinical data requirements, for example from post market surveillance, including requirements for post market clinical follow up.
5.2 Clinical evaluation process
The clinical evaluation shall follow a defined, documented and methodologically sound process. The first step in this process is the development of the clinical evaluation plan. In line with the clinical evaluation plan, methodologically sound procedures shall be employed to collect, appraise and analyse clinical data relevant to the subject medical device and to determine whether there is sufficient clinical evidence to confirm compliance with the specified requirements. Clinical evaluation is a continuous process; the clinical evaluation report and associated documentation shall be updated on a regular basis with relevant data from post market surveillance and when information becomes available suggesting an appreciable change in the clinical benefit/risk balance. Inputs into the clinical evaluation shall include, as appropriate: — — — — —
Risk management report; Clinical evaluation plan; Literature review; Clinical investigation report(s); and Postmarket surveillance reports, including post market clinical follow up reports.
— — — —
Risk management report; Clinical investigation plan(s); Postmarket surveillance plans, including requirements for post market clinical follow up; Labelling, including information for use.
The clinical evaluation shall provide inputs into the following documents, as appropriate:
The manufacturer shall establish, implement, document and maintain an ongoing process which shall include: a. Development of the clinical evaluation plan based on device design, intended purpose, expected clinical benefits, intended patient population, treatment indications, and conditions and duration of use. This includes: — Identification of hazards and assessment of risks associated with the device which could impact device safety in use; — Definition of clinical safety endpoints and outcome measures relevant to the demonstration of achievement of the intended purpose, clinical benefit and device safety; — Device maturity or life cycle stage, development history or previously marketed similar devices. b. Identification of other state of the art or standard of care treatment options appropriate for use in the same indications and patient populations, to establish the state of the art and appropriate benchmarks for the identified clinical outcome measures and safety endpoints c. Collection or generation of clinical data and other evidence relevant to the demonstration of clinical safety, performance and benefit-risk assessment.
10
© ISO #### – All rights reserved
ISO #####-#:####(X)
d. Appraisal of the quality of the assembled clinical data and other evidence, to inform decisions on inclusion and to determine the strength of evidence for each outcome parameter, indication, patient population, device variants and combinations, over the device lifetime or duration of intended clinical benefit or identified clinical risks. e. Analysis of the clinical data and other evidence in relation to the specified performances, benefits, risks and outcome measures identified in step a, against the benchmarks established in step b, such that conclusions on safety, performance, clinical benefit and residual risks can be drawn. f. Identification of evidence insufficiency or data weaknesses, such that requirements for reduction of indications, additional pre-market clinical investigations or specific post market data collection measures can be determined. g. Development of the post market surveillance plan (including requirements for post market clinical follow up) based on the analysis of the clinical evidence in steps e and f. h. Implementation of the post market surveillance plan. i. Integration of the outputs of post market surveillance with the clinical evaluation process (including re-evaluation of the state of the art for benchmarks for safety and performance), the risk management report, post market surveillance plans and other associated clinical documentation.
Although clinical evaluation is a cyclical process, interdependencies between steps can create feedback loops within the cycle. Examples of such interdependencies include: — outputs from the SOTA evaluation can indicate that revisions to the risk management plan or the clinical evaluation plan are required prior to the collection of further clinical data; — if no relevant data are identified at the data collection stage, a re-evaluation of the clinical evaluation plan or clinical development plan can be necessary; — following data appraisal, it can be determined that there are insufficient clinical data meeting the inclusion criteria to feed into data analysis which can then prompt a revision of the clinical evaluation plan or postmarket surveillance plan; — conclusions from the clinical evaluation can indicate a revision to the clinical evaluation plan, clinical development plan, or risk analysis, or to the introduction of risk control measures such as a change to the indications for use, recorded in related risk management documentation.
© ISO #### – All rights reserved
11
ISO #####-#:####(X)
Figure 1 — Interdependencies and potential feedback loops within the clinical evaluation process Competencies required for the clinical evaluation
5.3 Competencies required for the clinical evaluation Top management shall provide evidence of its commitment to the clinical evaluation process by ensuring: — the provision of adequate resources; and — the assignment of competent personnel.
Persons performing clinical evaluation tasks shall be competent on the basis of education, training, skills and experience appropriate to the tasks assigned to them. Where appropriate, these persons shall have knowledge of and experience of: — research methodology (including clinical investigation design and biostatistics); — information management (e.g. qualification as an academic librarian; experience with relevant databases such as Embase and Medline); — regulatory requirements; — medical writing (e.g. post-graduate experience in a relevant science or in medicine; training and experience in medical writing, systematic review and clinical data appraisal) — knowledge of relevant device technology and its application; — knowledge of the diagnosis and management of the conditions intended to be diagnosed or managed by the device, including medical alternatives, treatment standards and technology (e.g. specialist clinical expertise in the relevant medical specialty).
Competencies required for particular parts of the clinical evaluation process are indicated in Table 2 Table 2 — Competency requirements for clinical evaluation
Clinical evaluation planning 12
regulatory
clinical
device
information Study management design
Medical biostatistics writing
© ISO #### – All rights reserved
ISO #####-#:####(X)
Data collection and literature review
Data appraisal Data analysis
Validation of conclusions Clinical investigation design
Note to entry 1: Clinical evaluation tasks can be performed by representatives of several functions, each contributing their specialist knowledge.
5.4 Clinical evaluation plan
Clinical evaluation activities shall be planned. The manufacturer shall establish and document a clinical evaluation plan (CEP) for the subject medical device in accordance with the clinical evaluation process (see 5.2).
5.4.1
Purpose of the clinical evaluation plan
The purpose of the CEP is to establish the requirements for demonstration of safety, performance and clinical benefit for a device over its intended lifetime. It thus sets out the strategy for obtaining sufficient clinical evidence in relation to the objectives of the clinical evaluation. What constitutes sufficient clinical evidence is determined by a critical appraisal of the existing clinical data that identifies the applicable clinical endpoints and timescales. Where necessary, a clinical investigation plan or clinical development plan shall be devised, based on a gap analysis that characterises the difference between the available data and what is deemed to constitute sufficient clinical evidence.
The CEP shall be reviewed at specified intervals over the lifecycle of the device and in response to changes in the clinical risk assessment. If the plan changes during the life cycle of the medical device, a record of the changes shall be maintained.
5.4.2
Clinical evaluation plan inputs
The CEP shall document, as a minimum, the following inputs:
1. identification and description of the subject medical device, including key design features, materials, how it is used, mechanism of action, and interactions with associated accessories, its intended purpose, including specification of indications, contra-indications, target patient population and intended users as appropriate 2. Device lifetime, including, where applicable: i. Duration of device performance in use ii. Duration of associated clinical benefit iii. Period over which risks associated with use of the device can be expected to persist after use
© ISO #### – All rights reserved
13
ISO #####-#:####(X)
3. A specification of the intended clinical benefits of the device. There should be sufficient definition to enable a mapping of anticipated clinical benefits to the relevant clinical outcome parameters.
Where a device has only indirect clinical benefits (for example, where it is used as an accessory to enable another device to achieve its intended purpose) there should be a consideration of whether clinical outcome parameters (e.g. associated with the primary device for an accessory) are required to demonstrate clinical performance, or an explanation as to why intended performance can be demonstrated through non-clinical evidence
4. Claims made for the device (for example in marketing literature) 5. Clinically-relevant outputs from the risk management process, including relevant outputs from safety engineering activities, existing clinical data and post market surveillance. 6. Indicative list of clinical endpoints (i.e. clinical output parameters) which could be used to provide clinical data to demonstrate achievement of the associated safety, performance and benefit parameters. Note A description of these endpoints does not mean that all must feature in the clinical evaluation.
7. Where applicable, a description of in vitro or other testing which could be used in lieu of or to supplement clinical data. 8. A specification of other state of the art diagnostic or treatment options available for the same indications (or, where not applicable, same intended purposes) and patient populations 9. A description of the device regulatory history, including significant changes to design or intended purpose arising from complaints and vigilance 10. A summary of or reference to any existing review of clinical data relevant to the demonstration of the safety, performance and clinical benefit of the device 11. A summary of the clinical development plan and the current status of the device within this plan. The clinical development plan should describe the progression from exploratory investigations, feasibility and pilot studies, to confirmatory investigations, such as pivotal clinical investigations, and PMCF.
5.5 Clinical evaluation plan outputs
Based on the above inputs, the clinical evaluation plan shall describe:
a. The scope of the planned clinical evaluation activities, including: i. characterisation of the key features of the subject device, with respect to clinical performance and utility ii. the applicable regulatory framework(s) and the specific regulatory requirements for which conformity is to be demonstrated b. The objectives of the clinical evaluation, including identification of: i. the specific treatment or diagnostic indications, patient populations and anatomical locations for which safety, performance and clinical benefit are to be demonstrated ii. the context of the evaluation in terms of clinical need and the state of the art iii. the specified requirements for safety, performance, clinical benefit and benefit/risk ratio that are to be demonstrated iv. the specified claims to be demonstrated v. any other requirements specific to particular regulatory regions c. Assignment of responsibilities and authorities d. the methods by which a critical analysis of the clinical data will be performed in relation to identified risks and benefits, based on an appropriate level of evidence and taking into account
14
© ISO #### – All rights reserved
ISO #####-#:####(X)
e. f.
g.
h. i.
j.
k. l.
the alternative treatment options available, in line with the need to provide sufficient clinical evidence the way the evaluation will be used for confirmation of safety, performance, clinical benefit and determination of residual risk and benefit/risk ratio The data sources that will be used to demonstrate clinical performance and clinical benefit, including: i. pre-market clinical investigations undertaken on the subject device or an equivalent device ii. data published in the scientific literature for the subject device or an equivalent device iii. post-market data relevant to clinical performance or benefit outcomes (for example, arising from PMCF) iv. data from national registries v. other suitably justified sources of evidence, including non-clinical evidence The data sources that will be used to evaluate device safety, including: i. Complaints and trending arising from PMS ii. Adverse events reports arising from pre-market clinical investigations or PMCF studies iii. Adverse events reported in the scientific literature for the subject device or similar devices iv. Data arising from registries or adverse event databases The means by which the data sources will be appraised for quality The means by which benchmarks for clinical and other outcomes will be determined, for example, by use of professional society guidelines, evaluation of state of the art through the literature, or guidance published by the target regulatory region(s), including: i. pre-market clinical investigations on similar devices ii. data published in the scientific literature for similar devices (if applicable) and alternative treatments specification of methods to be used for examination of qualitative and quantitative aspects of: i. clinical safety, with clear reference to the determination of residual risks and sideeffects ii. clinical performance iii. clinical benefit iv. claims Conclusions on overall benefit-risk in relation to the state of the art in medicine. Requirements for the review of the clinical evaluation, including frequency, competencies and inputs
6 Determination of the state of the art (SOTA) and benchmarks 6.1 Purpose of the SOTA analysis
The purpose of the SOTA analysis is to establish specific and measurable outcome parameters, and to set benchmarks for these outcome parameters against which the safety, performance and clinical benefit of the subject device can be evaluated.
There can be multiple means of demonstrating that an individual safety, performance or clinical benefit objective has been met. The purpose of the SOTA analysis is to determine, for each safety, performance or clinical benefit objective: • •
which endpoints or methods can be used for the demonstration of conformity with specified requirements what values or outcomes for these endpoints or methods are considered acceptable in relation to the SOTA
© ISO #### – All rights reserved
15
ISO #####-#:####(X)
6.2 Process for establishing the SOTA 6.2.1
Overall process description
The determination of the SOTA shall follow a defined, documented and methodologically sound process.
The process for establishing the SOTA includes:
6.2.2
— Definition of scope (see 6.2.2); — Data collection, including evaluation of potential data sources and justification for those included and excluded; — Data appraisal and weighting; — Data analysis, including, if applicable, discussion of the statistical or other methods used to justify benchmarks for clinical safety, performance and benefit endpoints; — Conclusions on the appropriate sources of evidence and their associated benchmarks for the demonstration of safety, performance and clinical benefit of the device under evaluation (DUE). Scope of the SOTA review
The scope of the SOTA review shall include:
— the devices and therapeutic treatment options to be evaluated, in the light of the intended purpose of the device and (where applicable) clinical indications; — the intended safety, performance and clinical benefits of the DUE in all relevant patient populations; — the devices and other therapeutic options used to identify the outcome measures relevant to the devices intended safety, performance and associated clinical benefits.
The scope of the SOTA review shall be based on the intended purpose of the device. For devices with specific treatment indications, the SOTA review shall cover all relevant treatment options, including medicinal and other non-surgical therapeutic options, alternative surgical interventions, alternative devices, similar and equivalent devices, where applicable.
For devices without specific treatment indications or direct clinical benefits, the SOTA review shall cover all therapeutic options with the same intended purpose, including intended patient population if applicable.
In some circumstances it can be acceptable to consider only devices within the same generic device group, for example: a. where there are no other suitable treatment options for the specific patient population, indication, or stage / severity of disease b. where the generic device group is well-established as the standard of care for a given diagnostic or therapeutic indication, and — there is applicable guidance at levels 1-2 (see 6.3.1) specifying requirements for relevant safety, performance and clinical benefit endpoints, and this guidance can be considered current; — the specified endpoints are sufficient to demonstrate the clinical safety, performance and benefit of the subject device; — evaluation of data retrieved at level 6 does not indicate that there are treatment options outside of the generic device group which offer a superior benefit-risk balance for the same indications and patient populations.
Where evaluation of all relevant devices and therapeutic options within the SOTA is considered unfeasible or unnecessary, the selection of included options shall be justified. The justification shall be documented and include: 16
© ISO #### – All rights reserved
ISO #####-#:####(X)
— representativeness of the included therapeutic options to current clinical best practice; — impact assessment and confirmation that restrictions of scope will not result in inappropriately low benchmarks for safety, performance and clinical benefit being established.
6.3 Data collection for evaluation of the SOTA 6.3.1
Sources of evidence for establishing the SOTA
The hierarchy of sources of clinical evidence for establishing the state of the art for safety, performance and clinical benefit is illustrated in Table 3. Table 3 — Hierarchy of clinical evidence sources
Level
Source
1a (highest)
Minimum safety / performance / benefit outcomes specified by regulatory requirements applicable in the target regulatory region
1c
Device-specific international standards which specify requirements for clinical evidence
1b 2a
Guidelines from medical societies within the target regulatory region
Guidelines from medical societies outside the target regulatory region, where justified
2b
Device-specific national standards relevant to the target regulatory region which specify requirements for clinical evidence
3
Systematic reviews of the scientific literature published in the peerreviewed scientific literature
4
Other literature reviews or meta-analyses
5
Other clinical data from which inferences on requirements for clinical safety, performance or benefit can be made (eg clinical opinion, analysis of vigilance database, grey literature, etc)
6
Non-clinical evidence, where justified
Due to limitations associated with potential data sources, the SOTA review shall as a minimum include consideration of all evidence available at levels 1-3. Lower levels of evidence may be included if conclusions drawn from these sources are incomplete. 6.3.2
Limitations of potential SOTA data sources
Account shall be taken of current best practice at the time of publication. Levels 1-2 represent current best practice at the time of their publication. However, the state of the art can evolve more quickly than these documents are updated. Review of the peer-reviewed scientific literature can indicate that a higher benchmark is required than specified by these sources. In addition, professional society guidelines might not address every aspect of safety, performance and clinical benefit required for the demonstration of conformity for a given device. Sources 1-2 should be coupled with any evidence available at level 3, to determine if a higher benchmark is applicable for specific safety, performance and clinical benefit endpoints.
Use of clinical best practice guidelines from medical societies outside the target regulatory region shall be justified. There can be differences in standard of care or patient populations between regions which make the published data unsuitable for setting benchmarks.
Meta-analyses or other forms of data pooling that are not published in the peer reviewed literature shall only be used to justify state of the art benchmarks if there is insufficient evidence at the higher levels. Meta-analyses are prone to bias, and there is significant potential for misinterpretation. If a meta-analysis © ISO #### – All rights reserved
17
ISO #####-#:####(X)
and other forms of data pooling is performed to establish the SOTA, measures taken to avoid bias shall be documented.
The use of non-clinical evidence for setting benchmarks should only be justified for very low risk, established devices, or those with no direct clinical benefit, and where no other relevant guidance at a higher level exists.
6.4 SOTA data selection 6.4.1
General
The selection of relevant publications or other data sources to establish benchmarks for safety, performance and clinical benefit shall be based on the relevance and quality of the sources from which the data are obtained. 6.4.2
6.4.2.1
SOTA relevance criteria
SOTA relevance criteria for devices with indications and direct clinical benefits
Relevance criteria for devices with indications and direct clinical benefits shall include: 1. publication date;
2. similarity of intended purpose, excluding mode of action;
3. completeness of outcome measures evaluated with respect to demonstration of safety, performance and clinical benefit for that intended purpose.
In general, when two or more data sources within levels 1-3 suggest benchmarks for the same outcome measures, patient populations and treatment indications, higher relevance shall be assigned to the most recent publication(s). The hierarchy of relevance with respect to similarity of intended purpose, from highest to lowest, is: 1. Therapeutic options with the same intended patient populations and indications (where applicable); 2. Therapeutic options with narrower intended patient populations and treatment indications, within the scope of the intended purpose of the DUE; 3. Therapeutic options with broader intended patient populations and treatment indications (where applicable) than the DUE; 4. Therapeutic options with different intended patient populations and treatment indications (where applicable) than the DUE.
The hierarchy of relevance with respect to completeness of outcome measures evaluated, from highest to lowest, is: 1. Outcome measures assessed, and values provided, relevant to all aspects of intended purpose of the DUE; 2. Outcome measures assessed, and values provided, relevant to some but not all aspects of the intended purpose of the device under evaluation; 3. Outcome measures relevant to some but not all aspects of the intended purpose of the device under evaluation assessed, but no values provided.
6.4.2.2
SOTA relevance criteria for devices without indications or direct clinical benefits
Relevance criteria for devices without indications and direct clinical benefits shall be based on similarity of intended purpose.
18
© ISO #### – All rights reserved
ISO #####-#:####(X)
6.4.3
SOTA quality criteria
Appraisal and ranking of literature data quality shall include consideration of study design, statistical validity of conclusions, and likelihood of bias. Ranking of other data sources used for the evaluation of the SOTA should follow the hierarchy presented in 6.3.1. 6.4.4
SOTA data analysis
Data retrieved and determined to be of sufficient quality for inclusion in the SOTA evaluation shall be used to determine relevant outcome parameters for each intended purpose, and to determine the values for these outcome parameters which can be considered representative of acceptable safety, performance or clinical benefit in relation to the SOTA treatment or therapeutic options with the same patient population and treatment indication.
Where there are discrepancies between included data sources, these shall be resolved based on consideration of the relevance and quality of the source, based on the criteria described in Sections 6.4.2 and 6.4.3. 6.4.5
SOTA conclusions
Outcome parameters and benchmarks derived from the SOTA review shall be mapped against the intended safety, performance and clinical benefit objectives of the device under evaluation, as defined in the CEP (see 5.5.b). Where the conclusions are considered insufficient to address all relevant aspects of safety, performance and clinical benefit, the next level of evidence shall be considered in the SOTA evaluation. The final set of outcome measures with their quantified benchmarks shall be documented and justified on the basis of quality, relevance, completeness and impact on standard of care.
7 Equivalence
7.1 General principles of equivalence for medical devices Clinical data obtained from the use of a device with which equivalence has been demonstrated can be used to demonstrate the safety, performance or clinical benefit of the device under evaluation.
To demonstrate equivalence between the subject device and a single medical device that achieves the same therapeutic result by the same means, all criteria for clinical, technical and biological equivalence shall be fulfilled.
7.2 Criteria for clinical equivalence
Medical devices can be considered clinically equivalent if they are used:
— for the same clinical condition or purpose;
— including similar severity and stage of disease;
— at the same site in the body;
— in a similar population, including as regards age, anatomy and physiology; — by the same kind of user;
— to achieve a similar relevant critical performance in view of the expected clinical effect for a specific intended purpose.
7.3 Criteria for technical equivalence
Medical devices can be considered technically equivalent if they:
— are of similar design; © ISO #### – All rights reserved
19
ISO #####-#:####(X)
— are used under similar conditions of use;
— have similar specifications and properties including physicochemical properties such as intensity of energy, tensile strength, viscosity, surface characteristics, wavelength or software algorithms; — use similar deployment methods, where relevant;
— have similar principles of operation and critical performance requirements.
7.4 Criteria for biological equivalence
Medical devices can be considered biologically equivalent if:
— the chemical characteristics of their materials are sufficiently similar, such that the composition and processing do not result in additional or different toxicological concerns, taking into account the release characteristics of substances, including degradation products and leachables; — the physical characteristics of their materials are sufficiently similar, such that the configuration, morphology, topography and tribology do not result in additional or different biocompatibility concerns; and
— their intended clinical use is sufficiently similar that the endpoints of biological evaluation identified in ISO 10993-1:2018, Clause A.1 [1] are identical.
Note The principles for establishing biological equivalence are explained in ISO 10993-18:2020, Annex C [2].
7.5 Justification and documentation of equivalence
The term “similar” used in clauses 7.2 to 7.4 shall be taken to mean that no clinically significant difference in the performance, safety or intended clinical benefits of the device would be triggered by the differences between the subject device and the equivalent device. The following information and justifications shall be documented in the Clinical Evaluation Report:
— All differences between the subject device and the equivalent device, including differences in manufacturing processes, surface treatments or modifications of material characteristics, including explanations as to why the differences are not expected to significantly affect the clinical performance and clinical safety of the subject device. — Clinically relevant specifications and measurements, in a comparative tabulation. — Comparative drawings or pictures to allow a comparison of shapes and sizes of elements that are in contact with the body, if relevant. — Reference to supporting non-clinical information (e.g. pre-clinical study reports) that support equivalence. — Confirmation of access to relevant proprietary information, if the equivalent device is made by a different manufacturer.
Note A suggested format for the demonstration of equivalence is included in Annex X.
8 Data collection 8.1 Sources of data Evidence used in the clinical evaluation may be sourced from:
20
— Clinical investigations; — Clinical investigations or other clinical experience published in the scientific literature; — PMCF studies;
© ISO #### – All rights reserved
ISO #####-#:####(X)
— — — —
8.1.1
Device registries; Other real world data, including data derived from surveys and electronic health records; Clinical data obtained from PMS, such as complaint trending and incident reports; Non-clinical evidence (performance only, where appropriately justified on the basis of device intended purpose, mode of action, novelty, and risk). Clinical investigation data
Clinical investigations undertaken by the manufacturer to generate the clinical data needed to demonstrate that a device meets the requirements for safety, performance, or clinical benefit as specified in the CEP shall be planned, conducted and reported in compliance with ISO 14155.
8.1.2
8.1.2.1
Literature search data
Literature search protocol
The purpose of a literature search protocol is to:
a. document research questions relevant to the clinical evaluation, including i.
ii.
and
research questions relevant to the safety, performance and clinical benefit of the device under evaluation; research questions relevant to the evaluation of the SOTA, and in particular outcomes achievable with other relevant diagnostic or therapeutic options;
b. specify the systematic methods and criteria for objective search and review by which the research questions are to be addressed by the literature review.
The literature search protocol shall include the following information:
a. the scope and objectives of the clinical evaluation relevant to the literature search (see Sections 5.2a and b);
b. the research question(s) to be addressed by the literature review; c. the literature search methods;
d. strategy for selection of relevant publications/studies (i.e. inclusion and exclusion criteria); e. appraisal criteria and analysis methods.
Amendments to or deviations from the protocol shall be justified and documented in the literature review report. Confirmation shall be obtained that any such amendments or deviations do not affect the outcome of the research or introduce new biases. Note 1 The literature search protocol has the potential to reduce the impact of writer bias, by providing transparency of methods and processes, and facilitates review of the planned literature review methods.
Note 2 The literature search protocol can be used to clarify the scope of the clinical evaluation and confirm the device description, including indications and claims.
Note 3 Separate literature search protocols may be developed to address SOTA, the device under evaluation, or any specific subsets of these searches.
8.1.2.2
Literature search report
The results of the literature search shall be included in the literature search report and shall contain the following information: a. the scope and objectives of the clinical evaluation (see Sections 5.2 a and b);
© ISO #### – All rights reserved
21
ISO #####-#:####(X)
b. characterisation of the SOTA, including a review of the clinical background, the public health impact of the target disease, identification of similar devices and alternative therapeutic options and current developments in research;
c. a detailed description of the subject device, including indication(s) or intended purpose, mode of action, intended patient population, claims and design innovations;
d. a description of preclinical studies, if relevant; e. literature search methods and results; f.
reasons for rejection of papers identified as relevant by the literature search but not meeting inclusion criteria;
g. a description of the clinical literature relevant to conformity assessment, with an appraisal of evidential value for each data set described: i.
ii.
prospective, randomised controlled trials; prospective, non-randomised trials;
iii. retrospective cohort studies and registry data; iv. case series;
v.
other clinical data, including the results of vigilance and post market surveillance and conclusions of meta-analyses and other systematic reviews;
h. an analysis of data on performance and clinical benefit;
i.
a review of complications and adverse events.
Note 1 to entry: For literature searches undertaken to establish the SOTA, the selection criteria described in 6.4 may be applied in lieu of the data appraisal listed in point g.
8.1.3
PMCF Studies
The PMCF plan is an output of the clinical evaluation and identified PMCF mechanisms represent an important source of clinical evidence in the post market phase. PMCF studies shall be designed to address residual risks identified by the evaluation of the clinical data and documented in the CER. The depth, extent and design of such studies shall be appropriate and proportionate to the risk and novelty of the device. 8.1.4
Registry data
Registry data shall be evaluated and weighted for inclusion in the CER based on quality and relevance. 8.1.5
Other forms of real-world evidence
There is guidance on use of RWD for pharma, but not so much for devices. Could this be developed in a horizontal standard, and specifics of implementation put in the vertical standard? eg, this guidance funded by EFPIA: https://rwe-navigator.eu/use-real-world-evidence/sources-of-realworld-data/ Also recent MHRA guidance.
9 Data appraisal
9.1 Literature data Publications identified by the literature search as directly relevant to the research question shall be appraised according to defined appraisal criteria, to determine a weighting for the contribution of each 22
© ISO #### – All rights reserved
ISO #####-#:####(X)
data set to the clinical evaluation on the basis of the methodological quality and scientific validity of each data set and its relevance to the clinical evaluation. Pivotal data sets (i.e. high quality data obtained from studies using the subject device or an equivalent device in the intended indication) and other highly relevant data sets shall be identified from the results of the appraisal. Papers used only for background information and to determine the SOTA may be appraised using different criteria.
See Table 4 for an example of an appraisal scheme in which the weighting is calculated as the product of the average gradings for relevance and quality. Table 4 — Appraisal Criteria
RELEVANCE Criterion
Question
Grading
Relevance of device
Were the data generated from the device in question?
3 Subject device
Relevance of application
Was the device used for the same intended use (e.g., methods of deployment, application, etc.)?
3 Same use
Relevance of population
Where the data generated from a patient group that is representative of the intended treatment population e.g., age, gender, etc.) and clinical condition (i.e., disease, including state and severity)?
Is the subject population relevant to the proposed intended use and the intended user of the device?
2 Equivalent device
1 Other device or therapy 2 Similar use
1 Different use
3 Same population
2 Similar population
1 Different population
Is the study generalizable and does the study population match the intended use population?
QUALITY (methodological quality and scientific validity) Criterion
Question
Grading
Study design
Was the design of the study appropriate and of high evidential value?
3 RCT or meta-analysis
Do the reports or collations of data contain sufficient information to be able to undertake a rational and objective assessment, taking into account the methodological factors below?
3 High quality
Data quality Study size
Outcome measures
Is the population size sufficient to provide results that are meaningful for the relevant population?
2 Large registry
case
1 Small case series
series,
2 Minor deficiencies
1 Inadequate information
Do the outcome measures reported reflect the intended performance of the device and the intended clinical benefits?
Are the outcome measures aligned with relevant clinical outcomes specified in the manufacturer’s clinical evaluation plan?
© ISO #### – All rights reserved
23
ISO #####-#:####(X)
RELEVANCE Criterion
Question
Follow-up
Is the duration of follow-up long enough to assess all relevant clinical outcomes and identify complications?
Grading
Statistical Significance Has a statistical analysis of the data been provided and is it appropriate? Clinical Significance
9.2 Registry data
Did the statistical analysis link selected safety and performance endpoints to the selection of sample size? Is the magnitude of the treatment effect observed clinically significant?
Include guidelines for validation / evaluation of quality of registry data? Can IMDRF guidance be built upon to make specific recommendations for assessment of data quality for JRI registries? • •
•
IMDRF/Registry WG/N42FINAL:2017 Methodological Principles in the Use of International Medical Device Registry Data http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech170316-methodological-principles.pdf IMDRF/Registry WG/N46 FINAL:2018 Tools for Assessing the Usability of Registries in Support of Regulatory Decision-Making https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2a hUKEwiMlZ6m7b3sAhVvURUIHQLfBH8QFjABegQIBBAC&url=http%3A%2F%2Fwww.imdrf.org%2 Fdocs%2Fimdrf%2Ffinal%2Ftechnical%2Fimdrf-tech-180327-usability-toolsn46.docx&usg=AOvVaw2ofbn7oSGCEabGDHZmFVEk IMDRF/REGISTRY WG/N33FINAL:2016 Principles of International System of Registries Linked to Other Data Sources and Tools http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech160930-principles-system-registries.pdf
Would ranking system based on these parameters from the N33 guidance be sufficient? Some relevant parameters from IMDRF “1. DEVICE: The registry contains sufficient information to uniquely identify the device. Ideally, the unique device identifier would be included, but when the UDI is not available, the registry would include a combination of identifiers (catalogue, number, manufacturer, description) that, in combination, will assist in uniquely identifying the device. 2. QUALITY IMPROVEMENT SYSTEM: The registry is part of a health care delivery quality improvement system or evolving into one as device technologies are diffused into practice and need continuing evaluation (including outlier identification). 3. BENEFICIAL CHANGE: The registry has established mechanisms to bring about beneficial change in health care delivery through stakeholder participation, ownership and integration into the relevant health care systems. 4. EFFICIENCY: The registry is embedded in the health care delivery system so that data collection occurs as part of care delivery (i.e., not overly burdensome, not highly complicated, not overly costly, etc.) and integrated with work flow of clinical teams. 5. ACTIONABLE DATA: The registry provides actionable information in a relevant and timely manner to decision makers. 6. TRANPARENCY: The governance structure, data access, and analytical processes of the registry are transparent 24
© ISO #### – All rights reserved
ISO #####-#:####(X)
7. LINKABILITY: Information in the registry can be linked with other data sources for enhancement including adequate follow up achievement. 8. TOTAL DEVICE LIFE-CYCLE: The registry can serve as infrastructure for seamless integration of evidence throughout the device life cycle.”
10 Data analysis 10.1 Purpose of data analysis 10.2 Data analysis methods This section should define levels of data stratification, eg: -
By device variants
-
Combinations
-
Treatment indications
-
Clinical outcome measures
And how this is mapped to specific requirements. How to determine when data against a certain parameter is “sufficient”? reference to statistical interpretation, p-values, minimum patient numbers, etc? Once the data sets have been selected and appraised, a critical analysis of the data shall be performed to evaluate the safety, performance, and benefit/risk profile. This analysis shall include a review of data on similar devices and alternative therapies to define the state of art and identify applicable hazards, such as those due to relevant technologies.
11 Clinical evaluation report (CER)
The information evaluated shall be summarised and the results of the evaluation documented in the clinical evaluation report. All information contributing to the evaluation shall be included or referenced in the clinical evaluation report such that the evaluation can be understood by an independent party and the methods employed, the relevant data sets, the assumptions made and the conclusions reached are evident. Traceability shall be established between the relevant data sets and the conclusions reached. The clinical evaluation report shall include at least the following: a. identification and description of the subject device (see 4.4.2 a-d.); b. a description of the methodology used for the clinical evaluation, including search and appraisal criteria and any deviations from the clinical evaluation plan; c. a description of the state of the art (see 5); d. a summary of the data sets relevant to the clinical evaluation of the subject device, including a critical appraisal of their contribution to the evaluation (see Annex X); e. analysis of the clinical data including any statistical analysis (see Annex X); f. an assessment of clinical risks and benefits; g. identification of residual risks and uncertainties requiring further evaluation and a plan for their evaluation, including plans for PMCF where relevant; h. conclusions in relation to the scope and objectives of the clinical evaluation (see 4.5 a and b); i. the identity and roles of individuals contributing to the clinical evaluation, including information indicating their qualification by training and experience for the role undertaken and, for individuals not employed by the manufacturer, a declaration of interests.
Note See Annex X for a suggested format for the clinical evaluation report. © ISO #### – All rights reserved
25
ISO #####-#:####(X)
Content for Annexes or possible figures:
26
© ISO #### – All rights reserved
ISO #####-#:####(X)
Annex A (informative)
Annex title e.g. Example of a figure and a table
A.1 Suggested template for a clinical evaluation plan
© ISO #### – All rights reserved
27
ISO #####-#:####(X)
A.2 Suggested content for a clinical evaluation report A.2.1 Executive Summary A.2.2 Scope and objectives A.2.3 Clinical context and state of the art A.2.4 Device under evaluation A.2.5 Clinical evidence summary and appraisal A.2.6 Analysis of clinical evidence A.2.7 Conclusions 1.
2.
Executive Summary Introduction
2.1. Objectives 6
6
2.2. Guidance Documents 2.3. Reference Material 2.4. Abbreviations 3.
7
7 7
Context of the Evaluation 8
3.1. Clinical Background
3.2. Therapeutic Options 3.3. Design Concept
4.
6
9
9 9
Description of the device and its intended application
4.1. Device description 9
4.2. Intended purpose, including indications and claims 4.3. Clinical risks
11
4.5. Comparison with Existing Products
4.6. Assessment of Equivalency
4.7. Design and Manufacture 16 5.
12
Literature Review 16
16
5.3. Summary of the Literature Review
16
28
11
11
5.1. Objectives of the Literature Review
5.2. Literature Review Methodology 16
9
© ISO #### – All rights reserved
ISO #####-#:####(X)
5.4. Conclusions from Literature Review
7.
Clinical Investigation
17
7.1. Clinical Data Requirements
7.2. Study Design and Objectives
17
17
7.3. Clinical Investigation Methodology
7.4. Results of the Clinical Investigation 7.5. Discussion of Clinical Results
16
17
17 17
7.6. Conclusions from the Clinical Investigation 8.
Post Market Surveillance 18
8.1. Post-market Clinical Follow-up
18
18
8.1.1. Study Design and Objectives- Clinical Registry 18 8.1.2. Clinical Investigation Methodology
8.2. Analysis of Complaints data 9.
Risk Assessment 18
18
18
9.1. Analysis of Product-related Risks 18
9.2. Clinical Risks
19
9.4. Clinical Benefits
20
9.3. Residual Risks and Uncertainties 19 9.5. Benefit:Risk Assessment 20
9.6. Conclusions of the Clinical Risk Assessment 10.
11.
Post-market Surveillance Plan 21
Conclusions
20
23
12. Authors’ and Reviewers’ CVs…………………………………………………………….25 CER Content
Executive summary Introduction Objectives
Guidance Documents Reference Material Abbreviations
Clinical Background
Therapeutic Options Design Concept
Device description
Indications and claims; Product Literature Device Classification
© ISO #### – All rights reserved
29
ISO #####-#:####(X)
Comparison with existing products Assessment Equivalency
of
Design and Manufacture Literature Review
Pre-clinical Assessment Clinical requirements
data
Premarket Investigation
Clinical
PMCF PMS
Published clinical data Product-related risks Clinical risks
Residual risks uncertainties Clinical benefits
and
Benefit:risk assessment Conclusions assessment
of
risk
PMS plan (including PMCF plan) Conclusions
30
© ISO #### – All rights reserved
ISO #####-#:####(X)
A.3 Flowcharts for consideration: Literature Review
Identify and analyse clinical data for equivalent products
Explain the clinical background
Characterise clinical need and design solution
Identify therapeutic options
Specify and justify design input
Clinical Evaluation Report
Technical File
Clinical risk assessment
Summarise preclinical design verification and risk analysis (e.g. RMR)
Summarise clinical investigation data (e.g. CSR)
Is clinical evidence sufficient? Summarise clinical data for alternative therapiess
Describe device, indications and claims
Summarise clinical data for predicate devices
Define the state of the art
No
Summarise existing PMS/PMCF data (e.g. PMS Report)
Yes
Define additional data requirements or corrective action (e.g. CEP)
Define and justify PMS requirements (e.g. PMS/PMCF Plan)
Collect required data or take necessary action
Repeat evaluation periodically
Clinical Evaluation Process
© ISO #### – All rights reserved
31
ISO #####-#:####(X)
Define search strategy and undertake search
Literature Review Request Form
Peer review by clinical specialist or other reviewers (if necessary)
Literature Review v1.3
(MEDDEV 2.7/1 A4)
Develop device description
Search output
Manufacturer’s review
(MEDDEV 2.7/1 A3)
Define clinical use and review criteria
Define responsibilities and timescales
Obtain clinical input from/via Medical Director
Literature Review v1.1
Sift from title and abstract
(MEDDEV 2.7/1 A5.1)
Bibliographic, appraisal and data extraction tables
Obtain full text papers
Define Scope (MEDDEV 2.7/1 A5.2)
Protocol (MEDDEV 2.7/1 A5.3)
Sift from full text papers
Appraisal of quality and relevance
Data extraction
(MEDDEV 2.7/1 A6)
Technical, regulatory and editorial review
Data synthesis and interpretation
(MEDDEV 2.7/1 A7, as appli cable)
Define selection criteria and quality assessment criteria (MEDDEV 2.7/1 A6)
Obtain client approval
32
Literature Review v1.2
Literature Review Process
Review budget estimate
© ISO #### – All rights reserved
Literature Review v1.0
ISO #####-#:####(X)
From From Literature Literature Review Review process process or or information information from from client client
Context of evaluation
Clinical need / Design input
State of the art
Design solution / intended clinical intervention
Indications and claims
Document in PowerPoint and/or prepare draft CER
Define Clinical Investigation Plan / study design
Identify key papers
Gap analysis
Characterise: - Clinical context - Patient population - Comparators - Intervention - Clinical outcomes
Identify key clinical endpoints and follow-up timescale
Statistical determination of sample size
Define clinical strategy
CEP Process
© ISO #### – All rights reserved
33
ISO #####-#:####(X)
A.4 Data sources
34
© ISO #### – All rights reserved
ISO #####-#:####(X)
A.5 Literature search methods
© ISO #### – All rights reserved
35
ISO #####-#:####(X)
A.6 Data appraisal A.6.1 Data appraisal criteria for determination of the SOTA A.6.2 Data appraisal criteria for the device under evaluation
36
© ISO #### – All rights reserved
ISO #####-#:####(X)
A.7 Data analysis
© ISO #### – All rights reserved
37
ISO #####-#:####(X)
A.8 Equivalence Suggest include of table from MDCG 2020-5
38
Kommentiert [JT1]: Needs to be adapted to align with criteria in this standard
© ISO #### – All rights reserved
ISO #####-#:####(X)
Bibliography [1] ISO 10993-1:2018, Biological evaluation of medical devices — Part 1: Evaluation and testing within a risk management process
[2] ISO 10992-18:2020, Biological evaluation of medical devices — Part 18: Chemical characterization of medical device materials within a risk management process IMDRF guidance MDCE WG/N56: Clinical Evaluation
IMDRF guidance MDCE WG/N55: Clinical Evidence – Key Definitions and Concepts
IMDRF guidance Registry WG/N42: Methodological Principles in the Use of International
IMDRF guidance Registry WG/N46: Tools for Assessing the Usability of Registries in Support of Regulatory Decision-Making EMA guidance: Wording of therapeutic indication - guide for assessors (europa.eu) FDA guidance on Real World Evidence
© ISO #### – All rights reserved
39
