Good Protocol Design

The Monkey
A former boss once told me this story. When I walk into his office he envisions me with a monkey on my back. As we begin our conversation the monkey jumps off my back and starts to create havoc, knocking papers to the floor, turning over the lamp, swinging from the chandelier, and kicking the waste can. My goal is to leave the office with the monkey on my boss's back; my boss's goal is for me to leave his office with the monkey on my back. When the conversation ends, the person who has something to do has the monkey on their back.

The same analogy can help you to write a protocol. The monkey is consensus on several key points about the study design. If you get consensus from the project team and top management, the monkey is on their back. If you have to start writing the protocol without agreement on these points, the monkey is on your back. With committed consensus, you can plan on 20-30 working days over a duration of 2-3 months to write a new protocol for a new technology. Without consensus it can take years.

Authority
To lay some background, let's look at the regulatory requirements for a clinical investigation. FDA regulations tell us that we shall have valid scientific evidence to support the safety and efficacy of medical devices. Valid scientific evidence [consists of] evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsibly be concluded by qualified experts that there is a reasonable assurance of safety and effectiveness of a device under its conditions of use. So the purpose of a clinical investigation in the States is to provide reasonable assurance of safety and effectiveness for the investigational device. [1. 21 CFR Part 860.7(c)(1-2) (paraphrased)]

For those of you concerned with clinical trials in Europe the path of authority is a little more complex. You'll begin with the Medical Device Directive which states the demonstration of conformity with the essential requirements must include a clinical evaluation in accordance with Annex X. [2. MDD 93-42-EEC I.6a)] A clinical evaluation is the assessment and analysis of [existing] clinical data pertaining to the medical device in order to verify the clinical safety and performance of the device. [3. MEDDEV 2.7.1 r3] If existing clinical data cannot verify safety and performance, then a clinical investigation must be done per ISO 14155 or the intended use reworded. The standard states that the justification for the design of the clinical investigation shall be based on the evaluation of the results of a clinical evaluation. [4. ISO 14155 Clause 5.3] This is the logic behind bringing a medical device to market in the EU via the literature route.

Start at the end
To leave the monkey on the project team's back, here are the key points on which you will want consensus.

Intended use
Intended use is a US regulatory term that refers to what you say in the labeling. It ties the physical device, and the words we use to describe it, into a single entity. [5. 21 CFR 801.4] Intended use is a broad concept—think verbs, function, and action. Consider the following examples:
  [1] A neurostimulator is intended for stimulating the vagus nerve;
  [2] An MRI scanner is intended for soft-tissue imaging using nuclear magnetic resonance;
  [3] A fetal oxygen monitor is intended for monitoring oxygen in a fetus.

The intended use dictates the specialty we need in an investigator, say orthopedist or neurologist, and begins to define the subject population.

Indications for use
Indications for use are the diseases or conditions for which the device will be used. Choose diseases, conditions, or procedures that have accepted medical definitions. The indications for use dictate the subject selection criteria. [6. 21 CFR 814.20(b)(3)(i)] Consider the following examples:
  [1] A neurostimulator indicated for prevention of epileptic seizures;
  [2] An MRI scanner indicated for detecting structural abnormalities;
  [3] A fetal oxygen monitor (hopefully) indicated for determining if a C-section is warranted.

Performance and device deficiencies
Performance speaks to the physical properties and operation of the device; for example, what range of light-waves does it emit, how hot does it get, do the buttons stick, how easy is it to use? Performance also speaks to the ISO concept of device deficiencies such as inadequacies with respect to the devices identity, quality, durability, reliability, or safety. The intended performance or concerns about lack of performance dictate several of the questions that you should include on the case report forms. [7. ISO 14155]

Consider the following examples:
  [1] The neurostimulator delivers xyz amps.
  [2] An MRI scanner has good contrast resolution.
  [3] A fetal oxygen monitor detects oxygen levels in fetal blood.

Safety
Safety refers to the absence of adverse events to subjects and the absence of serious, unanticipated adverse events to caregivers or bystanders. [8. ISO 14155] Recording and reporting of adverse events varies with the host country (the country in which the study is being implemented) and with the study risk. Every protocol requires a "risk assessment". The risk assessment is often written in tabular format with headings of hazard, frequency, severity, risk, mitigation, and residual risk.

A hazard is an adverse event that may occur to a subject, or a serious, unanticipated adverse event that may occur to a caregiver or bystander. Frequency refers to the rate at which the hazard is likely to occur; it can be estimated on any convenient scale such as 1-100%. Severity refers to the intensity of the event, such as mild, moderate, or severe; a three-point scale is useful for estimation. Risk is a function of frequency and severity; you can use any meaningful mathematical relationship, in the example the two scores have been added. Mitigation refers to actions that are taken to lessen or prevent the risk; and because you are writing a protocol you’ll focus on measures that can be taken by the investigator. Residual risk is the amount of risk left over after all mitigation measures are taken. Residual risk can be described as minor, requiring an outpatient procedure, or hospitalization, or by any other meaningful scale. [9. ISO 14155] Clearly the safety questions on the case report forms will come from the risk assessment. In the US, the risk assessment may cause the IRB to see the study as significant risk. Consider the following examples for residual risk:
  [1] Neurostimulalton of the vagus nerve may cause headaches.
  [2] MRI scanner may expose technical staff to NMR.
  [3] Placement of fetal oxygen monitor sensors may injure fetal soft tissue.

Target market
Who is going to influence the buying decision about your device within the hospital or clinic structure? Often this person is a good choice for clinical investigator. They know how to use the device, see its value, and can give meaningful feedback on performance.

Major competitive products or procedures
Controlled, randomized trials are not usually required for medical devices in order to gain clearance in the US, and a comparison to competitors does not enter into the equation in the EU. Nevertheless, it is useful to know what your competitors are claiming, how they word the intended use and indications for use, what clinical trials they have done, what predicate devices they referenced, and the like. If you do decide to do a randomized, controlled study, a thorough knowledge of your competition will help you decide on what product or procedure to use as a control.

Claims
A claim is an assertion of truth about your product. The claims you want to make dictate the endpoints (clinical measurements that indicate device success or failure), outcome measurements, whether a study is comparative or single armed, and many other features of investigation. The team may list several claims they want to make about the device; limit your selection to two or three. If you're a new company go after the easy claims first to get experience in conducting studies, getting FDA clearance, and commercializing a device. If your firm is experienced you may feel comfortable going after more complex, but desirable, claims in the first investigation.

Frequently the project team will settle on the claims to be investigated. And then you'll begin writing the protocol only to find there is no good methodology for measuring success or failure, the sample size makes the study unaffordable, there are too few patients to maintain a reasonable recruiting rate, there is no market for treating such patients, or some other insurmountable problem. You go back to the project team, rework the wording of the proposed claims, and resume writing the protocol. This is the monkey creating havoc.

Protocol Contents
There is no absolute format for a protocol, although FDA issued a guidance document in 1996, [10. FDA Statistical Guidance] and ISO 14155 contains an extensive checklist. Here is a suggested format that is a combination of the two:
  [01] Cover page, Table of contents, Executive summary.
  [02] Device description, photograph, accessories, description of comparator.
  [03] Contention of SR or NSR regulatory status.
  [04] Study objective.
  [05] Hypotheses, primary and secondary.
  [06] Endpoints and procedures, primary and secondary.
  [07] Medical risk analysis.
  [08] Study design.
  [09] Subjects.
  [10] Ethical considerations.
  [11] Sample Size.
  [12] Data collection.
  [13] Control of study devices.
  [14] Adverse event recording and reporting.
  [15] Monitoring plan.
  [16] Documentation.
  [17] Central laboratories.
  [18] Data management.
  [19] Data analysis.
  [20] Data monitoring board, Data safety monitoring board, Clinical events committee, if one will be used.
  [21] Personnel Responsibilities.
  [22] Return of unused inventory.
  [23] Study registration and publication policy.
  [24] Study cancellation.
  [25] Definitions.
  [26] References.
  [27] Attachments.

Internal Approvals
Internal approvals are another great opportunity for the monkey to create havoc. But you can keep him in his cage by using an approval form that identifies and limits each function's responsibilities. For example, the project team leader reviews the protocol to ensure you have described the device accurately and you are investigating the right device; marketing ensures you are investigating the right claims; manufacturing reviews to ensure they can actually make the investigational device; the medical manager assures you have an accurate and complete risk analysis; regulatory assures you have identified the regulatory status of the investigational device correctly; statistics reviews the sample size and data analysis plan; and so forth. Only your manager has the privilege of global review.

References
[1] 21 CFR Part 860.7(c)(1-2) (paraphrased).
[2] MDD 93-42-EEC I.6a).
[3] MEDDEV 2.7.1 r3.
[4] ISO 14155 Clause 5.3.
[5] 21 CFR 801.4.
[6] 21 CFR 814.20(b)(3)(i)
[7] ISO 14155 Clause 3.15.
[8] ISO 14155 Clause 3.2.
[9] ISO 14971 "Medical devices—Application of risk management to medical devices" (2007).
[10] FDA's Statistical Guidance for Clinical Trials of Non-Diagnostic Medical Devices (1996).