Acrobat Reader required to view PDF documents. If you do not have Acrobat Reader download it here:

The following proposal has been developed by the Cytopathology Education and Technology Consortium (CETC)* to provide guidance to the Food and Drug Administration (FDA) and other regulatory bodies regarding the evaluations of workload limits for cytotechnologists

*The CETC is composed of representatives from professional societies with interest in cytopathology. The representatives authoring this manuscript are:

Karen Atkison, MPA, CT (ASCP)
American Society of Cytopathology
Burlington, NC

Jacalyn Papillo, CT (ASCP)
American Society of Cytopathology
Burlington, VT

R. Marshall Austin, MD
College of American Pathologists
Charleston, SC

Marluce Bibbo, MD
International Academy of Cytology
Philadelphia, PA

George Birdsong, MD
American Society of Cytopathology
Atlanta, GA

Thomas A. Bonfiglio, MD
American Society for Clinical Pathology
Rochester, NY

Diane D. Davey, MD
American Society of Cytopathology
Lexington, KY

Rose Marie Gatscha, SCT (ASCP)
American Society for Cytotechnology
New York, NY

Lydia Howell, MD
Papanicolaou Society of Cytopathology
Sacramento, CA

Dina Mody, MD
College of American Pathologists
Houston, TX

Marianne Prey MD
College of American Pathologists
St. Louis, MO

Stephen Raab, MD
Papanicolaou Society of Cytopathology
Pittsburgh, PA

Volker Schneider, MD
International Academy of Cytology
Freiburg, Germany

Diane Solomon, MD
ex officio
Bethesda, MD

Background

As technologies emerge for automated, assisted-screening of cervicovaginal samples, guidelines for assessing daily workload of cytotechnologists utilizing these technologies is seen as an integral component in the evaluation of overall instrument performance. In the absence of workload limits for automated-assisted screening within the Clinical Laboratory Improvement Amendments of 1998 (CLIA’88), it is our judgment that daily workload performance should be included for assessment in pivotal clinical trials used by medical device manufacturers to support FDA pre-market approval applications. Subsequent changes to workload performance recommendations should also be based on well-designed evaluation studies.

Workload performance studies should provide information regarding safe and effective workload limits, and should use sensitivity and specificity calculations as the overall basis for interpreting results. Workload data elements should be provided in tabular form to appropriately convey the results obtained during the trial. The manufacturer should provide this information within the application to the FDA for inclusion in the labeling of the device.

The Pivotal Clinical Trial

It is our belief that the clinical trial should replicate the intended use of the device as nearly as possible.

  • Institutions participating in a clinical trial should represent laboratories with varying sample volumes and patient populations, with both average disease prevalence and high disease prevalence populations being represented.
  • Cytotechnologist participants should include individuals with varying levels of work experience, and varying levels of slide screening productivity.
    • A sufficient number of test subjects, and abnormal proportions, need be included to adequately assess sensitivity and specificity.
  • The number of cytotechnologists in the clinical trial should be high enough to adequately represent varying levels of work experience and various levels of screening productivity, but should be limited to the extent that each cytotechnologist sees a sufficient number of slides to insure multiple day’s workload for each participating cytotechnologist.
  • Cytotechnologists who participate in the manual review arm of the clinical trial should also participate in the automated, assisted-screening arm of the trial.

Workload: Data Collection and Calculation

The method chosen to record and assess workload during the clinical trial should replicate daily use conditions, as nearly as possible. The following guidelines apply:

  • Time spent reviewing clinical information and on clerical reporting methods should be as equivalent as possible in both arms of the trial. Available clinical information for each enrolled subject should be the same in both arms of the study.
  • Daily totals for the number of slides evaluated and the number of hours-spent evaluating slides should be recorded for each study arm.
  • When possible, the number of hours spent screening per day and the screening environment (e.g. breaks, meals) should be similar in both arms of the trial, and similar to the total daily screening time for each cytotechnologist participant.
  • The number of screening hours per day need not be constant from one cytotechnologist to another.
  • Daily workload for both arms should be expressed as “average slides/hour”, calculated by dividing the total number of slides per day by the total number of hours screened per day.
  • Manufacturers should not be required to collect workload statistics hour by hour during the clinical trial.
  • The use of independent time studies (stop-watch) to monitor workflow segments of the manual versus automated arm should not be required but may be done at the discretion of the manufacturer. Data derived from such studies can be made available in the product labeling.

Labeling Specific to Workload

Automated assisted screening systems require “abnormal” or “suspicious” slides triaged for full-slide review. In clinical practice, automated assisted screening with subsequent full side review of abnormals may be performed by a single technologist (Method A) or it may be performed by two cytotechnologists; with the first cytotechnologist performing the automated assisted review (triage) and the second cytotechnologist performing the full-slide review (Method B ). In addition, laboratories may desire the flexibility to use a combination of Methods A and B. When Method B is used, workload limits for hours spent doing full slide review will be the same as manual screening limits.

Workload data from clinical trials should indicate which method is being analyzed when calculating workload limits. Manufacturers should create a data set for the method(s) they have analyzed during clinical trials and submit these to the FDA as part of the PMA application, or to supplement subsequent labeling claims.

 

The data elements for workload should include the following:

 

Data Reported by Individual Cytotechnologist

  • Listing of each cytotechnologist participant (A-Z), leaving the true identity of each cytotechnologist confidential. The listing of cytotechnologists should be sorted by the number of years screening experience.

Manual Arm

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The calculated average number of hours screened per day in trial
  • The calculated average number of slides/hour
  • The calculated average number of slides/day
  • The extrapolated average number of slides per 7.5 hour day
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day)

Method A: Automated, Assisted-Screening Arm with Full Slide Review)

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The total number of days screening in trial
  • The calculated average number of hours screened per day in trial
  • The calculated average number of slides/hour for assisted screening with full slide review
  • The extrapolated average number of slides per 7.5 hour day for assisted screening with full slide review
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day)for assisted screening with full slide review

Method B: Automated, Assisted-Screening Arm without Full Slide Review)

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The total number of days screening in trial
  • The calculated average number of hours screened per day in trial
  • The total number of slides triaged to full-slide review
  • The total number of slide evaluation hours for assisted triage screening
  • The calculated average number of slides/hour for assisted triage screening
  • The calculated range of slides/hour for assisted triage screening (low day – high day)
  • The extrapolated average number of slides per 7.5 hour day for assisted triage screening
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day) for assisted triage screening
  • Where applicable, indicate slide accountability for the total number of slides archived (requiring no manual review by a cytotechnologist).

Data Collated and Reported by Site

Workload data should be reported in either an aggregate fashion for all sites, including mean, median and ranges, or by dividing into two groups: those sites with equivalent automated and manual sensitivity, and those sites with improved sensitivity in the automated arm. Workload data from site(s) with inferior automated screening performance compared to the manual-screening arm should not be included (exclusion of data for workload setting purposes only).

  • Listing of each site (A-Z), leaving the true identity of each site confidential.
  • Total number of cytotechnologists participating in study
  • Range of levels of experience

Manual Arm

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The calculated average number of hours screened per day in trial
  • The calculated average number of slides/hour
  • The calculated average number of slides/day
  • The extrapolated average number of slides per 7.5 hour day
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day)

Method A: Automated, Assisted-Screening Arm with Full Slide Review)

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The total number of days screening in trial
  • The calculated average number of hours screened per day in trial
  • The calculated average number of slides/hour for assisted screening with full slide review
  • The extrapolated average number of slides per 7.5 hour day for assisted screening with full slide review
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day) for assisted screening with full slide review

Method B: Automated, Assisted-Screening Arm without Full Slide Review)

  • The total number of study slides evaluated
  • The total number of slide evaluation hours
  • The total number of days screening in trial
  • The calculated average number of hours screened per day in trial
  • The total number of slides triaged to full-slide review
  • The total number of slide evaluation hours for assisted triage screening
  • The calculated average number of slides/hour for assisted triage screening
  • The calculated range of slides/hour for assisted triage screening (low day – high day)
  • The extrapolated average number of slides per 7.5 hour day for assisted triage screening
  • The extrapolated range of number of slides per 7.5 hour day (low day – high day)for assisted triage screening
  • Where applicable, indicate slide accountability for the total number of slides archived (requiring no manual review by a cytotechnologist).

Workload Maximums

For Method A, the actual numeric data for the extrapolated average number of slides per 7.5 hour day between the manual arm and the automated, assisted-screening arm should be compared to determine daily workload limits. For Method B, manufacturers must indicate that the data does not include full slide review and the actual numeric data for the calculated number of slides per hour between the manual arm and the automated, assisted-screening arm should be compared to determine hourly workload limits.

Limitations

Workload data collected during a clinical trial is limited not only in total numbers, but also in that a learning curve may be inherent with new technology. Workload levels achieved during the trial may be able to be surpassed once greater experience is gained with the technology. The above methodology of assessing workload is intended as a guideline for initial labeling and implementation upon FDA approval. Subsequent changes to workload performance recommendations should also be based on well-designed evaluation studies.

The initial labeling should contain a statement to the fact that:
“This product has been shown to be safe and effective, within the course of the clinical trial, at workloads levels listed in the product insert. The upper limit achieved during the trial represents an absolute maximum number of slides and is not to be employed as a performance target for individuals examining slides utilizing this technology. It is the responsibility of the Technical Supervisor (Medical Director) to establish initial individual workload limits and reassess to these limits every six months, within the range of product labeling. This technology has not been proven to be safe or effective at workload levels which exceed product labeling.”

Subsequent Workload Studies

It is recognized that over time, with increased utilization and greater experience with automated, assisted-screening, workload productivity in excess of initial labeling may be achieved without compromising sensitivity and specificity. A mechanism should be established for manufacturers who wish to prove reliability and to establish a higher productivity claim than that achieved during the pivotal clinical trial and initial product labeling.

Written and approved by the CETC on February 24, 2003.