search
Back to results

Breast Ultrasound Image Reviewed With Assistance of Deep Learning Algorithms

Primary Purpose

Breast Cancer, Breast Lesions, Breast Mass

Status
Unknown status
Phase
Not Applicable
Locations
United States
Study Type
Interventional
Intervention
Ultrasound Image review with CADe
Ultrasound Image review with CADx
Ultrasound Image manual review
Biopsy
Sponsored by
Samsung Medison
About
Eligibility
Locations
Arms
Outcomes
Full info

About this trial

This is an interventional device feasibility trial for Breast Cancer focused on measuring Breast cancer, Breast Imaging

Eligibility Criteria

19 Years - undefined (Adult, Older Adult)All SexesAccepts Healthy Volunteers

  1. Inclusion Criteria:

    • Adult females or males recommended for ultrasound-guided breast lesion biopsy or ultrasound follow-up with at least one suspicious lesion
    • Age > 18 years
    • Able to provide informed consent

  2. Exclusion Criteria:

    • Unable to read and understand English
    • Unable or unwilling to provide informed consent
    • A patient with current or previous diagnosis of breast cancer in the same quadrant
    • Unable or unwilling to undergo study procedures

  3. Subject Characteristics

    1. Number of Subjects: 300 subjects from 300 separate breast lesions can be acquired. If a subject has more than 1 suspicious lesion, each may be chosen by the radiologist attending as suitable for "second review".
    2. Gender and Age of Subjects: Adult females or males aged 18 years or older who meet all of the inclusion criteria and none of the exclusion criteria will be considered for enrollment. Minors are excluded as breast cancer is very rare in this age group.
    3. Racial and Ethnic Origin: There are no enrollment exclusions based on economic status, race, or ethnicity. Based on local and United States census data, the expected ethnic distribution will be approximately 26 Hispanic (approx. 16%) and 134 non-Hispanic people. Furthermore, the expected racial distribution is expected to be approximately 126 White (approx. 79% of the whole study), 21 Black or African America (13%), 8 Asian (5%), and 5 of other categories (3%).
    4. Vulnerable Subjects: It is unlikely that any UR students or employees will be enrolled unless their primary physician refers them to UR Medicine Breast Imaging at Red Creek for breast ultrasound and a suspicious lesion is found. We do not expect any of these referrals to be from staffs who work directly with the PIs.

Sites / Locations

  • University of Rochester

Arms of the Study

Arm 1

Arm 2

Arm 3

Arm Type

Active Comparator

Experimental

Experimental

Arm Label

Manual review

Review by S-Detect for Breast

Review with assistance of S-Detect for Breast

Arm Description

The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.

The same images will be separately processed by the artificial intelligence system (S-Detect for Breast) by Samsung. The two results, one by the radiologists and the other by artificial intelligence system, will be compared to statistically quantify equivalence (CADe).

Second, the images will be reviewed by the radiologists with the help of artificial intelligence system, which is an interactive tool automatically providing recommendations on BIRADS descriptor choices that can be modified by the radiologists. The radiologists, after selecting all the descriptors of BIRADS, will decide the assessment categories. These decisions will be compared with the ground truths generated from the biopsy results or a 24-month follow-up (CADx).

Outcomes

Primary Outcome Measures

Concordance rate
Breast Imaging Reporting and Data System descriptors suggested by S-Detect for Breast are in good agreement with those selected by experts. In other words, the Breast Imaging Reporting and Data System Lexicon values generated by S-Detect for Breast are not statistically different from the consensus of experts. Breast Imaging Reporting and Data System Assessment Category Score: The user makes the final decision on the Assessment Category Score. Using this Score, S-Detect displays the assessment description. Category 0: Incomplete - Need Additional Imaging Evaluation Category 1: Negative Category 2: Benign Category 3: Probably Benign Category 4a: Low suspicion for malignancy Category 4b: Moderate suspicion for malignancy Category 4c: High suspicion for Malignancy Category 5: Highly Suggestive of Malignancy Category 6: Known Biopsy-Proven Malignancy

Secondary Outcome Measures

Reporting time
Measure reporting time of Breast Imaging Reporting and Data System Lexicon value in Breast imaging by radiologists without S-Detect for Breast and also measured report time by radiologists with S-Detect for Breast.
Consensus
Evaluate the consensus between manually reading of Breast Imaging without assistance and Automatically detection results(Breast Imaging Reporting and Data System Lexicons). Average of consensus is evaluated in both of Expert group and non-expert group.
Accuracy
Comparing to the Breast Biopsy results, The accuracy of Breast Imaging results by radiologists with CADx will be evaluated.
Sensitivity
Comparing to the Breast Biopsy results, The sensitivity of Breast Imaging results by radiologists with CADx will be evaluated.
Specificity
Comparing to the Breast Biopsy results, The specificity of Breast Imaging results by radiologists with CADx will be evaluated.
Area Under Curve
Comparing to the Breast Biopsy results, Area Under Curve (ROC analysis) of Breast Imaging results by radiologists with CADx will be evaluated.

Full Information

First Posted
October 11, 2018
Last Updated
October 27, 2019
Sponsor
Samsung Medison
Collaborators
University of Rochester
search

1. Study Identification

Unique Protocol Identification Number
NCT03706534
Brief Title
Breast Ultrasound Image Reviewed With Assistance of Deep Learning Algorithms
Official Title
Breast Ultrasound Image Reviewed With Assistance of Deep Learning
Study Type
Interventional

2. Study Status

Record Verification Date
October 2019
Overall Recruitment Status
Unknown status
Study Start Date
September 20, 2018 (Actual)
Primary Completion Date
November 30, 2019 (Anticipated)
Study Completion Date
January 31, 2020 (Anticipated)

3. Sponsor/Collaborators

Responsible Party, by Official Title
Sponsor
Name of the Sponsor
Samsung Medison
Collaborators
University of Rochester

4. Oversight

Studies a U.S. FDA-regulated Drug Product
No
Studies a U.S. FDA-regulated Device Product
No
Product Manufactured in and Exported from the U.S.
No
Data Monitoring Committee
Yes

5. Study Description

Brief Summary
This study evaluates a second review of ultrasound images of breast lesions using an interactive "deep learning" (or artificial intelligence) program developed by Samsung Medical Imaging, to see if this artificial intelligence will help the Radiologist make more accurate diagnoses.
Detailed Description
Using ultrasound images prospectively acquired, the purpose of this study entails a second review of ultrasound images with suspicious breast lesions using an interactive "deep learning" (or artificial intelligence) program developed by SamsungMedison Co.,Ltd. The images will be reviewed by the radiologists twice: first without, and then with assistance of artificial intelligence program by SamsungMedison Co., Ltd. BIRADS system will be used in this study. The objectives of the study are twofold: to quantify the statistical equivalence of radiologists' opinion and AI's output (CADe), and to check BIRADS score-based diagnostic accuracy (CADx) that is gained by the Radiologists' use of this interactive tool

6. Conditions and Keywords

Primary Disease or Condition Being Studied in the Trial, or the Focus of the Study
Breast Cancer, Breast Lesions, Breast Mass
Keywords
Breast cancer, Breast Imaging

7. Study Design

Primary Purpose
Device Feasibility
Study Phase
Not Applicable
Interventional Study Model
Crossover Assignment
Model Description
This clinical study performed by multiple reader multiple case (MRMC) study design, where as set of clinical readers evaulate under multiple reading condition. All Interpreting physician(reader) independently read all of the cases. (fully-crossed design).
Masking
Outcomes Assessor
Masking Description
The study consisted of 10 readers with varying levels of training and experience providing analysis on a randomized set of 300 patients' breast ultrasound data with and without S-Detect for Breast. Two reading periods separated by at least 3-week washout, totaling 600 cases analyzed per reader. PI and her associate have knowledge about patients diagnosis and other information. So, they are exclueded in readers for "reviewing". And all breast US images are de-indentified.
Allocation
Randomized
Enrollment
300 (Actual)

8. Arms, Groups, and Interventions

Arm Title
Manual review
Arm Type
Active Comparator
Arm Description
The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.
Arm Title
Review by S-Detect for Breast
Arm Type
Experimental
Arm Description
The same images will be separately processed by the artificial intelligence system (S-Detect for Breast) by Samsung. The two results, one by the radiologists and the other by artificial intelligence system, will be compared to statistically quantify equivalence (CADe).
Arm Title
Review with assistance of S-Detect for Breast
Arm Type
Experimental
Arm Description
Second, the images will be reviewed by the radiologists with the help of artificial intelligence system, which is an interactive tool automatically providing recommendations on BIRADS descriptor choices that can be modified by the radiologists. The radiologists, after selecting all the descriptors of BIRADS, will decide the assessment categories. These decisions will be compared with the ground truths generated from the biopsy results or a 24-month follow-up (CADx).
Intervention Type
Device
Intervention Name(s)
Ultrasound Image review with CADe
Other Intervention Name(s)
S-Detect, S-Detect for Breast, CADe, Computer-Assisted Detection Device
Intervention Description
This software is a computer-aided detection (CADe) software application, designed to assist radiologist to analyze breast ultrasound images. S-Detect automatically segments and classifies shape, orientation, margin, lesion boundary, echo pattern, and posterior feature characteristics of user-selected region of interest. The device uses deep learning methods to perform tissue segmentation and classification of images.
Intervention Type
Device
Intervention Name(s)
Ultrasound Image review with CADx
Other Intervention Name(s)
S-Detect, S-Detect for Breast, CADx, Computer-Assisted Diagnostic Device
Intervention Description
This software is also a computer-assisted diagnostic(CADx) software application, designed to assist a medical doctor in determining diagnosis by presenting whether a lesion is malignant in a breast ultrasound image obtained from an ultrasound imaging device.
Intervention Type
Device
Intervention Name(s)
Ultrasound Image manual review
Other Intervention Name(s)
Convetional Ultrasound image
Intervention Description
The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored.
Intervention Type
Procedure
Intervention Name(s)
Biopsy
Intervention Description
Suspicious lesions found on breast ultrasound are then followed either by ultrasound guided biopsy or ultrasound imaging every 6 months for two years. For those who undergo biopsy, ultrasound provides images which are used to localize the lesion and guide the placement of the biopsy needle. The sample is sent to pathology for diagnosis, while the ultrasound guidance images are stored. For those who have imaging follow-up, ultrasound images of the breast mass are obtained, digitally stored and interpreted by the radiologist typically using BIRADS scheme.
Primary Outcome Measure Information:
Title
Concordance rate
Description
Breast Imaging Reporting and Data System descriptors suggested by S-Detect for Breast are in good agreement with those selected by experts. In other words, the Breast Imaging Reporting and Data System Lexicon values generated by S-Detect for Breast are not statistically different from the consensus of experts. Breast Imaging Reporting and Data System Assessment Category Score: The user makes the final decision on the Assessment Category Score. Using this Score, S-Detect displays the assessment description. Category 0: Incomplete - Need Additional Imaging Evaluation Category 1: Negative Category 2: Benign Category 3: Probably Benign Category 4a: Low suspicion for malignancy Category 4b: Moderate suspicion for malignancy Category 4c: High suspicion for Malignancy Category 5: Highly Suggestive of Malignancy Category 6: Known Biopsy-Proven Malignancy
Time Frame
2 days
Secondary Outcome Measure Information:
Title
Reporting time
Description
Measure reporting time of Breast Imaging Reporting and Data System Lexicon value in Breast imaging by radiologists without S-Detect for Breast and also measured report time by radiologists with S-Detect for Breast.
Time Frame
2 day
Title
Consensus
Description
Evaluate the consensus between manually reading of Breast Imaging without assistance and Automatically detection results(Breast Imaging Reporting and Data System Lexicons). Average of consensus is evaluated in both of Expert group and non-expert group.
Time Frame
2 day
Title
Accuracy
Description
Comparing to the Breast Biopsy results, The accuracy of Breast Imaging results by radiologists with CADx will be evaluated.
Time Frame
7 day
Title
Sensitivity
Description
Comparing to the Breast Biopsy results, The sensitivity of Breast Imaging results by radiologists with CADx will be evaluated.
Time Frame
7 day
Title
Specificity
Description
Comparing to the Breast Biopsy results, The specificity of Breast Imaging results by radiologists with CADx will be evaluated.
Time Frame
7 day
Title
Area Under Curve
Description
Comparing to the Breast Biopsy results, Area Under Curve (ROC analysis) of Breast Imaging results by radiologists with CADx will be evaluated.
Time Frame
7 day

10. Eligibility

Sex
All
Minimum Age & Unit of Time
19 Years
Accepts Healthy Volunteers
Accepts Healthy Volunteers
Eligibility Criteria
Inclusion Criteria: Adult females or males recommended for ultrasound-guided breast lesion biopsy or ultrasound follow-up with at least one suspicious lesion Age > 18 years Able to provide informed consent Exclusion Criteria: Unable to read and understand English Unable or unwilling to provide informed consent A patient with current or previous diagnosis of breast cancer in the same quadrant Unable or unwilling to undergo study procedures Subject Characteristics Number of Subjects: 300 subjects from 300 separate breast lesions can be acquired. If a subject has more than 1 suspicious lesion, each may be chosen by the radiologist attending as suitable for "second review". Gender and Age of Subjects: Adult females or males aged 18 years or older who meet all of the inclusion criteria and none of the exclusion criteria will be considered for enrollment. Minors are excluded as breast cancer is very rare in this age group. Racial and Ethnic Origin: There are no enrollment exclusions based on economic status, race, or ethnicity. Based on local and United States census data, the expected ethnic distribution will be approximately 26 Hispanic (approx. 16%) and 134 non-Hispanic people. Furthermore, the expected racial distribution is expected to be approximately 126 White (approx. 79% of the whole study), 21 Black or African America (13%), 8 Asian (5%), and 5 of other categories (3%). Vulnerable Subjects: It is unlikely that any UR students or employees will be enrolled unless their primary physician refers them to UR Medicine Breast Imaging at Red Creek for breast ultrasound and a suspicious lesion is found. We do not expect any of these referrals to be from staffs who work directly with the PIs.
Overall Study Officials:
First Name & Middle Initial & Last Name & Degree
Avice O'Connell
Organizational Affiliation
Department of Imaging Sciences, University of Rochester
Official's Role
Principal Investigator
First Name & Middle Initial & Last Name & Degree
Kevin Parker
Organizational Affiliation
Department of Electrical & Computer Engineering, University of Rochester
Official's Role
Principal Investigator
Facility Information:
Facility Name
University of Rochester
City
Rochester
State/Province
New York
ZIP/Postal Code
14642
Country
United States

12. IPD Sharing Statement

Plan to Share IPD
No

Learn more about this trial

Breast Ultrasound Image Reviewed With Assistance of Deep Learning Algorithms

We'll reach out to this number within 24 hrs