User Research
For each prototype, we follow a specific user testing methodology and retrieved user data that will be of chief importance to our research project. Therefore, it is important to follow the User Research sections of each prototype. Be free to find them on our repositories and inside each Wiki. One example of this is the prototype-breast-screening repository. Here, you will find the User Research section of the repository Wiki. However, on this page, we will describe our theoretical methods for the chosen User Research approach. As well as, the data acquired for this user exploration. More information of the User Research section is available across our private wiki on the meta-private repository for team usage. Unfortunately, you need to be a member of our team to access the restricted information. Additionally, our research is supported by a list of clinical contacts and list of patients. Concerning research items promoted by this work, we have additional information on our ResearchGate page.
Methodology ↑
When choosing a methodology for our project, there comes a point in which we need to move onto deciding how to conduct our User Research. Usually, this happens after we have worked out the objectives and study goals. However since the beginning, in our work, we will address Design Thinking (DT) and User-Centered Design (UCD) methodologies, both from the Human-Computer Interaction (HCI) literature. Our methodologies should not be seen as a concrete and inflexible approach to design. In fact, the set of identified component stages serve as a guide to the activities that we would typically carry out. In order to gain the purest and most informative insights of the project, stages might be switched, conducted concurrently and repeated several times in order to expand the solution space, and zero in on the best possible solutions. In essence, from the HCI literature, both DT and UCD methodologies are iterative, flexible and focused on collaboration between researchers and clinicians, with an emphasis on bringing ideas to life based on how real users think, feel and behave. For these reasons, we are applying the two methodologies to our work.
Use Cases ↑
Our Use Cases are Scenarios where the use of our several Prototypes may help improve medical imaging care. They were created to empower researchers to produce solutions that are clinically relevant, ethical, and effective. Each Use Case provides narrative descriptions and flow charts that specify the healthcare goal of the solution, the required clinical input, how it should integrate into the clinical workflow and how it will interface with users and tools. Furthermore, our Use Cases help radiologists and allied professionals by ensuring that our tools provide information that can be efficiently implemented into the daily workflow, and have the potential to improve the quality and efficiency of patient care. In this section, we follow the instructions and examples from the ACR Data Science Institute of the American College of Radiology (ACR) organization. Also, we are following the information available on the Radiology Use Cases documentation from the OpenMRS Wiki.
UML User Cases one depth bellow
For the Unified Modeling Language (UML), an Actor specifies a role played by a user or any system that interacts with the subject. An Actor models a type of role played by an entity that interacts with the subject (e.g., examining and diagnose patients), but which is external to the subject. Actor may represent roles played by human users (e.g., Physician, Radiologist, etc), external hardware (e.g., MG, US, MRI, etc), or other subjects (e.g., Modality, etc).
Examine patients, obtain medical histories, diagnose illnesses, or prescribe and administer treatment for people suffering from injury or disease. A physician, medical practitioner, medical doctor, or simply a doctor, is a professional who practices medicine. Physicians are concerned with promoting, maintaining, or restoring health through the study, diagnosis, prognosis, and treatment of disease, injury, and other physical and mental impairments.
A surgeon is a physician who performs surgery. The surgical profession is one of responsibility and leadership. The surgeon is responsible for the preoperative diagnosis of the patient, for performing the operation, and for providing the patient with postoperative surgical care and treatment. The surgeon is also looked upon as the leader of the surgical team.
A gynecologist deals with the health of the female reproductive system. Outside medicine, the term means "the science of women". Its counterpart is andrology, which deals with medical issues specific to the male reproductive system.
A physician who treats cancer and provides medical care for a person diagnosed with cancer. The field of oncology has three major areas: medical, surgical, and radiation. A medical oncologist treats cancer using chemotherapy or other medications, such as targeted therapy or immunotherapy.
Is a physician who has specialized training in obtaining and interpreting medical images, which makes him or her an imaging expert. Radiologists are medical doctors that specialize in diagnosing and treating injuries and diseases using medical imaging (radiology) procedures (exams/tests) such as MG, US or MRI.
Are medical professionals who create medical images of the human anatomy to aid radiologists and other doctors diagnose and treat illness and injury. Radiographers, also known as radiologic technologists, diagnostic radiographers, and medical radiation technologists are healthcare professionals who specialize in the imaging of human anatomy for the diagnosis and treatment of pathology.
Equipment or probes used to acquire images of the body (MG, US, and MRI). The imaging modalities used in medicine are based on a variety of energy sources, including light, electrons, lasers, X-Rays, radionuclides, ultrasound and nuclear magnetic resonance. In our work, we will focus just in three modalities: (1) MammoGraphy (MG), on both CranioCaudal (CC) and MedioLateral Oblique (MLO) views; (2) UltraSound (US); and (3) Magnetic Resonance Imaging (MRI).
Staff who schedule, prepare and acquire exams. The radiology staff role is responsible for insurance, software, security, oncology, radiology, typing, scheduling, general, education, imaging. This department is generally the first contact so the staff must present a friendly, courteous and professional demeanor.
A patient is any recipient of health care services performed by healthcare professionals. The patient is most often ill or injured and in need of treatment by a physician, nurse, psychologist, dentist, veterinarian, or other health care provider.
Breast Cancer Medical Imaging Diagnosis Tool
- Physician
- Oncologist
- Radiologist
The first use case aims at describing how users will perform and use our tools in a general way. For that, in this use case, we define in what way our tools are contemplating a Multimodality strategy concerning the several used modalities of medical imaging technologies.
- Lesions
Our tool is a breast cancer medical imaging diagnostic tool, which allows health professionals to visualize medical images searching for invasive lesions on breast cancer over a period of time (i.e., Follow-up). The tool uses medical images of women and data to support clinicians on the final diagnostic on a Multimodality strategy. As a definition, we consider Multimodality strategy as the use of the following modalities: (1) MG, on both CC and MLO views; (2) US; and (3) MRI.
A Multimodality of medical imaging exams (i.e., MG, US, and MRI) result in hundreds of images for radiologist review. Numerous acquired sequences require constant mental comparison and task switching during the process of interpretation. Such activity may result in medical error and operator exhaustion. The ability to accurately visualize such images could substantially improve the efficiency of interpretation for large imaging datasets. By collating sequences and time points together into a Multimodality strategy, annotated lesions or other inputs could be used to rapidly compare study series without shifting attention. Further, image attributes could be superimposed allowing image series to provide all information required for accurate interpretation. Such a method could also be applied for purposes of temporal comparison, both for identical and different modalities.
Multimodality acquisition (e.g., MG, US, and MRI) in the breast for lesion visualizations and characterization. Using a Multimodality strategy, all relevant lesion attributes are available on an overall visualization, allowing for rapid and accurate diagnosis.
The primary Use Case would likely be the integration of multiple modalities in our tool. Image diagnosis could be composed of MG, US, and MRI exams acquired on a breast anatomic location. Digital data from standard MG, US, and MRI breast exams are sent to a server, which can be accessed by the tool. The clinician then analyzes the image data and dynamically generates breast information for the final interpretation.
Breast Cancer Lesion Visualization Highlight
- Physician
- Oncologist
- Radiologist
The second use case aims to provide information regarding how to highlight breast lesions. Here, we provide a type of image visualization for a highlighted lesion. Therefore, it will supply the radiologist with appearing visualization of lesions and their pathologic forms. The radiologist will use this information to help guide their decision.
- Lesion
- Lesion annotations
- AI lesion identification with BI-RADS severity
Provide a type of image visualization for a highlighted lesion to supply the radiologist with appearing visualization of lesions and their pathologic forms. With this Use Case, we aim at highlighting the visualization of contours from several breast cancer lesions. The breast cancer lesions are twofold: (a) masses; and (2) calcifications.
Many radiology practices suffer from a lack of breast-trained radiologists. Patients undergoing breast cancer screening at these sites may be under-diagnosed or over-diagnosed dependent upon the skill level of the interpreting radiologists. Because most of the studies are negative on screening mammography, non-breast radiologists may not have the breadth of experience required to identify nuanced differences that separate malignant from benign lesions. A tool that enables radiologists to specify a lesion of interest and enhance the lesion visualization with pathologic highlights would increase confidence in medical decisions and curb unnecessary recall imaging and biopsies.
A radiologist reviewing a case notices that masses and calcifications in the left breast but has not encountered this specific appearance before. The radiologist highlights the area of concern on the MG and our tool provides the appearing lesions that the radiologist, with little experience on breast, is able to understand. All of the appearing lesions were benign on pathology and the radiologist, therefore, determines that this lesion is of no concern.
A radiologist is viewing the screening MG, US, and MRI images. A potential abnormality is identified and the radiologist draws the lesion contours. The contours can be of any size or shape specified by the radiologist. This image is then processed and sent to our internal database. The radiologist will use this information to help guide their decision and can close the panel to continue with their regular workflow.
Personas ↑
Our Personas are fictional clinical characters, which we create based upon our research to represent the different user types that might similarly use our system. For the presented information, we follow the article titled as "Personas – A Simple Introduction" from Interaction Design Foundation. However, two other scientific publications were also important to perform this Personas idea. One paper titled as "Developing Mobile Clinical Decision Support for Nursing Home Staff Assessment of Urinary Tract Infection using Goal-Directed Design", from Jones et al. who describe and applied Personas to their work.
Persona A ↑
Goals: To deliver effective care for his patients, contribute to medical literature, and keep abreast of technological developments.
Activities: Henriques Oliveira is a 28-year-old Intern from the Department of Radiology now living in Lisbon. He likes to cycles to work.
Attitudes: Mainly assesses and treats breast cancer inpatients but also sees patients in a specialist outpatient clinic.
Aptitudes: From the Department of Radiology oh Hospital Central (Public) in Sintra. He also works at a regional Clinic (Private) in Cascais.
Skills: Intern in Medical Radiology learning with Seniors regarding patients affected by, or at risk of, breast cancer.
Persona B ↑
Goals: To deliver effective care for his patients, contribute to medical literature, and keep abreast of technological developments.
Activities: Henriques Oliveira is a 28-year-old Intern from the Department of Radiology now living in Lisbon. He likes to cycles to work.
Attitudes: Mainly assesses and treats breast cancer inpatients but also sees patients in a specialist outpatient clinic.
Aptitudes: From the Department of Radiology oh Hospital Central (Public) in Sintra. He also works at a regional Clinic (Private) in Cascais.
Skills: Intern in Medical Radiology learning with Seniors regarding patients affected by, or at risk of, breast cancer.
Persona C ↑
Goals: To deliver effective care for his patients, contribute to medical literature, and keep abreast of technological developments.
Activities: Henriques Oliveira is a 28-year-old Intern from the Department of Radiology now living in Lisbon. He likes to cycles to work.
Attitudes: Mainly assesses and treats breast cancer inpatients but also sees patients in a specialist outpatient clinic.
Aptitudes: From the Department of Radiology oh Hospital Central (Public) in Sintra. He also works at a regional Clinic (Private) in Cascais.
Skills: Intern in Medical Radiology learning with Seniors regarding patients affected by, or at risk of, breast cancer.
Persona D ↑
Goals: To deliver effective care for his patients, contribute to medical literature, and keep abreast of technological developments.
Activities: Henriques Oliveira is a 28-year-old Intern from the Department of Radiology now living in Lisbon. He likes to cycles to work.
Attitudes: Mainly assesses and treats breast cancer inpatients but also sees patients in a specialist outpatient clinic.
Aptitudes: From the Department of Radiology oh Hospital Central (Public) in Sintra. He also works at a regional Clinic (Private) in Cascais.
Skills: Intern in Medical Radiology learning with Seniors regarding patients affected by, or at risk of, breast cancer.
Persona E ↑
Goals: To deliver effective care for his patients, contribute to medical literature, and keep abreast of technological developments.
Activities: Henriques Oliveira is a 28-year-old Intern from the Department of Radiology now living in Lisbon. He likes to cycles to work.
Attitudes: Mainly assesses and treats breast cancer inpatients but also sees patients in a specialist outpatient clinic.
Aptitudes: From the Department of Radiology oh Hospital Central (Public) in Sintra. He also works at a regional Clinic (Private) in Cascais.
Skills: Intern in Medical Radiology learning with Seniors regarding patients affected by, or at risk of, breast cancer.
User Stories ↑
Our User Stories are short statements that identify the user, as well as their needs and goals. It determines who are the users, what they need and why they need it. There is usually one User Story per each Persona. As we above present, there are multiple Personas.
As an example, we used the following codesqueeze structure:
"As a [role], I want [feature] because [reason]."
Scenarios ↑
In our project, Scenarios are situations where we want to capture how users are performing tasks. Scenarios must describe the motivations of the users, their tasks, goals and/or a question they need to be answered. Also, the Scenarios are suggestions for possible ways to accomplish these goals. Essentially, it is the development of a User Story relating the task to multiple target users. However, Scenarios can also be broken down into Use Cases that describe the flow of tasks in which any user takes a given functionality or path.
Storyboards ↑
Storyboards are the visual representations of how the user would react with our solutions. There are different types of Storyboards: sketches, illustrations and screenshots, slideshows and animated, live demos. For the purpose, we used the prototyping tool called Justinmind to support this section.
Tracking Users ↑
The focus on the UX goals, to drive analytics measurement plans, rather than tracking superficial metrics, provide us core goal identification, of a design, to meaningfully measure it. These UX goals should remain at the forefront when choosing how to measure the success of a design. Rather than jumping directly to what may be simple to track, these UX goals are critical to meaningful measurement. Therefore, we use several trackers in our solution and listed below.
List of used trackers:
Matomo (Piwik) ↑
The Matomo (Piwik) tracker has a full range of features that allow us to evaluate the entire clinician-journey of everyone who visits our platform. There is no other tool that offers such a complete range of features like this. We can connect to our Matomo Analytics account to access the project dashboard.
Username: mimbcd.ui
All we need to do is to feed it with some data by embedding the following tracking code into our platform, right before the </head> tag or in a .js file:
<!-- Matomo -->
var _paq = window._paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="https://breastscreening.matomo.cloud/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', '1']);
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
})();
<!-- End Matomo Code -->
For some further customization of our tracking code, the JavaScript Tracking Client guide might be helpful. Nevertheless, it is recommended for having a look at the User Guides and FAQs to learn more about all the many features.
Related Issues ↑
User Test Evaluations ↑
When reporting results from a user test, we should focus primarily on our findings and recommendations that are differentiated by levels of severity. Including the pertinent information from the test plan and presenting just enough details so that the method is identifiable. By keeping the sections short, we should use tables to display the metrics and use visual examples to demonstrate problem areas, when possible. The following list is presenting the resources of our User Test Evaluations, formally transcribed in a document as User Testing and Analysis (UTA) guide under this research work. In our research works, we use the term UTA to denominate the several iterations of our user studies. For example, the following Test 1, Test 2, etc., are instances of UTA1, UTA2, etc., respectively. Another important external piece of information, yet more complete, is the one presented on both meta-private and prototype-breast-screening repositories. Unfortunately, some of this information is private.
List of user test evaluations resources:
Test 1 ↑
Test 2 (Pre-Master Thesis of Francisco Maria Calisto) ↑
Test 3 (Post-Master Thesis) ↑
Test 4: Single-Modality vs Multi-Modality ↑
Test 4: Assistant ↑
Test 5: Multi-Classifier Prototype ↑
Test 6: First Functional Co-Variables Classifier ↑
Test 7: Multi-Modality vs Assistant ↑
Test 8: Demographic & Domain Study ↑
Test 9: Scalable Interactions (Pre-Master Thesis of Hugo Lencastre) ↑
-
[GitHub] Master Project: Breast Cancer Multimodality Scalable Interactions
-
[GitHub] Master Thesis: Breast Cancer Multimodality Scalable Interactions
-
[GitHub] Extended Abstract: Breast Cancer Multimodality Scalable Interactions
Test 10: Lesion Bounding Boxes vs Real Lesion Contours (Pre-Master Thesis of Nádia Mourão) ↑
-
[Sheet] UTA10: Highlight of the Lesion in Breast Cancer Diagnosis
-
[Sheet] UTA10: 2D Breast Cancer Diagnosis Explainable Visualizations
Test 11: Assertiveness-based Assistant (Pre-Master Thesis of João Fernandes) ↑
Test 12: Lexical Annotating Tool (Pre-Master Thesis of Miguel Bastos) ↑
Test 13: AI-based Recommendations (Pre-Master Thesis of Vicente Sobral) ↑
Test 14: Assertiveness-based Researchers (CMU | HCII) ↑
For more information, please follow the private information concerning this section inside the wiki of the meta-private repository. Unfortunately, you need to be a member of our team to access the restricted information.