publications

2023

1. ICCV

   PGFed: Personalize Each Client’s Global Objective for Federated Learning

   Jun Luo, Matias Mendieta, Chen Chen, and Shandong Wu

   In International Conference on Computer Vision (ICCV), 2023

   (Oral, top 1.8%)

   Abstract arXiv Code Poster Slides

   Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client’s empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client’s risk is estimated through a first-order approximation for other clients’ adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients’ empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods.

2. ICCV

   FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning

   Guangyu Sun, Matias Mendieta, Jun Luo, Shandong Wu, and Chen Chen

   In International Conference on Computer Vision (ICCV), 2023

   Abstract arXiv Code Poster

   Personalized Federated Learning (PFL) represents a promising solution for decentralized learning in heterogeneous data environments. Partial model personalization has been proposed to improve the efficiency of PFL by selectively updating local model parameters instead of aggregating all of them. However, previous work on partial model personalization has mainly focused on Convolutional Neural Networks (CNNs), leaving a gap in understanding how it can be applied to other popular models such as Vision Transformers (ViTs). In this work, we investigate where and how to partially personalize a ViT model. Specifically, we empirically evaluate the sensitivity to data distribution of each type of layer. Based on the insights that the self-attention layer and the classification head are the most sensitive parts of a ViT, we propose a novel approach called FedPerfix, which leverages plugins to transfer information from the aggregated model to the local client as a personalization. Finally, we evaluate the proposed approach on CIFAR-100, OrganAMNIST, and Office-Home datasets and demonstrate its effectiveness in improving the model’s performance compared to several advanced PFL methods.

3. ISBI

   Human Not in the Loop: Objective Sample Difficulty Measures for Curriculum Learning

   Zhengbo Zhou, Jun Luo, Gene Kitamura, Dooman Arefan, and Shandong Wu

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2023

   (Oral)

   Abstract arXiv DOI

   Curriculum learning is a learning method that trains models in a meaningful order from easier to harder samples. A key here is to devise automatic and objective difficulty measures of samples. In the medical domain, previous work applied domain knowledge from human experts to qualitatively assess classification difficulty of medical images to guide curriculum learning, which requires extra annotation efforts, relies on subjective human experience, and may introduce bias. In this work, we propose a new automated curriculum learning technique using the variance of gradients (VoG) to compute an objective difficulty measure of samples and evaluated its effects on elbow fracture classification from X-ray images. Specifically, we used VoG as a metric to rank each sample in terms of the classification difficulty, where high VoG scores indicate more difficult cases for classification, to guide the curriculum training process We compared the proposed technique to a baseline (without curriculum learning), a previous method that used human annotations on classification difficulty, and anti-curriculum learning. Our experiment results showed comparable and higher performance for the binary and multi-class bone fracture classification tasks.

4. JNM

   Potential Use of Artificial Intelligence in Sincalide-Stimulated Cholescintigraphy: A Pilot Study

   Jun Luo, Dooman Arefan, Anil Vasireddi, Shandong Wu, and Nghi Nguyen

   Journal of Nuclear Medicine

   Abstract URL

   Sincalide-stimulated cholescintigraphy (SSC) is frequently performed to calculate the gallbladder ejection fraction, which aids in diagnosing functional gallbladder disorder. In this pilot study, we explored an artificial intelligence (AI)-based application for gallbladder radioactivity tracking.

2022

1. IJCAI

   Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

   Jun Luo, and Shandong Wu

   In International Joint Conference on Artificial Intelligence (IJCAI), 2022

   (acceptance rate: 14.9%)

   Abstract arXiv DOI Slides

   The goal of conventional federated learning (FL) is to train a global model for a federation of clients with decentralized data, reducing the systemic privacy risk of centralized training. The distribution shift across non-IID datasets, also known as the data heterogeneity, often poses a challenge for this one-global-model-fits-all solution. In this work, we propose APPLE, a personalized cross-silo FL framework that adaptively learns how much each client can benefit from other clients’ models. We also introduce a method to flexibly control the focus of training APPLE between global and local objectives. We empirically evaluate our method’s convergence and generalization behavior and performed extensive experiments on two benchmark datasets and two medical imaging datasets under two non-IID settings. The results show that the proposed personalized FL framework, APPLE, achieves state-of-the-art performance compared to several other personalized FL approaches in the literature.

2. ISBI

   FedSLD: Federated Learning with Shared Label Distribution for Medical Image Classification

   Jun Luo, and Shandong Wu

   In IEEE International Symposium on Biomedical Imaging (ISBI), 2022

   (Oral)

   Abstract arXiv DOI Slides

   Machine learning in medical research, by nature, needs careful attention on obeying the regulations of data privacy, making it difficult to train a machine learning model over gathered data from different medical centers. Failure of leveraging data of the same kind may result in poor generalizability for the trained model. Federated learning (FL) enables collaboratively training a joint model while keeping the data decentralized for multiple medical centers. However, federated optimizations often suffer from the heterogeneity of the data distribution across medical centers. In this work, we propose Federated Learning with Shared Label Distribution (FedSLD) for classification tasks, a method that assumes knowledge of the label distributions for all the participating clients in the federation. FedSLD adjusts the contribution of each data sample to the local objective during optimization given knowledge of the distribution, mitigating the instability brought by data heterogeneity across all clients. We conduct extensive experiments on four publicly available image datasets with different types of non-IID data distributions. Our results show that FedSLD achieves better convergence performance than the compared leading FL optimization algorithms, increasing the test accuracy by up to 5.50 percentage points.

3. SPIE

   Deep Curriculum Learning in Task Space for Multi-Class Based Mammography Diagnosis

   Jun Luo, Dooman Arefan, Margarita Zuley, Jules Sumkin, and Shandong Wu

   In SPIE Medical Imaging 2022

   (Oral)

   Abstract arXiv DOI Slides

   Mammography is used as a standard screening procedure for the potential patients of breast cancer. Over the past decade, it has been shown that deep learning techniques have succeeded in reaching near-human performance in a number of tasks, and its application in mammography is one of the topics that medical researchers most concentrate on. In this work, we propose an end-to-end Curriculum Learning (CL) strategy in task space for classifying the three categories of Full-Field Digital Mammography (FFDM), namely Malignant, Negative, and False recall. Specifically, our method treats this three-class classification as a "harder" task in terms of CL, and create an "easier" sub-task of classifying False recall against the combined group of Negative and Malignant. We introduce a loss scheduler to dynamically weight the contribution of the losses from the two tasks throughout the entire training process. We conduct experiments on an FFDM datasets of 1,709 images using 5-fold cross validation. The results show that our curriculum learning strategy can boost the performance for classifying the three categories of FFDM compared to the baseline strategies for model training.

2021

1. SPIE

   Medical knowledge-Guided Deep Curriculum Learning for Elbow Fracture Diagnosis from X-Ray Images

   Jun Luo, Gene Kitamura, Emine Doganay, Dooman Arefan, and Shandong Wu

   In SPIE Medical Imaging 2021

   (Oral)

   Abstract DOI PDF Slides

   Elbow fractures are one of the most common fracture types. Diagnoses on elbow fractures often need the help of radiographic imaging to be read and analyzed by a specialized radiologist with years of training. Thanks to the recent advances of deep learning, a model that can classify and detect different types of bone fractures needs only hours of training and has shown promising results. However, most existing deep learning models are purely data-driven, lacking incorporation of known domain knowledge from human experts. In this work, we propose a novel deep learning method to diagnose elbow fracture from elbow X-ray images by integrating domain-specific medical knowledge into a curriculum learning framework. In our method, the training data are permutated by sampling without replacement at the beginning of each training epoch. The sampling probability of each training sample is guided by a scoring criterion constructed based on clinically known knowledge from human experts, where the scoring indicates the diagnosis difficultness of different elbow fracture subtypes. We also propose an algorithm that updates the sampling probabilities at each epoch, which is applicable to other sampling-based curriculum learning frameworks. We design an experiment with 1865 elbow X-ray images for a fracture/normal binary classification task and compare our proposed method to a baseline method and a previous method using multiple metrics. Our results show that the proposed method achieves the highest classification performance. Also, our proposed probability update algorithm boosts the performance of the previous method.

2. MICCAI-W

   Knowledge-Guided Multiview Deep Curriculum Learning for Elbow Fracture Classification

   Jun Luo, Gene Kitamura, Dooman Arefan, Emine Doganay, Ashok Panigrahy, and Shandong Wu

   In MICCAI Workshop on Machine Learning in Medical Imaging

   Abstract arXiv DOI Code Poster Slides

   Elbow fracture diagnosis often requires patients to take both frontal and lateral views of elbow X-ray radiographs. In this paper, we propose a multiview deep learning method for an elbow fracture subtype classification task. Our strategy leverages transfer learning by first training two single-view models, one for frontal view and the other for lateral view, and then transferring the weights to the corresponding layers in the proposed multiview network architecture. Meanwhile, quantitative medical knowledge was integrated into the training process through a curriculum learning framework, which enables the model to first learn from "easier" samples and then transition to "harder" samples to reach better performance. In addition, our multiview network can work both in a dual-view setting and with a single view as input. We evaluate our method through extensive experiments on a classification task of elbow fracture with a dataset of 1,964 images. Results show that our method outperforms two related methods on bone fracture study in multiple settings, and our technique is able to boost the performance of the compared methods. The code is available at https://github.com/ljaiverson/multiview-curriculum.

3. ARRS

   Multi-View-Enabled Deep Learning for Automated Radiographic View Classification and Fracture Detection of the Elbow

   Emine Doganay, Gene Kitamura, Luo Yang, Jun Luo, and Shandong Wu

   American Roentgen Ray Society (ARRS) Annual Meeting

   (Oral)

   URL Slides
4. AAOMR

   Image Enhancement of Ultra-Low Dose CBCT Images Using a Deep Generative Model

   Suvendra Vijayan, Jun Luo, Shandong Wu, and Anitha Potluri

   American Academy of Oral and Maxillofacial Radiology (AAOMR) Annual Session

   (Oral)

   URL PDF