Shao-Yuan Lo

Shao-Yuan Lo 羅紹元

I am a Research Scientist at Honda Research Institute USA. My research lies in Machine Learning, Computer Vision, and Trustworthy AI. Specifically, my recent research focuses on Multimodal Large Language Models, Visual Scene Understanding, and Adversarial Machine Learning.

I obtained my Ph.D. in Electrical and Computer Engineering at Johns Hopkins University in 2023, advised by Prof. Vishal M. Patel. Before that, I received my M.S. (Electronics Engineering) and B.S. (EECS Honors Program) degrees from National Chiao Tung University in 2019 and 2017, respectively, advised by Prof. Hsueh-Ming Hang. I also spent wonderful times as an Applied Scientist Intern at Amazon Just Walk Out team in summer 2022 and at Amazon Astro team in summer 2021.

Email / CV / Google Scholar / Github / LinkedIn / Twitter

News

09/2024: Invited talk at Jilin University (remote).

07/2023: 1 paper accepted to ECCV 2024.

02/2024: 2 papers accepted to CVPR 2024. Won the Best Student Paper Award at SPIE Medical Imaging 2024!!

01/2024: Invited talks at Academia Sinica and National Yang Ming Chiao Tung University.

07/2023: Joined Honda Research Institute USA as a Research Scientist. See you in San Jose, CA!!

03/2023: Defended my PhD dissertation!!

02/2023: 1 paper accepted to CVPR 2023.

09/2022: Accepted to the Google CS Research Mentorship Program.

07/2022: 1 paper accepted to IEEE T-PAMI, and 1 paper accepted to IROS 2022.

05/2022: Joined Amazon (Just Walk Out team) as an Applied Scientist Intern. See you in Seattle, WA!!

01/2022: Invited talks at Academia Sinica and National Yang Ming Chiao Tung University.

12/2021: 1 paper accepted to IEEE T-IP.

06/2021: Invited talk at CVPR 2021 Tutorial on Adversarial Machine Learning in Computer Vision.

05/2021: Joined Amazon (Astro team) as an Applied Scientist Intern. See you in Bellevue, WA!!

12/2019: Won the Best Paper Award at ACM Multimedia Asia 2019!!

08/2019: Joined Johns Hopkins University as a PhD student. See you in Baltimore, MD!!

Publications
	StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Yuxiang Guo, Faizan Siddiqui, Yang Zhao, Rama Chellappa, Shao-Yuan Lo arXiv, 2024 paper / bibtex
	Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models Yuchen Yang, Kwonjoon Lee, Behzad Dariush, Yinzhi Cao, Shao-Yuan Lo European Conference on Computer Vision (ECCV), 2024 paper / code / bibtex
	Uncertainty-aware Action Decoupling Transformer for Action Anticipation Hongji Guo, Nakul Agarwal, Shao-Yuan Lo, Kwonjoon Lee, Qiang Ji IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [Highlight] paper / bibtex
	Can’t Make an Omelette without Breaking Some Eggs: Plausible Action Anticipation using Large Video-Language Models Himangi Mittal, Nakul Agarwal, Shao-Yuan Lo, Kwonjoon Lee IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024 paper / bibtex
	Patient-specific 3D Volumetric CBCT Image Reconstruction with Single X-ray Projection Using Denoising Diffusion Probabilistic Model Shaoyan Pan, Shao-Yuan Lo, Chih-Wei Chang, Ella Salari, Tonghe Wang, Justin Roper, Aparna H. Kesarwala, Xiaofeng Yang SPIE Medical Imaging (SPIE MI), 2024 [Best Student Paper Award] paper / bibtex
	Adaptive Batch Normalization Networks for Adversarial Robustness Shao-Yuan Lo, Vishal M. Patel IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS), 2024 [Oral] paper / slides / poster / bibtex
	Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation Shao-Yuan Lo, Poojan Oza, Sumanth Chennupati, Alejandro Galindo, Vishal M. Patel IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 paper / slides / poster / bibtex
	Deep Learning-based Multi-Organ CT Segmentation with Adversarial Data Augmentation Shaoyan Pan, Shao-Yuan Lo, Min Huang, Chaoqiong Ma, Jacob Wynne, Tonghe Wang, Tian Liu, Xiaofeng Yang SPIE Medical Imaging (SPIE MI), 2023 paper / bibtex
	Adversarially Robust One-class Novelty Detection Shao-Yuan Lo, Poojan Oza, Vishal M. Patel IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2022 paper / slides / bibtex
	Learning Feature Decomposition for Domain Adaptive Monocular Depth Estimation Shao-Yuan Lo, Wei Wang, Jim Thomas, Jingjing Zheng, Vishal M. Patel, Cheng-Hao Kuo IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022 paper / blog / bibtex
	Exploring Adversarially Robust Training for Unsupervised Domain Adaptation Shao-Yuan Lo, Vishal M. Patel Asian Conference on Computer Vision (ACCV), 2022 paper / slides / bibtex
	Defending Against Multiple and Unforeseen Adversarial Videos Shao-Yuan Lo, Vishal M. Patel IEEE Transactions on Image Processing (T-IP), 2021 [Journal presentation at ICIP 2022] paper / slides / bibtex
	Error Diffusion Halftoning Against Adversarial Examples Shao-Yuan Lo, Vishal M. Patel IEEE International Conference on Image Processing (ICIP), 2021 paper / slides / bibtex
	Overcomplete Representations Against Adversarial Videos Shao-Yuan Lo, Jeya Maria Jose Valanarasu, Vishal M. Patel IEEE International Conference on Image Processing (ICIP), 2021 paper / slides / bibtex
	MultAV: Multiplicative Adversarial Videos Shao-Yuan Lo, Vishal M. Patel IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS), 2021 paper / slides / bibtex
	Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation Shao-Yuan Lo, Hsueh-Ming Hang, Sheng-Wei Chan, Jing-Jhih Lin ACM International Conference on Multimedia in Asia (ACM MM Asia), 2019 [Best Paper Award] paper / project page / slides / bibtex
	Exploring Semantic Segmentation on the DCT Representation Shao-Yuan Lo, Hsueh-Ming Hang ACM International Conference on Multimedia in Asia (ACM MM Asia), 2019 [Oral] paper / slides / bibtex
	Multi-Class Lane Semantic Segmentation using Efficient Convolutional Networks Shao-Yuan Lo, Hsueh-Ming Hang, Sheng-Wei Chan, Jing-Jhih Lin IEEE International Workshop on Multimedia Signal Processing (MMSP), 2019 paper / slides / bibtex
	Incorporating Luminance, Depth and Color Information by a Fusion-based Network for Semantic Segmentation Shang-Wei Hung, Shao-Yuan Lo, Hsueh-Ming Hang IEEE International Conference on Image Processing (ICIP), 2019 [Oral] paper / project page / slides / bibtex
	Efficient Road Lane Marking Detection with Deep Learning Ping-Rong Chen, Shao-Yuan Lo, Hsueh-Ming Hang, Sheng-Wei Chan, Jing-Jhih Lin IEEE International Conference on Digital Signal Processing (DSP), 2018 paper / slides / bibtex

Dissertations
	Robust Computer Vision Against Adversarial Examples and Domain Shifts Shao-Yuan Lo Ph.D. Thesis, Johns Hopkins University, 2023 thesis / bibtex
	Real-Time Semantic Segmentation Networks for Autonomous Driving Shao-Yuan Lo Master Thesis, National Chiao Tung University, 2019 thesis / bibtex

Industrial Experience

Research Scientist at Honda Research Institute USA, San Jose, CA (July 2023 - Now)

Applied Scientist Intern at Amazon (Just Walk Out team), Seattle, WA (May 2022 - Aug 2022)

Applied Scientist Intern at Amazon (Astro team), Bellevue, WA (May 2021 - Aug 2021)

Awards

Robert F. Wagner All-Conference Best Student Paper Award, SPIE Medical Imaging 2024 (2024)

CVPR DEI Travel Award, IEEE/CVF CVPR 2023 (2023)

Google CS Research Mentorship Program, Google Research (2022)

Government Scholarship to Study Abroad, Ministry of Education, Taiwan (2020)

First-Year Doctoral Fellowship, Johns Hopkins University (2019)

Government Scholarship to World Top 100 Universities, Ministry of Education, Taiwan (2019)

Best Paper Award, ACM Multimedia Asia 2019 (2019)

Best Master Thesis Award, Chinese Image Processing and Pattern Recognition Society (2019)

Students’ Outstanding Contribution Award, National Chiao Tung University (2019)

Dean's List, National Chiao Tung University (2017)

Scholarship to Overseas Exchange Program, National Chiao Tung University (2016)

WINTEK Outstanding Freshman Scholarship, WINTEK Corp. and National Chiao Tung University (2013)

Invited Talks

09/19/2024: “Multimodal Large Language Models for Driving Safety Applications.” At Jilin University, China (Remote). Host: Prof. Hongxia Xie.

01/09/2024: “Robust Computer Vision Against Adversarial Examples and Domain Shifts.” At National Yang Ming Chiao Tung University, Taiwan. Host: Prof. Wen-Hsiao Peng.

12/14/2023: “Robust Computer Vision Against Adversarial Examples and Domain Shifts.” At Academia Sinica, Taiwan. Host: Prof. Jun-Cheng Chen.

04/14/2023: “Robust Computer Vision Against Adversarial Examples and Domain Shifts.” At Computational Cognition, Vision, and Learning (CCVL) Lab, Johns Hopkins University. Host: Prof. Alan Yuille.

01/10/2022: “Defending Against Multiple and Unforeseen Adversarial Videos.” At National Yang Ming Chiao Tung University, Taiwan. Host: Prof. Wen-Hsiao Peng.

01/05/2022: “Defending Against Multiple and Unforeseen Adversarial Videos.” At Academia Sinica, Taiwan. Host: Prof. Jun-Cheng Chen.

06/19/2021: “Adversarial Attacks and Defenses in Videos.” At CVPR 2021 Tutorial on Adversarial Machine Learning in Computer Vision, Virtual. Host: Prof. Cihang Xie.

Academic Activities

Associate Editor: ICRA 2025

Area Chair: CVPR 2024 AI4CG Workshop

Journal Reviewer: IEEE T-PAMI, IEEE T-IP, IEEE RA-L, IEEE T-CSVT, IEEE T-SMC, IEEE JETCAS, Pattern Recognition, Neurocomputing, Medical Physics

Conference Reviewer: CVPR, ICCV, ECCV, ICLR, AAAI, WACV, ACCV, ICIP, ICPR, AVSS

Teaching Assistant: Deep Learning (EN.520.638), Johns Hopkins University, Spring (2021, 2022, 2023)

Press Coverage

10/11/2022: I was reported on Formosa Television (民視新聞), EBC News (東森新聞), and New Tang Dynasty Television (新唐人電視台) when I was seving as the Vice President of the Taiwanese Student Association at Johns Hopkins University.

06/01/2019: I was reported on Liberty Times (自由時報), United Daily News (聯合報), and Commercial Times (工商時報) when I was graduating from National Chiao Tung University.

07/19/2013: I was reported on China Times (中國時報) when I was graduating from senior high school (National Experimental High School at Hsinchu Science Park).

06/04/2010: I was reported on Liberty Times (自由時報) when I was graduating from junior high school.

unique visitors since Oct 2020

Template from Jon Barron