Mingyu Ouyang

CS Ph.D. Student, National University of Singapore

Portrait of Mingyu Ouyang

I am a second-year Ph.D. student in Computer Science at the National University of Singapore, co-advised by Prof. Hwee Tou Ng and Prof. Mike Zheng Shou. I completed my Master's at NUS, and obtained my Bachelor's degree from Wuhan University.

I am with Show Lab, NUS and ARC Lab, Tencent IEG on multimodal digital agent research. I previously interned at SenseTime Research Singapore and Tencent Map, Beijing.

I am interested in the interaction and productivity gains enabled by digital (computer-use/game) agents. Recently, I am focusing on game environments, as games offer rich visual observations, long-horizon goals, and dynamic mechanics. Improvements in perception, reasoning, and control in games will support the future embodiment of digital agents.

Email: yyyangwhu@gmail.com


Updates
Earlier updates (2023–2025)
  • 09/2025 SlideTailor was accepted by AAAI 2026.
  • 01/2025 Started Ph.D. in School of Computing, NUS.
  • 06/2024 DCTransformer was accepted by IEEE Transactions on Image Processing.
  • 03/2024 AssistGUI was accepted by CVPR 2024.
  • 09/2023 Joined Show Lab, NUS and started an internship at SenseTime Research Singapore.
  • 08/2023 Started Master's in Data Science and Machine Learning at Faculty of Science, NUS.

Publications (* equal contribution; † corresponding author)
Player-1 Gym: synthetic 2D and 3D game environments and training overview

Player-1: Learning to Play from Synthetic Experience
Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou†, Hwee Tou Ng†M. Ouyang, Y. Wang, S. Hu, X. Yin, H. Sun, G. Wang, J. Pu, C. Cheng, H. Chen, Y. Shan, M. Z. Shou†, H. T. Ng†
Under review, 2026

Synthetic environments improve transferable agent capabilities: An agentic post-training framework that learns general game control beyond supervised demonstrations.

GameWorld demo showing multimodal agents interacting across browser games

GameWorld:Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
Mingyu Ouyang*, Siyuan Hu*, Kevin Qinghong Lin, Hwee Tou Ng†, Mike Zheng Shou†M. Ouyang*, S. Hu*, K. Q. Lin, H. T. Ng†, M. Z. Shou†
ECCV 2026, Best Paper Runner-Up Award, Multimodal Digital Agents Workshop @ ECCV 2026

Can VLMs play games as humans do? GameWorld evaluates game agents (both computer-use agents and general-purpose agents) with 34 games and 170 tasks.

Two FocusUI examples progressing from original interfaces to saliency maps, masked tokens, and animated clicks on target buttons

FocusUI:Efficient UI Grounding via Position-Preserving Visual Token Selection
Mingyu Ouyang, Kevin Qinghong Lin, Mike Zheng Shou†, Hwee Tou Ng†M. Ouyang, K. Q. Lin, M. Z. Shou†, H. T. Ng†
CVPR 2026, CVPR Compute Transparency Champion Award

Makes UI agents' perception more efficient: FocusUI selects instruction-relevant visual tokens for low-cost UI grounding.

SlideTailor workflow: paper and preferences become slides, a speech script, and a video presentation

SlideTailor:Personalized Presentation Slide Generation for Scientific Papers
Wenzheng Zeng*, Mingyu Ouyang*, Langyuan Cui*, Hwee Tou Ng†W. Zeng*, M. Ouyang*, L. Cui*, H. T. Ng†
AAAI 2026

Generates preference-aligned slides for scientific papers.

Computer Use out-of-the-box demo controlling a desktop browser from a mobile instruction

The Dawn of GUI Agent:A Preliminary Case Study with Claude 3.5 Computer Use
Siyuan Hu, Mingyu Ouyang, Difei Gao, Mike Zheng Shou†S. Hu, M. Ouyang, D. Gao, M. Z. Shou†
Technical Report 2024, GitHub 2k stars

An out-of-the-box framework for desktop GUI agents, supporting multiple OSs and models.

AssistGUI demo placing text behind a person in Adobe After Effects

AssistGUI: Task-Oriented PC Graphical User Interface Automation
Difei Gao, Lei Ji, Zechen Bai, Mingyu Ouyang, Peiran Li, Dongxing Mao, Qinchen Wu, Weichen Zhang, Peiyi Wang, Xiangwu Guo, Hengxu Wang, Luowei Zhou, Mike Zheng Shou†D. Gao, L. Ji, Z. Bai, M. Ouyang, P. Li, D. Mao, Q. Wu, W. Zhang, P. Wang, X. Guo, H. Wang, L. Zhou, M. Z. Shou†
CVPR 2024

A desktop GUI agent framework for task decomposition, GUI parsing, action generation, and reflection.

DCTransformer comparison showing a compressed JPEG and the recovered high-quality image

JPEG Quantized Coefficient Recovery via DCT Domain Spatial-Frequential Transformer
Mingyu Ouyang, Zhenzhong Chen†M. Ouyang, Z. Chen†
IEEE Transactions on Image Processing 2024

Recovers JEPG-quantized DCT coefficients with spatial-frequency modeling.


Education
National University of Singapore
Ph.D. in Computer Science
National University of Singapore
M.Sc. in Data Science and Machine Learning
Wuhan University
B.Sc. in Geospatial Informatics and Digitalized Technology

Experiences
ARC Lab, Tencent IEG
Research Intern (Project Up)
UseIt AI
Agent Dev
SenseTime Research Singapore
Research Intern
Tencent Map
Algorithm Dev Intern

Miscellaneous
  • I enjoy football games (Visca Barça! 💙❤️). I played for and served as the captain of the Wuhan University football team, where we won the championship at the 16th Hubei Province Games in 2022. "Salid y disfrutad."