Hi, I’m Shijie. I am a Scientist at A*STAR in Singapore, working on embodied intelligence and multimodal foundation models. I am the Principal Investigator of an A*STAR Career Development Fund project (2025–2028). I received my Ph.D. in Computer Science from the University of Bonn, advised by Prof. Juergen Gall, and my M.S. from Nankai University, advised by Prof. Ming-Ming Cheng. Previously, I was a research intern at Qualcomm AI Research in Amsterdam and Intel Labs in Munich.
My research is in computer vision and 3D perception, with robots as a platform for learning and acting in the physical world. My goal is to build robots that learn from their own deployment: understanding what happened, deciding what to update, and retaining a lesson only while it remains valid. This work brings together three directions:
Spatial perception and reasoning: Grounding decisions in 3D scenes and world representations.
I study how to reconstruct, understand, and reason about the physical world, from large-scale 3D perception to language-guided grounding. A central question is how to connect observations to the right object or event, and gather further evidence when the cause of a failure is unclear. Representative work: SeeGround, MonoSR, and One Agent to Guide Them All.
Temporal understanding and memory: Recognising the past, anticipating the future, and knowing what to retain.
I model how activities and scenes unfold over time, connecting action segmentation, forecasting, and memory for long-horizon reasoning. I am interested in how robots can track changing conditions and determine when an earlier lesson should be re-verified, revised, or withdrawn. Representative work: MS-TCN++, Thinking Ahead, and PRISM.
Efficient, cost-aware embodied intelligence: Choosing when to look, ask, act, and learn.
I develop embodied systems that weigh the value of additional evidence against the cost of sensing, computation, and human help. This extends to deciding whether a deployment experience calls for an update to memory, a reusable skill, or model weights, with verification and rollback when needed. Representative work: SpatialReasoner, Talk2Escape, and Ask When It Pays.
I’m always open to collaboration and in-depth discussions. Get in touch.
News
- 2027One paper was accepted to WACV.
- 2026-09Starting from 2026.9, I serve as an Associate Editor for ICRA 2027 and IEEE Robotics and Automation Letters (RA-L).
- 2026-05We will host RobotPAD workshop (NeurIPS26, Paris), EMR workshop (ECCV26, Malmö).
- 2026-05I will give an invited short talk at Singapore Vision Day 2026 (NUS).
- 2026One paper was accepted to CVPR, one paper was accepted to ICLR, two papers were accepted to ECCV, one paper was accepted to CoRL, two papers were accepted to IROS, one paper was accepted to RA-L, one paper was accepted to BMVC, and one paper was accepted to ICASSP.
- 2025-10We got the 2nd place in Challenge of Multimodal Robot Learning in InternUtopia and Real World and 1st place in simulation stage at IROS2025.
- 2025Two papers were accepted to ICCV, and one paper was accepted to CVPR.
- 2024.4I successfully defended my PhD thesis and began my new role as a Scientist at A*STAR.
Publications
Selected Publications
arXiv & Preprints12 papers
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift
Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation
IntentionNav: A Benchmark for Intent-Driven Object Navigation from Implicit Human Instruction
AI for Auto-Research: Roadmap & User Guide
Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances
PRISM: Planning and Reasoning with Intent in Simulated embodied environments
Think, Then Look: Active Spatial Reasoning for House-Scale 3D Scene Understanding
Improving the Generalization of Segmentation Foundation Models via Weakly-Supervised and Unsupervised Adaptation
DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completion
Multi-View Industrial Anomaly Detection with Epipolar Constrained Cross-View Fusion
2027
Motion Subspace Canonicalization for Design-Intent Evaluation in CAD Sketches
2026
PanDA: Panoptic Domain Adaptation for Multimodal Perception in Autonomous Driving
Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
Thinking Ahead: Foresight Intelligence in MLLMs and World Models
MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
Perception-Aware Multimodal Spatial Reasoning from Monocular Images
Talk2Escape: Conversational Grounding for Vision-and-Language Navigation
Efficient Online Robotic Reinforcement Learning with Explicit Manifold Alignment
Prioritizing Faithfulness: Efficient Zero-Shot Novel View Synthesis via Homography-Guided SA-RePaint
Sam-Guided Multi-View Fusion for Weakly Supervised 3D Point Cloud Segmentation
2025
Global-Aware Monocular Semantic Scene Completion with State Space Models
Future-Aware Interaction Network For Motion Forecasting
Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding
VaLID: Variable-Length Input Diffusion for Novel View Synthesis
SiFH: Siamese Frequency Harmonization Self-Supervised Learning for Motion Forecasting
2024
Tfnet: Exploiting temporal cues for fast and accurate lidar semantic segmentation
2023
MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation
Foresight Social-aware Reinforcement Learning for Robot Navigation
2022
Dual Pyramid Generative Adversarial Networks for Semantic Image Synthesis
2021
Moving object segmentation in 3D LiDAR data: A learning-based approach exploiting sequential data
Multi-scale interaction for real-time lidar data segmentation on an embedded platform
Rethinking 3-D LiDAR point cloud segmentation
Spatial-temporal consistency network for low-latency trajectory forecasting
2020
MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation
Pose Refinement Graph Convolutional Network for Skeleton-based Action Recognition
Refinedbox: Refining for fewer and high-quality object proposals
2019
Joint salient object detection and existence prediction
2018
Del: Deep embedding learning for efficient image segmentation
Direct line guidance odometry
Structured skip list: A compact data structure for 3D reconstruction
No papers match your search.
Honors & Fellowships
- Global Young Scientists Summit (GYSS) 2027 — Selected to represent A*STAR.
- KIT YIG Prep Pro Remote Fellowship (2026–2028)
- DAAD AINeT Fellow (2026)
- IROS 2025 Challenge on Multimodal Robot Learning — Second Place, $5K Award — Track 1: Vision-Language Manipulation in Open Tabletop Environments.
Academic Service
- Associate Editor: IEEE Robotics and Automation Letters (RA-L); ICRA 2027 (Robot Learning)
- Area Chair: ICME 2026, BMVC 2026, IJCNN 2027
- Workshop Organizer:
- Reviewer: Journals: TPAMI, IJCV, TNNLS, TIP, TMM, RA-L, PR; Conferences: CVPR, ICCV, ECCV, ICLR, NeurIPS, AAAI, IJCAI, KDD, ACM MM, BMVC, ICRA, IROS, RSS
Grants
A*STAR Career Development Fund (CDF)Apr 2025–Mar 2028
Towards Intelligent Embodied 3D Perception: Learning to Understand, Anticipate, and Interact in Complex Environments
Invited Talks
- [2026] Invited talk at 2nd Singapore Vision Day, National University of Singapore.
- Invited seminars at National University of Singapore, Monash University, Macquarie University, University of Adelaide, Aalto University, KU Leuven, and Télécom Paris.