Selected Publications

2024

• Conflictbank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM. Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, Yu Cheng. NeurIPS 2024
• MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution. Wei Tao, Yucheng Zhou, Wenqiang Zhang, Yu Cheng. NeurIPS 2024
• On Giant’s Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion. Chenghao Fan, Zhenyi Lu, Wei Wei, Jie Tian, Xiaoye Qu, Dangyang Chen, Yu Cheng. NeurIPS 2024
• Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging. Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, Yu Cheng. NeurIPS 2024
• LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training. Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, Yu Cheng. EMNLP 2024
• SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information. Jiashuo Sun, Jihai Zhang, Yucheng Zhou, Zhaochen Su, Xiaoye Qu, Yu Cheng. EMNLP 2024
• On the Universal Truthfulness Hyperplane Inside LLMs. Junteng Liu, Shiqi Chen, Yu Cheng, Junxian He. EMNLP 2024
• MoE-I^2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition. Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, Bo Yuan. EMNLP 2024
• Timo: Towards Better Temporal Reasoning for Language Models. Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, Yu Cheng. COLM 2024
• Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning. JIHAI ZHANG, Xiang Lan, Xiaoye Qu, Yu Cheng, Mengling Feng, Bryan Hooi. ECCV 2024
• Rethinking Weakly-supervised Video Temporal Grounding From a Game Perspective. Xiang Fang, Zeyu Xiong, Wanlong Fang, Xiaoye Qu, Chen Chen, Jianfeng Dong, Keke Tang, Pan Zhou, Yu Cheng. ECCV 2024
• Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval using Language. Xiang Fang, Wanlong Fang, Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Renfu Li, Zichuan Xu, Lixing Chen, Panpan Zheng, Yu Cheng. MM 2024
• MoE-RBench: Towards Building Reliable Language Models with Sparse Mixture-of-Experts. Guanjie Chen, Xinyu Zhao, Tianlong Chen, Yu Cheng. ICML 2024
• LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models. Tianci Liu, Haoyu Wang, Shiyang Wang, Yu Cheng, Jing Gao. ICML 2024
• Multimodal Instruction Tuning with Conditional Mixture of LoRA. Ying Shen, Zhiyang Xu, Qifan Wang, Yu Cheng, Wenpeng Yin, Lifu Huang. ACL 2024
• Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? Zhaochen Su, Juntao Li, Jun Zhang, Tong Zhu, Xiaoye Qu, Pan Zhou, Yan Bowen, Yu Cheng, Min Zhang. ACL 2024
• Confidence is not Timeless: Modeling Temporal Validity for Rule-based Temporal Knowledge Graph Forecasting. Rikui Huang, Wei Wei, Xiaoye Qu, Shengzhe Zhang, Dangyang Chen, Yu Cheng. ACL 2024
• Mitigating Boundry Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models. Zhenyi Lu, Jie Tian, Wei Wei, Xiaoye Qu, Yu Cheng, Wenfeng Xie, Dangyang Chen. ACL 2024
• Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning. Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, Lifu Huang. ACL 2024
• Towards Robust Temporal Activity Localization Learning with Noisy Labels. Daizong Liu, Xiaoye Qu, Xiang Fang, Jianfeng Dong, Pan Zhou, Guoshun Nan, Keke Tang, Wanlong Fang, Yu Cheng. LREC-COLING 2024
• Reinforcement Learning with Token-level Feedback for Controllable Text Generation. Wendi Li, Wei Wei, Kaihe xu, Wenfeng xie, Dangyang Chen, Yu Cheng. NAACL 2024
• GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions. Woojeong Jin, Subhabrata Mukherjee, Yu Cheng, Yelong Shen, Weizhu Chen, Ahmed Hassan Awadallah, Damien Jose, Xiang Ren. NAACL 2024
• Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy. Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen. ICLR 2024
• Sparse MoE with Language Guided Routing for Multilingual Machine Translation. Xinyu Zhao, Xuxi Chen, Yu Cheng, Tianlong Chen. ICLR 2024
• Transform-Equivariant Consistency Learning for Temporal Sentence Grounding. Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Zichuan Xu, Haozhao Wang, Xing Di, Weining Lu, Yu Cheng. ACM Transactions on Multimedia Computing, Communications and Applications
• CR-MoE: Consistent Routed Mixture-of-Experts for Scaling Contrastive Learning. Ziyu Jiang, Guoqing Zheng, Yu Cheng, Ahmed Hassan Awadallah, Zhangyang Wang. Transactions on Machine Learning Research (TMLR)
• Enhancing Low-Resource Relation Representations through Multi-View Decoupling. Chenghao Fan, Wei Wei, Xiaoye Qu, Zhenyi Lu, Xie Wenfeng, Yu Cheng. AAAI 2024
• Unsupervised Domain Adaptative Temporal Sentence Localization with Mutual Information Maximization. Daizong Liu, Xiang Fang, Xiaoye Qu, Jianfeng Dong, He Yan, Yang Yang, Yu Cheng. AAAI 2024
• ProS: Facial Omni-Representation Learning via Prototype-based Self-Distillation. Xing Di, Yiyu Zheng, Xiaoming Liu, Yu Cheng. WACV 2024

2023

• DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, Bo Li. NeurIPS 2023 (Outstanding Paper Award)
• Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding. Xiang Fang, Daizong Liu, Wanlong Fang, Pan Zhou, Yu Cheng, Keke Tang, Kai Zou. EMNLP 2023
• Low-Switching Policy Gradient with Exploration via Online Sensitivity Sampling. Yunfan Li, Yiran Wang, Yu Cheng, Lin Yang. ICML 2023
• Local Byte Fusion for Neural Machine Translation. Makesh Narsimhan Sreedhar, Xiangpeng Wan, Yu Cheng, Junjie Hu. ACL 2023
• DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models. Xuxi Chen, Tianlong Chen, Weizhu Chen, Zhangyang Wang, Ahmed Hassan Awadallah, Yu Cheng. ACL 2023
• You Are Catching My Attention: Are Vision Transformers Bad Learners Under Backdoor Attacks? Zenghui Yuan, Pan Zhou, Kai Zou, Yu Cheng. CVPR 2023
• Transform-Equivariant Consistency Learning for Temporal Sentence Grounding. Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Zichuan Xu, Haozhao Wang, Xing Di, Weining Lu, Yu Cheng. CVPR 2023
• What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression. Mengnan Du, Subhabrata Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Hassan Awadallah. EACL 2023
• Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao. ICLR 2023
• Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis. Wan-Cyuan Fan, Yen-Chun Chen, DongDong Chen, Yu Cheng, Lu Yuan, Yu-Chiang Frank Wang. AAAI 2023
• Hypotheses Tree Building for One-Shot Temporal Sentence Localization. Daizong Liu, Xiang Fang, Pan Zhou, Xing Di, Weining Lu, Yu Cheng. AAAI 2023
• Filling the Information Gap between Video and Query for Language-Driven Moment Retrieval. Daizong Liu, Xiaoye Qu, Jianfeng Dong, Guoshun Nan, Pan Zhou, Zichuan Xu, Lixing Chen, He Yan, Yu Cheng. ACMMM 2023

2022

• RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL. Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Yu Cheng, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, Zhouhan Lin. EMNLP 2022
• Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding. Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong. EMNLP 2022
• M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design. Hanxue Liang, Zhiwen Fan, Rishov Sarkar, Ziyu Jiang, Tianlong Chen, Kai Zou, Yu Cheng, Cong Hao, Zhangyang Wang. NeurIPS 2022
• Point Cloud Domain Adaptation via Masked Local 3D Structure Prediction. Hanxue Liang, Hehe Fan, Zhiwen Fan, Yi Wang, Tianlong Chen, Yu Cheng, Zhangyang Wang. ECCV 2022
• DnA: Improving Few-Shot Transfer Learning with Low-Rank Decomposition and Alignment. Ziyu Jiang, Tianlong Chen, Xuxi Chen, Yu Cheng, Luowei Zhou, Lu Yuan, Ahmed Awadallah, Zhangyang Wang. ECCV 2022
• Scalable Learning to Optimize: A Learned Optimizer Can Train Big Models. Xuxi Chen, Tianlong Chen, Yu Cheng, Weizhu Chen, Ahmed Awadallah, Zhangyang Wang. ECCV 2022
• MA-CLIP: Towards Modality-Agnostic Contrastive Language-Image Pre-training. Haoxuan You, Luowei Zhou, Bin Xiao, Noel C Codella, Yu Cheng, Ruochen Xu, Shih-Fu Chang, Lu Yuan. ECCV 2022
• SemAttack: Natural Textual Attacks via Different Semantic Spaces. Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, Bo Li. NAACL 2022
• A Good Prompt Is Worth Millions of Parameters? Low-resource Prompt-based Learning for Vision-Language Models. Woojeong Jin, Yu Cheng, Yelong Shen, Weizhu Chen, Xiang Ren. ACL 2022
• The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy. Tianlong Chen, Zhenyu Zhang, Yu Cheng, Ahmed Awadallah, Zhangyang Wang. CVPR 2022
• Memory-guided Semantic Learning Network for Temporal Sentence Grounding. Daizong Liu, Xiaoye Qu, Xing Di, Yu Cheng, Zichuan Xu, Pan Zhou. AAAI 2022
• Unsupervised Temporal Video Grounding with Deep Semantic Clustering. Daizong Liu, Xiaoye Qu, Yinzhen Wang, Xing Di, Kai Zou, Yu Cheng, Zichuan Xu, Pan Zhou. AAAI 2022
• Playing Lottery Tickets with Vision and Language. Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu. AAAI 2022
• Efficient Robust Training via Backward Smoothing. Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, Jingjing Liu. AAAI 2022
• Adversarial Feature Augmentation and Normalization for Visual Recognition. Tianlong Chen, Yu Cheng, Zhe Gan, Jianfeng Wang, Lijuan Wang, Jingjing Liu, Zhangyang Wang. Transactions on Machine Learning Research (TMLR)

2021

• Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models. Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, Bo Li. NeurIPS 2021
• VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation. Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu. NeurIPS 2021
• Chasing Sparsity in Vision Transformers: An End-to-End Exploration. Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang. NeurIPS 2021
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective. Tianlong Chen, Yu Cheng, Zhe Gan, Jingjing Liu, Zhangyang Wang. NeurIPS 2021
• The Elastic Lottery Ticket Hypothesis. Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang. NeurIPS 2021
• Maxva: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients. Chen Zhu, Yu Cheng, Zhe Gan, Furong Huang, Jingjing Liu, Tom Goldstein. ECML 2021
• EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets. Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang, Jingjing Liu. ACL 2021
• Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding. Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng, Jingjing Liu. ACL 2021
• Deep Co-Attention Network for Multi-View Subspace Learning. Lecheng Zheng, Yu Cheng, Hongxia Yang, Nan Cao, Jingrui He. WWW 2021
• InfoBERT: Improving Robustness of Language Models from an Information Theoretic Perspective. Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, Jingjing Liu. ICLR 2021
• APo-VAE: Text Generation in Hyperbolic Space. Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, Jingjing Liu. NAACL 2021
• Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning. Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, Shiqi Xu. NAACL 2021
• Context-aware Biaffine Localizing Network for Temporal Sentence Grounding. Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Yu Cheng, Wei Wei, Zichuan Xu, Yulai Xie. CVPR 2021
• UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training. Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu. CVPR 2021
• EnlightenGAN: Deep Light Enhancement without Paired Supervision. Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, Zhangyang Wang. IEEE Transactions on Image Processing (TIP)
• Meta Module Network for Compositional Visual Reasoning. Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Wang, Jingjing Liu. WACV 2021 (Best Student Paper Honorable Mention)

2020

• Contextual Text Style Transfer. Yu Cheng, Zhe Gan, Yizhe Zhang, Oussama Elachqar, Dianqi Li, Jingjing Liu. EMNLP 2020
• HERO: Hierarchical Encoder for Video+ Language Omni-representation Pre-training. Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu. EMNLP 2020
• Multi-Fact Correction in Abstractive Text Summarization. Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, Jingjing Liu. EMNLP 2020
• Cross-Thought for Sentence Encoder Pre-training. Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, Jingjing Liu. EMNLP 2020
• Contrastive Distillation on Intermediate Representations for Language Model Compression. Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, Jingjing Liu. EMNLP 2020
• Uniter: Universal Image-text Representation Learning. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, Jingjing Liu. ECCV 2020
• Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models. Jize Cao, Zhe Gan, Yu Cheng, Licheng Yu, Yen-Chun Chen, Jingjing Liu. ECCV 2020
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning. Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, Jingjing Liu. NeurIPS 2020
• Graph Optimal Transport for Cross-Domain Alignment. Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, Jingjing Liu. ICML 2020
• Freelb: Enhanced Adversarial Training for Language Understanding. Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Thomas Goldstein, Jingjing Liu. ICLR 2020
• Sequential Attention GAN for Interactive Image Editing. Yu Cheng, Zhe Gan, Yitong Li, Jingjing Liu, Jianfeng Gao. ACMMM 2020
• Fine-grained Iterative Attention Network for Temporal Language Localization in Videos. Xiaoye Qu, Pengwei Tang, Zhikang Zhou, Yu Cheng, Jianfeng Dong, Pan Zhou. ACMMM 2020
• Distilling the Knowledge of BERT for Text Generation. Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, Jingjing Liu. ACL 2020
• INSET: Sentence Infilling with Inter-sentential Generative Pre-training. Yichen Huang, Yizhe Zhang, Oussama Elachqar, Yu Cheng. ACL 2020
• Discourse-Aware Neural Extractive Model for Text Summarization. Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu. ACL 2020
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning. Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, Zhangyang Wang. CVPR 2020
• VIOLIN: A Large-Scale Dataset for Video-and-Language Inference. Jingzhou Liu, Wenhu Chen, Yu Cheng, Zhe Gan, Licheng Yu, Yiming Yang, Jingjing Liu. CVPR 2020
• BachGAN: High-Resolution Image Synthesis from Salient Object Layout. Yandong Li, Yu Cheng, Zhe Gan, Licheng Yu, Liqiang Wang, Jingjing Liu. CVPR 2020
• Contrastively Smoothed Class Alignment for Unsupervised Domain Adaptation. Shuyang Dai, Yu Cheng, Yizhe Zhang, Zhe Gan, Jingjing Liu, Lawrence Carin. ACCV 2020
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling. Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, Graham Neubig. AAAI 2020