Yiren Jian

I am a research scientist at ByteDance Inc, specializing in multi-modal foundational models. I earned my PhD in Computer Science from Dartmouth College in January 2024, where I worked on Machine Learning research under the mentorship of Soroush Vosoughi. During my PhD study, I have been research scientist interns at NEC Labs America (2021), Snap Research (2022) and ByteDance AML (2023).

During the initial three years of my program at Dartmouth, I collaborated closely with Lorenzo Torresani on Computer Vision. Prior to attending Dartmouth, I earned a MS degree in Biophysics with Chen Zeng at The George Washington University.

Research

My research spans a wide range of topics within the field of machine learning, focusing on computer vision, natural language processing and computational science. More specifically, I have first-author works on:

Pretraining generative visual-language [NeurIPS] and video-language [arXiv] models
Multi-modal sentence embeddings [NeurIPS]
Few-shot Learning in vision [AAAI] and language [NAACL, NAACL]
AI for science: learning structural patterns in RNA [BMC Bioinformatics, arXiv] and RNA-protein complexes [Nature Communications], and modeling the T-cell peptide interactions [KDD]

Full publication records can be found on my Google Scholar.

	InfiMM: Advancing Multimodal Understanding with an Open-Sourced Visual Language Model Haogeng Liu, Quanzeng You, Yiqi Wang, Xiaotian Han, Bohan Zhai, Yongfei Liu, Wentao Chen, Yiren Jian, Yunzhe Tao, Jianbo Yuan, Ran He, Hongxia Yang arXiv, 2024 code / bibtex
	Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction Yiren Jian, Tingkai Liu, Yunzhe Tao, Chunhui Zhang, Soroush Vosoughi, Hongxia Yang arXiv, 2023 code / bibtex
	GEM: Generating Engaging Multimodal Content Chongyang Gao, Yiren Jian, Natalia Denisenko, Soroush Vosoughi, V.S. Subrahmanian International Joint Conference on Artificial Intelligence (IJCAI), 2024 code / bibtex
	Bootstrapping Vision-Language Learning with Decoupled Language Pre-training Yiren Jian, Chongyang Gao, Soroush Vosoughi Neural Information Processing Systems (NeurIPS), 2023 (Spotlight) code / poster / slides / bibtex
	Knowledge from Large-Scale Protein Contact Prediction Models Can be Transferred to the Data-Scarce RNA Contact Prediction Task Yiren Jian^†, Chongyang Gao, Chen Zeng, Yunjie Zhao, Soroush Vosoughi^† arXiv, 2023 code / bibtex
	Evaluating Native-like Structures of RNA-protein Complexes Through the Deep Learning Method Chengwei Zeng^#, Yiren Jian^#, Soroush Vosoughi, Chen Zeng, Yunjie Zhao Nature Communications, 2023 code / bibtex
	Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings Yiren Jian, Chongyang Gao, Soroush Vosoughi Neural Information Processing Systems (NeurIPS), 2022 code / poster / slides / video / bibtex
	T-Cell Receptor-Peptide Interaction Prediction with Physical Model Augmented Pseudo-Labeling Yiren Jian, Erik Kruus, Martin Renqiang Min ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022 (Oral Presentation) code / poster / slides / bibtex
	Embedding Hallucination for Few-shot Language Fine-tuning Yiren Jian, Chongyang Gao, Soroush Vosoughi Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), 2022 code / poster / slides / video / bibtex
	Contrastive Learning for Prompt-based Few-shot Language Learners Yiren Jian, Chongyang Gao, Soroush Vosoughi Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), 2022 code / poster / slides / video / bibtex
	Label Hallucination for Few-shot Classification Yiren Jian, Lorenzo Torresani AAAI Conference on Artificial Intelligence (AAAI), 2022 code / poster / slides / video / appendix / bibtex
	MetaPix: Domain Transfer for Semantic Segmentation by Meta Pixel Weighting Yiren Jian, Chongyang Gao Image and Vision Computing, 2021 code / bibtex
	Task Meta-Transfer from Limited Parallel Labels Yiren Jian, Karim Ahmed, Lorenzo Torresani NeurIPS (Meta-Learning Workshop), 2020 code / poster / video / bibtex
	DIRECT: RNA Contact Predictions by Interating Structural Patterns Yiren Jian, Xiaonan Wang, Jiadi Qiu, Huiwen Wang, Zhichao Liu, Yunjie Zhao, Chen Zeng BMC Bioinformatics, 2019 bibtex