a hidden state is worth a thousand words.

About me

I am Bokai Xu, I received bachelor degree at CUHK Shenzhen. After that I work as an engineer at Modelbest Inc. My long-term passion is understanding the connection between deep neural networks and biological brain, and the nature of human conciousness. But currently, I do works in omnimodal streaming language models, which is a foundation for further research.

Open for collaboration

If you have any questions about our work or would like to collaborate, please feel free to contact via Email / Email / WeChat. Also, our team at Modelbest Inc. and THUNLP is open for undergraduate/graduate research internship or engineering in omnimodal streaming LLM / multimodal LLM / speech LLM.

News

• Jan 19, 2025: Our omni foundation model, MiniCPM-o-2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone released, where I contributed to end-to-end speech pretraining & streaming speech generation and advanced voice mode.

• Oct 14, 2024: Our paper, pure VLM-based RAG VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents released, contributed to training and evaluation framework of VLM-based dense retrieval models.

• Sep 4, 2024: Our work, text dense retrieval model MiniCPM-Embedding released, contributed to contrastive learning train & eval framework.

• Aug 5, 2024: Our model MiniCPM-V-2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone released, where I contributed to enhanced OCR capability and multi-image understanding capability.

• July 1, 2024: Converted to full-time machine learning engineer at Modelbest Inc., my leader is Dr. Yuan Yao, and my partner is Mr. Junbo Cui.

• Jan 9, 2024: Joined Modelbest Inc. as a machine learning intern, my mentors are Dr. Huadong Wang and Mr. Shi Yu.

• May 23, 2023: Our paper UltraChat: Enhancing Chat Language Models by Scaling High-quality Instructional Conversations released.

• Apr 17 2023: Our paper Tool Learning with Foundation Models released.

• Apr 3, 2023: Joined ICBI@SIAT as a part-time research assistant. I work closely with Dr. Chaoyu Yang and Prof. Pengcheng Zhou on brain reconstruction.

• June 1, 2022: Joined THUNLP@THU as a part-time research assistant, my mentor is Dr. Huadong Wang.