Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models

Authors: Hu, Yushi, Stretcu, Otilia, Lu, Chun-Ta, Viswanathan, Krishnamurthy, Hata, Kenji, Luo, Enming, Krishna, Ranjay, Fuxman, Ariel

Source: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2024 IEEE/CVF Conference on. :9590-9601 Jun, 2024

Relation: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

Authors: Hu, Yushi, Liu, Benlin, Kasai, Jungo, Wang, Yizhong, Ostendorf, Mari, Krishna, Ranjay, Smith, Noah A.

Source: 2023 IEEE/CVF International Conference on Computer Vision (ICCV) ICCV Computer Vision (ICCV), 2023 IEEE/CVF International Conference on. :20349-20360 Oct, 2023

Relation: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

PromptCap: Prompt-Guided Image Captioning for VQA with GPT-3

Authors: Hu, Yushi, Hua, Hang, Yang, Zhengyuan, Shi, Weijia, Smith, Noah A., Luo, Jiebo

Source: 2023 IEEE/CVF International Conference on Computer Vision (ICCV) ICCV Computer Vision (ICCV), 2023 IEEE/CVF International Conference on. :2951-2963 Oct, 2023

Relation: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

Decoding-Time Language Model Alignment with Multiple Objectives

Authors: Shi, Ruizhe, Chen, Yifang, Hu, Yushi, Liu, Alisa, Hajishirzi, Hannaneh, Smith, Noah A., Du, Simon S.

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2406.18853

Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation

Authors: Collins, Katherine M., Kim, Najoung, Bitton, Yonatan, Rieser, Verena, Omidshafiei, Shayegan, Hu, Yushi, Chen, Sherol, Dutta, Senjuti, Chang, Minsuk, Lee, Kimin, Liang, Youwei, Evans, Georgina, Singla, Sahil, Li, Gang, Weller, Adrian, He, Junfeng, Ramachandran, Deepak, Dvijotham, Krishnamurthy Dj

Subject Terms: Computer Science - Machine Learning, Computer Science - Computation and Language, Computer Science - Computer Vision and Pattern Recognition

Access URL: http://arxiv.org/abs/2406.16807

Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models

Authors: Hu, Yushi, Shi, Weijia, Fu, Xingyu, Roth, Dan, Ostendorf, Mari, Zettlemoyer, Luke, Smith, Noah A, Krishna, Ranjay

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2406.09403

BLINK: Multimodal Large Language Models Can See but Not Perceive

Authors: Fu, Xingyu, Hu, Yushi, Li, Bangzheng, Feng, Yu, Wang, Haoyu, Lin, Xudong, Roth, Dan, Smith, Noah A., Ma, Wei-Chiu, Krishna, Ranjay

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2404.12390

Training Language Models to Generate Text with Citations via Fine-grained Rewards

Authors: Huang, Chengyu, Wu, Zeqiu, Hu, Yushi, Wang, Wenya

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.04315

Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models

Authors: Hu, Yushi, Stretcu, Otilia, Lu, Chun-Ta, Viswanathan, Krishnamurthy, Hata, Kenji, Luo, Enming, Krishna, Ranjay, Fuxman, Ariel

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2312.03052

Two Watts is All You Need: Enabling In-Detector Real-Time Machine Learning for Neutrino Telescopes Via Edge Computing

Authors: Jin, Miaochen, Hu, Yushi, Argüelles, Carlos A.

Subject Terms: High Energy Physics - Experiment, Astrophysics - Instrumentation and Methods for Astrophysics

Access URL: http://arxiv.org/abs/2311.04983

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation

Authors: Cho, Jaemin, Hu, Yushi, Garg, Roopal, Anderson, Peter, Krishna, Ranjay, Baldridge, Jason, Bansal, Mohit, Pont-Tuset, Jordi, Wang, Su

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2310.18235

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

Authors: Wu, Zeqiu, Hu, Yushi, Shi, Weijia, Dziri, Nouha, Suhr, Alane, Ammanabrolu, Prithviraj, Smith, Noah A., Ostendorf, Mari, Hajishirzi, Hannaneh

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2306.01693

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

Authors: Su, Hongjin, Shi, Weijia, Kasai, Jungo, Wang, Yizhong, Hu, Yushi, Ostendorf, Mari, Yih, Wen-tau, Smith, Noah A., Zettlemoyer, Luke, Yu, Tao

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2212.09741

PromptCap: Prompt-Guided Task-Aware Image Captioning

Authors: Hu, Yushi, Hua, Hang, Yang, Zhengyuan, Shi, Weijia, Smith, Noah A, Luo, Jiebo

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2211.09699

Binding Language Models in Symbolic Languages

Authors: Cheng, Zhoujun, Xie, Tianbao, Shi, Peng, Li, Chengzu, Nadkarni, Rahul, Hu, Yushi, Xiong, Caiming, Radev, Dragomir, Ostendorf, Mari, Zettlemoyer, Luke, Smith, Noah A., Yu, Tao

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2210.02875

Acoustic Span Embeddings for Multilingual Query-by-Example Search

Authors: Hu, Yushi, Settle, Shane, Livescu, Karen

Source: 2021 IEEE Spoken Language Technology Workshop (SLT) Spoken Language Technology Workshop (SLT), 2021 IEEE. :935-942 Jan, 2021

Relation: 2021 IEEE Spoken Language Technology Workshop (SLT)

Unsupervised Learning of Hierarchical Conversation Structure

Authors: Lu, Bo-Ru, Hu, Yushi, Cheng, Hao, Smith, Noah A., Ostendorf, Mari

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2205.12244

In-Context Learning for Few-Shot Dialogue State Tracking

Authors: Hu, Yushi, Lee, Chia-Hsuan, Xie, Tianbao, Yu, Tao, Smith, Noah A., Ostendorf, Mari

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2203.08568

Acoustic span embeddings for multilingual query-by-example search

Authors: Hu, Yushi, Settle, Shane, Livescu, Karen

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2011.11807

Multilingual Jointly Trained Acoustic and Written Word Embeddings

Authors: Hu, Yushi, Settle, Shane, Livescu, Karen

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2006.14007