Computation and Language 100
☆ LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
Shang Yang, Junxian Guo, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, Song Han
Large language models (LLMs) have shown remarkable potential in processing
long sequences, yet efficiently serving these long-context models remains
challenging due to the quadratic computational complexity of attention in the
prefilling stage and the large memory footprint of the KV cache in the decoding
stage. To address these issues, we introduce LServe, an efficient system that
accelerates long-sequence LLM serving via hybrid sparse attention. This method
unifies different hardware-friendly, structured sparsity patterns for both
prefilling and decoding attention into a single framework, where computations
on less important tokens are skipped block-wise. LServe demonstrates the
compatibility of static and dynamic sparsity in long-context LLM attention.
This design enables multiplicative speedups by combining these optimizations.
Specifically, we convert half of the attention heads to nearly free streaming
heads in both the prefilling and decoding stages. Additionally, we find that
only a constant number of KV pages is required to preserve long-context
capabilities, irrespective of context length. We then design a hierarchical KV
page selection policy that dynamically prunes KV pages based on query-centric
similarity. On average, LServe accelerates LLM prefilling by up to 2.9x and
decoding by 1.3-2.1x over vLLM, maintaining long-context accuracy. Code is
released at https://github.com/mit-han-lab/omniserve.
comment: Accepted by MLSys 2025. Code available at:
https://github.com/mit-han-lab/omniserve
☆ Interpretable Text Embeddings and Text Similarity Explanation: A Primer
Text embeddings and text embedding models are a backbone of many AI and NLP
systems, particularly those involving search. However, interpretability
challenges persist, especially in explaining obtained similarity scores, which
is crucial for applications requiring transparency. In this paper, we give a
structured overview of interpretability methods specializing in explaining
those similarity scores, an emerging research area. We study the methods'
individual ideas and techniques, evaluating their potential for improving
interpretability of text embeddings and explaining predicted similarities.
☆ Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning
Large language models (LLMs) often fail to ask effective questions under
uncertainty, making them unreliable in domains where proactive
information-gathering is essential for decisionmaking. We present ALFA, a
framework that improves LLM question-asking by (i) decomposing the notion of a
"good" question into a set of theory-grounded attributes (e.g., clarity,
relevance), (ii) controllably synthesizing attribute-specific question
variations, and (iii) aligning models via preference-based optimization to
explicitly learn to ask better questions along these fine-grained attributes.
Focusing on clinical reasoning as a case study, we introduce the MediQ-AskDocs
dataset, composed of 17k real-world clinical interactions augmented with 80k
attribute-specific preference pairs of follow-up questions, as well as a novel
expert-annotated interactive healthcare QA task to evaluate question-asking
abilities. Models aligned with ALFA reduce diagnostic errors by 56.6% on
MediQ-AskDocs compared to SOTA instruction-tuned LLMs, with a question-level
win-rate of 64.4% and strong generalizability. Our findings suggest that
explicitly guiding question-asking with structured, fine-grained attributes
offers a scalable path to improve LLMs, especially in expert application
domains.
comment: 22 pages, 8 figures, 8 tables
★ FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
Weilin Zhao, Tengyu Pan, Xu Han, Yudi Zhang, Ao Sun, Yuxiang Huang, Kaihuo Zhang, Weilun Zhao, Yuxuan Li, Jianyong Wang, Zhiyuan Liu, Maosong Sun
Speculative sampling has emerged as an important technique for accelerating
the auto-regressive generation process of large language models (LLMs) by
utilizing a draft-then-verify mechanism to produce multiple tokens per forward
pass. While state-of-the-art speculative sampling methods use only a single
layer and a language modeling (LM) head as the draft model to achieve
impressive layer compression, their efficiency gains are substantially reduced
for large-vocabulary LLMs, such as Llama-3-8B with a vocabulary of 128k tokens.
To address this, we present FR-Spec, a frequency-ranked speculative sampling
framework that optimizes draft candidate selection through vocabulary space
compression. By constraining the draft search to a frequency-prioritized token
subset, our method reduces LM Head computation overhead by 75% while ensuring
the equivalence of the final output distribution. Experiments across multiple
datasets demonstrate an average of 1.12$\times$ speedup over the
state-of-the-art speculative sampling method EAGLE-2.
☆ Prompt-to-Leaderboard
Evan Frick, Connor Chen, Joseph Tennyson, Tianle Li, Wei-Lin Chiang, Anastasios N. Angelopoulos, Ion Stoica
Large language model (LLM) evaluations typically rely on aggregated metrics
like accuracy or human preference, averaging across users and prompts. This
averaging obscures user- and prompt-specific variations in model performance.
To address this, we propose Prompt-to-Leaderboard (P2L), a method that produces
leaderboards specific to a prompt. The core idea is to train an LLM taking
natural language prompts as input to output a vector of Bradley-Terry
coefficients which are then used to predict the human preference vote. The
resulting prompt-dependent leaderboards allow for unsupervised task-specific
evaluation, optimal routing of queries to models, personalization, and
automated evaluation of model strengths and weaknesses. Data from Chatbot Arena
suggest that P2L better captures the nuanced landscape of language model
performance than the averaged leaderboard. Furthermore, our findings suggest
that P2L's ability to produce prompt-specific evaluations follows a power law
scaling similar to that observed in LLMs themselves. In January 2025, the
router we trained based on this methodology achieved the \#1 spot in the
Chatbot Arena leaderboard. Our code is available at this GitHub link:
https://github.com/lmarena/p2l.
☆ CLIPPER: Compression enables long-context synthetic data generation
LLM developers are increasingly reliant on synthetic data, but generating
high-quality data for complex long-context reasoning tasks remains challenging.
We introduce CLIPPER, a compression-based approach for generating synthetic
data tailored to narrative claim verification - a task that requires reasoning
over a book to verify a given claim. Instead of generating claims directly from
the raw text of the book, which results in artifact-riddled claims, CLIPPER
first compresses the book into chapter outlines and book summaries and then
uses these intermediate representations to generate complex claims and
corresponding chain-of-thoughts. Compared to naive approaches, CLIPPER produces
claims that are more valid, grounded, and complex. Using CLIPPER, we construct
a dataset of 19K synthetic book claims paired with their source texts and
chain-of-thought reasoning, and use it to fine-tune three open-weight models.
Our best model achieves breakthrough results on narrative claim verification
(from 28% to 76% accuracy on our test set) and sets a new state-of-the-art for
sub-10B models on the NoCha leaderboard. Further analysis shows that our models
generate more detailed and grounded chain-of-thought reasoning while also
improving performance on other narrative understanding tasks (e.g.,
NarrativeQA).
☆ GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks
Jianwen Luo, Yiming Huang, Jinxiang Meng, Fangyu Lei, Shizhu He, Xiao Liu, Shanshan Jiang, Bin Dong, Jun Zhao, Kang Liu
Large Language Models (LLMs) have shown great promise in tool-making, yet
existing frameworks often struggle to efficiently construct reliable toolsets
and are limited to single-task settings. To address these challenges, we
propose GATE (Graph-based Adaptive Tool Evolution), an adaptive framework that
dynamically constructs and evolves a hierarchical graph of reusable tools
across multiple scenarios. We evaluate GATE on open-ended tasks (Minecraft),
agent-based tasks (TextCraft, DABench), and code generation tasks (MATH, Date,
TabMWP). Our results show that GATE achieves up to 4.3x faster milestone
completion in Minecraft compared to the previous SOTA, and provides an average
improvement of 9.23% over existing tool-making methods in code generation tasks
and 10.03% in agent tasks. GATE demonstrates the power of adaptive evolution,
balancing tool quantity, complexity, and functionality while maintaining high
efficiency. Code and data are available at
\url{https://github.com/ayanami2003/GATE}.
comment: 8 pages of main text, 38 pages of appendices
☆ Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation
Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark
Reasoning about images with rich text, such as charts and documents, is a
critical application of vision-language models (VLMs). However, VLMs often
struggle in these domains due to the scarcity of diverse text-rich
vision-language data. To address this challenge, we present CoSyn, a framework
that leverages the coding capabilities of text-only large language models
(LLMs) to automatically create synthetic text-rich multimodal data. Given input
text describing a target domain (e.g., "nutrition fact labels"), CoSyn prompts
an LLM to generate code (Python, HTML, LaTeX, etc.) for rendering synthetic
images. With the underlying code as textual representations of the synthetic
images, CoSyn can generate high-quality instruction-tuning data, again relying
on a text-only LLM. Using CoSyn, we constructed a dataset comprising 400K
images and 2.7M rows of vision-language instruction-tuning data. Comprehensive
experiments on seven benchmarks demonstrate that models trained on our
synthetic data achieve state-of-the-art performance among competitive
open-source models, including Llama 3.2, and surpass proprietary models such as
GPT-4V and Gemini 1.5 Flash. Furthermore, CoSyn can produce synthetic pointing
data, enabling VLMs to ground information within input images, showcasing its
potential for developing multimodal agents capable of acting in real-world
environments.
comment: 20 pages, 19 figures, 9 tables, website:
https://yueyang1996.github.io/cosyn/
☆ Revealing and Mitigating Over-Attention in Knowledge Editing
Large Language Models have demonstrated superior performance across a wide
range of tasks, but they still exhibit undesirable errors due to incorrect
knowledge learned from the training data. To avoid this, knowledge editing
methods emerged to precisely edit the specific model knowledge via efficiently
modifying a very small percentage of parameters. % However, those methods can
lead to the problem of Specificity Failure: when the content related to the
edited knowledge occurs in the context, it can inadvertently corrupt other
pre-existing knowledge. However, those methods can lead to the problem of
Specificity Failure, where the existing knowledge and capabilities are severely
degraded due to editing. Our preliminary indicates that Specificity Failure
primarily stems from the model's attention heads assigning excessive attention
scores to entities related to the edited knowledge, thereby unduly focusing on
specific snippets within the context, which we denote as the Attention Drift
phenomenon. To mitigate such Attention Drift issue, we introduce a simple yet
effective method Selective Attention Drift Restriction}(SADR), which introduces
an additional regularization term during the knowledge editing process to
restrict changes in the attention weight distribution, thereby preventing undue
focus on the edited entity. Experiments on five frequently used strong LLMs
demonstrate the effectiveness of our method, where SADR can significantly
mitigate Specificity Failure in the predominant knowledge editing tasks.
★ Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
Multi-head Latent Attention (MLA) is an innovative architecture proposed by
DeepSeek, designed to ensure efficient and economical inference by
significantly compressing the Key-Value (KV) cache into a latent vector.
Compared to MLA, standard LLMs employing Multi-Head Attention (MHA) and its
variants such as Grouped-Query Attention (GQA) exhibit significant cost
disadvantages. Enabling well-trained LLMs (e.g., Llama) to rapidly adapt to MLA
without pre-training from scratch is both meaningful and challenging. This
paper proposes the first data-efficient fine-tuning method for transitioning
from MHA to MLA (MHA2MLA), which includes two key components: for partial-RoPE,
we remove RoPE from dimensions of queries and keys that contribute less to the
attention scores, for low-rank approximation, we introduce joint SVD
approximations based on the pre-trained parameters of keys and values. These
carefully designed strategies enable MHA2MLA to recover performance using only
a small fraction (0.3% to 0.6%) of the data, significantly reducing inference
costs while seamlessly integrating with compression techniques such as KV cache
quantization. For example, the KV cache size of Llama2-7B is reduced by 92.19%,
with only a 0.5% drop in LongBench performance.
comment: 16 pages, 8 figures
★ LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models
Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, Juanzi Li
Existing Large Vision-Language Models (LVLMs) can process inputs with context
lengths up to 128k visual and text tokens, yet they struggle to generate
coherent outputs beyond 1,000 words. We find that the primary limitation is the
absence of long output examples during supervised fine-tuning (SFT). To tackle
this issue, we introduce LongWriter-V-22k, a SFT dataset comprising 22,158
examples, each with multiple input images, an instruction, and corresponding
outputs ranging from 0 to 10,000 words. Moreover, to achieve long outputs that
maintain high-fidelity to the input images, we employ Direct Preference
Optimization (DPO) to the SFT model. Given the high cost of collecting human
feedback for lengthy outputs (e.g., 3,000 words), we propose IterDPO, which
breaks long outputs into segments and uses iterative corrections to form
preference pairs with the original outputs. Additionally, we develop
MMLongBench-Write, a benchmark featuring six tasks to evaluate the
long-generation capabilities of VLMs. Our 7B parameter model, trained with
LongWriter-V-22k and IterDPO, achieves impressive performance on this
benchmark, outperforming larger proprietary models like GPT-4o. Code and data:
https://github.com/THU-KEG/LongWriter-V
☆ Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs
While large language models demonstrate remarkable capabilities at
task-specific applications through fine-tuning, extending these benefits across
diverse languages is essential for broad accessibility. However, effective
cross-lingual transfer is hindered by LLM performance gaps across languages and
the scarcity of fine-tuning data in many languages. Through analysis of LLM
internal representations from over 1,000+ language pairs, we discover that
middle layers exhibit the strongest potential for cross-lingual alignment.
Building on this finding, we propose a middle-layer alignment objective
integrated into task-specific training. Our experiments on slot filling,
machine translation, and structured text generation show consistent
improvements in cross-lingual transfer, especially to lower-resource languages.
The method is robust to the choice of alignment languages and generalizes to
languages unseen during alignment. Furthermore, we show that separately trained
alignment modules can be merged with existing task-specific modules, improving
cross-lingual capabilities without full re-training. Our code is publicly
available (https://github.com/dannigt/mid-align).
☆ Measuring Faithfulness of Chains of Thought by Unlearning Reasoning Steps
When prompted to think step-by-step, language models (LMs) produce a chain of
thought (CoT), a sequence of reasoning steps that the model supposedly used to
produce its prediction. However, despite much work on CoT prompting, it is
unclear if CoT reasoning is faithful to the models' parameteric beliefs. We
introduce a framework for measuring parametric faithfulness of generated
reasoning, and propose Faithfulness by Unlearning Reasoning steps (FUR), an
instance of this framework. FUR erases information contained in reasoning steps
from model parameters. We perform experiments unlearning CoTs of four LMs
prompted on four multi-choice question answering (MCQA) datasets. Our
experiments show that FUR is frequently able to change the underlying models'
prediction by unlearning key steps, indicating when a CoT is parametrically
faithful. Further analysis shows that CoTs generated by models post-unlearning
support different answers, hinting at a deeper effect of unlearning.
Importantly, CoT steps identified as important by FUR do not align well with
human notions of plausbility, emphasizing the need for specialized alignment
☆ eC-Tab2Text: Aspect-Based Text Generation from e-Commerce Product Tables NAACL 2025
Large Language Models (LLMs) have demonstrated exceptional versatility across
diverse domains, yet their application in e-commerce remains underexplored due
to a lack of domain-specific datasets. To address this gap, we introduce
eC-Tab2Text, a novel dataset designed to capture the intricacies of e-commerce,
including detailed product attributes and user-specific queries. Leveraging
eC-Tab2Text, we focus on text generation from product tables, enabling LLMs to
produce high-quality, attribute-specific product reviews from structured
tabular data. Fine-tuned models were rigorously evaluated using standard
Table2Text metrics, alongside correctness, faithfulness, and fluency
assessments. Our results demonstrate substantial improvements in generating
contextually accurate reviews, highlighting the transformative potential of
tailored datasets and fine-tuning methodologies in optimizing e-commerce
workflows. This work highlights the potential of LLMs in e-commerce workflows
and the essential role of domain-specific datasets in tailoring them to
industry-specific challenges.
comment: NAACL 2025 (Industry Track)
☆ Optimizing Model Selection for Compound AI Systems
Compound AI systems that combine multiple LLM calls, such as self-refine and
multi-agent-debate, achieve strong performance on many AI tasks. We address a
core question in optimizing compound systems: for each LLM call or module in
the system, how should one decide which LLM to use? We show that these LLM
choices have a large effect on quality, but the search space is exponential. We
propose LLMSelector, an efficient framework for model selection in compound
systems, which leverages two key empirical insights: (i) end-to-end performance
is often monotonic in how well each module performs, with all other modules
held fixed, and (ii) per-module performance can be estimated accurately by an
LLM. Building upon these insights, LLMSelector iteratively selects one module
and allocates to it the model with the highest module-wise performance, as
estimated by an LLM, until no further gain is possible. LLMSelector is
applicable to any compound system with a bounded number of modules, and its
number of API calls scales linearly with the number of modules, achieving
high-quality model allocation both empirically and theoretically. Experiments
with popular compound systems such as multi-agent debate and self-refine using
LLMs such as GPT-4o, Claude 3.5 Sonnet and Gemini 1.5 show that LLMSelector
confers 5%-70% accuracy gains compared to using the same LLM for all modules.
☆ From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Our ability to continuously acquire, organize, and leverage knowledge is a
key feature of human intelligence that AI systems must approximate to unlock
their full potential. Given the challenges in continual learning with large
language models (LLMs), retrieval-augmented generation (RAG) has become the
dominant way to introduce new information. However, its reliance on vector
retrieval hinders its ability to mimic the dynamic and interconnected nature of
human long-term memory. Recent RAG approaches augment vector embeddings with
various structures like knowledge graphs to address some of these gaps, namely
sense-making and associativity. However, their performance on more basic
factual memory tasks drops considerably below standard RAG. We address this
unintended deterioration and propose HippoRAG 2, a framework that outperforms
standard RAG comprehensively on factual, sense-making, and associative memory
tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in
HippoRAG and enhances it with deeper passage integration and more effective
online use of an LLM. This combination pushes this RAG system closer to the
effectiveness of human long-term memory, achieving a 7% improvement in
associative memory tasks over the state-of-the-art embedding model while also
exhibiting superior factual knowledge and sense-making memory capabilities.
This work paves the way for non-parametric continual learning for LLMs. Our
code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.
comment: Code and data to be released at:
https://github.com/OSU-NLP-Group/HippoRAG
☆ Rapid Word Learning Through Meta In-Context Learning
Humans can quickly learn a new word from a few illustrative examples, and
then systematically and flexibly use it in novel contexts. Yet the abilities of
current language models for few-shot word learning, and methods for improving
these abilities, are underexplored. In this study, we introduce a novel method,
Meta-training for IN-context learNing Of Words (Minnow). This method trains
language models to generate new examples of a word's usage given a few
in-context examples, using a special placeholder token to represent the new
word. This training is repeated on many new words to develop a general
word-learning ability. We find that training models from scratch with Minnow on
human-scale child-directed language enables strong few-shot word learning,
comparable to a large language model (LLM) pre-trained on orders of magnitude
more data. Furthermore, through discriminative and generative evaluations, we
demonstrate that finetuning pre-trained LLMs with Minnow improves their ability
to discriminate between new words, identify syntactic categories of new words,
and generate reasonable new usages and definitions for new words, based on one
or a few in-context examples. These findings highlight the data efficiency of
Minnow and its potential to improve language model performance in word learning
tasks.
☆ ReVision: A Dataset and Baseline VLM for Privacy-Preserving Task-Oriented Visual Instruction Rewriting
Efficient and privacy-preserving multimodal interaction is essential as AR,
VR, and modern smartphones with powerful cameras become primary interfaces for
human-computer communication. Existing powerful large vision-language models
(VLMs) enabling multimodal interaction often rely on cloud-based processing,
raising significant concerns about (1) visual privacy by transmitting sensitive
vision data to servers, and (2) their limited real-time, on-device usability.
This paper explores Visual Instruction Rewriting, a novel approach that
transforms multimodal instructions into text-only commands, allowing seamless
integration of lightweight on-device instruction rewriter VLMs (250M
parameters) with existing conversational AI systems, enhancing vision data
privacy. To achieve this, we present a dataset of over 39,000 examples across
14 domains and develop a compact VLM, pretrained on image captioning datasets
and fine-tuned for instruction rewriting. Experimental results, evaluated
through NLG metrics such as BLEU, METEOR, and ROUGE, along with semantic
parsing analysis, demonstrate that even a quantized version of the model
(<500MB storage footprint) can achieve effective instruction rewriting, thus
enabling privacy-focused, multimodal AI applications.
comment: 12 pages, 7 figures, 3 tables
☆ Harnessing PDF Data for Improving Japanese Large Multimodal Models
Large Multimodal Models (LMMs) have demonstrated strong performance in
English, but their effectiveness in Japanese remains limited due to the lack of
high-quality training data. Current Japanese LMMs often rely on translated
English datasets, restricting their ability to capture Japan-specific cultural
knowledge. To address this, we explore the potential of Japanese PDF data as a
training resource, an area that remains largely underutilized. We introduce a
fully automated pipeline that leverages pretrained models to extract image-text
pairs from PDFs through layout analysis, OCR, and vision-language pairing,
removing the need for manual annotation. Additionally, we construct instruction
data from extracted image-text pairs to enrich the training data. To evaluate
the effectiveness of PDF-derived data, we train Japanese LMMs and assess their
performance on the Japanese LMM Benchmark. Our results demonstrate substantial
improvements, with performance gains ranging from 3.9% to 13.8% on Heron-Bench.
Further analysis highlights the impact of PDF-derived data on various factors,
such as model size and language models, reinforcing its value as a multimodal
resource for Japanese LMMs. We plan to make the source code and data publicly
available upon acceptance.
comment: 15 pages, 8 figures
☆ SurveyX: Academic Survey Automation via Large Language Models
Xun Liang, Jiawei Yang, Yezhaohui Wang, Chen Tang, Zifan Zheng, Simin Niu, Shichao Song, Hanyu Wang, Bo Tang, Feiyu Xiong, Keming Mao, Zhiyu li
Large Language Models (LLMs) have demonstrated exceptional comprehension
capabilities and a vast knowledge base, suggesting that LLMs can serve as
efficient tools for automated survey generation. However, recent research
related to automated survey generation remains constrained by some critical
limitations like finite context window, lack of in-depth content discussion,
and absence of systematic evaluation frameworks. Inspired by human writing
processes, we propose SurveyX, an efficient and organized system for automated
survey generation that decomposes the survey composing process into two phases:
the Preparation and Generation phases. By innovatively introducing online
reference retrieval, a pre-processing method called AttributeTree, and a
re-polishing process, SurveyX significantly enhances the efficacy of survey
composition. Experimental evaluation results show that SurveyX outperforms
existing automated survey generation systems in content quality (0.259
improvement) and citation quality (1.76 enhancement), approaching human expert
performance across multiple evaluation dimensions. Examples of surveys
generated by SurveyX are available on www.surveyx.cn
comment: 15 pages, 16 figures
☆ Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, Chong Luo
Inspired by the success of DeepSeek-R1, we explore the potential of
rule-based reinforcement learning (RL) in large reasoning models. To analyze
reasoning dynamics, we use synthetic logic puzzles as training data due to
their controllable complexity and straightforward answer verification. We make
some key technical contributions that lead to effective and stable RL training:
a system prompt that emphasizes the thinking and answering process, a stringent
format reward function that penalizes outputs for taking shortcuts, and a
straightforward training recipe that achieves stable convergence. Our 7B model
develops advanced reasoning skills-such as reflection, verification, and
summarization-that are absent from the logic corpus. Remarkably, after training
on just 5K logic problems, it demonstrates generalization abilities to the
challenging math benchmarks AIME and AMC.
☆ Tree-of-Debate: Multi-Persona Debate Trees Elicit Critical Thinking for Scientific Comparative Analysis
With the exponential growth of research facilitated by modern technology and
improved accessibility, scientific discoveries have become increasingly
fragmented within and across fields. This makes it challenging to assess the
significance, novelty, incremental findings, and equivalent ideas between
related works, particularly those from different research communities. Large
language models (LLMs) have recently demonstrated strong quantitative and
qualitative reasoning abilities, and multi-agent LLM debates have shown promise
in handling complex reasoning tasks by exploring diverse perspectives and
reasoning paths. Inspired by this, we introduce Tree-of-Debate (ToD), a
framework which converts scientific papers into LLM personas that debate their
respective novelties. To emphasize structured, critical reasoning rather than
focusing solely on outcomes, ToD dynamically constructs a debate tree, enabling
fine-grained analysis of independent novelty arguments within scholarly
articles. Through experiments on scientific literature across various domains,
evaluated by expert researchers, we demonstrate that ToD generates informative
arguments, effectively contrasts papers, and supports researchers in their
literature review.
comment: Code available at: https://github.com/pkargupta/tree-of-debate
☆ Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning NAACL 2025
Fact verification (FV) aims to assess the veracity of a claim based on
relevant evidence. The traditional approach for automated FV includes a
three-part pipeline relying on short evidence snippets and encoder-only
inference models. More recent approaches leverage the multi-turn nature of LLMs
to address FV as a step-by-step problem where questions inquiring additional
context are generated and answered until there is enough information to make a
decision. This iterative method makes the verification process rational and
explainable. While these methods have been tested for encyclopedic claims,
exploration on domain-specific and realistic claims is missing. In this work,
we apply an iterative FV system on three medical fact-checking datasets and
evaluate it with multiple settings, including different LLMs, external web
search, and structured reasoning using logic predicates. We demonstrate
improvements in the final performance over traditional approaches and the high
potential of step-by-step FV systems for domain-specific claims.
comment: Accepted to NAACL 2025 (Main)
☆ On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems NAACL 2025
Retrieval-augmented generation (RAG) has emerged as an approach to augment
large language models (LLMs) by reducing their reliance on static knowledge and
improving answer factuality. RAG retrieves relevant context snippets and
generates an answer based on them. Despite its increasing industrial adoption,
systematic exploration of RAG components is lacking, particularly regarding the
ideal size of provided context, and the choice of base LLM and retrieval
method. To help guide development of robust RAG systems, we evaluate various
context sizes, BM25 and semantic search as retrievers, and eight base LLMs.
Moving away from the usual RAG evaluation with short answers, we explore the
more challenging long-form question answering in two domains, where a good
answer has to utilize the entire context. Our findings indicate that final QA
performance improves steadily with up to 15 snippets but stagnates or declines
beyond that. Finally, we show that different general-purpose LLMs excel in the
biomedical domain than the encyclopedic one, and that open-domain evidence
retrieval in large corpora is challenging.
comment: Accepted to Findings of NAACL 2025
★ TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
Jianling Li, Shangzhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, Haojie Wang, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun
Triton, a high-level Python-like language designed for building efficient GPU
kernels, is widely adopted in deep learning frameworks due to its portability,
flexibility, and accessibility. However, programming and parallel optimization
still require considerable trial and error from Triton developers. Despite
advances in large language models (LLMs) for conventional code generation,
these models struggle to generate accurate, performance-optimized Triton code,
as they lack awareness of its specifications and the complexities of GPU
programming. More critically, there is an urgent need for systematic
evaluations tailored to Triton. In this work, we introduce TritonBench, the
first comprehensive benchmark for Triton operator generation. TritonBench
features two evaluation channels: a curated set of 184 real-world operators
from GitHub and a collection of operators aligned with PyTorch interfaces.
Unlike conventional code benchmarks prioritizing functional correctness,
TritonBench also profiles efficiency performance on widely deployed GPUs
aligned with industry applications. Our study reveals that current
state-of-the-art code LLMs struggle to generate efficient Triton operators,
highlighting a significant gap in high-performance code generation. TritonBench
will be available at https://github.com/thunlp/TritonBench.
☆ Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of LLMs
Zongxia Li, Lorena Calvo-Bartolomé, Alexander Hoyle, Paiheng Xu, Alden Dima, Juan Francisco Fung, Jordan Boyd-Graber
A common use of NLP is to facilitate the understanding of large document
collections, with a shift from using traditional topic models to Large Language
Models. Yet the effectiveness of using LLM for large corpus understanding in
real-world applications remains under-explored. This study measures the
knowledge users acquire with unsupervised, supervised LLM-based exploratory
approaches or traditional topic models on two datasets. While LLM-based methods
generate more human-readable topics and show higher average win probabilities
than traditional models for data exploration, they produce overly generic
topics for domain-specific datasets that do not easily allow users to learn
much about the documents. Adding human supervision to the LLM generation
process improves data exploration by mitigating hallucination and
over-genericity but requires greater human effort. In contrast, traditional.
models like Latent Dirichlet Allocation (LDA) remain effective for exploration
but are less user-friendly. We show that LLMs struggle to describe the haystack
of large corpora without human help, particularly domain-specific data, and
face scaling and hallucination limitations due to context length constraints.
Dataset available at https://huggingface. co/datasets/zli12321/Bills.
comment: 21 Pages. LLM for Data Exploration and content analysis
☆ HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States
The integration of additional modalities increases the susceptibility of
large vision-language models (LVLMs) to safety risks, such as jailbreak
attacks, compared to their language-only counterparts. While existing research
primarily focuses on post-hoc alignment techniques, the underlying safety
mechanisms within LVLMs remain largely unexplored. In this work , we
investigate whether LVLMs inherently encode safety-relevant signals within
their internal activations during inference. Our findings reveal that LVLMs
exhibit distinct activation patterns when processing unsafe prompts, which can
be leveraged to detect and mitigate adversarial inputs without requiring
extensive fine-tuning. Building on this insight, we introduce HiddenDetect, a
novel tuning-free framework that harnesses internal model activations to
enhance safety. Experimental results show that {HiddenDetect} surpasses
state-of-the-art methods in detecting jailbreak attacks against LVLMs. By
utilizing intrinsic safety-aware patterns, our method provides an efficient and
scalable solution for strengthening LVLM robustness against multimodal threats.
Our code will be released publicly at
https://github.com/leigest519/HiddenDetect.
☆ SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
M-A-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, Kang Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, Chujie Zheng, Kaixing Deng, Shuyue Guo, Shian Jia, Sichao Jiang, Yiyan Liao, Rui Li, Qinrui Li, Sirun Li, Yizhi Li, Yunwen Li, Dehua Ma, Yuansheng Ni, Haoran Que, Qiyao Wang, Zhoufutu Wen, Siwei Wu, Tianshun Xing, Ming Xu, Zhenzhu Yang, Zekun Moore Wang, Junting Zhou, Yuelin Bai, Xingyuan Bu, Chenglin Cai, Liang Chen, Yifan Chen, Chengtuo Cheng, Tianhao Cheng, Keyi Ding, Siming Huang, Yun Huang, Yaoru Li, Yizhe Li, Zhaoqun Li, Tianhao Liang, Chengdong Lin, Hongquan Lin, Yinghao Ma, Zhongyuan Peng, Zifan Peng, Qige Qi, Shi Qiu, Xingwei Qu, Yizhou Tan, Zili Wang, Chenqing Wang, Hao Wang, Yiya Wang, Yubo Wang, Jiajun Xu, Kexin Yang, Ruibin Yuan, Yuanhao Yue, Tianyang Zhan, Chun Zhang, Jingyang Zhang, Xiyue Zhang, Xingjian Zhang, Yue Zhang, Yongchi Zhao, Xiangyu Zheng, Chenghua Zhong, Yang Gao, Zhoujun Li, Dayiheng Liu, Qian Liu, Tianyu Liu, Shiwen Ni, Junran Peng, Yujia Qin, Wenbo Su, Guoyin Wang, Shi Wang, Jian Yang, Min Yang, Meng Cao, Xiang Yue, Zhaoxiang Zhang, Wangchunshu Zhou, Jiaheng Liu, Qunshu Lin, Wenhao Huang, Ge Zhang
Large language models (LLMs) have demonstrated remarkable proficiency in
mainstream academic disciplines such as mathematics, physics, and computer
science. However, human knowledge encompasses over 200 specialized disciplines,
far exceeding the scope of existing benchmarks. The capabilities of LLMs in
many of these specialized fields-particularly in light industry, agriculture,
and service-oriented disciplines-remain inadequately evaluated. To address this
gap, we present SuperGPQA, a comprehensive benchmark that evaluates
graduate-level knowledge and reasoning capabilities across 285 disciplines. Our
benchmark employs a novel Human-LLM collaborative filtering mechanism to
eliminate trivial or ambiguous questions through iterative refinement based on
both LLM responses and expert feedback. Our experimental results reveal
significant room for improvement in the performance of current state-of-the-art
LLMs across diverse knowledge domains (e.g., the reasoning-focused model
DeepSeek-R1 achieved the highest accuracy of 61.82% on SuperGPQA), highlighting
the considerable gap between current model capabilities and artificial general
intelligence. Additionally, we present comprehensive insights from our
management of a large-scale annotation process, involving over 80 expert
annotators and an interactive Human-LLM collaborative system, offering valuable
methodological guidance for future research initiatives of comparable scope.
☆ Sentence Smith: Formally Controllable Text Transformation and its Application to Evaluation of Text Embedding Models
We propose the Sentence Smith framework that enables controlled and specified
manipulation of text meaning. It consists of three main steps: 1. Parsing a
sentence into a semantic graph, 2. Applying human-designed semantic
manipulation rules, and 3. Generating text from the manipulated graph. A final
filtering step (4.) ensures the validity of the applied transformation. To
demonstrate the utility of Sentence Smith in an application study, we use it to
generate hard negative pairs that challenge text embedding models. Since the
controllable generation makes it possible to clearly isolate different types of
semantic shifts, we can gain deeper insights into the specific strengths and
weaknesses of widely used text embedding models, also addressing an issue in
current benchmarking where linguistic phenomena remain opaque. Human validation
confirms that the generations produced by Sentence Smith are highly accurate.
☆ Entity Framing and Role Portrayal in the News ACL
Tarek Mahmoud, Zhuohan Xie, Dimitar Dimitrov, Nikolaos Nikolaidis, Purificação Silvano, Roman Yangarber, Shivam Sharma, Elisa Sartori, Nicolas Stefanovitch, Giovanni Da San Martino, Jakub Piskorski, Preslav Nakov
We introduce a novel multilingual hierarchical corpus annotated for entity
framing and role portrayal in news articles. The dataset uses a unique taxonomy
inspired by storytelling elements, comprising 22 fine-grained roles, or
archetypes, nested within three main categories: protagonist, antagonist, and
innocent. Each archetype is carefully defined, capturing nuanced portrayals of
entities such as guardian, martyr, and underdog for protagonists; tyrant,
deceiver, and bigot for antagonists; and victim, scapegoat, and exploited for
innocents. The dataset includes 1,378 recent news articles in five languages
(Bulgarian, English, Hindi, European Portuguese, and Russian) focusing on two
critical domains of global significance: the Ukraine-Russia War and Climate
Change. Over 5,800 entity mentions have been annotated with role labels. This
dataset serves as a valuable resource for research into role portrayal and has
broader implications for news analysis. We describe the characteristics of the
dataset and the annotation process, and we report evaluation results on
fine-tuned state-of-the-art multilingual transformers and hierarchical
zero-shot learning using LLMs at the level of a document, a paragraph, and a
sentence.
comment: 23 pages, 12 figures. Submitted to ACL Rolling Review (ARR)
☆ From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT
The generative capabilities of LLM models present opportunities in
accelerating tasks and concerns with the authenticity of the knowledge it
produces. To address the concerns, we present a computational approach that
systematically evaluates the factual accuracy of biomedical knowledge that an
LLM model has been prompted to generate. Our approach encompasses two
processes: the generation of disease-centric associations and the verification
of them using the semantic knowledge of the biomedical ontologies. Using
ChatGPT as the select LLM model, we designed a set of prompt-engineering
processes to generate linkages between diseases, drugs, symptoms, and genes to
establish grounds for assessments. Experimental results demonstrate high
accuracy in identifying disease terms (88%-97%), drug names (90%-91%), and
genetic information (88%-98%). The symptom term identification accuracy was
notably lower (49%-61%), as verified against the DOID, ChEBI, SYMPTOM, and GO
ontologies accordingly. The verification of associations reveals literature
coverage rates of (89%-91%) among disease-drug and disease-gene associations.
The low identification accuracy for symptom terms also contributed to the
verification of symptom-related associations (49%-62%).
comment: 26 pages, 6 figures, In Review with a Cell Press Journal
☆ Data-Efficient Pretraining with Group-Level Data Influence Modeling
Data-efficient pretraining has shown tremendous potential to elevate scaling
laws. This paper argues that effective pretraining data should be curated at
the group level, treating a set of data points as a whole rather than as
independent contributors. To achieve that, we propose Group-Level Data
Influence Modeling (Group-MATES), a novel data-efficient pretraining method
that captures and optimizes group-level data utility. Specifically, Group-MATES
collects oracle group-level influences by locally probing the pretraining model
with data sets. It then fine-tunes a relational data influence model to
approximate oracles as relationship-weighted aggregations of individual
influences. The fine-tuned model selects the data subset by maximizing its
group-level influence prediction, with influence-aware clustering to enable
efficient inference. Experiments on the DCLM benchmark demonstrate that
Group-MATES achieves a 10% relative core score improvement on 22 downstream
tasks over DCLM-Baseline and 5% over individual-influence-based methods,
establishing a new state-of-the-art. Further analyses highlight the
effectiveness of relational data influence models in capturing intricate
interactions between data points.
☆ I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search
Recent advancements in large language models (LLMs) have shown remarkable
potential in automating machine learning tasks. However, existing LLM-based
agents often struggle with low-diversity and suboptimal code generation. While
recent work has introduced Monte Carlo Tree Search (MCTS) to address these
issues, limitations persist in the quality and diversity of thoughts generated,
as well as in the scalar value feedback mechanisms used for node selection. In
this study, we introduce Introspective Monte Carlo Tree Search (I-MCTS), a
novel approach that iteratively expands tree nodes through an introspective
process that meticulously analyzes solutions and results from parent and
sibling nodes. This facilitates a continuous refinement of the node in the
search tree, thereby enhancing the overall decision-making process.Furthermore,
we integrate a Large Language Model (LLM)-based value model to facilitate
direct evaluation of each node's solution prior to conducting comprehensive
computational rollouts. A hybrid rewarding mechanism is implemented to
seamlessly transition the Q-value from LLM-estimated scores to actual
performance scores. This allows higher-quality nodes to be traversed
earlier.Applied to the various ML tasks, our approach demonstrates a6\%
absolute improvement in performance compared to the strong open-source AutoML
agents, showcasing its effectiveness in enhancing agentic AutoML systems.
☆ Bridging the Gap: Transforming Natural Language Questions into SQL Queries via Abstract Query Pattern and Contextual Schema Markup
Large language models have demonstrated excellent performance in many tasks,
including Text-to-SQL, due to their powerful in-context learning capabilities.
They are becoming the mainstream approach for Text-to-SQL. However, these
methods still have a significant gap compared to human performance, especially
on complex questions. As the complexity of questions increases, the gap between
questions and SQLs increases. We identify two important gaps: the structural
mapping gap and the lexical mapping gap. To tackle these two gaps, we propose
PAS-SQL, an efficient SQL generation pipeline based on LLMs, which alleviates
gaps through Abstract Query Pattern (AQP) and Contextual Schema Markup (CSM).
AQP aims to obtain the structural pattern of the question by removing
database-related information, which enables us to find structurally similar
demonstrations. CSM aims to associate database-related text span in the
question with specific tables or columns in the database, which alleviates the
lexical mapping gap. Experimental results on the Spider and BIRD datasets
demonstrate the effectiveness of our proposed method. Specifically, PAS-SQL +
GPT-4o sets a new state-of-the-art on the Spider benchmark with an execution
accuracy of 87.9\%, and achieves leading results on the BIRD dataset with an
execution accuracy of 64.67\%.
☆ How to Get Your LLM to Generate Challenging Problems for Evaluation
The pace of evolution of Large Language Models (LLMs) necessitates new
approaches for rigorous and comprehensive evaluation. Traditional human
annotation is increasingly impracticable due to the complexities and costs
involved in generating high-quality, challenging problems. In this work, we
introduce CHASE, a unified framework to synthetically generate challenging
problems using LLMs without human involvement. For a given task, our approach
builds a hard problem in a bottom-up manner from simpler components. Moreover,
our framework decomposes the generation process into independently verifiable
sub-tasks, thereby ensuring a high level of quality and correctness. We
implement CHASE to create evaluation benchmarks across three diverse domains:
(1) document-based question answering, (2) repository-level code completion,
and (3) math reasoning. The performance of state-of-the-art LLMs on these
synthetic benchmarks lies in the range of 40-60% accuracy, thereby
demonstrating the effectiveness of our framework at generating challenging
problems. We publicly release our benchmarks and code.
☆ Data-Constrained Synthesis of Training Data for De-Identification
Many sensitive domains -- such as the clinical domain -- lack widely
available datasets due to privacy risks. The increasing generative capabilities
of large language models (LLMs) have made synthetic datasets a viable path
forward. In this study, we domain-adapt LLMs to the clinical domain and
generate synthetic clinical texts that are machine-annotated with tags for
personally identifiable information using capable encoder-based NER models. The
synthetic corpora are then used to train synthetic NER models. The results show
that training NER models using synthetic corpora incurs only a small drop in
predictive performance. The limits of this process are investigated in a
systematic ablation study -- using both Swedish and Spanish data. Our analysis
shows that smaller datasets can be sufficient for domain-adapting LLMs for data
synthesis. Instead, the effectiveness of this process is almost entirely
contingent on the performance of the machine-annotating NER models trained
using the original data.
comment: Under review
☆ Explanations of Deep Language Models Explain Language Representations in the Brain
Recent advances in artificial intelligence have given rise to large language
models (LLMs) that not only achieve human-like performance but also share
computational principles with the brain's language processing mechanisms. While
previous research has primarily focused on aligning LLMs' internal
representations with neural activity, we introduce a novel approach that
leverages explainable AI (XAI) methods to forge deeper connections between the
two domains. Using attribution methods, we quantified how preceding words
contribute to an LLM's next-word predictions and employed these explanations to
predict fMRI recordings from participants listening to the same narratives. Our
findings demonstrate that attribution methods robustly predict brain activity
across the language network, surpassing traditional internal representations in
early language areas. This alignment is hierarchical: early-layer explanations
correspond to the initial stages of language processing in the brain, while
later layers align with more advanced stages. Moreover, the layers more
influential on LLM next-word prediction$\unicode{x2014}$those with higher
attribution scores$\unicode{x2014}$exhibited stronger alignment with neural
activity. This work establishes a bidirectional bridge between AI and
neuroscience. First, we demonstrate that attribution methods offer a powerful
lens for investigating the neural mechanisms of language comprehension,
revealing how meaning emerges from preceding context. Second, we propose using
brain alignment as a metric to evaluate the validity of attribution methods,
providing a framework for assessing their biological plausibility.
☆ AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO
Large Language Models (LLMs) have demonstrated impressive capabilities in
language processing, yet they often struggle with tasks requiring genuine
visual spatial reasoning. In this paper, we introduce a novel two-stage
training framework designed to equip standard LLMs with visual reasoning
abilities for maze navigation. First, we leverage Supervised Fine Tuning (SFT)
on a curated dataset of tokenized maze representations to teach the model to
predict step-by-step movement commands. Next, we apply Group Relative Policy
Optimization (GRPO)-a technique used in DeepSeekR1-with a carefully crafted
reward function to refine the model's sequential decision-making and encourage
emergent chain-of-thought behaviors. Experimental results on synthetically
generated mazes show that while a baseline model fails to navigate the maze,
the SFT-trained model achieves 86% accuracy, and further GRPO fine-tuning
boosts accuracy to 93%. Qualitative analyses reveal that GRPO fosters more
robust and self-corrective reasoning, highlighting the potential of our
approach to bridge the gap between language models and visual spatial tasks.
These findings offer promising implications for applications in robotics,
autonomous navigation, and other domains that require integrated visual and
sequential reasoning.
☆ InstructAgent: Building User Controllable Recommender via LLM Agent WWW2025
Wujiang Xu, Yunxiao Shi, Zujie Liang, Xuying Ning, Kai Mei, Kun Wang, Xi Zhu, Min Xu, Yongfeng Zhang
Traditional recommender systems usually take the user-platform paradigm,
where users are directly exposed under the control of the platform's
recommendation algorithms. However, the defect of recommendation algorithms may
put users in very vulnerable positions under this paradigm. First, many
sophisticated models are often designed with commercial objectives in mind,
focusing on the platform's benefits, which may hinder their ability to protect
and capture users' true interests. Second, these models are typically optimized
using data from all users, which may overlook individual user's preferences.
Due to these shortcomings, users may experience several disadvantages under the
traditional user-platform direct exposure paradigm, such as lack of control
over the recommender system, potential manipulation by the platform, echo
chamber effects, or lack of personalization for less active users due to the
dominance of active users during collaborative learning. Therefore, there is an
urgent need to develop a new paradigm to protect user interests and alleviate
these issues. Recently, some researchers have introduced LLM agents to simulate
user behaviors, these approaches primarily aim to optimize platform-side
performance, leaving core issues in recommender systems unresolved. To address
these limitations, we propose a new user-agent-platform paradigm, where agent
serves as the protective shield between user and recommender system that
enables indirect exposure. To this end, we first construct four recommendation
datasets, denoted as $\dataset$, along with user instructions for each record.
comment: WWW2025@HCRS
☆ Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
Knowledge editing allows for efficient adaptation of large language models
(LLMs) to new information or corrections without requiring full retraining.
However, prior methods typically focus on either single-language editing or
basic multilingual editing, failing to achieve true cross-linguistic knowledge
synchronization. To address this, we present a simple and practical
state-of-the-art (SOTA) recipe Cross-Lingual Knowledge Democracy Edit (X-KDE),
designed to propagate knowledge from a dominant language to other languages
effectively. Our X-KDE comprises two stages: (i) Cross-lingual Edition
Instruction Tuning (XE-IT), which fine-tunes the model on a curated parallel
dataset to modify in-scope knowledge while preserving unrelated information,
and (ii) Target-language Preference Optimization (TL-PO), which applies
advanced optimization techniques to ensure consistency across languages,
fostering the transfer of updates. Additionally, we contribute a high-quality,
cross-lingual dataset, specifically designed to enhance knowledge transfer
across languages. Extensive experiments on the Bi-ZsRE and MzsRE benchmarks
show that X-KDE significantly enhances cross-lingual performance, achieving an
average improvement of +8.19%, while maintaining high accuracy in monolingual
settings.
☆ LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning
Long context understanding remains challenging for large language models due
to their limited context windows. This paper presents Long Input Fine-Tuning
(LIFT), a novel framework for long-context modeling that can improve the
long-context performance of arbitrary (short-context) LLMs by dynamically
adapting model parameters based on the long input. Importantly, LIFT, rather
than endlessly extending the context window size to accommodate increasingly
longer inputs in context, chooses to store and absorb the long input in
parameter. By fine-tuning the long input into model parameters, LIFT allows
short-context LLMs to answer questions even when the required information is
not provided in the context during inference. Furthermore, to enhance LIFT
performance while maintaining the original in-context learning (ICL)
capabilities, we introduce Gated Memory, a specialized attention adapter that
automatically balances long input memorization and ICL. We provide a
comprehensive analysis of the strengths and limitations of LIFT on long context
understanding, offering valuable directions for future research.
comment: arXiv admin note: text overlap with arXiv:2412.13626
☆ Length-Controlled Margin-Based Preference Optimization without Reference Model
Direct Preference Optimization (DPO) is a widely adopted offline algorithm
for preference-based reinforcement learning from human feedback (RLHF),
designed to improve training simplicity and stability by redefining reward
functions. However, DPO is hindered by several limitations, including length
bias, memory inefficiency, and probability degradation. To address these
challenges, we propose Length-Controlled Margin-Based Preference Optimization
(LMPO), a more efficient and robust alternative. LMPO introduces a uniform
reference model as an upper bound for the DPO loss, enabling a more accurate
approximation of the original optimization objective. Additionally, an average
log-probability optimization strategy is employed to minimize discrepancies
between training and inference phases. A key innovation of LMPO lies in its
Length-Controlled Margin-Based loss function, integrated within the
Bradley-Terry framework. This loss function regulates response length while
simultaneously widening the margin between preferred and rejected outputs. By
doing so, it mitigates probability degradation for both accepted and discarded
responses, addressing a significant limitation of existing methods. We evaluate
LMPO against state-of-the-art preference optimization techniques on two
open-ended large language models, Mistral and LLaMA3, across six conditional
benchmarks. Our experimental results demonstrate that LMPO effectively controls
response length, reduces probability degradation, and outperforms existing
approaches. The code is available at \url{https://github.com/gengxuli/LMPO}.
☆ How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation
Recently, LLMs have garnered increasing attention across academic disciplines
for their potential as human digital twins, virtual proxies designed to
replicate individuals and autonomously perform tasks such as decision-making,
problem-solving, and reasoning on their behalf. However, current evaluations of
LLMs primarily emphasize dialogue simulation while overlooking human behavior
simulation, which is crucial for digital twins. To address this gap, we
introduce BehaviorChain, the first benchmark for evaluating LLMs' ability to
simulate continuous human behavior. BehaviorChain comprises diverse,
high-quality, persona-based behavior chains, totaling 15,846 distinct behaviors
across 1,001 unique personas, each with detailed history and profile metadata.
For evaluation, we integrate persona metadata into LLMs and employ them to
iteratively infer contextually appropriate behaviors within dynamic scenarios
provided by BehaviorChain. Comprehensive evaluation results demonstrated that
even state-of-the-art models struggle with accurately simulating continuous
human behavior.
☆ NAVIG: Natural Language-guided Analysis with Vision Language Models for Image Geo-localization
Image geo-localization is the task of predicting the specific location of an
image and requires complex reasoning across visual, geographical, and cultural
contexts. While prior Vision Language Models (VLMs) have the best accuracy at
this task, there is a dearth of high-quality datasets and models for analytical
reasoning. We first create NaviClues, a high-quality dataset derived from
GeoGuessr, a popular geography game, to supply examples of expert reasoning
from language. Using this dataset, we present Navig, a comprehensive image
geo-localization framework integrating global and fine-grained image
information. By reasoning with language, Navig reduces the average distance
error by 14% compared to previous state-of-the-art models while requiring fewer
than 1000 training samples. Our dataset and code are available at
https://github.com/SparrowZheyuan18/Navig/.
☆ PEARL: Towards Permutation-Resilient LLMs ICLR 2025
The in-context learning (ICL) capability of large language models (LLMs)
enables them to perform challenging tasks using provided demonstrations.
However, ICL is highly sensitive to the ordering of demonstrations, leading to
instability in predictions. This paper shows that this vulnerability can be
exploited to design a natural attack - difficult for model providers to detect
- that achieves nearly 80% success rate on LLaMA-3 by simply permuting the
demonstrations. Existing mitigation methods primarily rely on post-processing
and fail to enhance the model's inherent robustness to input permutations,
raising concerns about safety and reliability of LLMs. To address this issue,
we propose Permutation-resilient learning (PEARL), a novel framework based on
distributionally robust optimization (DRO), which optimizes model performance
against the worst-case input permutation. Specifically, PEARL consists of a
permutation-proposal network (P-Net) and the LLM. The P-Net generates the most
challenging permutations by treating it as an optimal transport problem, which
is solved using an entropy-constrained Sinkhorn algorithm. Through minimax
optimization, the P-Net and the LLM iteratively optimize against each other,
progressively improving the LLM's robustness. Experiments on synthetic
pre-training and real-world instruction tuning tasks demonstrate that PEARL
effectively mitigates permutation attacks and enhances performance. Notably,
despite being trained on fewer shots and shorter contexts, PEARL achieves
performance gains of up to 40% when scaled to many-shot and long-context
scenarios, highlighting its efficiency and generalization capabilities.
comment: ICLR 2025
☆ Multi-Record Web Page Information Extraction From News Websites
In this paper, we focused on the problem of extracting information from web
pages containing many records, a task of growing importance in the era of
massive web data. Recently, the development of neural network methods has
improved the quality of information extraction from web pages. Nevertheless,
most of the research and datasets are aimed at studying detailed pages. This
has left multi-record "list pages" relatively understudied, despite their
widespread presence and practical significance.
To address this gap, we created a large-scale, open-access dataset
specifically designed for list pages. This is the first dataset for this task
in the Russian language. Our dataset contains 13,120 web pages with news lists,
significantly exceeding existing datasets in both scale and complexity. Our
dataset contains attributes of various types, including optional and
multi-valued, providing a realistic representation of real-world list pages.
These features make our dataset a valuable resource for studying information
extraction from pages containing many records.
Furthermore, we proposed our own multi-stage information extraction methods.
In this work, we explore and demonstrate several strategies for applying
MarkupLM to the specific challenges of multi-record web pages. Our experiments
validate the advantages of our methods.
By releasing our dataset to the public, we aim to advance the field of
information extraction from multi-record pages.
☆ Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
This paper investigates the efficacy of RWKV, a novel language model
architecture known for its linear attention mechanism, for generating sentence
embeddings in a zero-shot setting. I conduct a layer-wise analysis to evaluate
the semantic similarity captured by embeddings from different hidden layers of
a pre-trained RWKV model. The performance is assessed on the Microsoft Research
Paraphrase Corpus (MRPC) dataset using Spearman correlation and compared
against a GloVe-based baseline. My results indicate that while RWKV embeddings
capture some semantic relatedness, they underperform compared to the GloVe
baseline in terms of Spearman correlation. I also analyze the inference time
and GPU memory usage, highlighting the computational trade-offs associated with
RWKV embeddings. The findings suggest that while RWKV offers potential
advantages in terms of linear scaling, its zero-shot sentence embedding quality
for semantic similarity tasks requires further investigation and potential
task-specific fine-tuning to match or exceed simpler baselines.
comment: 17 pages, 3 tables, preprint on ArXiV, includes detailed analysis of
RWKV for semantic similarity tasks
☆ Reward Models Identify Consistency, Not Causality
Reward models (RMs) play a crucial role in aligning large language models
(LLMs) with human preferences and enhancing reasoning quality. Traditionally,
RMs are trained to rank candidate outputs based on their correctness and
coherence. However, in this work, we present several surprising findings that
challenge common assumptions about RM behavior. Our analysis reveals that
state-of-the-art reward models prioritize structural consistency over causal
correctness. Specifically, removing the problem statement has minimal impact on
reward scores, whereas altering numerical values or disrupting the reasoning
flow significantly affects RM outputs. Furthermore, RMs exhibit a strong
dependence on complete reasoning trajectories truncated or incomplete steps
lead to significant variations in reward assignments, indicating that RMs
primarily rely on learned reasoning patterns rather than explicit problem
comprehension. These findings hold across multiple architectures, datasets, and
tasks, leading to three key insights: (1) RMs primarily assess coherence rather
than true reasoning quality; (2) The role of explicit problem comprehension in
reward assignment is overstated; (3) Current RMs may be more effective at
ranking responses than verifying logical validity. Our results suggest a
fundamental limitation in existing reward modeling approaches, emphasizing the
need for a shift toward causality-aware reward models that go beyond
consistency-driven evaluation.
comment: 16 pages
☆ FIND: Fine-grained Information Density Guided Adaptive Retrieval-Augmented Generation for Disease Diagnosis
Retrieval-Augmented Large Language Models (LLMs), which integrate external
knowledge into LLMs, have shown remarkable performance in various medical
domains, including clinical diagnosis. However, existing RAG methods struggle
to effectively assess task difficulty to make retrieval decisions, thereby
failing to meet the clinical requirements for balancing efficiency and
accuracy. So in this paper, we propose FIND (\textbf{F}ine-grained
\textbf{In}formation \textbf{D}ensity Guided Adaptive RAG), a novel framework
that improves the reliability of RAG in disease diagnosis scenarios. FIND
incorporates a fine-grained adaptive control module to determine whether
retrieval is necessary based on the information density of the input. By
optimizing the retrieval process and implementing a knowledge filtering module,
FIND ensures that the retrieval is better suited to clinical scenarios.
Experiments on three Chinese electronic medical record datasets demonstrate
that FIND significantly outperforms various baseline methods, highlighting its
effectiveness in clinical diagnosis tasks.
☆ Behavioral Analysis of Information Salience in Large Language Models
Large Language Models (LLMs) excel at text summarization, a task that
requires models to select content based on its importance. However, the exact
notion of salience that LLMs have internalized remains unclear. To bridge this
gap, we introduce an explainable framework to systematically derive and
investigate information salience in LLMs through their summarization behavior.
Using length-controlled summarization as a behavioral probe into the content
selection process, and tracing the answerability of Questions Under Discussion
throughout, we derive a proxy for how models prioritize information. Our
experiments on 13 models across four datasets reveal that LLMs have a nuanced,
hierarchical notion of salience, generally consistent across model families and
sizes. While models show highly consistent behavior and hence salience
patterns, this notion of salience cannot be accessed through introspection, and
only weakly correlates with human perceptions of information salience.
☆ A Statistical Case Against Empirical Human-AI Alignment
Empirical human-AI alignment aims to make AI systems act in line with
observed human behavior. While noble in its goals, we argue that empirical
alignment can inadvertently introduce statistical biases that warrant caution.
This position paper thus advocates against naive empirical alignment, offering
prescriptive alignment and a posteriori empirical alignment as alternatives. We
substantiate our principled argument by tangible examples like human-centric
decoding of language models.
comment: 24 pages, 2 figures, 5 tables
☆ ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
Self-awareness, i.e., the ability to assess and correct one's own generation,
is a fundamental aspect of human intelligence, making its replication in large
language models (LLMs) an important yet challenging task. Previous works tackle
this by employing extensive reinforcement learning or rather relying on large
external verifiers. In this work, we propose Refine via Intrinsic
Self-Verification (ReVISE), an efficient and effective framework that enables
LLMs to self-correct their outputs through self-verification. The core idea of
ReVISE is to enable LLMs to verify their reasoning processes and continually
rethink reasoning trajectories based on its verification. We introduce a
structured curriculum based upon online preference learning to implement this
efficiently. Specifically, as ReVISE involves two challenging tasks (i.e.,
self-verification and reasoning correction), we tackle each task sequentially
using curriculum learning, collecting both failed and successful reasoning
paths to construct preference pairs for efficient training. During inference,
our approach enjoys natural test-time scaling by integrating self-verification
and correction capabilities, further enhanced by our proposed confidence-aware
decoding mechanism. Our experiments on various reasoning tasks demonstrate that
ReVISE achieves efficient self-correction and significantly improves reasoning
performance.
☆ Can LLMs Predict Citation Intent? An Experimental Analysis of In-context Learning and Fine-tuning on Open LLMs
This work investigates the ability of open Large Language Models (LLMs) to
predict citation intent through in-context learning and fine-tuning. Unlike
traditional approaches that rely on pre-trained models like SciBERT, which
require extensive domain-specific pretraining and specialized architectures, we
demonstrate that general-purpose LLMs can be adapted to this task with minimal
task-specific data. We evaluate twelve model variations across five prominent
open LLM families using zero, one, few, and many-shot prompting to assess
performance across scenarios. Our experimental study identifies the
top-performing model through extensive experimentation of in-context
learning-related parameters, which we fine-tune to further enhance task
performance. The results highlight the strengths and limitations of LLMs in
recognizing citation intents, providing valuable insights for model selection
and prompt engineering. Additionally, we make our end-to-end evaluation
framework and models openly available for future use.
☆ Less is More: Improving LLM Alignment via Preference Data Selection
Direct Preference Optimization (DPO) has emerged as a promising approach for
aligning large language models with human preferences. While prior work mainly
extends DPO from the aspect of the objective function, we instead improve DPO
from the largely overlooked but critical aspect of data selection.
Specifically, we address the issue of parameter shrinkage caused by noisy data
by proposing a novel margin-maximization principle for dataset curation in DPO
training. To accurately estimate margins for data selection, we propose a
dual-margin guided approach that considers both external reward margins and
implicit DPO reward margins. Extensive experiments demonstrate that our method
reduces computational cost dramatically while improving performance.
Remarkably, by using just 10\% of the Ultrafeedback dataset, our approach
achieves 3\% to 8\% improvements across various Llama and Mistral series models
on the AlpacaEval 2.0 benchmark. Furthermore, our approach seamlessly extends
to iterative DPO, yielding a roughly 3\% improvement with 25\% online data,
while further reducing training time. These results highlight the potential of
data selection strategies for advancing preference optimization.
☆ Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
Bytes form the basis of the digital world and thus are a promising building
block for multimodal foundation models. Recently, Byte Language Models (BLMs)
have emerged to overcome tokenization, yet the excessive length of bytestreams
requires new architectural paradigms. Therefore, we present the Multiscale Byte
Language Model (MBLM), a model-agnostic hierarchical decoder stack that allows
training with context windows of $5$M bytes on single GPU in full model
precision. We thoroughly examine MBLM's performance with Transformer and Mamba
blocks on both unimodal and multimodal tasks. Our experiments demonstrate that
hybrid architectures are efficient in handling extremely long byte sequences
during training while achieving near-linear generational efficiency. To the
best of our knowledge, we present the first evaluation of BLMs on visual Q\&A
tasks and find that, despite serializing images and the absence of an encoder,
a MBLM with pure next token prediction can match custom CNN-LSTM architectures
with designated classification heads. We show that MBLMs exhibit strong
adaptability in integrating diverse data representations, including pixel and
image filestream bytes, underlining their potential toward omnimodal foundation
models. Source code is publicly available at:
https://github.com/ai4sd/multiscale-byte-lm
comment: Under Review
☆ LLM-based User Profile Management for Recommender System ACL 2025
The rapid advancement of Large Language Models (LLMs) has opened new
opportunities in recommender systems by enabling zero-shot recommendation
without conventional training. Despite their potential, most existing works
rely solely on users' purchase histories, leaving significant room for
improvement by incorporating user-generated textual data, such as reviews and
product descriptions. Addressing this gap, we propose PURE, a novel LLM-based
recommendation framework that builds and maintains evolving user profiles by
systematically extracting and summarizing key information from user reviews.
PURE consists of three core components: a Review Extractor for identifying user
preferences and key product features, a Profile Updater for refining and
updating user profiles, and a Recommender for generating personalized
recommendations using the most current profile. To evaluate PURE, we introduce
a continuous sequential recommendation task that reflects real-world scenarios
by adding reviews over time and updating predictions incrementally. Our
experimental results on Amazon datasets demonstrate that PURE outperforms
existing LLM-based methods, effectively leveraging long-term user information
while managing token limitations.
comment: Submitted to ACL 2025
☆ LoRA-GGPO: Mitigating Double Descent in LoRA Fine-Tuning via Gradient-Guided Perturbation Optimization
Large Language Models (LLMs) have achieved remarkable success in natural
language processing, but their full fine-tuning remains resource-intensive.
Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation
(LoRA), have emerged as a practical solution by approximating parameter updates
with low-rank matrices. However, LoRA often exhibits a "double descent"
phenomenon during fine-tuning, where model performance degrades due to
overfitting and limited expressiveness caused by low-rank constraints. To
address this issue, we propose LoRA-GGPO (Gradient-Guided Perturbation
Optimization), a novel method that leverages gradient and weight norms to
generate targeted perturbations. By optimizing the sharpness of the loss
landscape, LoRA-GGPO guides the model toward flatter minima, mitigating the
double descent problem and improving generalization. Extensive experiments on
natural language understanding (NLU) and generation (NLG) tasks demonstrate
that LoRA-GGPO outperforms LoRA and its state-of-the-art variants. Furthermore,
extended experiments specifically designed to analyze the double descent
phenomenon confirm that LoRA-GGPO effectively alleviates this issue, producing
more robust and generalizable models. Our work provides a robust and efficient
solution for fine-tuning LLMs, with broad applicability in real-world
scenarios. The code is available at https://github.com/llm172/LoRA-GGPO.
☆ CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models
Large Language Model-based Multi-Agent Systems (LLM-MASs) have demonstrated
remarkable real-world capabilities, effectively collaborating to complete
complex tasks. While these systems are designed with safety mechanisms, such as
rejecting harmful instructions through alignment, their security remains
largely unexplored. This gap leaves LLM-MASs vulnerable to targeted
disruptions. In this paper, we introduce Contagious Recursive Blocking Attacks
(Corba), a novel and simple yet highly effective attack that disrupts
interactions between agents within an LLM-MAS. Corba leverages two key
properties: its contagious nature allows it to propagate across arbitrary
network topologies, while its recursive property enables sustained depletion of
computational resources. Notably, these blocking attacks often involve
seemingly benign instructions, making them particularly challenging to mitigate
using conventional alignment methods. We evaluate Corba on two widely-used
LLM-MASs, namely, AutoGen and Camel across various topologies and commercial
models. Additionally, we conduct more extensive experiments in open-ended
interactive LLM-MASs, demonstrating the effectiveness of Corba in complex
topology structures and open-source models. Our code is available at:
https://github.com/zhrli324/Corba.
☆ Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generation
We propose a new framework for zero-shot generation of synthetic tabular
data. Using the large language model (LLM) GPT-4o and plain-language prompting,
we demonstrate the ability to generate high-fidelity tabular data without
task-specific fine-tuning or access to real-world data (RWD) for pre-training.
To benchmark GPT-4o, we compared the fidelity and privacy of LLM-generated
synthetic data against data generated with the conditional tabular generative
adversarial network (CTGAN), across three open-access datasets: Iris, Fish
Measurements, and Real Estate Valuation. Despite the zero-shot approach, GPT-4o
outperformed CTGAN in preserving means, 95% confidence intervals, bivariate
correlations, and data privacy of RWD, even at amplified sample sizes. Notably,
correlations between parameters were consistently preserved with appropriate
direction and strength. However, refinement is necessary to better retain
distributional characteristics. These findings highlight the potential of LLMs
in tabular data synthesis, offering an accessible alternative to generative
adversarial networks and variational autoencoders.
comment: 12 pages, 7 figures, 5 tables
☆ MultiSlav: Using Cross-Lingual Knowledge Transfer to Combat the Curse of Multilinguality
Artur Kot, Mikołaj Koszowski, Wojciech Chojnowski, Mieszko Rutkowski, Artur Nowakowski, Kamil Guttmann, Mikołaj Pokrywka
Does multilingual Neural Machine Translation (NMT) lead to The Curse of the
Multlinguality or provides the Cross-lingual Knowledge Transfer within a
language family? In this study, we explore multiple approaches for extending
the available data-regime in NMT and we prove cross-lingual benefits even in
0-shot translation regime for low-resource languages. With this paper, we
provide state-of-the-art open-source NMT models for translating between
selected Slavic languages. We released our models on the HuggingFace Hub
(https://hf.co/collections/allegro/multislav-6793d6b6419e5963e759a683) under
the CC BY 4.0 license. Slavic language family comprises morphologically rich
Central and Eastern European languages. Although counting hundreds of millions
of native speakers, Slavic Neural Machine Translation is under-studied in our
opinion. Recently, most NMT research focuses either on: high-resource languages
like English, Spanish, and German - in WMT23 General Translation Task 7 out of
8 task directions are from or to English; massively multilingual models
covering multiple language groups; or evaluation techniques.
☆ Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases
Rena Gao, Xuetong Wu, Tatsuki Kuribayashi, Mingrui Ye, Siya Qi, Carsten Roever, Yuanxing Liu, Zheng Yuan, Jey Han Lau
This study evaluates Large Language Models' (LLMs) ability to simulate
non-native-like English use observed in human second language (L2) learners
interfered with by their native first language (L1). In dialogue-based
interviews, we prompt LLMs to mimic L2 English learners with specific L1s
(e.g., Japanese, Thai, Urdu) across seven languages, comparing their outputs to
real L2 learner data. Our analysis examines L1-driven linguistic biases, such
as reference word usage and avoidance behaviors, using information-theoretic
and distributional density measures. Results show that modern LLMs (e.g.,
Qwen2.5, LLAMA3.3, DeepseekV3, GPT-4o) replicate L1-dependent patterns observed
in human L2 data, with distinct influences from various languages (e.g.,
Japanese, Korean, and Mandarin significantly affect tense agreement, and Urdu
influences noun-verb collocations). Our results reveal the potential of LLMs
for L2 dialogue generation and evaluation for future educational applications.
☆ How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?
Sergey Pletenev, Maria Marina, Daniil Moskovskiy, Vasily Konovalov, Pavel Braslavski, Alexander Panchenko, Mikhail Salnikov
The performance of Large Language Models (LLMs) on many tasks is greatly
limited by the knowledge learned during pre-training and stored in the model's
parameters. Low-rank adaptation (LoRA) is a popular and efficient training
technique for updating or domain-specific adaptation of LLMs. In this study, we
investigate how new facts can be incorporated into the LLM using LoRA without
compromising the previously learned knowledge. We fine-tuned
Llama-3.1-8B-instruct using LoRA with varying amounts of new knowledge. Our
experiments have shown that the best results are obtained when the training
data contains a mixture of known and new facts. However, this approach is still
potentially harmful because the model's performance on external
question-answering benchmarks declines after such fine-tuning. When the
training data is biased towards certain entities, the model tends to regress to
few overrepresented answers. In addition, we found that the model becomes more
confident and refuses to provide an answer in only few cases. These findings
highlight the potential pitfalls of LoRA-based LLM updates and underscore the
importance of training data composition and tuning parameters to balance new
knowledge integration and general model capabilities.
☆ Towards a Perspectivist Turn in Argument Quality Assessment NAACL 2025
The assessment of argument quality depends on well-established logical,
rhetorical, and dialectical properties that are unavoidably subjective:
multiple valid assessments may exist, there is no unequivocal ground truth.
This aligns with recent paths in machine learning, which embrace the
co-existence of different perspectives. However, this potential remains largely
unexplored in NLP research on argument quality. One crucial reason seems to be
the yet unexplored availability of suitable datasets. We fill this gap by
conducting a systematic review of argument quality datasets. We assign them to
a multi-layered categorization targeting two aspects: (a) What has been
annotated: we collect the quality dimensions covered in datasets and
consolidate them in an overarching taxonomy, increasing dataset comparability
and interoperability. (b) Who annotated: we survey what information is given
about annotators, enabling perspectivist research and grounding our
recommendations for future actions. To this end, we discuss datasets suitable
for developing perspectivist models (i.e., those containing individual,
non-aggregated annotations), and we showcase the importance of a controlled
selection of annotators in a pilot study.
comment: Accepted to NAACL 2025
☆ MLGym: A New Framework and Benchmark for Advancing AI Research Agents
Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, Dieuwke Hupkes, Ricardo Silveira Cabral, Tatiana Shavrina, Jakob Foerster, Yoram Bachrach, William Yang Wang, Roberta Raileanu
We introduce Meta MLGym and MLGym-Bench, a new framework and benchmark for
evaluating and developing LLM agents on AI research tasks. This is the first
Gym environment for machine learning (ML) tasks, enabling research on
reinforcement learning (RL) algorithms for training such agents. MLGym-bench
consists of 13 diverse and open-ended AI research tasks from diverse domains
such as computer vision, natural language processing, reinforcement learning,
and game theory. Solving these tasks requires real-world AI research skills
such as generating new ideas and hypotheses, creating and processing data,
implementing ML methods, training models, running experiments, analyzing the
results, and iterating through this process to improve on a given task. We
evaluate a number of frontier large language models (LLMs) on our benchmarks
such as Claude-3.5-Sonnet, Llama-3.1 405B, GPT-4o, o1-preview, and Gemini-1.5
Pro. Our MLGym framework makes it easy to add new tasks, integrate and evaluate
models or agents, generate synthetic data at scale, as well as develop new
learning algorithms for training agents on AI research tasks. We find that
current frontier models can improve on the given baselines, usually by finding
better hyperparameters, but do not generate novel hypotheses, algorithms,
architectures, or substantial improvements. We open-source our framework and
benchmark to facilitate future research in advancing the AI research
capabilities of LLM agents.
comment: 35 pages, 12 figures, 10 tables
☆ Stories that (are) Move(d by) Markets: A Causal Exploration of Market Shocks and Semantic Shifts across Different Partisan Groups
Macroeconomic fluctuations and the narratives that shape them form a mutually
reinforcing cycle: public discourse can spur behavioural changes leading to
economic shifts, which then result in changes in the stories that propagate. We
show that shifts in semantic embedding space can be causally linked to
financial market shocks -- deviations from the expected market behaviour.
Furthermore, we show how partisanship can influence the predictive power of
text for market fluctuations and shape reactions to those same shocks. We also
provide some evidence that text-based signals are particularly salient during
unexpected events such as COVID-19, highlighting the value of language data as
an exogenous variable in economic forecasting. Our findings underscore the
bidirectional relationship between news outlets and market shocks, offering a
novel empirical approach to studying their effect on each other.
☆ Enhancing Language Multi-Agent Learning with Multi-Agent Credit Re-Assignment for Interactive Environment Generalization
LLM-based agents have made significant advancements in interactive
environments, such as mobile operations and web browsing, and other domains
beyond computer using. Current multi-agent systems universally excel in
performance, compared to single agents, but struggle with generalization across
environments due to predefined roles and inadequate strategies for generalizing
language agents. The challenge of achieving both strong performance and good
generalization has hindered the progress of multi-agent systems for interactive
environments. To address these issues, we propose CollabUIAgents, a multi-agent
reinforcement learning framework with a novel multi-agent credit re-assignment
(CR) strategy, assigning process rewards with LLMs rather than
environment-specific rewards and learning with synthesized preference data, in
order to foster generalizable, collaborative behaviors among the role-free
agents' policies. Empirical results show that our framework improves both
performance and cross-environment generalizability of multi-agent systems.
Moreover, our 7B-parameter system achieves results on par with or exceed strong
closed-source models, and the LLM that guides the CR. We also provide insights
in using granular CR rewards effectively for environment generalization, and
accommodating trained LLMs in multi-agent systems.
comment: 24 pages, under review
☆ StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following
Multi-turn instruction following capability constitutes a core competency of
large language models (LLMs) in real-world applications. Existing evaluation
benchmarks predominantly focus on fine-grained constraint satisfaction and
domain-specific capability assessment, yet overlook the crucial structural
dependency between dialogue turns that distinguishes multi-turn from
single-turn interactions. This structural dependency not only reflects user
intent but also establishes a second dimension for instruction following
evaluation beyond constraint satisfaction. To address this gap, we propose
StructFlowBench, a multi-turn instruction following benchmark with structural
flow modeling. The benchmark innovatively defines a structural flow framework
comprising six fundamental inter-turn relationships, which not only introduces
novel structural constraints for model evaluation but also serves as generation
parameters for creating customized dialogue flows tailored to specific
scenarios. Adopting established LLM-based automatic evaluation methodologies,
we conduct systematic evaluations of 13 leading open-source and closed-source
LLMs. Experimental results reveal significant deficiencies in current models'
comprehension of multi-turn dialogue structures. The code is available at
\url{https://github.com/MLGroupJLU/StructFlowBench}.
comment: 18 pages, 8 figures, 8 tables
☆ How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation
Jailbreak attacks, where harmful prompts bypass generative models' built-in
safety, raise serious concerns about model vulnerability. While many defense
methods have been proposed, the trade-offs between safety and helpfulness, and
their application to Large Vision-Language Models (LVLMs), are not well
understood. This paper systematically examines jailbreak defenses by reframing
the standard generation task as a binary classification problem to assess model
refusal tendencies for both harmful and benign queries. We identify two key
defense mechanisms: safety shift, which increases refusal rates across all
queries, and harmfulness discrimination, which improves the model's ability to
distinguish between harmful and benign inputs. Using these mechanisms, we
develop two ensemble defense strategies-inter-mechanism ensembles and
intra-mechanism ensembles-to balance safety and helpfulness. Experiments on the
MM-SafetyBench and MOSSBench datasets with LLaVA-1.5 models show that these
strategies effectively improve model safety or optimize the trade-off between
safety and helpfulness.
☆ NLoRA: Nyström-Initiated Low-Rank Adaptation for Large Language Models
Parameter-efficient fine-tuning (PEFT) is essential for adapting large
language models (LLMs), with low-rank adaptation (LoRA) being the most popular
approach. However, LoRA suffers from slow convergence, and some recent LoRA
variants, such as PiSSA, primarily rely on Singular Value Decomposition (SVD)
for initialization, leading to expensive computation. To mitigate these
problems, we use the Nystr\"om method, which follows a three-matrix
manipulation. We first introduce StructuredLoRA (SLoRA), which investigates
adding a small intermediate matrix between the low-rank matrices A and B.
Secondly, we propose Nystr\"omLoRA (NLoRA), which leverages Nystr\"om-based
initialization for SLoRA to improve its effectiveness and efficiency. Finally,
we propose IntermediateTune (IntTune), which explores fine-tuning exclusively
on the intermediate matrix of NLoRA to further boost LLM efficiency. We
evaluate our methods on five natural language generation (NLG) tasks and eight
natural language understanding (NLU) tasks. On GSM8K, SLoRA and NLoRA achieve
accuracies of 56.48% and 57.70%, surpassing LoRA by 33.52% and 36.41%, with
only 3.67 million additional trainable parameters. IntTune improves average NLG
performance over LoRA by 7.45% while using only 1.25% of its parameters. These
results demonstrate the efficiency and effectiveness of our approach in
enhancing model performance with minimal parameter overhead.
☆ Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
Handling long-context sequences efficiently remains a significant challenge
in large language models (LLMs). Existing methods for token selection in
sequence extrapolation either employ a permanent eviction strategy or select
tokens by chunk, which may lead to the loss of critical information. We propose
Efficient Selective Attention (ESA), a novel approach that extends context
length by efficiently selecting the most critical tokens at the token level to
compute attention. ESA reduces the computational complexity of token selection
by compressing query and key vectors into lower-dimensional representations. We
evaluate ESA on long sequence benchmarks with maximum lengths up to 256k using
open-source LLMs with context lengths of 8k and 32k. ESA outperforms other
selective attention methods, especially in tasks requiring the retrieval of
multiple pieces of information, achieving comparable performance to
full-attention extrapolation methods across various tasks, with superior
results in certain tasks.
comment: 14 pages,2 figures
☆ Argument-Based Comparative Question Answering Evaluation Benchmark
Irina Nikishina, Saba Anwar, Nikolay Dolgov, Maria Manina, Daria Ignatenko, Viktor Moskvoretskii, Artem Shelmanov, Tim Baldwin, Chris Biemann
In this paper, we aim to solve the problems standing in the way of automatic
comparative question answering. To this end, we propose an evaluation framework
to assess the quality of comparative question answering summaries. We formulate
15 criteria for assessing comparative answers created using manual annotation
and annotation from 6 large language models and two comparative question
asnwering datasets. We perform our tests using several LLMs and manual
annotation under different settings and demonstrate the constituency of both
evaluations. Our results demonstrate that the Llama-3 70B Instruct model
demonstrates the best results for summary evaluation, while GPT-4 is the best
for answering comparative questions. All used data, code, and evaluation
results are publicly
available\footnote{\url{https://anonymous.4open.science/r/cqa-evaluation-benchmark-4561/README.md}}.
comment: 8 pages, 7 Tables, 13 Figures, 18 pages with Appendix
☆ Enhancing Smart Environments with Context-Aware Chatbots using Large Language Models
This work presents a novel architecture for context-aware interactions within
smart environments, leveraging Large Language Models (LLMs) to enhance user
experiences. Our system integrates user location data obtained through UWB tags
and sensor-equipped smart homes with real-time human activity recognition (HAR)
to provide a comprehensive understanding of user context. This contextual
information is then fed to an LLM-powered chatbot, enabling it to generate
personalised interactions and recommendations based on the user's current
activity and environment. This approach moves beyond traditional static chatbot
interactions by dynamically adapting to the user's real-time situation. A case
study conducted from a real-world dataset demonstrates the feasibility and
effectiveness of our proposed architecture, showcasing its potential to create
more intuitive and helpful interactions within smart homes. The results
highlight the significant benefits of integrating LLM with real-time activity
and location data to deliver personalised and contextually relevant user
experiences.
comment: 11 pages, 3 figures
☆ Optimal word order for non-causal text generation with Large Language Models: the Spanish case
Andrea Busto-Castiñeira, Silvia García-Méndez, Francisco de Arriba-Pérez, Francisco J. González-Castaño
Natural Language Generation (NLG) popularity has increased owing to the
progress in Large Language Models (LLMs), with zero-shot inference
capabilities. However, most neural systems utilize decoder-only causal
(unidirectional) transformer models, which are effective for English but may
reduce the richness of languages with less strict word order, subject omission,
or different relative clause attachment preferences. This is the first work
that analytically addresses optimal text generation order for non-causal
language models. We present a novel Viterbi algorithm-based methodology for
maximum likelihood word order estimation. We analyze the non-causal
most-likelihood order probability for NLG in Spanish and, then, the probability
of generating the same phrases with Spanish causal NLG. This comparative
analysis reveals that causal NLG prefers English-like SVO structures. We also
analyze the relationship between optimal generation order and causal
left-to-right generation order using Spearman's rank correlation. Our results
demonstrate that the ideal order predicted by the maximum likelihood estimator
is not closely related to the causal order and may be influenced by the
syntactic structure of the target sentence.
☆ PredictaBoard: Benchmarking LLM Score Predictability
Lorenzo Pacchiardi, Konstantinos Voudouris, Ben Slater, Fernando Martínez-Plumed, José Hernández-Orallo, Lexin Zhou, Wout Schellaert
Despite possessing impressive skills, Large Language Models (LLMs) often fail
unpredictably, demonstrating inconsistent success in even basic common sense
reasoning tasks. This unpredictability poses a significant challenge to
ensuring their safe deployment, as identifying and operating within a reliable
"safe zone" is essential for mitigating risks. To address this, we present
PredictaBoard, a novel collaborative benchmarking framework designed to
evaluate the ability of score predictors (referred to as assessors) to
anticipate LLM errors on specific task instances (i.e., prompts) from existing
datasets. PredictaBoard evaluates pairs of LLMs and assessors by considering
the rejection rate at different tolerance errors. As such, PredictaBoard
stimulates research into developing better assessors and making LLMs more
predictable, not only with a higher average performance. We conduct
illustrative experiments using baseline assessors and state-of-the-art LLMs.
PredictaBoard highlights the critical need to evaluate predictability alongside
performance, paving the way for safer AI systems where errors are not only
minimised but also anticipated and effectively mitigated. Code for our
benchmark can be found at
https://github.com/Kinds-of-Intelligence-CFI/PredictaBoard
☆ An Enhancement of Jiang, Z., et al.s Compression-Based Classification Algorithm Applied to News Article Categorization
This study enhances Jiang et al.'s compression-based classification algorithm
by addressing its limitations in detecting semantic similarities between text
documents. The proposed improvements focus on unigram extraction and optimized
concatenation, eliminating reliance on entire document compression. By
compressing extracted unigrams, the algorithm mitigates sliding window
limitations inherent to gzip, improving compression efficiency and similarity
detection. The optimized concatenation strategy replaces direct concatenation
with the union of unigrams, reducing redundancy and enhancing the accuracy of
Normalized Compression Distance (NCD) calculations. Experimental results across
datasets of varying sizes and complexities demonstrate an average accuracy
improvement of 5.73%, with gains of up to 11% on datasets containing longer
documents. Notably, these improvements are more pronounced in datasets with
high-label diversity and complex text structures. The methodology achieves
these results while maintaining computational efficiency, making it suitable
for resource-constrained environments. This study provides a robust, scalable
solution for text classification, emphasizing lightweight preprocessing
techniques to achieve efficient compression, which in turn enables more
accurate classification.
comment: 11 pages, 5 figures, 1 table
☆ Natural Language Generation
This book provides a broad overview of Natural Language Generation (NLG),
including technology, user requirements, evaluation, and real-world
applications. The focus is on concepts and insights which hopefully will remain
relevant for many years, not on the latest LLM innovations. It draws on decades
of work by the author and others on NLG.
The book has the following chapters: Introduction to NLG; Rule-Based NLG;
Machine Learning and Neural NLG; Requirements; Evaluation; Safety, Maintenance,
and Testing; and Applications. All chapters include examples and anecdotes from
the author's personal experiences, and end with a Further Reading section.
The book should be especially useful to people working on applied NLG,
including NLG researchers, people in other fields who want to use NLG, and
commercial developers. It will not however be useful to people who want to
understand the latest LLM technology.
There is a companion site with more information at
https://ehudreiter.com/book/
comment: This is a preprint of the following work: Ehud Reiter, Natural
Language Generation, 2024, Springer reproduced with permission of Springer
Nature Switzerland AG. The final authenticated version is available online
at: http://dx.doi.org/10.1007/978-3-031-68582-8
☆ Early-Exit and Instant Confidence Translation Quality Estimation
Quality estimation is omnipresent in machine translation, for both evaluation
and generation. Unfortunately, quality estimation models are often opaque and
computationally expensive, making them impractical to be part of large-scale
pipelines. In this work, we tackle two connected challenges: (1) reducing the
cost of quality estimation at scale, and (2) developing an inexpensive
uncertainty estimation method for quality estimation. To address the latter, we
introduce Instant Confidence COMET, an uncertainty-aware quality estimation
model that matches the performance of previous approaches at a fraction of
their costs. We extend this to Early-Exit COMET, a quality estimation model
that can compute quality scores and associated confidences already at early
model layers, allowing us to early-exit computations and reduce evaluation
costs. We also apply our model to machine translation reranking. We combine
Early-Exit COMET with an upper confidence bound bandit algorithm to find the
best candidate from a large pool without having to run the full evaluation
model on all candidates. In both cases (evaluation and reranking) our methods
reduce the required compute by 50% with very little degradation in performance.
☆ Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models
Artem Vazhentsev, Lyudmila Rvanova, Ivan Lazichny, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov
Uncertainty quantification (UQ) is a prominent approach for eliciting
truthful answers from large language models (LLMs). To date, information-based
and consistency-based UQ have been the dominant UQ methods for text generation
via LLMs. Density-based methods, despite being very effective for UQ in text
classification with encoder-based models, have not been very successful with
generative LLMs. In this work, we adapt Mahalanobis Distance (MD) - a
well-established UQ technique in classification tasks - for text generation and
introduce a new supervised UQ method. Our method extracts token embeddings from
multiple layers of LLMs, computes MD scores for each token, and uses linear
regression trained on these features to provide robust uncertainty scores.
Through extensive experiments on eleven datasets, we demonstrate that our
approach substantially improves over existing UQ methods, providing accurate
and computationally efficient uncertainty scores for both sequence-level
selective generation and claim-level fact-checking tasks. Our method also
exhibits strong generalization to out-of-domain data, making it suitable for a
wide range of LLM-based applications.
☆ A Survey on Data Contamination for Large Language Models
Recent advancements in Large Language Models (LLMs) have demonstrated
significant progress in various areas, such as text generation and code
synthesis. However, the reliability of performance evaluation has come under
scrutiny due to data contamination-the unintended overlap between training and
test datasets. This overlap has the potential to artificially inflate model
performance, as LLMs are typically trained on extensive datasets scraped from
publicly available sources. These datasets often inadvertently overlap with the
benchmarks used for evaluation, leading to an overestimation of the models'
true generalization capabilities. In this paper, we first examine the
definition and impacts of data contamination. Secondly, we review methods for
contamination-free evaluation, focusing on three strategies: data
updating-based methods, data rewriting-based methods, and prevention-based
methods. Specifically, we highlight dynamic benchmarks and LLM-driven
evaluation methods. Finally, we categorize contamination detecting methods
based on model information dependency: white-Box, gray-Box, and black-Box
detection approaches. Our survey highlights the requirements for more rigorous
evaluation protocols and proposes future directions for addressing data
contamination challenges.
☆ Unstructured Evidence Attribution for Long Context Query Focused Summarization
Large language models (LLMs) are capable of generating coherent summaries
from very long contexts given a user query. Extracting and properly citing
evidence spans could help improve the transparency and reliability of these
summaries. At the same time, LLMs suffer from positional biases in terms of
which information they understand and attend to, which could affect evidence
citation. Whereas previous work has focused on evidence citation with
predefined levels of granularity (e.g. sentence, paragraph, document, etc.), we
propose the task of long-context query focused summarization with unstructured
evidence citation. We show how existing systems struggle to generate and
properly cite unstructured evidence from their context, and that evidence tends
to be "lost-in-the-middle". To help mitigate this, we create the Summaries with
Unstructured Evidence Text dataset (SUnsET), a synthetic dataset generated
using a novel domain-agnostic pipeline which can be used as supervision to
adapt LLMs to this task. We demonstrate across 5 LLMs of different sizes and 4
datasets with varying document types and lengths that LLMs adapted with SUnsET
data generate more relevant and factually consistent evidence than their base
models, extract evidence from more diverse locations in their context, and can
generate more relevant and consistent summaries.
comment: 24 pages; 21 figures; 5 tables
♻ ☆ Large Language Model Confidence Estimation via Black-Box Access
Estimating uncertainty or confidence in the responses of a model can be
significant in evaluating trust not only in the responses, but also in the
model as a whole. In this paper, we explore the problem of estimating
confidence for responses of large language models (LLMs) with simply black-box
or query access to them. We propose a simple and extensible framework where, we
engineer novel features and train a (interpretable) model (viz. logistic
regression) on these features to estimate the confidence. We empirically
demonstrate that our simple framework is effective in estimating confidence of
Flan-ul2, Llama-13b, Mistral-7b and GPT-4 on four benchmark Q\&A tasks as well
as of Pegasus-large and BART-large on two benchmark summarization tasks with it
surpassing baselines by even over $10\%$ (on AUROC) in some cases.
Additionally, our interpretable approach provides insight into features that
are predictive of confidence, leading to the interesting and useful discovery
that our confidence models built for one LLM generalize zero-shot across others
on a given dataset.
♻ ☆ The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
In this paper, we analyze the computational limitations of Mamba and
State-space Models (SSMs) by using the circuit complexity framework. Despite
Mamba's stateful design and recent attention as a strong candidate to
outperform Transformers, we have demonstrated that both Mamba and SSMs with
$\mathrm{poly}(n)$-precision and constant-depth layers reside within the
$\mathsf{DLOGTIME}$-uniform $\mathsf{TC}^0$ complexity class. This result
indicates Mamba has the same computational capabilities as Transformer
theoretically, and it cannot solve problems like arithmetic formula problems,
boolean formula value problems, and permutation composition problems if
$\mathsf{TC}^0 \neq \mathsf{NC}^1$. Therefore, it challenges the assumption
Mamba is more computationally expressive than Transformers. Our contributions
include rigorous proofs showing that Selective SSM and Mamba architectures can
be simulated by $\mathsf{DLOGTIME}$-uniform $\mathsf{TC}^0$ circuits, and they
cannot solve problems outside $\mathsf{TC}^0$.
comment: CPAL 2025
♻ ☆ Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning
Recent advancements in 3D Large Language Models (3DLLMs) have highlighted
their potential in building general-purpose agents in the 3D real world, yet
challenges remain due to the lack of high-quality robust instruction-following
data, leading to limited discriminative power and generalization of 3DLLMs. In
this paper, we introduce Robin3D, a powerful 3DLLM trained on large-scale
instruction-following data generated by our novel data engine, Robust
Instruction Generation (RIG) engine. RIG generates two key instruction data: 1)
the Adversarial Instruction-following data, which features mixed negative and
positive samples to enhance the model's discriminative understanding. 2) the
Diverse Instruction-following data, which contains various instruction styles
to enhance model's generalization. As a result, we construct 1 million
instruction-following data, consisting of 344K Adversarial samples, 508K
Diverse samples, and 165K benchmark training set samples. To better handle
these complex instructions, Robin3D first incorporates Relation-Augmented
Projector to enhance spatial understanding, and then strengthens the object
referring and grounding ability through ID-Feature Bonding. Robin3D
consistently outperforms previous methods across five widely-used 3D multimodal
learning benchmarks, without the need for task-specific fine-tuning. Notably,
we achieve a 7.8\% improvement in the grounding task (Multi3DRefer) and a 6.9\%
improvement in the captioning task (Scan2Cap).
comment: 8 pages
♻ ☆ How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations NAACL 2025
Multimodal foundation models aim to create a unified representation space
that abstracts away from surface features like language syntax or modality
differences. To investigate this, we study the internal representations of
three recent models, analyzing the model activations from semantically
equivalent sentences across languages in the text and speech modalities. Our
findings reveal that: 1) Cross-modal representations converge over model
layers, except in the initial layers specialized at text and speech processing.
2) Length adaptation is crucial for reducing the cross-modal gap between text
and speech, although current approaches' effectiveness is primarily limited to
high-resource languages. 3) Speech exhibits larger cross-lingual differences
than text. 4) For models not explicitly trained for modality-agnostic
representations, the modality gap is more prominent than the language gap.
comment: NAACL 2025
♻ ☆ Safety Evaluation of DeepSeek Models in Chinese Contexts
Wenjing Zhang, Xuejiao Lei, Zhaoxiang Liu, Ning Wang, Zhenhong Long, Peijun Yang, Jiaojiao Zhao, Minjie Hua, Chaoyang Ma, Kai Wang, Shiguo Lian
Recently, the DeepSeek series of models, leveraging their exceptional
reasoning capabilities and open-source strategy, is reshaping the global AI
landscape. Despite these advantages, they exhibit significant safety
deficiencies. Research conducted by Robust Intelligence, a subsidiary of Cisco,
in collaboration with the University of Pennsylvania, revealed that DeepSeek-R1
has a 100\% attack success rate when processing harmful prompts. Additionally,
multiple safety companies and research institutions have confirmed critical
safety vulnerabilities in this model. As models demonstrating robust
performance in Chinese and English, DeepSeek models require equally crucial
safety assessments in both language contexts. However, current research has
predominantly focused on safety evaluations in English environments, leaving a
gap in comprehensive assessments of their safety performance in Chinese
contexts. In response to this gap, this study introduces CHiSafetyBench, a
Chinese-specific safety evaluation benchmark. This benchmark systematically
evaluates the safety of DeepSeek-R1 and DeepSeek-V3 in Chinese contexts,
revealing their performance across safety categories. The experimental results
quantify the deficiencies of these two models in Chinese contexts, providing
key insights for subsequent improvements. It should be noted that, despite our
efforts to establish a comprehensive, objective, and authoritative evaluation
benchmark, the selection of test samples, characteristics of data distribution,
and the setting of evaluation criteria may inevitably introduce certain biases
into the evaluation results. We will continuously optimize the evaluation
benchmark and periodically update this report to provide more comprehensive and
accurate assessment outcomes. Please refer to the latest version of the paper
for the most recent evaluation results and conclusions.
comment: 12 pages, 2 tables, 7 figures
♻ ☆ Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generation
Despite the remarkable capabilities of Large Language Models (LLMs) in
various NLP tasks, they remain vulnerable to hallucinations due to their
limited parametric knowledge and lack of domain-specific expertise.
Retrieval-Augmented Generation (RAG) addresses this challenge by incorporating
external document retrieval to augment the knowledge base of LLMs. In this
approach, RAG retrieves document chunks from an external corpus in response to
a query, which are then used as context for the downstream language model to
generate an answer. However, these retrieved knowledge sources often include
irrelevant or erroneous information, undermining the effectiveness of RAG in
downstream tasks. To overcome this limitation, we introduce a compact,
efficient, and pluggable module designed to refine external knowledge sources
before feeding them to the generator. The module reconstructs retrieved content
by extracting the most relevant and supportive information and reorganising it
into a concise, query-specific format. Through a three-stage training paradigm
- comprising supervised fine-tuning, contrastive multi-task learning, and
reinforcement learning-based alignment - it prioritises critical knowledge and
aligns it with the generator's preferences. This method enables LLMs to produce
outputs that are more accurate, reliable, and contextually appropriate.
comment: 14 pages
♻ ☆ Reading between the Lines: Can LLMs Identify Cross-Cultural Communication Gaps?
In a rapidly globalizing and digital world, content such as book and product
reviews created by people from diverse cultures are read and consumed by others
from different corners of the world. In this paper, we investigate the extent
and patterns of gaps in understandability of book reviews due to the presence
of culturally-specific items and elements that might be alien to users from
another culture. Our user-study on 57 book reviews from Goodreads reveal that
83\% of the reviews had at least one culture-specific difficult-to-understand
element. We also evaluate the efficacy of GPT-4o in identifying such items,
given the cultural background of the reader; the results are mixed, implying a
significant scope for improvement. Our datasets are available here:
https://github.com/sougata-ub/reading_between_lines
♻ ☆ metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models ICLR 2025
Large Language Models (LLMs) vary in their abilities on a range of tasks.
Initiatives such as the Open LLM Leaderboard aim to quantify these differences
with several large benchmarks (sets of test items to which an LLM can respond
either correctly or incorrectly). However, high correlations within and between
benchmark scores suggest that (1) there exists a small set of common underlying
abilities that these benchmarks measure, and (2) items tap into redundant
information and the benchmarks may thus be considerably compressed. We use data
from n > 5000 LLMs to identify the most informative items of six benchmarks,
ARC, GSM8K, HellaSwag, MMLU, TruthfulQA and WinoGrande (with d = 28,632 items
in total). From them we distill a sparse benchmark, metabench, that has less
than 3% of the original size of all six benchmarks combined. This new sparse
benchmark goes beyond point scores by yielding estimators of the underlying
benchmark-specific abilities. We show that these estimators (1) can be used to
reconstruct each original individual benchmark score with, on average, 1.24%
root mean square error (RMSE), (2) reconstruct the original total score with
0.58% RMSE, and (3) have a single underlying common factor whose Spearman
correlation with the total score is r = 0.94.
comment: accepted for publication at ICLR 2025
♻ ☆ Certified Robustness Under Bounded Levenshtein Distance ICLR 2025
Text classifiers suffer from small perturbations, that if chosen
adversarially, can dramatically change the output of the model. Verification
methods can provide robustness certificates against such adversarial
perturbations, by computing a sound lower bound on the robust accuracy.
Nevertheless, existing verification methods incur in prohibitive costs and
cannot practically handle Levenshtein distance constraints. We propose the
first method for computing the Lipschitz constant of convolutional classifiers
with respect to the Levenshtein distance. We use these Lipschitz constant
estimates for training 1-Lipschitz classifiers. This enables computing the
certified radius of a classifier in a single forward pass. Our method, LipsLev,
is able to obtain $38.80$% and $13.93$% verified accuracy at distance $1$ and
$2$ respectively in the AG-News dataset, while being $4$ orders of magnitude
faster than existing approaches. We believe our work can open the door to more
efficient verification in the text domain.
comment: Accepted in ICLR 2025
♻ ☆ SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters ICLR 2025
Existing preference optimization objectives for language model alignment
require additional hyperparameters that must be extensively tuned to achieve
optimal performance, increasing both the complexity and time required for
fine-tuning large language models. In this paper, we propose a simple yet
effective hyperparameter-free preference optimization algorithm for alignment.
We observe that promising performance can be achieved simply by optimizing
inverse perplexity, which is calculated as the inverse of the exponentiated
average log-likelihood of the chosen and rejected responses in the preference
dataset. The resulting simple learning objective, SimPER, is easy to implement
and eliminates the need for expensive hyperparameter tuning and a reference
model, making it both computationally and memory efficient. Extensive
experiments on widely used real-world benchmarks, including MT-Bench,
AlpacaEval 2, and 10 key benchmarks of the Open LLM Leaderboard with 5 base
models, demonstrate that SimPER consistently and significantly outperforms
existing approaches-even without any hyperparameters or a reference model . For
example, despite its simplicity, SimPER outperforms state-of-the-art methods by
up to 5.7 points on AlpacaEval 2 and achieves the highest average ranking
across 10 benchmarks on the Open LLM Leaderboard. The source code for SimPER is
publicly available at: https://github.com/tengxiao1/SimPER.
comment: ICLR 2025
♻ ☆ OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking
Zekun Xi, Wenbiao Yin, Jizhan Fang, Jialong Wu, Runnan Fang, Ningyu Zhang, Jiang Yong, Pengjun Xie, Fei Huang, Huajun Chen
Machine writing with large language models often relies on
retrieval-augmented generation. However, these approaches remain confined
within the boundaries of the model's predefined scope, limiting the generation
of content with rich information. Specifically, vanilla-retrieved information
tends to lack depth, novelty, and suffers from redundancy, which negatively
impacts the quality of generated articles, leading to shallow, unoriginal, and
repetitive outputs. To address these issues, we propose OmniThink, a
slow-thinking machine writing framework that emulates the human-like process of
iterative expansion and reflection. The core idea behind OmniThink is to
simulate the cognitive behavior of learners as they slowly deepen their
knowledge of the topics. Experimental results demonstrate that OmniThink
improves the knowledge density of generated articles without compromising
metrics such as coherence and depth. Human evaluations and expert feedback
further highlight the potential of OmniThink to address real-world challenges
in the generation of long-form articles.
comment: Code is available at https://github.com/zjunlp/OmniThink
♻ ☆ CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs
Chinese, as a linguistic system rich in depth and complexity, is
characterized by distinctive elements such as ancient poetry, proverbs, idioms,
and other cultural constructs. However, current Large Language Models (LLMs)
face limitations in these specialized domains, highlighting the need for the
development of comprehensive datasets that can assess, continuously update, and
progressively improve these culturally-grounded linguistic competencies through
targeted training optimizations. To address this gap, we introduce CKnowEdit,
the first-ever Chinese knowledge editing dataset designed to correct
linguistic, factual, and logical errors in LLMs. We collect seven types of
knowledge from a wide range of sources, including classical texts, idioms, and
content from Baidu Tieba Ruozhiba, taking into account the unique polyphony,
antithesis, and logical structures inherent in the Chinese language. By
analyzing this dataset, we highlight the challenges current LLMs face in
mastering Chinese. Furthermore, our evaluation of state-of-the-art knowledge
editing techniques reveals opportunities to advance the correction of Chinese
knowledge. Code and dataset are available at
https://github.com/zjunlp/EasyEdit.
comment: Ongoing work; project website is available at
https://zjunlp.github.io/project/CKnowEdit code and dataset are available at
https://github.com/zjunlp/EasyEdit
♻ ☆ Non-Contextual BERT or FastText? A Comparative Analysis
Natural Language Processing (NLP) for low-resource languages, which lack
large annotated datasets, faces significant challenges due to limited
high-quality data and linguistic resources. The selection of embeddings plays a
critical role in achieving strong performance in NLP tasks. While contextual
BERT embeddings require a full forward pass, non-contextual BERT embeddings
rely only on table lookup. Existing research has primarily focused on
contextual BERT embeddings, leaving non-contextual embeddings largely
unexplored. In this study, we analyze the effectiveness of non-contextual
embeddings from BERT models (MuRIL and MahaBERT) and FastText models (IndicFT
and MahaFT) for tasks such as news classification, sentiment analysis, and hate
speech detection in one such low-resource language Marathi. We compare these
embeddings with their contextual and compressed variants. Our findings indicate
that non-contextual BERT embeddings extracted from the model's first embedding
layer outperform FastText embeddings, presenting a promising alternative for
low-resource NLP.
♻ ☆ Extracting Sentence Embeddings from Pretrained Transformer Models
Pre-trained transformer models shine in many natural language processing
tasks and therefore are expected to bear the representation of the input
sentence or text meaning. These sentence-level embeddings are also important in
retrieval-augmented generation. But do commonly used plain averaging or prompt
templates sufficiently capture and represent the underlying meaning? After
providing a comprehensive review of existing sentence embedding extraction and
refinement methods, we thoroughly test different combinations and our original
extensions of the most promising ones on pretrained models. Namely, given 110 M
parameters, BERT's hidden representations from multiple layers, and many
tokens, we try diverse ways to extract optimal sentence embeddings. We test
various token aggregation and representation post-processing techniques. We
also test multiple ways of using a general Wikitext dataset to complement
BERT's sentence embeddings. All methods are tested on eight Semantic Textual
Similarity (STS), six short text clustering, and twelve classification tasks.
We also evaluate our representation-shaping techniques on other static models,
including random token representations. Proposed representation extraction
methods improve the performance on STS and clustering tasks for all models
considered. Very high improvements for static token-based models, especially
random embeddings for STS tasks, almost reach the performance of BERT-derived
representations. Our work shows that the representation-shaping techniques
significantly improve sentence embeddings extracted from BERT-based and simple
baseline models.
comment: Postprint update
♻ ☆ Revisiting In-context Learning Inference Circuit in Large Language Models ICLR 2025
In-context Learning (ICL) is an emerging few-shot learning paradigm on
Language Models (LMs) with inner mechanisms un-explored. There are already
existing works describing the inner processing of ICL, while they struggle to
capture all the inference phenomena in large language models. Therefore, this
paper proposes a comprehensive circuit to model the inference dynamics and try
to explain the observed phenomena of ICL. In detail, we divide ICL inference
into 3 major operations: (1) Input Text Encode: LMs encode every input text (in
the demonstrations and queries) into linear representation in the hidden states
with sufficient information to solve ICL tasks. (2) Semantics Merge: LMs merge
the encoded representations of demonstrations with their corresponding label
tokens to produce joint representations of labels and demonstrations. (3)
Feature Retrieval and Copy: LMs search the joint representations of
demonstrations similar to the query representation on a task subspace, and copy
the searched representations into the query. Then, language model heads capture
these copied label representations to a certain extent and decode them into
predicted labels. Through careful measurements, the proposed inference circuit
successfully captures and unifies many fragmented phenomena observed during the
ICL process, making it a comprehensive and practical explanation of the ICL
inference process. Moreover, ablation analysis by disabling the proposed steps
seriously damages the ICL performance, suggesting the proposed inference
circuit is a dominating mechanism. Additionally, we confirm and list some
bypass mechanisms that solve ICL tasks in parallel with the proposed circuit.
comment: 37 pages, 41 figures, 8 tables. ICLR 2025 Accepted. Camera-ready
Version
♻ ☆ M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation
Recent advancements in large language models (LLMs) have given rise to the
LLM-as-a-judge paradigm, showcasing their potential to deliver human-like
judgments. However, in the field of machine translation (MT) evaluation,
current LLM-as-a-judge methods fall short of learned automatic metrics. In this
paper, we propose Multidimensional Multi-Agent Debate (M-MAD), a systematic
LLM-based multi-agent framework for advanced LLM-as-a-judge MT evaluation. Our
findings demonstrate that M-MAD achieves significant advancements by (1)
decoupling heuristic MQM criteria into distinct evaluation dimensions for
fine-grained assessments; (2) employing multi-agent debates to harness the
collaborative reasoning capabilities of LLMs; (3) synthesizing
dimension-specific results into a final evaluation judgment to ensure robust
and reliable outcomes. Comprehensive experiments show that M-MAD not only
outperforms all existing LLM-as-a-judge methods but also competes with
state-of-the-art reference-based automatic metrics, even when powered by a
suboptimal model like GPT-4o mini. Detailed ablations and analysis highlight
the superiority of our framework design, offering a fresh perspective for
LLM-as-a-judge paradigm. Our code and data are publicly available at
https://github.com/SU-JIAYUAN/M-MAD.
comment: Code and data are available at https://github.com/SU-JIAYUAN/M-MAD
♻ ☆ More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li
As large language models (LLMs) process increasing context windows, the
memory usage of KV cache has become a critical bottleneck during inference. The
mainstream KV compression methods, including KV pruning and KV quantization,
primarily focus on either token or precision dimension separately. However,
these works leaving the trade-off between these two orthogonal dimensions
largely under-explored. In this paper, we comprehensively investigate the
token-precision trade-off in KV cache compression.Experiments demonstrate that
storing more tokens in the KV cache with lower precision,a strategy we term
quantized pruning, can significantly enhance the long-context performance of
LLMs. In-depth analysis of the token-precision trade-off across key aspects
demonstrates that, quantized pruning achieves substantial improvements in
retrieval-related tasks and consistently performs well across varying input
lengths. Furthermore, quantized pruning demonstrates notable stability and
effectiveness across different KV pruning methods, quantization strategies, and
model scales. These findings offer valuable insights into optimizing KV cache
compression through balanced token-precision trade-off strategies. Our code is
available at https://github.com/zhzihao/QPruningKV.
comment: 13pages,9 figures
♻ ☆ T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
Text-to-image (T2I) models have rapidly advanced, enabling the generation of
high-quality images from text prompts across various domains. However, these
models present notable safety concerns, including the risk of generating
harmful, biased, or private content. Current research on assessing T2I safety
remains in its early stages. While some efforts have been made to evaluate
models on specific safety dimensions, many critical risks remain unexplored. To
address this gap, we introduce T2ISafety, a safety benchmark that evaluates T2I
models across three key domains: toxicity, fairness, and bias. We build a
detailed hierarchy of 12 tasks and 44 categories based on these three domains,
and meticulously collect 70K corresponding prompts. Based on this taxonomy and
prompt set, we build a large-scale T2I dataset with 68K manually annotated
images and train an evaluator capable of detecting critical risks that previous
work has failed to identify, including risks that even ultra-large proprietary
models like GPTs cannot correctly detect. We evaluate 12 prominent diffusion
models on T2ISafety and reveal several concerns including persistent issues
with racial fairness, a tendency to generate toxic content, and significant
variation in privacy protection across the models, even with defense methods
like concept erasing. Data and evaluator are released under
https://github.com/adwardlee/t2i_safety.
♻ ☆ Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples
The evaluation of cross-lingual semantic search capabilities of models is
often limited to existing datasets from tasks such as information retrieval and
semantic textual similarity. To allow for domain-specific evaluation, we
introduce Cross Lingual Semantic Discrimination (CLSD), a novel cross-lingual
semantic search task that does not require a large evaluation corpus, only
parallel sentences of the language pair of interest within the target domain.
This task focuses on the ability of a model to cross-lingually rank the true
parallel sentence higher than challenging distractors generated by a large
language model. We create a case study of our introduced CLSD task for the
language pair German-French in the news domain. Within this case study, we find
that models that are also fine-tuned for retrieval tasks benefit from pivoting
through English, while bitext mining models perform best directly
cross-lingually. A fine-grained similarity analysis enabled by our distractor
generation strategy indicate that different embedding models are sensitive to
different types of perturbations.
♻ ☆ How Much Do LLMs Hallucinate across Languages? On Multilingual Estimation of LLM Hallucination in the Wild
In the age of misinformation, hallucination -- the tendency of Large Language
Models (LLMs) to generate non-factual or unfaithful responses -- represents the
main risk for their global utility. Despite LLMs becoming increasingly
multilingual, the vast majority of research on detecting and quantifying LLM
hallucination are (a) English-centric and (b) focus on machine translation (MT)
and summarization, tasks that are less common ``in the wild'' than open
information seeking. In contrast, we aim to quantify the extent of LLM
hallucination across languages in knowledge-intensive long-form question
answering. To this end, we train a multilingual hallucination detection model
and conduct a large-scale study across 30 languages and 6 open-source LLM
families. We start from an English hallucination detection dataset and rely on
MT to generate (noisy) training data in other languages. We also manually
annotate gold data for five high-resource languages; we then demonstrate, for
these languages, that the estimates of hallucination rates are similar between
silver (LLM-generated) and gold test sets, validating the use of silver data
for estimating hallucination rates for other languages. For the final rates
estimation, we build a knowledge-intensive QA dataset for 30 languages with
LLM-generated prompts and Wikipedia articles as references. We find that, while
LLMs generate longer responses with more hallucinated tokens for
higher-resource languages, there is no correlation between length-normalized
hallucination rates of languages and their digital representation. Further, we
find that smaller LLMs exhibit larger hallucination rates than larger models.