arxiv
30 articles about arxiv in AI news
Hugging Face OCRs 27,000 arXiv Papers to Markdown with Open 5B Model
Hugging Face CEO Clement Delangue announced the OCR conversion of 27,000 arXiv papers to Markdown using an open 5B-parameter model and 16 parallel jobs on L40S GPUs. This demonstrates a scalable, open-source pipeline for large-scale academic document processing.
New arXiv Paper Proposes LLM-Generated 'Reference Documents' to Speed Up
A new arXiv preprint introduces a method for efficient LLM-based reranking. It uses LLMs to generate 'reference documents' that help dynamically truncate long ranked lists and optimize batch processing, achieving up to 66% speedup on TREC benchmarks.
New arXiv Study Finds No Saturation Point for Data in Traditional Recommender Systems
A new arXiv preprint systematically tests how recommendation model performance scales with training data size. Using 10 algorithm variants across 11 large datasets, the research finds that normalized performance (NDCG@10) generally keeps improving up to 100 million interactions, with no clear saturation point for typical models.
arXiv Paper Proposes Federated Multi-Agent System with AI Critics for Network Fault Analysis
A new arXiv paper introduces a collaborative control algorithm for AI agents and critics in a federated multi-agent system, providing convergence guarantees and applying it to network telemetry fault detection. The system maintains agent privacy and scales with O(m) communication overhead for m modalities.
arXiv Paper Proposes 'Connections' Word Game as New Benchmark for AI Agent Social Intelligence
A new arXiv preprint introduces the improvisational word game 'Connections' as a benchmark for evaluating social intelligence in AI agents. It requires agents to gauge the cognitive states of others, testing collaborative reasoning beyond individual knowledge retrieval.
Agent Judges with Big Five Personas Match Human Evaluators, Show Logarithmic Score Saturation in New arXiv Study
A new arXiv study shows LLM agents conditioned with Big Five personalities produce evaluations indistinguishable from humans. Crucially, quality scores saturate logarithmically with panel size, while discovering unique issues follows a slower power law.
arXiv Survey Maps KV Cache Optimization Landscape: 5 Strategies for Million-Token LLM Inference
A comprehensive arXiv review categorizes five principal KV cache optimization techniques—eviction, compression, hybrid memory, novel attention, and combinations—to address the linear memory scaling bottleneck in long-context LLM inference. The analysis finds no single dominant solution, with optimal strategy depending on context length, hardware, and workload.
Federated Fine-Tuning Benchmark Shows QLoRA Nears Centralized Accuracy on
Sherpa.ai's arXiv benchmark shows federated fine-tuning with QLoRA matches centralized accuracy on four healthcare and finance datasets, outperforming isolated single-institution learning under non-IID conditions.
DPAA Debiases GNN Recommenders by Reweighting Message Passing
arXiv paper 2605.11145 proposes DPAA, a debiasing framework for GNN-based CF that applies adaptive weighting during message passing, outperforming prior methods.
MM-LLM Framework Boosts Recommendation AUC 0.35%, Online Metrics 0.02%
arXiv paper proposes LLaMA2-based MM-LLM framework for recommendation, achieving 0.35% AUC gain and 0.02% online lift at scale.
LLMs Fail at Implicit Travel Constraints, New Benchmark Shows
LLMs fail at implicit travel constraints, a new arXiv paper decomposes planning into 5 atomic skills, finding structural biases and ineffective self-correction.
LLM-Based Customer Digital Twins Predict Preferences with 87.7% Accuracy
A new arXiv paper proposes using LLM-based 'customer digital twins' (CDTs) — agents built from individual Reddit review histories via RAG — to perform conjoint analysis. The CDTs predict actual user preferences with 87.73% accuracy in a computer monitor case study, offering a scalable alternative to traditional market research.
DharmaOCR: New Small Language Models Set State-of-the-Art for Structured
A new arXiv preprint presents DharmaOCR, a pair of small language models (7B & 3B params) fine-tuned for structured OCR. They introduce a new benchmark and use Direct Preference Optimization to drastically reduce 'text degeneration'—a key cause of performance failures—while outputting structured JSON. The models claim superior accuracy and lower cost than proprietary APIs.
New Research Proposes CPGRec
A new arXiv paper introduces CPGRec, a three-module framework for video game recommendations. It aims to solve the common trade-off between accuracy and diversity by using strict game connections and leveraging category/popularity data. Experiments on a Steam dataset show promising results.
Indexing Multimodal LLMs for Large-Scale Image Retrieval
A new arXiv paper proposes using Multimodal LLMs (MLLMs) for instance-level image-to-image retrieval. By prompting models with paired images and converting next-token probabilities into scores, the method enables training-free re-ranking. It shows superior robustness to clutter and occlusion compared to specialized models, though struggles with severe appearance changes.
FeCoSR: A Federated Framework for Cross-Market Sequential Recommendation
A new arXiv paper introduces FeCoSR, a federated collaboration framework for cross-market sequential recommendation. It tackles data isolation and market heterogeneity by enabling many-to-many collaborative training with a novel loss function, showing advantages over traditional transfer approaches.
New Research Proposes Authority-aware Generative Retrieval (AuthGR) for
A new arXiv paper introduces an Authority-aware Generative Retriever (AuthGR) framework. It uses multimodal signals to score document trustworthiness and trains a model to prioritize authoritative sources. Large-scale online A/B tests on a commercial search platform report significant improvements in user engagement and reliability.
New Research Adapts Deep Interest Network for Time-Sensitive
A new arXiv paper details a recommendation engine for daily fantasy sports that explicitly models time-sensitivity and urgency. The system adapts the Deep Interest Network (DIN) architecture with real-time urgency features and temporal positional encodings, achieving a significant performance gain over a traditional baseline.
Bi-Predictability: A New Real-Time Metric for Monitoring LLM
A new arXiv paper introduces 'bi-predictability' (P), an information-theoretic measure, and a lightweight Information Digital Twin (IDT) architecture to monitor the structural integrity of multi-turn LLM conversations in real-time. It detects a 'silent uncoupling' regime where outputs remain semantically sound but the conversational thread degrades, offering a scalable tool for AI assurance.
New Research Proposes Unified LLM Framework for Need-Driven Service
A new arXiv paper introduces a large language model framework that unifies living need prediction and service recommendation for local life services. It uses behavioral clustering to filter noise and a curriculum learning + RL strategy to navigate complex decision paths. Experiments show it significantly improves both need prediction and recommendation accuracy.
LLM-HYPER: A Training-Free Framework for Cold-Start Ad CTR Prediction
A new arXiv paper introduces LLM-HYPER, a framework that treats large language models as hypernetworks to generate parameters for click-through rate estimators in a training-free manner. It uses multimodal ad content and few-shot prompting to infer feature weights, drastically reducing the cold-start period for new promotional ads and has been deployed on a major U.S. e-commerce platform.
Is Sliding Window All You Need? An Open Framework for Long-Sequence
A new arXiv paper provides a complete, open-source framework for training long-sequence recommender systems using sliding windows. It demonstrates up to +6.34% recall gains on retail data and introduces a novel embedding layer for large vocabularies, making the technique practical for academic and industrial research.
SID-Coord: A New Framework for Balancing Memorization and Generalization
A new arXiv paper introduces SID-Coord, a framework that integrates trainable Semantic IDs (SIDs) with traditional Hashed IDs (HIDs) in ranking models. It aims to solve the memorization-generalization trade-off, improving performance on long-tail items. Online A/B tests in a production short-video search system showed statistically significant improvements in engagement metrics.
ContextSim: A New LLM Framework for Context-Aware Recommender System Simulation
A new arXiv preprint introduces ContextSim, a framework that uses LLM agents to simulate users interacting with recommender systems within realistic daily scenarios (time, location, needs). Experiments show it generates more human-aligned interactions and that RS parameters optimized with it yield improved real-world engagement.
FashionStylist: New Expert-Annotated Dataset Aims to Unify Multimodal
A new arXiv preprint introduces FashionStylist, a dataset with professional fashion annotations for item grounding, outfit completion, and outfit evaluation. It aims to address the fragmentation in existing fashion AI benchmarks by providing expert-level reasoning data.
New Research Establishes State-of-the-Art for Virtual Try-Off with
A new arXiv paper introduces a systematic framework for Virtual Try-Off (VTOFF)—reconstructing a garment's canonical form from a worn image. The Dual-UNet Diffusion model achieves state-of-the-art results on standard datasets, providing foundational insights for this emerging computer vision task.
CoDiS: A Causal Framework for Cross-Domain Sequential Recommendation
A new arXiv paper introduces CoDiS, a framework for Cross-Domain Sequential Recommendation that uses causal inference to disentangle domain-shared and domain-specific user preferences while addressing context confounding and gradient conflicts. It outperforms state-of-the-art baselines on three real-world datasets.
Beyond Dense Connectivity: Explicit Sparsity for Scalable Recommendation
A new arXiv paper introduces SSR, a framework that builds explicit sparsity into recommendation model architectures. It addresses the inefficiency of dense models (like MLPs) when processing high-dimensional, sparse user data, showing superior performance and scalability on datasets including AliExpress.
Walmart Research Proposes Unified Training for Sponsored Search Retrieval
A new arXiv preprint details Walmart's novel bi-encoder training framework for sponsored search retrieval. It addresses the limitations of using user engagement as a sole training signal by combining graded relevance labels, retrieval priors, and engagement data. The method outperformed the production system in offline and online tests.
SMTPO: A New Framework for Multi-Turn Conversational Recommendation Using Simulated Users and RL
A new arXiv paper introduces SMTPO, a framework for conversational recommender systems. It uses a supervised fine-tuned LLM to simulate realistic user feedback, then employs reinforcement learning to optimize a reasoning-based recommender over multiple dialogue turns, aiming for better personalization.