web research
30 articles about web research in AI news
Webify MCP: Cut Web Research Token Costs by 97% in Claude Code
Webify MCP slashes web research token usage by 97% in Claude Code using DOM graph + BM25 retrieval. Install with one pip command and let Claude auto-route web queries through it.
RiskWebWorld: A New Benchmark Exposes the Limits of AI for E-commerce Risk
Researchers introduced RiskWebWorld, a realistic benchmark for testing GUI agents on 1,513 authentic e-commerce risk management tasks. It reveals a major capability gap, showing even the best models fail over 50% of the time, highlighting the immaturity of AI for high-stakes operational automation.
Google DeepMind: Web Environment, Not Model Weights, Is Key AI Agent Attack Surface
Google DeepMind researchers present a systematic framework showing that the web environment itself—not just the model—is a primary attack surface for AI agents. In benchmarks, hidden prompt injections hijacked agents in up to 86% of scenarios, with memory poisoning attacks exceeding 80% success.
Ego2Web Benchmark Bridges Egocentric Video and Web Agents, Exposing Major Performance Gaps
Researchers introduce Ego2Web, the first benchmark requiring AI agents to understand real-world first-person video and execute related web tasks. Their novel Ego2WebJudge evaluation method achieves 84% human agreement, while state-of-the-art agents perform poorly across all task categories.
OpenCSF: A 1.5TB Free Computer Science Library Emerges from Unstructured Web Data
A new open-source dataset called OpenCSF has been compiled, containing 1.5TB of computer science materials scraped from public web sources. It provides a massive, free corpus for AI training and research in software engineering and CS education.
MiRA Framework Boosts Gemma3-12B to 43% Success Rate on WebArena-Lite, Surpassing GPT-4 and WebRL
Researchers propose MiRA, a milestone-based RL framework that improves long-horizon planning in LLM agents. It boosts Gemma3-12B's web navigation success from 6.4% to 43%, outperforming GPT-4-Turbo (17.6%) and the previous SOTA WebRL (38.4%).
AI Agents Learn to Plan Like Humans: New Framework Solves Complex Web Tasks
Researchers have developed STRUCTUREDAGENT, a hierarchical planning framework that enables AI web agents to tackle complex, multi-step tasks by using dynamic AND/OR trees and structured memory. The system achieves 46.7% success on challenging shopping tasks, outperforming existing methods.
New Benchmark Exposes Critical Gaps in AI's Ability to Navigate the Visual Web
Researchers unveil BrowseComp-V³, a challenging new benchmark testing multimodal AI's ability to perform deep web searches combining text and images. Even top models score only 36%, revealing fundamental limitations in visual-text integration and complex reasoning.
Open-Source FaceSwap Tool Enables Real-Time Webcam Swaps
Developer Gurisingh has released a free, open-source tool for real-time face-swapping on webcams. It works with live video calls and requires only a single source photo.
Tiny Fish Improves Live Web Usability for AI Coding Agents
Tiny Fish has released a tool that makes the live web significantly more usable for AI coding agents. This addresses a critical failure point where agent workflows often break down during real-world web interactions.
Keygraph Launches Shannon AI to Automate Web App Security Testing
Keygraph has launched 'Shannon,' an AI agent that autonomously hacks web applications to find security flaws. This positions AI as an offensive security tool for proactive defense.
Emergence WebVoyager: A New Benchmark Exposes Inconsistencies in Web Agent Evaluation
A new study introduces Emergence WebVoyager, a standardized benchmark for evaluating web-based AI agents. It reveals significant performance inconsistencies, showing OpenAI Operator's success rate is 68.6%, not 87%. This highlights a critical need for rigorous, transparent testing in agent development.
Alumnium MCP Hits 98.5% on WebVoyager: How to Add SOTA Browsing to Claude Code
The open-source Alumnium MCP server, which acts as a high-level browser subagent for Claude Code, just set a new state-of-the-art benchmark score. Install it to offload complex web tasks.
AI Coding Agent Rewrites Canon Webcam Software in Rust, Fixes Persistent Crashes
A developer used an AI coding agent to rewrite Canon's official, crash-prone webcam software. The agent produced a fully functional Rust application overnight, solving a problem that had persisted for years.
Open-Source 'Manus Alternative' Emerges: Fully Local AI Agent with Web Browsing, Code Execution, and Voice Input
An open-source project has been released that replicates core features of AI agent platforms like Manus—autonomous web browsing, multi-language code execution, and voice input—while running entirely locally on user hardware with no external API dependencies.
Stop Claude Code's Web Fetches from Burning 700K Tokens on HTML Junk
A new MCP server, token-enhancer, strips scripts, nav bars, and ads from web pages before they hit Claude's context, cutting token waste by 90%+.
AI2's MolmoWeb: Open 8B-Parameter Web Agent Navigates Using Screenshots, Challenges Proprietary Systems
The Allen Institute for AI released MolmoWeb, a fully open web agent that operates websites using only screenshots. The 8B-parameter model outperforms other open models and approaches proprietary performance, with all training data and weights publicly released.
Google DeepMind Unveils Gemini-Powered Browser That Generates Websites in Real-Time
Google DeepMind has demonstrated a browser prototype powered by Gemini 3.1 Flash-Lite that generates complete HTML/CSS websites dynamically based on user prompts and navigation context, shifting from static page retrieval to on-demand interface generation.
Firecrawl MCP Server: When to Upgrade from Fetch MCP for Web Scraping
Firecrawl's MCP server offers 12+ tools for advanced web scraping, but its 500-credit free tier and complex pricing mean you should only install it for specific, complex data extraction tasks.
China's DeepSeek-R1: Open-Source AI Agent Runs Locally with Web Search, Code Generation, and Built-In Computer
Chinese AI company DeepSeek has released DeepSeek-R1, a fully open-source AI agent that runs locally on personal computers with web search capabilities, code generation, and built-in computer functionality. The model represents a significant move toward accessible, self-contained AI systems outside the dominant U.S. ecosystem.
Cloudflare CEO Predicts AI Bot Traffic Will Surpass Human Web Traffic by 2027
Cloudflare CEO Matthew Prince forecasts that automated bot traffic will exceed human web traffic within three years, driven by the proliferation of AI agents. This projection highlights a fundamental shift in internet infrastructure demands.
Open-Source Web UI 'LLM Studio' Enables Local Fine-Tuning of 500+ Models, Including GGUF and Multimodal
LLM Studio, a free and open-source web interface, allows users to fine-tune over 500 large language models locally on their own hardware. It supports GGUF-quantized models, vision, audio, and embedding models across Mac, Windows, and Linux.
WebMCP: Turn Any Web Page into a Claude Code Tool with This Chrome Flag
WebMCP lets Claude Code interact directly with web pages via a Chrome extension, turning browsing sessions into structured data sources without scraping.
Open-Source Breakthrough Promises 'Invisible' Web Scraping Capabilities
A new 100% open-source Python library called 'ScrapeNinja' claims to make web scraping virtually undetectable by bot detection systems. The tool reportedly mimics human browsing patterns to bypass anti-scraping measures while remaining completely transparent and community-driven.
Crawlee: The Open-Source Web Scraping Library That Evades Modern Bot Detection
Crawlee, a 100% open-source Python library, enables developers to build web scrapers that bypass modern anti-bot systems with features like proxy rotation, headless browser support, and automatic retries.
Scrapy Revolutionizes Web Scraping: How This Open-Source Framework Is Democratizing Data Extraction
Scrapy, a powerful Python framework, enables developers to extract structured data from any website locally, eliminating SaaS dependencies and cloud costs. With 15+ years of production use and 59K GitHub stars, it offers enterprise-grade scraping capabilities for free.
How AI Agents Are Learning to Scrape the Web and Fine-Tune Models in One Go
A developer has integrated web scraping capabilities into HuggingFace's fine-tuning skill, enabling AI agents to collect data from protected platforms and automatically train custom models. This breakthrough addresses a major bottleneck in AI development workflows.
Perplexity's pplx-embed: The Bidirectional Breakthrough Transforming Web-Scale AI Retrieval
Perplexity has launched pplx-embed, a new family of multilingual embedding models that set state-of-the-art benchmarks for web-scale retrieval. Built on Qwen3 architecture with bidirectional attention, these models specifically address the noise and complexity of real-world web data.
The Proxy-Free Web Scraping Revolution: How AI APIs Are Changing Data Collection
A new generation of web scraping APIs eliminates the need for manual proxy management, handling thousands of pages automatically while avoiding blocks. This represents a major shift toward AI-driven data collection infrastructure.
Google Launches Deep Research Max Agent on Gemini 3.1 Pro
Google DeepMind rolled out Deep Research Max and standard Deep Research agents on Gemini 3.1 Pro, enabling autonomous web and proprietary data research via the Gemini API. The Max variant uses extended test-time compute for thorough asynchronous reports.