on device
30 articles about on device in AI news
Roboflow's RF-DETR Model Ported to Apple MLX, Enabling Real-Time On-Device Instance Segmentation
Roboflow's RF-DETR object detection model is now available on Apple's MLX framework, enabling real-time instance segmentation on Apple Silicon devices. This port unlocks new on-device visual analysis applications for robotics and augmented vision-language models.
Google Releases Magenta RealTime 2 for Open-Weight Music Generation
Google released Magenta RealTime 2 on Hugging Face, the only open-weights model for real-time continuous music generation on device with ~200ms latency.
MobileMem: On-Device Memory From a Year of Phone Data
MobileMem trains an LLM on a year of mobile data for on-device memory. Paper and code released, but no benchmarks disclosed.
Amazon Designs Custom AI Silicon for Future Devices, Panay Says
Amazon hardware chief Panos Panay confirmed Amazon is designing its own end-to-end silicon for some devices, signaling a strategic push into custom AI hardware.
Apple Core AI Runs Models On-Device, Zero Server Calls
Apple launched Core AI for on-device model inference on Apple silicon. Zero server calls, supports Qwen, Mistral, SAM3 across devices.
OpenMedKit Adds GLiNER for On-Device PII Detection on iPhone
OpenMedKit is adding the GLiNER zero-shot named entity recognition framework to its toolkit, expanding its on-device, privacy-preserving PII detection capabilities for healthcare data on iPhones.
HUOZIIME: A Research Framework for On-Device LLM-Powered Input Methods
A new research paper introduces HUOZIIME, a personalized on-device input method powered by a lightweight LLM. It uses a hierarchical memory mechanism to capture user-specific input history, enabling privacy-preserving, real-time text generation tailored to individual writing styles.
ModelBest Hits $1B+ Valuation for On-Device Foundation Models
ModelBest, a Chinese developer of on-device AI foundation models, raised several hundred million RMB, reaching a valuation exceeding $1 billion. The funding will accelerate its push to deploy efficient models directly on smartphones and IoT devices.
Ethan Mollick: Gemma 4 Impressive On-Device, But Agentic Workflows Doubted
Wharton professor Ethan Mollick finds Google's Gemma 4 powerful for on-device use but is skeptical about its ability to execute true agentic workflows, citing limitations in judgment and self-correction.
Apple's On-Device Reranking Model for Private Visual Search: A Technical Breakdown
Analysis of Apple's Enhanced Visual Search system that uses multimodal features, geo-signals, and index debiasing to identify landmarks entirely on-device. This represents a significant advancement in privacy-preserving AI for visual recognition.
Apple Reportedly Gains Full Internal Access to Google's Gemini for On-Device Model Distillation
A report claims Apple's AI deal with Google includes full internal model access, enabling distillation of Gemini's reasoning into smaller, on-device models. This would allow Apple to build specialized, efficient AI without relying solely on cloud APIs.
KAIST Develops 'SoulMate' AI Chip for Real-Time, On-Device Personalization
KAIST researchers have developed a new AI semiconductor, 'SoulMate,' that enables real-time, on-device learning of user habits and preferences. The chip combines RAG and LoRA for instant personalization while consuming minimal power, aiming for commercialization by 2027.
Stanford's OpenJarvis: The Open-Source Framework Bringing Personal AI Agents to Your Device
Stanford researchers have released OpenJarvis, an open-source framework for building personal AI agents that operate entirely on-device. This local-first approach prioritizes privacy and autonomy while providing tools, memory, and learning capabilities.
Open-Source Project Unlocks Apple's On-Device AI for Any Device on Your Network
Perspective Intelligence Web, an open-source project, enables any device with a browser to access Apple's powerful on-device AI models running locally on a Mac. This MIT-licensed solution addresses privacy concerns by keeping all processing on your private network while extending Apple Intelligence capabilities to Windows, Linux, Android, and Chromebook devices.
Edge AI Breakthrough: Qwen3.5 2B Runs Locally on iPhone 17 Pro, Redefining On-Device Intelligence
Alibaba's Qwen3.5 2B model now runs locally on iPhone 17 Pro devices, marking a significant breakthrough in edge AI. This development enables sophisticated language processing without cloud dependency, potentially transforming mobile AI applications and user privacy paradigms.
Google's AI Edge Gallery Arrives on iPhone: A Privacy-First Revolution in On-Device Intelligence
Google AI Edge Gallery has launched on iOS, bringing true on-device function calling to iPhones for the first time. Powered by the compact 270M parameter FunctionGemma model, it enables natural voice commands to trigger phone actions like calendar events and flashlight toggles—completely offline.
Google's AICore Beta Enables On-Device Gemini Nano 4 Downloads for Android Phones
A new beta of Google's AICore system service enables users to download Gemini Nano 4 Full and Gemini Nano 4 Fast models directly onto compatible Android phones, including those with Snapdragon 8 Elite Gen 5 chips. This moves beyond pre-installed AI to user-initiated model management.
Apple's Private Cloud Compute: Leak Suggests 4x M2 Ultra Cluster for On-Device AI Offload
A leak suggests Apple's Private Cloud Compute for AI may be built on clusters of four M2 Ultra chips, potentially offering high-performance, private server-side processing for iPhone AI tasks. This would mark Apple's strategic move into dedicated, privacy-focused AI infrastructure.
Perplexity AI Launches On-Device Search Engine: Privacy-First AI Comes Home
A new privacy-first AI search engine called Perplexity AI now runs entirely on users' own hardware, eliminating cloud data transmission. This breakthrough represents a significant shift toward decentralized, secure AI processing that protects user queries from corporate surveillance.
The Laptop Agent Revolution: How 24B-Parameter Models Are Redefining On-Device AI
Liquid's LFM2-24B-A2B model runs locally on laptops, selecting tools in under 400ms. Its hybrid architecture enables sparse activation, making powerful AI agents practical for regulated industries and developers without cloud dependencies.
Apple's Neural Engine Jailbroken: Researchers Unlock On-Device AI Training Capabilities
A researcher has reverse-engineered Apple's private Neural Engine APIs to enable direct transformer training on M-series chips, bypassing CoreML restrictions. This breakthrough could enable battery-efficient local model training and fine-tuning without cloud dependency.
Matic Robot Vacuum Ships 20-Year-Old Vision Research
Navneet Dalal's 50,000-cited INRIA vision research now ships in Matic, a 5-camera robot vacuum with on-device NVIDIA processing. The product maps homes in 3D in 20 minutes, 20 years after the underlying patents were filed.
Qwen 3.7 27B Draws Best Local-Model Pelican, Simon Willison Says
Qwen 3.7 27B, as a 17GB GGUF, drew Simon Willison's best local pelican-bicycle image. Signals on-device generation crossed a quality threshold.
OpenAI's First Device: Doughnut-Shaped Speaker, No Display
OpenAI's first device is a doughnut-shaped, display-less smart speaker, per Bloomberg. It's battery-powered with moving parts for interaction, targeting voice-first AI computing.
PrismML Shrinks Qwen 3.6 to iPhone 17 Pro, Apple Eyes Deal
PrismML compressed Alibaba's 36B-parameter Qwen 3.6 to run on an iPhone 17 Pro, drawing Apple's interest for on-device AI without cloud latency.
llada.cpp Cuts LLaDA-8B Latency 17-42x on Mobile NPU
llada.cpp, the first NPU-aware dLLM inference framework, cuts LLaDA-8B latency 17-42x on smartphones, enabling real-time on-device generation.
Apple AFM Core Advanced: Sparse, Multimodal, iPhone 17 Pro Only
Apple AFM Core Advanced is sparse, multimodal, and exclusive to iPhone 17 Pro, M3+ Mac, M4+ iPad, while AFM Core is dense for other devices.
Memory Supply Squeeze Hits Non-AI Sectors as DRAM Prices Double
DRAM prices surged 93-98% QoQ in Q1 2026 as AI data centers consume fab capacity, nine industry groups warned the Trump administration on June 3, threatening supply for automotive, telecom, and medical devices.
MIT Hackathon Team Builds Wearable AI for Physical Movement Guidance
MIT hackathon team builds wearable AI for real-time physical movement guidance via sensors and on-device inference, demoed by @kimmonismus.
Developer Achieves 395x RTFx on M5 Max with Fastest Parakeet v3 for Apple ANE
Developer @mweinbach has optimized the Parakeet v3 speech recognition model for Apple's Neural Engine, achieving a 395x real-time factor on an M5 Max chip. This represents a significant performance leap for on-device AI inference on Apple Silicon.