Google Launches AI Inference “LiteRT.js“

Google Launches AI Inference “LiteRT.js“

Google has introduced LiteRT.js, a new high-performance JavaScript runtime designed to run machine learning and generative AI models directly inside web browsers using local device hardware acceleration. The release aims to make browser-based AI significantly faster, cheaper, and more private by moving inference from the cloud to the user’s device.  

LiteRT.js is part of Google’s broader LiteRT ecosystem, the successor to TensorFlow Lite, which powers AI experiences across billions of devices worldwide. While LiteRT already supports Android, iOS, desktop, and embedded platforms, LiteRT.js extends those capabilities to the web with native support for modern browser acceleration technologies such as WebGPU and future WebNN support.  

What LiteRT.js Brings to Web Developers

According to Google, LiteRT.js offers several advantages over traditional browser AI approaches:

• ⚡ Hardware-accelerated inference through WebGPU

• 🔒 Fully local execution with improved privacy

• 💰 Zero inference costs after model download

• 📶 Offline AI experiences

• 🧠 Support for both ML and GenAI workloads

• 🔄 Easier migration path from PyTorch models

One of the biggest improvements is the model conversion pipeline. Developers can convert models directly from PyTorch, TensorFlow, or JAX into .tflite models without requiring the complex conversion chains commonly used by TensorFlow.js workflows.  

Built for the New Era of Web AI

Google says LiteRT.js can power a wide range of browser-native AI experiences, including:

• Real-time image processing

• Speech recognition

• Document understanding

• AI assistants

• Computer vision applications

• On-device LLM inference

By running models locally, applications benefit from lower latency while avoiding server-side GPU costs and API dependencies.  

Works Alongside TensorFlow.js

Rather than replacing TensorFlow.js, Google positions LiteRT.js as a complementary runtime.

Developers can continue using TensorFlow.js for preprocessing and postprocessing while swapping the model execution layer with LiteRT.js for improved performance. Migration typically involves replacing model loading while leaving most application logic unchanged.  

Why It Matters

The AI industry is rapidly moving beyond cloud-only inference.

Running AI directly inside browsers offers several advantages:

• Lower operating costs

• Improved privacy

• Reduced latency

• Better offline support

• Lower infrastructure requirements

For developers, LiteRT.js lowers the barrier to building AI-native web applications that feel instantaneous and don’t require expensive backend infrastructure. As WebGPU adoption accelerates, browser-based AI could become a viable alternative for many workloads that previously required cloud GPUs.