Posted by Jolanda Verhoef, Senior Developer Relations Engineer, Android Developer Relations

0713%20Jetpacker%20Blog.png

Building GenAI features in your app usually means navigating through various models, APIs and architecture choices: 

  • Execution location: Where does your model run? On device, in the cloud, or both?
  • Complexity: How complex is your setup? Are you doing a single inference call or do you need a more agentic flow?
  • In-app or Android System: Should your feature be built into your Android app or does it fit better as an Android system integration?

In this blog post series we'll navigate these choices with you. We will take you along on a journey, starting with a basic mobile app and transforming it into a personalized, intelligent, and agentic experience.

Jetpacker: a demo travel app

Jetpacker is a technical showcase app that our team built from the ground up for this year's Google I/O (built using Antigravity). At its core, Jetpacker helps users plan, explore, and enjoy their next big adventure. It shows an overview of your trips, the itinerary of each trip, and details of each event on that trip. Of course following all best practices of Android development, including a beautifully expressive Material UI design.

And best of all? It's fully open source!

Today we are publishing a series of technical blog posts diving deep into each of these features. We’ll provide detailed implementation steps, code snippets, and architectural insights to help you build your own intelligent Android applications.

On-device intelligence

on-device-features.png

On-device features in Jetpacker: Summarizing trip itineraries, managing expenses, and voice notes

Using an on-device model comes with no additional cloud inference costs, means you don't have to worry about internet connectivity, and lets users be confident that private information will be processed locally, on the device, without any of their data being sent to the cloud.

In Jetpacker, we chose on-device inference for three of our features:

  • The trip overview feature transforms a messy, multi-day itinerary into a concise, actionable summary. It leverages Gemini Nano through the ML Kit GenAI APIs to process data locally on the device. We consider this a nice-to-have feature where we don't want to incur extra cloud costs, making on-device inference the right choice.
  • The expense tracker automatically extracts structured data from receipt images to help users track their travel spending. It uses the multimodal capabilities of Gemini Nano 4 through the ML Kit GenAI APIs. We choose an on-device solution so that any privacy-sensitive information on the receipt images never leaves the user's device.
  • The audio diary records, transcribes, and categorizes voice notes into relevant trip activities. It is powered by the ML Kit Speech Recognition and GenAI Prompt APIs. We chose an on-device solution for privacy and connectivity reasons.

Cloud & hybrid inference

cloud-hybrid-features.png

Cloud and hybrid features in Jetpacker: Museum assistant with web grounding, hybrid restaurant review drafting, and hotel support chat featuring custom-routed live translation.

Sometimes your use-case requires AI models with greater world knowledge or a much larger context window and with greater ability in handling complex tasks. In that case, we can switch from running an on-device model to using a cloud model instead.

Or, if you want to get the best of both worlds, you can use hybrid inference to dynamically choose either a cloud or on-device model at runtime. This allows us to lower costs by moving inference to the device when it is available, but at the same time support all Android devices running the app.

In Jetpacker, we implemented several features using cloud or hybrid inference:

  • The place Q&A feature answers user questions about specific locations by grounding responses in real-world data. It uses Firebase AI Logic integrated with Google Maps and web context. Using a cloud model is necessary here for its greater world knowledge.
  • The review drafting feature helps users compose detailed reviews for the places they have visited. It leverages both on-device and cloud models through Firebase AI Logic's new Hybrid inference API. This is a feature we wanted to make available to all app users, so we're using a cloud model as a fallback when an on-device model is unavailable.
  • The automatic chat translation dynamically translates chat messages in real time to facilitate seamless communication, demonstrating custom hybrid inference logic. Again, we want this feature to be available to all app users, but at the same time have some specific considerations on when to choose on-device versus cloud.

System integration

While not a feature you see in the app itself, the Android system integration opens up the app's core capabilities directly to the Android operating system. It uses the AppFunctions API to integrate with system-level intelligence.

In-app agentic workflows (coming soon!)

agentic-feature-booking-assistant%20(1).png

The booking assistant shows several in-progress flight bookings, asking the user for input before making a final booking.

Agenticness introduces a higher level of autonomy, enabling models to act as agents. Instead of a single inference call, an agent works towards a specific goal via an orchestration loop that allows it to reason, use tools, and adapt its path. Depending on your requirements, these intelligent agents can run either in the cloud, directly on-device, or in a hybrid setup.

For Jetpacker we added a booking assistant that automates end-to-end booking workflows directly within the application to streamline reservations. It is built using A2UI and ADK running in the cloud. The Android app functions as a front-end to the multi-agentic system running in the cloud.

Learn more

Check out the other parts of this blog post series:

Part 1 (this post!): Introduction of the app and a high-level overview.
Part 2: On-device intelligence. Deep-dive into ML Kit’s GenAI APIs and Gemini Nano to build privacy-first features like itinerary summarization, receipt parsing, and local audio processing.
Part 3: Hybrid and cloud reasoning. Explore how to use Firebase AI Logic to ground LLM answers in real-world data like Google Maps and web context.
Part 4: System integration. Integrating with the Android intelligence system using AppFunctions.
Part 5 (coming soon): In-app agentic workflows. Extend the app with an end-to-end booking assistant powered by A2UI and ADK.

Interested in more on Android Development? Follow Android Developers on YouTube or LinkedIn!