Integrating Gemini into a Flutter App the Right Way

Dropping an LLM call behind a button is easy. Shipping one that survives real users — flaky networks, rate limits, cancelled screens, and the occasional 500 from the model — is where most Flutter integrations fall apart. This is the checklist I run through before an AI feature goes anywhere near production.

Keep the key off the device

The single most common mistake is bundling an API key into the app. Anything shipped in a binary can be extracted, and a leaked Gemini key is a billing incident waiting to happen.

Put the model call behind your own backend — a Cloud Function, a small server, or Firebase AI Logic, which brokers Gemini access through Firebase Auth so no raw key ever ships. The client talks to your endpoint; your endpoint holds the secret.

final response = await _client.post(
  Uri.parse('$backendUrl/ask'),
  headers: {'Authorization': 'Bearer $idToken'},
  body: jsonEncode({'prompt': prompt}),
);

The Flutter app authenticates the user; the backend authenticates to Gemini. Two separate trust boundaries.

Stream tokens, don't block

A model can take several seconds to finish a full answer. Blocking the UI on the complete response feels broken. Stream the tokens instead and render them as they arrive, so the first words show up almost immediately.

Expose the stream from a service

Model your service around a Stream<String> so the UI never touches the transport layer.

Stream<String> streamAnswer(String prompt) async* {
  final request = http.Request('POST', Uri.parse('$backendUrl/ask/stream'))
    ..headers['Authorization'] = 'Bearer $idToken'
    ..body = jsonEncode({'prompt': prompt});
 
  final response = await _client.send(request);
  await for (final chunk in response.stream.transform(utf8.decoder)) {
    yield chunk;
  }
}

Accumulate on the UI side

In the widget, fold each chunk into the visible text. With Riverpod this stays declarative:

final answerProvider = StreamProvider.autoDispose.family<String, String>(
  (ref, prompt) async* {
    final buffer = StringBuffer();
    await for (final chunk in ref.read(aiServiceProvider).streamAnswer(prompt)) {
      buffer.write(chunk);
      yield buffer.toString();
    }
  },
);

autoDispose matters here: when the user leaves the screen, the provider tears down and the underlying request is cancelled instead of billing you for tokens nobody will read.

Handle failure like it's the common case

LLM calls fail more often than a typical REST endpoint. Plan for four buckets and give each a distinct UI:

• Network errors — offer a retry, keep the typed prompt.
• Rate limits (429) — back off and tell the user to wait, don't hammer.
• Content blocked — Gemini may refuse a prompt; show why, don't crash.
• Timeouts — cap the request and fail gracefully rather than spinning forever.

try {
  await for (final partial in streamAnswer(prompt)) {
    state = AsyncData(partial);
  }
} on RateLimitException catch (e) {
  state = AsyncError('Please wait ${e.retryAfter}s', StackTrace.current);
} on TimeoutException {
  state = AsyncError('The model took too long. Try again.', StackTrace.current);
}

Respect rate limits before the server does

The free tier gives you room to build, but a chatty UI can burn through requests fast. Two guards go a long way:

1. Debounce input-driven calls so you aren't firing on every keystroke.
2. Serialize in-flight requests per screen — cancel the previous stream before starting a new one.

Both are cheap on the client and save you from 429s that the user would otherwise see.

Design an offline-first fallback

Mobile means intermittent connectivity. When the model is unreachable, the feature shouldn't feel dead:

• Cache the last successful answer keyed by prompt, so repeat questions resolve instantly.
• For narrow tasks — classification, extraction, simple intent detection — ship an on-device model with TensorFlow Lite or ML Kit and fall back to it when the network is gone.
• At minimum, surface a clear "You're offline" state instead of a spinner that never resolves.

Future<String> answer(String prompt) async {
  if (await _connectivity.isOnline) {
    return _gemini.complete(prompt);
  }
  return _cache.get(prompt) ?? _onDevice.classify(prompt);
}

The pre-ship checklist

Before you call the integration done:

• No API key in the client binary
• Responses stream and render incrementally
• Every failure bucket has its own UI state
• Requests cancel when the screen is disposed
• Input is debounced and in-flight calls are serialized
• There's a sensible offline path

Get these six right and the AI feature stops being a demo and starts being something you can hand to real users.