Measure the core before adding another input
PocketCook's committed core has no camera preview, frame encoder or camera transport. This lesson preserves the existing roadmap URL while separating a practical voice evaluation from an advanced camera design exercise. Do not search the app for a camera toggle or present this chapter as an implemented visual assistant.
Start by writing expected behavior before asking Gemini. Evaluation is not collecting impressive replies. It is checking whether a specified user task succeeds, including when the assistant lacks information or an operation is unavailable.
Build a task-level evaluation sheet
Use the bundled Tomato pasta recipe for consistency. For each case, record recipe step, exact question, expected behavior, actual outcome, model, device, route and whether the result was a live run or a simulation. Do not store API keys or incidental conversations.
| Case | Expected behavior | Evidence |
|---|---|---|
| “What ingredients do I need?” | Answer agrees with the selected recipe | Compare with Recipes.kt |
| “What should I do at this step?” | Guidance matches the app's current position | Note current UI step before asking |
| “Can you repeat only that action?” | Concise relevant repeat | Live observation |
| Interrupt during a response | Old queued speech does not resume | Recording or timestamped notes with consent |
| “Start a timer” | No claim that an actual timer was started | Compare voice with app state |
| Ambiguous substitution | Clarify rather than invent a specific allergy guarantee | Task notes |
| Network unavailable | Recoverable state; manual steps remain usable | Device failure run |
These are expectations for evaluation, not promises that every stochastic reply will satisfy them. A failed case belongs in the report. Repeat a representative question several times to expose variability instead of keeping only a favorable answer.
Latency needs defined endpoints
Separate at least three measurements: start action to setup acknowledgement; end of the user's utterance to first audible response; interruption event to stopped old playback. The first uses app events, the second includes speech detection and model processing, and the third measures a local response to a known event. A stopwatch estimate should be labeled approximate.
Case ID:
Source revision / model / date:
Phone and Android version / speaker or headset:
Start condition and expected result:
Connection latency:
End-of-utterance to first audible response:
Interruption-event to playback stopped:
Observed result and repetitions:
Measurement method and limitations:
No benchmark threshold is claimed by this course. Record your baseline first, then choose a budget appropriate to the product. A ten-minute PCM unit simulation verifies ordering across many fragments; it is not ten minutes of networking, acoustic echo or model behavior.
Understand the layers of evidence
The core was built and tested with 19 unit tests, five emulator tests and two physical-device audio tests; see its pinned verification report. Voice success was also reported by the author. Those results do not establish a complete device matrix, a Bluetooth guarantee or a sustained cloud-session benchmark.
Your learning evidence should distinguish deterministic tests, platform tests and live task evaluation. If API access is unavailable, complete the offline and test sections and mark live cases unverified. Do not substitute prerecorded or fake answers and describe them as Gemini.
Advanced design: what would a useful camera add?
A possible extension lets a user deliberately share a frame to ask about an ingredient. Specify an explicit enable/disable action, permission denial, a visible preview, frame age and bounded preparation/sending. Prefer the newest useful frame over an ever-growing backlog. Stale visual context can be worse than no context if the phone moved or the ingredient changed.
The Live API overview describes image support. Actual camera work also needs a device-side capture implementation, which is not included in this source revision. Review the relevant model and API limits again when implementing it rather than copying a limit from an old lesson.
Frame absence should lead to a clarification, not a claim that the assistant can see the kitchen. Appearance alone cannot establish that food is safe. Keep the design exercise focused on recognizing supported context and communicating uncertainty, not issuing safety guarantees.
Finish with a defensible report
Submit the table with at least seven cases, repeated live attempts where available, one failure analysis and a proposed next experiment. Include the exact source commit and your own changes. A polished screenshot motivates learners; only reproducible behavior and clearly bounded claims establish that the feature works.
Course study guide · Hands-on codelab · Pinned Android source