The project
Build a voice note assistant with push-to-talk, transcript correction, interruptible playback and a text-only fallback.
Prerequisites
Complete the six teaching modules and their exercises. Keep the design notes and failure cases: they form the starting evidence for this project.
Build in increments
1. What a voice AI pipeline contains
Understand the responsibilities behind a spoken conversation. Apply the module decision to this project. Your checkpoint: You can name each latency and correctness boundary.
2. Capturing speech and detecting turns
Collect intentional input and preserve user control. Apply the module decision to this project. Your checkpoint: The user can always tell whether the app is recording.
3. Transcription and language understanding
Keep uncertain speech separate from confirmed intent. Apply the module decision to this project. Your checkpoint: An interim transcript cannot trigger a permanent action.
4. Speech synthesis and playback
Deliver understandable audio without losing application state. Apply the module decision to this project. Your checkpoint: Text and audio represent the same approved user-facing response.
5. Interruptions and conversation state
Handle overlapping speech without mixing turns. Apply the module decision to this project. Your checkpoint: No output from a superseded turn can re-enter active playback.
6. Latency, accessibility and reliability
Evaluate the entire voice experience under realistic conditions. Apply the module decision to this project. Your checkpoint: Your voice feature remains usable when speech input or output is unavailable.
Acceptance criteria
- Display when the microphone is active
- Measure capture-to-first-audio latency by pipeline stage
- Stop stale speech on interruption and release audio resources
Evidence to submit
- A repository or reproducible design artifact with setup instructions and tested tool versions.
- A diagram showing components, data flow, authority and failure boundaries.
- An evaluation table with inputs, expected results, actual results and explanations of failures.
- A demonstration of the normal path and at least two relevant failure paths.
- A short decision record covering alternatives, known limitations and the next improvement.
Review rubric
| Dimension | Ready to demonstrate | Revisit when |
|---|---|---|
| Understanding | Explain each boundary and why it exists | You can only repeat framework terminology |
| Implementation | Reproduce the core behavior from a clean setup | Hidden local state is required |
| Reliability | Show bounded failures and useful recovery | The demo only works on the happy path |
| Evidence | Present observed outcomes and limitations | Success is inferred from a generated answer |
| Ownership | Modify a requirement and explain the consequences | You cannot adapt the implementation independently |
Do not treat a completed reading checklist as proof of project readiness. Revisit any module whose checkpoint you cannot demonstrate.