Running the full test suite in CI catches regressions before they reach users. The challenge is keeping the pipeline fast — emulator setup is the biggest time sink.
Unit Tests (Fast, No Emulator)
# .github/workflows/unit-tests.yml
jobs:
unit-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with: { java-version: '17', distribution: 'temurin', cache: 'gradle' }
- name: Run unit tests
run: ./gradlew testDebugUnitTest
- name: Upload results
if: always()
uses: actions/upload-artifact@v3
with:
name: unit-test-results
path: "**/build/reports/tests/testDebugUnitTest/"
- name: Publish test report
uses: dorny/test-reporter@v1
if: always()
with:
name: Unit Tests
path: "**/build/test-results/testDebugUnitTest/*.xml"
reporter: java-junit
Instrumented Tests (Emulator Required)
instrumented-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with: { java-level: '17', distribution: 'temurin', cache: 'gradle' }
# Enable KVM for hardware acceleration (2-5x faster emulator boot)
- name: Enable KVM group perms
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | \
sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: Run instrumented tests
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 33
arch: x86_64
profile: Nexus 6
avd-name: test_avd
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim
disable-animations: true
script: ./gradlew connectedDebugAndroidTest
- name: Upload results
if: always()
uses: actions/upload-artifact@v3
with:
name: instrumented-test-results
path: "**/build/reports/androidTests/connected/"
Compose UI Tests in CI
Compose UI tests use createComposeRule() and run on the emulator as instrumented tests:
@get:Rule
val rule = createComposeRule()
@Test
fun articleList_showsTitle() {
rule.setContent {
ArticleListScreen(
articles = listOf(Article("1", "Hello World", "", 0L, ""))
)
}
rule.onNodeWithText("Hello World").assertIsDisplayed()
}
These run automatically under connectedDebugAndroidTest. No special CI config needed.
Screenshot Tests (Paparazzi — No Emulator)
screenshot-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with: { java-version: '17', distribution: 'temurin', cache: 'gradle' }
- name: Verify screenshots
run: ./gradlew verifyPaparazziDebug
- name: Upload diffs on failure
if: failure()
uses: actions/upload-artifact@v3
with:
name: screenshot-diffs
path: "**/build/paparazzi/failures/"
Emulator Matrix (Multi-API Testing)
instrumented-test:
runs-on: ubuntu-latest
strategy:
matrix:
api-level: [29, 31, 33]
include:
- api-level: 29
arch: x86
- api-level: 31
arch: x86_64
- api-level: 33
arch: x86_64
steps:
- uses: reactivecircus/android-emulator-runner@v2
with:
api-level: ${{ matrix.api-level }}
arch: ${{ matrix.arch }}
script: ./gradlew connectedDebugAndroidTest
Flaky Test Quarantine
- name: Run tests (retry on flake)
run: |
./gradlew connectedDebugAndroidTest || \
./gradlew connectedDebugAndroidTest || \
./gradlew connectedDebugAndroidTest
Better: mark known flaky tests with @FlakyTest and exclude from the main suite:
@FlakyTest(bugId = 123)
@Test
fun occasionallyFlakyTest() { /* ... */ }
# Exclude flaky tests from regular CI run
./gradlew connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.notAnnotation=androidx.test.filters.FlakyTest
Key Takeaways
| Test Type | CI Tool | Speed |
|---|---|---|
| Unit tests | testDebugUnitTest | Fast (< 2 min) |
| Screenshot tests | Paparazzi verifyPaparazziDebug | Fast (< 5 min, no emulator) |
| Instrumented | connectedDebugAndroidTest + emulator runner | Slow (10–20 min) |
| Multi-API matrix | GitHub Actions strategy.matrix | Parallel, scales linearly |
| Flaky tests | @FlakyTest annotation + exclusion filter | Quarantine, don't delete |