Executors & ThreadPool Tuning

Kotlin coroutines abstract over thread pools, but the thread pools are still there. Dispatchers.IO is backed by a ThreadPoolExecutor. Knowing how thread pools work — and how to tune them — lets you size your work queues correctly, diagnose throughput bottlenecks, and understand why your coroutine-based code behaves the way it does under load.

What Is a Thread Pool?

A thread pool is a managed collection of pre-created threads sitting idle, waiting for work. Submitting a task to the pool picks an idle thread (or queues the task if all threads are busy), runs it, and returns the thread to the pool when done.

Benefits over creating a thread per task:

• No creation cost — threads are expensive to create (~1 ms, ~1 MB stack)
• Bounded resource usage — you control how many threads exist
• Reuse — threads return to the pool rather than being destroyed

ThreadPoolExecutor

All standard executors in Java/Kotlin are backed by ThreadPoolExecutor:

Copy codeThreadPoolExecutor(
    int corePoolSize,        // min threads kept alive, even if idle
    int maximumPoolSize,     // max threads ever created
    long keepAliveTime,      // how long idle non-core threads survive
    TimeUnit unit,
    BlockingQueue<Runnable> workQueue, // holds tasks when all threads are busy
    ThreadFactory threadFactory,
    RejectedExecutionHandler handler   // what happens when queue is full
)

The lifecycle of a submitted task:

Copy codeSubmit task
│
├─ If active threads < corePoolSize → create new thread (even if others are idle)
│
├─ If threads >= corePoolSize → try to queue in workQueue
│
├─ If queue is full and threads < maximumPoolSize → create new non-core thread
│
└─ If queue is full and threads == maximumPoolSize → RejectedExecutionHandler fires

The Four Standard Executors

FixedThreadPool

Copy codeval executor = Executors.newFixedThreadPool(4)

• corePoolSize == maximumPoolSize == n
• Unbounded LinkedBlockingQueue
• Use for: predictable, CPU-bound workloads where you want hard concurrency limits

Risk: Unbounded queue means tasks pile up without bound if processing is slower than submission. Under a traffic spike, memory can grow until OOM.

CachedThreadPool

Copy codeval executor = Executors.newCachedThreadPool()

• corePoolSize = 0, maximumPoolSize = Integer.MAX_VALUE
• SynchronousQueue (no buffering — task must be handed off immediately)
• Idle threads live for 60 seconds
• Use for: short-lived tasks with unpredictable bursts; scales up fast

Risk: Can create thousands of threads under extreme load, starving the system.

SingleThreadExecutor

Copy codeval executor = Executors.newSingleThreadExecutor()

• One thread, unbounded queue
• Guarantees serial execution order
• Use for: writing to a shared file, serializing DB migrations, ordered event processing

ScheduledThreadPool

Copy codeval scheduler = Executors.newScheduledThreadPool(2)
scheduler.scheduleAtFixedRate({ pollServer() }, 0, 30, TimeUnit.SECONDS)
scheduler.schedule({ sendHeartbeat() }, 5, TimeUnit.SECONDS)

• Use for: polling, retries with delay, heartbeats

On Android, prefer WorkManager for deferred work that must survive process death. Use ScheduledThreadPool only for in-process, in-session scheduling.

Work Queue Types

The queue determines how tasks wait when all threads are busy:

For Android background work with bounded memory: ArrayBlockingQueue + CallerRunsPolicy is a safe combination:

Copy codeval executor = ThreadPoolExecutor(
    2, 4,
    60L, TimeUnit.SECONDS,
    ArrayBlockingQueue(20),
    Executors.defaultThreadFactory(),
    ThreadPoolExecutor.CallerRunsPolicy() // caller thread runs the task on queue full
)

CallerRunsPolicy provides natural backpressure — if the pool is full, the calling thread does the work itself, slowing the submission rate.

How Dispatchers.IO Is Tuned

Dispatchers.IO uses a shared ThreadPoolExecutor with:

• corePoolSize = max(64, number of CPU cores)
• No hard maximum — it can grow beyond 64 if all threads are blocked
• limitedParallelism(n) carves out a sub-pool to prevent one subsystem from using all IO threads

Copy code// Prevent one feature from monopolizing Dispatchers.IO
private val imageIoDispatcher = Dispatchers.IO.limitedParallelism(4)

suspend fun downloadImage(url: String) = withContext(imageIoDispatcher) {
    // Only 4 concurrent downloads, even if IO has 64 threads
}

Sizing Your Thread Pool

The formula depends on work type:

IO-bound work (waiting on network/disk):

threads ≈ number_of_cores × (1 + wait_time / compute_time)

If threads spend 90% of time waiting: threads = cores × 10

CPU-bound work (computation):

threads ≈ number_of_cores (± 1)

More threads than cores causes context-switching overhead without benefit.

Practical example — fetching 100 images on a 4-core device:

• Each fetch takes ~200ms total; 10ms compute, 190ms waiting
• Optimal IO threads ≈ 4 × (1 + 190/10) = 4 × 20 = 80 threads

This is why Dispatchers.IO defaults to 64 — it's tuned for IO-bound work.

Coroutine Dispatcher vs Executor: When to Use Which

Converting an Executor to a coroutine Dispatcher:

Copy codeval myExecutor = Executors.newFixedThreadPool(4)
val myDispatcher = myExecutor.asCoroutineDispatcher()

viewModelScope.launch(myDispatcher) {
    processChunk(data)
}

// Clean up when done
myDispatcher.close()
myExecutor.shutdown()

Key Takeaways