Twitter's home timeline is one of the most-read data sets in the world. Understanding how it's generated server-side and how the client caches it reveals tradeoffs every engineer faces with any high-scale feed.
Server-Side: Fanout Models
Push Fanout (Write-Heavy)
When a user tweets, the tweet is immediately pushed to the home timeline of every follower:
User A tweets → write to A's 10,000 followers' timeline caches → tweet immediately readable
- Pros: reads are instant (pre-computed timeline); very fast for readers
- Cons: write amplification; celebrities with 100M followers = 100M cache writes per tweet
Pull Fanout (Read-Heavy)
When a user opens the app, the timeline is computed by fetching tweets from all followed accounts:
User opens app → fetch latest tweets from 500 followed accounts → merge and sort → display
- Pros: no write amplification; always fresh
- Cons: slow read; N+1 query problem
Hybrid (Twitter's Actual Approach)
- Regular users: push fanout
- Celebrities (>1M followers): pull fanout at read time
- Pre-compute timelines for active users; evict timelines of inactive users (< 30 days)
Client-Side: Timeline Caching Strategy
@Entity(tableName = "timeline")
data class TimelineEntry(
@PrimaryKey val tweetId: String,
val authorId: String,
val authorName: String,
val authorHandle: String,
val authorAvatarUrl: String,
val text: String,
val mediaUrls: List<String>,
val likeCount: Int,
val retweetCount: Int,
val replyCount: Int,
val createdAt: Long,
val fetchedAt: Long = System.currentTimeMillis(),
// Denormalized for display; accept eventual consistency
val isLikedByMe: Boolean = false,
val isRetweetedByMe: Boolean = false
)
@Dao
interface TimelineDao {
@Query("SELECT * FROM timeline ORDER BY created_at DESC LIMIT 200")
fun observeTimeline(): Flow<List<TimelineEntry>>
@Query("DELETE FROM timeline WHERE fetched_at < :cutoff")
suspend fun evictOlderThan(cutoff: Long)
@Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insertAll(entries: List<TimelineEntry>)
}
Cache Invalidation Strategy
class TimelineCacheManager(private val dao: TimelineDao) {
// Keep only 200 tweets in cache; evict anything older than 1 week
suspend fun trimCache() {
val oneWeekAgo = System.currentTimeMillis() - 7 * 24 * 60 * 60 * 1000L
dao.evictOlderThan(oneWeekAgo)
// Room Query to keep only top 200 by createdAt
}
// Invalidate and refresh on pull-to-refresh
suspend fun refresh(api: TimelineApi) {
val fresh = api.getTimeline(cursor = null, count = 50)
dao.insertAll(fresh.map { it.toEntity() })
trimCache()
}
}
Showing the "New Tweets" Indicator
@HiltViewModel
class TimelineViewModel @Inject constructor(
private val dao: TimelineDao,
private val api: TimelineApi
) : ViewModel() {
val timeline: StateFlow<List<TimelineEntry>> = dao.observeTimeline()
.stateIn(viewModelScope, SharingStarted.WhileSubscribed(5000), emptyList())
private val _newTweetCount = MutableStateFlow(0)
val newTweetCount: StateFlow<Int> = _newTweetCount
private var newestTweetId: String? = null
init {
pollForNewTweets()
}
private fun pollForNewTweets() = viewModelScope.launch {
while (isActive) {
delay(60_000) // check every minute
newestTweetId?.let { sinceId ->
try {
val newTweets = api.getTimeline(sinceId = sinceId, count = 10)
if (newTweets.isNotEmpty()) {
_newTweetCount.value = newTweets.size
}
} catch (e: IOException) {
// Ignore; try again next poll
}
}
}
}
fun onScrolledToTop(newestId: String) {
newestTweetId = newestId
_newTweetCount.value = 0
}
fun onRefreshRequested() = viewModelScope.launch {
_newTweetCount.value = 0
// Invalidate pager or refresh Room cache
}
}
Handling Retweets and Quotes in the Feed
sealed class TimelineItem {
data class OriginalTweet(val entry: TimelineEntry) : TimelineItem()
data class Retweet(val retweeter: UserSummary, val original: TimelineEntry) : TimelineItem()
data class QuoteTweet(val entry: TimelineEntry, val quoted: TimelineEntry) : TimelineItem()
data class Thread(val tweets: List<TimelineEntry>) : TimelineItem()
}
fun List<TimelineEntry>.toTimelineItems(): List<TimelineItem> {
return mapNotNull { entry ->
when {
entry.retweetedFrom != null -> TimelineItem.Retweet(
retweeter = entry.author,
original = findOriginal(entry.retweetedFrom) ?: return@mapNotNull null
)
entry.quotedTweetId != null -> TimelineItem.QuoteTweet(
entry = entry,
quoted = findOriginal(entry.quotedTweetId) ?: return@mapNotNull null
)
else -> TimelineItem.OriginalTweet(entry)
}
}
}
Key Takeaways
| Pattern | Rule |
|---|---|
| Hybrid fanout | Regular users push; celebrities pull at read time |
| Local Room cache | Display cached timeline instantly on launch; sync in background |
| 200 tweet limit | Don't cache more than 200 on device; evict oldest first |
| New tweet indicator | Poll every 60s; show count banner; don't jump scroll |
| Pull-to-refresh | Invalidate top of cache; prepend new tweets |
| Denormalize for display | Store authorName in timeline entry; avoids join on every render |