Kotlin Coroutines as the Missing Adapter for TDLib's Async Callbacks
Without changing the asynchronous nature, only the code organization: Kotlin coroutine practice in a Telegram project
Looking back at a project I did four years ago, I used TDLib to fetch Telegram group data.
The underlying API is typical Java asynchronous callback style.
The real time sink was not sending any single request, but organizing async steps like querying groups, joining groups, paging, querying users, and downloading files into a complete business flow.
Later, I reorganized this code with Kotlin coroutines.
I did not change the asynchronous nature of the underlying API, only the way business code is organized.
This is also a deepening understanding I had of Kotlin coroutines:
Coroutines are a linearized expression of asynchronous callbacks.
1. Project background: the business is not complex, the async flow is
Four years ago, I did a Telegram data-fetching project, using Telegram's official TDLib underneath.
The business involved was not particularly complex, mainly consisting of:
- Parsing group info from a group link.
- Searching for public groups and joining them.
- Joining groups through invite links.
- Getting basic group info.
- Fetching historical messages with pagination.
- Getting the group member list.
- Querying user profiles based on message senders.
- Downloading images, files, and user avatars from messages.
If you only look at the business requirements, these are all ordinary features.
There was also an interesting situation at the time.
The PM was more focused on business design.
For example, how a group should be searched, what info should be fetched after joining, how much historical message to get, and how images and files should be handled.
These discussions are fine in themselves.
But sometimes, a business flow that looks simple when drawn on a whiteboard does not mean the technical implementation is simple either.
Behind a single query action, there may be multiple async requests; an ordinary-looking pagination requirement may actually require deciding the next step based on the last request's result; a media download may involve multiple concurrent tasks and completion state management.
These were the things I cared about at the time.
To put it bluntly, a business flow can be designed beautifully, but if you don't account for the underlying API's async mechanism, the dependencies between requests, and the concurrency handling approach, the eventual technical implementation becomes very complex.
A good business design should not stop at the business flow diagram; it should also consider the complexity of the technical implementation.
And that was the problem I actually wanted to solve.
TDLib's API is async callback style; the true complexity lies not in any single request, but in the dependencies between requests.
For example, joining a group may require first parsing the link, then determining the group type, then performing a search or join operation, and finally getting the group details.
Fetching historical messages is not a single interface call. You need to decide based on the returned result whether there is a next page, then continue requesting until a stop condition is met.
If the fetched messages contain media, you also need to continue issuing file download requests.
When you combine these steps, you get a large number of async operations.
At the time, the team focused more on how single APIs were called, while I was concerned with how complex the whole business flow became once these APIs were combined.
At that time, I mainly considered two implementation approaches:
- Continue using Callback, writing business logic inside the callbacks.
- Use Java synchronization tools like Lock, CountDownLatch to wait for async requests to complete, then continue execution.
Both approaches solve the problem, but neither is ideal.
I wanted to find a way to let business code be organized in a normal execution order while preserving the asynchronous nature of the underlying API.
2. Java async programming: Callback Hell, or thread waiting?
TDLib's Java API is inherently asynchronous.
When you call an interface, you pass a request object and a callback; after the underlying processing completes, the result is returned via the callback.
The simplified call form looks like this:
client.send(function, new Client.ResultHandler() {
@Override
public void onResult(TdApi.Object result) {
// handle the result
}
});
A single request is not a problem.
But when a business flow requires multiple requests to execute sequentially, the problem appears.
2.1 Callback Hell
Suppose we need to join a group based on an invite link, then fetch group details.
The business flow is roughly:
If we use only Callback, the code becomes this:
client.send(
new TdApi.CheckChatInviteLink(inviteLink),
result -> {
if (result instanceof TdApi.ChatInvite) {
client.send(
new TdApi.JoinChatByInviteLink(inviteLink),
joinResult -> {
if (joinResult instanceof TdApi.Chat) {
TdApi.Chat chat = (TdApi.Chat) joinResult;
client.send(
new TdApi.GetChat(chat.id),
chatResult -> {
// process final result
}
);
}
}
);
}
}
);
This only shows three steps.
If we keep adding error handling, state checks, message pagination, and user info lookup, the nesting only gets deeper.
What's worse, the business logic is scattered across multiple callbacks.
A developer has to keep switching between different callbacks to understand the full business flow.
The business flow is supposed to be linear, but the code is split into multiple layers by callbacks.
2.2 Using sync-wait tools to wrap async calls as blocking calls
Another way is to use Java synchronization tools.
For example, after sending a request, wait for the callback to return the result, then continue with subsequent logic.
A simplified example:
public TdApi.Object sendFunction(
TdApi.Function function) {
CountDownLatch latch = new CountDownLatch(1);
AtomicReference<TdApi.Object> result =
new AtomicReference<>();
client.send(function, response -> {
result.set(response);
latch.countDown();
});
try {
latch.await();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
return result.get();
}
This approach looks like it solves the callback nesting problem.
The caller can write code in sequence:
TdApi.Object result1 = sendFunction(function1);
TdApi.Object result2 = sendFunction(function2);
TdApi.Object result3 = sendFunction(function3);
But it does not actually solve the async problem.
It just makes the calling thread wait for the async result.
If this happens on the main thread, it can cause an ANR; if many requests use this approach, it also introduces thread resource and concurrency management issues.
You can put both approaches side by side:
One leads code into callback nesting; the other leads threads into waiting.
2.3 Historical message pagination: waiting for the next page result
Fetching historical messages requires repeatedly deciding whether to continue based on the previous page's result.
For example:
public List<TdApi.Message> getMessages(
long chatId,
long fromMessageId,
int limit) {
List<TdApi.Message> messages = new ArrayList<>();
long messageId = fromMessageId;
while (true) {
TdApi.Messages result =
(TdApi.Messages) sendFunction(
new TdApi.GetChatHistory(
chatId,
messageId,
0,
limit,
false
)
);
if (result.messages.length == 0) {
break;
}
for (TdApi.Message message : result.messages) {
messages.add(message);
}
messageId = result.messages[
result.messages.length - 1
].id;
if (result.messages.length < limit) {
break;
}
}
return messages;
}
The point here is not the pagination algorithm itself.
It is that each request for the next page requires waiting for the previous page to return, and only then can you decide the next step.
Without the sync wrapper, pagination logic would have to live inside callbacks, either recursing or issuing further requests.
2.4 Media download: CountDownLatch
Consider another scenario.
After fetching a batch of messages, you may need to download the images or files in them.
If you want several files to download concurrently and then continue only after all complete, you need to handle task completion state.
In Java you can use CountDownLatch:
CountDownLatch latch =
new CountDownLatch(files.size());
for (TdApi.File file : files) {
client.send(
new TdApi.DownloadFile(
file.id,
32,
0,
0,
true
),
result -> {
// process download result
latch.countDown();
}
);
}
latch.await();
// continue after all download tasks complete
This style can express the need to wait for multiple tasks to finish.
But it also introduces extra synchronization management:
- You need to maintain the counter.
- You need to ensure every task calls
countDown()correctly. - You need to consider exceptions, timeouts, and task failures.
await()blocks the current thread.
The business logic itself is not complex, yet it requires dealing with a fair amount of concurrency detail.
That is also why I decided later to use coroutines to reorganize the code.
3. Kotlin coroutines: wrapping callbacks into suspend functions
Coroutines do not automatically turn TDLib's async API into a sync API.
They require an adapter layer that converts callbacks into Kotlin suspend functions.
At that time, the most central piece of the project was this layer.
3.1 From Callback to Continuation
Here is a simplified implementation:
suspend fun <T : TdApi.Object> sendFunctionAsync(
function: TdApi.Function<T>
): T = suspendCoroutine { continuation ->
Context.client.send(function) { result ->
continuation.resume(result as T)
}
}
What this code does is simple:
- Call TDLib's
send()method. - Register a result callback.
- The current coroutine suspends.
- When TDLib returns a result,
continuation.resume()resumes the coroutine.
There is a very important concept here:
What suspends is the coroutine, not the execution thread.
While waiting for TDLib to return, the current thread can continue with other tasks.
A sequence diagram can show this process:
sequenceDiagram
participant C as Coroutine
participant A as Adapter
participant T as TDLib
participant R as ResponseReceiver
C->>A: sendFunctionAsync()
A->>T: client.send()
Note over C: coroutine suspends
T-->>R: async result
R-->>A: execute ResultHandler
A-->>C: continuation.resume()
Note over C: coroutine resumes
This is the most important difference from blocking waits like CountDownLatch.await() or Object.wait().
From the caller's perspective, the original async callback has been wrapped into a suspend function that directly returns a result.
For example:
val chat = sendFunctionAsync(
TdApi.GetChat(chatId)
)
val messages = sendFunctionAsync(
TdApi.GetChatHistory(
chatId,
0,
0,
100,
false
)
)
The code executes in sequence.
But underneath it is still TDLib async requests.
We did not change the async nature, only the code organization.
3.2 From Callback to sequential code
Compare the two styles.
Java Callback:
client.send(function1, result1 -> {
client.send(function2, result2 -> {
client.send(function3, result3 -> {
// final processing
});
});
});
Kotlin Coroutine:
val result1 = sendFunctionAsync(function1)
val result2 = sendFunctionAsync(function2)
val result3 = sendFunctionAsync(function3)
// final processing
Both snippets express the same business flow.
The difference is that the Kotlin version does not require nesting the subsequent business logic inside callbacks.
For tasks that need to run sequentially, code can naturally be organized in business order.
4. Business refactoring: letting async flow return to the business itself
After completing the basic wrapping, the real value is in the changes at the business layer.
4.1 Search groups and join groups
For example, the user provides a Telegram group link.
Depending on the link type, different processing flows apply:
Using coroutines, business code can be organized like this:
suspend fun joinChat(
inviteLink: String
): TdApi.Chat {
val invite = sendFunctionAsync(
TdApi.CheckChatInviteLink(inviteLink)
)
return when (invite) {
is TdApi.ChatInvite -> {
sendFunctionAsync(
TdApi.JoinChatByInviteLink(inviteLink)
)
}
else -> {
throw IllegalStateException(
"Unsupported invite result"
)
}
}
}
This only shows the basic structure for the invite link flow.
Public groups can be fetched with SearchPublicChat and then joined with JoinChat.
If you are already a group member, you also need to handle the result returned by TDLib accordingly, then fetch the final group info.
The key point is: these business steps no longer need to be nested layer by layer through Callback.
Each step can be called independently, and they can be combined according to business needs.
4.2 Historical message pagination
Pagination logic can also be converted from callback recursion into a normal loop.
suspend fun getChatMessages(
chatId: Long,
limit: Int
): List<TdApi.Message> {
val result = mutableListOf<TdApi.Message>()
var fromMessageId = 0L
while (true) {
val response = sendFunctionAsync(
TdApi.GetChatHistory(
chatId,
fromMessageId,
0,
limit,
false
)
)
if (response.messages.isEmpty()) {
break
}
result.addAll(response.messages)
val nextMessageId = response.messages.last().id
// prevent the pagination cursor from not advancing
if (nextMessageId == fromMessageId) {
break
}
fromMessageId = nextMessageId
if (response.messages.size < limit) {
break
}
}
return result
}
Previously, you would have to issue the next page request inside a callback.
Now you just need a while loop.
Each request suspends the current coroutine, and continues after the result returns.
Of course, a real project also needs to handle pagination boundaries, duplicate pages, request exceptions, and business stop conditions based on what TDLib returns.
But at least, pagination logic has finally returned to what it should be.
5. Concurrency handling: CountDownLatch becomes async / awaitAll
After solving the sequential execution problem, there is still one important scenario: concurrency.
For example, after fetching a batch of group messages, you need to look up user profiles based on the message sender IDs.
If you query one by one, it is inefficient.
If you use Callback, you need to maintain multiple callbacks and task completion state.
If you use CountDownLatch, you have to manually manage the count.
Kotlin coroutines can directly use async and awaitAll.
5.1 Concurrent user profile lookup
suspend fun getUsers(
userIds: List<Long>
): List<TdApi.User> = coroutineScope {
userIds
.distinct()
.map { userId ->
async {
sendFunctionAsync(
TdApi.GetUser(userId)
)
}
}
.awaitAll()
}
This code expresses the intent clearly:
mapcreates a coroutine task for each user.asyncissues concurrent requests.awaitAll()waits for all tasks to complete.- Finally returns all user profiles.
No need to manually maintain a counter, and no need to manage thread waiting yourself.
However, there is still an engineering issue.
If userIds has tens of thousands of entries, directly creating tens of thousands of concurrent tasks is not necessarily reasonable.
Coroutines reduce the complexity of concurrency orchestration, but that does not mean you can ignore control over the number of concurrent operations.
You can use Semaphore to limit the maximum concurrency:
suspend fun getUsers(
userIds: List<Long>
): List<TdApi.User> = coroutineScope {
val semaphore = Semaphore(10)
userIds
.distinct()
.map { userId ->
async {
semaphore.withPermit {
sendFunctionAsync(
TdApi.GetUser(userId)
)
}
}
}
.awaitAll()
}
Here, at most 10 requests are allowed to enter the actual request phase at a time.
The specific concurrency number should be decided based on TDLib's request capability, server limits, and business needs.
5.2 Concurrent file download
The same idea works for media downloads:
suspend fun downloadFiles(
files: List<TdApi.File>
) = coroutineScope {
files.map { file ->
async {
sendFunctionAsync(
TdApi.DownloadFile(
file.id,
32,
0,
0,
true
)
)
}
}.awaitAll()
}
Compared to CountDownLatch, the coroutine version expresses the relationship between tasks more easily.
We no longer need to care about how the counter changes, nor manually manage waiting and wake-ups.
We only need to describe:
Start these tasks, and wait for all of them to complete.
6. Not every Callback should be converted to suspend
The earlier discussion focused on one-shot requests.
But TDLib also has another type of async interaction: continuous Updates events.
For example:
- New messages arrive.
- Group info changes.
- File download progress updates.
- User status changes.
These events are different from requests like GetChat or GetUser.
One-shot requests are usually:
send request -> get result -> end
Continuous events are:
register listener -> receive events continuously -> unregister listener
For the former, using suspend is more appropriate.
For the latter, using Flow is more natural.
For example, you can use callbackFlow to convert callback events into a Flow:
fun observeUpdates(): Flow<TdApi.Update> =
callbackFlow {
val listener = object : UpdateListener {
override fun onUpdate(update: TdApi.Update) {
trySend(update)
}
}
client.addUpdateListener(listener)
awaitClose {
client.removeUpdateListener(listener)
}
}
Here, UpdateListener is just an illustrative interface; the actual implementation should be based on the listener mechanism TDLib uses.
Through callbackFlow, the underlying continuous Callback can be converted into a Kotlin Flow.
The responsibilities of the two can be understood like this:
| Async operation | Kotlin expression | Applicable scenario |
|---|---|---|
| One-shot request | suspend | query group, query user |
| Continuous event | Flow | new messages, status updates |
| Concurrent tasks | async | concurrent queries, concurrent downloads |
| Wait for multiple tasks | awaitAll | wait for all tasks to finish |
Not every Callback should be changed to suspend; instead, you should choose the appropriate expression based on the semantics of the async operation.
7. Looking back: what did coroutines really change?
Looking back at this project, the initial problem was actually not that TDLib's API was not easy to use.
TDLib itself is an async library; it handles data interaction through request callbacks and an Updates mechanism.
The problem was how business code organized these async operations.
Through a coroutine adapter layer, we can isolate Callback at the bottom.
Business layer
|
v
Coroutine API
|
v
Callback Adapter
|
v
TDLib
The business layer does not need to know whether the underlying uses Callback, thread pools, or some other async mechanism.
An adapter layer in the middle handles the conversion.
This is, in fact, an adapter design idea.
Looking back, we had several options:
| Implementation | Main problem |
|---|---|
| Callback | layers of nested callbacks, business flow easily scattered |
| Lock / Wait | turns async waiting into thread blocking |
| CountDownLatch | manually manage concurrent task completion state |
| Kotlin Coroutine | organize business flow with suspend functions and structured concurrency |
Coroutines do not make the underlying requests truly synchronous.
Nor do they make TDLib's async mechanism disappear.
They simply provide a more natural way of expression:
- Sequential tasks use suspend functions.
- Concurrent tasks use
async. - Waiting for multiple tasks uses
awaitAll(). - Continuous events use Flow.
- Task lifecycle is managed through structured concurrency.
Business flows that previously had to be expressed through nested callbacks can now be expressed with sequential code.
Concurrent waits that previously had to be maintained manually can now be left to coroutines.
This is what I think is the most valuable part of Kotlin coroutines.
Async itself is not the problem; how to organize async operations is the problem.
Coroutines do not eliminate Callback; they just isolate Callback into an adapter layer.
When we stop focusing attention on callbacks, locks, and counters, and instead return to the business flow, the code truly becomes clear.
It is like how people communicate.
We want to tell a story in its natural order, rather than having to wait for another callback to tell us what to say next after each sentence.
Coroutines do not change the nature of async, they just finally let async code be organized the way human thinking works.
References
-
Telegram's official open-source TDLib project, including source code, Java interfaces, and usage examples.
-
TDLib official getting-started documentation, introducing TDLib's basic usage and its async request mechanism.