fix: Optimize Slack channel fetching to avoid rate limiting issues (#12542)

### Problem
The Slack integration fails when fetching channels from workspaces with
many channels due to rate limiting errors. The current implementation
makes a single API call requesting both public and private channels
simultaneously with `types: 'public_channel,private_channel'`, which
causes Slack's API to apply complex filtering and hit rate limits more
frequently.

  When testing with a csutomer workspace containing 157 channels:
- Combined request: Hit rate limits after a few pages, required 22+ API
calls with long delays
- Separate requests: Private channels (1 channel) load instantly, public
channels (185 channels) load quickly
  
 
  ### Solution

  Split the channel fetching into two sequential steps:
1. **Fetch private channels first** with `limit: 1000` (expects very
few)
  2. **Fetch public channels second** with pagination as needed

This approach leverages the fact that Slack's API handles single-type
requests much more efficiently than mixed-type requests, avoiding the
rate limiting issues entirely while maintaining the same functionality.
This commit is contained in:
Muhsin Keloth
2025-09-29 14:41:48 +05:30
committed by GitHub
parent 487209574c
commit 6c6aaf573c

View File

@@ -24,13 +24,31 @@ class Integrations::Slack::ChannelBuilder
end
def channels
conversations_list = slack_client.conversations_list(types: 'public_channel,private_channel', exclude_archived: true)
# Split channel fetching into separate API calls to avoid rate limiting issues.
# Slack's API handles single-type requests (public OR private) much more efficiently
# than mixed-type requests (public AND private). This approach eliminates rate limits
# that occur when requesting both channel types simultaneously.
channel_list = []
# Step 1: Fetch all private channels in one call (expect very few)
private_channels = fetch_channels_by_type('private_channel')
channel_list.concat(private_channels)
# Step 2: Fetch public channels with pagination
public_channels = fetch_channels_by_type('public_channel')
channel_list.concat(public_channels)
channel_list
end
def fetch_channels_by_type(channel_type, limit: 1000)
conversations_list = slack_client.conversations_list(types: channel_type, exclude_archived: true, limit: limit)
channel_list = conversations_list.channels
while conversations_list.response_metadata.next_cursor.present?
conversations_list = slack_client.conversations_list(
cursor: conversations_list.response_metadata.next_cursor,
types: 'public_channel,private_channel',
exclude_archived: true
types: channel_type,
exclude_archived: true,
limit: limit
)
channel_list.concat(conversations_list.channels)
end