fix: Optimize Slack channel fetching to avoid rate limiting issues (#12542)
### Problem The Slack integration fails when fetching channels from workspaces with many channels due to rate limiting errors. The current implementation makes a single API call requesting both public and private channels simultaneously with `types: 'public_channel,private_channel'`, which causes Slack's API to apply complex filtering and hit rate limits more frequently. When testing with a csutomer workspace containing 157 channels: - Combined request: Hit rate limits after a few pages, required 22+ API calls with long delays - Separate requests: Private channels (1 channel) load instantly, public channels (185 channels) load quickly ### Solution Split the channel fetching into two sequential steps: 1. **Fetch private channels first** with `limit: 1000` (expects very few) 2. **Fetch public channels second** with pagination as needed This approach leverages the fact that Slack's API handles single-type requests much more efficiently than mixed-type requests, avoiding the rate limiting issues entirely while maintaining the same functionality.
This commit is contained in:
@@ -24,13 +24,31 @@ class Integrations::Slack::ChannelBuilder
|
||||
end
|
||||
|
||||
def channels
|
||||
conversations_list = slack_client.conversations_list(types: 'public_channel,private_channel', exclude_archived: true)
|
||||
# Split channel fetching into separate API calls to avoid rate limiting issues.
|
||||
# Slack's API handles single-type requests (public OR private) much more efficiently
|
||||
# than mixed-type requests (public AND private). This approach eliminates rate limits
|
||||
# that occur when requesting both channel types simultaneously.
|
||||
channel_list = []
|
||||
|
||||
# Step 1: Fetch all private channels in one call (expect very few)
|
||||
private_channels = fetch_channels_by_type('private_channel')
|
||||
channel_list.concat(private_channels)
|
||||
|
||||
# Step 2: Fetch public channels with pagination
|
||||
public_channels = fetch_channels_by_type('public_channel')
|
||||
channel_list.concat(public_channels)
|
||||
channel_list
|
||||
end
|
||||
|
||||
def fetch_channels_by_type(channel_type, limit: 1000)
|
||||
conversations_list = slack_client.conversations_list(types: channel_type, exclude_archived: true, limit: limit)
|
||||
channel_list = conversations_list.channels
|
||||
while conversations_list.response_metadata.next_cursor.present?
|
||||
conversations_list = slack_client.conversations_list(
|
||||
cursor: conversations_list.response_metadata.next_cursor,
|
||||
types: 'public_channel,private_channel',
|
||||
exclude_archived: true
|
||||
types: channel_type,
|
||||
exclude_archived: true,
|
||||
limit: limit
|
||||
)
|
||||
channel_list.concat(conversations_list.channels)
|
||||
end
|
||||
|
||||
Reference in New Issue
Block a user