This fixes the agent-bot webhook delivery path so transient upstream
failures follow the expected delivery lifecycle. Existing fallback
behavior is preserved, and fallback actions are applied only after
delivery attempts are exhausted.
To reproduce, configure an agent-bot webhook endpoint to return 429/500
for message events. Before this fix, failure handling could be applied
too early; after this fix, delivery attempts complete first and then
existing fallback handling runs.
Tested with:
- bundle exec rspec spec/jobs/agent_bots/webhook_job_spec.rb
spec/lib/webhooks/trigger_spec.rb
- bundle exec rubocop spec/jobs/agent_bots/webhook_job_spec.rb
spec/lib/webhooks/trigger_spec.rb
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Adds an account-level setting `keep_pending_on_bot_failure` to control
whether conversations should move from pending to open when agent bot
webhooks fail.
Some users experience occasional message drops and don't want
conversations to automatically reopen due to transient bot failures.
This setting gives accounts control over that behavior. This is a
temporary setting which will be removed in future once a proper fix for
it is done, so it is not added in the UI.
## Description
Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).
This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
- **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues
The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.
## API Documentation
### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```
### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```
---
## 1. Search All Resources
**Endpoint:** `GET /api/v1/accounts/{account_id}/search`
Returns results from all searchable resources (contacts, conversations,
messages, articles).
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [...],
"conversations": [...],
"messages": [...],
"articles": [...]
}
}
```
---
## 2. Search Contacts
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`
Search contacts by name, email, phone number, or identifier with
optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [
{
"id": 42,
"email": "john@example.com",
"name": "John Doe",
"phone_number": "+1234567890",
"identifier": "user_123",
"additional_attributes": {},
"created_at": 1701234567
}
]
}
}
```
---
## 3. Search Conversations
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`
Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"conversations": [
{
"id": 123,
"display_id": 45,
"inbox_id": 1,
"status": "open",
"messages": [...],
"meta": {...}
}
]
}
}
```
---
## 4. Search Messages (Advanced)
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`
Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.
### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |
### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp
### Example Requests
**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response (Success)
```json
{
"payload": {
"messages": [
{
"id": 789,
"content": "I need a refund for my purchase",
"message_type": "incoming",
"created_at": 1701234567,
"conversation_id": 123,
"inbox_id": 3,
"sender": {
"id": 42,
"type": "contact"
}
}
]
}
}
```
### Example Response (90-day limit exceeded)
```json
{
"error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`
---
## 5. Search Articles
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`
Search help center articles by title or content.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"articles": [
{
"id": 456,
"title": "Installation Guide",
"slug": "installation-guide",
"portal_slug": "help",
"account_id": 1,
"category_name": "Getting Started",
"status": "published",
"updated_at": 1701234567
}
]
}
}
```
---
## Technical Implementation
### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)
### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)
---
## Type of change
- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)
---
## How Has This Been Tested?
### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
- Individual filters (`from`, `inbox_id`, time range)
- Combined filters
- Permission validation for inbox access
- Feature flag checks
- 90-day limit enforcement
- Error handling for exceeded time limits
### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb
# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```
### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:
```bash
# 1. Create test data
bundle exec rake search:setup_test_data
# 2. Start OpenSearch
mise elasticsearch-start
# 3. Reindex messages
rails runner "Message.search_index.import Message.all"
# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"
# 5. Test via API or Rails console
```
---
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---
## Additional Notes
### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
- Set `OPENSEARCH_URL` environment variable
- Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
- `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search
### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages
### Breaking Changes
- None. All new parameters are optional and backward compatible.
### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`
### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
# Changelog
When an agent bot webhook fails, we now flip any pending conversation
back to an open state so a human agent can pick
it up immediately. There will be an clear activity message giving the
team clear visibility into what went
wrong. This keeps customers from getting stuck in limbo when their
connected bot goes offline.
# Testing instructions
1. Initial setup: Create an agent bot with a working webhook URL and
connect it to a test inbox. Send a message from a
contact (e.g., via the widget) so a conversation is created; it should
enter the Pending state while the bot handles
the reply.
2. Introduce failure: Edit that agent bot and swap the webhook URL for a
dummy endpoint that will fail. Have the same
contact send another message in the existing conversation. Because the
webhook call now fails, the conversation should flip from Pending back
to Open, making it visible to agents. Also verify the activity message
3. New conversation check: With the dummy URL still in place, start a
brand-new conversation from a contact. When the
bot tries (and fails) to respond, confirm that the conversation appears
immediately as Open rather than remaining Pending. Also the activity
message is visible
4. Subsequent messages in open conversations will show no change
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
