LeadMagnet/leadchat
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

feat: Advanced Search Backend (#12917)

Browse Source 
## Description

Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).

This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
   - **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues

The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.

## API Documentation

### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```

### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```

---

## 1. Search All Resources

**Endpoint:** `GET /api/v1/accounts/{account_id}/search`

Returns results from all searchable resources (contacts, conversations,
messages, articles).

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|

### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "contacts": [...],
    "conversations": [...],
    "messages": [...],
    "articles": [...]
  }
}
```

---

## 2. Search Contacts

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`

Search contacts by name, email, phone number, or identifier with
optional time filtering.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |

### Example Requests

**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "contacts": [
      {
        "id": 42,
        "email": "john@example.com",
        "name": "John Doe",
        "phone_number": "+1234567890",
        "identifier": "user_123",
        "additional_attributes": {},
        "created_at": 1701234567
      }
    ]
  }
}
```

---

## 3. Search Conversations

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`

Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |

### Example Requests

**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "conversations": [
      {
        "id": 123,
        "display_id": 45,
        "inbox_id": 1,
        "status": "open",
        "messages": [...],
        "meta": {...}
      }
    ]
  }
}
```

---

## 4. Search Messages (Advanced)

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`

Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.

### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |

### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp

### Example Requests

**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response (Success)
```json
{
  "payload": {
    "messages": [
      {
        "id": 789,
        "content": "I need a refund for my purchase",
        "message_type": "incoming",
        "created_at": 1701234567,
        "conversation_id": 123,
        "inbox_id": 3,
        "sender": {
          "id": 42,
          "type": "contact"
        }
      }
    ]
  }
}
```

### Example Response (90-day limit exceeded)
```json
{
  "error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`

---

## 5. Search Articles

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`

Search help center articles by title or content.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |

### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "articles": [
      {
        "id": 456,
        "title": "Installation Guide",
        "slug": "installation-guide",
        "portal_slug": "help",
        "account_id": 1,
        "category_name": "Getting Started",
        "status": "published",
        "updated_at": 1701234567
      }
    ]
  }
}
```

---

## Technical Implementation

### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)

### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)

---

## Type of change

- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)

---

## How Has This Been Tested?

### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
  - Individual filters (`from`, `inbox_id`, time range)
  - Combined filters
  - Permission validation for inbox access
  - Feature flag checks
  - 90-day limit enforcement
  - Error handling for exceeded time limits

### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb

# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```

### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:

```bash
# 1. Create test data
bundle exec rake search:setup_test_data

# 2. Start OpenSearch
mise elasticsearch-start

# 3. Reindex messages
rails runner "Message.search_index.import Message.all"

# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"

# 5. Test via API or Rails console
```

---

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---

## Additional Notes

### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
  - Set `OPENSEARCH_URL` environment variable
  - Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
  - `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search

### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages

### Breaking Changes
- None. All new parameters are optional and backward compatible.

### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`

### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)

---------

Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
This commit is contained in:
Vinay Keerthi
2026-01-07 15:30:49 +05:30
committed by GitHub
parent 566de02385
commit 59cbf57e20
49 changed files with 3816 additions and 385 deletions
Download Patch File Download Diff File Expand all files Collapse all files

240
spec/controllers/api/v1/accounts/search_controller_spec.rb
View File

@@ -69,6 +69,80 @@ RSpec.describe 'Search', type: :request do
expect(response_data[:payload].keys).to contain_exactly(:contacts)
expect(response_data[:payload][:contacts].length).to eq 1
end
it 'returns last_activity_at in contact search results' do
contact = create(:contact, email: 'activity@test.com', account: account, last_activity_at: 3.days.ago)
get "/api/v1/accounts/#{account.id}/search/contacts",
headers: agent.create_new_auth_token,
params: { q: 'activity' },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
contact_result = response_data[:payload][:contacts].first
expect(contact_result[:last_activity_at]).to eq(contact.last_activity_at.to_i)
expect(contact_result).not_to have_key(:created_at)
end
context 'with advanced_search feature enabled', :opensearch do
before do
account.enable_features!('advanced_search')
end
it 'filters contacts by since parameter' do
create(:contact, email: 'old@test.com', account: account, last_activity_at: 10.days.ago)
create(:contact, email: 'recent@test.com', account: account, last_activity_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/contacts",
headers: agent.create_new_auth_token,
params: { q: 'test', since: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
contact_emails = response_data[:payload][:contacts].pluck(:email)
expect(contact_emails).to include('recent@test.com')
expect(contact_emails).not_to include('old@test.com')
end
it 'filters contacts by until parameter' do
create(:contact, email: 'old@test.com', account: account, last_activity_at: 10.days.ago)
create(:contact, email: 'recent@test.com', account: account, last_activity_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/contacts",
headers: agent.create_new_auth_token,
params: { q: 'test', until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
contact_emails = response_data[:payload][:contacts].pluck(:email)
expect(contact_emails).to include('old@test.com')
expect(contact_emails).not_to include('recent@test.com')
end
it 'filters contacts by both since and until parameters' do
create(:contact, email: 'veryold@test.com', account: account, last_activity_at: 20.days.ago)
create(:contact, email: 'old@test.com', account: account, last_activity_at: 10.days.ago)
create(:contact, email: 'recent@test.com', account: account, last_activity_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/contacts",
headers: agent.create_new_auth_token,
params: { q: 'test', since: 15.days.ago.to_i, until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
contact_emails = response_data[:payload][:contacts].pluck(:email)
expect(contact_emails).to include('old@test.com')
expect(contact_emails).not_to include('veryold@test.com', 'recent@test.com')
end
end
end
end
@@ -94,6 +168,104 @@ RSpec.describe 'Search', type: :request do
expect(response_data[:payload].keys).to contain_exactly(:conversations)
expect(response_data[:payload][:conversations].length).to eq 1
end
context 'with advanced_search feature enabled', :opensearch do
before do
account.enable_features!('advanced_search')
end
it 'filters conversations by since parameter' do
unique_id = SecureRandom.hex(8)
old_contact = create(:contact, email: "old-#{unique_id}@test.com", account: account)
recent_contact = create(:contact, email: "recent-#{unique_id}@test.com", account: account)
old_conversation = create(:conversation, account: account, contact: old_contact)
recent_conversation = create(:conversation, account: account, contact: recent_contact)
create(:message, conversation: old_conversation, account: account, content: 'message 1')
create(:message, conversation: recent_conversation, account: account, content: 'message 2')
create(:inbox_member, user: agent, inbox: old_conversation.inbox)
create(:inbox_member, user: agent, inbox: recent_conversation.inbox)
# Bypass CURRENT_TIMESTAMP default
# rubocop:disable Rails/SkipsModelValidations
Conversation.where(id: old_conversation.id).update_all(last_activity_at: 10.days.ago)
Conversation.where(id: recent_conversation.id).update_all(last_activity_at: 2.days.ago)
# rubocop:enable Rails/SkipsModelValidations
get "/api/v1/accounts/#{account.id}/search/conversations",
headers: agent.create_new_auth_token,
params: { q: unique_id, since: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
conversation_display_ids = response_data[:payload][:conversations].pluck(:id)
expect(conversation_display_ids).to eq([recent_conversation.display_id])
end
it 'filters conversations by until parameter' do
unique_id = SecureRandom.hex(8)
old_contact = create(:contact, email: "old-#{unique_id}@test.com", account: account)
recent_contact = create(:contact, email: "recent-#{unique_id}@test.com", account: account)
old_conversation = create(:conversation, account: account, contact: old_contact)
recent_conversation = create(:conversation, account: account, contact: recent_contact)
create(:message, conversation: old_conversation, account: account, content: 'message 1')
create(:message, conversation: recent_conversation, account: account, content: 'message 2')
create(:inbox_member, user: agent, inbox: old_conversation.inbox)
create(:inbox_member, user: agent, inbox: recent_conversation.inbox)
# Bypass CURRENT_TIMESTAMP default
# rubocop:disable Rails/SkipsModelValidations
Conversation.where(id: old_conversation.id).update_all(last_activity_at: 10.days.ago)
Conversation.where(id: recent_conversation.id).update_all(last_activity_at: 2.days.ago)
# rubocop:enable Rails/SkipsModelValidations
get "/api/v1/accounts/#{account.id}/search/conversations",
headers: agent.create_new_auth_token,
params: { q: unique_id, until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
conversation_display_ids = response_data[:payload][:conversations].pluck(:id)
expect(conversation_display_ids).to eq([old_conversation.display_id])
end
it 'filters conversations by both since and until parameters' do
unique_id = SecureRandom.hex(8)
very_old_contact = create(:contact, email: "veryold-#{unique_id}@test.com", account: account)
old_contact = create(:contact, email: "old-#{unique_id}@test.com", account: account)
recent_contact = create(:contact, email: "recent-#{unique_id}@test.com", account: account)
very_old_conversation = create(:conversation, account: account, contact: very_old_contact)
old_conversation = create(:conversation, account: account, contact: old_contact)
recent_conversation = create(:conversation, account: account, contact: recent_contact)
create(:message, conversation: very_old_conversation, account: account, content: 'message 1')
create(:message, conversation: old_conversation, account: account, content: 'message 2')
create(:message, conversation: recent_conversation, account: account, content: 'message 3')
create(:inbox_member, user: agent, inbox: very_old_conversation.inbox)
create(:inbox_member, user: agent, inbox: old_conversation.inbox)
create(:inbox_member, user: agent, inbox: recent_conversation.inbox)
# Bypass CURRENT_TIMESTAMP default
# rubocop:disable Rails/SkipsModelValidations
Conversation.where(id: very_old_conversation.id).update_all(last_activity_at: 20.days.ago)
Conversation.where(id: old_conversation.id).update_all(last_activity_at: 10.days.ago)
Conversation.where(id: recent_conversation.id).update_all(last_activity_at: 2.days.ago)
# rubocop:enable Rails/SkipsModelValidations
get "/api/v1/accounts/#{account.id}/search/conversations",
headers: agent.create_new_auth_token,
params: { q: unique_id, since: 15.days.ago.to_i, until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
conversation_display_ids = response_data[:payload][:conversations].pluck(:id)
expect(conversation_display_ids).to eq([old_conversation.display_id])
end
end
end
end
@@ -175,6 +347,74 @@ RSpec.describe 'Search', type: :request do
expect(response_data[:payload][:articles].length).to eq 15 # Default per_page is 15
end
context 'with advanced_search feature enabled', :opensearch do
before do
account.enable_features!('advanced_search')
end
it 'filters articles by since parameter' do
portal = create(:portal, account: account)
old_article = create(:article, title: 'Old Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 10.days.ago)
recent_article = create(:article, title: 'Recent Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/articles",
headers: agent.create_new_auth_token,
params: { q: 'test', since: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
article_ids = response_data[:payload][:articles].pluck(:id)
expect(article_ids).to include(recent_article.id)
expect(article_ids).not_to include(old_article.id)
end
it 'filters articles by until parameter' do
portal = create(:portal, account: account)
old_article = create(:article, title: 'Old Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 10.days.ago)
recent_article = create(:article, title: 'Recent Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/articles",
headers: agent.create_new_auth_token,
params: { q: 'test', until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
article_ids = response_data[:payload][:articles].pluck(:id)
expect(article_ids).to include(old_article.id)
expect(article_ids).not_to include(recent_article.id)
end
it 'filters articles by both since and until parameters' do
portal = create(:portal, account: account)
very_old_article = create(:article, title: 'Very Old Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 20.days.ago)
old_article = create(:article, title: 'Old Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 10.days.ago)
recent_article = create(:article, title: 'Recent Article test', account: account, portal: portal,
author: agent, status: 'published', updated_at: 2.days.ago)
get "/api/v1/accounts/#{account.id}/search/articles",
headers: agent.create_new_auth_token,
params: { q: 'test', since: 15.days.ago.to_i, until: 5.days.ago.to_i },
as: :json
expect(response).to have_http_status(:success)
response_data = JSON.parse(response.body, symbolize_names: true)
article_ids = response_data[:payload][:articles].pluck(:id)
expect(article_ids).to include(old_article.id)
expect(article_ids).not_to include(very_old_article.id, recent_article.id)
end
end
end
end
end

2
spec/lib/webhooks/trigger_spec.rb
View File

@@ -18,7 +18,9 @@ describe Webhooks::Trigger do
before do
ActiveJob::Base.queue_adapter = :test
allow(GlobalConfig).to receive(:get_value).and_call_original
allow(GlobalConfig).to receive(:get_value).with('WEBHOOK_TIMEOUT').and_return(webhook_timeout)
allow(GlobalConfig).to receive(:get_value).with('DEPLOYMENT_ENV').and_return(nil)
end
after do

493
spec/services/search_service_spec.rb
View File

@@ -95,6 +95,7 @@ describe SearchService do
let(:search_type) { 'Message' }
it 'uses LIKE search when search_with_gin feature is disabled' do
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
@@ -105,6 +106,7 @@ describe SearchService do
end
it 'uses GIN search when search_with_gin feature is enabled' do
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(true)
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
@@ -119,11 +121,13 @@ describe SearchService do
message3 = create(:message, account: account, inbox: inbox, content: 'Harry is a wizard apprentice')
# Test with GIN search
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(true)
gin_search = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
gin_results = gin_search.perform[:messages].map(&:id)
# Test with LIKE search
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
like_search = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
like_results = like_search.perform[:messages].map(&:id)
@@ -133,6 +137,108 @@ describe SearchService do
expect(gin_results).to include(message.id, message2.id, message3.id)
end
end
# rubocop:disable RSpec/MultipleMemoizedHelpers
context 'when filtering messages with time, sender, and inbox', :opensearch do
let!(:agent) { create(:user, account: account) }
let!(:inbox2) { create(:inbox, account: account) }
let!(:old_message) do
create(:message, account: account, inbox: inbox, content: 'old wizard message', sender: harry, created_at: 80.days.ago)
end
let!(:recent_message) do
create(:message, account: account, inbox: inbox, content: 'recent wizard message', sender: harry, created_at: 1.day.ago)
end
let!(:agent_message) do
create(:message, account: account, inbox: inbox, content: 'wizard from agent', sender: agent, created_at: 1.day.ago)
end
let!(:inbox2_message) do
create(:message, account: account, inbox: inbox2, content: 'wizard in inbox2', sender: harry, created_at: 1.day.ago)
end
before do
account.enable_features!('advanced_search')
create(:inbox_member, inbox: inbox2, user: user)
end
it 'filters messages by time range with LIKE search' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', since: 50.days.ago.to_i, search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(recent_message.id, agent_message.id, inbox2_message.id)
expect(results.map(&:id)).not_to include(old_message.id)
end
it 'filters messages by time range with GIN search' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(true)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', since: 50.days.ago.to_i, search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(recent_message.id, agent_message.id, inbox2_message.id)
expect(results.map(&:id)).not_to include(old_message.id)
end
it 'filters messages by sender (contact)' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', from: "contact:#{harry.id}", search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(recent_message.id, old_message.id, inbox2_message.id)
expect(results.map(&:id)).not_to include(agent_message.id)
end
it 'filters messages by sender (agent)' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', from: "agent:#{agent.id}", search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(agent_message.id)
expect(results.map(&:id)).not_to include(recent_message.id, old_message.id, inbox2_message.id)
end
it 'filters messages by inbox' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', inbox_id: inbox2.id, search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(inbox2_message.id)
expect(results.map(&:id)).not_to include(recent_message.id, old_message.id, agent_message.id)
end
it 'combines multiple filters' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
params = { q: 'wizard', since: 50.days.ago.to_i, inbox_id: inbox.id, from: "contact:#{harry.id}", search_type: 'Message' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Message')
results = search.perform[:messages]
expect(results.map(&:id)).to include(recent_message.id)
expect(results.map(&:id)).not_to include(old_message.id, agent_message.id, inbox2_message.id)
end
end
# rubocop:enable RSpec/MultipleMemoizedHelpers
end
context 'when conversation search' do
@@ -183,6 +289,91 @@ describe SearchService do
expect(results.length).to eq(15) # Default per_page is 15
end
end
context 'when filtering contacts with time caps', :opensearch do
let!(:old_contact) { create(:contact, name: 'Old Potter', email: 'old@test.com', account: account, last_activity_at: 100.days.ago) }
let!(:recent_contact) { create(:contact, name: 'Recent Potter', email: 'recent@test.com', account: account, last_activity_at: 1.day.ago) }
before do
account.enable_features!('advanced_search')
end
it 'caps since to 90 days ago and excludes older contacts' do
params = { q: 'Potter', since: 100.days.ago.to_i, search_type: 'Contact' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Contact')
results = search.perform[:contacts]
expect(results.map(&:id)).not_to include(old_contact.id)
expect(results.map(&:id)).to include(recent_contact.id)
end
it 'caps until to 90 days from now' do
params = { q: 'Potter', until: 100.days.from_now.to_i, search_type: 'Contact' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Contact')
results = search.perform[:contacts]
# Both contacts should be included since their last_activity_at is before the capped time
expect(results.map(&:id)).to include(recent_contact.id)
end
end
context 'when filtering conversations with time caps', :opensearch do
let!(:old_conversation) { create(:conversation, contact: harry, inbox: inbox, account: account, last_activity_at: 100.days.ago) }
let!(:recent_conversation) { create(:conversation, contact: harry, inbox: inbox, account: account, last_activity_at: 1.day.ago) }
before do
account.enable_features!('advanced_search')
end
it 'caps since to 90 days ago and excludes older conversations' do
params = { q: 'Harry', since: 100.days.ago.to_i, search_type: 'Conversation' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Conversation')
results = search.perform[:conversations]
expect(results.map(&:id)).not_to include(old_conversation.id)
expect(results.map(&:id)).to include(recent_conversation.id)
end
it 'caps until to 90 days from now' do
params = { q: 'Harry', until: 100.days.from_now.to_i, search_type: 'Conversation' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Conversation')
results = search.perform[:conversations]
# Both conversations should be included since their last_activity_at is before the capped time
expect(results.map(&:id)).to include(recent_conversation.id)
end
end
context 'when filtering articles with time caps', :opensearch do
let!(:old_article) do
create(:article, title: 'Old Magic Guide', account: account, portal: portal, author: user, status: 'published', updated_at: 100.days.ago)
end
let!(:recent_article) do
create(:article, title: 'Recent Magic Guide', account: account, portal: portal, author: user, status: 'published', updated_at: 1.day.ago)
end
before do
account.enable_features!('advanced_search')
end
it 'caps since to 90 days ago and excludes older articles' do
params = { q: 'Magic', since: 100.days.ago.to_i, search_type: 'Article' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Article')
results = search.perform[:articles]
expect(results.map(&:id)).not_to include(old_article.id)
expect(results.map(&:id)).to include(recent_article.id)
end
it 'caps until to 90 days from now' do
params = { q: 'Magic', until: 100.days.from_now.to_i, search_type: 'Article' }
search = described_class.new(current_user: user, current_account: account, params: params, search_type: 'Article')
results = search.perform[:articles]
# Both articles should be included since their updated_at is before the capped time
expect(results.map(&:id)).to include(recent_article.id)
end
end
end
describe '#message_base_query' do
@@ -260,4 +451,306 @@ describe SearchService do
expect(search.send(:use_gin_search)).to be false
end
end
describe '#advanced_search with filters', if: Message.respond_to?(:search) do
let(:params) { { q: 'test' } }
let(:search_type) { 'Message' }
before do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(true)
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
allow(Message).to receive(:search).and_return([])
end
context 'when advanced_search feature flag is disabled' do
it 'ignores filters and falls back to standard search' do
allow(ChatwootApp).to receive(:advanced_search_allowed?).and_return(false)
contact = create(:contact, account: account)
inbox2 = create(:inbox, account: account)
params = { q: 'test', from: "contact:#{contact.id}", inbox_id: inbox2.id, since: 3.days.ago.to_i }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(search_service).not_to receive(:advanced_search)
search_service.perform
end
end
context 'when filtering by from parameter' do
let(:contact) { create(:contact, account: account) }
let(:agent) { create(:user, account: account) }
it 'filters messages from specific contact' do
params = { q: 'test', from: "contact:#{contact.id}" }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
sender_type: 'Contact',
sender_id: contact.id
)
)
).and_return([])
search_service.perform
end
it 'filters messages from specific agent' do
params = { q: 'test', from: "agent:#{agent.id}" }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
sender_type: 'User',
sender_id: agent.id
)
)
).and_return([])
search_service.perform
end
it 'ignores invalid from parameter format' do
params = { q: 'test', from: 'invalid:format' }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_not_including(:sender_type, :sender_id)
)
).and_return([])
search_service.perform
end
end
context 'when filtering by time range' do
it 'defaults to 90 days ago when no since parameter is provided' do
params = { q: 'test' }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(gte: be_within(1.second).of(Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS.days.ago))
)
)
).and_return([])
search_service.perform
end
it 'silently caps since timestamp to 90 day limit when exceeded' do
since_timestamp = (Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS * 2).days.ago.to_i
params = { q: 'test', since: since_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(gte: be_within(1.second).of(Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS.days.ago))
)
)
).and_return([])
search_service.perform
end
it 'filters messages since timestamp when within 90 day limit' do
since_timestamp = 3.days.ago.to_i
params = { q: 'test', since: since_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(gte: Time.zone.at(since_timestamp))
)
)
).and_return([])
search_service.perform
end
it 'filters messages until timestamp' do
until_timestamp = 5.days.ago.to_i
params = { q: 'test', until: until_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(lte: Time.zone.at(until_timestamp))
)
)
).and_return([])
search_service.perform
end
it 'filters messages within time range' do
since_timestamp = 5.days.ago.to_i
until_timestamp = 12.hours.ago.to_i
params = { q: 'test', since: since_timestamp, until: until_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(
gte: Time.zone.at(since_timestamp),
lte: Time.zone.at(until_timestamp)
)
)
)
).and_return([])
search_service.perform
end
it 'silently caps until timestamp to 90 days from now when exceeded' do
until_timestamp = 100.days.from_now.to_i
params = { q: 'test', until: until_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
created_at: hash_including(lte: be_within(1.second).of(90.days.from_now))
)
)
).and_return([])
search_service.perform
end
end
context 'when filtering by inbox_id' do
let!(:inbox2) { create(:inbox, account: account) }
before do
create(:inbox_member, user: user, inbox: inbox2)
end
it 'filters messages from specific inbox' do
params = { q: 'test', inbox_id: inbox2.id }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(inbox_id: inbox2.id)
)
).and_return([])
search_service.perform
end
it 'ignores inbox filter when user lacks access' do
restricted_inbox = create(:inbox, account: account)
params = { q: 'test', inbox_id: restricted_inbox.id }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_not_including(inbox_id: restricted_inbox.id)
)
).and_return([])
search_service.perform
end
end
context 'when combining multiple filters' do
it 'applies all filters together' do
test_contact = create(:contact, account: account)
test_inbox = create(:inbox, account: account)
create(:inbox_member, user: user, inbox: test_inbox)
since_timestamp = 3.days.ago.to_i
params = { q: 'test', from: "contact:#{test_contact.id}", inbox_id: test_inbox.id, since: since_timestamp }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
expect(Message).to receive(:search).with(
'test',
hash_including(
where: hash_including(
sender_type: 'Contact',
sender_id: test_contact.id,
inbox_id: test_inbox.id,
created_at: hash_including(gte: Time.zone.at(since_timestamp))
)
)
).and_return([])
search_service.perform
end
end
end
describe '#advanced_search_with_fallback' do
let(:params) { { q: 'test' } }
let(:search_type) { 'Message' }
context 'when Elasticsearch is unavailable' do
it 'falls back to LIKE search when Elasticsearch connection fails' do
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
params = { q: 'test' }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
allow(search_service).to receive(:advanced_search).and_raise(Faraday::ConnectionFailed.new('Connection refused'))
expect(search_service).to receive(:filter_messages_with_like).and_call_original
expect { search_service.perform }.not_to raise_error
end
it 'falls back to GIN search when Elasticsearch is unavailable and GIN is enabled' do
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(true)
params = { q: 'test' }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
allow(search_service).to receive(:advanced_search).and_raise(Searchkick::Error.new('Elasticsearch unavailable'))
expect(search_service).to receive(:filter_messages_with_gin).and_call_original
expect { search_service.perform }.not_to raise_error
end
it 'applies filters correctly in SQL fallback when Elasticsearch fails' do
allow(account).to receive(:feature_enabled?).and_call_original
allow(account).to receive(:feature_enabled?).with('advanced_search').and_return(true)
allow(account).to receive(:feature_enabled?).with('search_with_gin').and_return(false)
test_contact = create(:contact, account: account)
create(:message, account: account, inbox: inbox, content: 'test message', sender: test_contact, created_at: 1.day.ago)
params = { q: 'test', from: "contact:#{test_contact.id}", since: 2.days.ago.to_i }
search_service = described_class.new(current_user: user, current_account: account, params: params, search_type: search_type)
allow(search_service).to receive(:advanced_search).and_raise(Faraday::ConnectionFailed.new('Connection refused'))
results = search_service.perform[:messages]
expect(results).not_to be_empty
expect(results.first.sender_id).to eq(test_contact.id)
end
end
end
end

40
spec/support/opensearch_check.rb Normal file
View File

@@ -0,0 +1,40 @@
# frozen_string_literal: true
require 'net/http'
require 'json'
RSpec.configure do |config|
# Run OpenSearch connectivity check only for specs tagged with :opensearch
config.before(:context, :opensearch) do
opensearch_url = ENV.fetch('OPENSEARCH_URL', nil)
next if opensearch_url.blank?
puts "\n==== OpenSearch Connectivity Check ===="
puts "OPENSEARCH_URL: #{opensearch_url}"
begin
uri = URI.parse("#{opensearch_url}/_cluster/health")
response = Net::HTTP.get_response(uri)
raise "OpenSearch is not reachable at #{opensearch_url}. HTTP Status: #{response.code}" unless response.is_a?(Net::HTTPSuccess)
health = JSON.parse(response.body)
status = health['status']
puts "Cluster status: #{status}"
puts "Cluster name: #{health['cluster_name']}"
puts "Number of nodes: #{health['number_of_nodes']}"
raise "OpenSearch cluster is not healthy. Status: #{status}" unless %w[green yellow].include?(status)
puts '✓ OpenSearch is healthy and ready for tests'
puts "========================================\n\n"
rescue StandardError => e
puts "\n❌ ERROR: Failed to connect to OpenSearch"
puts "Message: #{e.message}"
puts "========================================\n\n"
raise "OpenSearch connectivity check failed: #{e.message}"
end
end
end