Commit Graph

19 Commits

Author SHA1 Message Date
Vinay Keerthi
59cbf57e20 feat: Advanced Search Backend (#12917)
## Description

Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).

This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
   - **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues

The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.

## API Documentation

### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```

### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```

---

## 1. Search All Resources

**Endpoint:** `GET /api/v1/accounts/{account_id}/search`

Returns results from all searchable resources (contacts, conversations,
messages, articles).

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|

### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "contacts": [...],
    "conversations": [...],
    "messages": [...],
    "articles": [...]
  }
}
```

---

## 2. Search Contacts

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`

Search contacts by name, email, phone number, or identifier with
optional time filtering.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |

### Example Requests

**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "contacts": [
      {
        "id": 42,
        "email": "john@example.com",
        "name": "John Doe",
        "phone_number": "+1234567890",
        "identifier": "user_123",
        "additional_attributes": {},
        "created_at": 1701234567
      }
    ]
  }
}
```

---

## 3. Search Conversations

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`

Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |

### Example Requests

**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "conversations": [
      {
        "id": 123,
        "display_id": 45,
        "inbox_id": 1,
        "status": "open",
        "messages": [...],
        "meta": {...}
      }
    ]
  }
}
```

---

## 4. Search Messages (Advanced)

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`

Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.

### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |

### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp

### Example Requests

**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response (Success)
```json
{
  "payload": {
    "messages": [
      {
        "id": 789,
        "content": "I need a refund for my purchase",
        "message_type": "incoming",
        "created_at": 1701234567,
        "conversation_id": 123,
        "inbox_id": 3,
        "sender": {
          "id": 42,
          "type": "contact"
        }
      }
    ]
  }
}
```

### Example Response (90-day limit exceeded)
```json
{
  "error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`

---

## 5. Search Articles

**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`

Search help center articles by title or content.

### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |

### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
  -H "api_access_token: YOUR_ACCESS_TOKEN"
```

### Example Response
```json
{
  "payload": {
    "articles": [
      {
        "id": 456,
        "title": "Installation Guide",
        "slug": "installation-guide",
        "portal_slug": "help",
        "account_id": 1,
        "category_name": "Getting Started",
        "status": "published",
        "updated_at": 1701234567
      }
    ]
  }
}
```

---

## Technical Implementation

### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)

### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)

---

## Type of change

- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)

---

## How Has This Been Tested?

### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
  - Individual filters (`from`, `inbox_id`, time range)
  - Combined filters
  - Permission validation for inbox access
  - Feature flag checks
  - 90-day limit enforcement
  - Error handling for exceeded time limits

### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb

# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```

### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:

```bash
# 1. Create test data
bundle exec rake search:setup_test_data

# 2. Start OpenSearch
mise elasticsearch-start

# 3. Reindex messages
rails runner "Message.search_index.import Message.all"

# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"

# 5. Test via API or Rails console
```

---

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---

## Additional Notes

### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
  - Set `OPENSEARCH_URL` environment variable
  - Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
  - `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search

### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages

### Breaking Changes
- None. All new parameters are optional and backward compatible.

### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`

### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)

---------

Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2026-01-07 15:30:49 +05:30
Sivin Varghese
f726dc2419 chore: Adds URL-based search and tab selection (#12663)
# Pull Request Template

## Description

This PR enables URL-based search and tab selection, allowing search
queries and active tabs to persist in the URL for easy sharing.

Fixes
[CW-5766](https://linear.app/chatwoot/issue/CW-5766/cannot-impersonate-an-account),
https://github.com/chatwoot/chatwoot/issues/12623

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

### Loom video

https://www.loom.com/share/422a1d61f3fe4278a88e352ef98d2b78?sid=35fabee7-652f-4e17-83bd-e066a3bb804c

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
2025-10-27 11:17:04 +05:30
Shivam Mishra
a0cc27faaf feat: show articles in the list only if feature is enabled (#11607)
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-05-29 14:19:35 +05:30
Shivam Mishra
b1120ae7fb feat: allow searching articles in omnisearch (#11558)
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-05-28 13:50:50 +05:30
Sivin Varghese
bd94e5062d chore: Search improvements (#10801)
- Adds pagination support for search.
- Use composition API on all search related component.
- Minor UI improvements.
- Adds missing specs

Loom video
https://www.loom.com/share/5b01afa5c9204e7d97ff81b215621dde?sid=82ca6d22-ca8c-4d5e-8740-ba06ca4051ba
2025-02-03 19:34:50 +05:30
Shivam Mishra
7fd8b4d03a feat: update colors for v4 (#10660)
Porting changes from https://github.com/chatwoot/chatwoot/pull/10552

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Pranav <pranavrajs@gmail.com>
2025-01-15 17:13:03 +05:30
Shivam Mishra
c51a458c25 style: apply fixes for eslint issues [cw-3590] (#10210)
These fixes are all auto generated and can be merged directly

Fixes the following issues

1. Event used on components should be hypenated
2. Attribute orders in components
3. Use `unmounted` instead of `destroyed`
4. Add explicit `emits` declarations for components, autofixed [using
this
script](https://gist.github.com/scmmishra/6f549109b96400006bb69bbde392eddf)


We ignore the top level v-if for now, we will fix it later
2024-10-03 15:02:12 +05:30
Shivam Mishra
42f6621afb feat: Vite + vue 3 💚 (#10047)
Fixes https://github.com/chatwoot/chatwoot/issues/8436
Fixes https://github.com/chatwoot/chatwoot/issues/9767
Fixes https://github.com/chatwoot/chatwoot/issues/10156
Fixes https://github.com/chatwoot/chatwoot/issues/6031
Fixes https://github.com/chatwoot/chatwoot/issues/5696
Fixes https://github.com/chatwoot/chatwoot/issues/9250
Fixes https://github.com/chatwoot/chatwoot/issues/9762

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2024-10-02 00:36:30 -07:00
Sojan Jose
58e78621ba chore: Custom Roles to manage permissions [ UI ] (#9865)
In admin settings, this Pr will add the UI for managing custom roles (
ref: https://github.com/chatwoot/chatwoot/pull/9995 ). It also handles
the routing logic changes to accommodate fine-tuned permissions.

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2024-09-17 11:40:11 -07:00
Sivin Varghese
b4b308336f feat: Eslint rules (#9839)
# Pull Request Template

## Description

This PR adds new eslint rules to the code base.

**Error rules**

|    Rule name     | Type | Files updated |
| ----------------- | --- | - |
| `vue/block-order`  | error  |    |
| `vue/component-name-in-template-casing`  | error  |    |
| `vue/component-options-name-casing`  | error  |    |
| `vue/custom-event-name-casing`  | error  |    |
| `vue/define-emits-declaration`  | error  |    |
| `vue/no-unused-properties`  | error  |    |
| `vue/define-macros-order`  | error  |    |
| `vue/define-props-declaration`  | error  |    |
| `vue/match-component-import-name`  | error  |    |
| `vue/next-tick-style`  | error  |    |
| `vue/no-bare-strings-in-template`  | error  |    |
| `vue/no-empty-component-block`  | error  |    |
| `vue/no-multiple-objects-in-class`  | error  |    |
| `vue/no-required-prop-with-default`  | error  |    |
| `vue/no-static-inline-styles`  | error  |    |
| `vue/no-template-target-blank`  | error  |    |
| `vue/no-this-in-before-route-enter`  | error  |    |
| `vue/no-undef-components`  | error  |    |
| `vue/no-unused-emit-declarations`  | error  |    |
| `vue/no-unused-refs`  | error  |    |
| `vue/no-use-v-else-with-v-for`  | error  |    |
| `vue/no-useless-v-bind`  | error  |    |
| `vue/no-v-text`  | error  |    |
| `vue/padding-line-between-blocks`  | error  |    |
| ~`vue/prefer-prop-type-boolean-first`~ | ~error~ |  (removed this
rule, cause a bug in displaying custom attributes) |
| `vue/prefer-separate-static-class`  | error  |    |
| `vue/prefer-true-attribute-shorthand`  | error  |    |
| `vue/require-explicit-slots`  | error  |    |
| `vue/require-macro-variable-name`  | error  |    |


**Warn rules**

|    Rule name     | Type | Files updated |
| ---- | ------------- | ------------- |
| `vue/no-root-v-if`  | warn  |    |


Fixes https://linear.app/chatwoot/issue/CW-3492/vue-eslint-rules

## Type of change

- [x] New feature (non-breaking change which adds functionality)


## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Fayaz Ahmed <fayazara@gmail.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
2024-08-05 14:02:16 +05:30
Sivin Varghese
12af2fe026 feat: Remove the usage of clickAway mixin with directive (#9323) 2024-04-30 10:35:14 +05:30
Sivin Varghese
40ec0d109a feat: Dark Mode (#7471) 2023-07-21 22:10:25 +05:30
Pranav Raj S
985b3f3bfc chore: Reset the base font-size to 16px instead of 10px (#7455)
Co-authored-by: Nithin David Thomas <1277421+nithindavid@users.noreply.github.com>
2023-07-04 03:23:00 +05:30
Nithin David Thomas
c0e905b5d8 fix: Search improvements and bug fixes [CW-1604] (#6985)
* fix: Search improvements and bug fixes

* Resets tab to all on search

* Fix index bug with tabs

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2023-04-27 13:20:29 +05:30
Nithin David Thomas
402428fb4d feat: Splits search api by resources to improve query time [cw-47] (#6942)
* feat: Splits search api by resources to improve query time

* Review fixes

* Spacing fixes

* Update app/javascript/dashboard/modules/search/components/SearchView.vue

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>

* Review fixes

* Refactor searchview

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2023-04-25 17:59:38 +05:30
Nithin David Thomas
f3aba84063 fix: Fixes scroll behavior for search page (#6717) 2023-03-21 12:10:08 +05:30
Nithin David Thomas
d2aa5f4c69 fix: Reverts popover styling for search page (#6688) 2023-03-20 11:04:56 +05:30
Nithin David Thomas
cae3ac94cd feat: Search improvements for conversations (#6645)
* feat: Shows search as a popover

* Reverts search from popover to page

* Fixes review comments on usability

* Fixes keyboard navigation issues
2023-03-14 13:09:43 +05:30
Nithin David Thomas
88ed028a06 feat: Revamps search to use new search API's (#6582)
* feat: Revamps search to use new search API's

* Fixes search result spacing

* Fixes message result

* Fixes issue with empty search results

* Remove console errors

* Remove console errors

* Fix console errors, canned responses

* Fixes message rendering on results

* Highlights search term

* Fixes html rendering for emails

* FIxes email rendering issues

* Removes extra spaces and line breaks

---------

Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
2023-03-03 20:58:21 +05:30