## Summary
This Enterprise-only feature automatically fetches a favicon for
companies created with a domain, and adds a batch task to backfill
missing avatars for existing companies. The flow only targets companies
that do not already have an attached avatar, so existing avatars are
left untouched.
## Demo
https://github.com/user-attachments/assets/d050334e-769f-4e46-b6e7-f7423727a192
## What changed
- Added `Avatar::AvatarFromFaviconJob` to build a Google favicon URL
from the company domain and fetch it through `Avatar::AvatarFromUrlJob`
- Triggered favicon fetching from `Company` with `after_create_commit`
- Added `Companies::FetchAvatarsJob` to batch existing companies that
are missing avatars
- Added `companies:fetch_missing_avatars` under `enterprise/lib/tasks`
- Kept the company-specific implementation inside the Enterprise
boundary
- Stubbed the new favicon request in unrelated specs that now hit this
callback indirectly
- Updated a couple of CI-sensitive specs that were failing due to
callback side effects / reload-safe exception assertions
## How to verify
1. Create a company in Enterprise with a valid domain and no avatar.
2. Confirm that a favicon-based avatar gets attached shortly after
creation.
3. Create another company with a domain and an avatar already attached.
4. Confirm that the existing avatar is not replaced.
5. Run `companies:fetch_missing_avatars`.
6. Confirm that existing companies without avatars get one, while
companies that already have avatars remain unchanged.
## Notes
- This change does not refresh or overwrite existing company avatars
- Favicon fetching only runs for companies with a present domain
- The branch includes the latest `develop`
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
## Description
This PR includes cron job to delete the orphans
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces a scheduled cleanup for conversations missing `contact` or
`inbox`.
>
> - Adds `Internal::RemoveOrphanConversationsService` to batch-delete
orphan conversations (scoped by optional `account`, within a
configurable `days` window) with progress logging
> - New `Internal::RemoveOrphanConversationsJob` that invokes the
service; scheduled via `config/schedule.yml` to run every 12 hours on
`housekeeping` queue
> - Refactors rake task `chatwoot:ops:cleanup_orphan_conversations` to
use the service and report `total_deleted` after confirmation
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
59a24715cc59f048d08db3f588cde6fa036f3166. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
## Description
Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).
This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
- **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues
The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.
## API Documentation
### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```
### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```
---
## 1. Search All Resources
**Endpoint:** `GET /api/v1/accounts/{account_id}/search`
Returns results from all searchable resources (contacts, conversations,
messages, articles).
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [...],
"conversations": [...],
"messages": [...],
"articles": [...]
}
}
```
---
## 2. Search Contacts
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`
Search contacts by name, email, phone number, or identifier with
optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [
{
"id": 42,
"email": "john@example.com",
"name": "John Doe",
"phone_number": "+1234567890",
"identifier": "user_123",
"additional_attributes": {},
"created_at": 1701234567
}
]
}
}
```
---
## 3. Search Conversations
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`
Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"conversations": [
{
"id": 123,
"display_id": 45,
"inbox_id": 1,
"status": "open",
"messages": [...],
"meta": {...}
}
]
}
}
```
---
## 4. Search Messages (Advanced)
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`
Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.
### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |
### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp
### Example Requests
**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response (Success)
```json
{
"payload": {
"messages": [
{
"id": 789,
"content": "I need a refund for my purchase",
"message_type": "incoming",
"created_at": 1701234567,
"conversation_id": 123,
"inbox_id": 3,
"sender": {
"id": 42,
"type": "contact"
}
}
]
}
}
```
### Example Response (90-day limit exceeded)
```json
{
"error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`
---
## 5. Search Articles
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`
Search help center articles by title or content.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"articles": [
{
"id": 456,
"title": "Installation Guide",
"slug": "installation-guide",
"portal_slug": "help",
"account_id": 1,
"category_name": "Getting Started",
"status": "published",
"updated_at": 1701234567
}
]
}
}
```
---
## Technical Implementation
### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)
### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)
---
## Type of change
- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)
---
## How Has This Been Tested?
### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
- Individual filters (`from`, `inbox_id`, time range)
- Combined filters
- Permission validation for inbox access
- Feature flag checks
- 90-day limit enforcement
- Error handling for exceeded time limits
### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb
# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```
### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:
```bash
# 1. Create test data
bundle exec rake search:setup_test_data
# 2. Start OpenSearch
mise elasticsearch-start
# 3. Reindex messages
rails runner "Message.search_index.import Message.all"
# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"
# 5. Test via API or Rails console
```
---
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---
## Additional Notes
### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
- Set `OPENSEARCH_URL` environment variable
- Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
- `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search
### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages
### Breaking Changes
- None. All new parameters are optional and backward compatible.
### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`
### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
This PR fixes the HTTP 500 timeout errors occurring when deleting SLA
policies that have large volumes of historical data.
The fix moves the deletion workflow to asynchronous background
processing using the existing `DeleteObjectJob`.
By offloading heavy cascaded deletions (applied SLAs, SLA events,
conversation nullifications) from the request cycle, the API can now
return immediately while the cleanup continues in the background
avoiding the `Rack::Timeout::RequestTimeoutException`. This ensures that
SLA policies can be deleted reliably, regardless of data size.
### Problem
Deleting an SLA policy via `DELETE
/api/v1/accounts/{account_id}/sla_policies/{id}` fails consistently with
`Rack::Timeout::RequestTimeoutException (15s)` for policies with large
amounts of related data.
Because the current implementation performs all dependent deletions
**synchronously**, Rails processes:
- `has_many :applied_slas, dependent: :destroy` (thousands)
- Each `AppliedSla#destroy` → triggers destruction of many `SlaEvent`
records
- `has_many :conversations, dependent: :nullify` (thousands)
This processing far exceeds the Rack timeout window and consistently
triggers HTTP 500 errors for users.
### Solution
This PR applies the same pattern used successfully in Inbox deletion.
**Move deletion to async background jobs**
- Uses `DeleteObjectJob` for centralized, reliable cleanup.
- Allows the DELETE API call to respond immediately.
**Chunk large datasets**
- Records are processed in **batches of 5,000** to reduce DB load and
avoid job timeouts.
# Pull Request Template
## Description
This PR includes,
1. **Channel-specific formatting and menu options** for the rich reply
editor.
2. **Removal of the plain reply editor** and full **standardization** on
the rich reply editor across all channels.
3. **Fix for multiple canned responses insertion:**
* **Before:** The plain editor only allowed inserting canned responses
at the beginning of a message, making it impossible to combine multiple
canned responses in a single reply. This caused inconsistent behavior
across the app.
* **Solution:** Replaced the plain reply editor with the rich
(ProseMirror) editor to ensure a unified experience. Agents can now
insert multiple canned responses at any cursor position.
4. **Floating editor menu** for the reply box to improve accessibility
and overall user experience.
5. **New Strikethrough formatting option** added to the editor menu.
---
**Editor repo PR**:
https://github.com/chatwoot/prosemirror-schema/pull/36
Fixes https://github.com/chatwoot/chatwoot/issues/12517,
[CW-5924](https://linear.app/chatwoot/issue/CW-5924/standardize-the-editor),
[CW-5679](https://linear.app/chatwoot/issue/CW-5679/allow-inserting-multiple-canned-responses-in-a-single-message)
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
### Screenshot
**Dark**
<img width="850" height="345" alt="image"
src="https://github.com/user-attachments/assets/47748e6c-380f-44a3-9e3b-c27e0c830bd0"
/>
**Light**
<img width="850" height="345" alt="image"
src="https://github.com/user-attachments/assets/6746cf32-bf63-4280-a5bd-bbd42c3cbe84"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com>
## Description
This PR reverts the migration from the `annotate` gem to `annotaterb`
introduced in PR #12845. The annotation errors reported in #11673
persist with both gems, and the old `annotate` gem handles the errors
more gracefully by continuing to process other models instead of
crashing.
**Testing reveals both gems fail with the same underlying issue:**
**Old annotate gem (3.2.0):**
```
Unable to annotate app/models/installation_config.rb: no implicit conversion of Hash into String
Unable to annotate app/models/installation_config.rb: no implicit conversion of nil into Array
Model files unchanged.
```
(Logs error but continues processing)
**New annotaterb gem (4.20.0):**
```
❯ bundle exec annotaterb models
ruby/3.4.4/lib/ruby/gems/3.4.0/gems/reline-0.3.6/lib/reline/terminfo.rb:2: warning: ruby/3.4.4/lib/ruby/3.4.0/fiddle.rb was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0.
You can add fiddle to your Gemfile or gemspec to silence this warning.
Also please contact the author of reline-0.3.6 to request adding fiddle into its gemspec.
Annotating models
bundler: failed to load command: annotaterb (ruby/3.4.4/bin/annotaterb)
ruby/3.4.4/lib/ruby/3.4.0/psych/parser.rb:62:in 'Psych::Parser#_native_parse': no implicit conversion of Hash into String (TypeError)
_native_parse @handler, yaml, path
^^^^^^^^^^^^^^^^^^^^
from ruby/3.4.4/lib/ruby/3.4.0/psych/parser.rb:62:in 'Psych::Parser#parse'
from ruby/3.4.4/lib/ruby/3.4.0/psych.rb:457:in 'Psych.parse_stream'
from ruby/3.4.4/lib/ruby/3.4.0/psych.rb:401:in 'Psych.parse'
from ruby/3.4.4/lib/ruby/3.4.0/psych.rb:325:in 'Psych.safe_load'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activerecord-7.1.5.2/lib/active_record/coders/yaml_column.rb:37:in 'ActiveRecord::Coders::YAMLColumn::SafeCoder#load'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activerecord-7.1.5.2/lib/active_record/coders/column_serializer.rb:37:in 'ActiveRecord::Coders::ColumnSerializer#load'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activerecord-7.1.5.2/lib/active_record/type/serialized.rb:22:in 'ActiveRecord::Type::Serialized#deserialize'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activemodel-7.1.5.2/lib/active_model/attribute.rb:175:in 'ActiveModel::Attribute::FromDatabase#type_cast'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activemodel-7.1.5.2/lib/active_model/attribute.rb:43:in 'ActiveModel::Attribute#value'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activemodel-7.1.5.2/lib/active_model/attribute_set.rb:37:in 'block in ActiveModel::AttributeSet#to_hash'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activesupport-7.1.5.2/lib/active_support/core_ext/enumerable.rb:78:in 'block in Enumerable#index_with'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activesupport-7.1.5.2/lib/active_support/core_ext/enumerable.rb:78:in 'Array#each'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activesupport-7.1.5.2/lib/active_support/core_ext/enumerable.rb:78:in 'Enumerable#index_with'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activemodel-7.1.5.2/lib/active_model/attribute_set.rb:37:in 'ActiveModel::AttributeSet#to_hash'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/activerecord-7.1.5.2/lib/active_record/model_schema.rb:499:in 'ActiveRecord::ModelSchema::ClassMethods#column_defaults'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/model_wrapper.rb:68:in 'AnnotateRb::ModelAnnotator::ModelWrapper#column_defaults'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/model_wrapper.rb:139:in 'block in AnnotateRb::ModelAnnotator::ModelWrapper#built_attributes'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/model_wrapper.rb:136:in 'Array#map'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/model_wrapper.rb:136:in 'AnnotateRb::ModelAnnotator::ModelWrapper#built_attributes'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/column_annotation/annotation_builder.rb:15:in 'AnnotateRb::ModelAnnotator::ColumnAnnotation::AnnotationBuilder#build'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:52:in 'block in AnnotateRb::ModelAnnotator::Annotation::AnnotationBuilder::Annotation#columns'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:51:in 'Array#map'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:51:in 'AnnotateRb::ModelAnnotator::Annotation::AnnotationBuilder::Annotation#columns'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:26:in 'AnnotateRb::ModelAnnotator::Annotation::AnnotationBuilder::Annotation#body'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:35:in 'AnnotateRb::ModelAnnotator::Annotation::AnnotationBuilder::Annotation#build'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotation/annotation_builder.rb:71:in 'AnnotateRb::ModelAnnotator::Annotation::AnnotationBuilder#build'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/project_annotator.rb:43:in 'AnnotateRb::ModelAnnotator::ProjectAnnotator#build_instructions_for_file'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/project_annotator.rb:17:in 'block in AnnotateRb::ModelAnnotator::ProjectAnnotator#annotate'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/project_annotator.rb:13:in 'Array#map'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/project_annotator.rb:13:in 'AnnotateRb::ModelAnnotator::ProjectAnnotator#annotate'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotator.rb:21:in 'AnnotateRb::ModelAnnotator::Annotator#do_annotations'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/model_annotator/annotator.rb:8:in 'AnnotateRb::ModelAnnotator::Annotator.do_annotations'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/commands/annotate_models.rb:17:in 'AnnotateRb::Commands::AnnotateModels#call'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/runner.rb:38:in 'AnnotateRb::Runner#run'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/lib/annotate_rb/runner.rb:11:in 'AnnotateRb::Runner.run'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/annotaterb-4.20.0/exe/annotaterb:18:in '<top (required)>'
from ruby/3.4.4/bin/annotaterb:25:in 'Kernel#load'
from ruby/3.4.4/bin/annotaterb:25:in '<top (required)>'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli/exec.rb:58:in 'Kernel.load'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli/exec.rb:58:in 'Bundler::CLI::Exec#kernel_load'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli/exec.rb:23:in 'Bundler::CLI::Exec#run'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli.rb:455:in 'Bundler::CLI#exec'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/vendor/thor/lib/thor/command.rb:28:in 'Bundler::Thor::Command#run'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in 'Bundler::Thor::Invocation#invoke_command'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/vendor/thor/lib/thor.rb:527:in 'Bundler::Thor.dispatch'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli.rb:35:in 'Bundler::CLI.dispatch'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/vendor/thor/lib/thor/base.rb:584:in 'Bundler::Thor::Base::ClassMethods#start'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/cli.rb:29:in 'Bundler::CLI.start'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/exe/bundle:28:in 'block in <top (required)>'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/lib/bundler/friendly_errors.rb:117:in 'Bundler.with_friendly_errors'
from ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bundler-2.5.16/exe/bundle:20:in '<top (required)>'
from ruby/3.4.4/bin/bundle:25:in 'Kernel#load'
from ruby/3.4.4/bin/bundle:25:in '<main>'
```
(Crashes immediately, stops all processing)
**Root cause:** The `InstallationConfig` model uses YAML serialization
(`serialize :serialized_value, coder: YAML`) on a JSONB database column.
When annotation tools read column defaults, PostgreSQL returns JSONB as
a Hash, but YAML expects a String, causing the type error.
The migration to annotaterb doesn't solve the problem - both gems
encounter the same error. The old gem is preferable as it continues
working despite the error.
Reverts #12845
Related to #11673
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
1. Reverted commit 559d1b657
2. Ran `bundle install` to reinstall annotate gem v3.2.0
3. Ran `RAILS_ENV=development bundle exec annotate`
- Result: Logs errors for InstallationConfig but completes successfully
4. Re-applied the annotaterb changes and tested `bundle exec annotaterb
models`
- Result: Crashes with full stack trace and stops processing
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] New and existing unit tests pass locally with my changes
---
*Edited to truncate environment-specific info from error dump*
## Description
The `annotate` gem has been deprecated and users are experiencing
annotation errors with the new Rails 7 `serialize` syntax. This PR
migrates to `annotaterb`, the actively maintained fork.
Users reported errors when running `make db`:
```
Unable to annotate app/models/installation_config.rb: no implicit conversion of Hash into String
Unable to annotate app/models/installation_config.rb: no implicit conversion of nil into Array
```
This PR updates the Gemfile and rake configuration to use `annotaterb`
instead.
Fixes#11673
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Tested locally with the following steps:
1. Run `bundle install` - successfully installed annotaterb 4.20.0
2. Run `RAILS_ENV=development bundle exec rails db:chatwoot_prepare` -
completed without annotation errors
3. Run `RAILS_ENV=development bundle exec rails annotate_rb:models` -
successfully annotated all models including InstallationConfig
4. Verified InstallationConfig model annotations are present and correct
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] New and existing unit tests pass locally with my changes
## Description
Implements company backfill migration infrastructure for existing
contacts. This is **Part 1 of 2** for the company model production
rollout as described in
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production).
Creates jobs and services to associate existing contacts with companies
based on their email domains, filtering out free email providers (gmail,
yahoo, etc.) and disposable addresses.
**What's included:**
- Business email detector service with ValidEmail2 (uses
`disposable_domain?` to avoid DNS lookups)
- Per-account batch job to process contacts for one account
- Orchestrator job to iterate all accounts
- Rake task: `bundle exec rake companies:backfill`
~~*NOTE*: I'm using a hard-coded approach to determine if something is a
"business" email by filtering out emails that are usually personal. I've
also added domains that are common to some of our customers' regions.
This should be simpler. I looked into `Valid_Email2` and I couldn't find
anything to dictate whether an email is a personal email or a business
one. I don't think the approach used in the frontend is valid here.~~
UPDATE: Using `email_provider_info` gem instead.
**Pending - Part 2 (separate PR):** Real-time company creation for new
contacts
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
```bash
# Run all new tests
bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\
spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\
spec/enterprise/jobs/migration/company_backfill_job_spec.rb
# Run RuboCop
bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\
enterprise/app/jobs/migration/company_account_batch_job.rb \\
enterprise/app/jobs/migration/company_backfill_job.rb \\
lib/tasks/companies.rake
```
**Performance optimization:**
- Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX
lookups (discovered via tcpdump analysis - `disposable?` was making
network calls for every email, causing 100x slowdown)
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
This PR is the fix for MFA changes, to not generate auth tokens without
MFA verification in case MFA is enabled for the account
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Description
In Rails 7.1+, accessing `schema_format` via `ActiveRecord::Base` is no
longer supported, causing a method error.
The correct approach is to use
`ActiveRecord.schema_format`, which aligns
with the public API.
This issue occurs locally because the environment uses newer versions of
Ruby and Rails than others.
Fixes: https://github.com/chatwoot/chatwoot/issues/11594
Co-authored-by: Sojan Jose <sojan@pepalo.com>
# Pull Request Template
## Description
Updates the public swagger spec to be OpenAPI 3.0.4 compliant. Regarding
#7893, I'm investigating the use of
[oas_rails](https://github.com/a-chacon/oas_rails) to auto-generate the
documentation along with correct expected payloads.
Mostly fixes#10531, related to #7893
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How Has This Been Tested?
Copied spec into swagger editor, reviewed outputs:
Note that this shows two errors with the `DELETE` endpoints as
technically these should not have a `requestBody` - in which case we
should be making use of another HTTP verb or another endpoint.
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Daniel Jimenez <devildan.knex@gmail.com>
Co-authored-by: Daniel Jimenez <daniel.jimenez@spark64.com>
The current task for loading `GeoLite2-City.mmdb` doesn't work for all build types. This PR addresses this and move the task to initializer to ensure consistency across environments.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Sojan Jose <sojan.official@gmail.com>
Since we download the GeoIP database during worker/server initialization, there is a high chance of spamming the server with too many requests for downloads, especially if the number of web and worker nodes is high. This PR provides the ability to specify a custom URL for the GeoLite database download, configurable via an environment variable. This helps in distributing the load and avoiding server overload during the initialization process
This PR refactors the schema we introduced in #7518 based on the feedback from production tests. Here is the change log
- Decouple Inbox association to a new table inbox_response_sources -> this lets us share the same response source between multiple inboxes
- Add a status field to responses. This ensures that, by default, responses are created in pending status. You can do quality assurance before making them active. In future, this status can be leveraged by the bot to auto-generate response questions from conversations which require a handoff
- Add response_source association to responses and remove hard dependency from response_documents. This lets users write free-form question answers based on conversations, which doesn't necessarily need a response source.
* Add annotate gem to the project
* Annotate models, fixtures, factories and model_specs
* Keep annotations only in Models
* Remove unwanted changes in model specs
* Exclude auto_annotate_models from rubocop
