## Summary
- The conversation transcript endpoint rate limit is hardcoded at 30
requests/hour per account with no way to override it
- Self-hosted users with active accounts hit this limit and get 429
errors across all channels
- Add `RATE_LIMIT_CONVERSATION_TRANSCRIPT` env var (default: `1000`) to
make it configurable, consistent with other throttles like
`RATE_LIMIT_CONTACT_SEARCH` and `RATE_LIMIT_REPORTS_API_ACCOUNT_LEVEL`
## Description
Make the $velma Redis connection pool size configurable via
`REDIS_VELMA_SIZE` environment variable (default: 5, matching current
behavior)
The $velma pool is used exclusively by Rack::Attack for rate limiting
and was the only Redis pool with a hardcoded size
## Fixes
Under high traffic, the hardcoded $velma pool (size: 5) causes
connection contention. Every HTTP request passes through Rack::Attack
middleware, which requires a $velma Redis connection. When
`WEB_CONCURRENCY=2` and `RAILS_MAX_THREADS=10` (20 concurrent threads),
the 4:1 thread-to-connection ratio causes threads to queue for up to 1
second (the pool timeout), resulting in intermittent request latency
spikes during traffic bursts.
The $alfred pool was already configurable via REDIS_ALFRED_SIZE — this
change brings $velma to parity.
## Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Low Risk**
> Low risk: changes only Redis connection pool sizing for Rack::Attack;
misconfiguration could cause rate-limiting Redis contention or extra
connections but no data/auth logic changes.
>
> **Overview**
> Makes the `velma` Redis connection pool (used by Rack::Attack)
configurable via a new `REDIS_VELMA_SIZE` env var, replacing the
previously hardcoded pool size.
>
> Documents `REDIS_VELMA_SIZE` in `.env.example` alongside the existing
`REDIS_ALFRED_SIZE` setting.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
dcbc946f2e1d7356dc743178ca46cdf12cb25c78. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
The existing /api health check endpoint creates a new Redis connection
on every request and checks both Redis and Postgres availability. During
peak traffic, this creates unnecessary load and can cause cascading
failures when either service is slow - instances get marked unhealthy,
traffic shifts to remaining instances, which then also fail health
checks.
The new /health endpoint:
- Returns immediately with 200 {"status":"woot"}
- Skips all middleware and authentication
- No Redis or Postgres dependency
- Suitable for health checks that only need to verify the web server is
responding
## Description
Increased the alfred pool size to 10, so that each worker has enough
redis connection thread pool to work
## Type of change
Please delete options that are not relevant.
- [ ] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces configurability for the Redis pool used by `alfred`.
>
> - `$alfred` `ConnectionPool` size now reads from
`ENV['REDIS_ALFRED_SIZE']` with a default of `5`
> - Adds `REDIS_ALFRED_SIZE=10` to `.env.example`
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
96cdff8c0ea40f82a57d70be053780e87384ed47. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: tanmay <tanmay-chatwoot@tanmays-MacBook-Pro.local>
## Context
Sidekiq logs only showed the Sidekiq wrapper class and JID, which wasn’t
helpful when debugging ActiveJobs.
## Changes
- Updated `ChatwootDequeuedLogger` to log the actual `ActiveJob class`
and `job_id` instead of the generic Sidekiq wrapper and JID.
> Example
> ```
> Dequeued ActionMailer::MailDeliveryJob
123e4567-e89b-12d3-a456-426614174000 from default
> ```
- Remove sidekiq worker and unify everything to `ActiveJob`
Previously, email replies were handled inside workers. There was no
execution logs. This meant if emails silently failed (as reported by a
customer), we had no way to trace where the issue happened, the only
assumption was “no error = mail sent.”
By moving email handling into jobs, we now have proper execution logs
for each attempt. This makes it easier to debug delivery issues and
would have better visibility when investigating customer reports.
Fixes
https://linear.app/chatwoot/issue/CW-5538/emails-are-not-sentdelivered-to-the-contact
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
## Linear:
- https://github.com/chatwoot/chatwoot/issues/486
## Description
This PR implements Multi-Factor Authentication (MFA) support for user
accounts, enhancing security by requiring a second form of verification
during login. The feature adds TOTP (Time-based One-Time Password)
authentication with QR code generation and backup codes for account
recovery.
## Type of change
- [ ] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
- Added comprehensive RSpec tests for MFA controller functionality
- Tested MFA setup flow with QR code generation
- Verified OTP validation and backup code generation
- Tested login flow with MFA enabled/disabled
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
We now support searching within the actual message content, email
subject lines, and audio transcriptions. This enables a faster, more
accurate search experience going forward. Unlike the standard message
search, which is limited to the last 3 months, this search has no time
restrictions.
The search engine also accounts for small variations in queries. Minor
spelling mistakes, such as searching for slck instead of Slack, will
still return the correct results. It also ignores differences in accents
and diacritics, so searching for Deja vu will match content containing
Déjà vu.
We can also refine searches in the future by criteria such as:
- Searching within a specific inbox
- Filtering by sender or recipient
- Limiting to messages sent by an agent
Fixes https://github.com/chatwoot/chatwoot/issues/11656
Fixes https://github.com/chatwoot/chatwoot/issues/10669
Fixes https://github.com/chatwoot/chatwoot/issues/5910
---
Rake tasks to reindex all the messages.
```sh
bundle exec rake search:all
```
Rake task to reindex messages from one account only
```sh
bundle exec rake search:account ACCOUNT_ID=1
```
Rails 7.1 ships with connection-pooling enabled by default for
`RedisCacheStore` (see rails/rails#45235).
Because we already wrap our Redis clients in our own `ConnectionPool`
($alfred / $velma), the upgrade resulted in a double-wrapped object and
runtime errors such as:
NoMethodError: undefined method `get` for an instance of ConnectionPool
This patch:
* Passes `pool: false` when instantiating `RedisCacheStore` in
`config/initializers/rack_attack.rb`, telling Rails to use the pool we
supply instead of building its own.
* Adds an inline comment explaining the rationale.
* Adds a TODO in `config/initializers/01_redis.rb` suggesting a future
simplification: switch to plain Redis clients and let Rails manage the
pool.
Reference docs:
* rails/rails#45235 – “Enable connection pooling by default for
MemCacheStore and RedisCacheStore” -
https://github.com/rails/rails/pull/45235
* Rails 7.1 Caching Guide – 2.1.1 “Connection Pool Options” (use `pool:
false`) [Ruby on Rails
Guides](https://guides.rubyonrails.org/v7.1/caching_with_rails.html)
Migration Guide: https://chwt.app/v4/migration
This PR imports all the work related to Captain into the EE codebase. Captain represents the AI-based features in Chatwoot and includes the following key components:
- Assistant: An assistant has a persona, the product it would be trained on. At the moment, the data at which it is trained is from websites. Future integrations on Notion documents, PDF etc. This PR enables connecting an assistant to an inbox. The assistant would run the conversation every time before transferring it to an agent.
- Copilot for Agents: When an agent is supporting a customer, we will be able to offer additional help to lookup some data or fetch information from integrations etc via copilot.
- Conversation FAQ generator: When a conversation is resolved, the Captain integration would identify questions which were not in the knowledge base.
- CRM memory: Learns from the conversations and identifies important information about the contact.
---------
Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
- Throttle reports API requests at the account level
- Throttle reports API requests at the user level for dashboard users as
well as API users
Co-authored-by: Sojan Jose <sojan@pepalo.com>
40 % of Chatwoot's current log volume is from state transition logs generated by `rack-timeout`, which are logged at the `INFO` level. This PR reduce the noise in logs and set RACK::TIMEOUT to log at `error` level
The current task for loading `GeoLite2-City.mmdb` doesn't work for all build types. This PR addresses this and move the task to initializer to ensure consistency across environments.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Sojan Jose <sojan.official@gmail.com>
This PR addresses several issues related to logging:
- Enabling Lograge broke certain non-API URLs, such as password reset.
This occurred due to the user ID tagging we had in Lograge, which has
now been limited to API pages only.
- Disabled the start and done logs in Sidekiq.
- Investigated why Sidekiq logs weren’t being output as JSON. This is
due to the use of ActiveJob instead of Sidekiq for the job base classes.
**Potential Options for Converting ActiveJob Logs to JSON:**
- https://glozer.rocks/ojb
- https://learnedreverie.medium.com/activejob-logs-as-json-6912403d8c81
- https://github.com/roidrage/lograge/pull/226
- Fix git-hash in super admin instance health page for Heroku installations
- Fix git boundary not set error for docker/heroku deployments which remove .git directory
This PR refactors the schema we introduced in #7518 based on the feedback from production tests. Here is the change log
- Decouple Inbox association to a new table inbox_response_sources -> this lets us share the same response source between multiple inboxes
- Add a status field to responses. This ensures that, by default, responses are created in pending status. You can do quality assurance before making them active. In future, this status can be leveraged by the bot to auto-generate response questions from conversations which require a handoff
- Add response_source association to responses and remove hard dependency from response_documents. This lets users write free-form question answers based on conversations, which doesn't necessarily need a response source.
- The message sender was incorrect. Incoming messages were created under the contact's name, and outgoing messages were created under the user's name instead of the reverse.
- The seed user's email address was incorrect in the message data.
- The Sendmail configuration overrode the Letter Opener config which made it difficult to test the email.
- This PR also fixes an ESLint lint issue on develop.
This commit introduces the ability to associate response sources to an inbox, allowing external webpages to be parsed by Chatwoot. The parsed data is converted into embeddings for use with GPT models when managing customer queries.
The implementation relies on the `pgvector` extension for PostgreSQL. Database migrations related to this feature are handled separately by `Features::ResponseBotService`. A future update will integrate these migrations into the default rails migrations, once compatibility with Postgres extensions across all self-hosted installation options is confirmed.
Additionally, a new GitHub action has been added to the CI pipeline to ensure the execution of specs related to this feature.
