9967101b48
## PR#1: Reporting events rollup — model and write path Reporting queries currently hit the `reporting_events` table directly. This works, but the table grows linearly with event volume, and aggregation queries (counts, averages over date ranges) get progressively slower as accounts age. This PR introduces a pre-aggregated `reporting_events_rollups` table that stores daily per-metric, per-dimension (account/agent/inbox) totals. The write path is intentionally decoupled from the read path — rollup rows are written inline from the event listener via upsert, and a backfill service exists to rebuild historical data from raw events. Nothing reads from this table yet. The write path activates when an account has a `reporting_timezone` set (new account setting). The `reporting_events_rollup` feature flag controls only the future read path, not writes — so rollup data accumulates silently once timezone is configured. A `MetricRegistry` maps raw event names to rollup column semantics in one place, keeping the write and (future) read paths aligned. ### What changed - Migration for `reporting_events_rollups` with a unique composite index for upsert - `ReportingEventsRollup` model - `reporting_timezone` account setting with IANA timezone validation - `MetricRegistry` — single source of truth for event-to-metric mappings - `RollupService` — real-time upsert from event listener - `BackfillService` — rebuilds rollups for a given account + date from raw events - Rake tasks for interactive backfill and timezone setup - `reporting_events_rollup` feature flag (disabled by default) ### How to test 1. Set a `reporting_timezone` on an account (`Account.first.update!(reporting_timezone: 'Asia/Kolkata')`) 2. Resolve a conversation or trigger a first response 3. Check `ReportingEventsRollup.where(account_id: ...)` — rows should appear 4. Run backfill: `bundle exec rake reporting_events_rollup:backfill` and verify historical data populates --------- Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
38 lines
1.1 KiB
Ruby
38 lines
1.1 KiB
Ruby
class CreateReportingEventsRollup < ActiveRecord::Migration[7.1]
|
|
def change
|
|
create_rollups_table
|
|
add_rollups_indexes
|
|
end
|
|
|
|
private
|
|
|
|
def create_rollups_table
|
|
create_table :reporting_events_rollups do |t|
|
|
t.integer :account_id, null: false
|
|
t.date :date, null: false
|
|
t.string :dimension_type, null: false
|
|
t.bigint :dimension_id, null: false
|
|
t.string :metric, null: false
|
|
t.bigint :count, default: 0, null: false
|
|
t.float :sum_value, default: 0.0, null: false
|
|
t.float :sum_value_business_hours, default: 0.0, null: false
|
|
|
|
t.timestamps
|
|
end
|
|
end
|
|
|
|
def add_rollups_indexes
|
|
add_index :reporting_events_rollups,
|
|
[:account_id, :date, :dimension_type, :dimension_id, :metric],
|
|
unique: true, name: 'index_rollup_unique_key'
|
|
|
|
add_index :reporting_events_rollups,
|
|
[:account_id, :metric, :date],
|
|
name: 'index_rollup_timeseries'
|
|
|
|
add_index :reporting_events_rollups,
|
|
[:account_id, :dimension_type, :date],
|
|
name: 'index_rollup_summary'
|
|
end
|
|
end
|