feat(rollup): add models and write path [1/3] (#13796)
## PR#1: Reporting events rollup — model and write path Reporting queries currently hit the `reporting_events` table directly. This works, but the table grows linearly with event volume, and aggregation queries (counts, averages over date ranges) get progressively slower as accounts age. This PR introduces a pre-aggregated `reporting_events_rollups` table that stores daily per-metric, per-dimension (account/agent/inbox) totals. The write path is intentionally decoupled from the read path — rollup rows are written inline from the event listener via upsert, and a backfill service exists to rebuild historical data from raw events. Nothing reads from this table yet. The write path activates when an account has a `reporting_timezone` set (new account setting). The `reporting_events_rollup` feature flag controls only the future read path, not writes — so rollup data accumulates silently once timezone is configured. A `MetricRegistry` maps raw event names to rollup column semantics in one place, keeping the write and (future) read paths aligned. ### What changed - Migration for `reporting_events_rollups` with a unique composite index for upsert - `ReportingEventsRollup` model - `reporting_timezone` account setting with IANA timezone validation - `MetricRegistry` — single source of truth for event-to-metric mappings - `RollupService` — real-time upsert from event listener - `BackfillService` — rebuilds rollups for a given account + date from raw events - Rake tasks for interactive backfill and timezone setup - `reporting_events_rollup` feature flag (disabled by default) ### How to test 1. Set a `reporting_timezone` on an account (`Account.first.update!(reporting_timezone: 'Asia/Kolkata')`) 2. Resolve a conversation or trigger a first response 3. Check `ReportingEventsRollup.where(account_id: ...)` — rows should appear 4. Run backfill: `bundle exec rake reporting_events_rollup:backfill` and verify historical data populates --------- Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
This commit is contained in:
@@ -85,11 +85,13 @@ class Account < ApplicationRecord
|
||||
validates_with JsonSchemaValidator,
|
||||
schema: SETTINGS_PARAMS_SCHEMA,
|
||||
attribute_resolver: ->(record) { record.settings }
|
||||
validate :validate_reporting_timezone
|
||||
|
||||
store_accessor :settings, :auto_resolve_after, :auto_resolve_message, :auto_resolve_ignore_waiting
|
||||
|
||||
store_accessor :settings, :audio_transcriptions, :auto_resolve_label
|
||||
store_accessor :settings, :captain_models, :captain_features
|
||||
store_accessor :settings, :reporting_timezone
|
||||
store_accessor :settings, :keep_pending_on_bot_failure
|
||||
store_accessor :settings, :captain_auto_resolve_mode
|
||||
include AccountCaptainAutoResolve
|
||||
@@ -215,6 +217,12 @@ class Account < ApplicationRecord
|
||||
# method overridden in enterprise module
|
||||
end
|
||||
|
||||
def validate_reporting_timezone
|
||||
return if reporting_timezone.blank? || ActiveSupport::TimeZone[reporting_timezone].present?
|
||||
|
||||
errors.add(:reporting_timezone, I18n.t('errors.account.reporting_timezone.invalid'))
|
||||
end
|
||||
|
||||
def remove_account_sequences
|
||||
ActiveRecord::Base.connection.exec_query("drop sequence IF EXISTS camp_dpid_seq_#{id}")
|
||||
ActiveRecord::Base.connection.exec_query("drop sequence IF EXISTS conv_dpid_seq_#{id}")
|
||||
|
||||
48
app/models/reporting_events_rollup.rb
Normal file
48
app/models/reporting_events_rollup.rb
Normal file
@@ -0,0 +1,48 @@
|
||||
# == Schema Information
|
||||
#
|
||||
# Table name: reporting_events_rollups
|
||||
#
|
||||
# id :bigint not null, primary key
|
||||
# count :bigint default(0), not null
|
||||
# date :date not null
|
||||
# dimension_id :bigint not null
|
||||
# dimension_type :string not null
|
||||
# metric :string not null
|
||||
# sum_value :float default(0.0), not null
|
||||
# sum_value_business_hours :float default(0.0), not null
|
||||
# created_at :datetime not null
|
||||
# updated_at :datetime not null
|
||||
# account_id :integer not null
|
||||
#
|
||||
# Indexes
|
||||
#
|
||||
# index_rollup_summary (account_id,dimension_type,date)
|
||||
# index_rollup_timeseries (account_id,metric,date)
|
||||
# index_rollup_unique_key (account_id,date,dimension_type,dimension_id,metric) UNIQUE
|
||||
#
|
||||
|
||||
class ReportingEventsRollup < ApplicationRecord
|
||||
belongs_to :account
|
||||
|
||||
# Store string values directly in the database for better readability and debugging
|
||||
enum :dimension_type, %w[account agent inbox team].index_by(&:itself)
|
||||
enum :metric, %w[
|
||||
resolutions_count
|
||||
first_response
|
||||
resolution_time
|
||||
reply_time
|
||||
bot_resolutions_count
|
||||
bot_handoffs_count
|
||||
].index_by(&:itself)
|
||||
|
||||
validates :account_id, presence: true
|
||||
validates :date, presence: true
|
||||
validates :dimension_type, presence: true
|
||||
validates :dimension_id, presence: true
|
||||
validates :metric, presence: true
|
||||
validates :count, numericality: { greater_than_or_equal_to: 0 }
|
||||
|
||||
scope :for_date_range, ->(start_date, end_date) { where(date: start_date..end_date) }
|
||||
scope :for_dimension, ->(type, id) { where(dimension_type: type, dimension_id: id) }
|
||||
scope :for_metric, ->(metric) { where(metric: metric) }
|
||||
end
|
||||
Reference in New Issue
Block a user