Chinh (lelouvincx) / 2026-03-13

Notes

• Tasks

• Done

  • TODO Search for a tool to summarize youtube videos

• TODO Check failed dbt tests in #data-ops-bot, list all issues, then create corresponding linear issues for them

• DONE DAT-554

• DONE DAT-555

• fct_job_queue_performance

  • 1-Ampcode 2-Ampcode 3-Ampcode (fix) 4-Ampcode (migrate) 5-Ampcode (migrate continue)
  • PR dbt (reviewing): https://github.com/holistics/dbt/pull/854
  • PR Holistics (draft): https://github.com/holistics/internal-aml-project/pull/69
  • This model is used by dataset Job Queue Performance, then used by dashboard [WIP] Job Queue Performace 3.0
    • Not a big impact because no one is using this dashboard right now.
    • The team is probably using this dashboard Report Job Queue Performance Monitoring.
  • But is this dbt model essential? Worth keeping?
    • This model estimates, at minute-level, how loaded each tenant queue is and how many were waiting vs total active jobs.
      • Minute-level => very big.
    • https://holistics.slack.com/archives/C79QB3FEV/p1755662645561689
      • There is already a solution for this: dashboards those let customers monitor directly in their tenant (via embedding). Similar to Usage Monitoring.
      • In Job Monitoring -> Analytics.
    • This dbt model is staled.
  • Should we remove it?
    • Is there any use case in the future that we can consider using it?
      • I don’t think so. The only use case is Job Monitoring, which is already addressed by squad performance.
    • Is it safe if we remove this model? Yes, we can recreate it from fct_jobs because that model already has key inputs:
      • created_at, start_time, end_time
      • assigned_queue
      • queue_limit
      • queue_valid_from
      • tenant/region keys
    • The cost for storing this table: <$1/month.
    • Yes. Remove it.
  • DONE Write a notion document about this and let teammate review => https://www.notion.so/holistics/DE-212-336f89dc7e498006a2a0e78622de854c

• DONE Write downstream models

  • Discuss with anh Triet this morning. The point is, this model must be at minute-level, to help debug tracing.
  • Decision:
    • Keep this dbt model, refactor to fix the bigquery resource limit
      • Cap the incremental lookback window to 7 days for fct_job_queue_performance to prevent BigQuery resource limit errors when _dbt_max_partition falls behind.
      • Caveat: as long as the schedule does not paused for longer than 7 days, it’s okay.
    • Clean up dashboards on bi.holistics.io
      • One dashboard to clean up is: [WIP] Job Queue Performace 3.0
      • Other dashboards we can convert to 4.0:
        • Job Queue Performance Estimation (v2)
        • Report Job Queue Performance Monitoring
  • PR review done. Backfill completed.
  • Now proceed to clean up BI layer

  • Dashboard	Decision	Reason	Output
    [WIP] Job Queue Performance 3.0	Delete	Unused WIP dashboard	DONE
    Job Queue Performance Estimation (v2)	Migrate	Used by Dat/Triet for performance debug tracing. For example: Timo case.	Job Queue Performance. Note: still keep the dashboard since it is shared with customers (shareable link)
    Report Job Queue Performance Monitoring	Keep (migrate to 4.0)	Check for redundancy with v2	DONE
    Tenant execeeds job queue limit	Keep		DONE

  • BI clean up done. Note: for migrating the dashboard Job Queue Performance, still keep the old dashboard because it is shared with customers (shareable link).
  • Close this task.

• mart_product__dataset_datamodel_dimensions (parse json)

  • Opencode
  • Ampcode
  • Delegate to Thuan (junior) to debug.
  • I already know the bug.
  • Done: https://linear.app/holistics/issue/DAT-567

• unique_mart_product__dataset_datamodel_dependencies_skey

  - [Ampcode](https://ampcode.com/threads/T-019d0598-aa1b-7779-a563-4b5f42594a81)
  - Done: https://linear.app/holistics/issue/DAT-575
  

  • Skipped models: itg_customer_support__organizations, mart_customer_support__organizations, mart_product__activities, mart_product__activities_backfilled_event, mart_product__as_code_objects, mart_product__tag_assignments, mart_product__tenant_features_usage_monthly

• DONE Onboarding call with Showbie

• ((69b21fb9-6670-4bab-a135-2054dd790815))

• DONE Fix DAT-554 Elementary dbt test n_records column missing