Accelerating Real-Time Analytics with Amazon QuickSight and SPICE

Challenge: Slow Ad-Hoc Reporting and Aurora Database Overload from Analytics Queries

TargetBay’s BayEngage platform powers campaign delivery for over 4,500 e-commerce merchants. The platform captures billions of campaign events — opens, clicks, bounces, and conversions — within Amazon Aurora MySQL clusters. Internal teams rely on real-time insights to monitor campaign performance, detect failures, and drive product decisions. Three critical issues were undermining this capability:

• Slow, Costly Ad-Hoc Reporting: Direct SQL queries against large Aurora tables caused multi-minute response times, production database CPU spikes, and risk to transactional workloads. Complex analytical queries competed for resources with real-time campaign processing.
• Fragmented Data Architecture: Customer and event data was distributed across multiple Aurora clusters, preventing unified reporting. Teams could not easily join tenant, campaign, and engagement data across cluster boundaries for cross-cutting analysis.
• Delayed Error Visibility: Campaign or batch job failures were typically discovered only during next-day reporting cycles, meaning issues that could have been caught in minutes went undetected for hours, impacting merchant campaigns and eroding platform reliability.

Solution: Deploying Amazon QuickSight with SPICE In-Memory Engine for Near Real-Time Analytics

FactualMinds deployed Amazon QuickSight with the SPICE in-memory engine to decouple analytics from production databases, providing near real-time reporting without impacting transactional workloads.

Data Ingestion and SPICE Configuration:

• Configured QuickSight to ingest data from each Aurora MySQL cluster independently
• Scheduled incremental SPICE refreshes every 5 to 15 minutes, ensuring dashboards reflect current state without continuous database queries
• Stored key tables — tenants, campaigns, batches, and events — in SPICE for rapid in-memory access

Unified Cross-Cluster Reporting:

• Joined tenant, batch, and campaign tables from multiple SPICE data sources within QuickSight
• Created unified dashboards spanning tenants and regions, providing a single-pane view that was previously impossible with direct database queries

Live Dashboards and Monitoring:

• Built dashboards covering campaign engagement metrics (CTR, open rate, bounce rate), batch processing success and failure rates, and top-performing tenants
• Configured threshold-based alerts and anomaly detection directly within QuickSight, enabling immediate notification when metrics deviate from expected patterns

Self-Service Analytics:

• Empowered business teams to build ad-hoc visualizations — funnels, heat maps, time series, and pivot tables — without requiring engineering involvement or database access

AWS Services Used for QuickSight SPICE Analytics

• Amazon QuickSight with SPICE — In-memory analytics engine with scheduled incremental refresh
• Amazon Aurora MySQL — Source database clusters for campaign and tenant data
• Amazon CloudWatch — Infrastructure monitoring and alerting
• AWS IAM — Access control for QuickSight dashboards and data sources

Results: Near Real-Time Analytics with 5–15 Min QuickSight SPICE Refresh Cycles

The QuickSight deployment transformed TargetBay’s analytics capabilities from reactive and fragmented to near real-time and self-service:

• Near real-time analytics with 5 to 15 minute refresh cycles replacing multi-minute ad-hoc queries against production databases
• Eliminated reporting lag by surfacing campaign failures, anomalies, and performance degradation within minutes rather than the following day
• Reduced Aurora database overhead by offloading analytical workloads to SPICE, freeing production clusters to focus on transactional processing
• Self-service analytics for business users enabling product, marketing, and operations teams to explore data independently without engineering support
• Instant anomaly detection and alerts through threshold-based monitoring that proactively flags issues before they escalate to merchant-facing impact

For more on Amazon QuickSight consulting and real-time analytics on AWS, see our analytics service page.