Tweet Pulse
Polymarket Intelligence
Role
PM-builder · Bobi Labs build
Duration
Q4 2024 · Q1 2025 (live, ongoing)
Status
Live · client-hosted · ongoing maintenance
The problem
Polymarket runs prediction markets on Elon Musk: how many tweets in an hour, will he post about a topic, will silence break a record. The traders following these markets needed real-time intelligence: tweet velocity right now, regime shifts (calm to volatile, neutral to directional), historical patterns, alerts when something just changed. Existing tools either ran on minute-bar data (too slow), buried this audience inside a generic crypto-twitter feed, or paywalled the live data.
The technical challenge was sub-second dashboard response on data updating every few seconds from rate-limited APIs, with three-tier user economics: anonymous landing, free signup unlocks the dashboard, pro subscription unlocks real-time alerts. The client needed it to scale to a community of subscribers, survive tweet-volume spikes, and run unattended.
The approach
Three-layer caching strategy. Postgres materialized views for the heavy aggregates (11 views, trigger-refreshed on new tweet writes with a 15s debounce). Server-side TTL cache in front of the database with thundering-herd protection so a flash crowd hits cache, not Postgres. Client-side TanStack Query cache with per-query auto-refresh intervals. The hot dashboard path serves from MVs in under 500ms cold load.
Real-time tweet ingest as a separate Deno service running on EC2, polling the X API every 5 seconds, deduping across multiple data sources. Survives restarts via systemd. A second EC2 service runs the scheduler that fires Supabase edge functions on intervals (Telegram alerts every minute, MV refresh fallback every 5 minutes, supplementary ingest jobs).
Regime detection runs as a Postgres view classifying tweet velocity across rolling windows. Telegram alerts fire on transitions with deep-links back to the dashboard. Stripe handles the pro-tier subscription. Auth via Supabase OAuth.
What shipped
- ▪Real-time dashboard (stats banner, market heatmap, deep-data views, on-this-day patterns) with sub-500ms render on aggregated data updating every few seconds.
- ▪11 materialized views and 41 query patterns powering the read path, refreshed by triggers on source-table writes with a debounced cron fallback.
- ▪X API tweet poller and edge-function scheduler running as systemd services on a single t4g.micro EC2.
- ▪Regime detection algorithm classifying tweet activity across rolling time windows, with Telegram alerts on regime transitions.
- ▪Stripe subscription flow with webhook reconciliation. Auth via Supabase OAuth.
Stack & decisions
Next.js + TypeScript
App Router with server actions for all data flows. No API routes except the Stripe webhook.
Supabase + Postgres
11 materialized views (3 layers), 41 query patterns, RLS for the authenticated dashboard. Trigger-refreshed MVs on new tweet writes.
EC2 (t4g.micro, ARM64)
Two Deno services on a single instance: tweet ingest (5s poll) and edge-function scheduler. Both as systemd, auto-restart on crash. The client wanted unattended ops; systemd-on-EC2 gave that without paging anyone.
Stripe
Subscription model. Webhook reconciles to Supabase for entitlements.
Telegram Bot API
Alerts on regime transitions, deep-linked back to the dashboard.
X API
Rate-limit-aware polling with multi-source dedup. Hardest engineering work was the rate-limit budgeting.
Outcome
- →Live in production, serving a community of traders. Free signup base + pro tier wired for monetization via Stripe.
- →Sub-500ms dashboard cold load on aggregated data updating every few seconds.
- →Telegram alerts in under 30 seconds of regime transition.
- →Auto-restarting systemd services have run unattended through the operational period without manual intervention.
Have something we should ship?
60-minute scoping call, then a clear plan for how we'd approach it.