RENEWA Knowledge Base

Background Jobs

2 min read

Background Jobs

BullMQ job queues with Redis for async processing. All recurring or background work goes through backend/src/lib/jobs/ — setInterval() / setTimeout() recursion / in-process loops are forbidden (duplicate work across replicas, drop on crash, no retry/DLQ). Bare setInterval outside lib/jobs/ is caught by quick-lint.sh.

Since I#2013, processors are batch presentation (see Persistence Topology): stateless Job<Data> → JobResult functions, deps injected via DI, never importing @/db, and never consuming container.services — they see only the ProcessorContainer view (domain + infrastructure + crossCutting, no services slice; see Dependency Injection). Legacy processor→service deps (e.g. entraSync, billingEmail) are allowlisted and migrate on touch.

Job System Structure

All job infrastructure lives in backend/src/lib/jobs/:

FilePurpose
factory.tsQueue + worker creation and lifecycle (getNotificationQueue, getOperationsQueue, getRenditionQueue)
config.tsRedis connection config per environment
types.tsJob type definitions, payload interfaces, QueueNames
schedulers.tsRecurring job registration (repeat.pattern cron syntax)
health.tsQueue health monitoring and metrics
index.tsPublic API exports

Queues

Three general-purpose queues (QueueNames in types.ts, as of 2026-06):

QueuePurposeExample processors
notificationsEmail/SMS dispatchprocessors/notifications.ts, billing-email.ts, document-reminder.ts, rejection-bundle.ts
renditionsPDF→PNG conversion + image thumbnailsprocessors/rendition.ts, processors/pdf-conversion.ts (Stage 1 enqueues Stage 2 rendition jobs)
operationsLong-running sync/maintenance (concurrency 1, mutex-guarded singletons)processors/operations.ts, entra-sync.ts, fulfillment-chain-reconciliation.ts

Scheduled work registered in schedulers.ts includes reminder-scheduler (hourly), rejection-bundle-processor (every minute), escalation-scheduler (hourly), entra-sync (hourly at :30), token-cleanup, and HubSpot webhook log cleanup.

Specialized HubSpot Queues

Dedicated queues for HubSpot Integration (rate-limited to respect API limits):

  • hubspot-sync-queue.ts → processors/hubspotSyncProcessor.ts
  • hubspot-webhook-event-queue.ts → processors/hubspotWebhookEventProcessor.ts
  • hubspot-association-recon-queue.ts → processors/hubspotAssociationReconProcessor.ts

Backfill Jobs

backend/src/lib/jobs/rendition-backfill.ts finds original Files without thumbnail renditions and enqueues generation jobs — idempotent, safe on every startup and as a scheduled job (not a one-time migration).

Process Supervision

Workers run in-process inside the backend: backend/src/index.ts starts the notification, operations, rendition, and HubSpot workers at boot (skipped when NODE_ENV=test). supervisord.conf supervises the backend and nginx processes in the container — there are no separate worker processes.

Redis Connection

Cloud environments use Upstash Redis via the ioredis-compatible UPSTASH_REDIS_URL (BullMQ requires ioredis, not the REST API); local/test use a direct Redis connection. Configured in lib/jobs/config.ts.

See Also

Graph View

Backlinks