Boilerplate

This boilerplate demonstrates how to implement Server-Sent Events (SSE) in Fastify, using MinIO as a local S3 storage and event-driven processing for large file imports. Ideal for developers looking to integrate real-time progress tracking in file processing workflows.

Technical Stack

• Backend: Fastify
• Storage: MinIO (S3-compatible)
• Job Scheduler: Database polling every 1 minute
• Event Handling: Fastify EventEmitter
• Streaming: Server-Sent Events (SSE)

Key Features

• File Import Handling: Accepts a users.csv file (1k users) via the /file_import endpoint.
• S3 Storage with MinIO: Files are stored locally using a Docker-based MinIO service.
• Scheduled Processing: A job runs every 1 minute to check for pending file imports.
• Real-Time Progress Updates: Uses EventEmitter to emit progress percentage and processed rows.
• SSE Endpoint for Live Tracking: Exposes an SSE endpoint for frontend apps to receive real-time updates.

Use Cases

This boilerplate is perfect for learning how to implement SSE in Fastify, handling real-time updates for batch processing tasks like CSV imports, ETL pipelines, and background jobs.

With this setup, you can efficiently manage file processing while keeping users informed of the progress—all in real time! 🚀