What is Broadway in Elixir and how does it handle data pipelines?

Broadway is Elixir's concurrent, multi-stage data processing pipeline library built on top of GenStage. It provides a higher-level API for processing data from producers (SQS, Kafka, RabbitMQ, etc.) through processing stages and into batch consumers. Define a pipeline: defmodule MyPipeline do; use Broadway; def start_link(_), do: Broadway.start_link(__MODULE__, name: __MODULE__, producer: [module: {BroadwaySQS.Producer, queue_url: url}], processors: [default: [concurrency: 10]], batchers: [default: [batch_size: 100, batch_timeout: 1000]]); def handle_message(_, msg, _), do: process(msg); def handle_batch(_, msgs, _, _), do: bulk_insert(msgs); end. Features: automatic back-pressure, rate limiting, acknowledgement management, error handling with retry/discard semantics, metrics (message rate, processing time), and graceful shutdown. Broadway is ideal for processing message queues, event streams, and any data pipeline requiring controlled throughput with back-pressure.