Intermediate
Big Data & Data Engineering
Q49 / 100
Why is it important for pipeline tasks in Airflow to be idempotent?
Correct! Well done.
Incorrect.
The correct answer is A) So that retries or backfills after a failure do not produce duplicate or inconsistent data
A
Correct Answer
So that retries or backfills after a failure do not produce duplicate or inconsistent data
Explanation
Pipelines often need to be retried or backfilled; idempotent tasks (e.g., using overwrite instead of append) ensure re-running produces the same correct result.
Progress
49/100