What is the mixture of experts (MoE) architecture?

A Combining predictions from multiple separate models B A neural network architecture routing inputs to a subset of "expert" feed-forward networks, enabling sparse activation of a much larger total parameter count C An ensemble method combining expert ML practitioners' models D A multi-task learning framework with specialized heads