MoERouting.h File Reference

Mixture of Experts (MoE) routing logic. More...

#include <ExpertRouter.hpp>
#include <algorithm>

Include dependency graph for MoERouting.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces
namespace	MoERouting

Functions
int	MoERouting::collapse_top2_if_dominant (float &top1Weight, float &top2Weight, float dominantExpertCutoff=0.8f)
	Collapses the top 2 experts into a single expert if the top one is highly dominant.
void	MoERouting::route_top2_base_experts (const FactorizedInput &inp, int &e0, int &e1, float &w0, float &w1)
	Identifies the top 2 base experts for a given factorized input.

Detailed Description

Mixture of Experts (MoE) routing logic.

Provides utility functions to select the top active experts for the neural network based on the input features.