Best Node Selection
Stream Manager has a built in mechanism for selecting & providing the best (optimal) node for an appropriate streaming operation. This mechanism allows stream manager to select an optimal node based on state of available resources on it.
Best node selection logic uses a
metric-weight evaluation mechanism to determine the optimal candidate. When a publisher/subscriber request arrives on the Stream Manager requesting an origin/edge for publishing/subscribing, it creates a pool of nodes suitable for the request.Stream Manager then evaluates the
node-score for each candidate node in the pool. The node having the best
node-score is then returned to the client as per request.
InstanceMetricsRuleManager component in Stream Manager uses predefined
metric rules, where specified metrics are configured with predefined
metric weight which project their importance relative to other metrics.
Each metric in the metric evaluation system can be put in one of the two categorizes -
HIGHERBETTER. For metric of type
LOWERBETTER, a increase in value leads to decrease in weight and for a metric of type
HIGHERBETTER increase in value causes increase in weight.
Currently all of defined metrics are of
Stream Manager, internally deals with two types of metrics :
STATIC: Metrics that are directly available via node reports.
DYNAMIC: Metrics that are not directly available, but are derived using
Metrics that affect the best node selection algorithm are defined in the configuration file
net weight or
node score is evaluated for all candate nodes for a node type using all the metrics defined that influence the algorithm, the current value of each metric and assigned weight for the respective metric.The
node score refers to the sum of all metric weights for a node. The node with the highest score is preferred over other candidate nodes.