Request Routing between clusters

  • Fields in HTTP Header is used to define routing rule for distributing traffic in cluster level, for example:
    • host, path, query, cookie, method, etc.
  • BFE provide a “Condition” expression to define how to route message with special header. This is routing rule in cluster level load balance.
  • If multiple rules are configured, BFE would match the rules in sequence. Matching procedure stop if one rule is matched.

Example

  • A product “demo”, which needs process three kinds of traffic: static content traffic, “post” traffic, other traffic, then we can define three clusters ：

    • demo-static：serve static content
    • demo-post：serve post message
    • demo-main: serve other traffic

Sub cluster level load balance

  • In sub cluster level, load balance rules also can be configured. The rules define traffic weight which is distributed to each sub cluster。

  • A special virtual sub cluster “BLACKHOLE” could be used to discard traffic。

  • Consider following configuration：

    • Two IDC：IDC1, IDC2
    • Two BFE cluster：BFE1, BFE2
    • Two backend sub cluster：sub-cluster-1, sub-cluster-2
    • BFE1：{sub-cluster-1:w11，sub-cluster-2:w12, Blackhole:w1B}
    • BFE2：{sub-cluster-1:w21，sub-cluster-2:w22, Blackhole:w2B}

  • Based on the weight in configuration, BFE distribute traffic to backend sub cluster.

    • For example，if weight configuration {w11, w12, w1B} ={45，45，10}, percent of traffic to sub cluster is sub-cluster-1, sub-cluster-2 and Blackhole is 45%, 45% and 10%.

Instance level load balance

  • Usually, a sub cluster is composed of multiple instances. Within sub cluster, WRR（weighted round robin) is used to distribute message among instance。
  • Instance can be assign different weight based on its capacity。

Health check of instance

BFE do health check for each backend instance. A instance has following two states:

  • NORMAL state：the instance act normally in processing message.

State switch:

  • NORMAL to CHECKING, when：

    • Consecutive failure, in connecting or sending message to the instance, exceed a threshhod.

  • CHECKING to NORMAL, when：

    • BFE receive correct response from backend instance for health check request.

Message retry in failure

  • Re-route message within same sub cluster.
  • Re-route message to other sub cluster.

TCP connection between BFE and backend instance support：

  • short-lived connection：BFE route each request message to backend server with a new established TCP connection.

  • connection pool：

    • BFE maintain connection pool to instances.

    • To an incoming request：

      • if there is an available connection, reuse it.
      • else establish a new TCP connection.

    • After finish processing a request via a connection:

      • If current size of connection pool is less than configured number, the connection is added into the pool.
      • Else close the connection directly.

Session stickiness

  • Source IP
  • Field in request header, cookie etc.

Keep session in different routing level:

  • Instance level: message of a session is sent to same instance.