Simulation tools

The simulation client is a machine which will create and subscribe to topics with configurable message rates and sizes.Because it is sometimes necessary in simulating large load to use multiple client machines, the user does not interactwith the simulation client directly, but instead delegates their requests to the simulation controller, which will thensend signals to clients to start incurring load. The client implementation is in the class.

To Start a simulation client, use the pulsar-perf script with the command simulation-client as follows:

The simulation controller send signals to the simulation clients, requesting them to create new topics, stop oldtopics, change the load incurred by topics, as well as several other tasks. It is implemented in the classorg.apache.pulsar.testclient.LoadSimulationController and presents a shell to the user as an interface to sendcommand with.

To start a simulation controller, use the pulsar-perf script with the command simulation-controller as follows:

• Create a group of topics with a producer and a consumer
  • trade_group <tenant> <group> <num_namespaces> [—rate <message rate per second>] [—rand-rate <lower bound>,<upper bound>] [—separation <separation between creating topics in ms>] [—size <message size in bytes>] [—topics-per-namespace <number of topics to create per namespace>]
• Change the configuration of an existing topic
  • change <tenant> <namespace> <topic> [—rate <message rate per second>] [—rand-rate <lower bound>,<upper bound>] [—size <message size in bytes>]
• Change the configuration of a group of topics
• Shutdown a previously created group of topics
  • stop_group <tenant> <group>
• Copy the historical data from one ZooKeeper to another and simulate based on the message rates and sizes in thathistory
  • copy <tenant> <source zookeeper> <target zookeeper> [—rate-multiplier value]
• Simulate the load of the historical data on the current ZooKeeper (should be same ZooKeeper being simulated on)
  • simulate <tenant> <zookeeper> [—rate-multiplier value]

The commands copy, simulate, and stream are very similar but have significant differences. is used whenyou want to simulate the load of a static, external ZooKeeper on the ZooKeeper you are simulating on. Thus,source zookeeper should be the ZooKeeper you want to copy and target zookeeper should be the ZooKeeper you aresimulating on, and then it will get the full benefit of the historical data of the source in both load managerimplementations. simulate on the other hand takes in only one ZooKeeper, the one you are simulating on. It assumesthat you are simulating on a ZooKeeper that has historical data for SimpleLoadManagerImpl and creates equivalenthistorical data for ModularLoadManagerImpl. Then, the load according to the historical data is simulated by theclients. Finally, stream takes in an active ZooKeeper different than the ZooKeeper being simulated on and streamsload data from it and simulates the real-time load. In all cases, the optional rate-multiplier argument allows theuser to simulate some proportion of the load. For instance, using —rate-multiplier 0.05 will cause messages tobe sent at only 5% of the rate of the load that is being simulated.

To observe the behavior of the load manager in these simulations, one may utilize the broker monitor, which isimplemented in org.apache.pulsar.testclient.BrokerMonitor. The broker monitor will print tabular load data to theconsole as it is updated using watchers.

The console will then continuously print load data until it is interrupted.