Simulate Faults on Physical Machines

You can use PhysicalMachineChaos to simulate the faults of network, disk, pressure, JVM, time, and others in physical or virtual machines. Before using PhysicalMachineChaos provided by Chaos Mesh, you need to deploy Chaosd in your physical or virtual machines. The version mapping between Chaos Mesh and Chaosd is as follows:

Before creating PhysicalMachineChaos experiments using Chaos Mesh, you need to deploy Chaosd in service mode to all physical or virtual machines that are going to be injected with faults. After deploying Chaosd, run Chaosd server as follows:

1. Start Chaosd server:

   • After generating the TLS certificate through Chaosctl, run the following command to start Chaosd in service mode:

     note

     The TLS certificates are saved to the default output directory of Chaosctl. If you manually specified another directory when generating certificates, replace the directory in the command line with the corresponding one.

• If the TLS certificate is not generated through Chaosctl, you can run the following command to start Chaosd in service mode. However, for the security of your clusters, this is not recommended.

1. Open Chaos Dashboard, and click NEW EXPERIMENT on the page to create a new experiment:

   

2. Click PHYSIC in Experiment Type and choose a specific type of the experiment, such as NETWORK ATTACK. Then, choose a specific behavior and fill out the corresponding configurations:

3. Fill out the experiment information to specify the range and the planned duration of the experiment:

4. Submit the experiment.

1. Write the experimental configurations to the “physicalmachine.yaml” file. For example:

   The experimental configurations above send HTTP requests to Chaosd service in the specified physical machines or virtual machines to trigger network latency experiments.

2. Create the experiment using kubectl. The command is as follows:

Each fault action has its own specific configurations. The following section introduces various fault types and their corresponding configuration methods.

CPU stress

To simulate a CPU stress scenario, you need to set the action to “stress-cpu”. For the corresponding configurations, refer to Parameters of simulating CPU stress.

Memory stress

To simulate a memory stress scenario, you need to set the action to “stress-mem”. For the corresponding configurations, refer to Parameters of simulating memory stress.

Disk read load

To simulate a disk read load scenario, you need to set the action to “disk-read-payload”. For the corresponding configurations, refer to Parameters of simulating disk read load.

Disk write load

To simulate a disk write load scenario, you need to set the action to “disk-write-payload”. For the corresponding configurations, refer to Parameters of simulating disk write load.

Disk fill

Network corruption

To simulate a network corruption scenario, you need to set the action to “network-corrupt”. For the corresponding configurations, refer to Parameters of simulating network corruption.

Network latency

To simulate a network latency scenario, you need to set the action to “network-delay”. For the corresponding configurations, refer to Parameters of simulating network latency.

Network duplication

To simulate a network duplication scenario, you need to set the action to “network-duplicate”. For the corresponding configurations, refer to Parameters of simulating network duplication.

Network loss

To simulate a network loss scenario, you need to set the action to “network-loss”. For the corresponding configurations, refer to Parameters of simulating network loss.

Network partition

To simulate a network partition scenario, you need to set the action to “network-partition”. For the corresponding configurations, refer to Parameters of simulating network partition.

DNS fault

To simulate a DNS fault scenario, you need to set the action to “network-dns”. For the corresponding configurations, refer to Parameters of simulating DNS fault.

Process fault

To simulate a process fault scenario, you need to set the action to “process”. For the corresponding configurations, refer to Parameters of simulating process fault.

Throw custom exceptions for JVM applications

To simulate a scenario that JVM applications throw custom exceptions, you need to set the action to “jvm-exception”. For the corresponding configurations, refer to Parameters for throwing custom exceptions.

Increase method latency for JVM applications

To simulate a scenario that JVM applications increase method latency, you need to set the action to “jvm-latency”. For the corresponding configurations, refer to Parameters for increasing method latency.

Modify return values of methods for JVM applications

To simulate a scenario that JVM applications modify return values of methods, you need to set the action to “jvm-return”. For the corresponding configurations, refer to Parameters for modifying return values of a method.

Trigger garbage collection for JVM applications

To simulate a scenario that JVM applications trigger garbage collection, you need to set the action to “jvm-gc”. For the corresponding configurations, refer to Parameters for triggering garbage collection.

Trigger faults using Byteman configuration files for JVM applications

Time offset

To simulate a time offset scenario, you need to set the action to “clock”. For the corresponding configurations, refer to Parameters of simulating time offset.