TiDB TPC-C Performance Test Report -- v6.2.0 vs. v6.1.0

Test overview

This test aims at comparing the TPC-C performance of TiDB v6.2.0 and v6.1.0 in the Online Transactional Processing (OLTP) scenario. The results show that compared with v6.1.0, the TPC-C performance of v6.2.0 is declined by 2.00%.

Test environment (AWS EC2）

Hardware configuration

Software version

Parameter configuration

TiDB v6.2.0 and TiDB v6.1.0 use the same configuration.

TiDB parameter configuration

log.level: "error"
prepared-plan-cache.enabled: true
tikv-client.max-batch-wait-time: 2000000

TiKV parameter configuration

raftstore.apply-max-batch-size: 2048
raftstore.apply-pool-size: 3
raftstore.store-max-batch-size: 2048
raftstore.store-pool-size: 2
readpool.storage.normal-concurrency: 10
server.grpc-concurrency: 6

TiDB global variable configuration

set global tidb_hashagg_final_concurrency=1;
set global tidb_hashagg_partial_concurrency=1;
set global tidb_enable_async_commit = 1;
set global tidb_enable_1pc = 1;
set global tidb_guarantee_linearizability = 0;
set global tidb_enable_clustered_index = 1;
set global tidb_prepared_plan_cache_size=1000;

HAProxy configuration - haproxy.cfg

For more details about how to use HAProxy on TiDB, see Best Practices for Using HAProxy in TiDB.

global                                     # Global configuration.
   pidfile     /var/run/haproxy.pid        # Writes the PIDs of HAProxy processes into this file.
   maxconn     4000                        # The maximum number of concurrent connections for a single HAProxy process.
   user        haproxy                     # The same with the UID parameter.
   group       haproxy                     # The same with the GID parameter. A dedicated user group is recommended.
   nbproc      64                          # The number of processes created when going daemon. When starting multiple processes to forward requests, ensure that the value is large enough so that HAProxy does not block processes.
   daemon                                  # Makes the process fork into background. It is equivalent to the command line "-D" argument. It can be disabled by the command line "-db" argument.

defaults                                   # Default configuration.
   log global                              # Inherits the settings of the global configuration.
   retries 2                               # The maximum number of retries to connect to an upstream server. If the number of connection attempts exceeds the value, the backend server is considered unavailable.
   timeout connect  2s                     # The maximum time to wait for a connection attempt to a backend server to succeed. It should be set to a shorter time if the server is located on the same LAN as HAProxy.
   timeout client 30000s                   # The maximum inactivity time on the client side.
   timeout server 30000s                   # The maximum inactivity time on the server side.

listen tidb-cluster                        # Database load balancing.
   bind 0.0.0.0:3390                       # The Floating IP address and listening port.
   mode tcp                                # HAProxy uses layer 4, the transport layer.
   balance leastconn                      # The server with the fewest connections receives the connection. "leastconn" is recommended where long sessions are expected, such as LDAP, SQL and TSE, rather than protocols using short sessions, such as HTTP. The algorithm is dynamic, which means that server weights might be adjusted on the fly for slow starts for instance.
   server tidb-1 10.9.18.229:4000 check inter 2000 rise 2 fall 3       # Detects port 4000 at a frequency of once every 2000 milliseconds. If it is detected as successful twice, the server is considered available; if it is detected as failed three times, the server is considered unavailable.
   server tidb-2 10.9.39.208:4000 check inter 2000 rise 2 fall 3
   server tidb-3 10.9.64.166:4000 check inter 2000 rise 2 fall 3

Prepare test data

1. Deploy TiDB v6.2.0 and v6.1.0 using TiUP.
2. Create a database named tpcc: create database tpcc;.
3. Use BenchmarkSQL to import the TPC-C 5000 Warehouse data: tiup bench tpcc prepare --warehouse 5000 --db tpcc -H 127.0.0.1 -P 4000.
4. Run the tiup bench tpcc run -U root --db tpcc --host 127.0.0.1 --port 4000 --time 1800s --warehouses 5000 --threads {{thread}} command to perform stress tests on TiDB via HAProxy. For each concurrency, the test takes 30 minutes.
5. Extract the tpmC data of New Order from the result.

Test result

Compared with v6.1.0, the TPC-C performance of v6.2.0 is declined by 2.00%.