Repository navigation
Optimize the performance and stability of TiDB running on AWS i3.xlarge/i3.2xlarge #18025
Description
Description
Optimize the performance and stability of TiDB running on 8 virtual cores(vcores) environment(like AWS i3.2xlarge or m5.2xlarge).
Currently, in order to gain a better performance, one TiKV instance may require 12 or more vcores, this is not cost-effective. CPU is more expensive than memory/disk in the cloud environment, the total number of CPU vcore is constant in a given machine, for example, 96vcores, and only can allocate six 4xlarge(16vcore) type instances. Adding more memory or disk to a machine is easy to archive.
In order to reduce the cost of the TiDB cluster on the cloud, we should reduce the CPU core requirement for TiKV at first. There are too many thread pools and other threads in TiKV, too many thread pools and threads will cause bad performance if there are only 8 vcores.
Category
Cost-Effective, Performance
Value
Reduce cost on cloud/on-per.
Task List
- Reduce overhead of the regular threads
- Reduce split cost
- Reduce scheduler cost
- Reduce the cost of gc worker
- Reduce thread pools
- Combine scheduler pool to unified thread pools
- [TBD]Combine apply pool and raftstore pool
- Reduce other threads
Workload Estimation
200
Time
GanttProgress: 10%
GanttStart: 2020-08-01
GanttDue: 2020-11-30
Activity
iamxy commentedon Jun 29, 2020
For general users AWS i3.2xlarge as the starting instance to deploy TiDB, the cost is still high. The public cloud recommends 8 GiB of memory per vCPU as the best specification for running a database. If we can make 4vCPU and 32GB memory as the baseline hardware requirements for TiKV node, this will be significant for users.
Another ideal suggestion is to use EBS storage instead of the local disk by default. This is not because EBS is cheaper than local disks, but opposite the high provisioned IOPS EBS may be more expensive. But it can bring more flexibility and reliability, which allows users to have more instance types to choose, and the user's data will have more redundancy, thereby enhancing the system's fault tolerance. More importantly, the use of EBS is a necessary condition for elastic scheduling, because the node can be scaled vertically without transmitting data.
The following is a list containing recommended instance types (for TiKV nodes)
AWS
GCP
[-]Performance is good enough on AWS i3.2xlarge[/-][+]Performance is good enough on AWS i3.xlarge/i3.2xlarge[/+][-]Performance is good enough on AWS i3.xlarge/i3.2xlarge[/-][+]Optimize the performance and stability of TiDB running on AWS i3.xlarge/i3.2xlarge[/+]iamxy commentedon Jul 29, 2020
From cost considerations, it is recommended that we focus on supporting Amazon EBS gp2 volumes
See: https://aws.amazon.com/ebs/volume-types/
and Standard provisioned Persistent Disk for Google Cloud
See: https://cloud.google.com/compute/docs/disks
zhangjinpeng87 commentedon Jul 30, 2020
@iamxy I think you are talking about this issue #18024
iamxy commentedon Jul 30, 2020
Yes, thank you for linking.