溫馨提示×

溫馨提示×

您好,登錄后才能下訂單哦!

密碼登錄×
登錄注冊×
其他方式登錄
點擊 登錄注冊 即表示同意《億速云用戶服務條款》

cassandra-stress 壓測使用介紹

發布時間:2020-07-15 00:11:50 來源:網絡 閱讀:1430 作者:無鋒劍 欄目:大數據

Cassandra從很早的版本就自帶了cassandra-stress壓力測試工具,它的使用方法在cassandra-stress后添加命令和選項。其中常用的命令一般只用到:write、read、mixed、user。其中單純的write和read只測試讀和寫,mixed則測試同時讀寫。user是2.1之后新增的,通過自定義配置文件,在配置文件中可以指定insert和query查詢語句。
命令command沒有以-開頭,而選項[options]有兩種方式:-選項名稱 選項值或者選項名稱=選項值。

常用的選項有

counter_read: 多個并發讀,必須首先通過counter_write測試填充群集。
counter_write:多個并發寫。
legacy:傳統模式的支持。
mixed:混合模式和可配置的讀寫比例和分布。集群必須先寫測試數據填充。
read: 多個并發讀取。必須首先通過寫入測試填充群集。
write: 針對群集的多個并發寫入。
user:  交錯用戶提供具有可配置比率和分布的查詢。
version: 打印cassandra-stress版本。
print: 打印定義輸出 

子選項

-COL

列詳細信息,例如大小和計數分布,數據生成器

用法:
-col names =?[slice] [super =?] [comparator =?] [timestamp =?] [size = DIST(?)]
 要么 
-col [n = DIST(?)] [slice] [super =?] [comparator =?] [timestamp =?] [size = DIST(?)]
-rate

速率

使用以下選項設置費率:
-rate threads=N [throttle=N] [fixed=N]
配置項:
threads=N  并發運行的客戶端數量。
throttle=N 所有客戶端的每秒操作達到最大速率,默認值為0。
fixed=N    期望所有客戶每秒的固定運行率。默認值為0。

或者
-rate [threads>=N] [threads<=N] [auto]
參數:
threads > = N :同時運行至少這么多客戶端。默認值為4。
threads <= N :最多同時運行這么多客戶端。默認值為1000。
auto 一旦吞吐量飽和,就停止增加線程。
-errors

如何處理壓力測試期間遇到的錯誤

用法:
-errors [retries = N] [ignore] [skip-read-validation]
retries=N 失敗前嘗試次數。
ignore 忽略錯誤。
skip-read-validation 跳過讀取驗證和消息輸出。
-graph

壓力測試生成結果圖表,可以將多個測試一起繪制成圖表。

用法:
-graph file=? [revision=?] [title=?] [op=?]
-log

日志設置

用法:
level=verbose
or
-log [level=?] [no-summary] [file=?] [hdrfile=?] [interval=?] [no-settings] [no-progress] [show-queries] [query-log-file=?]
-mode

Thrift or CQL 選項

用法:
-mode thrift [smart] [user=?] [password=?]
  or 
-mode native [unprepared] cql3 [compression=?] [port=?] [user=?] [password=?] [auth-provider=?] [maxPending=?] [connectionsPerHost=?] [protocolVersion=?]
  or
-mode simplenative [prepared] cql3 [port=?]
-node

要連接的節點

用法:
-node [datacenter=?] [whitelist] [file=?] []
-port

指定用于連接Cassandra節點的端口。9042端口用于native協議的客戶端連接。

-port [native=?] [thrift=?] [jmx=?]
:-schema

表結構設置

用法:
-schema [replication(?)] [keyspace=?] [compaction(?)] [compression=?]
-sendto

指定要將壓力命令發送到的服務器。

用法:
-sendto <host>

-tokenrange
令牌范圍設置。

用法:
-tokenrange [no-wrap] [split-factor =?] [savedata =?]

額外選項

profile=?:指定YAML配置文件,需要自己編寫DML,插入,查詢;(只能作為user選項的子選項)
ops(?):   指定操作類型和數量,比如ops(inserts=1),或者ops(queries=2),其中queries需要用指定的查詢名稱代替;(只能作為user選項的子選項)
n=?:      指定操作數量,比如要寫入1萬條數據,n=10000; 要讀取1000條數據,n=1000;
err<?:  指定均值的標準誤差; 達到此值時, cassandra-stress將結束。默認值為0.02;
truncate=?: 是否需要清空表,可選項有:never(默認值),one,always;
cl=?:     一致性級別,可選項有:ONE,QUORUM,LOCAL_QUORUM,EACH_QUORUM,ALL,ANY,LOCAL_ONE(默認值);(只能作為user選項的子選項)
no-warmup:不要預熱過程,冷啟動任務。

--注意
選項名稱,選項值必須放在 子選項選項值前面,比如正確的用法:truncate=one -node xxx

簡單讀寫壓測示例

#插入(寫入)一百萬行
cassandra-stress write n = 1000000 -rate threads = 50 -node  172.20.101.166 -port native=9042

#讀二十萬行。
cassandra-stress讀n = 200000 -rate threads = 50 -node 172.20.101.157 -port native=9042

#讀取行持續3分鐘。

cassandra-stress read duration = 3m -rate threads = 50 -node 172.20.101.164 -port native=9042

?;旌献x寫持續5分鐘。
cassandra-stress mixed duration = 5m -rate threads = 50 -node 172.20.101.164 -port native=9042

#首先讀取200,000行而不預熱50,000行。
cassandra-stress read n = 200000 no-warmup -rate threads = 50 -node 172.20.101.160, 172.20.101.166 -port native=9042

#通過身份驗證運行cassandra-stress
以下示例顯示使用-mode選項提供用戶名和密碼:
cassandra-stress -mode native cql3 user = cassandra password = cassandra no-warmup cl = QUORUM

復雜壓測示例:

100萬條數據寫入,一致性級別為Local_Quorum,客戶端線程數=500個,2個列,副本數據=3個

cassandra-stress write n=1000000 cl=LOCAL_QUORUM -rate threads=500 \
    -col "size=fixed(2048)" "n=fixed(32)" -schema "replication(factor=3)" -node 172.20.101.157 -port native=9042

使用user profile 配置yaml

#
# This is an example YAML profile for cassandra-stress
#
# insert data
# cassandra-stress user profile=/home/jake/stress1.yaml ops(insert=1)
#
# read, using query simple1:
# cassandra-stress profile=/home/jake/stress1.yaml ops(simple1=1)
#
# mixed workload (90/10)
# cassandra-stress user profile=/home/jake/stress1.yaml ops(insert=1,simple1=9)

#
# Keyspace info
#
keyspace: load_test

#
# The CQL for creating a keyspace (optional if it already exists)
#
keyspace_definition: |
  CREATE KEYSPACE load_test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
#
# Table info
#
table: event2

#
# The CQL for creating a table you wish to stress (optional if it already exists)
#
table_definition: |
  CREATE TABLE event2 (
        cookie_id int,
        timestamp timestamp,
        event_name text,
        session_id uuid,
        page text,
        device text,
        PRIMARY KEY(cookie_id, timestamp, event_name, session_id)
  ) WITH CLUSTERING ORDER BY (timestamp DESC)

#
# Optional meta information on the generated columns in the above table
# The min and max only apply to text and blob types
# The distribution field represents the total unique population
# distribution of that column across rows.  Supported types are
# 
#      EXP(min..max)                        An exponential distribution over the range [min..max]
#      EXTREME(min..max,shape)              An extreme value (Weibull) distribution over the range [min..max]
#      GAUSSIAN(min..max,stdvrng)           A gaussian/normal distribution, where mean=(min+max)/2, and stdev is (mean-min)/stdvrng
#      GAUSSIAN(min..max,mean,stdev)        A gaussian/normal distribution, with explicitly defined mean and stdev
#      UNIFORM(min..max)                    A uniform distribution over the range [min, max]
#      FIXED(val)                           A fixed distribution, always returning the same value
#      Aliases: extr, gauss, normal, norm, weibull
#
#      If preceded by ~, the distribution is inverted
#
# Defaults for all columns are size: uniform(4..8), population: uniform(1..100B), cluster: fixed(1)
#
columnspec:
  - name: cookie_id
    population: uniform(1..100M)   # the range of unique values to select for the field (default is 100Billion)
  - name: timestamp
    size: fixed(13)
    population: uniform(1..100M)
  - name: event_name
    size: uniform(5..10)
    population: uniform(1..100M)
  - name: session_id
    size: fixed(32)
    population: uniform(1..100M)
  - name: page
    size: gaussian(16..64)
    population: uniform(1..100M)
  - name: device
    size: fixed(4)
    population: uniform(1..10)

insert:
  partitions: uniform(1..50)       # number of unique partitions to update in a single operation
                                  # if batchcount > 1, multiple batches will be used but all partitions will
                                  # occur in all batches (unless they finish early); only the row counts will vary
  batchtype: UNLOGGED               # type of batch to use
  select: fixed(10)/10       # uniform chance any single generated CQL row will be visited in a partition;
                                  # generated for each partition independently, each time we visit it

#
# A list of queries you wish to run against the schema
#
queries:
   simple1:
      cql: select * from event2 where cookie_id = ?
      fields: samerow             # samerow or multirow (select arguments from the same row, or randomly from all rows in the partition)
Collapse

user profile 測試案例-1

不要預熱過程,冷啟動任務,插入1000000萬條數據,讀寫比例3比1 ,一致性級別設置為:QUORUM
cassandra-stress user profile=./pttest-cassandra.yaml n=1000000 ops(insert=3,simple1=1) no-warmup cl=QUORUM -node 172.20.101.157 -port native=9042

結果

Results:
Op rate                   :    1,562 op/s  [insert: 1,164 op/s, simple1: 398 op/s]
Partition rate            :   30,063 pk/s  [insert: 29,667 pk/s, simple1: 396 pk/s]
Row rate                  :   30,063 row/s [insert: 29,667 row/s, simple1: 396 row/s]
Latency mean              :   28.2 ms [insert: 37.0 ms, simple1: 2.7 ms]
Latency median            :    4.3 ms [insert: 6.5 ms, simple1: 1.1 ms]
Latency 95th percentile   :  126.2 ms [insert: 142.5 ms, simple1: 7.7 ms]
Latency 99th percentile   :  230.4 ms [insert: 254.1 ms, simple1: 23.8 ms]
Latency 99.9th percentile :  476.8 ms [insert: 554.2 ms, simple1: 96.6 ms]
Latency max               : 2581.6 ms [insert: 2,581.6 ms, simple1: 2,264.9 ms]
Total partitions          :    999,932 [insert: 986,756, simple1: 13,176]
Total errors              :          0 [insert: 0, simple1: 0]
Total GC count            : 43
Total GC memory           : 12.565 GiB
Total GC time             :    3.0 seconds
Avg GC time               :   69.4 ms
StdDev GC time            :   13.4 ms
Total operation time      : 00:00:33

Improvement over 181 threadCount: -7%

user profile 測試案例-2

不要預熱過程,冷啟動任務,500并發,插入1000000萬條數據,讀寫比例3比1 ,一致性級別設置為:QUORUM
cassandra-stress user profile=./pttest-cassandra.yaml n=1000000 ops(insert=3,simple1=1) no-warmup cl=QUORUM -rate threads=500 -node 172.20.101.157 -port native=9042

Results:
Op rate                   :    1,496 op/s  [insert: 1,145 op/s, simple1: 396 op/s]
Partition rate            :   28,763 pk/s  [insert: 29,221 pk/s, simple1: 394 pk/s]
Row rate                  :   28,763 row/s [insert: 29,221 row/s, simple1: 394 row/s]
Latency mean              :   49.5 ms [insert: 64.0 ms, simple1: 7.5 ms]
Latency median            :    4.9 ms [insert: 7.0 ms, simple1: 1.3 ms]
Latency 95th percentile   :  251.3 ms [insert: 286.0 ms, simple1: 29.7 ms]
Latency 99th percentile   :  513.0 ms [insert: 561.0 ms, simple1: 118.3 ms]
Latency 99.9th percentile :  909.1 ms [insert: 940.0 ms, simple1: 477.1 ms]
Latency max               : 1416.6 ms [insert: 1,416.6 ms, simple1: 820.0 ms]
Total partitions          :    999,847 [insert: 986,542, simple1: 13,305]
Total errors              :          0 [insert: 0, simple1: 0]
Total GC count            : 41
Total GC memory           : 12.277 GiB
Total GC time             :    3.0 seconds
Avg GC time               :   72.3 ms
StdDev GC time            :   25.0 ms
Total operation time      : 00:00:34

發現并發并沒有提升效率!

user profile 測試案例-3

連接多節點,
不要預熱過程,冷啟動任務,插入1000000萬條數據,讀寫比例3比1 ,一致性級別設置為:QUORUM

cassandra-stress user profile=./pttest-cassandra.yaml n=1000000 ops(insert=3,simple1=1) no-warmup cl=QUORUM -node 172.20.101.157,172.20.101.160,172.20.101.167 -port native=9042

結果:

Results:
Op rate                   :    1,570 op/s  [insert: 1,181 op/s, simple1: 389 op/s]
Partition rate            :   30,638 pk/s  [insert: 30,250 pk/s, simple1: 389 pk/s]
Row rate                  :   30,638 row/s [insert: 30,250 row/s, simple1: 389 row/s]
Latency mean              :   32.6 ms [insert: 42.4 ms, simple1: 2.6 ms]
Latency median            :    4.4 ms [insert: 6.6 ms, simple1: 1.3 ms]
Latency 95th percentile   :  128.6 ms [insert: 146.9 ms, simple1: 7.7 ms]
Latency 99th percentile   :  283.4 ms [insert: 336.3 ms, simple1: 21.6 ms]
Latency 99.9th percentile : 2334.1 ms [insert: 2,344.6 ms, simple1: 78.0 ms]
Latency max               : 2730.5 ms [insert: 2,730.5 ms, simple1: 414.7 ms]
Total partitions          :    999,987 [insert: 987,306, simple1: 12,681]
Total errors              :          0 [insert: 0, simple1: 0]
Total GC count            : 43
Total GC memory           : 12.817 GiB
Total GC time             :    3.1 seconds
Avg GC time               :   72.1 ms
StdDev GC time            :   14.9 ms
Total operation time      : 00:00:32

user profile 測試案例-4

生成測試結果圖
連接多節點,不要預熱過程,冷啟動任務,插入1000000萬條數據,讀寫比例3比1 ,一致性級別設置為:QUORUM

cassandra-stress user profile=./pttest-cassandra.yaml n=1000000 ops(insert=3,simple1=1) no-warmup cl=QUORUM -graph file=test.html title=test revision=test1 -node 172.20.101.157,172.20.101.160,172.20.101.167 -port native=9042

 Results:
Op rate                   :    1,602 op/s  [insert: 1,204 op/s, simple1: 398 op/s]
Partition rate            :   30,941 pk/s  [insert: 30,543 pk/s, simple1: 398 pk/s]
Row rate                  :   30,941 row/s [insert: 30,543 row/s, simple1: 398 row/s]
Latency mean              :   15.6 ms [insert: 20.1 ms, simple1: 2.1 ms]
Latency median            :    4.1 ms [insert: 5.9 ms, simple1: 1.2 ms]
Latency 95th percentile   :   67.2 ms [insert: 74.7 ms, simple1: 5.4 ms]
Latency 99th percentile   :  111.9 ms [insert: 118.9 ms, simple1: 11.7 ms]
Latency 99.9th percentile :  274.2 ms [insert: 289.4 ms, simple1: 71.3 ms]
Latency max               : 2231.4 ms [insert: 2,231.4 ms, simple1: 271.1 ms]
Total partitions          :    999,996 [insert: 987,127, simple1: 12,869]
Total errors              :          0 [insert: 0, simple1: 0]
Total GC count            : 41
Total GC memory           : 12.276 GiB
Total GC time             :    2.7 seconds
Avg GC time               :   66.9 ms
StdDev GC time            :   15.4 ms
Total operation time      : 00:00:32

Improvement over 81 threadCount: -8%

參考文檔:
https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsCStress.html
https://www.instaclustr.com/deep-diving-cassandra-stress-part-3-using-yaml-profiles/
https://zqhxuyuan.github.io/2015/10/15/Cassandra-Stress/

向AI問一下細節

免責聲明:本站發布的內容(圖片、視頻和文字)以原創、轉載和分享為主,文章觀點不代表本網站立場,如果涉及侵權請聯系站長郵箱:is@yisu.com進行舉報,并提供相關證據,一經查實,將立刻刪除涉嫌侵權內容。

AI

亚洲午夜精品一区二区_中文无码日韩欧免_久久香蕉精品视频_欧美主播一区二区三区美女