Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
[info] welcome to sbt 1.7.1 (Eclipse Adoptium Java 17.0.6)
[info] loading settings for project cs206-demos-build-build-build from metals.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project/project/project
[info] loading settings for project cs206-demos-build-build from metals.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project/project
[success] Generated .bloop/cs206-demos-build-build.json
[success] Total time: 0 s, completed Mar 5, 2023, 12:55:52 PM
[info] loading settings for project cs206-demos-build from metals.sbt,plugins.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project
[success] Generated .bloop/cs206-demos-build.json
[success] Total time: 0 s, completed Mar 5, 2023, 12:55:52 PM
[info] loading settings for project cs206-demos from build.sbt ...
[info] set current project to cs206-demos (in build file:/Users/mbovel/cs206-demos/)
[success] Total time: 0 s, completed Mar 5, 2023, 12:55:53 PM
[info] compiling 7 Scala sources and 1 Java source to /Users/mbovel/cs206-demos/target/scala-3.2.0/classes ...
[warn] there was 1 deprecation warning; re-run with -deprecation for details
[warn] one warning found
[info] done compiling
[info] running org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator /Users/mbovel/cs206-demos/target/scala-3.2.0/classes /Users/mbovel/cs206-demos/target/scala-3.2.0/src_managed/jmh /Users/mbovel/cs206-demos/target/scala-3.2.0/resource_managed/jmh default
Processing 18 classes from /Users/mbovel/cs206-demos/target/scala-3.2.0/classes with "reflection" generator
Writing out Java source to /Users/mbovel/cs206-demos/target/scala-3.2.0/src_managed/jmh and resources to /Users/mbovel/cs206-demos/target/scala-3.2.0/resource_managed/jmh
[info] compiling 22 Java sources to /Users/mbovel/cs206-demos/target/scala-3.2.0/classes ...
[info] done compiling
[info] running (fork) org.openjdk.jmh.Main -wi 5 -w 2 -i 5 -r 2 -f 4 -rf JSON -rff benchmarks/results/01-AppendBenchmark-openjdk-2.json AppendBenchmark
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench00_prepend
[info] # Parameters: (seqType = List)
[info] # Run progress: 0.00% complete, ETA 00:05:20
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 351700723.794 ops/s
[info] # Warmup Iteration 2: 442326063.471 ops/s
[info] # Warmup Iteration 3: 482146561.904 ops/s
[info] # Warmup Iteration 4: 483658907.401 ops/s
[info] # Warmup Iteration 5: 482756851.941 ops/s
[info] Iteration 1: 483905096.217 ops/s
[info] Iteration 2: 480055700.561 ops/s
[info] Iteration 3: 486374433.407 ops/s
[info] Iteration 4: 485267406.361 ops/s
[info] Iteration 5: 485538971.150 ops/s
[info] # Run progress: 6.25% complete, ETA 00:05:05
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 350719571.619 ops/s
[info] # Warmup Iteration 2: 451206807.714 ops/s
[info] # Warmup Iteration 3: 490100763.137 ops/s
[info] # Warmup Iteration 4: 487314789.690 ops/s
[info] # Warmup Iteration 5: 488113654.210 ops/s
[info] Iteration 1: 487971215.935 ops/s
[info] Iteration 2: 485010242.963 ops/s
[info] Iteration 3: 491266982.891 ops/s
[info] Iteration 4: 490008642.469 ops/s
[info] Iteration 5: 490948599.274 ops/s
[info] # Run progress: 12.50% complete, ETA 00:04:45
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 363573271.893 ops/s
[info] # Warmup Iteration 2: 462069479.886 ops/s
[info] # Warmup Iteration 3: 492345947.173 ops/s
[info] # Warmup Iteration 4: 492191071.421 ops/s
[info] # Warmup Iteration 5: 492139581.795 ops/s
[info] Iteration 1: 492520442.587 ops/s
[info] Iteration 2: 489770324.541 ops/s
[info] Iteration 3: 492411394.728 ops/s
[info] Iteration 4: 492768058.846 ops/s
[info] Iteration 5: 490506614.579 ops/s
[info] # Run progress: 18.75% complete, ETA 00:04:24
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 364944800.124 ops/s
[info] # Warmup Iteration 2: 465323902.710 ops/s
[info] # Warmup Iteration 3: 494130840.274 ops/s
[info] # Warmup Iteration 4: 494837429.876 ops/s
[info] # Warmup Iteration 5: 494424176.116 ops/s
[info] Iteration 1: 488339537.372 ops/s
[info] Iteration 2: 494676555.216 ops/s
[info] Iteration 3: 492751118.977 ops/s
[info] Iteration 4: 492657421.495 ops/s
[info] Iteration 5: 494607578.811 ops/s
[info] Result "benchmarks.AppendBenchmark.bench00_prepend":
[info] 489367816.919 ±(99.9%) 3425286.291 ops/s [Average]
[info] (min, avg, max) = (480055700.561, 489367816.919, 494676555.216), stdev = 3944564.787
[info] CI (99.9%): [485942530.628, 492793103.210] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench00_prepend
[info] # Parameters: (seqType = Array)
[info] # Run progress: 25.00% complete, ETA 00:04:04
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 237676.397 ops/s
[info] # Warmup Iteration 2: 273790.235 ops/s
[info] # Warmup Iteration 3: 275467.418 ops/s
[info] # Warmup Iteration 4: 275771.821 ops/s
[info] # Warmup Iteration 5: 277771.672 ops/s
[info] Iteration 1: 272648.090 ops/s
[info] Iteration 2: 275371.603 ops/s
[info] Iteration 3: 275254.768 ops/s
[info] Iteration 4: 274080.371 ops/s
[info] Iteration 5: 273171.142 ops/s
[info] # Run progress: 31.25% complete, ETA 00:03:44
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 237634.994 ops/s
[info] # Warmup Iteration 2: 273381.633 ops/s
[info] # Warmup Iteration 3: 273584.442 ops/s
[info] # Warmup Iteration 4: 273792.013 ops/s
[info] # Warmup Iteration 5: 278053.274 ops/s
[info] Iteration 1: 272667.336 ops/s
[info] Iteration 2: 274319.970 ops/s
[info] Iteration 3: 274584.893 ops/s
[info] Iteration 4: 275925.086 ops/s
[info] Iteration 5: 276867.875 ops/s
[info] # Run progress: 37.50% complete, ETA 00:03:23
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 233125.681 ops/s
[info] # Warmup Iteration 2: 275188.759 ops/s
[info] # Warmup Iteration 3: 275966.930 ops/s
[info] # Warmup Iteration 4: 272522.073 ops/s
[info] # Warmup Iteration 5: 273351.390 ops/s
[info] Iteration 1: 272635.686 ops/s
[info] Iteration 2: 274204.618 ops/s
[info] Iteration 3: 275334.117 ops/s
[info] Iteration 4: 265310.543 ops/s
[info] Iteration 5: 272905.198 ops/s
[info] # Run progress: 43.75% complete, ETA 00:03:03
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 237741.675 ops/s
[info] # Warmup Iteration 2: 275667.300 ops/s
[info] # Warmup Iteration 3: 274290.395 ops/s
[info] # Warmup Iteration 4: 276886.796 ops/s
[info] # Warmup Iteration 5: 275197.592 ops/s
[info] Iteration 1: 272920.686 ops/s
[info] Iteration 2: 273755.777 ops/s
[info] Iteration 3: 277562.979 ops/s
[info] Iteration 4: 278794.623 ops/s
[info] Iteration 5: 274939.898 ops/s
[info] Result "benchmarks.AppendBenchmark.bench00_prepend":
[info] 274162.763 ±(99.9%) 2338.355 ops/s [Average]
[info] (min, avg, max) = (265310.543, 274162.763, 278794.623), stdev = 2692.853
[info] CI (99.9%): [271824.408, 276501.118] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench01_append
[info] # Parameters: (seqType = List)
[info] # Run progress: 50.00% complete, ETA 00:02:43
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 15907.896 ops/s
[info] # Warmup Iteration 2: 17543.361 ops/s
[info] # Warmup Iteration 3: 17562.433 ops/s
[info] # Warmup Iteration 4: 17506.153 ops/s
[info] # Warmup Iteration 5: 17606.084 ops/s
[info] Iteration 1: 17416.510 ops/s
[info] Iteration 2: 17595.374 ops/s
[info] Iteration 3: 17571.971 ops/s
[info] Iteration 4: 17586.792 ops/s
[info] Iteration 5: 17632.775 ops/s
[info] # Run progress: 56.25% complete, ETA 00:02:22
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 15921.777 ops/s
[info] # Warmup Iteration 2: 17614.063 ops/s
[info] # Warmup Iteration 3: 17664.387 ops/s
[info] # Warmup Iteration 4: 17708.036 ops/s
[info] # Warmup Iteration 5: 17754.180 ops/s
[info] Iteration 1: 17572.550 ops/s
[info] Iteration 2: 17798.175 ops/s
[info] Iteration 3: 17649.892 ops/s
[info] Iteration 4: 17711.858 ops/s
[info] Iteration 5: 17829.488 ops/s
[info] # Run progress: 62.50% complete, ETA 00:02:02
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 16069.859 ops/s
[info] # Warmup Iteration 2: 16463.425 ops/s
[info] # Warmup Iteration 3: 16269.710 ops/s
[info] # Warmup Iteration 4: 16324.154 ops/s
[info] # Warmup Iteration 5: 16297.561 ops/s
[info] Iteration 1: 16890.848 ops/s
[info] Iteration 2: 16196.471 ops/s
[info] Iteration 3: 16421.244 ops/s
[info] Iteration 4: 16265.258 ops/s
[info] Iteration 5: 16513.462 ops/s
[info] # Run progress: 68.75% complete, ETA 00:01:41
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 15278.443 ops/s
[info] # Warmup Iteration 2: 16907.713 ops/s
[info] # Warmup Iteration 3: 16914.745 ops/s
[info] # Warmup Iteration 4: 16906.849 ops/s
[info] # Warmup Iteration 5: 16919.514 ops/s
[info] Iteration 1: 16457.784 ops/s
[info] Iteration 2: 16449.054 ops/s
[info] Iteration 3: 16808.030 ops/s
[info] Iteration 4: 16859.095 ops/s
[info] Iteration 5: 16712.004 ops/s
[info] Result "benchmarks.AppendBenchmark.bench01_append":
[info] 17096.932 ±(99.9%) 507.909 ops/s [Average]
[info] (min, avg, max) = (16196.471, 17096.932, 17829.488), stdev = 584.909
[info] CI (99.9%): [16589.023, 17604.841] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench01_append
[info] # Parameters: (seqType = Array)
[info] # Run progress: 75.00% complete, ETA 00:01:21
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 282434.850 ops/s
[info] # Warmup Iteration 2: 327567.263 ops/s
[info] # Warmup Iteration 3: 324002.695 ops/s
[info] # Warmup Iteration 4: 330791.419 ops/s
[info] # Warmup Iteration 5: 329071.154 ops/s
[info] Iteration 1: 326898.733 ops/s
[info] Iteration 2: 294169.651 ops/s
[info] Iteration 3: 307854.566 ops/s
[info] Iteration 4: 289012.308 ops/s
[info] Iteration 5: 302312.117 ops/s
[info] # Run progress: 81.25% complete, ETA 00:01:01
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 161724.285 ops/s
[info] # Warmup Iteration 2: 244821.235 ops/s
[info] # Warmup Iteration 3: 243622.158 ops/s
[info] # Warmup Iteration 4: 308415.193 ops/s
[info] # Warmup Iteration 5: 315329.987 ops/s
[info] Iteration 1: 292714.247 ops/s
[info] Iteration 2: 314064.207 ops/s
[info] Iteration 3: 302659.810 ops/s
[info] Iteration 4: 257611.097 ops/s
[info] Iteration 5: 166652.528 ops/s
[info] # Run progress: 87.50% complete, ETA 00:00:40
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 136955.280 ops/s
[info] # Warmup Iteration 2: 224680.879 ops/s
[info] # Warmup Iteration 3: 223524.474 ops/s
[info] # Warmup Iteration 4: 118912.147 ops/s
[info] # Warmup Iteration 5: 133508.554 ops/s
[info] Iteration 1: 113239.365 ops/s
[info] Iteration 2: 111431.865 ops/s
[info] Iteration 3: 131214.719 ops/s
[info] Iteration 4: 73828.760 ops/s
[info] Iteration 5: 73766.813 ops/s
[info] # Run progress: 93.75% complete, ETA 00:00:20
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 149663.322 ops/s
[info] # Warmup Iteration 2: 131250.281 ops/s
[info] # Warmup Iteration 3: 151107.370 ops/s
[info] # Warmup Iteration 4: 197408.259 ops/s
[info] # Warmup Iteration 5: 192316.822 ops/s
[info] Iteration 1: 225051.819 ops/s
[info] Iteration 2: 292209.120 ops/s
[info] Iteration 3: 302920.295 ops/s
[info] Iteration 4: 300481.202 ops/s
[info] Iteration 5: 288371.057 ops/s
[info] Result "benchmarks.AppendBenchmark.bench01_append":
[info] 238323.214 ±(99.9%) 77612.877 ops/s [Average]
[info] (min, avg, max) = (73766.813, 238323.214, 326898.733), stdev = 89379.105
[info] CI (99.9%): [160710.337, 315936.091] (assumes normal distribution)
[info] # Run complete. Total time: 00:05:27
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
[info] extra caution when trusting the results, look into the generated code to check the benchmark still
[info] works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
[info] different JVMs are already problematic, the performance difference caused by different Blackhole
[info] modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
[info] Benchmark (seqType) Mode Cnt Score Error Units
[info] AppendBenchmark.bench00_prepend List thrpt 20 489367816.919 ± 3425286.291 ops/s
[info] AppendBenchmark.bench00_prepend Array thrpt 20 274162.763 ± 2338.355 ops/s
[info] AppendBenchmark.bench01_append List thrpt 20 17096.932 ± 507.909 ops/s
[info] AppendBenchmark.bench01_append Array thrpt 20 238323.214 ± 77612.877 ops/s
[info] Benchmark result is saved to benchmarks/results/01-AppendBenchmark-openjdk-2.json
[success] Total time: 333 s (05:33), completed Mar 5, 2023, 1:01:26 PM