Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
[info] welcome to sbt 1.7.1 (Eclipse Adoptium Java 17.0.6)
[info] loading settings for project cs206-demos-build-build-build from metals.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project/project/project
[info] loading settings for project cs206-demos-build-build from metals.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project/project
[success] Generated .bloop/cs206-demos-build-build.json
[success] Total time: 0 s, completed Mar 4, 2023, 11:07:36 PM
[info] loading settings for project cs206-demos-build from metals.sbt,plugins.sbt ...
[info] loading project definition from /Users/mbovel/cs206-demos/project
[success] Generated .bloop/cs206-demos-build.json
[success] Total time: 0 s, completed Mar 4, 2023, 11:07:37 PM
[info] loading settings for project cs206-demos from build.sbt ...
[info] set current project to cs206-demos (in build file:/Users/mbovel/cs206-demos/)
[success] Total time: 0 s, completed Mar 4, 2023, 11:07:37 PM
[info] compiling 7 Scala sources and 1 Java source to /Users/mbovel/cs206-demos/target/scala-3.2.0/classes ...
[warn] there was 1 deprecation warning; re-run with -deprecation for details
[warn] one warning found
[info] done compiling
[info] running org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator /Users/mbovel/cs206-demos/target/scala-3.2.0/classes /Users/mbovel/cs206-demos/target/scala-3.2.0/src_managed/jmh /Users/mbovel/cs206-demos/target/scala-3.2.0/resource_managed/jmh default
Processing 18 classes from /Users/mbovel/cs206-demos/target/scala-3.2.0/classes with "reflection" generator
Writing out Java source to /Users/mbovel/cs206-demos/target/scala-3.2.0/src_managed/jmh and resources to /Users/mbovel/cs206-demos/target/scala-3.2.0/resource_managed/jmh
[info] compiling 22 Java sources to /Users/mbovel/cs206-demos/target/scala-3.2.0/classes ...
[info] done compiling
[info] running (fork) org.openjdk.jmh.Main -wi 5 -w 2 -i 5 -r 2 -f 4 -rf JSON -rff benchmarks/results/01-AppendBenchmark-openjdk-1.json AppendBenchmark
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench00_prepend
[info] # Parameters: (seqType = List)
[info] # Run progress: 0.00% complete, ETA 00:05:20
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 348314640.382 ops/s
[info] # Warmup Iteration 2: 440656733.630 ops/s
[info] # Warmup Iteration 3: 480539678.347 ops/s
[info] # Warmup Iteration 4: 478571725.024 ops/s
[info] # Warmup Iteration 5: 471545764.287 ops/s
[info] Iteration 1: 481930720.032 ops/s
[info] Iteration 2: 472562223.553 ops/s
[info] Iteration 3: 478494308.304 ops/s
[info] Iteration 4: 477268796.501 ops/s
[info] Iteration 5: 484340793.210 ops/s
[info] # Run progress: 6.25% complete, ETA 00:05:06
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 354459633.839 ops/s
[info] # Warmup Iteration 2: 444895303.405 ops/s
[info] # Warmup Iteration 3: 481534544.224 ops/s
[info] # Warmup Iteration 4: 474739620.384 ops/s
[info] # Warmup Iteration 5: 487221152.181 ops/s
[info] Iteration 1: 485673631.657 ops/s
[info] Iteration 2: 475776674.594 ops/s
[info] Iteration 3: 486181054.466 ops/s
[info] Iteration 4: 483642555.872 ops/s
[info] Iteration 5: 487516728.373 ops/s
[info] # Run progress: 12.50% complete, ETA 00:04:46
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 347642244.768 ops/s
[info] # Warmup Iteration 2: 445497978.928 ops/s
[info] # Warmup Iteration 3: 472791640.751 ops/s
[info] # Warmup Iteration 4: 468161657.750 ops/s
[info] # Warmup Iteration 5: 470855585.390 ops/s
[info] Iteration 1: 473450159.452 ops/s
[info] Iteration 2: 484301909.935 ops/s
[info] Iteration 3: 491654138.003 ops/s
[info] Iteration 4: 487470415.874 ops/s
[info] Iteration 5: 492293770.892 ops/s
[info] # Run progress: 18.75% complete, ETA 00:04:25
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 351873176.694 ops/s
[info] # Warmup Iteration 2: 453276885.691 ops/s
[info] # Warmup Iteration 3: 488173998.859 ops/s
[info] # Warmup Iteration 4: 480333341.035 ops/s
[info] # Warmup Iteration 5: 481854073.801 ops/s
[info] Iteration 1: 482733215.013 ops/s
[info] Iteration 2: 486281484.891 ops/s
[info] Iteration 3: 481221035.557 ops/s
[info] Iteration 4: 484273203.420 ops/s
[info] Iteration 5: 487276153.752 ops/s
[info] Result "benchmarks.AppendBenchmark.bench00_prepend":
[info] 483217148.667 ±(99.9%) 4724403.219 ops/s [Average]
[info] (min, avg, max) = (472562223.553, 483217148.667, 492293770.892), stdev = 5440629.773
[info] CI (99.9%): [478492745.448, 487941551.887] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench00_prepend
[info] # Parameters: (seqType = Array)
[info] # Run progress: 25.00% complete, ETA 00:04:05
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 232076.689 ops/s
[info] # Warmup Iteration 2: 271973.894 ops/s
[info] # Warmup Iteration 3: 275638.109 ops/s
[info] # Warmup Iteration 4: 274522.566 ops/s
[info] # Warmup Iteration 5: 272707.132 ops/s
[info] Iteration 1: 271748.520 ops/s
[info] Iteration 2: 273014.734 ops/s
[info] Iteration 3: 272563.068 ops/s
[info] Iteration 4: 272166.072 ops/s
[info] Iteration 5: 273470.347 ops/s
[info] # Run progress: 31.25% complete, ETA 00:03:44
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 235215.622 ops/s
[info] # Warmup Iteration 2: 270938.421 ops/s
[info] # Warmup Iteration 3: 270419.938 ops/s
[info] # Warmup Iteration 4: 273408.312 ops/s
[info] # Warmup Iteration 5: 272375.184 ops/s
[info] Iteration 1: 269327.491 ops/s
[info] Iteration 2: 269025.765 ops/s
[info] Iteration 3: 273528.800 ops/s
[info] Iteration 4: 271925.280 ops/s
[info] Iteration 5: 275233.564 ops/s
[info] # Run progress: 37.50% complete, ETA 00:03:24
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 230595.057 ops/s
[info] # Warmup Iteration 2: 272342.787 ops/s
[info] # Warmup Iteration 3: 276015.231 ops/s
[info] # Warmup Iteration 4: 272367.984 ops/s
[info] # Warmup Iteration 5: 273383.329 ops/s
[info] Iteration 1: 271906.413 ops/s
[info] Iteration 2: 269770.111 ops/s
[info] Iteration 3: 274532.297 ops/s
[info] Iteration 4: 274142.930 ops/s
[info] Iteration 5: 272262.573 ops/s
[info] # Run progress: 43.75% complete, ETA 00:03:03
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 236305.070 ops/s
[info] # Warmup Iteration 2: 269805.041 ops/s
[info] # Warmup Iteration 3: 275202.223 ops/s
[info] # Warmup Iteration 4: 276049.181 ops/s
[info] # Warmup Iteration 5: 276258.479 ops/s
[info] Iteration 1: 272998.572 ops/s
[info] Iteration 2: 275537.033 ops/s
[info] Iteration 3: 274510.101 ops/s
[info] Iteration 4: 272839.275 ops/s
[info] Iteration 5: 272998.071 ops/s
[info] Result "benchmarks.AppendBenchmark.bench00_prepend":
[info] 272675.051 ±(99.9%) 1553.827 ops/s [Average]
[info] (min, avg, max) = (269025.765, 272675.051, 275537.033), stdev = 1789.390
[info] CI (99.9%): [271121.224, 274228.878] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench01_append
[info] # Parameters: (seqType = List)
[info] # Run progress: 50.00% complete, ETA 00:02:43
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 15779.588 ops/s
[info] # Warmup Iteration 2: 17267.103 ops/s
[info] # Warmup Iteration 3: 17583.667 ops/s
[info] # Warmup Iteration 4: 17466.471 ops/s
[info] # Warmup Iteration 5: 17520.267 ops/s
[info] Iteration 1: 17397.772 ops/s
[info] Iteration 2: 17380.976 ops/s
[info] Iteration 3: 17530.405 ops/s
[info] Iteration 4: 17464.689 ops/s
[info] Iteration 5: 17587.442 ops/s
[info] # Run progress: 56.25% complete, ETA 00:02:22
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 15900.045 ops/s
[info] # Warmup Iteration 2: 17458.777 ops/s
[info] # Warmup Iteration 3: 17563.226 ops/s
[info] # Warmup Iteration 4: 17548.318 ops/s
[info] # Warmup Iteration 5: 17602.615 ops/s
[info] Iteration 1: 17475.678 ops/s
[info] Iteration 2: 17310.880 ops/s
[info] Iteration 3: 17687.887 ops/s
[info] Iteration 4: 17522.416 ops/s
[info] Iteration 5: 17690.410 ops/s
[info] # Run progress: 62.50% complete, ETA 00:02:02
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 16615.014 ops/s
[info] # Warmup Iteration 2: 18071.053 ops/s
[info] # Warmup Iteration 3: 18288.683 ops/s
[info] # Warmup Iteration 4: 18162.275 ops/s
[info] # Warmup Iteration 5: 18162.073 ops/s
[info] Iteration 1: 18265.766 ops/s
[info] Iteration 2: 18112.100 ops/s
[info] Iteration 3: 18360.725 ops/s
[info] Iteration 4: 18258.664 ops/s
[info] Iteration 5: 18345.359 ops/s
[info] # Run progress: 68.75% complete, ETA 00:01:42
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 16266.308 ops/s
[info] # Warmup Iteration 2: 18147.553 ops/s
[info] # Warmup Iteration 3: 18369.462 ops/s
[info] # Warmup Iteration 4: 18216.634 ops/s
[info] # Warmup Iteration 5: 18188.207 ops/s
[info] Iteration 1: 18384.933 ops/s
[info] Iteration 2: 18270.939 ops/s
[info] Iteration 3: 18494.194 ops/s
[info] Iteration 4: 18303.357 ops/s
[info] Iteration 5: 18257.426 ops/s
[info] Result "benchmarks.AppendBenchmark.bench01_append":
[info] 17905.101 ±(99.9%) 369.369 ops/s [Average]
[info] (min, avg, max) = (17310.880, 17905.101, 18494.194), stdev = 425.366
[info] CI (99.9%): [17535.732, 18274.470] (assumes normal distribution)
[info] # JMH version: 1.36
[info] # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10
[info] # VM invoker: /private/var/root/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 5 iterations, 2 s each
[info] # Measurement: 5 iterations, 2 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: benchmarks.AppendBenchmark.bench01_append
[info] # Parameters: (seqType = Array)
[info] # Run progress: 75.00% complete, ETA 00:01:21
[info] # Fork: 1 of 4
[info] # Warmup Iteration 1: 277270.258 ops/s
[info] # Warmup Iteration 2: 333449.170 ops/s
[info] # Warmup Iteration 3: 331790.975 ops/s
[info] # Warmup Iteration 4: 332878.299 ops/s
[info] # Warmup Iteration 5: 329429.787 ops/s
[info] Iteration 1: 331081.459 ops/s
[info] Iteration 2: 333868.462 ops/s
[info] Iteration 3: 334945.438 ops/s
[info] Iteration 4: 331814.290 ops/s
[info] Iteration 5: 333273.386 ops/s
[info] # Run progress: 81.25% complete, ETA 00:01:01
[info] # Fork: 2 of 4
[info] # Warmup Iteration 1: 285724.464 ops/s
[info] # Warmup Iteration 2: 332229.089 ops/s
[info] # Warmup Iteration 3: 334550.466 ops/s
[info] # Warmup Iteration 4: 331105.929 ops/s
[info] # Warmup Iteration 5: 331058.216 ops/s
[info] Iteration 1: 330723.120 ops/s
[info] Iteration 2: 329277.386 ops/s
[info] Iteration 3: 333763.126 ops/s
[info] Iteration 4: 332543.489 ops/s
[info] Iteration 5: 332450.099 ops/s
[info] # Run progress: 87.50% complete, ETA 00:00:40
[info] # Fork: 3 of 4
[info] # Warmup Iteration 1: 282792.484 ops/s
[info] # Warmup Iteration 2: 332264.581 ops/s
[info] # Warmup Iteration 3: 333864.657 ops/s
[info] # Warmup Iteration 4: 335450.388 ops/s
[info] # Warmup Iteration 5: 327403.770 ops/s
[info] Iteration 1: 328751.305 ops/s
[info] Iteration 2: 333965.071 ops/s
[info] Iteration 3: 331672.666 ops/s
[info] Iteration 4: 330989.878 ops/s
[info] Iteration 5: 331698.674 ops/s
[info] # Run progress: 93.75% complete, ETA 00:00:20
[info] # Fork: 4 of 4
[info] # Warmup Iteration 1: 281533.742 ops/s
[info] # Warmup Iteration 2: 325284.652 ops/s
[info] # Warmup Iteration 3: 334341.357 ops/s
[info] # Warmup Iteration 4: 305619.356 ops/s
[info] # Warmup Iteration 5: 324531.639 ops/s
[info] Iteration 1: 329579.506 ops/s
[info] Iteration 2: 332412.783 ops/s
[info] Iteration 3: 333842.392 ops/s
[info] Iteration 4: 333166.228 ops/s
[info] Iteration 5: 331942.273 ops/s
[info] Result "benchmarks.AppendBenchmark.bench01_append":
[info] 332088.052 ±(99.9%) 1458.243 ops/s [Average]
[info] (min, avg, max) = (328751.305, 332088.052, 334945.438), stdev = 1679.315
[info] CI (99.9%): [330629.809, 333546.294] (assumes normal distribution)
[info] # Run complete. Total time: 00:05:26
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
[info] extra caution when trusting the results, look into the generated code to check the benchmark still
[info] works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
[info] different JVMs are already problematic, the performance difference caused by different Blackhole
[info] modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
[info] Benchmark (seqType) Mode Cnt Score Error Units
[info] AppendBenchmark.bench00_prepend List thrpt 20 483217148.667 ± 4724403.219 ops/s
[info] AppendBenchmark.bench00_prepend Array thrpt 20 272675.051 ± 1553.827 ops/s
[info] AppendBenchmark.bench01_append List thrpt 20 17905.101 ± 369.369 ops/s
[info] AppendBenchmark.bench01_append Array thrpt 20 332088.052 ± 1458.243 ops/s
[info] Benchmark result is saved to benchmarks/results/01-AppendBenchmark-openjdk-1.json
[success] Total time: 333 s (05:33), completed Mar 4, 2023, 11:13:10 PM