Repository navigation
Expand file tree
/
Copy pathbenchmark.txt
More file actions
289 lines (277 loc) · 20.6 KB
/
Copy pathbenchmark.txt
File metadata and controls
289 lines (277 loc) · 20.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< dev.openfeature:sdk >-------------------------
[INFO] Building OpenFeature Java SDK 1.23.0
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ sdk ---
[INFO] Deleting /home/local_toddbaert/git/java-sdk/target
[INFO]
[INFO] --- flatten:1.8.0:clean (flatten.clean) @ sdk ---
[INFO] Deleting /home/local_toddbaert/git/java-sdk/.flattened-pom.xml
[INFO]
[INFO] --- toolchains:3.3.0:select-jdk-toolchain (default) @ sdk ---
[INFO] Not using an external toolchain as the current JDK 'JDK[/usr/lib/jvm/java-21-openjdk-amd64]' matches the requirements.
[INFO]
[INFO] --- resources:3.5.0:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/local_toddbaert/git/java-sdk/src/main/resources
[INFO]
[INFO] --- flatten:1.8.0:flatten (flatten) @ sdk ---
[INFO] Generating flattened POM of project dev.openfeature:sdk:jar:1.23.0...
[INFO]
[INFO] --- resources:3.5.0:copy-resources (copy-license) @ sdk ---
[INFO] Copying 1 resource from . to target/classes/META-INF
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ sdk ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 91 source files with javac [debug release 11] to target/classes
[INFO] Annotation processing is enabled because one or more processors were found
on the class path. A future release of javac may disable annotation processing
unless at least one processor is specified by name (-processor), or a search
path is specified (--processor-path, --processor-module-path), or annotation
processing is enabled explicitly (-proc:only, -proc:full).
Use -Xlint:-options to suppress this message.
Use -proc:none to disable annotation processing.
[INFO] /home/local_toddbaert/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Some input files use or override a deprecated API.
[INFO] /home/local_toddbaert/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/local_toddbaert/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Some input files use unchecked or unsafe operations.
[INFO] /home/local_toddbaert/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- toolchains:3.3.0:select-jdk-toolchain (default) @ sdk ---
[INFO] Not using an external toolchain as the current JDK 'JDK[/usr/lib/jvm/java-21-openjdk-amd64]' matches the requirements.
[INFO]
[INFO] --- resources:3.5.0:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/local_toddbaert/git/java-sdk/src/main/resources
[INFO]
[INFO] --- flatten:1.8.0:flatten (flatten) @ sdk ---
[INFO] Generating flattened POM of project dev.openfeature:sdk:jar:1.23.0...
[INFO]
[INFO] --- resources:3.5.0:copy-resources (copy-license) @ sdk ---
[INFO] Copying 1 resource from . to target/classes/META-INF
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ sdk ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.5.0:testResources (default-testResources) @ sdk ---
[INFO] Copying 2 resources from src/test/resources to target/test-classes
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ sdk ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 101 source files with javac [debug release 11] to target/test-classes
[INFO] Annotation processing is enabled because one or more processors were found
on the class path. A future release of javac may disable annotation processing
unless at least one processor is specified by name (-processor), or a search
path is specified (--processor-path, --processor-module-path), or annotation
processing is enabled explicitly (-proc:only, -proc:full).
Use -Xlint:-options to suppress this message.
Use -proc:none to disable annotation processing.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/DomainScopedProviderSpecTest.java: Some input files use or override a deprecated API.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/DomainScopedProviderSpecTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/HookSpecTest.java: Some input files use unchecked or unsafe operations.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/HookSpecTest.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] >>> jmh:0.2.2:benchmark (default-cli) > process-test-resources @ sdk >>>
[INFO]
[INFO] --- toolchains:3.3.0:select-jdk-toolchain (default) @ sdk ---
[INFO] Not using an external toolchain as the current JDK 'JDK[/usr/lib/jvm/java-21-openjdk-amd64]' matches the requirements.
[INFO]
[INFO] --- resources:3.5.0:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/local_toddbaert/git/java-sdk/src/main/resources
[INFO]
[INFO] --- flatten:1.8.0:flatten (flatten) @ sdk ---
[INFO] Generating flattened POM of project dev.openfeature:sdk:jar:1.23.0...
[INFO]
[INFO] --- resources:3.5.0:copy-resources (copy-license) @ sdk ---
[INFO] Copying 1 resource from . to target/classes/META-INF
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ sdk ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.5.0:testResources (default-testResources) @ sdk ---
[INFO] Copying 2 resources from src/test/resources to target/test-classes
[INFO]
[INFO] <<< jmh:0.2.2:benchmark (default-cli) < process-test-resources @ sdk <<<
[INFO]
[INFO]
[INFO] --- jmh:0.2.2:benchmark (default-cli) @ sdk ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 101 source files to /home/local_toddbaert/git/java-sdk/target/test-classes
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/TelemetryTest.java: Some input files use or override a deprecated API.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/TelemetryTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/multiprovider/BaseStrategyTest.java: Some input files use unchecked or unsafe operations.
[INFO] /home/local_toddbaert/git/java-sdk/src/test/java/dev/openfeature/sdk/multiprovider/BaseStrategyTest.java: Recompile with -Xlint:unchecked for details.
[INFO] Executing the JMH benchmarks
# JMH version: 1.37
# VM version: JDK 21.0.12.1, OpenJDK 64-Bit Server VM, 21.0.12.1+1-1-24.04.4-Ubuntu
# VM invoker: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
# VM options: -Xmx1024m -XX:+UnlockExperimentalVMOptions -XX:+UseEpsilonGC
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: <none>
# Measurement: 1 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: dev.openfeature.sdk.benchmark.AllocationBenchmark.run
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
Iteration 1: Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build as described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org.mockito/org/mockito/Mockito.html#0.3
WARNING: A Java agent has been loaded dynamically (/home/local_toddbaert/.m2/repository/net/bytebuddy/byte-buddy-agent/1.18.14-jdk5/byte-buddy-agent-1.18.14-jdk5.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
num #instances #bytes class name (module)
-------------------------------------------------------
1: 175386 37568880 [B (java.base@21.0.12.1)
2: 77126 4105488 [Ljava.lang.Object; (java.base@21.0.12.1)
3: 164 3424000 [Ljdk.internal.vm.FillerElement; (java.base@21.0.12.1)
4: 113801 2731224 java.util.ArrayList (java.base@21.0.12.1)
5: 50000 2400000 dev.openfeature.sdk.LayeredEvaluationContext
6: 93931 2254344 java.lang.String (java.base@21.0.12.1)
7: 50000 2000000 dev.openfeature.sdk.FlagEvaluationDetails
8: 50000 2000000 dev.openfeature.sdk.ProviderEvaluation
9: 50000 1600000 dev.openfeature.sdk.HookSupportData
10: 50000 1600000 dev.openfeature.sdk.SharedHookContext
11: 21146 1300632 [I (java.base@21.0.12.1)
12: 39904 1276928 java.util.ArrayList$Itr (java.base@21.0.12.1)
13: 50004 1200096 dev.openfeature.sdk.ImmutableContext
14: 50000 1200000 dev.openfeature.sdk.HookContext
15: 50000 1200000 dev.openfeature.sdk.Pair
16: 20479 1146824 jdk.internal.org.objectweb.asm.SymbolTable$Entry (java.base@21.0.12.1)
17: 50014 800224 java.util.Optional (java.base@21.0.12.1)
18: 50004 800064 dev.openfeature.sdk.NoOpProvider$$Lambda/0x00007871b802f550
19: 50000 800000 dev.openfeature.sdk.DefaultHookData
20: 19335 773400 dev.openfeature.sdk.ProviderEvaluation$ProviderEvaluationBuilderImpl
21: 12079 579792 java.util.HashMap (java.base@21.0.12.1)
22: 23844 572256 net.bytebuddy.description.type.TypeDescription$Generic$OfNonGenericType$ForLoadedType
23: 23639 567336 java.lang.StringBuilder (java.base@21.0.12.1)
24: 366 556080 [C (java.base@21.0.12.1)
25: 4343 521720 java.lang.Class (java.base@21.0.12.1)
26: 11896 475840 net.bytebuddy.description.type.TypeDescription$ForLoadedType
27: 1570 430760 [J (java.base@21.0.12.1)
28: 13261 424352 java.util.AbstractList$Itr (java.base@21.0.12.1)
29: 4745 417560 java.lang.reflect.Method (java.base@21.0.12.1)
30: 6511 416704 java.net.URL (java.base@21.0.12.1)
31: 6888 385728 jdk.internal.org.objectweb.asm.Label (java.base@21.0.12.1)
32: 584 372928 [Ljdk.internal.org.objectweb.asm.SymbolTable$Entry; (java.base@21.0.12.1)
33: 12890 338976 [Ljava.lang.Class; (java.base@21.0.12.1)
34: 9810 313920 net.bytebuddy.description.method.MethodDescription$ForLoadedMethod
35: 9726 311232 java.util.concurrent.ConcurrentHashMap$Node (java.base@21.0.12.1)
36: 5594 268512 net.bytebuddy.description.method.ParameterDescription$Latent
37: 1193 267232 jdk.internal.org.objectweb.asm.MethodWriter (java.base@21.0.12.1)
38: 7606 243392 java.util.concurrent.ConcurrentLinkedQueue$Itr (java.base@21.0.12.1)
39: 10003 240072 java.lang.Double (java.base@21.0.12.1)
40: 7487 239584 java.util.HashMap$Node (java.base@21.0.12.1)
41: 2561 223192 [Ljava.util.HashMap$Node; (java.base@21.0.12.1)
42: 8680 208320 net.bytebuddy.description.type.TypeDescription$Generic$Visitor$Substitutor$ForAttachment
43: 4206 201888 jdk.internal.org.objectweb.asm.Frame (java.base@21.0.12.1)
44: 2513 180936 java.lang.reflect.Field (java.base@21.0.12.1)
45: 5604 179328 java.util.AbstractList$SubList (java.base@21.0.12.1)
46: 5436 173952 java.io.File (java.base@21.0.12.1)
47: 1600 166400 java.util.jar.JarFile$JarFileEntry (java.base@21.0.12.1)
48: 6923 166152 net.bytebuddy.description.ByteCodeElement$Token$TokenList
49: 10006 160096 dev.openfeature.sdk.Value
50: 10004 160064 dev.openfeature.sdk.ImmutableStructure
51: 6262 150288 net.bytebuddy.description.annotation.AnnotationList$Explicit
52: 3682 147280 java.lang.invoke.MethodType (java.base@21.0.12.1)
53: 5424 130176 net.bytebuddy.implementation.bytecode.StackManipulation$Size
54: 186 126112 [Ljava.util.concurrent.ConcurrentHashMap$Node; (java.base@21.0.12.1)
55: 2613 125424 java.lang.invoke.MemberName (java.base@21.0.12.1)
56: 1537 121216 [S (java.base@21.0.12.1)
57: 2938 117520 java.util.LinkedHashMap$Entry (java.base@21.0.12.1)
58: 4575 109800 net.bytebuddy.description.type.TypeList$Generic$Explicit
59: 5808 92928 jdk.internal.util.StrongReferenceKey (java.base@21.0.12.1)
60: 2975 91480 [Ljava.lang.String; (java.base@21.0.12.1)
61: 1602 89712 java.util.zip.ZipFile$ZipFileInputStream (java.base@21.0.12.1)
62: 1809 86832 jdk.internal.ref.CleanerImpl$PhantomCleanableRef (java.base@21.0.12.1)
63: 2710 86720 net.bytebuddy.description.method.ParameterDescription$ForLoadedParameter$OfMethod
64: 3533 84792 net.bytebuddy.description.type.TypeDescription$Generic$LazyProjection$ForLoadedReturnType
65: 2565 82080 net.bytebuddy.description.type.TypeDescription$Generic$LazyProjection$OfMethodParameter
66: 3363 80712 net.bytebuddy.description.method.ParameterList$ForTokens
67: 1602 76896 java.util.zip.ZipFile$ZipFileInflaterInputStream (java.base@21.0.12.1)
68: 3067 73608 java.util.Arrays$ArrayList (java.base@21.0.12.1)
69: 1796 71840 jdk.internal.org.objectweb.asm.Handler (java.base@21.0.12.1)
70: 2970 71280 jdk.internal.org.objectweb.asm.ByteVector (java.base@21.0.12.1)
71: 1686 67440 java.util.WeakHashMap$Entry (java.base@21.0.12.1)
72: 2755 66120 jdk.internal.org.objectweb.asm.Edge (java.base@21.0.12.1)
73: 1151 64456 net.bytebuddy.jar.asm.Label
74: 1600 64000 java.security.CodeSource (java.base@21.0.12.1)
75: 3734 59744 java.lang.Object (java.base@21.0.12.1)
76: 1490 59600 jdk.internal.loader.URLClassPath$JarLoader$2 (java.base@21.0.12.1)
77: 1832 58624 java.util.AbstractList$ListItr (java.base@21.0.12.1)
78: 1047 52136 [Ljava.lang.invoke.LambdaForm$Name; (java.base@21.0.12.1)
79: 788 50432 jdk.internal.org.objectweb.asm.FieldWriter (java.base@21.0.12.1)
80: 748 47872 java.util.LinkedHashMap (java.base@21.0.12.1)
81: 1917 46008 net.bytebuddy.description.type.TypeList$Explicit
82: 375 45656 [Lnet.bytebuddy.jar.asm.Label;
83: 1068 42720 java.io.ObjectStreamField (java.base@21.0.12.1)
84: 1323 42336 java.lang.invoke.LambdaForm$Name (java.base@21.0.12.1)
85: 1321 42272 java.util.TreeMap$ValueIterator (java.base@21.0.12.1)
86: 1740 41760 java.util.Arrays$ArrayItr (java.base@21.0.12.1)
87: 1729 41496 java.lang.PublicMethods$MethodList (java.base@21.0.12.1)
88: 719 40264 net.bytebuddy.jar.asm.SymbolTable$Entry
89: 1249 39968 java.util.HashMap$KeyIterator (java.base@21.0.12.1)
90: 1207 38624 jdk.internal.org.objectweb.asm.Type (java.base@21.0.12.1)
91: 1602 38448 java.util.zip.ZipFile$InflaterCleanupAction (java.base@21.0.12.1)
92: 245 37240 jdk.internal.org.objectweb.asm.ClassWriter (java.base@21.0.12.1)
93: 1100 35200 jdk.internal.util.WeakReferenceKey (java.base@21.0.12.1)
94: 2181 34896 net.bytebuddy.description.type.TypeDescription$Generic$AnnotationReader$Delegator$ForLoadedMethodReturnType
95: 153 34272 net.bytebuddy.jar.asm.MethodWriter
96: 1171 33144 [Ljava.lang.reflect.Parameter; (java.base@21.0.12.1)
97: 559 28504 [Ljava.lang.reflect.Method; (java.base@21.0.12.1)
98: 509 28504 net.bytebuddy.description.method.MethodDescription$Latent
99: 368 26496 java.lang.reflect.Constructor (java.base@21.0.12.1)
100: 299 26312 net.bytebuddy.jar.asm.Context
101: 1564 25024 java.security.SecureClassLoader$1 (java.base@21.0.12.1)
102: 1564 25024 java.security.SecureClassLoader$CodeSourceKey (java.base@21.0.12.1)
103: 623 24920 java.lang.NoSuchFieldException (java.base@21.0.12.1)
104: 1482 23712 java.util.zip.CRC32 (java.base@21.0.12.1)
105: 308 23408 [Ljava.lang.ref.SoftReference; (java.base@21.0.12.1)
106: 963 23112 java.util.Collections$1 (java.base@21.0.12.1)
107: 354 22656 java.lang.Class$ReflectionData (java.base@21.0.12.1)
108: 1036 22176 [Ljdk.internal.org.objectweb.asm.Type; (java.base@21.0.12.1)
109: 456 21888 jdk.internal.org.objectweb.asm.AnnotationWriter (java.base@21.0.12.1)
110: 545 21800 java.util.StringJoiner (java.base@21.0.12.1)
111: 544 21760 java.lang.reflect.Parameter (java.base@21.0.12.1)
112: 853 21688 [Lnet.bytebuddy.implementation.bytecode.StackManipulation;
113: 535 21400 java.lang.ref.SoftReference (java.base@21.0.12.1)
114: 667 21344 net.bytebuddy.jar.asm.Type
115: 645 20640 java.util.HashMap$EntryIterator (java.base@21.0.12.1)
116: 642 20544 net.bytebuddy.description.method.ParameterList$ForLoadedExecutable$OfMethod
117: 852 20448 java.lang.invoke.ResolvedMethodName (java.base@21.0.12.1)
118: 848 20352 net.bytebuddy.description.method.MethodDescription$TypeToken
truncated...
Total 1512944 85381512
0.562 s/op
+totalAllocatedBytes: 85381512.000 bytes
+totalAllocatedInstances: 1512944.000 instances
+totalHeap: 1044512768.000 bytes
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalAllocatedBytes":
85381512.000 bytes
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalAllocatedInstances":
1512944.000 instances
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalHeap":
1044512768.000 bytes
# Run complete. Total time: 00:00:00
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
extra caution when trusting the results, look into the generated code to check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
different JVMs are already problematic, the performance difference caused by different Blackhole
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
Benchmark Mode Cnt Score Error Units
AllocationBenchmark.run ss 0.562 s/op
AllocationBenchmark.run:+totalAllocatedBytes ss 85381512.000 bytes
AllocationBenchmark.run:+totalAllocatedInstances ss 1512944.000 instances
AllocationBenchmark.run:+totalHeap ss 1044512768.000 bytes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.682 s
[INFO] Finished at: 2026-10-08T12:50:39-04:00
[INFO] ------------------------------------------------------------------------