Scoring System

The primary purpose of the Scalionix scoring system is to convert heterogeneous benchmark measurements into a single normalized performance model while preserving the characteristics of each individual workload.

Different benchmark categories naturally produce different measurement units.

For example:

• JSON measures bytes processed per second.
• Hashing measures bytes processed per second.
• Encryption measures bytes processed per second.
• Compression measures bytes processed per second.
• Collections measure operations per second.
• Concurrency measures synchronization operations per second.
• Compilation measures completed software builds per second.
• Storage measures bytes per second, files per second, or filesystem operations per second depending on the workload.

Because these measurements cannot be compared directly, every workload is normalized against deterministic reference values.

The resulting normalized scores allow workloads with fundamentally different units to participate in a common scoring model while preserving their relative performance.

Design Goals

The scoring system has several primary design goals.

• Platform independence.
• Deterministic score generation.
• Stable long-term comparisons.
• Hardware-independent normalization.
• Future extensibility.
• Resistance to measurement noise.

Every benchmark category follows exactly the same scoring pipeline regardless of the underlying workload.

Scoring Pipeline

Every benchmark workload passes through the same normalization process.

            Raw Benchmark Result
                     │
                     ▼
             Reference Matching
                     │
                     ▼
            Throughput Normalization
                     │
                     ▼
             Single / Multi Score
                     │
                     ▼
             Scaling Score
                     │
                     ▼
             Weighted Combination
                     │
                     ▼
             Workload Score
                     │
                     ▼
             Scenario Score
                     │
                     ▼
             Category Score
                     │
                     ▼
             Compute / Storage Score

Every stage performs a specific transformation while preserving deterministic benchmark behavior.

Reference Matching

Every workload produced by the benchmark is matched against exactly one predefined reference entry.

A reference uniquely identifies a benchmark workload using multiple dimensions.

These dimensions include:

• Benchmark category
• Scenario
• Dataset size
• Workload variant

Additional dimensions may be introduced in future scoring versions without affecting existing benchmark reports.

Only successfully matched workloads participate in score calculation.

Unmatched workloads are reported separately through benchmark diagnostics.

Normalization

After a matching reference has been located, the measured throughput is normalized against the reference throughput.

The benchmark supports two normalization directions.

Higher Is Better

Used by workloads where larger throughput indicates better performance.

Examples include:

• Serialization
• Hashing
• Encryption
• Compilation
• Storage throughput

Lower Is Better

Used by workloads where lower execution time indicates better performance.

Although the current benchmark primarily uses throughput-based measurements, the scoring engine supports both normalization models.

Reference Platform

All benchmark references are calibrated against a predefined reference workstation.

The current reference platform consists of:

Processor

AMD Ryzen 9 7950X3D

Memory

64 GB Kingston Fury Renegade DDR5-6400 CL32

Storage

Samsung 9100 Pro 1 TB PCIe Gen5 NVMe SSD

Reference values are calibrated so that the reference workstation produces approximately ten thousand points across benchmark categories.

This calibration simplifies long-term interpretation while preserving proportional score scaling.

Future reference platforms may be introduced through new scoring versions without invalidating historical benchmark reports.

Workload Scores

Every successfully executed workload produces up to three normalized scores.

Single-thread Score

Measures workload performance using one execution worker.

Multi-thread Score

Measures the highest throughput achieved across all evaluated worker configurations.

Scaling Score

Measures parallel execution efficiency relative to single-thread execution.

These three values describe different aspects of workload behavior.

Some benchmark categories intentionally omit one or more of these values when they are not applicable.

Compilation, for example, evaluates only native compiler parallelism and therefore contributes exclusively through its Multi-thread Score.

Weighted Combination

Workload scores are combined using weighted geometric averaging.

The default scoring weights are:

Single-thread

35%

Multi-thread

60%

Scaling

5%

This weighting intentionally prioritizes sustained production throughput while still rewarding efficient single-thread execution and parallel scalability.

Using a geometric mean instead of an arithmetic mean prevents one exceptionally high metric from completely masking weaknesses in another metric.

As a result, balanced hardware configurations receive more representative overall scores.

Scenario Scores

Many benchmark scenarios execute multiple workload configurations.

Examples include:

• Multiple dataset sizes
• Different block sizes
• Multiple object counts
• Compression variants
• Durability modes

All workloads belonging to the same scenario are aggregated into a single Scenario Score.

This aggregation produces stable scenario-level results while reducing sensitivity to individual workload variance.

Category Scores

Scenario scores are aggregated into Category Scores.

Each category therefore produces one deterministic normalized score representing the practical performance of that benchmark domain.

Category scores also include detailed diagnostics describing:

• Executed workloads
• Matched references
• Rejected workloads
• Missing references
• Successfully scored workloads

These diagnostics simplify benchmark validation and future reference calibration.

Compute Score

All compute-oriented benchmark categories contribute to the Compute Score.

These currently include:

• JSON
• Hashing
• Encryption
• Compression
• Collections
• Concurrency
• Compilation

Each category contributes using predefined category weights.

The resulting Compute Score represents the computational capability of the evaluated system independently from storage performance.

Storage Score

Storage workloads contribute exclusively to the Storage Score.

Storage intentionally remains independent from Compute because storage technology scales fundamentally differently from processor performance.

This separation preserves meaningful long-term comparisons across hardware generations.

The rationale behind this architectural decision was discussed in Chapter 6.

Scoring Versioning

Every benchmark report includes a scoring version.

Scoring versions define:

• Reference values
• Workload calibration
• Category aggregation
• Normalization behavior

Historical benchmark reports remain fully reproducible because the scoring version is stored together with every generated benchmark report.

Future scoring revisions therefore do not invalidate historical benchmark data.

Deterministic Results

The scoring system has been designed to produce deterministic results.

Given identical benchmark execution and identical scoring references, the generated scores will always be identical.

This property is essential for:

• Historical comparisons
• Regression analysis
• Hardware validation
• Public benchmark rankings
• Long-term performance tracking

Design Summary

The Scalionix scoring system transforms heterogeneous benchmark measurements into a unified performance model through deterministic reference normalization.

By combining workload normalization, scenario aggregation, category scoring, weighted geometric averaging, and versioned reference calibration, the scoring engine produces stable, reproducible, and meaningful benchmark scores across a wide variety of hardware platforms.

The resulting Compute Score and Storage Score provide independent yet complementary views of modern system performance, allowing accurate comparison of computational capability and storage subsystem efficiency without conflating fundamentally different performance domains.