Scoring Report

The Scoring Report contains the complete output of the benchmark scoring engine.

Unlike the Full Benchmark Report, which preserves raw execution results, the Scoring Report contains normalized benchmark scores generated from those execution results.

This report represents the complete scoring state of a benchmark execution.

Every workload score, scenario score, category score, normalization result, reference match, and scoring diagnostic is preserved.

The Scoring Report serves as the primary document for benchmark analysis and score validation.

Purpose

The Scoring Report has several primary objectives.

• Convert raw benchmark measurements into normalized scores.
• Preserve workload-level scoring information.
• Record benchmark diagnostics.
• Preserve reference matching information.
• Enable future score validation.

Unlike the Global Score Report, this report contains every intermediate scoring result.

Report Generation

The report is generated immediately after benchmark execution completes.

The scoring engine processes every successfully verified benchmark workload.

Each workload is matched against its corresponding reference before normalized scores are calculated.

Once all workloads have been processed, the scoring engine generates:

• Workload scores
• Scenario scores
• Category scores
• Compute Score
• Storage Score
• Scoring diagnostics

The resulting document completely describes how every benchmark score was calculated.

Report Structure

The report consists of several logical sections.

General Information

Contains benchmark metadata.

Compute Score

Overall compute performance.

Storage Score

Independent storage performance.

Benchmark Categories

Normalized scores for every benchmark category.

Scenario Scores

Aggregated scenario-level scores.

Workload Scores

Individual normalized benchmark workloads.

Diagnostics

Reference matching and scoring statistics.

Together these sections describe the complete scoring process.

Compute Score

The Compute Score represents the aggregated performance of all compute-oriented benchmark categories.

Current compute categories include:

• JSON
• Hashing
• Encryption
• Compression
• Collections
• Concurrency
• Compilation

Every category contributes according to its configured category weight.

Category aggregation is performed using the weighted geometric mean described in Chapter 7.

The Compute Score intentionally excludes every Storage workload.

Storage Score

Storage workloads are aggregated independently from Compute.

Storage Score includes:

• Sequential I/O
• Random I/O
• Small Files
• Metadata
• Directory Walk
• Synchronization

The Storage Score represents the overall practical efficiency of the storage subsystem.

Its calculation follows the same normalization rules as compute workloads while remaining completely independent from Compute Score.

Category Scores

Each benchmark category produces one normalized category score.

A category contains one or more benchmark scenarios.

Examples include:

            JSON
             │
             ▼
            Serialization
             │
             ▼
            Workloads

            Storage
             │
             ▼
            Sequential Read
             │
             ▼
            Workloads

            Compilation
             │
             ▼
            Typst Release
             │
             ▼
            Workload

Category scores therefore represent aggregated performance across all benchmark scenarios belonging to the same benchmark domain.

Scenario Scores

Every benchmark scenario contains one or more benchmark workloads.

Examples include:

JSON Serialization

• 64 KiB
• 256 KiB
• 1 MiB
• Multiple object counts

Storage Random Read

• Multiple dataset sizes
• Multiple block sizes

Compilation Typst

• Production build

Scenario aggregation combines these workload scores into one representative Scenario Score.

This reduces benchmark noise while preserving workload diversity.

Workload Scores

Workloads represent the smallest scoring unit.

Every workload records:

• Benchmark parameters
• Matched reference
• Measured throughput
• Normalized throughput
• Single-thread Score
• Multi-thread Score
• Scaling Score
• Combined Score
• Best worker configuration

These values represent the complete normalization result for one benchmark workload.

Diagnostics

Every benchmark category records detailed diagnostics.

Diagnostics include:

Input Samples

Number of benchmark measurements received.

Workload Groups

Number of workload groups created before scoring.

Matched References

Successfully matched reference entries.

Missing References

Workloads that could not be matched.

Successfully Scored Workloads

Workloads contributing to the final score.

Rejected Workloads

Measurements rejected by the scoring engine.

Missing Reference Keys

Complete identifiers describing unmatched workloads.

Rejected Workload Keys

Identifiers of workloads intentionally rejected.

These diagnostics greatly simplify benchmark maintenance and future reference calibration.

Reference Matching

Before a workload can be scored, the scoring engine must locate an appropriate reference.

Matching currently considers multiple workload dimensions.

Typical matching dimensions include:

• Benchmark category
• Scenario name
• Workload variant
• Dataset size

Additional dimensions may be introduced in future scoring versions without affecting historical benchmark reports.

Workloads without matching references are excluded from score calculation while remaining visible through diagnostics.

Worker Analysis

For workloads supporting multiple execution workers, the scoring engine records the best-performing worker configuration.

Examples include:

[16w → 9]

indicating that all nine evaluated workloads achieved their maximum throughput using sixteen execution workers.

This information provides valuable insight into scalability characteristics of the evaluated hardware.

Category Diagnostics

Every category maintains completely independent diagnostics.

This allows benchmark developers to identify problems affecting one category without influencing benchmark analysis elsewhere.

Examples include:

• Missing references affecting only Storage.
• Rejected workloads affecting only Compression.
• Verification failures affecting only Compilation.

Independent diagnostics significantly simplify benchmark evolution as additional benchmark categories are introduced.

Relationship to Other Reports

The Scoring Report is generated directly from the Full Benchmark Report.

The relationship between reports is illustrated below.

            Full Benchmark Report
                       │
                       ▼
                 Scoring Engine
                       │
                       ▼
                 Scoring Report
                       │
                       ├────────► Global Score Report
                       │
                       └────────► Benchmark Comparison

Unlike the Full Benchmark Report, the Scoring Report contains only normalized benchmark information.

Practical Applications

The Scoring Report is intended primarily for:

• Benchmark developers
• Hardware reviewers
• Benchmark validation
• Reference calibration
• Performance analysis
• Regression testing
• Future scoring revisions

It represents the most detailed description of benchmark scoring available within the Scalionix System Benchmark.

Long-Term Compatibility

The report has been designed for long-term compatibility.

Future benchmark releases may introduce:

• Additional benchmark categories
• Additional workload parameters
• New diagnostics
• Revised scoring models

Because every report includes explicit scoring version information, historical benchmark reports remain reproducible.

Design Summary

The Scoring Report represents the complete output of the benchmark normalization engine.

It records workload scores, scenario scores, category scores, Compute Score, Storage Score, diagnostics, reference matching, and benchmark aggregation in a deterministic and versioned format.

Together with the Full Benchmark Report, it provides a complete description of benchmark execution and score generation, allowing future score recalculation, regression analysis, benchmark validation, and long-term performance tracking.