Storage Benchmark

The Storage benchmark evaluates the practical performance of the complete storage subsystem rather than the theoretical bandwidth of a storage device.

Modern software interacts with persistent storage in many different ways. Some applications continuously stream large files, while others perform millions of metadata operations, create thousands of small files, execute random reads, synchronize write buffers, or recursively traverse complex directory trees.

Each of these workloads stresses a different part of the storage stack.

Unlike traditional storage benchmarks that primarily report sequential bandwidth or synthetic IOPS, the Scalionix Storage benchmark evaluates a diverse collection of realistic filesystem workloads representative of modern software engineering, backend infrastructure, databases, package managers, build systems, and distributed storage platforms.

The resulting Storage Score therefore represents the practical efficiency of the complete storage subsystem rather than the maximum performance of the storage hardware alone.

Objectives

The Storage benchmark has six primary objectives.

• Measure sustained sequential throughput.
• Measure random access performance.
• Measure filesystem metadata operations.
• Measure small file performance.
• Measure durability synchronization overhead.
• Measure filesystem traversal performance.

Together these workloads provide a comprehensive evaluation of practical storage behavior encountered during everyday software execution.

Why Storage?

Persistent storage remains one of the most important components of every computer system.

Nearly every application continuously performs storage operations.

Typical examples include:

• Software compilation
• Package managers
• Source code repositories
• Databases
• Object storage
• Backup software
• Artifact repositories
• Logging systems
• Telemetry collection
• Software installation

Although storage devices continue to increase in bandwidth, application performance frequently depends more on filesystem behavior than raw transfer speed.

Consequently, measuring only sequential bandwidth provides an incomplete representation of practical storage performance.

Benchmark Philosophy

The benchmark intentionally evaluates complete filesystem operations instead of isolated device benchmarks.

Every workload interacts with the operating system filesystem exactly as production software would.

Typical execution consists of:

         Create benchmark dataset
                    │
                    ▼
           Filesystem operation
                    │
                    ▼
       Operating system interaction
                    │
                    ▼
        Filesystem synchronization
                    │
                    ▼
              Verification
                    │
                    ▼
         Performance measurement

This execution model allows the benchmark to measure the combined behavior of:

• Storage hardware
• Filesystem implementation
• Operating system
• Kernel page cache
• Storage drivers
• Scheduler

rather than measuring only the storage controller.

Typical Storage Pipeline

              Prepare Dataset
                    │
                    ▼
       Execute Filesystem Operation
                    │
                    ▼
          Operating System Layer
                    │
                    ▼
             Storage Subsystem
                    │
                    ▼
           Verify Correctness
                    │
                    ▼
          Throughput Measurement

Storage Workload Categories

The Storage benchmark consists of six independent workload groups.

Each group measures a different aspect of storage subsystem behavior.

Sequential I/O

Large continuous file transfers.

Random I/O

Small block access across large datasets.

Small Files

Creation, reading, and deletion of thousands of files.

Metadata

Filesystem metadata operations.

Synchronization

Durability and persistence operations.

Directory Traversal

Recursive filesystem enumeration.

Each workload category contributes independently to the final Storage Score.

Sequential I/O

Sequential workloads measure continuous streaming performance.

Scenarios

storage.sequential.read
storage.sequential.write

The benchmark evaluates multiple dataset sizes together with multiple block sizes.

Sequential workloads primarily measure:

• Sustained storage throughput
• Filesystem buffering
• Page cache efficiency
• Storage controller performance

Unlike synthetic benchmarks, verification is performed after every operation.

Random I/O

Random access workloads evaluate storage behavior when reading or writing independent blocks distributed across the dataset.

Scenarios

storage.random.read
storage.random.write

Multiple block sizes are evaluated.

Typical configurations include:

• 4 KiB
• 16 KiB
• 64 KiB

Random workloads primarily measure:

• Random access latency
• Storage controller scheduling
• Flash translation layer efficiency
• Filesystem allocation behavior

The resulting measurements are expressed as operations per second.

Small File Operations

Many real-world applications manipulate enormous numbers of very small files.

Examples include:

• Source code repositories
• Package managers
• Build systems
• Software installation
• Configuration management

The benchmark evaluates three scenarios.

storage.small_files.create
storage.small_files.read
storage.small_files.delete

Multiple file counts and file sizes are evaluated.

These workloads stress:

• Inode allocation
• Directory management
• Metadata caching
• Filesystem allocation algorithms

rather than storage bandwidth.

Metadata Operations

Metadata operations are among the most frequently executed filesystem requests.

The benchmark currently evaluates:

storage.metadata.stat
storage.metadata.rename

These workloads exercise filesystem metadata handling independently from file contents.

Typical applications include:

• Build systems
• Package managers
• Compilers
• Source control systems
• Backup software

Metadata benchmarks frequently reveal filesystem bottlenecks invisible to bandwidth-oriented benchmarks.

Directory Traversal

Directory traversal evaluates recursive filesystem enumeration.

Scenario

storage.directory.walk

The benchmark recursively visits deterministic directory trees while collecting filesystem metadata.

Typical production examples include:

• Compiler dependency discovery
• Backup indexing
• Antivirus scanning
• Filesystem synchronization
• Source repository indexing

Unlike other Storage scenarios, Directory Walk executes only as a single-threaded workload.

Filesystem traversal is inherently limited by directory hierarchy traversal and operating system behavior.

Consequently, the benchmark evaluates absolute traversal efficiency rather than artificial parallel execution.

Synchronization Workloads

Durability guarantees are essential for many software systems.

The benchmark therefore evaluates explicit synchronization operations.

Scenarios

storage.sync.fsync
storage.sync.write_fsync

Multiple block sizes are evaluated together with different durability modes.

Synchronization workloads primarily measure:

• Flush latency
• Filesystem durability overhead
• Storage controller behavior
• Operating system synchronization cost

These scenarios are particularly relevant for databases, transactional systems, and persistent logging.

Worker Configuration

Most Storage workloads evaluate multiple worker configurations.

Representative worker counts include:

[ 1 / 2 / 4 / 6 / 8 / 12 / 16 / 20 / 24 / 32 ]

Workers operate on independent datasets whenever possible.

Directory Walk is the only exception.

Because recursive filesystem traversal is fundamentally sequential, it executes exclusively using a single worker.

The benchmark intentionally avoids artificial parallelization that would not represent practical application behavior.

Verification

Every Storage workload performs correctness verification.

Verification includes:

• Successful file creation
• Successful reads
• Successful writes
• Metadata validation
• Filesystem consistency
• Directory traversal completion

Benchmark results are accepted only when all filesystem operations complete successfully.

Measured Metrics

Different workload groups naturally produce different performance metrics.

Sequential Workloads

• Bytes per second

Random Workloads

• Operations per second

Small File Workloads

• Files per second

Metadata Workloads

• Operations per second

Synchronization Workloads

• Durability operations per second

Directory Traversal

• Directory operations per second

The scoring adapter normalizes all metrics into the common scoring model.

Score Calculation

Every Storage workload produces three normalized values.

Single-thread Score

Measures filesystem performance using one execution worker.

Multi-thread Score

Measures maximum throughput achieved across all evaluated worker configurations.

Scaling Score

Measures parallel scalability relative to the single-thread execution.

Directory Walk represents the only exception.

Because the workload executes exclusively using a single worker, its Multi-thread Score is intentionally equal to its Single-thread Score.

Its Scaling Score is fixed at the baseline reference value.

This allows the scenario to integrate naturally into the common scoring model while preserving realistic execution behavior.

Scenario scores are aggregated into the Storage category score.

Hardware Characteristics Measured

The Storage benchmark simultaneously exercises multiple layers of the storage stack rather than measuring the storage device in isolation.

Unlike compute-oriented benchmarks, storage performance depends on the interaction between hardware, operating system services, filesystem implementation, and storage controller behavior.

Consequently, Storage Score reflects the combined efficiency of the complete storage subsystem.

The benchmark measures several hardware characteristics simultaneously.

Storage Device

Sequential throughput, random access latency, internal parallelism, controller efficiency, flash translation layer behavior, and media performance directly influence benchmark results.

Filesystem

Filesystem allocation algorithms, metadata organization, journaling implementation, directory indexing, and caching strategies significantly affect practical storage performance.

Operating System

Kernel buffering, page cache management, asynchronous writeback, request scheduling, and synchronization mechanisms contribute to observed throughput.

Memory Subsystem

Large sequential transfers rely heavily on sustained memory bandwidth, while metadata workloads frequently benefit from cache efficiency.

Processor

Although Storage is not intended as a CPU benchmark, filesystem operations still require substantial processor resources for address translation, metadata processing, allocation management, checksum generation, and synchronization.

Storage Driver

Driver implementation, queue management, interrupt handling, and command submission latency contribute directly to measured throughput.

Because all of these components participate simultaneously, Storage Score represents the practical performance experienced by real applications rather than theoretical device specifications.

Why Storage Is Evaluated Separately

One of the primary architectural decisions of Scalionix System Benchmark is the separation of Compute Score and Storage Score.

This decision was made intentionally after evaluating how modern hardware evolves.

Processor performance continues to improve substantially with each generation.

Increasing core counts, wider execution engines, larger caches, improved branch prediction, higher memory bandwidth, and architectural enhancements allow compute-oriented workloads to scale significantly over time.

Storage hardware follows a fundamentally different evolution.

Although modern NVMe devices continue to improve, their maximum practical performance is constrained by several physical limits.

These include:

• PCI Express bandwidth
• NAND flash characteristics
• Controller architecture
• Storage protocols
• Filesystem overhead

As a result, storage performance does not scale proportionally with processor performance.

For example, replacing a 16-core desktop processor with a 64-core workstation processor may dramatically increase Compute Score.

The same system may produce only a relatively small improvement in Storage Score if both systems use comparable high-end NVMe devices.

Including Storage directly in the Compute Score would therefore distort the interpretation of overall system performance.

A workstation with substantially greater computational capability could appear artificially limited by storage hardware that has already approached the practical limits of current desktop technology.

For this reason, Storage is evaluated independently.

This allows both performance domains to evolve naturally without influencing each other.

• Compute Score represents computational capability.
• Storage Score represents filesystem and storage capability.

Both values remain equally important while preserving meaningful comparisons across different hardware generations.

Storage Reference Calibration

Storage references are calibrated independently from Compute references.

Every Storage workload defines deterministic reference throughput representing the target performance for a specific workload configuration.

Reference values are normalized to approximately ten thousand points on the benchmark reference platform.

Unlike Compute categories, Storage references are intentionally expected to evolve much more slowly.

Modern flagship NVMe devices already approach the practical bandwidth limits available on consumer desktop platforms.

Consequently, future recalibration of Storage references is expected to occur far less frequently than Compute reference updates.

This approach preserves long-term benchmark stability while allowing gradual adaptation as storage technology evolves.

Practical Interpretation

Storage Score should be interpreted independently from Compute Score.

High Storage scores generally indicate:

• Fast storage hardware
• Efficient filesystem implementation
• Low synchronization overhead
• Strong metadata performance
• Excellent small-file behavior
• Efficient operating system storage stack

Applications that frequently benefit from high Storage Score include:

• Software compilation
• Backup software
• Package managers
• Artifact repositories
• Databases
• Source code repositories
• Virtualization platforms
• Container runtimes
• Continuous integration systems

Because Storage Score reflects practical filesystem behavior rather than theoretical bandwidth, it often provides a much better indicator of real application performance than synthetic storage benchmarks.

Design Summary

The Storage benchmark evaluates the practical behavior of modern storage systems using deterministic workloads representative of real software engineering environments.

Rather than measuring only sequential bandwidth or synthetic IOPS, the benchmark evaluates multiple independent filesystem workloads including sequential transfers, random access, metadata operations, directory traversal, synchronization, and small file processing.

The resulting Storage Score therefore represents the complete efficiency of the storage subsystem experienced by real applications.

Unlike Compute Score, Storage Score is intentionally evaluated as an independent performance domain.

This separation ensures that future processor generations may continue to increase computational capability without artificially influencing storage evaluation.

Together, Compute Score and Storage Score provide a balanced representation of modern workstation performance while preserving meaningful comparisons across different hardware architectures, operating systems, and storage technologies.