Design Goals

The Scalionix System Benchmark was designed around a set of architectural principles intended to provide consistent, repeatable, and meaningful performance measurements across modern computer systems.

Rather than maximizing benchmark score variability or producing synthetic peak numbers, the framework focuses on measuring workloads that resemble those executed by real software development environments, backend infrastructure, and production systems.

Every design decision within the benchmark follows one or more of the principles described below.

Real-World Workloads

The benchmark intentionally avoids artificial instruction loops whose only purpose is to maximize processor utilization.

Instead, every category performs complete application workloads that involve realistic algorithms, data structures, memory access patterns, filesystem interaction, synchronization primitives, or software compilation.

Examples include:

• JSON serialization and deserialization
• Cryptographic hashing
• Authenticated encryption
• Lossless compression
• Container manipulation
• Thread synchronization
• Storage subsystem operations
• Complete software compilation

This approach produces benchmark results that more closely resemble practical machine performance during everyday engineering work.

Cross-Platform Consistency

One of the primary objectives of the framework is to produce comparable benchmark results across different operating systems.

Whenever possible, identical algorithms and workload definitions are executed on:

• Linux
• macOS
• Windows

Although operating systems inevitably differ in scheduler behavior, filesystem implementation, memory management, and system APIs, benchmark scenarios attempt to minimize these differences by performing equivalent computational work.

Platform-specific implementations are introduced only when operating system interfaces fundamentally differ.

Examples include:

• Filesystem synchronization APIs
• Storage device discovery
• Hardware information collection
• Compiler toolchain detection

Whenever platform-specific implementations are necessary, they are designed to produce equivalent benchmark semantics rather than identical implementation details.

Deterministic Execution

Benchmark repeatability is considered significantly more important than achieving the highest possible throughput.

Every workload executes deterministic input generated from predefined benchmark datasets.

The benchmark avoids using:

• Random execution order
• Adaptive optimization
• Dynamically changing datasets
• Workload mutation between iterations

Whenever random values are required, deterministic seeds are used to guarantee reproducible benchmark inputs.

This ensures that identical benchmark configurations perform identical computational work across different executions.

Reference-Based Scoring

Scalionix does not compare benchmark executions against other machines available at runtime.

Instead, every workload is evaluated against a calibrated reference throughput.

Each benchmark scenario has predefined reference values representing expected throughput for a calibrated reference platform.

Individual workload scores are calculated by comparing measured throughput against these reference values.

This design provides several advantages:

• Historical benchmark reports remain comparable
• Benchmark scores remain stable over time
• Results are independent of currently available machines
• New benchmark executions can be compared with reports generated years earlier

The reference calibration process is described in a dedicated chapter later in this document.

Independent Benchmark Categories

Each benchmark category measures one subsystem of the machine independently.

Examples include:

• Serialization
• Hashing
• Encryption
• Storage
• Compilation

The execution of one category does not affect the scoring logic of another category.

This separation allows individual benchmark categories to evolve independently without requiring changes to unrelated benchmark logic.

It also allows users to execute only selected benchmark categories when evaluating a particular subsystem.

Representative Scaling

Modern software rarely executes on a single processor core.

For this reason, benchmark workloads evaluate both computational performance and scalability across multiple worker configurations.

Most compute categories execute using a predefined worker matrix ranging from a single worker up to the maximum supported configuration.

This allows the benchmark to measure:

• Single-thread efficiency
• Parallel throughput
• Scalability
• Scheduler efficiency
• Synchronization overhead

The Compilation benchmark intentionally differs from this model.

Because production software compilation is almost exclusively performed using parallel build systems, compilation workloads execute only the representative multi-thread configuration using the machine’s physical processor core count.

This design reflects practical software engineering workflows rather than synthetic processor measurements.

Hardware Independence

The benchmark does not assume specific processor vendors, storage devices, or memory configurations.

Supported hardware includes:

• x86-64 processors
• ARM64 processors
• Desktop systems
• Laptops
• Workstations

Hardware characteristics are detected dynamically during benchmark initialization and recorded within the benchmark report.

Scoring remains independent of hardware manufacturer.

The benchmark evaluates measured performance rather than hardware specifications.

Extensibility

The benchmark architecture has been designed to allow new benchmark categories to be introduced without affecting existing implementations.

Each category defines:

• Execution model
• Request generator
• Benchmark tasks
• Scoring adapter
• Reference calibration
• Report serialization

As a result, new benchmark domains can be integrated with minimal impact on the remaining system.

Future benchmark categories may include additional serialization formats, networking workloads, message processing, or other production-oriented performance measurements while preserving compatibility with existing benchmark reports.

Long-Term Stability

Benchmark software is expected to evolve over time.

To preserve historical comparability, the framework introduces explicit versioning of benchmark scoring.

Every generated benchmark report contains:

• Benchmark version
• Scoring version
• Benchmark configuration
• Workload definitions
• Hardware snapshot
• Execution metadata

Changes to scoring algorithms or reference values are introduced through versioned scoring models rather than modifying historical benchmark reports.

This approach ensures that benchmark results generated using previous scoring models remain valid and reproducible.

Engineering-Oriented Philosophy

Scalionix System Benchmark was created primarily for software engineers.

The benchmark is intended to answer practical questions such as:

• Which workstation compiles software faster?
• Which processor scales better under parallel workloads?
• Which storage subsystem handles development workloads more efficiently?
• How does one machine compare against another using identical software tasks?

Rather than attempting to maximize synthetic benchmark numbers, the framework emphasizes meaningful engineering metrics that directly influence software development productivity.

Every benchmark category exists because it represents a workload commonly encountered in modern software engineering environments.

The benchmark therefore measures not only raw computational performance but also the practical efficiency of complete systems executing realistic software workloads.