Task 7 — Infrastructure Resource Orchestrator

Objective

Create an infrastructure resource orchestration system.

The system manages physical infrastructure and deploys application services across available compute resources.

The infrastructure contains:

• regions
• data centers
• racks
• servers
• server resources
• tenants
• services
• service replicas
• deployments
• resource quotas
• placement rules
• health checks
• maintenance windows
• failure events
• replica migrations

The implementation must determine where service replicas may run, whether deployment requirements can be satisfied, and how the system should react when infrastructure becomes unavailable.

The task focuses on modeling resource allocation, placement constraints, service health, failure handling, and state changes inside a distributed infrastructure system.

Domain Overview

Conceptually:

        Region
          |
          +-- DataCenter
                |
                +-- Rack
                      |
                      +-- Server
                            |
                            +-- CPU Capacity
                            +-- Memory Capacity
                            +-- Storage Capacity
                            +-- Network Capacity
                            |
                            +-- Service Replicas

Application-side relationships:

        Tenant
          |
          +-- ResourceQuota
          |
          +-- Service
                |
                +-- Deployment
                      |
                      +-- ServiceReplica
                      +-- PlacementRules
                      +-- HealthChecks

Operational relationships:

        Server
          |
          +-- MaintenanceWindow
          +-- FailureEvent
        
        ServiceReplica
          |
          +-- Placement
          +-- Health
          +-- MigrationHistory

Region

A region represents a large infrastructure location.

For example:

type Region struct {
    ID   string
    Name string
}

Example regions:

eu-west
eu-central
us-east

Data Center

A data center belongs to one region. For example:

type DataCenter struct {
    ID       string
    Name     string
    RegionID string
}

Each data center may contain multiple racks.

Rack

A rack belongs to one data center. For example:

type Rack struct {
    ID           string
    DataCenterID string
}

Each rack may contain multiple servers.

Server

Each server contains:

• ID
• Rack ID
• CPU Capacity
• Memory Capacity
• Storage Capacity
• Network Capacity
• Current Status

A possible model is:

type Server struct {
    ID            string
    RackID        string
    CPUCores      int
    MemoryGB      int
    StorageGB     int
    NetworkMbps   int
    Status        ServerStatus
}

Server Status

Supported server states may include:

• available
• maintenance
• failed
• disabled

A typed representation is recommended. For example:

type ServerStatus string

A server must not receive new workloads when its status is:

• maintenance
• failed
• disabled

Resource Capacity

The implementation must track both:

• total resources
• allocated resources

For each server. A useful model is:

type ResourceCapacity struct {
    CPUCores    int
    MemoryGB    int
    StorageGB   int
    NetworkMbps int
}

The available capacity is:

available = total - allocated

The system must never allocate more resources than the server physically provides.

Tenant

A tenant represents an owner of one or more services.

For example:

type Tenant struct {
    ID   string
    Name string
}

Each tenant has resource limits.

Resource Quota

A tenant may have limits such as:

• maximum CPU cores
• maximum memory
• maximum storage
• maximum replicas

For example:

type ResourceQuota struct {
    TenantID      string
    MaxCPUCores   int
    MaxMemoryGB   int
    MaxStorageGB  int
    MaxReplicas   int
}

The total resource consumption of all active tenant replicas must not exceed the configured quota.

Service

A service represents an application component.

For example:

type Service struct {
    ID       string
    TenantID string
    Name     string
}

Examples:

service-api
service-worker
service-auth
service-db-proxy

Deployment

A deployment defines how a service should run. A possible model is:

type Deployment struct {
    ID               string
    ServiceID        string
    DesiredReplicas  int
    Resources        ReplicaResources
    PlacementRules   PlacementRules
}

Replica Resources

Each replica requests resources. For example:

type ReplicaResources struct {
    CPUCores    int
    MemoryGB    int
    StorageGB   int
    NetworkMbps int
}

Example:

CPU = 4 cores
Memory = 8 GB
Storage = 50 GB
Network = 200 Mbps

Every replica of the deployment requires these resources.

Service Replica

A service replica represents one running instance. For example:

type ServiceReplica struct {
    ID           string
    DeploymentID string
    ServerID     string
    Role         ReplicaRole
    Status       ReplicaStatus
}

Possible roles:

• primary
• backup
• regular

Possible replica states:

• pending
• running
• unhealthy
• migrating
• stopped
• failed

Placement Rules

Deployments may define placement constraints. A possible model is:

type PlacementRules struct {
    MinimumDataCenters       int
    MaximumReplicasPerServer int
    SeparatePrimaryAndBackup bool
    SeparateReplicasByRack   bool
}

Additional restrictions may also be modeled where explicitly required.

Base Deployment Scenario

Create the following deployment:

Deployment ID:
deployment-api

Service:
service-api

Desired Replicas:
6

Each replica requires:

CPU = 4
Memory = 8 GB
Storage = 50 GB
Network = 200 Mbps

Placement rules:

• minimum 2 data centers
• maximum 2 replicas per server
• primary and backup replicas cannot share the same server
• servers under maintenance cannot receive workloads
• failed servers cannot receive workloads

The implementation must determine whether all six replicas can be placed.

Deterministic Placement

When several servers are equally valid, placement should be deterministic.

A reasonable rule is:

• DataCenter ID ascending
• Rack ID ascending
• Server ID ascending

The chosen rule must be documented.

The system should not produce random placement for identical input.

Placement Validation

A candidate server is valid only when all required conditions are satisfied.

At minimum:

• server status = available
• enough free CPU
• enough free memory
• enough free storage
• enough free network capacity
• maximum replicas per server not exceeded
• placement rules remain valid
• tenant quota remains valid

Placement Result

A possible result model is:

type ReplicaPlacement struct {
    ReplicaID    string
    ServerID     string
    RackID       string
    DataCenterID string
}

type DeploymentPlacementResult struct {
    DeploymentID string
    Successful   bool
    Placements   []ReplicaPlacement
    Errors       []string
}

If the complete deployment cannot be satisfied, the implementation must clearly report why.

Atomic Deployment Behavior

The implementation must define deployment behavior when only part of the requested replicas can be placed.

For the base task, use atomic deployment behavior:

• either all requested replicas can be placed
• or no new replicas are committed

This avoids leaving a partially created deployment without explicit intent.

An optional extension may support partial deployment.

Resource Reservation

When a placement succeeds, server resources must be reserved.

For each replica:

AllocatedCPU += replica.CPU
AllocatedMemory += replica.Memory
AllocatedStorage += replica.Storage
AllocatedNetwork += replica.Network

Resources must be released when a replica is permanently removed from the server.

Maintenance Window

A server may enter scheduled maintenance.

A possible model is:

type MaintenanceWindow struct {
    ID        string
    ServerID  string
    StartTime time.Time
    EndTime   time.Time
    Reason    string
}

During an active maintenance window:

new replicas cannot be placed on the server

The implementation should also determine whether already-running replicas must be migrated before maintenance begins.

For this task:

running replicas must be evacuated before planned maintenance

Failure Event

A server may fail unexpectedly.

A possible model is:

type FailureEvent struct {
    ID        string
    ServerID  string
    Timestamp time.Time
    Reason    string
}

When a server fails:

Server.Status = failed

Every running replica on that server becomes unavailable.

Failure Scenario

Assume:

server-17

fails.

The implementation must determine:

• which replicas were running on server-17
• which deployments are affected
• which services are degraded
• whether desired replica count is still satisfied
• whether placement rules are still satisfied

Service Health

A service may have one of the following health states:

• healthy
• degraded
• unavailable

A possible rule set:

        healthy:
            all desired replicas are running
        
        degraded:
            at least one replica is running,
            but fewer than desired replicas are available
        
        unavailable:
            no replicas are running

If primary/backup rules are used, role-specific availability must also be considered.

Replica Recovery

After a server failure, the orchestrator should attempt to restore the deployment to its desired state.

Conceptually:

• determine missing replicas
• find valid replacement servers
• reserve resources
• create replacement placements
• restore desired replica count

Recovery must still respect all original placement rules.

Migration

A replica migration moves a workload from one server to another. A possible model is:

type ReplicaMigration struct {
    ID             string
    ReplicaID      string
    SourceServerID string
    TargetServerID string
    Reason         string
    Status         MigrationStatus
}

Possible reasons:

• server failure
• planned maintenance
• manual relocation
• capacity rebalance

Migration Rules

The target server must satisfy the same requirements as a new placement.

Resources should not be released from the source location before the migration behavior is safely determined.

The exact migration execution strategy may be simplified, but the final state must remain consistent.

Health Check

Services may define health checks. For example:

type HealthCheck struct {
    ID           string
    ReplicaID    string
    Passed       bool
    CheckedAt    time.Time
    FailureCount int
}

A replica may become unhealthy without its physical server being failed.

The orchestrator must distinguish server failure from application health failure.

Unhealthy Replica Scenario

If one replica repeatedly fails health checks:

Replica.Status = unhealthy

The implementation should determine whether it still counts toward service availability. For this task:

only replicas with Status = running and healthy count as available

Tenant Quota Validation

Before creating a deployment, calculate the tenant’s projected total resource usage. For example:

        Current Tenant Usage:
            CPU = 20
            Memory = 40 GB
        
        New Deployment:
            6 replicas
            4 CPU each
            8 GB each

Projected additional usage:

CPU = 24
Memory = 48 GB

The deployment must be rejected when the projected result exceeds tenant quota.

Placement Rule Scenario

Create an additional deployment:

        Service:
            service-auth
        
        Replicas:
            4

Rules:

• minimum 2 data centers
• maximum 1 replica per server
• separate replicas by rack

The implementation must verify that:

• no two replicas share the same server
• no two replicas share the same rack
• at least two data centers are used

Capacity Failure Scenario

Create a request where enough total infrastructure capacity exists globally, but placement constraints prevent a valid deployment.

For example:

• 4 replicas requested
• enough CPU and memory exist
• only 2 valid racks exist
• rule requires 4 distinct racks

The result must report:

placement constraint failure

rather than incorrectly reporting insufficient CPU or memory.

Infrastructure Snapshot

A useful implementation may provide an infrastructure snapshot. For example:

type InfrastructureSnapshot struct {
    Regions      []Region
    DataCenters  []DataCenter
    Racks        []Rack
    Servers      []Server
    Replicas     []ServiceReplica
    Deployments  []Deployment
}

The snapshot may be used for reporting or testing.

Audit History

Every significant state-changing action should produce an audit event. Examples:

deployment created
replica placed
replica migrated
server entered maintenance
server failed
replica became unhealthy
recovery started
recovery completed

A possible model:

type AuditEvent struct {
    ID         string
    Timestamp  time.Time
    EntityType string
    EntityID   string
    Action     string
    Details    string
}

The audit log should preserve history instead of only storing final state.

Idempotent Deployment Request

Deployment creation requests should contain a request ID. For example:

type DeploymentRequest struct {
    RequestID    string
    Deployment   Deployment
}

If the same RequestID is processed twice, the system must not allocate the same deployment twice.

The second request should return the previously known result or report that the request was already processed.

Validation

The implementation should validate:

• duplicate entity IDs
• unknown RegionID
• unknown DataCenterID
• unknown RackID
• unknown ServerID
• unknown TenantID
• unknown ServiceID
• invalid resource capacity
• negative resource values
• desired replicas <= 0
• invalid placement rules
• tenant quota violations
• maintenance time ranges
• duplicate request IDs

Resource values must not be negative.

Required Operations

The system must support:

1. infrastructure creation
2. tenant creation
3. quota configuration
4. service creation
5. deployment creation
6. replica placement
7. placement validation
8. resource reservation
9. service health calculation
10. server maintenance
11. server failure handling
12. unhealthy replica handling
13. replica recovery
14. replica migration
15. audit history
16. idempotent deployment requests

Required Test Scenarios

Create tests for at least:

• successful deployment
• insufficient server capacity
• tenant quota exceeded
• maintenance server excluded
• failed server excluded
• minimum data-center rule
• maximum replicas per server
• rack separation rule
• server failure
• successful replica recovery
• failed replica recovery
• health-check failure
• migration
• duplicate RequestID
• resource release

Modeling Goal

The purpose of this task is not to recreate Kubernetes or another existing orchestration platform.

The goal is to model the fundamental relationships and rules involved in infrastructure orchestration.

A useful conceptual architecture is:

        Infrastructure Registry
                |
                +-- Regions
                +-- Data Centers
                +-- Racks
                +-- Servers
                |
                v
        Capacity Manager
                |
                v
        Placement Engine
                |
                +-- Resource Rules
                +-- Tenant Quotas
                +-- Placement Rules
                |
                v
        Replica Manager
                |
                +-- Deployment
                +-- Migration
                +-- Recovery
                |
                v
        Health Manager
                |
                +-- Server Failure
                +-- Health Checks
                |
                v
        Audit History

The important challenge is keeping infrastructure state, application state, resource allocation, and placement constraints consistent while the system changes over time.