ECS Remediation Agent

The ECS remediation agent is the first live remediation runner for Nirofy. It runs as a dedicated ECS Fargate service in your AWS account, polls core-platform for signed approved jobs, applies only scoped ECS mutations, verifies service health, and reports structured evidence back.

The default local development stack does not start this agent. That is intentional: the agent needs an agent token, AWS credentials, and task-role permissions that can mutate ECS services.

What It Can Do

Version 1 supports two ECS actions:

Nirofy actionAgent job kindECS API behavior
restart-serviceecs.force-new-deploymentCalls UpdateService with forceNewDeployment: true
scale-serviceecs.update-desired-countCalls UpdateService with bounded desiredCount

All v1 mutations require manual approval before core-platform creates a leaseable execution job.

Control Plane Flow

flowchart LR
  Diagnosis["AI diagnosis"] --> Approval["Manual approval"]
  Approval --> Job["Signed execution job"]
  Job --> Agent["ECS remediation agent"]
  Agent --> Preflight["DescribeServices preflight"]
  Preflight --> Update["UpdateService"]
  Update --> Verify["DescribeServices verification"]
  Verify --> Audit["Audit evidence + status"]

Core-platform remains the control plane for policy, approval, audit, and job signing. The agent only executes typed capabilities against its local allowed scope.

Local Commands

Run package checks explicitly:

bun run agent:typecheck
bun run agent:test
bun run agent:build

Start the agent explicitly for a scoped test:

bun run agent:dev

Build a local container image:

bun run docker:agent:build

Do not add the agent to bun run dev unless you are building a separate safe mock agent mode. Normal local development should not start a process that can mutate cloud infrastructure.

Register an Agent

Run migrations first:

bun run db:migrate

Then register or update the agent identity:

OPERYN_AGENT_ID=ecs-prod-us-east-1 \
OPERYN_AGENT_TOKEN=replace-with-random-secret \
OPERYN_ORG_ID=replace-with-org-id \
OPERYN_ENVIRONMENT=production \
AWS_ACCOUNT_ID=123456789012 \
AWS_REGION=us-east-1 \
OPERYN_AGENT_ALLOWED_SCOPES_JSON='{"awsAccountId":"123456789012","region":"us-east-1","ecsServices":[{"clusterName":"prod","serviceName":"checkout"}],"maxDesiredCount":4,"capabilities":["ecs.force-new-deployment","ecs.update-desired-count"]}' \
bun run agent:register

The registration script stores only a SHA-256 hash of OPERYN_AGENT_TOKEN.

Runtime Environment

Required:

VariableDescription
OPERYN_CORE_URLHTTPS URL for core-platform
OPERYN_AGENT_IDRegistered agent id
OPERYN_AGENT_TOKENBearer token and signed-job HMAC secret
OPERYN_ORG_IDNirofy organization id
OPERYN_ENVIRONMENTEnvironment served by this agent
AWS_ACCOUNT_IDAWS account allowed by local scope
AWS_REGIONAWS region allowed by local scope

Optional:

VariableDefaultDescription
OPERYN_AGENT_ALLOWED_SCOPES_JSON—Full local scope JSON
MAX_DESIRED_COUNT10Fallback max desired count when scope JSON is omitted
OPERYN_AGENT_HEARTBEAT_MS30000Heartbeat interval
OPERYN_AGENT_POLL_MS7500Job polling interval
OPERYN_AGENT_LEASE_SECONDS90Job lease duration
OPERYN_AGENT_STABLE_TIMEOUT_MS300000ECS stabilization timeout

Allowed Scope JSON

OPERYN_AGENT_ALLOWED_SCOPES_JSON is the agent's local hard stop. Even if core-platform sends a signed job, the agent rejects it unless it matches this scope.

{
  "awsAccountId": "123456789012",
  "region": "us-east-1",
  "ecsServices": [
    { "clusterName": "prod", "serviceName": "checkout" }
  ],
  "maxDesiredCount": 4,
  "capabilities": [
    "ecs.force-new-deployment",
    "ecs.update-desired-count"
  ]
}

ECS Deployment Files

Starter files live under infra/aws/:

  • ecs-remediation-agent-task-role-policy.json
  • ecs-remediation-agent-task-definition.example.json
  • ecs-remediation-agent-service.example.json

Replace the account, region, cluster, service, subnet, security group, image, and Secrets Manager placeholders before deploying.

IAM Requirements

The task role should allow only the target ECS service resources:

  • ecs:DescribeServices
  • ecs:UpdateService

Explicitly do not grant:

  • ecs:ExecuteCommand
  • ecs:RunTask
  • ecs:StopTask
  • iam:PassRole
  • wildcard mutation permissions

Use the NirofyRemediationEnabled=true ECS service tag as an additional guard.

Preflight and Verification

Before mutation, the agent checks:

  • service exists
  • service status is ACTIVE
  • deployment controller is ECS rolling deployment
  • requested account, region, cluster, service, and capability match local scope
  • scale desired count is within local and policy limits

After mutation, the agent polls DescribeServices until:

  • running count equals desired count
  • pending count is zero
  • no deployment reports failed rollout state
  • deployment set is stable

Timeouts mark the job failed and report the last observed ECS service state.

Dashboard Behavior

The Dashboard shows ECS-specific execution context for remediation actions:

  • runtime
  • account and region
  • cluster and service
  • operation
  • desired count for scale jobs
  • agent online eligibility

Approval for an ECS remediation action is disabled when no matching online agent is available for the target account, region, environment, and capability.

Production Checklist

  • Run migration 0040_add_ecs_remediation_agents.
  • Register the agent with bun run agent:register.
  • Store OPERYN_AGENT_TOKEN in Secrets Manager.
  • Build and push the agent image to ECR.
  • Replace all placeholders in infra/aws/*.json.
  • Attach the least-privilege task role.
  • Tag target ECS services with NirofyRemediationEnabled=true.
  • Deploy one agent service per account/region/environment scope.
  • Test against a non-critical ECS service before production.
  • Kill the agent mid-job once to verify lease expiry and recovery.