Skip to content
Open app

Self-hosting the keeper

Run the keeper yourself: against simulated paper vaults with no keys at all, against a local validator, or in Docker and on Railway, and point the web app at it.

On this page

Before you start#

You need a checkout of the Aqueduct monorepo, Node.js 24 or newer and pnpm. Install once from the repository root:

Terminal
pnpm install
One keeper per deployment

The keeper signs with the key stored in the program's config, and only one process should hold it. The devnet deployment is driven by the hosted keeper, so a self-hosted keeper either runs in paper mode or against a deployment of your own. Two keepers on one vault would fight over the same moves.

Run it with paper vaults#

By default the keeper uses the paper executor: two simulated vaults kept in a JSON file, seeded with 250,000 and 100,000 USDC, with the program's caps enforced. Rates, the sentinel, the planner and the API are all real. It needs no keys, wallet or RPC.

Terminal
pnpm --filter @aqueduct/keeper dev          # API on http://localhost:8787, restarts on code changes
pnpm --filter @aqueduct/keeper cycle:once   # one cycle for both tiers, printed as tables, then exits
pnpm --filter @aqueduct/keeper test
pnpm --filter @aqueduct/keeper typecheck

The keeper reads configuration from environment variables only and does not load a .env file by itself. Copy apps/keeper/.env.example to .env and either export the variables, or start it with Node's loader from apps/keeper:

Terminal
node --env-file=.env --import tsx src/index.ts

Configuration#

  • PORT
    Default
    8787
    Purpose
    HTTP API port
  • CYCLE_MINUTES
    Default
    10
    Purpose
    Minutes between cycles
  • EXECUTOR
    Default
    paper
    Purpose
    paper for simulated vaults, chain for the deployed program
  • RPC_URL
    Default
    https://api.devnet.solana.com
    Purpose
    Solana RPC for the chain executor
  • KEEPER_KEYPAIR
    Purpose
    Keeper secret key for the chain executor: base58, a JSON byte array, or a path to a keypair file. Must be the program's configured keeper
  • AQUEDUCT_DEPLOYMENT
    Default
    SDK bundle
    Purpose
    Path to a deployment JSON. Defaults to the SDK's deployment for the cluster
  • CLUSTER
    Default
    inferred
    Purpose
    Reported by /health: paper, or inferred from RPC_URL (devnet, mainnet-beta, localnet, custom)
  • FEE_COLLECT_MINUTES
    Default
    60
    Purpose
    Fee sweep interval (chain executor only)
  • DATA_DIR
    Default
    apps/keeper/data
    Purpose
    Where the keeper keeps its state files
  • PAPER_SEED
    Default
    bedrock=250000,rapids=100000
    Purpose
    Starting deposits of new paper vaults
  • OPENROUTER_API_KEY
    Purpose
    Enables the optional LLM second opinion in the sentinel
  • SENTINEL_MODEL
    Default
    typesafe/jev
    Purpose
    OpenRouter model for that second opinion
  • REBAL_*
    Purpose
    Rebalancer policy overrides; see Tuning
From apps/keeper/src/config.ts. Invalid values stop the process at startup.

Run it against a local validator#

To see the keeper sign real transactions, give it a deployment of its own on a local validator:

  1. Build and load the program
    Terminal
    anchor build
    solana-test-validator --reset --bpf-program BweEZpNS6bhcwjxgEgScgZSWRSkKC9tqmC2PpfZdFbhm target/deploy/aqueduct.so
  2. Create the vaults

    The setup script creates a test USDC mint, the config, both vaults and every strategy, seeds starter deposits, generates a keeper key at KEEPER_KEYPAIR_PATH if none exists there, and writes packages/sdk/src/deployments/localnet.json. It reuses the asset mint recorded in that file, so on a fresh validator delete the file first.

    Terminal
    RPC_URL=http://127.0.0.1:8899 CLUSTER=localnet KEEPER_KEYPAIR_PATH=/path/to/keeper.json pnpm --filter @aqueduct/sdk setup
  3. Start the keeper
    Terminal
    EXECUTOR=chain RPC_URL=http://127.0.0.1:8899 KEEPER_KEYPAIR=/path/to/keeper.json DATA_DIR=/tmp/aqueduct-keeper pnpm --filter @aqueduct/keeper dev

    The keeper warns at startup if its key is not the deployment's keeper; every transaction would then fail with Unauthorized.

Docker#

The image builds from the repository root, because the keeper depends on the SDK workspace package:

Terminal
docker build -f apps/keeper/Dockerfile -t aqueduct-keeper .
docker run --rm -p 8787:8787 -v keeper-data:/app/apps/keeper/data aqueduct-keeper

It runs the TypeScript sources with tsx on Node 24 and keeps its state in /app/apps/keeper/data, so mount a volume there.

Railway#

The hosted keeper runs on Railway from apps/keeper/railway.json: it builds the Dockerfile, redeploys on changes under apps/keeper or packages/sdk, checks /health at deploy time, restarts on failure and runs exactly one replica. To run your own:

  1. Create a service from the repository with the root as its build context; Railway picks up the config file.
  2. Attach a volume and set DATA_DIR to its mount path, so decisions and rebalancer memory survive redeploys.
  3. Set EXECUTOR, RPC_URL and KEEPER_KEYPAIR as service variables. Treat the keypair as a secret and never commit it.
  4. Keep one replica. Two would double every move.

State#

  • decisions.json
    Contents
    Decision log, latest targets, last cycle time and the rebalancer's memory (smoothed rates, cooldowns, 24h turnover)
    If lost
    The rebalancer restarts from on-chain balances; cooldowns and the turnover budget reset
  • rate-history.json
    Contents
    APYs the keeper recorded for venues without a history API
    If lost
    Those history charts start over
  • paper-vaults.json
    Contents
    Paper executor balances
    If lost
    Paper vaults restart from PAPER_SEED

The rebalancer writes its memory before it sends a transaction, so a crash and restart never repeat a move.

Point the app at your keeper#

The dashboard reads rates and decisions from the keeper API. In the app, open Settings, enter your keeper's base URL under Keeper API and choose Save & reload; it is stored in that browser only. For your own build of the web app, set it at build time:

apps/web/.env.local
NEXT_PUBLIC_KEEPER_URL=http://localhost:8787
NEXT_PUBLIC_DATA_SOURCE=chain                  # read vaults on-chain through the SDK instead of the built-in simulation
NEXT_PUBLIC_RPC_URL=https://api.devnet.solana.com
NEXT_PUBLIC_CLUSTER=devnet                     # or localnet

The API is described in Keeper HTTP API.

Operating it in production#

Coming with the next keeper release

The next keeper release is built for unattended operation: RPC retries and failover across several endpoints (RPC_URLS), a transaction sender that can never send the same move twice, graceful shutdown on redeploy, validated state with an automatic backup, structured JSON logs, Prometheus metrics, webhook alerts to Slack or Discord (low SOL, stalled cycles, failed moves, sentinel exits), and /ready and /status endpoints. The image will run as a non-root user with its state under /data. Its README carries the full configuration reference and an incident runbook.

Written against apps/keeper/src/config.ts, apps/keeper/src/chain/chainExecutor.ts, apps/keeper/Dockerfile, apps/keeper/railway.json and packages/sdk/scripts/setup.ts. If the code and this page disagree, the code wins.