> ## Documentation Index > Fetch the complete documentation index at: https://docs.iearena.org/llms.txt > Use this file to discover all available pages before exploring further. # CLI > Launch remote rollouts and browse jobs, trials, and secrets from the harbor CLI Please ensure you have either logged in with `harbor auth login` or minted an API key (see [Harbor Hub API key](/core-concepts/hosted-harbor/api-key)) ## Launching a Job The command `harbor run --launch` will attempt to launch a job under your personal organization The `--launch` flag can be combined with a singular agent/model combination (e.g. `harbor run --launch -a claude-code -m anthropic/fable-5 -d harbor/hello-world`). Agent/model sweeps must be launched using a config file (e.g. `harbor run --launch -c config.yaml`) ```yaml theme={"system"} # config.yaml job_name: opus-vs-codex-sweep organization: my-org # omit to use your personal org credential_mode: gateway # or "direct" n_attempts: 3 # 1-10 n_concurrent_trials: 32 # 1-1000 # Every agent runs against every task. Trials = n_attempts x tasks x agents. agents: - name: claude-code model_name: anthropic/claude-opus-4-1 secrets: [ANTHROPIC_API_KEY] - name: claude-code model_name: anthropic/claude-sonnet-4-5 secrets: [ANTHROPIC_API_KEY] - name: codex model_name: openai/gpt-5 secrets: [OPENAI_API_KEY, HF_TOKEN] n_concurrent: 8 # per-agent sub-limit under n_concurrent_trials env: RUST_LOG: info # nonsensitive only - secrets go in `secrets` - name: oracle # needs no inference credential secrets: [] datasets: - name: terminal-bench/terminal-bench-2-1 ref: "6" # `ref` or `version`, never both n_tasks: 25 exclude_task_names: - "flaky-*" tasks: - name: harbor/hello-world ref: latest # Supplied for this job only, read from your local environment at launch. job_secrets: HF_TOKEN: from_env: HF_TOKEN retry: max_retries: 2 include_exceptions: - EnvironmentStartError ``` ### Injecting Secrets There are two methods of injecting secrets into the hosted job: flags `--stored-secret ` and `--one-off-secret ENV_VAR_NAME=secret_value`, and config fields `job_secrets` and `secrets`. When using flags, `--stored-secret` will pull `` from the organization's stored credentials on the hub. `--one-off-secret` will encrypt the secret value and ensure that the value is not persisted in any job config files. The secret will be decrypted only when necessary and marked revoked after the job finishes. Flags apply secrets to every agent/model combination in the job config. In other words, if I launch with the following command: `harbor run --launch -c simple_config.yaml --stored-secret TEST_VAR` where `simple_config.yaml` is ```yaml theme={"system"} job_name: simple-config organization: my-org # omit to use your personal org agents: - name: claude-code model_name: anthropic/claude-opus-4-1 secrets: [ANTHROPIC_API_KEY] - name: claude-code model_name: anthropic/claude-sonnet-4-5 secrets: [ANTHROPIC_API_KEY] tasks: - name: harbor/hello-world ref: latest ``` The final resolved config would be as follows: ```yaml theme={"system"} job_name: simple-config organization: my-org agents: - name: claude-code model_name: anthropic/claude-opus-4-1 secrets: [ANTHROPIC_API_KEY, TEST_VAR] # notice TEST_VAR is used in both agents - name: claude-code model_name: anthropic/claude-sonnet-4-5 secrets: [ANTHROPIC_API_KEY, TEST_VAR] # notice TEST_VAR is used in both agents tasks: - name: harbor/hello-world ref: latest ``` | Flag | Value | Effect | | ------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------- | | `--dry-run` | — | Validate without queuing. Resolves tasks, agents, and the owning org, checks selections, and reports the trial count | | `--org` | `TEXT` | Organization that should own the hosted job. Defaults to your personal org | | `--credential-mode` | `gateway` \| `direct` | `gateway` proxies the key through the Hub, `direct` hands the real key to the agent | | `--stored-secret` | `NAME` | Select a secret already stored in the owning org. Repeatable | | `--one-off-secret` | `NAME[=VALUE]` | Supply a secret for this job only. Bare `NAME` reads it from your environment. Repeatable | | `--env-file` | `PATH` | Load a `.env`. Every name in it is selected, as with `--one-off-secret` | | `--registry-secret` | `HOST=NAME_OR_ID` | Pin a stored pull secret for a private image host. Repeatable | | `--no-secrets` | — | Launch with no credentials at all | `--launch` and `--upload` are mutually exclusive, so the upload-side flags do not apply to a hosted launch: `--upload`, `--public` / `--private`, and `--share`. ## Regrading Add `--launch` to regrade recorded Hub results with an updated verifier. This creates a new private job without rerunning the agent or changing the source. Specify `--org` for the launch organization's credits, permissions, and credentials. It must match the source organization, and you must have permission to launch there; shared read access alone is insufficient. ### Regrade a job ```bash theme={"system"} harbor job regrade --launch --org -d org/dataset@version harbor job regrade --launch --org -p ./updated-tasks ``` Use `-d` for datasets, `-p` for local task directories, or `-t` for registry tasks. These options are repeatable and can be combined. Verifiers match by task name. Harbor selects the latest trial attempts with recorded archives. ### Regrade a trial ```bash theme={"system"} harbor trial regrade --launch --org -t org/task@ref ``` Use exactly one `-t` or `-p`; for `-p`, provide a single task directory. Local verifier tasks are uploaded as temporary private packages. Add `--yes` to accept launch and upload without prompting. ### Requirements Hosted regrade currently supports single-step trials with separate verifiers. Source trials must have finished, and the replacement task must have the same name and be able to grade from the recorded artifacts. Sources must be Hub UUIDs. Use `--dry-run` to validate without creating a job. With local verifier tasks, it checks inputs without uploading; full source validation happens on launch. ### Options These options apply with `--launch`. See the [secret options](#injecting-secrets) for credential details. | Option | Description | | ----------------------------------- | ------------------------------------------------------------------------------ | | `--launch` | Run the regrade on Hosted Harbor. | | `--org NAME` | Launch organization. Required; must match the source organization. | | `-p, --task-path PATH` | Local verifier task. Jobs also accept parent directories; repeatable for jobs. | | `-t, --task REF` | Registry verifier task, such as `org/task@ref`. Repeatable for jobs. | | `-d, --dataset REF` | Dataset of verifier tasks. Job regrade only; repeatable. | | `-n, --n-concurrent N` | Concurrent regrades. Job regrade only. | | `--job-name NAME` | Name of the new hosted job. Job regrade only. | | `--trial-name NAME` | Currently sets the new hosted job's name for trial regrade. | | `--ve, --verifier-env KEY=VALUE` | Verifier environment variable. Repeatable. | | `--stored-secret NAME` | Stored verifier credential from the launch organization. Repeatable. | | `--one-off-secret NAME[=VALUE]` | Job-only credential; bare `NAME` reads your local environment. Repeatable. | | `--registry-secret HOST=NAME_OR_ID` | Select a stored credential for pulling private images. Repeatable. | | `--use-static-ip` | Use a static outbound IP. Otherwise, the server chooses its default. | | `-y, --yes` | Accept launch and local task upload without prompting. | | `--dry-run` | Validate without creating a job. | ## Listing Jobs Using the command `harbor hub job list` you can print out a list of all jobs visible to your user on the hub. | Flag | Value | Effect | | ------------ | ------------------------- | ----------------------------------------------------------------------- | | `--scope` | `my` \| `shared` \| `all` | Visibility scope. Defaults to `my`, the jobs your user owns | | `--search` | `TEXT` | Only jobs whose names contain the substring | | `--agent` | `NAME` | Filter by agent name. Repeatable, and multiple values match any of them | | `--provider` | `NAME` | Filter by model provider. Repeatable | | `--model` | `NAME` | Filter by model. Repeatable | ## Job Overview Use the command `harbor hub job show JOB_ID JOB_ID_2 ...` to print an overview similar to the one provided on the hub. The overview reports the number of trials, errors, and retries, the average return on each metric, the cost in USD, and token usage. ## Per-Task Job Breakdown Use the command `harbor hub job tasks JOB_ID` to see a per-task breakdown of a job. | Flag | Value | Effect | | ---------------------------------- | ------ | -------------------------------------------- | | `--search` | `TEXT` | Only tasks whose names contain the substring | | `--agent`, `--provider`, `--model` | `NAME` | As for [`job list`](#listing-jobs) | ## Trials Overview Use the command `harbor hub job trials JOB_ID JOB_ID_2 ...` to list trials across one or more jobs. In interactive terminals, you can switch pages to audit all trials without running a second command. | Flag | Value | Effect | | ---------------------------------- | ----------------------------------------------------- | ---------------------------------------------------- | | `--search` | `TEXT` | Only trials whose names contain the substring | | `--agent`, `--provider`, `--model` | `NAME` | As for [`job list`](#listing-jobs) | | `--limit` | `N` | Page size. Defaults to 100 here | | `--failed-only` | — | Only trials that errored or failed | | `--include-retries` | — | Include retry history, not just the latest execution | | `--sort-by` | `started_at` \| `task_name` \| `name` \| `error_type` | Sort column | | `--sort-order` | `asc` \| `desc` | Sort direction | ## Comparing 2 or More Jobs Use command `harbor hub job compare JOB_ID JOB_ID_2 ...` to get the side-by-side grid for performance on tasks ## Job Visibility Use command `harbor hub job shares JOB_ID` to see who a job is shared with. This returns orgs and users ## Renaming Jobs Use `harbor hub job rename JOB_ID "NEW NAME"` to change the name displayed on the Hub. This does not change the job ID, artifact identity, or submitted configuration. Add `--json` to print the rename RPC response. ## Transferring Job Ownership Use `harbor hub job transfer JOB_ID ORGANIZATION` to transfer a completed job and all of its trials to another organization you own. Job and trial IDs do not change. The destination must already have access to every execution dependency used by the job. The command prompts for confirmation; pass `--yes` / `-y` when scripting. By default, the source organization keeps shared read access and job-specific secrets are permanently removed. Use `--drop-source-access` to remove the source grant or `--keep-job-credentials` to retain and reauthorize secrets you originally supplied. ## Deleting Jobs Use command `harbor hub job delete JOB_ID JOB_ID_2 ...` to permanently delete jobs you own from the hub, including all of their trials and shares. The command prompts for confirmation before deleting anything; pass `--yes` / `-y` to skip the prompt (required when scripting or piping). Only the job's owner can delete a job. Jobs linked to a leaderboard submission and hosted jobs that are still running cannot be deleted. ## Hosted Job Status Use command `harbor hub job status JOB_ID` to get your job status. Returns counts of pending, running, failed, and completed trials. ## Trial Show Use command `harbor hub trial show TRIAL_ID` to show a single trial's metadata ## Downloading a Trial Use command `harbor hub trial download TRIAL_ID` to download a particular trial | Flag | Value | Effect | | -------------------- | ------ | ----------------------------------------------------------------------------- | | `--output-dir`, `-o` | `PATH` | Directory to materialize the trial into. Defaults to `./trials` | | `--overwrite` | — | Replace an existing trial directory | | `--trajectory` | — | Download only `trajectory.json`. Errors if the trial has no stored trajectory | ## Retrying a Hosted Trial For a trial in a job that was launched remotely, you can use the command `harbor hub trial retry TRIAL_ID` to retry those trials. These flags are cumulative filters. `harbor hub trial retry --job JOB_ID --failed-only` selects every trial in the job, then narrows to the ones that failed. | Flag | Value | Selects | | --------------- | -------- | ---------------------------------------- | | `--job` | `JOB_ID` | Every trial in that job | | `--search` | `TEXT` | Trials whose names contain the substring | | `--agent` | `TEXT` | Trials run with that agent | | `--provider` | `TEXT` | Trials run with that provider | | `--model` | `TEXT` | Trials run with that model | | `--task` | `TEXT` | Trials run against that task | | `--exception` | `TEXT` | Trials that failed with that exception | | `--failed-only` | — | Only trials that failed | | `--yes` | — | Skip confirmation | ## Canceling a Hosted Trial To cancel a hosted trial, use command `harbor hub trial cancel TRIAL_ID` | Flag | Value | Effect | | ---------- | -------- | --------------------------------------------------------------------- | | `--job` | `JOB_ID` | Cancel every trial in the job. Same as `harbor hub job cancel JOB_ID` | | `--all` | — | Cancel every currently running trial | | `--reason` | `TEXT` | Record a reason for the cancellation | | `--yes` | — | Skip confirmation | The selection filters from [`trial retry`](#retrying-a-hosted-trial) apply here too. ## Adding Secrets Use command `harbor hub secrets add NAME` to upload a secret to the Hub. | Flag | Value | Effect | | ------------- | -------- | ------------------------------------------------------------------- | | `--org` | `ORG_ID` | Organization to store the secret on. Defaults to your personal org | | `--job` | `JOB_ID` | Scope the secret to a single job instead of account-wide | | `--from-env` | — | Read the value from the local environment variable of the same name | | `--yes`, `-y` | — | Supersede an existing secret without prompting | ## Listing Secrets Use command `harbor hub secrets list` to list the names and metadata of your uploaded secrets. | Flag | Value | Effect | | ------------------- | -------- | -------------------------------------- | | `--org` | `ORG_ID` | Organization to list secrets for | | `--job` | `JOB_ID` | List secrets scoped to that job | | `--include-revoked` | — | Include secrets that have been revoked | ## Deleting Secrets Use command `harbor hub secrets delete` to revoke a secret. | Flag | Value | Effect | | --------- | -------- | ---------------------------------------------------------------------------------- | | `--org` | `ORG_ID` | Revoke a secret in that organization | | `--job` | `JOB_ID` | Revoke a secret scoped to that job | | `--purge` | — | Delete the record outright, so it no longer appears under `list --include-revoked` | ## Adding an Image Registry Secret Use command `harbor hub secrets registry add` to add an image registry secret. Also supports `secrets registry list` and `secrets registry delete`. | Flag | Value | Effect | | ------------- | ------ | ------------------------------------------------------------------- | | `--name` | `TEXT` | Display name used to select the credential later | | `--from-file` | `PATH` | Read the credential from a file, such as a GAR service account JSON | | `--yes`, `-y` | — | Supersede an existing credential without prompting | ## Shared Flags The listing commands above also accept these: | Flag | Value | Effect | | --------------- | ----- | ------------------------------------------------------ | | `--quiet`, `-q` | — | Print only IDs, for piping into `xargs` | | `--no-trunc` | — | Show full cell content, wrapping instead of truncating | | `--no-headers` | — | Omit the header row | | `--page` | `N` | Fetch one specific page, disabling interactive paging | | `--json` | — | Return the raw API response as JSON |