Existing plugins

Connect Harbor jobs to observability platforms.

Job plugins run in the Harbor process and react to job and trial events. Pass them at runtime with --plugin; plugins are not stored in config.json.

Install a plugin in the same Python environment as Harbor. Plugin credentials are Harbor process variables, not agent environment variables.

Discover plugins

Installed packages register a short plugin name. List them with:

harbor plugins list

Harbor provides two official plugins: langsmith and atif2otel. harbor plugins list shows the plugins installed in the environment running Harbor. A source checkout may already include both as editable workspace packages; PyPI installations must add the desired plugin package.

You can also pass a full module:Class import path to --plugin.

LangSmith

The LangSmith plugin syncs Harbor tasks to a LangSmith dataset and records each job as an experiment.

uv tool install "harbor[langsmith]"

export OPENAI_API_KEY="..."
export LANGSMITH_API_KEY="..."

harbor run \
  -t hello-world/hello-world \
  -a codex -m openai/gpt-5.6-sol \
  --plugin langsmith \
  --pk dataset_name=harbor-evals \
  --pk experiment_name=codex-baseline

Harbor experiments in LangSmith

LangSmith options

Option	Environment variable
dataset_name	HARBOR_LANGSMITH_DATASET
experiment_name	HARBOR_LANGSMITH_EXPERIMENT
experiment_id	HARBOR_LANGSMITH_EXPERIMENT_ID
endpoint	LANGSMITH_ENDPOINT
workspace_id	LANGSMITH_WORKSPACE_ID
sync_dataset	HARBOR_LANGSMITH_SYNC_DATASET
fail_fast	HARBOR_LANGSMITH_FAIL_FAST

api_key is also accepted, but LANGSMITH_API_KEY is recommended for credentials.

See the LangSmith docs and launch post.

ATIF to OpenTelemetry

The ATIF-to-OpenTelemetry plugin converts each agent’s ATIF trajectory into OpenTelemetry spans.

It converts agent/trajectory.json into spans for model calls, tool calls, and subagents.

uv tool install --with harbor-atif2otel harbor

Write files

Export all trials after the job finishes:

export OPENAI_API_KEY="..."

harbor run \
  -t hello-world/hello-world \
  -a codex -m openai/gpt-5.6-sol \
  --plugin atif2otel \
  --pk output_dir=./otel-traces \
  --pk encoding=json

Stream to MLflow

The built-in uploader exports each trial to an MLflow server when it finishes:

export OPENAI_API_KEY="..."
export MLFLOW_TRACKING_TOKEN="..."

harbor run \
  -t hello-world/hello-world \
  -a codex -m openai/gpt-5.6-sol \
  --plugin atif2otel \
  --pk endpoint=https://mlflow.example.com \
  --pk experiment_name=harbor-evals

ATIF-to-OpenTelemetry options

Option	Environment variable	Default
endpoint	OTEL_EXPORTER_OTLP_ENDPOINT	—
output_dir	HARBOR_OTEL_OUTPUT_DIR	—
experiment_name	MLFLOW_EXPERIMENT_NAME	Job name
token	MLFLOW_TRACKING_TOKEN	""
workspace	—	default
encoding	—	json
mode	—	auto

mode=auto streams when endpoint is set and writes files when output_dir is set. Set both to enable both outputs.

See the plugin source.

Use multiple plugins

--plugin and --plugin-kwarg are repeatable. Prefix each kwarg with its plugin name when using more than one plugin:

harbor run ... \
  --plugin langsmith \
  --plugin atif2otel \
  --pk langsmith.experiment_name=codex-baseline \
  --pk atif2otel.output_dir=./otel-traces

When running from a job config, keep plugin flags on the command line:

harbor run --config config.json --plugin langsmith

To build another integration, see Custom plugins.