/docs/metrics

Metrics & monitoring

The IICP directory exposes a Prometheus-compatible /api/metrics endpoint. Metrics are cached for 60 seconds and include node counts, heartbeat freshness, reputation scores, and discover latency percentiles.

This page covers the directory metrics. Each provider node also exposes its own Prometheus endpoint at GET /metrics on its serve port (default 9484) — per-node task counts, latency, and backend health (ADR-014). Directory reachability health answers whether the serving surface can be used; the separate registry backend_stability block answers whether the provider's local model backend is ready, cold/loading, unstable, or draining.

For operators wiring up monitoring. Both the directory and each node expose a Prometheus /metricsendpoint — this page lists what each metric means. Skip it unless you're setting up dashboards/alerts. Full reference below.

Prometheus scrape config

# prometheus.yml
scrape_configs:
  - job_name: iicp_directory
    scrape_interval: 60s          # metrics are cached 60s; scraping faster is wasteful
    static_configs:
      - targets: ['iicp.network']
    metrics_path: /api/metrics
    scheme: https

Rate limit: 30 requests/minute per IP. Prometheus scraping at 60s intervals stays well within the limit. No authentication required.

Metric reference

Metric	Type	Description
iicp_nodes_registered	gauge	Total registered nodes by region label. Includes all status levels (active + stale).
iicp_nodes_available	gauge	Heartbeating nodes (last heartbeat within 90 s) by region label. Public routability is a separate routing signal.
iicp_heartbeat_age_seconds	summary	Age in seconds since last heartbeat across heartbeating nodes. Quantiles: p50 and p95. A rising p95 indicates nodes dropping off the mesh.
iicp_discovery_score_p50	gauge	Median reputation score across discovered candidates (0–1). This evolving routing-history signal is separate from uptime, operational health, answer quality, and fraud evidence.

Region labels on iicp_nodes_registered and iicp_nodes_available match the region slugs returned by GET /api/v1/registry/nodes (e.g., eu-central, us-east).

Sample output

curl -sf https://iicp.network/api/metrics

# HELP iicp_nodes_registered Total registered IICP nodes by region.
# TYPE iicp_nodes_registered gauge
iicp_nodes_registered{region="eu-central"} 8

# HELP iicp_nodes_available Active IICP nodes (last_seen within 90 s) by region.
# TYPE iicp_nodes_available gauge
iicp_nodes_available{region="eu-central"} 8

# HELP iicp_heartbeat_age_seconds Age in seconds since last heartbeat for heartbeating nodes.
# TYPE iicp_heartbeat_age_seconds summary
iicp_heartbeat_age_seconds{quantile="0.5"} 18.4
iicp_heartbeat_age_seconds{quantile="0.95"} 27.1
iicp_heartbeat_age_seconds_count 8
iicp_heartbeat_age_seconds_sum 153.7

# HELP iicp_discovery_score_p50 Median reputation score across discovered/routable candidates (0-1).
# TYPE iicp_discovery_score_p50 gauge
iicp_discovery_score_p50 0.712

Suggested alerts

# Prometheus alerting rules
groups:
  - name: iicp
    rules:
      - alert: IicpNoActiveNodes
        expr: sum(iicp_nodes_available) == 0
        for: 2m
        annotations:
          summary: No active IICP nodes — discover endpoint will return empty results

      - alert: IicpHeartbeatLag
        expr: iicp_heartbeat_age_seconds{quantile="0.95"} > 60
        for: 5m
        annotations:
          summary: p95 heartbeat age > 60 s — nodes may be losing connectivity

      - alert: IicpLowReputation
        expr: iicp_discovery_score_p50 < 0.4
        for: 10m
        annotations:
          summary: Median reputation below 0.4 — inspect the score model and evidence basis; do not infer downtime or fraud

JSON stats endpoint

If you need structured JSON rather than Prometheus text format, use the stats endpoint:

curl -sf https://iicp.network/api/v1/stats

Returns a nested server object (active, heartbeating, public-routable, key-ready and downlevel node counts), probe aggregates, directory_health, resilience (visible, heartbeating, recovering/recent nodes and relay availability), sdk_adoption and the mesh_health object.mesh_health_federated remains null until federation is active. This is useful for dashboards and health checks without a Prometheus stack.

Interpreting recovery windows. active_nodes / public_routable_nodes are nodes a strict client can use now; heartbeating_nodes includes nodes still checking in even if a public route is settling. The resilience block prevents a sleeping laptop, tunnel cooldown, or relay restart from looking like a permanent zero-node mesh.

Node setup Registry API Live stats