Built-in Profiler

Quick Start

Set the HEW_PPROF environment variable, then open the URL in your browser:

HEW_PPROF=:6060 hew run my_program.hew
# Open http://localhost:6060/

That's it. No imports, no annotations, no config files. The profiler is built into the runtime and activated purely via environment variable.

Install and launch the observer

To explore actors in a desktop window and save sessions for later, use Hew Observe desktop. The commands below open the terminal observer.

Hew v0.9.3 includes the observer as the hew observe subcommand. Verify the installed SDK, start the profiled program with the Quick Start command above, then attach from another terminal:

hew --version
hew observe --addr localhost:6060

On Unix, hew run my_program.hew --profile uses a discoverable per-user socket, so the second command can be just hew observe. Use hew observe --demo to explore the dashboard with sample actor data.

Hew v0.9.3 includes the Observe v1 profiler and terminal observer. Keep the compiler and observer from the same SDK release together.

Follow the terminal user guide for screenshots, discovery and keyboard controls, or the browser dashboard guide for charts and profile exports. The illustrated manual covers all Observe interfaces.

How it works

When HEW_PPROF is set, the runtime spawns two additional OS threads alongside the normal actor scheduler:

• Sampler thread — captures a metrics snapshot every second into a ring buffer (5 minutes of history)
• HTTP server thread — serves a self-contained dashboard and JSON API

These are plain OS threads, not Hew actors, so the profiler remains responsive even if the actor scheduler is overloaded or deadlocked. The profiler has no external dependencies — the dashboard is a single HTML page with inline JavaScript, embedded in the binary at compile time.

Activation

The HEW_PPROF variable accepts a bind address:

Dashboard

The dashboard is a live-updating single-page app with five panels:

• Scheduler — tasks spawned/completed rates, active workers, work-steal rate
• Memory — bytes live (area chart), peak bytes, allocation count and rate
• Messages — send/receive rates, work steal count and rate
• Allocation Timeline — allocation rate over time (full-width area chart)
• Actors — table of live actors sorted by message count, with dispatch time averages, mailbox depth, and high-water marks

All panels auto-update every second via fetch polling. No WebSocket required.

JSON API

The profiler exposes ten versioned JSON endpoints for programmatic access or custom tooling, plus a plain-text scrape endpoint. Every JSON response uses this envelope and includes an X-Hew-Schema-Version: v1 header:

Hew v0.9.3 uses Observe v1. Observe's wire version is independent of the Hew release version, and consumers reject a mismatched envelope. The authoritative machine-readable contract is OpenAPI 3.1. In v1, signed and unsigned 64-bit integers are canonical decimal strings; bounded 16-bit and 32-bit fields remain JSON numbers. Unknown object fields and new taxonomy strings are accepted, while missing required fields, wrong types, and non-v1 envelopes are rejected.

{
  "schema_version": "v1",
  "data": {}
}

GET /api/metrics

Current snapshot of the basic scheduler, allocator, and TCP counters:

{
  "schema_version": "v1",
  "data": {
    "timestamp_secs": "42",
    "tasks_spawned": "1500",
    "tasks_completed": "1498",
    "steals": "23",
    "messages_sent": "5000",
    "messages_received": "4998",
    "active_workers": "3",
    "alloc_count": "12345",
    "dealloc_count": "12000",
    "bytes_allocated": "1048576",
    "bytes_freed": "1000000",
    "bytes_live": "48576",
    "peak_bytes_live": "65536",
    "tcp_bytes_read": "0",
    "tcp_bytes_written": "0",
    "tcp_accept_count": "0",
    "tcp_connect_count": "0",
    "tcp_error_count": "0"
  }
}

GET /api/actors

Per-actor statistics and mailbox depths:

{
  "schema_version": "v1",
  "data": [
    {
      "id": "1",
      "pid": "1",
      "actor_type": "Counter",
      "state": "idle",
      "msgs": "3000",
      "time_ns": "3170039",
      "mbox_depth": "0",
      "mbox_hwm": "2921"
    }
  ]
}

GET /api/memory

Current allocator statistics (alloc/dealloc counts, bytes allocated/freed/live, peak).

GET /api/metrics/history

Time-series ring buffer with up to 300 entries (5 minutes at 1-second intervals). Each entry contains the basic scheduler, allocator, and TCP counters, using short keys (t=timestamp, ts=tasks spawned, bl=bytes live, etc.) for compact transfer.

Additional endpoints

Trace continuity and attribution

Start a trace reader without stream_id or after; the response begins at the oldest event still retained. Continue with the returned stream epoch and exclusive cursor:

curl -fsS 'http://127.0.0.1:6060/api/traces?stream_id=<stream_id>&after=<next_cursor>&limit=256'

Both cursor parameters must be supplied together. Reads are non-destructive, so each observer advances its own cursor without consuming another observer's events. has_more reports whether another retained page is available. Because retention is bounded, a slow reader can lose old events; a non-null overrun reports the exact missing sequence range and count.

producer_instance_id changes after a process restart, while stream_id changes when tracing is cleared, reset, or re-enabled. Treat a 409 trace_stream_mismatch as a new observation segment rather than applying an old cursor to a new stream.

Trace events carry nullable subject, source_actor, and destination_actor fields plus explicit attribution values. Local sends include source and destination identities when the runtime knows them; external or remote_unknown endpoints remain null. Handler names are optional, so retain the raw message type rather than inventing missing actor or handler metadata.

Event timestamp_ns values come from the producer's monotonic clock and are authoritative for ordering and durations only within that clock domain. clock_anchor provides an approximate Unix-time correlation with explicit uncertainty; it is not a cross-node clock-synchronization guarantee. Do not infer cross-node causality or missing identities from timestamp proximity.

What to look for

Mailbox backpressure

If an actor's mbox_depth is consistently high or mbox_hwm is much larger than expected, the actor is receiving messages faster than it can process them. Consider splitting work across multiple actors, using bounded mailboxes, or reducing message volume.

Hot actors

Sort the actors table by message count or total time to find bottlenecks. An actor with high time_ns but low msgs has expensive per-message processing. An actor with high msgs but low time_ns is fast but heavily trafficked.

Memory leaks

Watch the Memory panel's bytes-live chart. In a healthy program, bytes live should stabilize after startup. A steadily growing line suggests a leak — likely actors accumulating state or collections that are never freed.

Scheduler saturation

If active_workers equals the total worker count (CPU cores) continuously, the scheduler is saturated. A high steal rate indicates load imbalance across workers.

Performance impact

The profiling allocator adds one AtomicU64::fetch_add per allocation and deallocation (~5–15 ns on x86_64). Per-message dispatch timing adds one Instant::now() pair per message (~20 ns). These counters are always active regardless of whether HEW_PPROF is set — the overhead is negligible for most workloads.

The HTTP server, sampler thread, and actor registry are only active when HEW_PPROF is set.

Example

actor Counter {
    var count: i64;
 
    receive fn tick() {
        count = count + 1;
    }
 
    receive fn report() {
        println(f"Counter: {count}");
    }
}
 
fn main() {
    let counter = spawn Counter { count: 0 };
 
    for _ in 0..3000 {
        counter.tick().expect("actor stopped");
    }
 
    sleep(2s);
    counter.report().expect("actor stopped");
}

HEW_PPROF=:6060 hew run profiler_demo.hew
# [hew-pprof] dashboard at http://127.0.0.1:6060/
# Open http://localhost:6060/ to see live charts

Profile Export

The profiler can export profiles in two standard formats for use with external tools like go tool pprof and gprof.

pprof (Google protobuf format)

Download a pprof-compatible heap profile while the program is running:

curl -o heap.pb.gz http://localhost:6060/debug/pprof/heap

# Analyze with Go's pprof tool
go tool pprof heap.pb.gz

# Or the standalone pprof tool
pprof -top heap.pb.gz
pprof -text heap.pb.gz

The heap profile models each actor as a "function" with two value types: alloc_objects (messages processed) and alloc_space (estimated bytes). A [runtime] entry captures global allocator statistics.

Flat profile (gprof-style text)

View a human-readable flat profile directly in your terminal:

curl http://localhost:6060/debug/pprof/profile

Flat profile:

Total processing time: 3.2 ms across 3 actor(s)
Memory: 184.4 KB live (194.4 KB peak), 549 allocations

  %       cumulative   self                self       total
 time      time(ms)   time(ms)    calls   us/call    us/call    name
 62.5           2.0        2.0     2000       1.0        1.0    Actor#1 (pid=1, state=idle)
 25.0           2.8        0.8      800       1.0        1.0    Actor#2 (pid=2, state=idle)
 12.5           3.2        0.4      200       2.0        2.0    Actor#3 (pid=3, state=idle)

Mailbox summary:
   Actor     Depth      HWM
   #1            0      150
   #2            0       80
   #3            0       20

Automatic export on exit

Set HEW_PROF_OUTPUT to automatically write profile files when the program exits. This works independently of HEW_PPROF — no dashboard needed.

# Write pprof heap profile on exit
HEW_PROF_OUTPUT=pprof ./my_hew_program

# Write gprof-style flat profile on exit
HEW_PROF_OUTPUT=flat ./my_hew_program

# Write both files
HEW_PROF_OUTPUT=both ./my_hew_program

Files are written to the current working directory when the scheduler shuts down.