Direct answer: compare CPU time and CPU rate only on a common interval
The monitor records a monitor_windows.cpu_ms aggregate by summing per-process process_cpu_ms_delta over the matched process family. The detailed process_samples.cpu_percent field instead calls process_cpu_percent, converting accumulated CPU ticks into a rate relative to elapsed monotonic wall time. Both depend on snapshots; neither is a kernel event ledger for every instruction executed by an agent.
This is a source audit of AgentSight v1.0.35 at commit c1574366122a90c451bca6a84713e28813ab95e2, not a new runtime benchmark. The calculation is especially important when a new process enters a sampled family, a short-lived subprocess falls between samples, or process IDs are reused.
The existing background monitoring guide owns the cadence and database architecture. This page covers the distinct engineering question of how to interpret the numeric resource fields and detect incomparable baselines.
Follow the two different CPU formulas
The process snapshot is built with sysinfo process data. Accumulated CPU milliseconds are converted to ticks using the platform clock-tick rate. For a process seen in both snapshots with matching starttime_ticks, process_cpu_ms_delta subtracts ticks, converts back to milliseconds, and rounds. When either the previous snapshot or matching process identity is missing, it returns zero.
For CPU percentage, process_cpu_percent looks up the old process by PID and divides the nonnegative difference in CPU ticks by measured elapsed wall time, then multiplies by 100. It does not normalize by host CPU count. A multithreaded process can therefore exceed 100%.
cpu_ms ~= round(delta_cpu_ticks / clock_ticks_per_second * 1000)
cpu_percent ~= (delta_cpu_ticks / clock_ticks_per_second) / delta_wall_seconds * 100The code paths diverge when no previous PID row exists. CPU milliseconds return zero, while CPU percentage falls back to lifetime average CPU use: accumulated ticks divided by elapsed process age (with a minimum age of 0.001 seconds). That rate is not directly comparable with an adjacent two-second CPU-time delta. The precise measurement window also depends on the real snapshot timestamps, not just the configured loop interval.
A subtle identity mismatch: PID reuse
The v1.0.35 CPU-millisecond delta explicitly rejects a previous PID whose starttime_ticks differs from the current process. The read/write byte-delta path is even more explicit: its prior-state map uses a ProcessKey composed of PID and start time.
The percentage function only checks that the previous snapshot contains the same numeric PID. If that PID was recycled for another process, its accumulated CPU ticks can be subtracted from the new occupant. A lower current counter is clamped to zero by saturating_sub; a higher current counter may produce a nonrepresentative rate. This is a code-level corner casedemonstrated by the guard mismatch, not evidence that a particular production run was affected. A safe downstream comparison keys process observations by PID and start time.
Nine source-derived interpretation cases
| Observation or synthetic input | Released source rule | Safe inference |
|---|---|---|
| Stable PID accumulates 400 ms of CPU time across a 2-second snapshot interval. | CPU delta uses the change in accumulated ticks; CPU% divides the CPU-seconds delta by measured wall seconds. | About 400 cpu_ms and 20% for that process, subject to tick rounding and actual sample interval. |
| Stable multithreaded process consumes 4 CPU-seconds during 2 wall seconds. | The CPU% expression multiplies CPU-seconds / wall-seconds by 100 without dividing by logical CPU count. | A result around 200% is possible; it is not automatically a corrupt reading. |
| A process appears in the current snapshot but not the previous snapshot. | process_cpu_ms_delta returns zero; process_cpu_percent falls back to lifetime ticks divided by process age. | The same first observation may have zero sampled CPU milliseconds and nonzero CPU%. They have different baselines. |
| A PID is reused between two snapshots with a different process start time. | CPU-ms delta checks starttime_ticks; CPU% matches the previous process only by PID. | The rate path can compare unrelated accumulated counters. This is a source-level identity risk, not proof of a witnessed failure. |
| A reused PID has fewer accumulated ticks than its old occupant. | CPU% subtracts the older counter with saturating_sub, potentially producing zero. | A zero rate in this edge case need not mean the new process did no work. |
| A process begins with no previous disk I/O baseline. | process_io_delta uses (pid, starttime_ticks) and returns zero when no prior key exists. | Zero read/write delta on the first observation is not evidence of no earlier lifetime I/O. |
| Two related processes map some of the same physical pages. | Monitor session rss_bytes is the saturating sum of sampled per-process resident bytes. | The family sum is not unique physical memory consumed by the session; shared pages may be counted more than once. |
| A 2-second monitor window has metrics but no process_samples rows. | Aggregate windows are inserted each tick; detailed process samples only cross 30-second boundaries. | Missing detail is expected sampling policy, not necessarily a zero-process or lost-collection result. |
| A session contains more than ten processes at a detail boundary. | bounded_process_samples chooses up to five highest and five lowest by CPU-ms, then RSS and I/O. | Detailed rows are a ranking sample, not an exhaustive process census or top ten by CPU%. |
The arithmetic inputs and PID-reuse cases are synthetic examples, not observed customer traces, field incidence estimates, or benchmark results. Tick conversion and scheduling can introduce rounding, while incomplete process observation can hide short-lived work.
Memory and I/O tell different stories again
The monitor converts each process RSS into bytes and adds those values across the matched process family. This is useful for locating a large family, but shared resident pages can be represented in multiple per-process RSS numbers. The summed rss_bytes is therefore not a precise unique-physical- memory measurement or a cgroup memory-usage counter.
Monitor read_bytes and write_bytes come from sampled process disk-usage counters. The monitor stores saturating differences between consecutive observations keyed by PID and start time. A process first seen on the current tick gets zero delta; that does not establish no I/O since process launch. These fields also do not represent every file operation, read syscall, or network byte transferred.
Per-session aggregate windows are written on the two-second monitor loop. Persisted process detail is selected only at 30-second crossings. If there are more than ten processes, the retained detail selects up to five highest and five lowest by a score ordered by CPU milliseconds, RSS, then I/O. It is not a complete inventory and is not sorted primarily by CPU%.
Reproduce the source inspection
These commands inspect the exact released implementation; they do not launch a monitor, read private session artifacts, or assume synthetic values were measured. Run them in a local checkout of the AgentSight product repository.
git checkout c1574366122a90c451bca6a84713e28813ab95e2
grep -n -E "fn process_cpu_percent|fn process_cpu_ms_delta|starttime_ticks|delta_wall" agentsight-capture/src/sources/proc.rs
grep -n -E "MONITOR_INTERVAL_SECS|DETAIL_SAMPLE_INTERVAL_SECS|fn aggregate_process_samples|fn process_io_delta|fn bounded_process_samples" collector/src/cmd_monitor.rs
grep -n -E "monitor_windows|process_samples|cpu_percent|rss_bytes" collector/src/cmd_monitor.rsTo evaluate an actual resource hotspot, compare consecutive complete snapshots for the same (pid, starttime_ticks), inspect elapsed wall time, and distinguish short-lived descendants from the currently sampled family. Preserve a provider-native agent timeline if you need to attribute the workload to a prompt or tool decision. A high CPU% does not identify which agent instruction caused the load.
Sources, scope, and limits
- Process snapshot CPU formula, first-sample fallback, PID and start-time behavior
- Monitor sampling, aggregation, I/O keys, ranking, and persisted SQLite fields
- AgentSight v1.0.35 release
Reviewed 11 November 2026 against the tagged v1.0.35 source. The finding applies to the snapshot-backed monitor code paths described above, not every CPU metric in the AgentSight UI, a complete system-wide CPU usage accounting, or a guarantee of process lifetime coverage. Future versions may change identity guards or metric semantics; pin the observed version when exporting or comparing historical monitor databases.