# Monitoring and troubleshooting

<div style="text-align: justify;">

The headless service has no window, so it reports through log files, the Windows event log and an optional local health endpoint. For support cases it bundles everything into one diagnostic package.

---

<h3 id="log-files" style="color: #203671; margin-top: 2.2em;">Log files</h3>

```
data\logs\veloxprint-20260925.log
data\logs\veloxprint-20260925_001.log        same day, after the size limit
data\logs\veloxprint-20260925.json           optional, logging.jsonFile
data\logs\lager-nord\veloxprint-...          instance with --config lager-nord.json
```

```
2026-09-25 07:16:48.120 +02:00 [INF] [Print] [#35] Task #35 received (WebSocket, Standard) -> WH-LABEL-01, 1 copy
2026-09-25 07:16:49.402 +02:00 [INF] [Print] [#35] Task #35 spooled to WH-LABEL-01, 1 page
2026-09-25 07:16:49.655 +02:00 [INF] [API] Task #35 confirmed as printed
```

Format: time with offset, level (`VRB`, `DBG`, `INF`, `WRN`, `ERR`, `FTL`), area (`API`, `WS`, `Print`, `Engine`), task id where known, message.

1. One file per day, a new file (`_001`, `_002`) when `logging.maxFileSizeMb` (250 MB) is reached.
2. Files older than `logging.retentionDays` (30) are deleted, by age, not by count.
3. Lines of a task carry its task id and trace id; the trace id links it to the history record and print task in VeloxFactory.
4. The API token never appears in any log.

---

<h3 id="event-log" style="color: #203671; margin-top: 2.2em;">Windows event log</h3>

Warnings and errors go to the event log *Application*, source **VeloxFactory Print Service**, with fixed event ids for monitoring systems such as Checkmk, PRTG, Zabbix or SCOM. The source is registered when the scheduled task is set up; without it the event log output stays off. Switch it off with `logging.eventLog`.

<table style="width: 100%; border-collapse: collapse;">
  <thead>
    <tr style="border-top: 1px solid #e6e8ef; border-bottom: 1px solid #e6e8ef;">
      <th style="text-align: left; padding: 6px 10px; white-space: nowrap;">Event id</th>
      <th style="text-align: left; padding: 6px 10px;">Meaning</th>
      <th style="text-align: left; padding: 6px 10px;">Action</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">1000</td><td style="padding: 6px 10px;">Engine started</td><td style="padding: 6px 10px;"></td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">1001</td><td style="padding: 6px 10px;">Engine stopped</td><td style="padding: 6px 10px;">Unexpected outside updates and reboots</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">2000</td><td style="padding: 6px 10px;">API not reachable</td><td style="padding: 6px 10px;">Network, DNS, proxy, VeloxFactory up?</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">2001</td><td style="padding: 6px 10px;">API token rejected (401/403)</td><td style="padding: 6px 10px;">Token revoked or rights missing, enter a new token</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">2002</td><td style="padding: 6px 10px;">Rate limit reached (429)</td><td style="padding: 6px 10px;">Occasionally harmless. Often: raise <code>API_RATE_LIMIT_PER_MINUTE</code> or the poll intervals</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">3000</td><td style="padding: 6px 10px;">WebSocket down longer than 5 minutes</td><td style="padding: 6px 10px;">Reverb, firewall, host, port, app key. Hybrid keeps printing by polling.</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">4000</td><td style="padding: 6px 10px;">Print failed</td><td style="padding: 6px 10px;">Error text contains the category</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">4001</td><td style="padding: 6px 10px;">Printer missing</td><td style="padding: 6px 10px;">Install machine-wide or set an alias</td></tr>
    <tr><td style="padding: 6px 10px;">5000</td><td style="padding: 6px 10px;">Configuration invalid</td><td style="padding: 6px 10px;">A changed configuration was not applied, the engine keeps the last valid one</td></tr>
  </tbody>
</table>

---

<h3 id="health" style="color: #203671; margin-top: 2.2em;">Local health endpoint</h3>

Optional, switched on under **Settings → Operation** (`health.enabled`, port `health.port`, default 8765). It listens only on `127.0.0.1`, needs no URL reservation and no administrator rights, and is not reachable from other machines. A monitoring agent on the same machine polls it.

```
GET http://localhost:8765/health
```

```json
{
  "status": "ok",
  "engine": "Running",
  "hostMode": "Headless",
  "version": "2.0.1",
  "startedAt": "2026-09-25T05:01:12+00:00",
  "api": {
    "state": "Ok",
    "lastApiSuccess": "2026-09-25T07:16:49+00:00",
    "lastApiError": null,
    "rateLimit": { "limit": 60, "remaining": 58, "rateLimitedUntil": null }
  },
  "websocket": { "state": "Subscribed", "since": "2026-09-25T05:01:14+00:00", "reconnectAttempt": 0 },
  "queue": { "active": 0, "waiting": 0, "retrying": 0, "outbox": 0 },
  "printers": { "ok": 3, "withProblems": 0 }
}
```

<table style="width: 100%; border-collapse: collapse;">
  <thead>
    <tr style="border-top: 1px solid #e6e8ef; border-bottom: 1px solid #e6e8ef;">
      <th style="text-align: left; padding: 6px 10px; white-space: nowrap;">HTTP status</th>
      <th style="text-align: left; padding: 6px 10px;">Condition</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">200, <code>"status": "ok"</code></td><td style="padding: 6px 10px;">Engine running or paused, and API state OK, unknown or rate limited</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px;">503, <code>"status": "degraded"</code></td><td style="padding: 6px 10px;">Everything else, e.g. token rejected or API unreachable</td></tr>
    <tr><td style="padding: 6px 10px;">no answer</td><td style="padding: 6px 10px;">Engine not running</td></tr>
  </tbody>
</table>

Worth alerting on besides the status: `queue.outbox` above 0 for a longer time (printed but not confirmed), `queue.retrying` above 0 (printer problem), `printers.withProblems` above 0, `websocket.state` not `Subscribed`.

---

<h3 id="support-package" style="color: #203671; margin-top: 2.2em;">Diagnostic package</h3>

**Info → Create support package**, or `--support-package C:\Temp\support.zip`. The ZIP contains:

<table style="width: 100%; border-collapse: collapse;">
  <tbody>
    <tr style="border-top: 1px solid #e6e8ef; border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>versions.txt</code></td><td style="padding: 6px 10px;">Program version, machine, account, Windows, .NET, configuration path, whether a token is stored</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>config.json</code></td><td style="padding: 6px 10px;">Configuration, anything resembling a token removed</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>engine-status.json</code></td><td style="padding: 6px 10px;">Last engine status (as in the health endpoint)</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>printers.txt</code></td><td style="padding: 6px 10px;">Local printers with scope, queue state, jobs, driver and driver version, port, dialog flag</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>journal-last-500.csv</code></td><td style="padding: 6px 10px;">The last 500 jobs of the journal</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>outbox.csv</code></td><td style="padding: 6px 10px;">Open confirmations and error reports: <code>task id;kind;attempts;next attempt;last error</code>. Empty is the normal case.</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>config-changes.csv</code></td><td style="padding: 6px 10px;">Last 500 configuration changes: time, account, changed keys</td></tr>
    <tr style="border-bottom: 1px solid #e6e8ef;"><td style="padding: 6px 10px; white-space: nowrap;"><code>scheduled-task.txt</code></td><td style="padding: 6px 10px;">XML definition of the scheduled task</td></tr>
    <tr><td style="padding: 6px 10px; white-space: nowrap;"><code>logs/</code></td><td style="padding: 6px 10px;">The 7 newest log files</td></tr>
  </tbody>
</table>

The API token is removed from every file.

---

<h3 id="problems" style="color: #203671; margin-top: 2.2em;">Common problems</h3>

<table style="width: 100%; border-collapse: collapse;">
  <thead>
    <tr style="border-top: 1px solid #e6e8ef; border-bottom: 1px solid #e6e8ef;">
      <th style="text-align: left; padding: 6px 10px;">Symptom</th>
      <th style="text-align: left; padding: 6px 10px;">Cause and fix</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Tasks stay pending, nothing in the log</td>
      <td style="padding: 6px 10px;">Broadcast id of the task is not in <code>receive.broadcastIds</code>, or the task has none and <code>includeWithoutBroadcastId</code> is off. Processing paused? Engine running (dashboard, Task Scheduler)?</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Tasks are printed, but only after a minute or two</td>
      <td style="padding: 6px 10px;">Tasks without broadcast id: VeloxFactory sends no event for them, only the poll finds them. Give them a broadcast id. Otherwise the WebSocket is not subscribed: check the WebSocket tile and <strong>Test connection</strong> (host, port, scheme, app key, firewall). A token without <code>report-print-task:read</code> fails at the channel authorization.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Prints from the window, fails from the scheduled task with <code>PrinterNotFound</code></td>
      <td style="padding: 6px 10px;">Printer only connected for your user profile. Install it machine-wide (<a href="/books/veloxfactory/page/printers-and-printing">Printers and printing</a>).</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Network printer fails from the scheduled task</td>
      <td style="padding: 6px 10px;">Task set up without stored password, or a local account against a domain print server. Use a domain account and set up the task again.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Error <code>[Timeout]</code> after 120 s</td>
      <td style="padding: 6px 10px;">Printer opens a dialog (PDF printers, OneNote, fax), or the driver hangs. Check the hint on the Printers page.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Only one copy instead of n</td>
      <td style="padding: 6px 10px;">Driver ignores the copy count. Set <em>Copies</em> to <em>One job per copy</em> for this printer.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Output shifted, cut off or too small</td>
      <td style="padding: 6px 10px;">Check scaling (<em>Actual size</em> for labels), paper size of the driver and auto rotate. The frame of the test page shows margins and scaling.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Very thin or missing lines</td>
      <td style="padding: 6px 10px;">Lines with width 0 in the report. Set <code>lineWidth</code> in the JRXML.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Printed, but pending in VeloxFactory</td>
      <td style="padding: 6px 10px;">Confirmation in the outbox (Queue tile <em>to confirm</em>, <code>outbox.csv</code>). The column <em>last error</em> names the reason. The task is not printed again, the confirmation follows as soon as the API answers.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Task shows error <em>Print state unknown after service restart</em></td>
      <td style="padding: 6px 10px;">The service ended while the task was printing. Check the printer; if nothing came out, use <strong>Print again</strong>.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Many 429 / event 2002</td>
      <td style="padding: 6px 10px;">Several services or scripts behind one IP share the limit. Raise <code>API_RATE_LIMIT_PER_MINUTE</code>, use Hybrid instead of short polling intervals.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Scheduled task shows result 0x1 / 0x3</td>
      <td style="padding: 6px 10px;">Exit code 1 configuration invalid, 3 token rejected. Start the window or <code>--validate-config</code>.</td>
    </tr>
    <tr style="border-bottom: 1px solid #e6e8ef;">
      <td style="padding: 6px 10px;">Window cannot save settings</td>
      <td style="padding: 6px 10px;">No write access to the data folder. Use <strong>Restart as administrator</strong>.</td>
    </tr>
    <tr>
      <td style="padding: 6px 10px;"><code>update.ps1</code> refuses to update</td>
      <td style="padding: 6px 10px;">Local changes in <code>app\</code>, or a window of the Print Service is still open under another user. Close it and run the update again.</td>
    </tr>
  </tbody>
</table>

</div>