Oxelone Docs

Search docs and API

Find a guide, a section of one, or an API operation by its name.

Open console (opens in a new tab)

The agent Troubleshooting

Troubleshooting

Start in the console. A machine's page shows its state, and the installer on the machine tells you why it stopped. Everything below comes from what the console and the installer actually say.

A machine is Installing or Stuck

A machine is Installing from the moment it enrolls until its first heartbeat, then Active. If it enrolled and no heartbeat arrives within 15 minutes, it becomes Stuck. The console says Waiting for its first heartbeat for Installing and No heartbeat since enrolment for Stuck.

Enrolling is not the same as reporting. The machine was enrolled, so the token is spent, but the agent has not yet reached the console. Check, in this order:

  1. Is the agent running? On the machine, run the agent's status command. It prints the machine's identity, server, bundle hash, last heartbeat and health.

    Windows
    macOS
    Linux
  2. Can the machine reach the console? The agent calls the console's address, the one in the install command.

  3. Read the log. See "Logs on the machine" below.

To start again, create a new install command and follow "Reinstall" below.

States at a glance

StateWhat it meansWhat you can do
InstallingEnrolled, no heartbeat yetWait up to 15 minutes. Suspend or Delete are available.
StuckEnrolled, never checked in after 15 minutesCheck the agent as above. Suspend or Delete are available.
ActiveThe agent has checked inNothing
SuspendedThe agent captures and enforces nothingResume, or Delete
UninstalledThe agent was removed on the machine with its removal keyDelete. The record is kept for its history
ArchivedA Stuck or Uninstalled machine, hidden from the default list after 30 daysDelete

Choose a view in the machines list to see the machines in a state.

The installer stopped

The installer prints a box that names what stopped it, what to do next, and whether your token is still valid. Most stops happen before the token is spent, and say Your token is unspent.

The installer saysWhat it means and what to do
This PowerShell is not elevatedOpen PowerShell with Run as administrator and run the same command.
No systemdThe Linux installer registers a systemd unit and this machine does not run systemd. Install on a machine that does.
An agent is already installed on this machineThe installer will not replace it without being told to. Re-run with -Replace (Windows) or --replace (macOS and Linux), together with the machine's removal key.
Removing the existing agent needs a removal keyYour enrollment token does not give authority to remove an agent. Issue a removal key and add it to the command.
The existing agent was not removedThe reason is printed in the box. Re-running is safe.
The previous agent's Windows service is still registeredWindows only. The previous agent's service was not removed. Follow the box.
This machine cannot run the agent as a serviceOn macOS and Linux, the agent must sit in a directory only root can write. Use the default install directory.
No agent release to installNo build is published for this operating system. A platform admin has to publish a build pack and make it the default, then you run the same command again.
Enrollment failed with exit code …The agent's own message is printed above the box. Whether your token was spent depends on that message.
Service registration failed with exit code …Windows only. The service did not register.

If enrollment itself is refused with enrollment credential is not recognised, expired, revoked, or already used, the token was used once, is older than 24 hours, or its profile was revoked. Create a new install command. A reusable secret behaves the same way once its machine count or expiry is used up. See Enrolling many machines.

At the end of a good install, a box titled Installed lists anything that still needs a person under Needs attention: no tray icon, no tool hooks, or a certificate that is not trusted yet. A machine whose profile includes TLS interception reads Degraded until it trusts the interception certificate.

Logs on the machine

The agent writes a log only when it runs as the installed service. An agent that you start by hand logs to its own console instead. The log is in a logs folder inside the agent's data folder, and it rotates daily.

PlatformLog folder
Windows%ProgramData%\Oxelone\agent\logs
macOS/Library/Application Support/Oxelone/agent/logs
Linux/var/lib/oxelone/agent/logs

On macOS and Linux only root can read the folder.

Raise the log level

Open the machine and go to Configuration › Log level. The levels are error, warn, info, debug and trace. By default the machine follows its profile. Choose Set on this machine to override it.

A person at the machine can also set the OXEL_LOG environment variable, which overrides every other setting.

A full tracing directive in OXEL_LOG, such as ureq_proto=trace, writes raw network bytes, including the compiled policy, into the log. While it is set, the log is customer content, and a diagnostics bundle collected then carries it to the server. Unset the variable and delete the logs folder before anyone collects diagnostics. A plain level such as debug does not do this.

Diagnostics

Open the machine and choose its Diagnostics tab. It has four sections.

SectionWhat it does
Round tripTest round trip has the agent measure a real call to the server. It answers on the machine's next heartbeat. It reports the network time, and the control-channel time: how long the command waited for a heartbeat.
Collect diagnosticsCollect diagnostics asks the agent to send its local logs. The bundle arrives after its next heartbeat.
BundlesThe bundles received. Open one to see the collection, the log and the counters.
CrashesThe last crash the agent reported: when, on which thread, and the error.

Running a round trip or collecting diagnostics needs the fleet:manage permission. Neither is offered for a machine whose agent is gone, such as an Uninstalled or Archived one.

What a bundle holds

A bundle is deliberately small:

  • Counters: names with numbers or yes/no values, one level deep. No text.
  • A log tail: the most recent part of the log. The bundle is capped at 256 KiB, so a long log is cut to its tail, and the bundle says so.
  • The collection: when it was received and collected, the agent version, the last error as a category, and the policy bundle the machine had.

A machine keeps its ten most recent bundles. Reading a bundle's log is recorded, with who looked. Delete this bundle? is permanent and recorded.

If a bundle says The agent found no log files on the machine, the agent was not running as the installed service, because it writes a log file only as one.

Remove or reinstall

Remove an agent

Open the machine, then Lifecycle (under More), and choose Issue key. The removal key belongs to that one machine. It is shown once, and issuing a new key invalidates the one before it. Run the uninstall command on the machine itself, as an administrator. Deployment has the commands.

The console never tells an agent to remove itself. Removal happens on the machine, with the key.

If the console that the agent enrolled against no longer has any record of the machine, because its database was rebuilt, there is nowhere to issue a key from. The installer and uninstaller take --orphaned for that. It is not authority by itself: the agent asks the server in its own configuration and removes itself only if that server answers that it has no record of the machine. If the server cannot be reached, or answers anything else, nothing is removed.

Reinstall

An agent already on the machine stops the installer, so you remove or replace it:

  • To replace it in one step, re-run the install command with the removal key and -Replace or --replace. The previous agent is removed first, and the new enrollment is linked to the machine it replaces.
  • To start clean, uninstall with the key, Delete the old machine, and install again.

Suspend, resume and delete

Suspend stops the agent acting on the machine and puts back what it changed. It is reversible: Resume restores it, and the agent does so on its next heartbeat, within a minute. Use it instead of removing an agent when you want a pause.

Delete is the final step, and is offered for a machine that is Suspended, Stuck, Installing, Uninstalled or Archived. It removes the machine from the console. If its agent comes back online, the agent removes itself. It cannot be undone, and the audit trail keeps who deleted it and when. A reinstall on that computer enrolls as a new machine, which can name the deleted one as the machine it replaces.

Changing a machine's lifecycle needs the fleet:manage permission.

Still stuck

Use New ticket in the account menu to tell Oxelone about a Bug or a Question. Copy diagnostics in the Help menu puts what the ticket would carry about the page you are on on the clipboard. Read any log text before you paste it into a ticket: a log can hold customer content if OXEL_LOG was set. See Tickets.