This page provides an overview of the locations Open OnDemand writes logs and useful session data and provides context around these locations.
In general administrators will want to note whether they need to see:
System type logs for authentication to the web-node itself or OOD’s own configuration.
Session data for things like connecting to interactive apps (Jupyter, Rstudio, Codeserver, etc.) from the user’s PUN, which is already running on the web-node.
These logs will provide information around authentication and internal Open OnDemand errors. They are very useful when initially installing and configuring Open OnDemand.
For logs relating to issues about:
Authenticating to the web-node
There are two locations to check depending on what information is needed:
OnDemand will log to this location for its own configuration errors.
Where OnDemand will log succsseful logins.
There are no entries for failed logins.
The NGINX logs are the output of the user PUN. These logs will capture things relevant to a particular user such as:
Debuggin issues related to job submissions for a user. For example, commands being issued to the scheduler (
qsub, etc) can be seen here by searching for
Issues related to PUNs crashing and/or pages not rendering correctly.
These logs are located at:
These logs by default have
root:root ownership. It may be beneficial
chown an appropriate staff group on these files for troubleshooting down the road.
These files will provide information around connections between the user’s PUN and a compute-node.
These logs provide information around connections and are also the working directory of the job and location
stdout for a job (typically to
output.log). Other apps will also output
information they may need there as well for connections and errors.
The files will be owned by the user and so admins will need to ensure they are either able to substitue user or escalate to root in order to see these files.
In general the session data or job submission files for apps across the dashboard, such as the Job Composer, Batch Connect, or Frame-renderer, all start from the root of:
User session data for batch connect apps can be seen from a user’s home directory at:
This file is used for the session data presented on the interactive apps page and data used to connect to the batch connect app.
There may be more than one file in the
<session id> directory, but in the Interactive Apps
page you can match the Session ID you see there to the directory with the desired
One important thing to note is if trying to launch a Jupyter or Rstudio session and encountering failures, the
output.log would show you things like what modules are being loaded and what kernels are available.
Suppose a user is having trouble connecting to a Codeserver session they created. To see what data is being used by this batch connect app for the connection, look in:
cat ~/ondemand/data/sys/dashboard/batch_connect/sys/<app>/output/<session id>/output.log
This should result in output that will give the logging information around what happened as this session was started to include ports, address, app version, and the token used for the connection.
You would also see any
WARN messages as well which will likely be beneficial to debug failed
connections or launches.