Hardware monitoring
We just added support to monitor hardware to our mobile-friendly web app for monitoring deep learning model training. You can use it to monitor hardware usage even if you are not training models.
Installation
You need to install our python package and some dependencies to start monitoring.
psutil is the package we use to track hardware (labml doesn't install it as a dependency).
To monitor GPUs (nvidia) you need to install py3nvml package as well.
You can install all of the above packages in a single command
pip install labml psutil py3nvml
If you have problems with your Python environment you can use our guide to setting up Python with conda locally, or on a remote computer.
Start monitoring
Once it's installed you can just run the following command to start monitoring. You will get a url to view the hardware usage.
You can also run it with nohup if you want it to run in the background on a remote computer.
Obtaining a token
This step is not necessary but useful if you are monitoring from time to time. By default when you run labml monitor it will give a URL to view the hardware usage. It can be annoying to get this url from the logs especially if you are running labml monitor on a remote computer.
In such cases you can obtain a token from app.labml.ai (Hamburger menu -> Settings). You can then add this to .labml/configs.yaml in your home directory.
Hosting your own server
labml monitor will start a process that monitors the hardware usage every few seconds and send the usage information to app.labml.ai by default. You can also host an instance of our app, by cloning the Github repository.
You can set the url of the server in .labml/configs.yaml.
web_api: https://hosted-labml-app.com
Join our Slack to discuss, and open an issue on Github if something breaks.
from Hacker News https://ift.tt/3c2zUYd
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.