-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
请问如何打印出训练过程中还有多少显存? #352
Comments
因为这边我想打印出现存还剩多少,以此来增加bachsize提高显存利用率来达到加快训练速度的效果。 |
请问若要断点续训如何操作呢? 这边我没有查看到相关参数,望解答一下,谢谢。
|
代码仓:https://github.com/mindspore-lab/mindyolo/tree/v0.3.0
环境:modelart、mindspore_2.2.12-cann_7.0.1.1-py_3.9-euler_2.10.7-aarch64-snt3p、
问题描述:
在使用过程中发现1000+图片、bachsize=64、num_parallel_workers=8时,一个epoch训练时长达到2.24分钟左右,请问如何打印出我还剩多少显存?以及训练一轮2分多时间正常么?若不正常请问有哪些解决方法呢?
请回复下,谢谢!
终端日志如下所示:
The text was updated successfully, but these errors were encountered: