You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for your code for the project! It is a really nice work!
I am confused about why using zero_module, may lead to the zero_grad between the input and the output. It is possible to correctly train the model parameter with the expected grad?
Is it true that zero module is the cause of zero grad? I'm not sure about this.
By the way, we used zero grad module based on a previous work, but by itself, it also has a positive impact faster learning as well (as shown in the previous works).
Activity
konpatp commentedon Sep 28, 2023
Is it true that zero module is the cause of zero grad? I'm not sure about this.
By the way, we used zero grad module based on a previous work, but by itself, it also has a positive impact faster learning as well (as shown in the previous works).
ZhangMingKun1 commentedon Sep 29, 2023
Thank you for your feedback! Could you please provide the paper title or GitHub link of the "previous works"?