You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I guess it's due to the optimizer setting, since I modified the update rule to: optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
the loss decreases dramatically.
@kopxiong It makes sense to change the hyperparameters. I've tried your suggestion, but the loss didn't drop to a satisfying level. Have you checked the y_pred of the final model?
@Andy-jqa @hunkim Sorry for the late reply. I think we can improve the model's performance from the following aspects:
add more layers or increase the number of nodes in each layer, since more units mean more powerful representation capability, like super(Model, self).__init__() self.l1 = torch.nn.Linear(8, 32) #(8, 6) self.l2 = torch.nn.Linear(32, 16) #(6, 4) self.l3 = torch.nn.Linear(16, 1) #(4, 1)
use Adam optimizer to RMSProp instead of SGD
use more epochs for training (like 2000 or more)
sigmoid activation function maybe not good, try ReLU or LeakyReLU (I tried ReLU, but didn't work)
maybe we also should add some regularization terms to avoid overfitting.
Activity
kopxiong commentedon Oct 13, 2017
I guess it's due to the optimizer setting, since I modified the update rule to:
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)the loss decreases dramatically.
drqiaojin commentedon Oct 13, 2017
@kopxiong It makes sense to change the hyperparameters. I've tried your suggestion, but the loss didn't drop to a satisfying level. Have you checked the
y_predof the final model?hunkim commentedon Oct 15, 2017
@kopxiong Any thoughts? Since the goal of these tutorials is understanding the concept, I did not pay attention to the accuracy.
kopxiong commentedon Oct 18, 2017
@Andy-jqa @hunkim Sorry for the late reply. I think we can improve the model's performance from the following aspects:
super(Model, self).__init__()self.l1 = torch.nn.Linear(8, 32) #(8, 6)self.l2 = torch.nn.Linear(32, 16) #(6, 4)self.l3 = torch.nn.Linear(16, 1) #(4, 1)maybe we also should add some regularization terms to avoid overfitting.
akramsystems commentedon Jun 18, 2018
set lr = 8 and epoch i set to 10000
got around 2% error
akramsystems commentedon Jun 18, 2018
also made more nodes in the hidden layer so i set my layers to
torch.nn.Linear(8, 30)
torch.nn.Linear(30, 10)
torch.nn.Linear(10, 1)
elcolie commentedon Dec 7, 2018
I can not get
lossbelow than0.6Any ideas?
file.zip