[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem on reproducing results of RNN on PTB #61

Open
tonystark940501 opened this issue Dec 26, 2018 · 2 comments
Open

Problem on reproducing results of RNN on PTB #61

tonystark940501 opened this issue Dec 26, 2018 · 2 comments

Comments

@tonystark940501
Copy link

Hi @quark0 , thanks for releasing the code. Really enjoy your paper. Here I have a problem and hope to get your help. I run the search code with 5 different random seeds, including the default seed and get 5 different RNN architectures. However, when I train these 5 architectures respectively from scratch, I get test ppl of 57.16, 61, 57.81, 60.99 and 57.53. None of them get a test ppl around 56. Is there anything I missed to get a robust result of 56.1 or 55.8 in the paper?
@quark0
Copy link
Owner
quark0 commented Dec 26, 2018 •
edited
Loading

You probably need to adjust the hyperparameters for the final evaluation. The default hyperparameters were tuned wrt the provided genotype but are likely suboptimal for the new architectures.

@tonystark940501
Copy link
Author

@quark0 Thanks for your response! Are there any suggestions for tunning hyperparams? Like what hyper parameters needed to be tuned, and the range. I see there`re a lot of hyper parameters needed to be tuned, including four dropout. Is it hard or expensive to do so?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@quark0 @tonystark940501