Skip to content
Snippets Groups Projects
Unverified Commit 6ff31512 authored by emailweixu's avatar emailweixu Committed by GitHub
Browse files

Minor refactoring for supervised training (#840)

* dropout for TransformerBlock

Include dropout to conform with the original implementation in "Attention is all you need".
Though it seems to hurt the unittest and ppo_babyai.py example.

* Configurable activation for TransformerBlock

Also only use position embedding for the first tranformer block of TransformerNetwork, which is the common practice.

* Slight refactoring of trainer for supervised learning

Also adds an alf conf example of language modeling task for supervised learning.

* Fix hypernetwork_algorithm_test

* Fix policy_trainer_test

* Address review comments
parent d9a75079
No related branches found
No related tags found
No related merge requests found
Showing
with 340 additions and 167 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment