* Upgrade to pytorch 1.8

* AdamW fix

* Remove trunc_normal

* Remove MixtureSameFamily

* Update doc requirements

* AdamW comment

* Remove unnecessary conversion

* Add torchtext

* Changed to python3.7; add torchtext

* Forwarding alf version number

* Fix rendering error in docker with python3.7

* The deadly triad for gym rendering: gym+pyglet+pillow

* More updates

* Add python3.7-dev in readme; minor carla docker update

* Minor fix

Sometime it is necessary to use it in order to make the ConvTraspose2D
to recover the original input size of the corresponding Conv2D.

* add alf.summary.images()

So we can add images to tensorboard, which is useful for showing the progress of image generating.

* alf.configurable for summary_utils.py

* Algorithm container

* Fixes for algorithm/contrainers.py

* Fix algorithms/containers and unittest

* loss_weight for LossAlg

* Loss for EncodingAlgorithm

* AlgorithmInterface

Unify interface for Algorithm and RLAlgorithm

* Update agent to use the new algorithm interface & fix EchoAlg bug

* Cleanup and update SacAlgorithm

* Update other algorithms

* Fix tests

* More tests and comments

* Address comments

* Address more review comments

* Fix comments

* add soft boundary transforms and functions

* address comments

* update

* fix mario example

* force frameformat to be channels_last

* simplify video recorder

* use np.full to create white images

* fix the bug of reward weight setting if checkpoint loaded

* create reward weights as buffers

* revert deletion

* * adding changes from linter. * squashing last few PR commits into one. * PR is still 1) uncertainty (auc) methods, 2) datagen refactoring, 3) minmax for functional particle VI.

* Fix some docstrings (#833)

* adding changes:
* adding test for eval_uncertainty
* removing some unused code for packages.
* created file sl_utils with loss functions and uncertainty related functions

* use random data instead of saving/loading mnist data for eval_uncertainty test, also incorporate changes from master

* * integrated alf configurable data_loaders
* fixed docstrings according to comments

Co-authored-by: emailweixu <emailweixu@gmail.com>

unnamed tuple and list are not correctly handled.
Also add more diagnositic information for map_structure and assert_same_structure errors.

* tasac algorithm

* fix summarizing value and return

* address comments and fix the Mode bug

* address more comments

* update

* Improve error reporting for alf.nest.get_field()

Previously, it is hard to find out what's wrong when get_field() fails.
This change make the error msg clearer by including all paths in the nest and the desired path.

Also improved the error reporting in config_util so that location of the error in config file will be displayed.

* Fix layers_test.py

Also:
1. implemented __repr__ for the schedulers
2. Fix issue int/int. Some of the progress value is int64 torch tensor, which can cause problem for division for LinearScheduler.

* dropout for TransformerBlock

Include dropout to conform with the original implementation in "Attention is all you need".
Though it seems to hurt the unittest and ppo_babyai.py example.

* Configurable activation for TransformerBlock

Also only use position embedding for the first tranformer block of TransformerNetwork, which is the common practice.

* Slight refactoring of trainer for supervised learning

Also adds an alf conf example of language modeling task for supervised learning.

* Fix hypernetwork_algorithm_test

* Fix policy_trainer_test

* Address review comments

* Add Conv2DBatchEnsemble layer and basic tests

* update docstring to address code reviews

When playing a trained model with alf snapshot, we should also set redirect the python path to its examples directory in case some conf files have been changed.

* Add support to speed limit penalty

* Address comments

* rendering interface

* image packing for saving image size

* fix dependency

* address comments

* Fix route length

* Some typo fixes

Previously, the docs of some of them are not generated. It seems to be caused
by the fact that some of those scripts try to define same flags when the module
is imported. So I moved the flags definitions into a function so that they are
defined only when the script is invoked.

* Utility to verify algorithm loaded from checkpoint is same as the original algorithm.

* Address comments

* Address comments

* support alf conf in grid_search

* address comments

* Perspective projection and visualization of world/ego points to camera image.

* Update

* Velocity in m/s

* Only render visible points

* Address comments

* Add option for waypoint rendering

* Fix another transpose

* More flexible Sequential

* Fix bug for NaiveParallelNetwork

* Address commants and allow Callable as elements of Sequential

* Fix accidentally inserted character

* fix use_entropy_reward for multi-step TD

* address comments

* more friendly option for snapshot

* using logging

* play alf snapshot

* typo

* fix train_play_test on CI

* address comments

* also redirect cnest.so path

* remove _alf_root global var

* also includes json files