How to globally set some parameters related to network optimization?

[lcy-seso]

Previously, we have some functions to globally set some parameters. In v2 API, the way to globally set these parameters changes a lot.

Can someone help me to make sure how each of them changes? Becuase these parameters are crucial to learning performance.

1. parameters moving into Optimizer (but they cannot be correctly recognized, a terrible BUG globally set parameters cannot work in V2 API #2488)

   default_momentum(val)
   default_decay_rate(val)
   default_gradient_clipping_threshold(val)

2. parameters I am not sure how to set.

   • It seems that the default initialization strategy becomes smart initialization,
   • but if I do not want it, for example, just use simple uniform initialization, how can I do to change the initialization method (and then only modify some parameters by hand), instead of modifying each parameter one by one.

   default_initial_smart(val)
   default_initial_std(val)
   default_initial_mean(val)
   default_initial_strategy(val)

3. parameters I not sure that they are still supported.

   • for training use both GPU and CPU default_device(val)

     • some layer in PaddlePaddle does not support running in GPU, for example, CRF layer. But running the entire model on CPU is really slow, in this situation, I hope to run the computation extensive parts of the model on GPU while the rest on CPU.
     • Previously, I can set a default device id, for example, GPU as default, and only change the device id of CRF layer to make it run on CPU, I am not sure how to run a model that some parts on GPU while other parts on GPU.

   • for model compression

   default_update_hooks(val)
   default_compact_func(val)

   • for cluster training

   default_num_batches_regularization(val)

[lcy-seso]

Hi, all, parsing the globally set parameters has bugs in v2 API, I am working on fixing this.