Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

预测时候, 遇到编译protobuf失败, 配置文件需要显式指定参数的lr和momentum #356

Closed
linrongyi opened this issue Nov 4, 2016 · 4 comments
Assignees
Labels

Comments

@linrongyi
Copy link

linrongyi commented Nov 4, 2016

用python wrapper执行预测的时候, 编译模型的conf出错.

在执行swig_paddle.GradientMachine.createFromConfigProto(conf.model_config)出出错, 提示

I1104 19:07:32.537729 17412 Util.cpp:126] Call runInitFunctions done.
[libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "paddle.ModelConfig" because it is missing required fields: parameters[14].learning_rate, parameters[14].m
omentum, parameters[16].learning_rate, parameters[16].momentum, parameters[17].learning_rate, parameters[17].momentum, parameters[18].learning_rate, parameters[18].momentum, parameters[19].l
earning_rate, parameters[19].momentum, parameters[20].learning_rate, parameters[20].momentum, parameters[21].learning_rate, parameters[21].momentum

我的Layer是

Layer(#
    name = "query_vec",
    type = "fc",
    active_type='',
    size = TERM_EMB_DIM,
    bias = False,
    inputs = Input("query",
    initial_std = 1.0 / math.sqrt(TERM_TOK_NUM) / 3.0,
    parameter_name = "_term_emb",
    sparse_update = True,
    sparse_remote_update = True)
)

如果加上 momentum, learning_rate 就好了, 也就是

Layer(#
    name = "query_vec",
    type = "fc",
    active_type='',
    size = TERM_EMB_DIM,
    bias = False,
    inputs = Input("query",
    initial_std = 1.0 / math.sqrt(TERM_TOK_NUM) / 3.0,
    parameter_name = "_term_emb",
    sparse_update = True,
    momentum = 0.0, learning_rate = 0.,
    sparse_remote_update = True)
)
@linrongyi linrongyi changed the title 预测时候, 遇到编译protobuf失败, 配置文件需要显示指定参数的lr和momentum 预测时候, 遇到编译protobuf失败, 配置文件需要显式指定参数的lr和momentum Nov 4, 2016
@reyoung
Copy link
Collaborator

reyoung commented Nov 5, 2016

@linrongyi 这里倒是没有任何限制。是因为在系统里混合使用了不同版本的Paddle。

这个错误是因为,你用了混合使用了 之前编译的py_paddle现在使用的paddle python包,即paddle包和py_paddle包二者版本不一致。请先将两个python包卸载,然后重新编译确保两个python包的版本一致。

卸载方式是 pip uninstall -y paddle py_paddle

这个事情引入的commit是 2c5a6ac ,原因是之前Paddle其实是在应该使用optional+default value的protobuf结构中,显示的赋值导致default value这个功能并没有使用。原来那么做的问题是会生成一个更大的protobuf结构,导致网络配置的单元测试看起来比较难看,同时也多占用了一些空间。

@backyes
Copy link
Contributor

backyes commented Nov 5, 2016

@linrongyi 插一句, 内部一键编译脚本编译出的paddle,应该不会出现这种python包不一致的问题,因为它是完全的安装不依赖系统python环境,每次编译安装将自动彻底清除老的安装目录(如果存在的话)。所以,可以试试一键编译脚本

@linrongyi
Copy link
Author

多谢 @reyoung @backyes , 问题已经搞定, 是因为在编译的时候swig_paddle是关闭的.
但是, 现在用paddle 预测工具执行predict 还是没有成功, 我们在另外一个ISSUE讨论.

@reyoung
Copy link
Collaborator

reyoung commented Nov 5, 2016

@linrongyi It will be nice if you want to add this situation to Paddle FAQ documentation. If you do not have time to do this, please let us know. We will add them to FAQ.

thisjiang pushed a commit to thisjiang/Paddle that referenced this issue Oct 28, 2021
wangxicoding pushed a commit to wangxicoding/Paddle that referenced this issue Dec 9, 2021
…rt-tokenizer-bos-path

Update distilbert tokenizer bos path
AnnaTrainingG pushed a commit to AnnaTrainingG/Paddle that referenced this issue Sep 19, 2022
danleifeng pushed a commit to danleifeng/Paddle that referenced this issue Sep 13, 2023
Co-authored-by: root <root@yq01-sys-hic-v100-box-a225-0610.yq01.baidu.com>
zmxdream pushed a commit to zmxdream/Paddle that referenced this issue Sep 19, 2023
Co-authored-by: root <root@yq01-sys-hic-v100-box-a225-0610.yq01.baidu.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants