Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[BUG] fix SymbolBlock constructor from user-defined graph #21044

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

khaotik
Copy link
Contributor

@khaotik khaotik commented May 29, 2022

Description

Fixes a bug where SymbolBlock does not copy user-given parameter attributes such as lr_mult. Simple reproducer:

#!/usr/bin/env python

import numpy as np
import mxnet as mx
import mxnet.symbol as mxs

DTYPE = np.float32
LR_MULT = 0.555
WD_MULT = 0.444

s_x = mxs.var('x', shape=(1,256,), dtype=DTYPE)
s_w = mxs.var('W', shape=(256,192), lr_mult=LR_MULT, wd_mult=WD_MULT, dtype=DTYPE)
s_b = mxs.var('b', shape=(1,192,), dtype=DTYPE, init=mx.init.Zero())
s_y = mxs.linalg.gemm(s_x, s_w, s_b)

fn = mx.gluon.SymbolBlock([s_y], [s_x])
fn.initialize()
fn.forward(mx.nd.random_uniform(-1., 1., shape=(1,256), dtype=DTYPE))

param_di = fn.collect_params()

assert param_di['W'].lr_mult == LR_MULT
assert param_di['W'].wd_mult == WD_MULT
assert not param_di['b'].data().asnumpy().any()

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • SymbolBlock now copies user-given wd_mult lr_mult init attributes from symbol.
  • stype attribute

Comments

  • stype attribute is still not copied. There's no MXSymbolInferStorageType C API. I think a complete fix would require some C API mods.

@khaotik khaotik requested a review from szha as a code owner May 29, 2022 01:46
@mxnet-bot
Copy link

Hey @khaotik , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [unix-cpu, unix-gpu, centos-gpu, centos-cpu, sanity, miscellaneous, windows-cpu, edge, clang, website, windows-gpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels May 29, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels May 29, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels May 30, 2022
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels May 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants