Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ graph ] Allow loss layer to be used in inference mode #2886

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

skykongkong8
Copy link
Member

  • Intention of the c5db575 commit was to completely remove the usage of loss and the label in inference.
  • However, there are cases where the loss layer is used in inference mode, such as the case of the loss layer being used as a metric which means that the formulation of the loss layer should not be determined by the execution mode, but the presence of the loss layer given by the user.
  • Thus, when loss layer is given when the model is formulated, the model will always have the loss layer, and it will expect to have a label vector later on (since the presence of the label vector cannot be detected on compile time.
  • On the contrary, when the loss layer is not given, the model will not have the loss layer, and it will not expect to have a label vector later on.

To summarize when inference mode,

loss label at inference time...
O O Loss layer is formulated, loss will be computed w.r.t. label
O X Loss layer is formulated, but no label -> emit error
X O Loss layer is not formulated, but label is given -> emit error
X X Loss layer is not formulated, label is empty as well -> inference w/o loss

Added TC will emit the output like below with model summary:

================================================================================
          Layer name          Layer type    Output dimension         Input layer
================================================================================
              input0               input           1:1:1:256                    
--------------------------------------------------------------------------------
    fully_connected0     fully_connected          1:1:1:1024              input0
--------------------------------------------------------------------------------
    fully_connected1     fully_connected          1:1:1:1024    fully_connected0
--------------------------------------------------------------------------------
    fully_connected2     fully_connected          1:1:1:1024    fully_connected1
--------------------------------------------------------------------------------
    fully_connected3     fully_connected           1:1:1:100    fully_connected2
--------------------------------------------------------------------------------
                mse0                 mse           1:1:1:100    fully_connected3
================================================================================

Self evaluation:

  1. Build test: [X]Passed [ ]Failed [ ]Skipped
  2. Run test: [X]Passed [ ]Failed [ ]Skipped

@skykongkong8 skykongkong8 changed the title [ graph ] Allow loss layer to be used in inference mode [ WIP ] [ graph ] Allow loss layer to be used in inference mode Jan 20, 2025
@skykongkong8 skykongkong8 force-pushed the pr/graph/loss_label_comb branch 2 times, most recently from c6ec999 to ce9615e Compare January 20, 2025 03:56
Copy link
Member

@DonghakPark DonghakPark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! especially Consider various case, add unittest and fix tflite export!! 👍

Copy link
Contributor

@EunjuYang EunjuYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@skykongkong8 skykongkong8 force-pushed the pr/graph/loss_label_comb branch from ce9615e to f7a15c6 Compare January 20, 2025 08:07
- Intention of the c5db575 commit was to completely remove the usage of loss and the label in inference.
- However, there are cases where the loss layer is used in inference mode, such as the case of the loss layer being used as a metric.
- Thus, when loss layer is given when the model is formulated, the model will always have the loss layer, and it will expect to have a label vector later on.
- On the contrary, when the loss layer is not given, the model will not have the loss layer, and it will not expect to have a label vector later on.

**Self evaluation:**
1. Build test:     [X]Passed [ ]Failed [ ]Skipped
2. Run test:     [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: skykongkong8 <ss.kong@samsung.com>
@skykongkong8 skykongkong8 force-pushed the pr/graph/loss_label_comb branch from f7a15c6 to 43e0804 Compare January 23, 2025 02:01
@skykongkong8 skykongkong8 changed the title [ WIP ] [ graph ] Allow loss layer to be used in inference mode [ graph ] Allow loss layer to be used in inference mode Jan 23, 2025
@skykongkong8 skykongkong8 self-assigned this Jan 23, 2025
- Current nntrainer inference mode does not consider label check as a mandatory process.

**Self evaluation:**
1. Build test:     [X]Passed [ ]Failed [ ]Skipped
2. Run test:     [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: skykongkong8 <ss.kong@samsung.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants