Skip to content
This repository has been archived by the owner on Aug 4, 2022. It is now read-only.

"Exported" model performs differently in custom frontend #79

Open
ghost opened this issue May 20, 2020 · 0 comments
Open

"Exported" model performs differently in custom frontend #79

ghost opened this issue May 20, 2020 · 0 comments

Comments

@ghost
Copy link

ghost commented May 20, 2020

Describe the bug
I use the nlp.js-app because it is a good interface to add entities, intents and slots. It's much easier than training it via the source code.

I add a manager.save() line in the server\trainers\nlpjs-process.js file so it exports the model as a model.nlp file when I train the model in the app. It's a stupid solution but the app doesn't allow exporting except in a .csv file that is totally incomplete.

Anyway, I want to use the model in my own chatbot application. For the frontend I am using vue-bot-ui
The user input is basically just fed into manager.process but upon boot the manager is of course trained with the exported model.

Please see the exact same conversation both in the app and then in my web application:
nlp

The slot filling doesn't work in my application. What you can see from the logs is that when I input "UPS" after the tracking number, you won't find the tracking number entity in the entities array. It won't remember the filled slot:

{ message: '723747234' }
{
  utterance: '723747234',
  locale: 'en',
  languageGuessed: false,
  localeIso2: 'en',
  language: 'English',
  domain: 'EN',
  classifications: [
    { label: 'parcel.damaged', value: 1 },
    { label: 'helpcenter-redirect', value: 0 },
    { label: 'integration-error', value: 0 },
    { label: 'None', value: 0 }
  ],
  intent: 'parcel.damaged',
  score: 1,
  entities: [
    {
      start: 0,
      end: 9,
      accuracy: 1,
      sourceText: '723747234',
      utteranceText: '723747234',
      entity: 'tracking number'
    }
  ],
  sourceEntities: [
...
    }
  ],
  sentiment: {
    score: 0,
    comparative: 0,
    vote: 'neutral',
    numWords: 1,
    numHits: 0,
    type: 'senticon',
    language: 'en'
  },
  actions: [],
  srcAnswer: 'Which carrier did you use?',
  answer: 'Which carrier did you use?',
  slotFill: {
    localeIso2: 'en',
    intent: 'parcel.damaged',
    entities: [ [Object] ],
    answer: 'We will claim {{slots.tracking.sourceText}} as damaged at the carrier.',
    srcAnswer: 'We will claim {{slots.tracking.sourceText}} as damaged at the carrier.',
    currentSlot: 'carrier'
  }
}
POST /chatbot/conversations/e0a7ff81-366d-4686-8f09-8dff19c593ed 200 87.092 ms - 28
{ message: 'UPS' }
{
  utterance: 'UPS',
  locale: 'en',
  languageGuessed: false,
  localeIso2: 'en',
  language: 'English',
  domain: 'EN',
  classifications: [
    { label: 'parcel.damaged', value: 1 },
    { label: 'helpcenter-redirect', value: 0 },
    { label: 'integration-error', value: 0 },
    { label: 'None', value: 0 }
  ],
  intent: 'parcel.damaged',
  score: 1,
  entities: [
    {
      start: 0,
      end: 2,
      len: 3,
      levenshtein: 0,
      accuracy: 1,
      option: 'UPS',
      sourceText: 'UPS',
      entity: 'carrier',
      utteranceText: 'UPS'
    }
  ],
  sourceEntities: [],
  sentiment: {
    score: 0.344,
    comparative: 0.344,
    vote: 'positive',
    numWords: 1,
    numHits: 1,
    type: 'senticon',
    language: 'en'
  },
...
}

To Reproduce
Steps to reproduce the behavior:

  1. Use the nlp-js.app to train a model
  2. add a manager.save() line in the server\trainers\nlpjs-process.js fie to export the model
  3. Train a manager with that model in a nlp-js application somewhere else
  4. See different behaviours on the screenshot above

Expected behavior
The exported model should show the same result on my own application as much on the app

nlp-js version 3.10.2 used for both applications (nlp-js.app & my own)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants