Duplicate concept answers breaks concepts page

[mahalakshme]

Need:

Some coded concepts have more than 15 answers and all are added at the same time. Hence the user sometimes by mistake adds the same concept answer twice. When it doesn't show a valid error, it is raised as a ticket.

Current scenario:

env: staging, prod

• Go to any organisation in webapp
• Go to a coded concept
• Add the same concept answer twice
• Results in 500 error
  Video to understand the issue:
  https://github.com/avniproject/avni-webapp/assets/10369963/674a69cf-aa27-4c4f-8e5c-39f8cd4db465

Expected:

• Show error 'Duplicate answer specified' on-clicking "Save" button and Oops error page should not display.
• Also handle the above when creating concepts directly in form. (InlineConcept)
• Removal of duplicate answer of previously existing answer concept should not result in voiding of earlier answer
• All duplicates except the first original answer should have duplicate error msg shown
• All different duplicates should be shown as errors at once

Old: ignore

Expected:

• Whenever duplicate concept answers are added show valid error message stating 'Duplicate answer present. Remove them to proceed.' like in the mockup below, below the last duplicate instead of showing Oops error page.

[mahalakshme]

concept page breaks. Added it to 5.1.0 since was raised by Vedant once, came as ticket and multiple discussions. To reduce the tickets added it to 5.1.0.

[arjunk]

I see in the database we have concepts with '&' and various other special characters also.

check using the following queries
select * from concept where name ~* '[^a-z0-9 ]' order by created_date_time desc; (last created till July)
select * from concept where name like '%&%'

What special characters are currently being supported/not supported? What is the basis of this? This change seems to have been done in the last few releases but couldn't find any relevant issue except this avniproject/avni-server#449 or release notes (Haven't checked code)
For the concepts where such names with special characters are already there, what problem is it or will it be causing? Do we need to migrate them?
Apart from giving an informative error message to the user, we need to document this.

[vinayvenu]

@mahalakshme I think I read the acceptance criteria wrong. Is there a need to remove these characters from concept answers?

[mahalakshme]

@vinayvenu @arjunk I think as part of security fixes, this change might have occured. But special characters are not allowed even after turning off enhanced validations in prod. We will need to check what has caused this change.

[mahalakshme]

moved the character issue to separate card: #1058

[vinayvenu]

Mockups - we can show the duplicates if possible?

[himeshr]

Sample Duplicate answer screenshots

[AchalaBelokar]

@himeshr I created a concept with ASH and ash it is accepting the value not showing the error. is it valid or not.?

[himeshr]

@himeshr I created a concept with ASH and ash it is accepting the value not showing the error. is it valid or not.?

@AchalaBelokar, In Avni, Concept names are case-sensitive, we allow users to create concepts with same name but with different case, which are used in different forms.. this is pretty common.

Now for Concept Answers, the only usecase i can think of, where concept answers with same name but different case might be needed, is when there is an external integration system, which accepts different case answers in different conditions.

So i would say that in principle, we should not error in-case there is more than 1 concept-answer with same name but different case.

@mahalakshme pls confirm if the duplicates check should ignore case and trailing whitespaces.

[mahalakshme]

@himeshr I will think and update

[mahalakshme]

@AchalaBelokar it does make sense to make the concepts to be considered duplicate independent of the case, but considering we have allowed it so far, it is now complex to correct the existing data. So we need not fix it as part of 10.1.0, will add it to a different release.

[dinesh2096]

• When we removed the option and try to add the same option in the concept it is still showing a error and also if we move the option and select the option and clicking on the save the error is removed but the save is not happening.

QA Reference video :

Click here to watch the video

[himeshr]

Handled additional test-cases:

• Recreation of previously saved voided concept answer
• Handling removal of duplicate non-voided answers when previously saved non-voided concept answer

[petmongrels]

We don't have any check on server for duplicate concepts. we should add a database constraint.

[himeshr]

We don't have any check on server for duplicate concepts. we should add a database constraint.

We do have it, checked in code as well as on local and prod openchs Databases for Concept table.

[himeshr]

Constraint also exists on DB for concept_answer..

[dinesh2096]

QA Reference :

Click here to watch video