Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Search administration & Ontology and Agents search #70

Merged
merged 11 commits into from
Apr 11, 2024

Conversation

syphax-bouazzouni
Copy link

@syphax-bouazzouni syphax-bouazzouni commented Feb 9, 2024

Prerequisites

Changes

Search administration

  • Add search administration endpoints to init schema and index batch (5f319b7)
  • Add admin search in collections (f156cdc)
  • Make the search admin use directly the solr connector (e33a97b)

Ontology and Agent Search

  • Add ontology and agent search endpoints (1fb990d)
  • Add agent and ontology search tests (10cf6f6)
  • Enforce solr models indexing one by one to prevent batch fails (374a693)
  • Index submission and agents metadata (1ab56c0)

Ontology content Search

  • Implement search ontologies content search endpoint (e378e64)
  • Add detType to search ontologies content search endpoint (309813c)
  • Add ontology search content types filter (7041245)
  • Fix content ontology search pagination (4e87904)

@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch 6 times, most recently from 2e6c459 to 92edbf3 Compare February 17, 2024 05:34
Copy link

codecov bot commented Feb 17, 2024

Codecov Report

Attention: Patch coverage is 76.69173% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 71.16%. Comparing base (85617ad) to head (7041245).
Report is 9 commits behind head on development.

❗ Current head 7041245 differs from pull request most recent head c5a95bc. Consider uploading reports for the commit c5a95bc to get more accurate results

Files Patch % Lines
controllers/admin_controller.rb 45.45% 24 Missing ⚠️
controllers/search_controller.rb 92.13% 7 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff               @@
##           development      #70      +/-   ##
===============================================
- Coverage        71.26%   71.16%   -0.10%     
===============================================
  Files               67       67              
  Lines             3508     3642     +134     
===============================================
+ Hits              2500     2592      +92     
- Misses            1008     1050      +42     
Flag Coverage Δ
unittests 71.16% <76.69%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from 92edbf3 to 10cf6f6 Compare February 17, 2024 06:05
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch 7 times, most recently from 2c41233 to e378e64 Compare February 29, 2024 01:54
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from d163b68 to 76846df Compare March 6, 2024 07:58
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from 61bec78 to 7041245 Compare March 19, 2024 14:16
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from df0f08b to 0a66e1d Compare April 11, 2024 18:41
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from ea7fb2f to 7cfb841 Compare April 11, 2024 19:04
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/index-ontology-agent-metadata branch from 7cfb841 to c5a95bc Compare April 11, 2024 19:08
@syphax-bouazzouni syphax-bouazzouni self-assigned this Apr 11, 2024
@syphax-bouazzouni syphax-bouazzouni changed the title Feature: index ontology agent metadata Feature: Search administration & Ontology and Agents search Apr 11, 2024
@syphax-bouazzouni syphax-bouazzouni merged commit 45b2a36 into development Apr 11, 2024
24 checks passed
syphax-bouazzouni added a commit that referenced this pull request May 22, 2024
…RI content negotiation and Ontology metadata and data indexation (#73)

* Feature: Migrate to virtuoso (#67)

* set up multiple triple store test environment

* optimize api tests

* Fix: update ncbo_annotator gem version (#71)

* update ncbo_annotator gem version

* update alegrograph to version 8.1.0

* Feature: use the new  SOLR Schema API instead of SOLR config files  (#68)

* update docker compose to use standard SOLR not the ontoportal configured

* update term search to use the new Schema API and remove config files

* update properties search to use the new Schema API & remove config files

* update class and properties schema to use the existent dynamic names

* Feature: resolving resources within specific ontologies, supporting various output formats(#69)

* remove useless line preventing sending the reset password email (#65)

* [ontoportal-bot] Gemfile.lock update

* Feature: api endpoint returns json-ld  for the element with that URI

* implement GET, POST requests, and GET /parse to submit INRATHES ontology

* Enhance tests using real data submission

* Enhance bin/ontoportal to make it able to run localy with UI

* Small fixes

- change controller name and test controller name
- remove /parse endpoint
- rackup to shotgun in bin/ontoportal

* Fix test dereference resource controller

- in json test, before we test the result we sort the hashes with the function (sort_nested_hash)
- in xml, ntriples and turtle, we split the result and the expected result, sort them and compare them

* update gemfile: add json-ld (3.0.2)

* change derefrencement namespacing and clean code

* Fix dereference resource tests expected resultsto handle parse triples

* fix xml serialization test for AG and Gb by cleaning the xml string

---------

Co-authored-by: OntoPortal Bot <ontoportal.bot.lirmm@gmail.com>
Co-authored-by: imadbourouche <ji_bourouche@esi.dz>

* Feature: URI drerfrencement content negotiation (#72)

* remove useless line preventing sending the reset password email (#65)

* [ontoportal-bot] Gemfile.lock update

* Feature: api endpoint returns json-ld  for the element with that URI

* implement GET, POST requests, and GET /parse to submit INRATHES ontology

* Enhance tests using real data submission

* Enhance bin/ontoportal to make it able to run localy with UI

* Small fixes

- change controller name and test controller name
- remove /parse endpoint
- rackup to shotgun in bin/ontoportal

* Fix test dereference resource controller

- in json test, before we test the result we sort the hashes with the function (sort_nested_hash)
- in xml, ntriples and turtle, we split the result and the expected result, sort them and compare them

* update gemfile: add json-ld (3.0.2)

* change derefrencement namespacing and clean code

* Fix dereference resource tests expected resultsto handle parse triples

* Feature: add content negotiation middleware

* Add headers to tests instead of output_format

* Apply middleware to only /ontologies/:acronym/resolve/:uri

* Add test cases for AllegroGraph and fix xml test

* move the content_negotiation middleware into rack folder and  module

* re-implement again the usage of  the output_format param if no format is given in the request header

* clean the tests for no more necessary checks

* clean and simplify the content negotiation middleware

* add the accepted format in the error response of resolvability endpoint

* refactor the content negotiation middleware code to be more clear

---------

Co-authored-by: Syphax bouazzouni <gs_bouazzouni@esi.dz>
Co-authored-by: OntoPortal Bot <ontoportal.bot.lirmm@gmail.com>

* Fix: user creation security  (#60)

* extract slice tests helper to the parent class for reusability

* add a test for the creation of an admin user

* enforce the security of admin user creation

* update slices controller to enforce admin security

* Fix: the content negotiation by removing a no needed require

* Feature: Indexation administration & Ontologies and Agents search  (#70)

* index submission and agents metadata

* add search administration endpoints to init schema and index batch

* add ontology and agent search endpoints

* add agent and ontology search tests

* add admin search in collections

* make the search admin use directly the solr connector

* implement search ontologies content search endpoint

* enforce solr models indexing one by one to prevent batch fails

* add detType tov search ontologies  content search endpoint

* fix content ontology search pagination

* add ontology search content types filter

* Feature: Add accessibility security to ontology metadata & content search results  (#74)

* add ontology accessibility restriction  to ontology metadata search

* add ontology accessibility restriction  to ontology content search

* add search results accessibility security test

* fix: enable user creation notification (#76)

* Fix: Invalidating cache on insert & fix Redis warning (#77)

* Merge pull request https://github.com/ncbo/ontologies_api/pull/120from ncbo/remove_redis-activesupport

Remove redis activesupport

* use the branch development of sparql client

* Feature: mappings statistics slices support (#78)

* restrict mapping statistics ontologies to the ontologies of the current slice

* add a test for the mappings slices support

* add test for mappings statistics slices support

---------

Co-authored-by: OntoPortal Bot <ontoportal.bot.lirmm@gmail.com>
Co-authored-by: imadbourouche <ji_bourouche@esi.dz>
Co-authored-by: Bilel Kihal <61744974+Bilelkihal@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant