Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store queries in DB table and index as OpenSearch percolators to allow for reverse lookup #1042

Open
Tracked by #1041
slint opened this issue Nov 4, 2024 · 0 comments · May be fixed by #1049
Open
Tracked by #1041

Store queries in DB table and index as OpenSearch percolators to allow for reverse lookup #1042

slint opened this issue Nov 4, 2024 · 0 comments · May be fixed by #1049

Comments

@slint
Copy link
Member

slint commented Nov 4, 2024

Rough design:

class Query(db.Model):

    id = db.Column(db.Integer, primary_key=True)

    score = db.Column(db.Integer, default=0)
    query_string = db.Column(db.Text, nullable=False)

    notes = db.Column(db.Text, nullable=True)

    active = db.Column(db.Boolean, default=True)


# Create the query (low-level API)
new_query = Query(
    query_string='"1080p download"',
    score=5,
)
db.session.add(new_query)
db.session.commit()

# Index as percolator (low-level API)
current_search_client.index(
    index="moderation-records",
    body={
        "query": {
            "query_string": {"query": new_query.query_string},
        },
        "score": new_query.score,
        "active": new_query.active,
    },
)

# High level
Query.create(
    query_string='"1080p download"',
    score=5,
)
db.session.commit()

# TODO: In the future we could have a REST API to hook-in the administration interface
"""POST /api/moderation/queries
{
    "query_string": "1080p download",
    "score": 5
}
"""

## Create mapping
record_mapping = current_search_client.indices.get("rdmrecords-records-record-v7.0.0")
current_search_client.create_index(
  index="moderation-records",
  body="mappings": {
    "properties": {
        # Inclue the "live" mapping
        **record_mapping,

        # Percolator-specific fields
        "query": {
            "type": "percolator"
        },
        "score": {
            "type": "integer"
        },
        "active": {
            "type": "boolean"
        }
    }
  }
)

## Usage - During moderation rules
matched_queries = current_search_client.search(
    index="moderation-records",
    body={
        "query": {
            "percolate": {
                "field": "query",
                "document": record.dumps(),
            }
        }
    },
)
@0einstein0 0einstein0 linked a pull request Nov 6, 2024 that will close this issue
@0einstein0 0einstein0 linked a pull request Nov 6, 2024 that will close this issue
@0einstein0 0einstein0 removed their assignment Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants