Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Samples for search #11338

Merged
merged 5 commits into from
May 11, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@
ImageAnalysisSkill,
Index,
Indexer,
IndexingSchedule,
IndexingParameters,
InputFieldMappingEntry,
KeepTokenFilter,
KeyPhraseExtractionSkill,
Expand Down Expand Up @@ -165,6 +167,8 @@
"ImageAnalysisSkill",
"Index",
"Indexer",
"IndexingSchedule",
"IndexingParameters",
"IndexAction",
"IndexDocumentsBatch",
"IndexingResult",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,21 +39,21 @@ def __init__(self, endpoint, credential, **kwargs):
endpoint=endpoint, sdk_moniker=SDK_MONIKER, **kwargs
) # type: _SearchServiceClient

def __enter__(self):
async def __aenter__(self):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

messed up aio in indexer :(

# type: () -> SearchIndexersClient
self._client.__enter__() # pylint:disable=no-member
await self._client.__aenter__() # pylint:disable=no-member
return self

def __exit__(self, *args):
async def __aexit__(self, *args):
# type: (*Any) -> None
return self._client.__exit__(*args) # pylint:disable=no-member
return await self._client.__aexit__(*args) # pylint:disable=no-member

def close(self):
async def close(self):
# type: () -> None
"""Close the :class:`~azure.search.documents.SearchIndexersClient` session.
"""Close the :class:`~azure.search.documents.aio.SearchIndexersClient` session.

"""
return self._client.close()
return await self._client.close()

@distributed_trace_async
async def create_indexer(self, indexer, **kwargs):
Expand Down
4 changes: 4 additions & 0 deletions sdk/search/azure-search-documents/samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ Then for common search index operations:

* Analyze text: [sample_analyze_text.py](sample_analyze_text.py) ([async version](async_samples/sample_analyze_text_async.py))

* CRUD operations for indexers: [sample_indexers_operations.py](sample_indexers_operations.py) ([async version](async_samples/sample_indexer_operations_async.py))

* General workflow of indexer, datasource and index: [sample_indexer_datasource_skillset.py](sample_indexer_datasource_skillset.py) ([async version](async_samples/sample_indexer_datasource_skillset.py))

## Prerequisites
* Python 2.7, or 3.5 or later is required to use this package (3.5 or later if using asyncio)
* You must have an [Azure subscription](https://azure.microsoft.com/free/)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# coding: utf-8

# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------

"""
FILE: sample_indexer_operations_async.py
DESCRIPTION:
This sample demonstrates how to get, create, update, or delete a Indexer.
USAGE:
python sample_indexer_operations_async.py

Set the environment variables with your own values before running the sample:
1) AZURE_SEARCH_SERVICE_ENDPOINT - the endpoint of your Azure Cognitive Search service
2) AZURE_SEARCH_API_KEY - your search API key
"""

import asyncio
import os

service_endpoint = os.getenv("AZURE_SEARCH_SERVICE_ENDPOINT")
key = os.getenv("AZURE_SEARCH_API_KEY")
connection_string = os.getenv("AZURE_STORAGE_CONNECTION_STRING")

from azure.core.credentials import AzureKeyCredential
from azure.search.documents import (
DataSource, DataContainer, DataSourceCredentials, Index, Indexer, SimpleField, edm
)
from azure.search.documents.aio import SearchServiceClient

service_client = SearchServiceClient(service_endpoint, AzureKeyCredential(key))
indexers_client = service_client.get_indexers_client()

async def create_indexer():
# create an index
index_name = "hotels"
fields = [
SimpleField(name="hotelId", type=edm.String, key=True),
SimpleField(name="baseRate", type=edm.Double)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for what looks like a more complete (enough) example, maybe add a hotelName that is a SearchableField with a few bells and whistles. As a consumer, this index would give me nothing useful (I'd see a base rate, but for whom?).

]
index = Index(name=index_name, fields=fields)
ind_client = service_client.get_indexes_client()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually, the method name should comply with the guidelines, i.e. that the model name is part of not only the client name ({Model}Client), but the method is get_{Model}Client (granted, in snake case idiomatic of python). Might consider doing before the UX study so, if people say "that's too long", we can take that back to archboard.

async with ind_client:
index = await ind_client.create_index(index)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The methods - now on sub-clients - should not repeat the model. I.e. just create. Same as below for other sub-clients.


# [START create_indexer_async]
heaths marked this conversation as resolved.
Show resolved Hide resolved
# create a datasource
ds_client = service_client.get_datasources_client()
credentials = DataSourceCredentials(connection_string=connection_string)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This connection string should just go onto the SearchIndexerDataSource. See #10860.

container = DataContainer(name='searchcontainer')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SearchIndexerDataContainer. See #10860.

ds = DataSource(name="async-indexer-datasource", type="azureblob", credentials=credentials, container=container)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SearchIndexerDataSource. See #10860.

async with ds_client:
data_source = await ds_client.create_datasource(ds)

# create an indexer
indexer = Indexer(name="async-sample-indexer", data_source_name="async-indexer-datasource", target_index_name="hotels")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SearchIndexer. See #10860.

async with indexers_client:
result = await indexers_client.create_indexer(indexer)
print("Create new Indexer - async-sample-indexer")
# [END create_indexer_async]

async def list_indexers():
# [START list_indexer_async]
async with indexers_client:
result = await indexers_client.get_indexers()
names = [x.name for x in result]
print("Found {} Indexers in the service: {}".format(len(result), ", ".join(names)))
# [END list_indexer_async]

async def get_indexer():
# [START get_indexer_async]
async with indexers_client:
result = await indexers_client.get_indexer("async-sample-indexer")
print("Retrived Indexer 'async-sample-indexer'")
return result
# [END get_indexer_async]

async def get_indexer_status():
# [START get_indexer_status_async]
async with indexers_client:
result = await indexers_client.get_indexer_status("async-sample-indexer")
print("Retrived Indexer status for 'async-sample-indexer'")
return result
# [END get_indexer_status_async]

async def run_indexer():
# [START run_indexer_async]
async with indexers_client:
result = await indexers_client.run_indexer("async-sample-indexer")
print("Ran the Indexer 'async-sample-indexer'")
return result
# [END run_indexer_async]

async def reset_indexer():
# [START reset_indexer_async]
async with indexers_client:
result = await indexers_client.reset_indexer("async-sample-indexer")
print("Reset the Indexer 'async-sample-indexer'")
return result
# [END reset_indexer_async]

async def delete_indexer():
# [START delete_indexer_async]
async with indexers_client:
indexers_client.delete_indexer("async-sample-indexer")
print("Indexer 'async-sample-indexer' successfully deleted")
# [END delete_indexer_async]

async def main():
# await create_indexer()
# await list_indexers()
# await get_indexer()
# await get_indexer_status()
# await run_indexer()
# await reset_indexer()
# await delete_indexer()
# await service_client.close()

if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
187 changes: 187 additions & 0 deletions sdk/search/azure-search-documents/samples/files/patient_data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you already have samples using medical data? Seems most others have been using the hotels sample. As long as you're consistent, though. If this is the first of new samples, you might consider a consistent model (i.e. hotels) for similarity across languages' samples.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having skimmed the contents as well, I'm a little worried the topic of conversation might be triggering to people who have been impacted by this tragedy. Hotels seem pretty safe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed it to use hotel data - agree with patient data being sensitive - yes i already had the data for a different project i've been working on

"agebracket": "20",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "30/01/2020",
"detectedcity": "Thrissur",
"detecteddistrict": "Thrissur",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "F",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "1",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "02/02/2020",
"detectedcity": "Alappuzha",
"detecteddistrict": "Alappuzha",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "2",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "03/02/2020",
"detectedcity": "Kasaragod",
"detecteddistrict": "Kasaragod",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "3",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "45",
"backupnotes": "Travel history to Italy and Austria",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "02/03/2020",
"detectedcity": "East Delhi (Mayur Vihar)",
"detecteddistrict": "East Delhi",
"detectedstate": "Delhi",
"estimatedonsetdate": "",
"gender": "M",
"nationality": "India",
"notes": "Travelled from Austria, Italy",
"patientnumber": "4",
"statecode": "DL",
"typeoftransmission": "Imported"
}
{
"agebracket": "20",
"backupnotes": "Student from Italy",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "30/01/2020",
"detectedcity": "Thrissur",
"detecteddistrict": "Thrissur",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "F",
"nationality": "India",
"notes": "Travelled from USA",
"patientnumber": "5",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "02/02/2020",
"detectedcity": "Mumbail",
"detecteddistrict": "Mumbail",
"detectedstate": "Maharashtra",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Family of infected",
"patientnumber": "6",
"statecode": "MH",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Death",
"dateannounced": "03/02/2020",
"detectedcity": "Kasaragod",
"detecteddistrict": "Kasaragod",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "7",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "23",
"backupnotes": "Travel history to Italy and Austria",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "02/03/2020",
"detectedcity": "East Delhi (Mayur Vihar)",
"detecteddistrict": "East Delhi",
"detectedstate": "Delhi",
"estimatedonsetdate": "",
"gender": "M",
"nationality": "India",
"notes": "Travelled from Austria, Italy",
"patientnumber": "8",
"statecode": "DL",
"typeoftransmission": "Imported"
}
{
"agebracket": "20",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "30/01/2020",
"detectedcity": "Thrissur",
"detecteddistrict": "Thrissur",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "F",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "9",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Infected",
"dateannounced": "02/02/2020",
"detectedcity": "Alappuzha",
"detecteddistrict": "Alappuzha",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "10",
"statecode": "KL",
"typeoftransmission": "Imported"
}
{
"agebracket": "",
"backupnotes": "Student from Wuhan",
"contractedfromwhichpatientsuspected": "",
"currentstatus": "Recovered",
"dateannounced": "03/02/2020",
"detectedcity": "Kasaragod",
"detecteddistrict": "Kasaragod",
"detectedstate": "Kerala",
"estimatedonsetdate": "",
"gender": "",
"nationality": "India",
"notes": "Travelled from Wuhan",
"patientnumber": "11",
"statecode": "KL",
"typeoftransmission": "Imported"
}
Loading