Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't create ElasticSearch JP index in migrate-all process #14117

Closed
shin1103 opened this issue Nov 26, 2023 · 0 comments · Fixed by #14118
Closed

Can't create ElasticSearch JP index in migrate-all process #14117

shin1103 opened this issue Nov 26, 2023 · 0 comments · Fixed by #14118
Assignees

Comments

@shin1103
Copy link
Contributor

Affected module
service

Describe the bug
Some ElasticSearch indexes are not created when launching a Docker image according to QuickStart.

To Reproduce

  1. Create Docker Image to install Japanese analyzer
FROM docker.elastic.co/elasticsearch/elasticsearch:8.10.2
RUN bin/elasticsearch-plugin install analysis-kuromoji
 docker build -t elasticsearch-kuromoji:8.10.2 .
  1. Edit Yaml file to use Docker Image above
    https://github.com/open-metadata/OpenMetadata/releases/download/1.2.2-release/docker-compose.yml

  2. Change ELASTICSEARCH_INDEX_MAPPING_LANG to JP
    https://docs.open-metadata.org/v1.2.x/deployment/kubernetes/helm-values

  3. Execute Docker Compose

docker compose -f docker-compose.yml up --detach
  1. Show Docker Logs, Name is execute_migrate_all
    Some index is fail to create. Here is part of the log.
21:16:29.345 [main] INFO  o.o.s.m.a.MigrationWorkflow - [MigrationWorkflow] WorkFlow Completed
21:16:35.793 [main] ERROR o.o.s.s.SearchRepository - Failed to read index Mapping file due to
java.lang.NullPointerException: Cannot invoke "java.io.InputStream.readAllBytes()" because "in" is null
        at org.openmetadata.service.search.SearchRepository.getIndexMapping(SearchRepository.java:190)
        at org.openmetadata.service.search.SearchRepository.updateIndex(SearchRepository.java:164)
        at org.openmetadata.service.search.SearchRepository.updateIndexes(SearchRepository.java:132)
        at org.openmetadata.service.util.TablesInitializer.execute(TablesInitializer.java:327)
        at org.openmetadata.service.util.TablesInitializer.main(TablesInitializer.java:209)
21:16:35.800 [main] ERROR o.o.s.s.e.ElasticSearchClient - Failed to create Elastic Search indexes due to
java.lang.NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "text" is null
        at es.org.apache.lucene.util.BytesRef.<init>(BytesRef.java:84)
        at es.org.elasticsearch.common.bytes.BytesArray.<init>(BytesArray.java:27)
        at es.org.elasticsearch.client.indices.CreateIndexRequest.source(CreateIndexRequest.java:261)
        at org.openmetadata.service.search.elasticsearch.ElasticSearchClient.createIndex(ElasticSearchClient.java:199)
        at org.openmetadata.service.search.SearchRepository.updateIndex(SearchRepository.java:168)
        at org.openmetadata.service.search.SearchRepository.updateIndexes(SearchRepository.java:132)
        at org.openmetadata.service.util.TablesInitializer.execute(TablesInitializer.java:327)
        at org.openmetadata.service.util.TablesInitializer.main(TablesInitializer.java:209)
21:16:35.860 [main] ERROR o.o.s.s.e.ElasticSearchClient - Failed to create alias for metadata_service_search_index due to
es.org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=index_not_found_exception, reason=no such index [metadata_service_search_index]]
        at es.org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:178)
        at es.org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2484)
        at es.org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2461)
        at es.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2184)
        at es.org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
        at es.org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
        at es.org.elasticsearch.client.IndicesClient.updateAliases(IndicesClient.java:737)
        at org.openmetadata.service.search.elasticsearch.ElasticSearchClient.createAliases(ElasticSearchClient.java:227)
        at org.openmetadata.service.search.SearchRepository.updateIndex(SearchRepository.java:170)
        at org.openmetadata.service.search.SearchRepository.updateIndexes(SearchRepository.java:132)
        at org.openmetadata.service.util.TablesInitializer.execute(TablesInitializer.java:327)
        at org.openmetadata.service.util.TablesInitializer.main(TablesInitializer.java:209)
        Suppressed: es.org.elasticsearch.client.ResponseException: method [POST], host [http://elasticsearch:9200], URI [/_aliases?master_timeout=30s&timeout=30s], stat
us line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [metadata_service_search_index]","resource.type":"index_or_alias","resource.id":"met
adata_service_search_index","index_uuid":"_na_","index":"metadata_service_search_index"}],"type":"index_not_found_exception","reason":"no such index [metadata_service_s
earch_index]","resource.type":"index_or_alias","resource.id":"metadata_service_search_index","index_uuid":"_na_","index":"metadata_service_search_index"},"status":404}
                at es.org.elasticsearch.client.RestClient.convertResponse(RestClient.java:347)
                at es.org.elasticsearch.client.RestClient.performRequest(RestClient.java:313)
                at es.org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
                at es.org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
                at es.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
                ... 8 common frames omitted

Expected behavior
All index is created successfully.

Version:

  • OS: MAC OS
  • Python version:3.9
  • OpenMetadata version: 1.2.2
  • OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==1.2.2]

Additional context
None

@harshach harshach moved this to Search in Release 1.2.3 Nov 27, 2023
harshach pushed a commit that referenced this issue Nov 28, 2023
* rename elasticsearch json filename same as english

* change name om_analyzer to om_analyzer_jp
@07Himank 07Himank moved this from Search to Done in Release 1.2.3 Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant