-
Notifications
You must be signed in to change notification settings - Fork 894
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Confusing opensearch dashboard import API #1723
Comments
Hello @charlielj88, Thank you for opening and calling out the lack of documentation. We should re-route this to the documentation repo for the site to be updated with the information. If you have an .ndjson file it means should use the
^ This supports uploading a ndjson file. I also believe it takes a json file as well. So from your example:
From the original post they most likely exported via the I understand that the base you have defined for OpenSearch Dashboards is If you are able to utilize the Thanks! |
Hello @kavilla, Thank you for your clarification, since I am using aws managed opensearch, the default opensearch dashboard url is appended with "/_dashboards". Yes I have tested port 443 is open and working. I have tested "https://{aws_opensearch_url}/_dashboards/api/status" and it could successfully return me the correct response. I have amended my original curl command accordingly as below, curl -X POST "https://{aws_opensearch_url}/_dashboards/api/saved_objects/_import?overwrite=true" -H "osd-xsrf: true" -k -u {username}:{password} --form file=@Metricbeat-kubernetes-overview.json However, I get the return response of "{"statusCode":401,"error":"Unauthorized","message":"Authentication required"}" The {username} and {password} in the curl command are the ones I can use to login to opensearch dashboard. Is there anything I missed here? Charlie |
Hello @charlielj88, Do you know if the user you are using has write access? |
Hi @kavilla, yes, this is confirmed, as I use the same opensearch credentials for metricbeat to write into the index. Charlie |
@kavilla, I also noticed that if I directly import the saved_object from UI, I have the following error, which seems the json format is not supported, |
Hello. I'm migrating from kibana spaces to opensearch-dashboards with multitenancy and i need a rest api to import dashboards for individual tenants.
running tenant is configured:
Any hints please on importing a dashboard for a specific tenant? Thank you. |
Finally find a way to convert the json to ndjson and now at least I could import metricbeat dashboards from UI... |
Hello everybody, Managed to get it working. Below the steps: Prerequisites:
# curl -k -XPUT -H'content-type: application/json' https://admin:admin@localhost:9200/_plugins/_security/api/tenants/tenant_one -d '{"description": "tenant one"}'
{"status":"CREATED","message":"'tenant_one' created."}
# curl -k -XPUT -H'content-type: application/json' https://admin:admin@localhost:9200/_plugins/_security/api/tenants/tenant_two -d '{"description": "tenant two"}'
{"status":"CREATED","message":"'tenant_two' created."}
# curl -k -XGET -u 'admin:admin' -c dashboards_cookie http://localhost:5601/api/login/
# curl -k -XGET -b dashboards_cookie http://localhost:5601/api/v1/configuration/account | jq
{
"data": {
"user_name": "admin",
"user_requested_tenant": "__user__",
[...]
}
}
# curl -k -XPOST -b dashboards_cookie -c dashboards_cookie -H'osd-xsrf: true' -H'content-type: application/json' http://localhost:5601/api/v1/multitenancy/tenant -d '{"tenant": "tenant_one", "username": "admin"}'
# curl -k -XGET -b dashboards_cookie http://localhost:5601/api/v1/configuration/account | jq
{
"data": {
"user_name": "admin",
"user_requested_tenant": "tenant_one",
[...]
}
}
# curl -k -XPOST -H'osd-xsrf: true' -b dashboards_cookie http://localhost:5601/api/saved_objects/_import?overwrite=true --form file=@export.ndjson | jq
{
"successCount": 3,
"success": true,
"successResults": [
{
"type": "index-pattern",
"id": "96f0ec20-ec90-11ec-9191-4dc9d4cc1f7d",
"meta": {
"title": "*__dobby_docs",
"icon": "indexPatternApp"
}
},
{
"type": "visualization",
"id": "e4aca2a0-ed6d-11ec-9191-4dc9d4cc1f7d",
"meta": {
"title": "cucu_dash",
"icon": "visualizeApp"
}
},
{
"type": "dashboard",
"id": "edeca590-ed6d-11ec-9191-4dc9d4cc1f7d",
"meta": {
"title": "cucu_the_dash",
"icon": "dashboardApp"
}
}
]
}
Reagrds, C. |
Any help/suggestion is highly appreciated. I am having trouble importing *.ndjson files using SavedObject API within a Lambda, what I am trying is to import a bunch of *.ndjson files (dashboards, index-templates etc) at the time of creating a stack (via CDK) that includes Opensearch & Dashboard services. The Opensearch is configured for Cognito authentication and browser logins are working fine. However the lambda (with write access/role to Opensearch domain) is having trouble with API, Does Opensearch dashboard service supports Lambda IAM role to access, I know Opensearch supports read/write from a Lambda using IAM role. I am reading AWS developer guide - "Loading credentials for a Node.js Lambda function" (https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-lambda.html)
The request has the headers set (not sure whether I am missing any header or Dashboard is looking for a specific header name)
|
You need to include the lambda execution role as 'Backend Role' in the corresponding Opensearch Dashboard Role. |
Apologise @subramaniant06 , I failed to notice your response I do have Lambda role mapped to
That snippet may be confusing, but what I am doing is to configure roles in opensearch via set of requests thats executed via a |
How are you making this API call? from lambda? This is possible only if you assume the master user role that you used to enable the Fine grained access for Opensearch. let requests: object[] = [{ method: 'PUT', path: '_plugins/_security/api/rolesmapping/all_access', body: { backend_roles: [ adminUserRoleArn, lambdaRoleArn ], hosts: [], users: [], } }] I'm able to create tenants, roles, monitors, destinations etc using Custom Resource Lambda with master user role as execution role, AWS NodeHttpClient with Sigv4. However i'm not able to do the multi-part upload using NodeHttpClient to import the index patterns, dashboards and visualizations. |
Guess we are in the same juncture of issue. The lambda execution role is set as The lambda that imports index-patterns/dashboards etc are triggered from S3 and creates an Axios multi-part request to import savedObjects. (this lambda has the same IAM role as the one that works with CustomResource). I am not sure what is happening in the background of Opensearch and unable to troubleshoot. Below is the typescript code that handles
by the way, I am using below API path for importing savedobjects, hope this is correct.
|
Hello, I was able to achieve using the curl -XPOST -c dashboards_cookie https://${os_endpoint}/_dashboards/auth/login -H "osd-xsrf: true" -H "Content-Type: application/json" -d '{"username":"${admin_username}","password":"${admin_password}"}' Thanks, |
Hi guys. Today I discovered another very confusing thing with importing dashboard to specific tenant. If I make tenant $ curl -k -w ", HTTP:%{http_code}" -X POST -uadmin:REDACTED "http://localhost:5601/api/saved_objects/_import?overwrite=true" -H "securitytenant: NGSupport" -H "osd-xsrf: true" --form file=@TestDashboard.ndjson {"successCount":3,"success":true,"successResults":[{"type":"index-pattern","id":"nginx-*","meta":{"title":"nginx-*","icon":"indexPatternApp"},"overwrite":true},{"type":"search","id":"149258c0-b816-11ed-89d4-ffdda45e5108","meta":{"title":"NginxRequests","icon":"discoverApp"}},{"type":"dashboard","id":"399571c0-b816-11ed-89d4-ffdda45e5108","meta":{"title":"NGSupport Dashboard","icon":"dashboardApp"}}]}, HTTP:200 Workaround for the problem is make dedicated internal user with read/write access to the tenant and using these credentials for importing. I expecting, that:
If I login to OpenSearch Dashboard as admin, via GUI I can import any objects to any tenants without any problems. Why API calls are different? |
Hello, this still seems to be an issue in OS 2.7. I am trying to get a saved object up with the following method:
The response is a 401 code from Opensearch. This file uploads fine from the gui. |
Hi,
I am using metricbeat-oss version 7.10.2 with aws opensearch 1.2 (compatibility mode enabled so trick the metricbeat with version 7.10.2). The opensearch connection is working fine. However, when I tried to follow the workaround approach in another post (#831) to manually curl to import the pre-existing metricbeat dashboards to opensearch, I could not trigger the API call successfully,
curl -k -XPOST -u {id}:{password} -H "osd-xsrf: true" -H 'content-type: application/json'
https://{opensearch url}:443/_dashboards/api/opensearch-dashboards/dashboards/import?exclude=index-patterh&force=true
-d@Metricbeat-kubernetes-overview.json
This API is quite puzzling to me,
Thank you.
Charlie
The text was updated successfully, but these errors were encountered: