Merge

.github/workflows/triage.yml

@@ -0,0 +1,13 @@
+name: Issue triage
+on:
+  issues:
+    types: [opened]
+jobs:
+  build:
+    name: Triage
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: Logerfo/triage-action@0.0.2
+      with:
+        github-token: ${{ secrets.ACCESS_TOKEN }}
+        label: "needs-triage"

CONTRIBUTING.md

@@ -0,0 +1 @@
+Refer to the [CORTX Contribution Guide](https://github.com/Seagate/cortx/blob/main/CONTRIBUTING.md) to contribute to the CORTX project and it's submodule repositories.

README.md

@@ -1,17 +1,16 @@
 
-
-# NSAL
+# CORTX-NSAL
 
 [![Codacy Badge](https://api.codacy.com/project/badge/Grade/6beb9e8fb4e64743b293431e91a8d787)](https://app.codacy.com/gh/Seagate/cortx-nsal?utm_source=github.com&utm_medium=referral&utm_content=Seagate/cortx-nsal&utm_campaign=Badge_Grade) [![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://github.com/Seagate/cortx-nsal/blob/main/LICENSE) [![Slack](https://img.shields.io/badge/chat-on%20Slack-blue")](https://join.slack.com/t/cortxcommunity/shared_invite/zt-femhm3zm-yiCs5V9NBxh89a_709FFXQ?)
 
-Namespace Abstraction Layer is created to facilitate the namespace management and abstraction on top of a object/cloud storage.
+The CORTX Namespace Abstraction Layer (NSAL) is created to facilitate the namespace management and abstraction on top of an object or cloud storage.
 
 ### Disclaimer
-Please refer to the shared disclaimer (https://github.com/Seagate/cortx-posix#disclaimer) about this code repository.
+Please refer to the shared [disclaimer](https://github.com/Seagate/cortx-posix#disclaimer) about this code repository.
 
 ## Components
 
-The Namespace abstraction layer is implemented using the following entities/components
+The Namespace abstraction layer is implemented using the following entitites and components:
 - Extended attributes (Xattrs)
 - KVtree
 - KVnode
@@ -20,3 +19,18 @@ The Namespace abstraction layer is implemented using the following entities/comp
 - KVstore
 
 Refer to the [CORTX-NSAL High Level Overview](doc/HighLevelOverview.md) for a high level overview over each component. 
+
+## Contribute to CORTX-NSAL
+
+We are excited about your interest in CORTX and hope you will join us. We take our community very seriously, and we are committed to creating a community built on respectful interactions and inclusivity, as documented in our [Code of conduct](CODE_OF_CONDUCT.md).
+
+Refer to the [CORTX Contribution Guide](CONTRIBUTING.md) that hosts all information about community values, code of conduct, how to contribute code and documentation, community and code style guide, and how to reach out to us.
+
+## Reach Out To Us
+
+- Please refer to our [Support](SUPPORT.md) section for support or to discuss things with your fellow CORTX Community members.
+- Click here to [view existing issues or to create new issues](https://github.com/Seagate/cortx-posix/issues).
+
+## Thank You!
+
+We thank you for stopping by to check out the CORTX Community. Seagate is fully dedicated to its mission to build open source technologies that help the world save unlimited data and solve challenging data problems. Join our mission to help reinvent a data-driven world.

SECURITY.md

@@ -0,0 +1,17 @@
+# CORTX Security Policy
+
+We take the security of the CORTX project seriously. If you find any security vulnerabilities in any of the CORTX-owned source code managed via our [GitHub Organisation](https://github.com/Seagate) repositories, please report it to us as described below.
+
+## Reporting Security Issues
+
+:warning: **Please do not publicly report any security vulnerabilities via GitHub issues.**
+
+Instead, please report them to us by mailing us at [cortx-questions@seagate.com](mailto:cortx-questionse@seagate.com). 
+
+#### Reporting Format
+
+To make your reporting meaningful and help us understand the nature and scope of the issue, please include as much information as possible. 
+
+#### Preferred Languages
+
+We prefer all communications to be in English.

SUPPORT.md

@@ -0,0 +1,8 @@
+Looking for support for CORTX parent or a repository?
+
+Consider some of these resources:
+
+-   Join our CORTX-Open Source Slack channel [![Slack](https://img.shields.io/badge/chat-on%20Slack-blue")](https://join.slack.com/t/cortxcommunity/shared_invite/zt-femhm3zm-yiCs5V9NBxh89a_709FFXQ?) to interact with community members and gets your questions answered.
+-   Join us in [Discussions](https://github.com/Seagate/cortx/discussions) to ask, answer, and discuss topics with your fellow CORTX contributors!
+-   If you'd like to contact us directly, drop us a mail at cortx-questions@seagate.com.
+-   We like to highlight the work and contributions of our community members—if you have solved an interesting challenge, or you are interested in sharing your experience or use cases, we want to talk to you! Please email our Community Manager rachel.novak@seagate.com or [schedule a meeting with us](https://outlook.office365.com/owa/calendar/CORTXCommunity@seagate.com/bookings/s/x8yMn2ODxUCOdhxvXkH4FA2) to share.

src/common/motr/m0common.c

@@ -42,6 +42,7 @@
 #include "lib/thread.h"
 #include "m0common.h"
 #include <motr/helpers/helpers.h>
+#include "operation.h"
 
 /* Key Iterator */
 
@@ -67,18 +68,28 @@ void m0_key_iter_fini(struct kvstore_iter *iter)
 {
 	struct m0_key_iter_priv *priv = m0_key_iter_priv(iter);
 
+	perfc_trace_inii(PFT_M0_KEY_ITER_FINISH, PEM_NSAL_NFS_TO_MOTR);
 	if (!priv->initialized)
 		goto out;
 
 	m0_bufvec_free(&priv->key);
 	m0_bufvec_free(&priv->val);
 
 	if (priv->op) {
+		perfc_trace_attr(PEA_M0_OP_SM_ID, priv->op->op_sm.sm_id);
+		perfc_trace_attr(PEA_M0_OP_SM_STATE, priv->op->op_sm.sm_state);
+
+		perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_FINISH);
 		m0_op_fini(priv->op);
+		perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_FINISH);
+
+		perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_FREE);
 		m0_op_free(priv->op);
+		perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_FREE);
 	}
 
 out:
+	perfc_trace_finii(PERFC_TLS_POP_DONT_VERIFY);
 	return;
 }
 
@@ -92,6 +103,7 @@ bool m0_key_iter_find(struct kvstore_iter *iter, const void* prefix,
 	struct m0_idx *index = iter->idx.index_priv;
 	int rc;
 
+	perfc_trace_inii(PFT_M0_KEY_ITER_FIND, PEM_NSAL_NFS_TO_MOTR);
 	if (prefix_len == 0)
 		rc = m0_bufvec_empty_alloc(key, 1);
 	else
@@ -107,17 +119,26 @@ bool m0_key_iter_find(struct kvstore_iter *iter, const void* prefix,
 
 	memcpy(priv->key.ov_buf[0], prefix, prefix_len);
 
+	perfc_trace_attr(PEA_TIME_ATTR_START_M0_IDX_OP);
 	rc = m0_idx_op(index, M0_IC_NEXT, &priv->key, &priv->val,
 		       priv->rcs, 0, op);
+	perfc_trace_attr(PEA_TIME_ATTR_END_M0_IDX_OP);
 
 	if (rc != 0) {
 		goto out_free_val;
 	}
 
-
+	perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_LAUNCH);
 	m0_op_launch(op, 1);
+	perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_LAUNCH);
+
+	perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_WAIT);
 	rc = m0_op_wait(*op, M0_BITS(M0_OS_STABLE),
 			M0_TIME_NEVER);
+	perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_WAIT);
+
+	perfc_trace_attr(PEA_M0_OP_SM_ID, (*op)->op_sm.sm_id);
+	perfc_trace_attr(PEA_M0_OP_SM_STATE, (*op)->op_sm.sm_state);
 
 	if (rc != 0) {
 		goto out_free_op;
@@ -135,8 +156,13 @@ bool m0_key_iter_find(struct kvstore_iter *iter, const void* prefix,
 
 out_free_op:
 	if (op && *op) {
+		perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_FINISH);
 		m0_op_fini(*op);
+		perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_FINISH);
+
+		perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_FREE);
 		m0_op_free(*op);
+		perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_FREE);
 	}
 
 out_free_val:
@@ -151,7 +177,7 @@ bool m0_key_iter_find(struct kvstore_iter *iter, const void* prefix,
 	}
 
 	iter->inner_rc = rc;
-
+	perfc_trace_finii(PERFC_TLS_POP_DONT_VERIFY);
 	return rc == 0;
 }
 
@@ -177,23 +203,33 @@ bool m0_key_iter_next(struct kvstore_iter *iter)
 	struct m0_idx *index = iter->idx.index_priv;
 	bool can_get_next = false;
 
+	perfc_trace_inii(PFT_M0_KEY_ITER_NEXT, PEM_NSAL_NFS_TO_MOTR);
 	assert(priv->initialized);
 
 	/* Motr API: "'vals' vector ... should contain NULLs" */
 	m0_bufvec_free_data(&priv->val);
 
+	perfc_trace_attr(PEA_TIME_ATTR_START_M0_IDX_OP);
 	iter->inner_rc = m0_idx_op(index, M0_IC_NEXT,
 				   &priv->key, &priv->val, priv->rcs,
 				   M0_OIF_EXCLUDE_START_KEY,  &priv->op);
+	perfc_trace_attr(PEA_TIME_ATTR_END_M0_IDX_OP);
 
 	if (iter->inner_rc != 0) {
 		goto out;
 	}
 
+	perfc_trace_attr(PEA_TIME_ATTR_START_NSAL_M0_OP_LAUNCH);
 	m0_op_launch(&priv->op, 1);
+	perfc_trace_attr(PEA_TIME_ATTR_END_NSAL_M0_OP_LAUNCH);
+
+	perfc_trace_attr(PEA_TIME_ATTR_START_M0_OP_WAIT);
 	iter->inner_rc = m0_op_wait(priv->op, M0_BITS(M0_OS_STABLE),
 				    M0_TIME_NEVER);
+	perfc_trace_attr(PEA_TIME_ATTR_END_M0_OP_WAIT);
 
+	perfc_trace_attr(PEA_M0_OP_SM_ID, priv->op->op_sm.sm_id);
+	perfc_trace_attr(PEA_M0_OP_SM_STATE, priv->op->op_sm.sm_state);
 	if (iter->inner_rc != 0) {
 		goto out;
 	}
@@ -204,6 +240,7 @@ bool m0_key_iter_next(struct kvstore_iter *iter)
 		can_get_next = true;
 
 out:
+	perfc_trace_finii(PERFC_TLS_POP_DONT_VERIFY);
 	return can_get_next;
 }
 

src/include/cfs_nsal_perfc.h

@@ -0,0 +1,103 @@
+/*
+ * Filename:	cfs_nsal_perfc.h
+ * Description:	This module defines performance counters and helpers.
+ *
+ * Copyright (c) 2020 Seagate Technology LLC and/or its Affiliates
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published
+ * by the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ * See the GNU Affero General Public License for more details.
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ * For any questions about this software or licensing,
+ * please email opensource@seagate.com or cortx-questions@seagate.com. 
+ */
+
+#ifndef __CFS_NSAL_PERF_COUNTERS_H_
+#define __CFS_NSAL_PERF_COUNTERS_H_
+/******************************************************************************/
+#include "perf/tsdb.h" /* ACTION_ID_BASE */
+#include "operation.h"
+#include <pthread.h>
+#include <string.h>
+#include "debug.h"
+#include "perf/perf-counters.h"
+
+enum perfc_nsal_function_tags {
+	PFT_KVS_START = PFTR_RANGE_4_START,
+
+	PFT_KVS_INIT,
+	PFT_KVS_FINI,
+	PFT_KVS_ALLOC,
+	PFT_KVS_FREE,
+	PFT_KVS_GET,
+	PFT_KVS_SET,
+	PFT_KVTREE_ITER_CH,
+
+	PFT_CORTX_KVS_INIT,
+	PFT_CORTX_KVS_FINISH,
+	PFT_CORTX_KVS_ALLOC,
+	PFT_CORTX_KVS_FREE,
+	PFT_CORTX_KVS_INDEX_CREATE,
+	PFT_CORTX_KVS_INDEX_DELETE,
+	PFT_CORTX_KVS_INDEX_OPEN,
+	PFT_CORTX_KVS_INDEX_CLOSE,
+	PFT_CORTX_KVS_GET_BIN,
+	PFT_CORTX_KVS4_GET_BIN,
+	PFT_CORTX_KVS_SET_BIN,
+	PFT_CORTX_KVS4_SET_BIN,
+	PFT_CORTX_KVS_DELETE_BIN,
+	PFT_CORTX_KVS_GEN_FID,
+	PFT_CORTX_KVS_PREFIX_ITER_FIND,
+	PFT_CORTX_KVS_PREFIX_ITER_NEXT,
+	PFT_CORTX_KVS_PREFIX_ITER_FINISH,
+	PFT_CORTX_KVS_ITER_GET_KV,
+	PFT_CORTX_KVS_GET_LIST_SIZE,
+
+	PFT_M0_KEY_ITER_FINISH,
+	PFT_M0_KEY_ITER_FIND,
+	PFT_M0_KEY_ITER_NEXT,
+
+	PFT_KVS_END = PFTR_RANGE_4_END,
+};
+
+enum perfc_nsal_entity_attrs {
+	PEA_KVS_START = PEAR_RANGE_4_START,
+
+	PEA_KVS_ALLOC_SIZE,
+	PEA_KVS_ALLOC_RES_RC,
+	PEA_KVS_KLEN,
+	PEA_KVS_VLEN,
+	PEA_KVS_RES_RC,
+
+	PEA_TIME_ATTR_START_NSAL_M0_OP_FINISH,
+	PEA_TIME_ATTR_END_NSAL_M0_OP_FINISH,
+	PEA_TIME_ATTR_START_NSAL_M0_OP_FREE,
+	PEA_TIME_ATTR_END_NSAL_M0_OP_FREE,
+	PEA_TIME_ATTR_START_NSAL_M0_OP_LAUNCH,
+	PEA_TIME_ATTR_END_NSAL_M0_OP_LAUNCH,
+	PEA_TIME_ATTR_START_NSAL_M0_OP_WAIT,
+	PEA_TIME_ATTR_END_NSAL_M0_OP_WAIT,
+
+	PEA_NS_RES_RC,
+	PEA_KVS_LIST_SIZE,
+
+	PEA_KVS_END = PEAR_RANGE_4_END
+};
+
+enum perfc_nsal_entity_maps {
+	PEM_KVS_START = PEMR_RANGE_4_START,
+
+	PEM_NSAL_NFS_TO_MOTR,
+	PEM_NSAL_TO_MOTR,
+	PEM_KVS_TO_NFS,
+
+	PEM_KVS_END = PEMR_RANGE_4_END
+};
+
+/******************************************************************************/
+#endif /* __CFS_NSAL_PERF_COUNTERS_H_ */

src/include/global_kvs.h

@@ -0,0 +1,58 @@
+/*
+ * Filename: global_kvs.h
+ * Description: Declaration for NSAL global kvs api
+ *
+ * Copyright (c) 2020 Seagate Technology LLC and/or its Affiliates
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published
+ * by the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ * See the GNU Affero General Public License for more details.
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ * For any questions about this software or licensing,
+ * please email opensource@seagate.com or cortx-questions@seagate.com.
+ */
+
+/** Global KV-Store operations **/
+/* The functionality provided by below APIs is dependent on basic
+ * framework provided kvstore.h
+ */ 
+
+#ifndef GLOBAL_KVS_H
+#define GLOBAL_KVS_H
+
+#include "kvstore.h"
+
+struct global_kvs_hdl {
+	kvs_idx_fid_t *fid;
+	struct kvstore *kvstore;
+	struct kvs_idx index;
+};
+
+/* Set an index mentioned by fid_str as global kvstore. Here it is assumed
+ * that the index has been created. Below call just opens the index. */
+int nsal_set_global_kvs(const char *fid_str);
+
+/* Allocate memory */
+int nsal_global_alloc(void **ptr, size_t size);
+
+/* Free the allocated memory */
+void nsal_global_free(void *ptr);
+
+/* Store KV pair in the global kvs */
+int nsal_global_set_kv(void *key, const size_t klen,
+		       void *value, const size_t vlen);
+
+/* Get KV pair stored in the global kvs */
+int nsal_global_get_kv(void *key, const size_t klen,
+		       void **value, size_t *vlen);
+
+/* Delete the KV pair stored in the global kvs */
+int nsal_global_del_kv(const void *key, size_t klen);
+
+
+#endif /*GLOBAL_KVS_H */

src/kvstore/CMakeLists.txt

@@ -1,5 +1,6 @@
 SET(base_kvstore_LIB_SRCS
     kvstore_base.c
+    global_kvs.c
 )
 
 add_library(base_kvstore OBJECT ${base_kvstore_LIB_SRCS})