Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#OBS-I335: dataset update fix #276

Merged
merged 5 commits into from
Nov 13, 2024
Merged

#OBS-I335: dataset update fix #276

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9ce2373
#OBS-I335: dataset update fix
JeraldJF Nov 13, 2024
a3c3459
#OBS-I335: loop bound issue fix
JeraldJF Nov 13, 2024
939893d
#OBS-I335: dataset updated as per feedbacks
JeraldJF Nov 13, 2024
c01be2a
#OBS-I334 - Fix the schema update functionality
SanthoshVasabhaktula Nov 13, 2024
6d1cb44
Merge branch 'hudi-spec-fix' of github.com:Sanketika-Obsrv/obsrv-api-…
SanthoshVasabhaktula Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 66 additions & 54 deletions api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,11 @@
if (dataset) {
if (dataset.api_version !== "v2") {
throw obsrvError(datasetId, "DATASET_API_VERSION_MISMATCH", "Draft dataset api version is not v2. Perform a read api call with mode=edit to migrate the dataset", "NOT_FOUND", 404)
}
if(dataset.version_key !== versionKey) {
}
if (dataset.version_key !== versionKey) {
throw obsrvError(datasetId, "DATASET_OUTDATED", "The dataset is outdated. Please try to fetch latest changes of the dataset and perform the updates", "CONFLICT", 409)
}
if(!_.includes([DatasetStatus.Draft, DatasetStatus.ReadyToPublish], dataset.status)) {
if (!_.includes([DatasetStatus.Draft, DatasetStatus.ReadyToPublish], dataset.status)) {
throw obsrvError(datasetId, "DATASET_NOT_IN_DRAFT_STATE_TO_UPDATE", "Dataset cannot be updated as it is not in draft state", "BAD_REQUEST", 400)
}
} else {
Expand All @@ -52,53 +52,57 @@
}

const datasetUpdate = async (req: Request, res: Response) => {

await validateRequest(req)
const datasetReq = req.body.request;
const datasetModel = await datasetService.getDraftDataset(datasetReq.dataset_id)
validateDataset(datasetModel, req)

const draftDataset = mergeDraftDataset(datasetModel, datasetReq);
const userID = (req as any)?.userID;
_.set(draftDataset, "updated_by", userID )
_.set(draftDataset, "updated_by", userID)
const response = await datasetService.updateDraftDataset(draftDataset);
ResponseHandler.successResponse(req, res, { status: httpStatus.OK, data: response });
}

const mergeDraftDataset = (datasetModel: Model<any, any> | null, datasetReq: any): Record<string, any> => {

const cache_config = _.get(datasetModel, ["dataset_config", "cache_config"]) || {}
const currentSchema = _.get(datasetModel, 'data_schema')
const fieldsRemoved = (datasetReq.data_schema) ? getMissingFieldsInNewSchema(datasetReq.data_schema, currentSchema) : []
const prev_dataset_config = _.get(datasetModel, ["dataset_config"])
const dataset: Record<string, any> = {
version_key: Date.now().toString(),
name: datasetReq.name || _.get(datasetModel, ["name"]),
id: _.get(datasetModel, ["id"])
}
if(datasetReq.validation_config) dataset["validation_config"] = datasetReq.validation_config
if(datasetReq.extraction_config) dataset["extraction_config"] = datasetReq.extraction_config
if(datasetReq.dedup_config) dataset["dedup_config"] = datasetReq.dedup_config
if(datasetReq.data_schema) dataset["data_schema"] = datasetReq.data_schema
if(datasetReq.dataset_config) dataset["dataset_config"] = { ...prev_dataset_config, ...datasetReq.dataset_config }
if(datasetReq.transformations_config || fieldsRemoved.length > 0)
if (datasetReq.validation_config) dataset["validation_config"] = datasetReq.validation_config
if (datasetReq.extraction_config) dataset["extraction_config"] = datasetReq.extraction_config
if (datasetReq.dedup_config) dataset["dedup_config"] = datasetReq.dedup_config
if (datasetReq.data_schema) dataset["data_schema"] = datasetReq.data_schema
const updatedDatasetConfig = _.merge(_.get(datasetModel, "dataset_config"), { ...datasetReq.dataset_config, cache_config })
if (datasetReq.dataset_config) dataset["dataset_config"] = updatedDatasetConfig
if (datasetReq.transformations_config || fieldsRemoved.length > 0)
dataset["transformations_config"] = mergeTransformationsConfig(_.get(datasetModel, ["transformations_config"]), datasetReq.transformations_config, fieldsRemoved)
if(datasetReq.denorm_config || fieldsRemoved.length > 0) dataset["denorm_config"] = mergeDenormConfig(_.get(datasetModel, ["denorm_config"]), datasetReq.denorm_config, fieldsRemoved)
if(datasetReq.connectors_config) dataset["connectors_config"] = mergeConnectorsConfig(_.get(datasetModel, ["connectors_config"]), datasetReq.connectors_config)
if(datasetReq.tags) dataset["tags"] = mergeTags(_.get(datasetModel, ["tags"]), datasetReq.tags)
if(datasetReq.sample_data) dataset["sample_data"] = datasetReq.sample_data
if(datasetReq.type) dataset["type"] = datasetReq.type
if(fieldsRemoved.length > 0) {
const keys_config = _.get(dataset["dataset_config"] ? dataset["dataset_config"] : prev_dataset_config, ["keys_config"])
if(keys_config['data_key'] in fieldsRemoved) {
if (datasetReq.denorm_config || fieldsRemoved.length > 0) dataset["denorm_config"] = mergeDenormConfig(_.get(datasetModel, ["denorm_config"]), datasetReq.denorm_config, fieldsRemoved)
if (datasetReq.connectors_config) dataset["connectors_config"] = mergeConnectorsConfig(_.get(datasetModel, ["connectors_config"]), datasetReq.connectors_config)
if (datasetReq.tags) dataset["tags"] = mergeTags(_.get(datasetModel, ["tags"]), datasetReq.tags)
if (datasetReq.sample_data) dataset["sample_data"] = datasetReq.sample_data
if (datasetReq.type) dataset["type"] = datasetReq.type
if (fieldsRemoved.length > 0) {
const keys_config = _.get(dataset, ["dataset_config", "keys_config"])
if (_.includes(fieldsRemoved, keys_config['data_key'])) {
keys_config['data_key'] = '';
}
if(keys_config['primary_key'] in fieldsRemoved) {
if (_.includes(fieldsRemoved, keys_config['primary_key'])) {
keys_config['primary_key'] = '';
}
if(keys_config['timestamp_key'] in fieldsRemoved) {
if (_.includes(fieldsRemoved, keys_config['partition_key'])) {
keys_config['partition_key'] = '';
}
if (_.includes(fieldsRemoved, keys_config['timestamp_key'])) {
keys_config['timestamp_key'] = '';
}
_.set(dataset["dataset_config"], 'keys_config', keys_config)
_.set(dataset["dataset_config"], 'keys_config', keys_config)
}
return dataset;
}
Expand All @@ -111,22 +115,30 @@
const fullPath = [...path, key].join('.');
if (!(key in newProperties)) {
removedFields.push(fullPath);
} else if (typeof oldProperties[key] === 'object' && typeof newProperties[key] === 'object') {
if (typeof oldProperties[key] === 'object' && oldProperties[key].properties) {
removedFields = removedFields.concat(getRemovedPropertiesFieldsNested(oldProperties[key].properties || {}, {}, [...path, key]));
}
}
else if (typeof oldProperties[key] === 'object' && typeof newProperties[key] === 'object') {
removedFields = removedFields.concat(
getRemovedPropertiesFieldsNested(oldProperties[key].properties || {}, newProperties[key].properties || {}, [...path, key])
getRemovedPropertiesFieldsNested(
oldProperties[key].properties || {},
newProperties[key].properties || {},
[...path, key]
)
);
}
}

return removedFields;
}

};


const getRemovedPropertiesFields = (oldSchema: any, newSchema: any): string[] => {
const oldProperties = oldSchema.properties || {};
const newProperties = newSchema.properties || {};
return getRemovedPropertiesFieldsNested(oldProperties, newProperties);
}

// Example usage
const removedFieldsNested = getRemovedPropertiesFields(oldSchema, newSchema);
return removedFieldsNested
Expand All @@ -135,41 +147,41 @@
const mergeTransformationsConfig = (currentConfigs: any, newConfigs: any, fieldsRemoved: string[]) => {

let updatedConfigs = currentConfigs;
if(newConfigs) {
const removeConfigs = _.map(_.filter(newConfigs, {action: "remove"}), "value.field_key")
const addConfigs = _.map(_.filter(newConfigs, {action: "upsert"}), "value")
if (newConfigs) {
const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.field_key")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that the newConfigs object is a valid array and has a reasonable length before using it in the loop. We can achieve this by adding a check to confirm that newConfigs is an array and limiting its length to a safe maximum value.

  • First, we will check if newConfigs is an array using Array.isArray.
  • Then, we will limit the length of newConfigs to a maximum value (e.g., 1000) to prevent potential DoS attacks.
Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -157,5 +157,7 @@
     let updatedConfigs = currentConfigs;
-    if (newConfigs && newConfigs.length) {
-        const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.field_key")
-        const addConfigs = _.map(_.filter(newConfigs, { action: "upsert" }), "value")
+    if (Array.isArray(newConfigs) && newConfigs.length > 0) {
+        const maxLength = 1000;
+        const validNewConfigs = newConfigs.slice(0, maxLength);
+        const removeConfigs = _.map(_.filter(validNewConfigs, { action: "remove" }), "value.field_key")
+        const addConfigs = _.map(_.filter(validNewConfigs, { action: "upsert" }), "value")
 
EOF
@@ -157,5 +157,7 @@
let updatedConfigs = currentConfigs;
if (newConfigs && newConfigs.length) {
const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.field_key")
const addConfigs = _.map(_.filter(newConfigs, { action: "upsert" }), "value")
if (Array.isArray(newConfigs) && newConfigs.length > 0) {
const maxLength = 1000;
const validNewConfigs = newConfigs.slice(0, maxLength);
const removeConfigs = _.map(_.filter(validNewConfigs, { action: "remove" }), "value.field_key")
const addConfigs = _.map(_.filter(validNewConfigs, { action: "upsert" }), "value")

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
const addConfigs = _.map(_.filter(newConfigs, { action: "upsert" }), "value")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfigs is an array before using its .length property in the loop. This can be done by adding a check to confirm that newConfigs is an instance of Array. If it is not, we can either initialize it as an empty array or handle the error appropriately.

Steps to fix:

  1. Add a check to ensure newConfigs is an array before using it in the loop.
  2. If newConfigs is not an array, initialize it as an empty array or handle the error.
Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -157,3 +157,3 @@
     let updatedConfigs = currentConfigs;
-    if (newConfigs && newConfigs.length) {
+    if (Array.isArray(newConfigs) && newConfigs.length) {
         const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.field_key")
EOF
@@ -157,3 +157,3 @@
let updatedConfigs = currentConfigs;
if (newConfigs && newConfigs.length) {
if (Array.isArray(newConfigs) && newConfigs.length) {
const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.field_key")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

updatedConfigs = _.unionWith(
addConfigs,
_.reject(currentConfigs, (config) => { return _.includes(removeConfigs, config.field_key)}),
(a, b) => {
_.reject(currentConfigs, (config) => { return _.includes(removeConfigs, config.field_key) }),
(a, b) => {
return a.field_key === b.field_key
}
}
)
}
if(fieldsRemoved.length > 0) {
updatedConfigs = _.reject(updatedConfigs, (config) => { return _.includes(fieldsRemoved, config.field_key)})
if (fieldsRemoved.length > 0) {
updatedConfigs = _.reject(updatedConfigs, (config) => { return _.includes(fieldsRemoved, config.field_key) })
}
return updatedConfigs
}

const mergeDenormConfig = (currentConfig: any, newConfig: any, fieldsRemoved: string[]) => {

let updatedConfigs = currentConfig.denorm_fields;
if(newConfig) {
const removeConfigs = _.map(_.filter(newConfig.denorm_fields, {action: "remove"}), "value.denorm_out_field")
const addConfigs = _.map(_.filter(newConfig.denorm_fields, {action: "upsert"}), "value")
if (newConfig) {
const removeConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "remove" }), "value.denorm_out_field")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfig.denorm_fields is an array and limit its length to a reasonable number before iterating over it. This can be done by adding a validation check before the loop.

  1. Check if newConfig.denorm_fields is an array.
  2. Limit the length of newConfig.denorm_fields to a reasonable maximum value (e.g., 1000) to prevent potential DoS attacks.
Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -178,5 +178,6 @@
     let updatedConfigs = currentConfig.denorm_fields;
-    if (_.get(newConfig, "denorm_fields")) {
-        const removeConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "remove" }), "value.denorm_out_field")
-        const addConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "upsert" }), "value")
+    const denormFields = _.get(newConfig, "denorm_fields");
+    if (Array.isArray(denormFields) && denormFields.length <= 1000) {
+        const removeConfigs = _.map(_.filter(denormFields, { action: "remove" }), "value.denorm_out_field")
+        const addConfigs = _.map(_.filter(denormFields, { action: "upsert" }), "value")
 
EOF
@@ -178,5 +178,6 @@
let updatedConfigs = currentConfig.denorm_fields;
if (_.get(newConfig, "denorm_fields")) {
const removeConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "remove" }), "value.denorm_out_field")
const addConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "upsert" }), "value")
const denormFields = _.get(newConfig, "denorm_fields");
if (Array.isArray(denormFields) && denormFields.length <= 1000) {
const removeConfigs = _.map(_.filter(denormFields, { action: "remove" }), "value.denorm_out_field")
const addConfigs = _.map(_.filter(denormFields, { action: "upsert" }), "value")

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
const addConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "upsert" }), "value")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfig.denorm_fields is a valid array before using its length property in the loop. This can be done by adding a check to confirm that newConfig.denorm_fields is an array and has a reasonable length. If it is not, we should handle the error appropriately.

Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -178,3 +178,3 @@
     let updatedConfigs = currentConfig.denorm_fields;
-    if (_.get(newConfig, "denorm_fields")) {
+    if (_.get(newConfig, "denorm_fields") && Array.isArray(newConfig.denorm_fields) && newConfig.denorm_fields.length < 1000) { // Validate newConfig.denorm_fields
         const removeConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "remove" }), "value.denorm_out_field")
EOF
@@ -178,3 +178,3 @@
let updatedConfigs = currentConfig.denorm_fields;
if (_.get(newConfig, "denorm_fields")) {
if (_.get(newConfig, "denorm_fields") && Array.isArray(newConfig.denorm_fields) && newConfig.denorm_fields.length < 1000) { // Validate newConfig.denorm_fields
const removeConfigs = _.map(_.filter(newConfig.denorm_fields, { action: "remove" }), "value.denorm_out_field")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

const denormFields = _.unionWith(
updatedConfigs = _.unionWith(
addConfigs,
_.reject(currentConfig.denorm_fields, (config) => { return _.includes(removeConfigs, config.denorm_out_field)}),
(a, b) => {
_.reject(currentConfig.denorm_fields, (config) => { return _.includes(removeConfigs, config.denorm_out_field) }),
(a, b) => {
return a.denorm_out_field === b.denorm_out_field
}
}
)
}
if(fieldsRemoved.length > 0) {
updatedConfigs = _.reject(currentConfig.denorm_fields, (config) => { return _.includes(fieldsRemoved, config.denorm_key)})
if (fieldsRemoved.length > 0) {
updatedConfigs = _.reject(updatedConfigs, (config) => { return _.includes(fieldsRemoved, config.denorm_key) })
}
return {
denorm_fields: updatedConfigs
Expand All @@ -178,8 +190,8 @@

const mergeConnectorsConfig = (currConfigs: any, newConfigs: any) => {

const removeConfigs = _.map(_.filter(newConfigs, {action: "remove"}), "value.connector_id")
const addConfigs = _.map(_.filter(newConfigs, {action: "upsert"}), "value")
const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.connector_id")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfigs is an array before using its .length property in the loop. This can be done by adding a check to confirm that newConfigs is an instance of an array. If it is not, we should handle the error appropriately, possibly by returning an empty array or an error response.

Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -200,2 +200,6 @@
 
+    if (!Array.isArray(newConfigs)) {
+        throw new Error("Invalid input: newConfigs should be an array");
+    }
+
     const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.connector_id")
EOF
@@ -200,2 +200,6 @@

if (!Array.isArray(newConfigs)) {
throw new Error("Invalid input: newConfigs should be an array");
}

const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.connector_id")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
const addConfigs = _.map(_.filter(newConfigs, { action: "upsert" }), "value")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfigs is an array before using its .length property in a loop. This can be done by adding a check to confirm that newConfigs is an instance of Array. If it is not, we should handle the error appropriately, such as returning an empty array or throwing an error.

Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -200,2 +200,6 @@
 
+    if (!(newConfigs instanceof Array)) {
+        throw new Error("Invalid input: newConfigs must be an array");
+    }
+
     const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.connector_id")
EOF
@@ -200,2 +200,6 @@

if (!(newConfigs instanceof Array)) {
throw new Error("Invalid input: newConfigs must be an array");
}

const removeConfigs = _.map(_.filter(newConfigs, { action: "remove" }), "value.connector_id")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

return _.unionWith(
_.map(addConfigs, (config) => {
Expand All @@ -191,17 +203,17 @@
version: config.version
}
}),
_.reject(currConfigs, (config) => { return _.includes(removeConfigs, config.connector_id)}),
(a, b) => {
_.reject(currConfigs, (config) => { return _.includes(removeConfigs, config.connector_id) }),
(a, b) => {
return a.connector_id === b.connector_id
}
}
)
}

const mergeTags = (currentTags: any, newConfigs: any) => {

const tagsToRemove = _.map(_.filter(newConfigs, {action: "remove"}), "value")
const tagsToAdd = _.map(_.filter(newConfigs, {action: "upsert"}), "value")
const tagsToRemove = _.map(_.filter(newConfigs, { action: "remove" }), "value")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfigs is an array before using its .length property in a loop. This can be done by adding a check to verify that newConfigs is an instance of Array. If it is not, we should handle the error appropriately, such as returning an empty array or throwing an error.

Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -222,2 +222,6 @@
 
+    if (!(newConfigs instanceof Array)) {
+        throw new Error("Invalid input: newConfigs must be an array");
+    }
+
     const tagsToRemove = _.map(_.filter(newConfigs, { action: "remove" }), "value")
EOF
@@ -222,2 +222,6 @@

if (!(newConfigs instanceof Array)) {
throw new Error("Invalid input: newConfigs must be an array");
}

const tagsToRemove = _.map(_.filter(newConfigs, { action: "remove" }), "value")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
const tagsToAdd = _.map(_.filter(newConfigs, { action: "upsert" }), "value")

Check failure

Code scanning / CodeQL

Loop bound injection High

Iteration over a user-controlled object with a potentially unbounded .length property from a
user-provided value
Loading
.

Copilot Autofix AI about 2 months ago

To fix the problem, we need to ensure that newConfigs is an array before using its .length property in a loop. This can be done by adding a check to verify that newConfigs is an instance of an array. If it is not, we should handle the error appropriately, such as returning an empty array or throwing an error.

Suggested changeset 1
api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
--- a/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
+++ b/api-service/src/controllers/DatasetUpdate/DatasetUpdate.ts
@@ -222,2 +222,6 @@
 
+    if (!Array.isArray(newConfigs)) {
+        throw new Error("Invalid input: newConfigs must be an array");
+    }
+
     const tagsToRemove = _.map(_.filter(newConfigs, { action: "remove" }), "value")
EOF
@@ -222,2 +222,6 @@

if (!Array.isArray(newConfigs)) {
throw new Error("Invalid input: newConfigs must be an array");
}

const tagsToRemove = _.map(_.filter(newConfigs, { action: "remove" }), "value")
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
return _.union(_.pullAll(currentTags, tagsToRemove), tagsToAdd)
}

Expand Down
Loading