Blob tag queries do not support tag keys with special characters except for _

[delreluca]

According to the docs both tag keys and values can contain the special characters +-.:=_/. However most of them need to be enclosed in double quotes for it to work in the query.

Azurite’s query parser always restricts tag keys to only allow _ as special character. This is correct when encountering an unquoted tag key I think, but it’s not correct when the tag key is quoted.

It is possible to set such a tag in Azurite and retrieve it back without any problems, so this is strictly a problem in the query parser it seems. I haven’t stepped through it because I don’t know TypeScript enough but these seems to be the relevant lines

Azurite/src/blob/persistence/QueryInterpreter/QueryParser.ts

Lines 317 to 348 in 1dd0784

	private ContainsInvalidTagKeyCharacter(key: string): boolean {
	for (let c of key) {
	if (!(c >= 'a' && c <= 'z' ||
	c >= 'A' && c <= 'Z' ||
	c >= '0' && c <= '9' ||
	c == '_')) {
	return true;
	}
	}
	return false;
	}
	private validateKey(key: string) {
	if (key.startsWith("@")) {
	if (this.conditionHeader) {
	this.query.throw("");
	}
	if (key !== "@container") {
	this.query.throw(`unsupported parameter '${key}'`);
	}
	// Key is @container, no need for further check.
	return;
	}
	if (!this.conditionHeader && ((key.length == 0) || (key.length > 128))) {
	this.query.throw('tag must be between 1 and 128 characters in length');
	}
	if (this.ContainsInvalidTagKeyCharacter(key)) {
	this.query.throw(`unexpected '${key}'`);
	}
	}

Azurite/src/blob/persistence/QueryInterpreter/QueryParser.ts

Lines 409 to 441 in 1dd0784

	if (isAKey) {
	const keyName = content.replace(new RegExp(`${openCharacter}${openCharacter}`, 'g'), openCharacter);
	this.validateKey(keyName);
	return new KeyNode(keyName);
	}
	else {
	const value = content.replace(new RegExp(`${openCharacter}${openCharacter}`, 'g'), openCharacter);
	this.validateValue(value);
	return new ConstantNode(value);
	}
	}
	/**
	* Visits the IDENTIFIER layer of the query syntax tree, returning the appropriate node.
	*
	* Identifiers are a sequence of characters which are not whitespace.
	*
	* @returns {IQueryNode}
	*/
	private visitKey(): IQueryNode {
	// A key name can be surrounded by double quotes.
	if (`"`.includes(this.query.peek())) {
	return this.visitString(true);
	}
	else {
	const identifier = this.query.take(
	c => !!c.trim() && c !== '=' && c != '>' && c !== '<'
	) || this.query.throw(`Expected a valid identifier, but found '${this.query.peek()}' instead.`);
	this.validateKey(identifier);
	return new KeyNode(identifier)
	}
	}
	}

There is a distinction between quoted and unquoted keys but both branches eventually call the same validation function that only accepts _ as special character.

Exemplary exception from the .NET SDK below. A query like this works in az CLI against actual Azure Blob Storage.

Azure.RequestFailedException: Error parsing query at or near character position 19: unexpected 'key.with.dots'
RequestId:6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Time:2024-12-14T16:20:31.299Z
Status: 400 (Error parsing query at or near character position 19: unexpected 'key.with.dots')
ErrorCode: InvalidQueryParameterValue

Additional Information:
QueryParameterName: where
QueryParameterValue:     "key.with.dots" = 'value'

Content:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidQueryParameterValue</Code>
  <Message>Error parsing query at or near character position 19: unexpected 'key.with.dots'
RequestId:6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Time:2024-12-14T16:20:31.299Z</Message>
  <QueryParameterName>where</QueryParameterName>
  <QueryParameterValue>    "key.with.dots" = 'value'</QueryParameterValue>
</Error>

Headers:
Server: Azurite-Blob/3.33.0
x-ms-error-code: InvalidQueryParameterValue
x-ms-request-id: 6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Date: Sat, 14 Dec 2024 16:20:31 GMT
Connection: keep-alive
Keep-Alive: REDACTED
Transfer-Encoding: chunked
Content-Type: application/xml

   at Azure.Storage.Blobs.ContainerRestClient.FilterBlobsAsync(Nullable`1 timeout, String where, String marker, Nullable`1 maxresults, IEnumerable`1 include, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.FindBlobsByTagsInternal(String marker, String expression, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.Models.FilterBlobsAsyncCollection.GetNextPageAsync(String continuationToken, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at MyApp.App.Worker.FindSomethingToWorkOn(CancellationToken cancellationToken) in /path/to/src/Worker.cs:line 32

[blueww]

@EmmaZhu
Would you please help to look at the blob tag issue?