No results for relationship with filter revoked = false

[Eiwanger]

Hello,

I tried to use the relationship code from Usage, where the query looks like this:
relationships = thesrc.query([ Filter('type', '=', 'relationship'), Filter('relationship_type', '=', rel_type), Filter('revoked', '=', False) ]).

With this query I get no results. But if I remove the Filter('revoked', '=', False) I get results.
thesrc is connected correctly with
collection = Collection(f"https://cti-taxii.mitre.org/stix/collections/{collection_Id["enterprise_attack"]}/")
thesrc = TAXIICollectionSource(collection).
Queries for course-of-action or intrusion-set for example work fine.
The error only occurs with a live connection. If I download the master branch I can access the relationships without problems.

Is this behaviour intentional?

[isaisabel]

The difference of behavior between local filesystem queries and TAXII queries is odd. It could be a difference in how the TAXII server handles absent fields VS a local MemoryStore or FileSystemSource -- I don't believe any relationship objects have the revoked field in either the true or false state. Further testing is likely necessary to determine the exact cause of the inconsistency.

Regardless, it's OK to remove that revoked filter on the get_relationships function. We don't revoke or deprecate relationship type objects at present, they're simply deleted when no longer relevant. The presence of that filter is mostly future-proofing of the code in case relationships are ever revoked in the future.

So it's fine to turn this:

relationships = thesrc.query([
    Filter('type', '=', 'relationship'),
    Filter('relationship_type', '=', rel_type),
    Filter('revoked', '=', False)
])

Into this:

relationships = thesrc.query([
    Filter('type', '=', 'relationship'),
    Filter('relationship_type', '=', rel_type)
])

For what it's worth, that relationship code is used in the ATT&CK Website source code. The ATT&CK website doesn't use the TAXII server however.

[isaisabel]

I'll look into the inconsistency and update the documentation to remove that line if need be.

[isaisabel]

After some testng it does appear that the TAXII server doesn't behave the same with the explicit removal of revoked objects which don't have the revoked field. Probably the code example here should be used for filtering revoked objects for this reason (The code example notes that .get() is used because the property may not be present: https://github.com/mitre/cti/blob/master/USAGE.md#removing-revoked-and-deprecated-objects

Here's the test script I used to determine that the behavior was inconsistent:

from stix2 import FileSystemSource, Filter, TAXIICollectionSource
from taxii2client.v20 import Collection # only specify v20 if your installed version is >= 2.0.0

collections = {
    "enterprise_attack": "95ecc380-afe9-11e4-9b6c-751b66dd541e",
    "mobile_attack": "2f669986-b40b-4423-b720-4396ca6a462b",
    "ics-attack": "02c3ef24-9cd4-48f3-a99f-b74ce24f1d34"
}

collection = Collection(f"https://cti-taxii.mitre.org/stix/collections/{collections['enterprise_attack']}/")
livesrc = TAXIICollectionSource(collection)
localsrc = FileSystemSource('./enterprise-attack')


def getRelationships_revokedexplicit(thesrc):
    return thesrc.query([
        Filter('type', '=', 'relationship'),
        Filter('relationship_type', '=', "uses"),
        Filter('revoked', '=', False)
    ])

def getRelationships(thesrc):
    return thesrc.query([
        Filter('type', '=', 'relationship'),
        Filter('relationship_type', '=', "uses")
    ])

print("live explicit revoked removal", len(getRelationships_revokedexplicit(livesrc)))
print("local explicit revoked removal", len(getRelationships_revokedexplicit(localsrc)))

print("live no explicit revoked removal", len(getRelationships(livesrc)))
print("local no explicit revoked removal", len(getRelationships(localsrc)))

$ python3 test.py
live explicit revoked removal 0
local explicit revoked removal 7340
live no explicit revoked removal 7340
local no explicit revoked removal 7340

I'll make note of this in the documentation including updating the relationship microlibrary to no longer specify the revoked field for relationships.

[isaisabel]

Following up to say I found some more issues stemming from the TAXII revoked weirdness and have updated the USAGE document accordingly.