Some single dockets are spread through multiple docket objects, need merging

[grossir]

Some dockets are spread through multiple docket objects, which need merging. Have seen it in mont and texapp. Seems to happen when docket number core exists

• texapp example
• mont example
• haw example
• delsuperct example

Related to #4256, it is the inverse: some dockets were not matched when they should have

---

Sentry Issue: COURTLISTENER-85X

texapp: more than 1 docket match for docket number '04-22-00669-CR'

[grossir]

This is a script to merge the dockets in this issue , which should be a single docket. To test it, check the commented lines on how to use clone_from_cl, which would require the branch that allows cloning clusters together with dockets to be merged

The dockets included are the ones reported on Sentry, which means they have been updated recently, after the logger call was put in place. We could make a lookup for all the duplicate dockets from these courts, if this sample looks OK

# Example query to get the docket ids
# https://www.courtlistener.com/api/rest/v3/dockets/?court__id=texapp&docket_number=14-22-00805-CV

# texapp example
# manage.py clone_from_cl --type search.Docket --id 63138128 63138127 61628798 61628797 --add-clusters
# texapp example 2
# manage.py clone_from_cl --type search.Docket --id 66610135 65750654 --add-clusters

# mont example
# manage.py clone_from_cl --type search.Docket --id 66906891 66769920 66683333 66556449 65640268 65392234 64686343 63558008  63359967 63273348 62979968 --add-clusters

# haw
# manage.py clone_from_cl --type search.Docket --id 66815418 66636686 65416047 --add-clusters

# del
# manage.py clone_from_cl --type search.Docket --id 6246543 4482784 --add-clusters


from cl.search.models import (
    Docket,
    DocketEntry,
    DocketTags,
    DocketPanel,
    Claim,
    BankruptcyInformation,
)
from cl.alerts.models import DocketAlert
from cl.audio.models import Audio
from cl.favorites.models import DocketTag, Note
from cl.people_db.models import (
    AttorneyOrganizationAssociation,
    PartyType,
    Role,
)
from cl.recap.models import PacerFetchQueue, ProcessingQueue
from cl.search.models import OpinionCluster
from django.db import transaction


models_that_reference_docket = [
    Audio,
    AttorneyOrganizationAssociation,
    PartyType,
    Role,
    PacerFetchQueue,
    ProcessingQueue,
    DocketEntry,
    DocketTags,
    DocketPanel,
    Claim,
    BankruptcyInformation,
    DocketAlert,
    DocketTag,
    Note,
    OpinionCluster,
]

need_merging = [
    ("texapp", "13-20-00065-CR"),
    ("texapp", "03-22-00689-CV"),
    ("texapp", "04-22-00669-CR"),
    ("texapp", "14-22-00805-CV"),
    ("texapp", "03-23-00775-CV"),
    ("texapp", "08-23-00123-CV"),    
    ("texapp", "03-24-00451-CR"),
    
    ("mont", "DA 21-0567"),
    ("mont", "DA 22-0484"),
    ("mont", "DA 22-0572"),
    ("mont", "DA 23-0044"),

    ("haw", "SCMF-12-0000538"),
    ("haw", "SCWC-18-0000099"),
    ("delch", "C.A. No. 2019-0812-KSJM"),
    ("delsuperct", "1201018188"),
]

for court_id, docket_number in need_merging:
    lookup = {"court_id": court_id, "docket_number": docket_number}
    dockets_qs = Docket.objects.filter(**lookup).order_by("-date_created")
    count = dockets_qs.count()
    
    print(f"Lookup: {lookup}")
    if count <= 1:
        print(f"{count} dockets for lookup. Skipping")
        continue

    # get all docket data, in descending date_created order
    # We will keep the oldest docket object
    *to_delete, to_keep = list(dockets_qs)
    dockets_to_delete = [d.id for d in to_delete]

    with transaction.atomic():
        # make all docket_id references point to the docket to keep
        for model in models_that_reference_docket:
            update_count = model.objects.filter(
                docket_id__in=dockets_to_delete
            ).update(docket_id=to_keep.id)
            if update_count:
                print(f"Updated {update_count} {model} objects")
            
        # Delete all dockets, except the one to keep
        deleted_dockets = Docket.objects.filter(
            id__in=dockets_to_delete
        ).delete()
        print(f"Deleted {deleted_dockets} dockets")

[mlissner]

Sounds like this should go to @flooie for review?

[flooie]

We spoke about this - and I think it makes the most sense to turn this into a PR and put it as a management command we can use again instead of a one-off script in the issue here. We think there are more dockets that need merging then these.