[feature request] stop an harvest job in progress

[virgilejarrige]

Hello,

I ran into the issue described here: https://groups.google.com/g/dataverse-community/c/O-NdDtgFrI0/m/os_KjdLxAQAJ

The process worked, but maybe it would be a good idea to have a button that would do the same thing on the harvest page? or maybe an API to reset stuck harvest?

Here is what i've done:
#manually get the harvest job's id:
select * from harvestingclient;

#fix the issue - where {ID} is the database id of the harvesting client.
UPDATE clientharvestrun SET harvestresult=0 WHERE harvestingclient_id={ID} AND harvestresult = 2;
UPDATE harvestingclient SET harvestingnow = FALSE WHERE id={ID};

#restart payara "just in case"
systemctl restart payara

Take care,

Virgile

[djbrooke]

Thanks @virgilejarrige, and good to see you at some #dataverse2021 sessions this week. I think it's a good idea to not have this be reliant on a DB update, but we should also examine and fix the failure cases that results in this condition in the first place as well.

[virgilejarrige]

Hey Danny!

We had two cases in which this happened:

1 - The harvest of an entire data repository (Nakala) - which worked but was so huge it made our postgresql bdd too big for the VM it was in. As it's on our "all-in-one" test server, we had to use a snapshot to restore it.
That was a noob mistake, but would have been nice to be able to stop it. ;-)

2 - The haverst of an ahp collection - which was ended with the DP Update and in the dashboard now we have "SUCCESS; 0 harvested, 0 deleted, 0 failed."
For this one, here are the parameters if you want to test it on your side:
URL: https://archives.ahp-numerique.fr/index.php/;oai
OAI Set: oai:archives.ahp-numerique.fr:ahpoai_406
Metadata Format: oai_dc
Schedule: None
Archive Type: Generic OAI archive
Archive URL: https://archives.ahp-numerique.fr

[mreekie]

sprint:

• this is an older one. Frequent request.
• Harvesting jobs can take a long time and currently you are stuck waiting for the end once it starts.
• This does not have an immediate solution.
• Solution is not likely complex.

Desired behavior is to stop the harvesting that is in progress as opposed to a pause state.
The current workaround is restarting the app server, so providing a stop will be a large improvement.

• small. e.g. implement a binary "stop" flag that client checks after every dataset import.

[mreekie]

Sprint

• orphaned in OnDeck in pm.sprint.2022_05_11

[mreekie]

Sprint:

• pm.sprint.2022_05_25 ended WIP

[mreekie]

Waiting on PR8753 to clear.
Leonid was looking to work on this but this is not important.
Phil - noted that customers have noticed this in the field.
Gustavo - let's push this to the sprint following this.

[pdurbin]

Is this issue in scope as well?

• Harvesting run infinitely stays at IN PROGRESS and DELETE IN PROGRESS #7052