Backup Status Endpoint Deprecated

[guitarhero23]

I had previously submitted a pull request which was accepted for the new backup endpoint for initiating a backup however nothing was included to correct the similar issue in returning the current status of the backup which is referenced in both the back_complete and backup_complete methods. I was going to just not show a status when creating/downloading the backup in my script however the backup_complete method which returns a boolean if complete references the same endpoint so if fails as well.

To Reproduce
Steps to reproduce the behavior:

from jira import JIRA
jira = JIRA(basic_auth=('username', 'password'), options = {'server':'https://domainhere.atlassian.net'})

StartBackup = JIRA.backup(jira)
BackupStatus = JIRA.backup_progress(jira)
BackupFinish = JIRA.backup_complete(jira)


StartBackup
while BackupFinish in False:
    # print(BackupStatus)
    print('Backup still in progress...')
    time.sleep(10)

else:
    JIRA.backup_download(jira)

Expected behavior
Return information about the current status of the Jira and when it is true download the backup

Stack Trace

jira.exceptions.JIRAError: JiraError HTTP 404 url: https://domainhere.atlassian.net/rest/obm/1.0/getprogress?_=1532377086814

Version Information
Python Interpreter: 3.6
jira-python: current
OS: Windows 10

I'll try and research and do a pull request if I find the new endpoint

[guitarhero23]

@hdost I might need someone else to code it out as I feel like I'll mangle it, however I think I can get someone to the point of understanding exactly how.

So the new progress endpoint is '/rest/backup/1/export/getProgress?taskId=' where you pass in the taskId. To get the task Id the endpoint is '/rest/backup/1/export/lastTaskId' (if another task was done at the same time could this grab the wrong one?)

So if you took in the taskId from /lastTaskId you'd get something like '17747' as the response, literally just the number. Then if you did '/rest/backup/1/export/getProgress?taskId=17747' you would get the following response,
{"status":"InProgress","description":"Cloud Export task","message":"Preparing database for export (it may take up to 30 minutes)","progress":0}

Now I'm not sure if the previous endpoint returned the same format response meaning the rest of the code can stay the same or not.

Thoughts?

[guitarhero23]

@hdost to add to my above, the following (crappy) code will successfully start a backup, check the status, and download the file to a specific directory when it's complete. I think I'll leave it to the experts to implement it in the library source.

import datetime
import time
import os
import shutil
import json
import requests




#Authentication
username = 'insertusername'
password = 'insertpassword'
domain = 'insertdomain'

start_backup_url = 'https://'+domain+'.atlassian.net/rest/backup/1/export/runbackup'
headers = {"Accept": "application/json","Content-Type": "application/json"}
start_backup_data = {"cbAttachments":"false","exportToCloud":"true"}


class Payload(object):
    def __init__(self,ParsedJson):
        self.__dict__ = json.loads(ParsedJson)


#Start a backup
StartBackup = requests.post(start_backup_url,data=json.dumps(start_backup_data),headers=headers,auth=(username,password))
StartBackup



TaskIdUrl = 'https://'+domain+'.atlassian.net/rest/backup/1/export/lastTaskId'
GetTaskId = requests.get(TaskIdUrl,headers=headers,auth=(username,password))

#Get TaskId
TaskId = GetTaskId.text
print(GetTaskId.text)

#Get Backup Progress
current_progress = 0
while current_progress < 100:
    get_progress_url = 'https://'+domain+'.atlassian.net/rest/backup/1/export/getProgress?taskId=' + TaskId
    get_progress = requests.get(get_progress_url, headers=headers, auth=(username,password))
    returned_get_progress_json = get_progress.json()
    parsed_get_progress_json = json.dumps(returned_get_progress_json)
    payload_get_progress_json = Payload(parsed_get_progress_json)
    cp = json.loads(parsed_get_progress_json)
    current_progress = cp['progress']
    print('Current Progress ' + str(current_progress)+'%')
    print('Backup still in progress... '+ str(datetime.datetime.now()))

    time.sleep(5)

print('Backup complete, downloading now')

get_progress_url = 'https://'+domain+'.atlassian.net/rest/backup/1/export/getProgress?taskId=' + TaskId
get_progress = requests.get(get_progress_url, headers=headers, auth=(username, password))
returned_get_progress_json = get_progress.json()
parsed_get_progress_json = json.dumps(returned_get_progress_json)
cp = json.loads(parsed_get_progress_json)
unique_download_url = cp['result']



download_url = 'https://'+domain+'.atlassian.net/plugins/servlet/'+unique_download_url
download_request = requests.get(download_url, headers=headers, auth=(username, password))
with open('C:\\Users\\travissimpson\\Downloads\\jira-export.zip', 'wb') as f:
    f.write(download_request.content)

[hdost]

I have confidence that you can get this working, the place you need to fix is

jira/jira/client.py

Lines 2715 to 2802 in 6ffa0f0

	def backup(self, filename='backup.zip', attachments=False):
	"""Will call jira export to backup as zipped xml. Returning with success does not mean that the backup process finished."""
	if self.deploymentType == 'Cloud':
	url = self._options['server'] + '/rest/backup/1/export/runbackup'
	payload = json.dumps({"cbAttachments": attachments})
	self._options['headers']['X-Requested-With'] = 'XMLHttpRequest'
	else:
	url = self._options['server'] + '/secure/admin/XmlBackup.jspa'
	payload = {'filename': filename}
	try:
	r = self._session.post(url, headers=self._options['headers'], data=payload)
	if r.status_code == 200:
	return True
	else:
	logging.warning(
	'Got %s response from calling backup.' % r.status_code)
	return r.status_code
	except Exception as e:
	logging.error("I see %s", e)
	def backup_progress(self):
	"""Return status of cloud backup as a dict.
	Is there a way to get progress for Server version?
	"""
	epoch_time = int(time.time() * 1000)
	if self.deploymentType == 'Cloud':
	url = self._options['server'] + '/rest/obm/1.0/getprogress?_=%i' % epoch_time
	else:
	logging.warning(
	'This functionality is not available in Server version')
	return None
	r = self._session.get(
	url, headers=self._options['headers'])
	# This is weird. I used to get xml, but now I'm getting json
	try:
	return json.loads(r.text)
	except Exception:
	import defusedxml.ElementTree as etree
	progress = {}
	try:
	root = etree.fromstring(r.text)
	except etree.ParseError as pe:
	logging.warning('Unable to find backup info. You probably need to initiate a new backup. %s' % pe)
	return None
	for k in root.keys():
	progress[k] = root.get(k)
	return progress
	def backup_complete(self):
	"""Return boolean based on 'alternativePercentage' and 'size' returned from backup_progress (cloud only)."""
	if self.deploymentType != 'Cloud':
	logging.warning(
	'This functionality is not available in Server version')
	return None
	status = self.backup_progress()
	perc_complete = int(re.search(r"\s([0-9]*)\s",
	status['alternativePercentage']).group(1))
	file_size = int(status['size'])
	return perc_complete >= 100 and file_size > 0
	def backup_download(self, filename=None):
	"""Download backup file from WebDAV (cloud only)."""
	if self.deploymentType != 'Cloud':
	logging.warning(
	'This functionality is not available in Server version')
	return None
	remote_file = self.backup_progress()['fileName']
	local_file = filename or remote_file
	url = self._options['server'] + '/webdav/backupmanager/' + remote_file
	try:
	logging.debug('Writing file to %s' % local_file)
	with open(local_file, 'wb') as file:
	try:
	resp = self._session.get(url, headers=self._options['headers'], stream=True)
	except Exception:
	raise JIRAError()
	if not resp.ok:
	logging.error("Something went wrong with download: %s" % resp.text)
	raise JIRAError(resp.text)
	for block in resp.iter_content(1024):
	file.write(block)
	except JIRAError as je:
	logging.error('Unable to access remote backup file: %s' % je)
	except IOError as ioe:
	logging.error(ioe)
	return None

I would just go ahead and submit a PR.

[asqui]

I agree with @hdost... @guitarhero23: Have a go at fixing it in the library -- we will review your change before it gets merged, and can help with suggestions or corrections.

If you get stuck on the process of how to clone the repo and create a pull request, we can help you with that too. :-)

[montaz708]

Hey guys, a while ago I referenced this solution to add to our company's workflow for reporting off of our Jira data. As of recently it looks like Jira changed the way you can authenticate to use their services. You have to set up a token and then you can create a HTTPBasicAuth object which comes from requests.auth. So building off of @guitarhero23 's solution, everywhere you see auth=(username, password) you would have to replace it with that HTTPBasicAuth object. I just do auth = HTTPBasicAuth(username, token). Thanks for this solution, it really enabled our team to get the data we needed.

[guitarhero23]

Good to hear @montaz708 and yes, with the changes they made you're free to just pass the token in place of the password and everything should work as expected. Speaking of which I believe this bug can be closed now since my pull request was accepted, I think. It was a year ago so I forget, I'll try and look it up.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.