I think the download URL in EE has changed.

[HanDuwol]

The source code below is the source code I added the status_code code to check. Is there any place to know the information of the download URL? Or does anyone know?

url = "https://earthexplorer.usgs.gov/download/12864/LC81150342020112LGN00/STANDARD/EE" def _download(self, url, output_dir, chunk_size=1024): """Download remote file given its URL.""" with self.session.get(url, stream=True, allow_redirects=True) as r: if r.status_code == 404: raise "404 not found..." file_size = int(r.headers['Content-Length']) Common.Parameter.FileSize = file_size with tqdm(total=file_size, unit_scale=True, unit='B', unit_divisor=1024) as pbar: local_filename = r.headers['Content-Disposition'].split('=')[-1] local_filename = os.path.join(output_dir, local_filename) with open(local_filename, 'wb') as f: for chunk in r.iter_content(chunk_size=chunk_size): if chunk: f.write(chunk) pbar.update(chunk_size) return local_filename

[dbb1215]

i think is not a bug ,i user netcore to download ，this url(https://earthexplorer.usgs.gov/download/12864/LC81150342020112LGN00/STANDARD/EE),retrun 404 page not found.

[HanDuwol]

Thanks for the feedback, could you ever know your download link? I downloaded the material from that link until last month!

[dbb1215]

Thanks for the feedback, could you ever know your download link? I downloaded the material from that link until last month!

now,i can't download,i not have a function to solve it,so,i'm so sorry.

[HanDuwol]

If possible, could you please provide a link if there was a notice that USGS' EarthExplorer does not provide a download service?

[yannforget]

Yes the download URLs changed again. They are also depreciating the current API and replacing it with a new one. But the documentation is offline for now.

I'm gonna have a look today, thanks for the bug reports!

[yannforget]

It seems that the new download URL is:

https://earthexplorer.usgs.gov/download/{dataset_id}/{scene_id}/EE/

with the following dataset IDs:

DATASETS = {
    "LANDSAT_TM_C1": "5e83d08ee1e2890b",
    "LANDSAT_ETM_C1": "5e83a506f72553b6",
    "LANDSAT_8_C1": "5e83d0b656b77cf3"
}

[gavrilobos]

Is this new download URL working for you?

I managed to patch the login, but am getting an
{"errorMessage":"Unable to find product configuration","isPending":false,"url":null}
from the server when I try to download a scene (link https://earthexplorer.usgs.gov/download/5e83d0b656b77cf3/LC81721122013331LGN01/EE for example.)

The landsatxplore scene search does generate an 'Download URL' field for each scene, however when i open it I have to manually press the button on the page in order to generate a download for the needed dataset, not sure how or if I can automate it to parse the button in JS.

[gavrilobos]

I managed to patch the login, but am getting an
{"errorMessage":"Unable to find product configuration","isPending":false,"url":null}
from the server when I try to download a scene (link https://earthexplorer.usgs.gov/download/5e83d0b656b77cf3/LC81721122013331LGN01/EE for example.)

To elaborate more on the issue, this is an error when I try to access from a web browser.

In python, in the _download() function, the 'r' (self.session.get(url, stream=True, allow_redirects=True) as r) is an Error 404, for r.headers I'm getting
{'Connection': 'Keep-Alive', 'Server': 'Apache', 'Pragma': 'no-cache', 'Expires': 'Thu, 19 Nov 1981 08:52:00 GMT', 'Transfer-Encoding': 'chunked', 'Content-Type': 'text/html; charset=UTF-8', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains, max-age=31536000', 'Keep-Alive': 'timeout=15, max=79', 'Cache-Control': 'no-store, no-cache, must-revalidate', 'Date': 'Sun, 02 Aug 2020 07:58:35 GMT', 'X-Frame-Options': 'SAMEORIGIN'}
and the code breaks on the file_size = int(r.headers['Content-Length']) part of the code, getting an KeyError: 'content-length' .

I have also skipped the logged_in() check, because I couldn't find the EE_LOGOUT_URL in rsp.text , could it be possible that I'm in fact not logged in then, and thus getting the download errors?

[yannforget]

Hi,

Yes, the link was working 3 days ago but it's not the case any more. The URL is now returning JSON data so I think they are in the process of implementing their new M2M (machine-to-machine) API. I am gonna try to get more information.

Thanks for the bug reports. Note that you can also use pylandsat which is similar but does not rely on EarthExplorer.

[gavrilobos]

Thank you so very much.

The pylandsat library works perfectly.

[yannforget]

I pushed a new version to pypi (v0.7) that should fix the issue.

[HanDuwol]

I pushed a new version to pypi (v0.7) that should fix the issue.

Thank you so much for reflecting my report!

I think it's extracted with a double quotation mark (") in the name of the download file

local_filename = r.headers['Content-Disposition'].split('=')[-1]
→ "LC08_L1TP_115036_20200421_20200508_01_T1.tar.gz"

local_filename = r.headers['Content-Disposition'].split('=')[-1].split('"')[1]
I modified it to confirm that it was downloaded normally!

Thanks!

[yannforget]

Thanks! v0.8 should fix that