Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

a single esptool command for erasing and flashing #6765

Merged
merged 4 commits into from
Nov 14, 2019

Conversation

d-a-v
Copy link
Collaborator

@d-a-v d-a-v commented Nov 13, 2019

Per #6755 (comment)
(thanks @projectgus for the hints)

esptool:

  • a single esptool command for erasing (all or a region) and flashing
  • update to esptool-2.8

fixes #6755

@d-a-v d-a-v added this to the 2.6.1 milestone Nov 13, 2019
@d-a-v
Copy link
Collaborator Author

d-a-v commented Nov 13, 2019

@ChazTuna @janzieg @s0170071 @f5soh
Can you please try this pull request
(I also updated the alpha release channel with it)

@f5soh
Copy link
Contributor

f5soh commented Nov 13, 2019

Can you please try this pull request

esptool v2.8 looks good, tested basic programmer without dtr and Nodemcu v0.9 : full erase, partial erase and only sketch

A little improvement will be output scrolling while writing.

esptool.py cmdline: ['--chip', 'esp8266', '--port', '/dev/ttyUSB0', '--baud', '115200', '--before', 'no_reset', '--after', 'soft_reset', 'write_flash', '0x0', '/tmp/arduino_build_628729/Wifi_Angle_meter_mma8452_esp01.ino.bin']
esptool.py v2.8
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 84:f3:eb:b4:e1:72
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 1MB
Compressed 405648 bytes to 265513...

Writing at 0x00000000... (5 %)
Writing at 0x00004000... (11 %)
Writing at 0x00008000... (17 %)
Writing at 0x0000c000... (23 %)
Writing at 0x00010000... (29 %)
Writing at 0x00014000... (35 %)
Writing at 0x00018000... (41 %)
Writing at 0x0001c000... (47 %)
Writing at 0x00020000... (52 %)
Writing at 0x00024000... (58 %)
Writing at 0x00028000... (64 %)
Writing at 0x0002c000... (70 %)
Writing at 0x00030000... (76 %)
Writing at 0x00034000... (82 %)
Writing at 0x00038000... (88 %)
Writing at 0x0003c000... (94 %)
Writing at 0x00040000... (100 %)
Wrote 405648 bytes (265513 compressed) at 0x00000000 in 33.2 seconds (effective 97.8 kbit/s)...
Hash of data verified.

Leaving...
Soft resetting...

@d-a-v
Copy link
Collaborator Author

d-a-v commented Nov 13, 2019

A little improvement will be output scrolling while writing.

That, I thought it was only an effect on my setup (linux, xcfe) but it appears to be true even on windows.
I think that is the way the arduino IDE console interprets the \r character

Copy link
Collaborator

@earlephilhower earlephilhower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor tweaks. I still need to run it to verify but it seems good.

tools/upload.py Show resolved Hide resolved
tools/upload.py Outdated Show resolved Hide resolved
tools/upload.py Outdated Show resolved Hide resolved
tools/upload.py Show resolved Hide resolved
tools/upload.py Outdated Show resolved Hide resolved
@earlephilhower
Copy link
Collaborator

A little improvement will be output scrolling while writing.

That, I thought it was only an effect on my setup (linux, xcfe) but it appears to be true even on windows.
I think that is the way the arduino IDE console interprets the \r character

I think it's all on the IDE here (as in we can't do anything on our end), but maybe if we disabled any buffering on stdout/stderr in the Python before calling esptool it might do something?

Adding -u after the python executable to the command line in platform.txt would do that. Whether it would affect esptool or the IDE remains to be seen.

@earlephilhower
Copy link
Collaborator

Just tested it, and -u has no effect on the IDE. So the scrolling issue isn't something we can adjust.

@JAndrassy
Copy link
Contributor

is the upload.py still necessary? can't esptool command line be directly configured in platform.txt?

@earlephilhower
Copy link
Collaborator

Not the way it's being done here. We simulate the erase_region bit by writing 0xffs (i.e. erased) to the block, and need to generate a file to make that happen (hence the Python code above)

@JAndrassy
Copy link
Contributor

Not the way it's being done here. We simulate the erase_region bit by writing 0xffs (i.e. erased) to the block, and need to generate a file to make that happen (hence the Python code above)

Espressif recommends blank.bin

@f5soh
Copy link
Contributor

f5soh commented Nov 13, 2019

I think it's all on the IDE here (as in we can't do anything on our end), but maybe if we disabled any buffering on stdout/stderr in the Python before calling esptool it might do something?

Previous esptool.exe was using \n instead of \r

@d-a-v
Copy link
Collaborator Author

d-a-v commented Nov 13, 2019

@JAndrassy our temp file is filled with 0xff just like espressif's blank.bin is.
Thanks to python we don't need to store it.

@earlephilhower
Copy link
Collaborator

I think it's all on the IDE here (as in we can't do anything on our end), but maybe if we disabled any buffering on stdout/stderr in the Python before calling esptool it might do something?

Previous esptool.exe was using \n instead of \r

Ah, it might make sense to put a PR in esptool.py repo to check if isatty() and if not (the case when run from the IDE) use normal endlines \n...

Copy link
Collaborator

@earlephilhower earlephilhower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @JAndrassy might have a point about just using esptool directly now, but given we need to convert 921k to 460k anyway I don't see how we can avoid having the separate script (which originally was only there because we needed 2 calls to esptool and Arduino doesn't support numbered .upload.2.xxx recipes.)

But looks workable now and avoids multiple resets, so I'm for merging it.

@d-a-v
Copy link
Collaborator Author

d-a-v commented Nov 13, 2019

It will be easier when/if we have another write_flash option like --erase_region along with --erase_flash

@devyte devyte merged commit b52c527 into esp8266:master Nov 14, 2019
earlephilhower added a commit to earlephilhower/esptool that referenced this pull request Nov 14, 2019
When esptool is run from the Arduino IDE, the output window doesn't
scroll because of the \rs used to make the console update status on a
single line.

To avoid this, don't attempt to output everything on a single line when
scripted (i.e. not running on a TTY) by replacing \r with \n.

See esp8266/Arduino#6765 for some discussion.
earlephilhower added a commit to earlephilhower/esptool that referenced this pull request Nov 14, 2019
When esptool is run from the Arduino IDE, the output window doesn't
scroll because of the \rs used to make the console update status on a
single line.

To avoid this, don't attempt to output everything on a single line when
scripted (i.e. not running on a TTY) by replacing \r with \n.

See esp8266/Arduino#6765 for some discussion.
earlephilhower added a commit to earlephilhower/Arduino that referenced this pull request Nov 16, 2019
PR esp8266#6765 introduced an incompatibility with the existing Java uploaders
for SPIFFS and LittleFS,  breaking them because of the upload.py
parameter format change to support single stage erase/upload.

Add in patch to silently eat the single --end and to parse the write
address and filename properly as generated by calls from the plugins,
while retaining compatibility with the current IDE changes.
devyte pushed a commit that referenced this pull request Nov 17, 2019
* Make upload.py compatible with existing FS upload

PR #6765 introduced an incompatibility with the existing Java uploaders
for SPIFFS and LittleFS,  breaking them because of the upload.py
parameter format change to support single stage erase/upload.

Add in patch to silently eat the single --end and to parse the write
address and filename properly as generated by calls from the plugins,
while retaining compatibility with the current IDE changes.

* Clean upload.py, make platform use write_flash

Make upload.py more concise and pythonic.

Use the "write_flash" argument in platform.txt for uploads instead of
assuming anything that is a file is the bin to upload.
@ChazTuna
Copy link

@ChazTuna @janzieg @s0170071 @f5soh
Can you please try this pull request
(I also updated the alpha release channel with it)

New board version 2.6.1 with fixes now works.. Thank you :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flashing fails with Arduino IDE 1.8.9 / Linux Mint
7 participants