What sort of http file throughput speed should I expect for an esp8266? #209
Comments
|
What size 'chunks' are you sending? I have found 512 btyes seem to work - any more and it crashes. A 1/2 Mb jpg file is actually quite large - are you sure that it can't be optismised smaller? |
|
Just what the server asks for! From the docs (and as illustated by the bitmap server example) 9 | Get response fragment. The server is ready to transmit another fragment of the response. The val1argument contains the number of bytes that may be transmitted. The callback returns either a String or ArrayBuffer. When all data of the request has been returned, the callback returns undefined. on esp8266, from (my) memory, it's in the region of 2000 bytes or so. A 1/2 M jpg file was just what I had to hand - it's just a test file, not what I'd use in the real world. It was meant to be about that size just to give me some idea of the speed I'd get. |
|
512 bytes made no difference I'm afraid. Anyway, I've just noticed I'm not sending a content-length header, so I'll just drop that in and see if it makes any difference... |
|
OK, so adding the content-length header has resulted in a speed up in the region of 4x, so that file loads in the browser in roughly 3 seconds. That is about what I'd expected/hoped for from an esp8266. I'm still not quite sure about how many bytes i can send at once without crashing http or using chunked encoding, but I'll have a play and a further read about it before posting another issue, if I need to. |
This how I found the 512 limit by experimenting. If the content is less than 512 - I just serve the string, if longer then chunk it. I think we are on the same path here... I'm just a bit further along! I've stopped using the spiffs file system ( I had set up so I could do a http put to update the content) . Now I serving the content from the |
|
The arduino asyncwebserver does 7Mbps so we know that the hardware is capable. The regular webserver is slower and speed compatible with moddable. Unfortunately the asyncweb is not stable. |
How was this measured? |
|
Connected my desktop computer to the esp32 wifi and streamed data. It is very fast. |
|
[EDITED add esp32] The gz file is 89kb. I found this curl script to measure: This is from windows using the simulator: and this is on the esp8266: esp32: or: esp 8266: esp32: |
|
The thread is for the 8266, but here is a comparision. This is the asyncwebserver streaming from ram with esp32. The server callback gives a buffer and its size for you to fill with data and wait for the next call. These are the buffer sizes on the first calls: |
|
This is the page load of my app: win sim: esp8266: esp32: |
|
Ah, the search for an easy to use performant espXX server! I've not used the arduino async one for a couple of years, as I remember it was good, but prone to crash. I checked a few months ago and development looked dead, sadly. I was surprised to not find it forked and further developed, maybe I didn't look hard enough. |
|
@wilberforce : "I think we are on the same path here... I'm just a bit further along! " .... |
|
A few notes:
I'm closing this issue as its (interesting) discussion seems to have run its course. If there are new findings, please open a new issue focused on that. Thank you. |
|
One more note... Nagle's algorithm may be relevant here. It is enabled by default in lwip. As an experiment, you can easily disable it by adding the following line to the top of |
|
A little more... The next push (later today) of the Moddable SDK includes support to disable Nagle's algorithm from script. It is disabled by default now in the HTTP server. I adapted the import {Server} from "http"
let server = new Server({});
server.callback = function(message, value) {
if (Server.status === message) {
this.path = value;
this.remaining = 10 * 1024 * 1024;
this.buffer = new ArrayBuffer(1);
}
if (Server.prepareResponse === message)
return {headers: ["Content-type", "application/octet-stream"], body: true};
if (Server.responseFragment === message) {
if (this.remaining <= 0)
return;
this.remaining -= value;
let buffer = this.buffer;
if (buffer.byteLength !== value) {
delete this.buffer;
buffer = this.buffer = new ArrayBuffer(value);
}
return buffer;
}
}N.B. The reuse of the N.B. Disabling the Nagle algorithm doesn't make a huge difference, but it seems like the right thing to do in the server given the way modSocket implements writes. |
|
Thanks Peter, much appreciated.
…On Thu, Jul 11, 2019 at 11:53 PM Peter Hoddie ***@***.***> wrote:
A little more...
The next push (later today) of the Moddable SDK includes support to
disable Nagle's algorithm from script. It is disabled by default now in the
HTTP server.
I adapted the httpserverchunked example
<https://github.com/Moddable-OpenSource/moddable/blob/public/examples/network/http/httpserverchunked/main.js>
for benchmarking. That is below. It generates an 10 MB stream of zero data.
On an ESP8266 it runs at about 330 KB/sec on our office network downloading
to browsers on macOS. On an ESP32, it runs at about 520 KB/sec. On both
devices, there is some variation (both faster and slower). There wasn't any
special effort to optimize the network environment, Wi-Fi antenna
placement, etc. The benchmarks were run on a release build.
import {Server} from "http"
let server = new Server({});server.callback = function(message, value) {
if (Server.status === message) {
this.path = value;
this.remaining = 10 * 1024 * 1024;
this.buffer = new ArrayBuffer(1);
}
if (Server.prepareResponse === message)
return {headers: ["Content-type", "application/octet-stream"], body: true};
if (Server.responseFragment === message) {
if (this.remaining <= 0)
return;
this.remaining -= value;
let buffer = this.buffer;
if (buffer.byteLength !== value) {
delete this.buffer;
buffer = this.buffer = new ArrayBuffer(value);
}
return buffer;
}
}
N.B. The reuse of the ArrayBuffer is to minimize the load on the garbage
collector. It makes a measurable difference when there are hundreds of
relatively large blocks being allocated per second.
N.B. Disabling the Nagle algorithm doesn't make a huge difference, but it
seems like the right thing to do in the server given the way modSocket
implements writes.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#209?email_source=notifications&email_token=AD7WV4XSRZUOHWKE5IBAYR3P662VJA5CNFSM4HXTZO2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYGLEQ#issuecomment-510682514>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AD7WV4XBWFTXF6FH3ABZFI3P662VJANCNFSM4HXTZO2A>
.
|
So, I basically modified the bitmapspooler example to "spool" a file and serve it over http chunked, which works. Testing it on a roughly 1/2 Meg jpg file it seems pretty slow - it loads in the browser almost a line at a time, taking in the region of 15 seconds in total.
I do appreciate that this is very limited hardware so I don't expect miracles from it, but I was hoping for something a bit faster!
I don't actually hope to serve largish bitmaps from it, but I do hope to serve a relatively meaty javascript bundle (react and a heap of css etc) gZipped.
Does that seem in the right region to you?
If so, can it be made faster?
If not, and it should be faster than that, I'm wondering what I might be doing wrong?
I'd test on my esp32 to see how that goes there (faster, I'm sure), but I'm having some other issues with that:( !
The text was updated successfully, but these errors were encountered: