-
Notifications
You must be signed in to change notification settings - Fork 688
Add a simple Job queue in default port. #1685
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
jerry-core/jerry-port.h
Outdated
| double jerry_port_get_current_time (void); | ||
|
|
||
| #ifndef CONFIG_DISABLE_ES2015_PROMISE_BUILTIN | ||
| #include "ecma-jobqueue.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think including this is not a good idea. IMHO we shouldn't make internal parts public. It would be great to separate the private and public parts more clearly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, the external can expose interface to internal, the reverse can't.
I find the definition of jerry_property_descriptor_t in jerryscript.h is same as ecma_property_descriptor_t in ecma-globals.h. So I think I can use the similar method in jerry-port.h: define the same struct with different name.
|
I wonder if it makes sense to make this more generic (and extensible w/o having to change ecma-jobqueue.h to add new types). I suspect that each type will have its own callback/handler, so we could remove the explicit type field and use a function pointer instead: Furthermore, I'd probably try to avoid exposing "internals" (i.e. Note I added The downside is you probably have to do 2 allocations (1 for the |
|
I like the way how you trying to do this. Jobs must depend on the actual port. Nice work. |
| return; | ||
| } | ||
|
|
||
| ecma_job_process (job_p); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ecma_job_process should probably be exposed in jerry-port.h somehow, to avoid reaching into internals.
Hmm, this is the first time the port needs to call back into JrS, I think?
Is there a convention to indicate in a port header that a function declaration is there to indicate the port can/needs to call it? (I'm used to adding extern as a hint, but don't know if that's a widespread convention or not.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems passing the function pointer can solve this problem.
If the job_p is defined like you suggest
typedef struct
{
ecma_job_handler_t handler; /** < callback that will execute the job */
void *user_data_p;
} ecma_job_t;
When get the job_p from the queue, directly call job_p->handler(job_p)
So there is no ecma_job_process in internal, and we need to implement all process-functions for different types of jobs in ecma-jobqueue.c.
| } | ||
|
|
||
| ecma_job_process (job_p); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Who/what is responsible for freeing the job_p? (Right now it's being leaked, I think.)
Related; I guess if there's a "job allocator" API (like I described vaguely in the other comment) and all jobs are allocated by JrS, it can also take responsibility for freeing them after executing the callback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my designs, they should be freed in the end of ecma_job_process, in ecma_job_process_promise_reaction_job or ecma_job_process_promise_thenable_job.
Because in the PR, no one allocate job yet, so i didn't write the "release" code
| */ | ||
| typedef struct | ||
| { | ||
| ecma_job_type_t type; /**< specific type of the job */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
enums are compiler specific, sometimes int, sometimes unsigned char. I would prefer to use an uint8_t here.
|
Are jobs are always resolved in-order? |
| jerry_port_queueitem_t *tail_p; /**< point to the tail item of the queue*/ | ||
| } jerry_port_jobqueue_t; | ||
|
|
||
| jerry_port_jobqueue_t queue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Static writable declarations are not allowed in JerryScript, you need to move this into jcontext.h.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can I include jcontext.h in port?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, because the queue's implementation heavily depends on the actual port. If the OS already have the event queue, you can directly use that. Here I just provide the simplest one in default port.
So I don't think it is suitable to be in jcontext.h. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't noticed that this is port specific. Leave it then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably add static here in any case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please add a static at least.
I think the "resolve" here means "process job", then answer is YES. |
|
Another commit submitted.
|
|
|
||
|
|
||
|
|
||
| #ifndef CONFIG_DISABLE_ES2015_PROMISE_BUILTIN |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra newlines.
jerry-core/jerry-port.h
Outdated
| #ifndef CONFIG_DISABLE_ES2015_PROMISE_BUILTIN | ||
|
|
||
|
|
||
| typedef struct jerry_job_t jerry_job_t; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra newlines.
| jerry_port_queueitem_t *tail_p; /**< point to the tail item of the queue*/ | ||
| } jerry_port_jobqueue_t; | ||
|
|
||
| jerry_port_jobqueue_t queue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't noticed that this is port specific. Leave it then.
jerry-core/jerry-port.h
Outdated
|
|
||
| void jerry_port_jobqueue_init (void); | ||
| void jerry_port_jobqueue_enqueue (jerry_job_t *job_p); | ||
| void jerry_port_jobqueue_run (void); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I see there is no need for exporting jerry_port_jobqueue_init and jerry_port_jobqueue_run functions. I confused later because I expected that ports must share them. The only function needed is jerry_port_jobqueue_enqueue and we expect that the callback is eventually called by the port. The how is the own business of the port.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternative idea:
Make jerry_port_jobqueue_enqueue expect two arguments: a callback and a job pointer. The port needs a queue item represenation anyway and adding two values shouldn't be a bigger issue than adding one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
about the Alternative idea:
The callback is already in the jerry_job_t, what is the benefit that we separate them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
about jerry_port_jobqueue_init and jerry_port_jobqueue_run:
You are right! Those two functions are called by the port, and jerry-core will never call them directly. My mistake
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benefit: you can drop the jerry_job_t type. This would simplify a lot of definitions later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer to keep the handler in the jerry_job_t, because it establish a connection between the callback and job (job actually is callback's arguments) in the jerry internal. Otherwise, the connection only exists in the port's queue item.
jerry-core/jerry-port.h
Outdated
|
|
||
| struct jerry_job_t | ||
| { | ||
| jerry_job_handler_t handler; /**< the handler funtion for the job */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a jerry_job_handler_t type? We could simply write it here:
void (*handler) (struct jerry_job_t *);
I don't really like the typedef above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, typedef is cleaner, and used in many place like jerryscript.h and jmem.h
f661285 to
5fbd362
Compare
| struct ecma_job_t | ||
| { | ||
| ecma_job_handler_t handler; /**< the handler funtion for the job */ | ||
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually these are my issues with the current approach. An ecma_job_t is defined here, which is basically a jerry_job_t. Furthermore both definitions require a circular definitions, and that is never a good practice. The structures below also follows some class in C programming model.
My suggestion is: keep the callback at jerry_port.h/ The enqueue get the callback and a user pointer. Since the callback is not stored in the job, ecma_job_promise_reaction_job_t and ecma_job_promise_thenable_job_t do not need to depend on anything. We will use different callbacks for reaction and thenable (what a strange word) so there will be no confusion. I think we need less type casts overall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then the callback in jerryport.h should be
typedef void (*jerry_job_handler_t) (uintptr_t);
void jerry_port_jobqueue_enqueue (jerry_job_handler_t, uintptr_t);
and in every callback body, we need to cast the uintptr_t to some specific job_t (e.g. ecma_job_promise_reaction_job_t)
Am I right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, except I would use void* instead of uintptr_t. Casting would be needed anyway since we have multiple job types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But at least we don't need casting a struct* to void*.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a question: a job does not have a return value? What happens with exceptions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You remind me that the job_handler should have return value, and if exception happens, perform implementation defined unhandled exception processing.. In the simple queue, I will stop the queue running. I will update the patch soon
|
What is your plan for engine shut-down? I suspect we should empty the queue and not allow inserting more items. Or what should happen? The multiple queues part is also unclear for me. Do we need multiple queues for a single JerryScript instance? |
|
I think one typical flow after introducing jobqueue/promise will be For now, only one queue (PromiseQueue) in Jerry. If we plan to support |
|
updated the patch, the job handler should return the exception if it happens. |
| { | ||
| jerry_value_t ret; | ||
|
|
||
| while (1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if it makes sense to put this loop into an ecma_... file and let it call out to the port for:
- Getting the next item (you already have this:
jerry_port_jobqueue_dequeue()) - Handling the case when there are no items in the queue ("If all Job Queues are empty, the result is implementation defined.") => maybe add
jerry_port_jobqueue_handle_queue_empty()? - Handling an error result ("If result is an abrupt completion, perform implementation defined unhandled exception processing.") => maybe add
jerry_port_jobqueue_handle_error()?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the loop should be in port file.
When people port the jerry-engine into a embedded project, the project/OS may already have its own queue mechanism/event loop, for example, the libtuv for iotjs. Then the loop in jerry might conflict the loop in the outer project/OS
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
while (true)
|
|
||
| typedef uint32_t (*jerry_job_handler_t) (void *); | ||
|
|
||
| void jerry_port_jobqueue_enqueue (jerry_job_handler_t handler, void *job_p); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Nice, this way any binding that needs to execute something asynchronously (for example setTimeout()) can use this to schedule the callback. In other words, the JobQueue feature is also useful for other (non-ES2015 Promise dependent) use cases.
Because of this, I think it makes sense to make a separate CONFIG_DISABLE_ES2015_JOBQUEUE feature flag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is too early to set the seperate CONFIG_DISABLE_ES2015_JOBQUEUE, the reasons are as follows:
- for now, only promise will use jobqueue. Enabling only one of "JOBQUEUE" and "PROMISE" does not make much sense.
- in the future, when we plan to support
module(which also use jobqueue), we will enable jobqueue by#if !defined(CONFIG_DISABLE_PROMISE) || !defined(CONFIG_DISABLE_MODULE) - will
setTimeoutuse the jobqueue? it is out of jerry-core's scope, because setTimeout is a external api. The binding code can directly use the jobqueue in the port no matter whether job queue is enabled in jerry-core. (That is another reason why I think the loop should not be in jerry-core)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, you're right. I've thought about this a bit more and I'm starting to doubt whether my comment here makes sense. The source of the ES6 job queue events/jobs is from things internal to JerryScript (for now only Promise), while for something like setTimeout(), the event source is the native/host system, which would put an event into the (existing) native event loop.
Yeah, that is also what I understand from the spec. I think because uses of JobQueues by the ES6 spec itself don't have any prioritization requirements, I think the current |
@zherczeg Hmm.. Did you see that ES6 introduced a section "ECMAScript Initialization()" which talks about the first time "NextJob" is called? http://www.ecma-international.org/ecma-262/6.0/#sec-ecmascript-initialization I wonder if it makes sense to add a public helper to |
|
About the ECMAScript Initialization, we dont have to implement it with jobqueue and enqueue the ScriptEvaluationJob in the first place. Instead, we can directly call the jerry_parse/jerry_run first, then enter the loop to process the jobs/events (both iotjs and nodejs are using the above mechanism). I agree with @zherczeg that only At least, I suggest that we only export |
Ah, you're right. Putting the loop into JrS would force everyone to change any existing event loop, which is not nice.
SGTM. |
zherczeg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The core is pretty good now. Several style fixes needed though.
| struct jerry_port_queueitem_t | ||
| { | ||
| jerry_port_queueitem_t *next_p; /**< point to next item */ | ||
| jerry_job_handler_t handler; /**< the hanlder for the job*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/**< the handler for the job */
| typedef struct | ||
| { | ||
| jerry_port_queueitem_t *head_p; /**< point to the head item of the queue */ | ||
| jerry_port_queueitem_t *tail_p; /**< point to the tail item of the queue*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/**< points to the head item of the queue */
/**< points to the tail item of the queue */
| jerry_port_queueitem_t *tail_p; /**< point to the tail item of the queue*/ | ||
| } jerry_port_jobqueue_t; | ||
|
|
||
| jerry_port_jobqueue_t queue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please add a static at least.
|
|
||
| queue.head_p = item_p; | ||
| item_p->next_p = NULL; | ||
| queue.tail_p = queue.head_p; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
queue.tail_p = item_p;
| { | ||
| jerry_value_t ret; | ||
|
|
||
| while (1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
while (true)
| #include "jerry-port-default.h" | ||
| #include "jrt.h" | ||
| #include "jmem.h" | ||
| #include "jerryscript.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be alphabetic order.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for reminding me. I just knew the rule.
| #include "ecma-globals.h" | ||
| #include "jerry-port.h" | ||
| #include "ecma-jobqueue.h" | ||
| #include "ecma-helpers.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be in alphabetic order.
| if (jerry_value_has_error_flag (ret)) | ||
| { | ||
| return ret; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if no error flag, we also need to release the ret value. Will add in the updated patch soon
|
@zherczeg updated. |
Job queue (event loop) is the basis of Promise. Each port should implement their own job queue. JerryScript-DCO-1.0-Signed-off-by: Zidong Jiang zidong.jiang@intel.com
|
|
||
| if (jerry_value_has_error_flag (ret)) | ||
| { | ||
| return ret; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading the NextJob section again (http://www.ecma-international.org/ecma-262/6.0/#sec-nextjob-result) I'm not sure whether the loop (recursion in the spec) should be stopped. It doesn't say that it returns after an error result. I guess you could argue that whether or not to return could fall under the implementation specific part.
That said, I think it's more useful to not return (keep looping) and instead call out to another port function to handle the error result, for example by using jerry_port_console() to print out the error message. I think something like that is what most people will do in actual projects.
I think it makes sense to expose a void jerry_port_handle_uncaught_error(jerry_value_t error) into jerry-port.h. Any user binding code that uses async callbacks can then also use the same interface to deal with uncaught errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's more useful to not return (keep looping) and instead call out to another port function to handle the error result
In Node.js, the uncaught error will stop the loop
setTimeout(function(){console.log(x)}, 100); //error: no reference
setTimeout(function(){console.log("will not show this")}, 200);
I know that the following code will run correctly
Promise.resolve(1).then(function(){console.log(x});
setTimeout(function(){console.log("WILL run here"), 100};
Because although function(){console.log(x} will throw error, it is not the result of NextJob, it is the step 7 of http://www.ecma-international.org/ecma-262/6.0/#sec-promisereactionjob
expose a handler into
jerry-port.h
I doubt it. The caller of the handler is only the port-specific queue_run function, not the jerry-core.
The default port's queue is designed for main-unix.c as a simple demo, and it is only a simplest queue which cannot even handle real-async event. So in my design, the uncaught error will stop the queue, then developers can write their own error-process function in main-unix.c (usually print the error message). I wonder if it is necessary to increase the complexity of the default queue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In Node.js, the uncaught error will stop the loop
Ah, interesting. Chrome's behavior is different, it does actually print the "will not show this" log.
OK, let's just leave it as-is.
zherczeg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Great patch!
LaszloLango
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Job queue is the basis of Promise.
as the spec says
Each port should implement their own job queue.
Job queue is very similar to the
event loopof node.js and iot.js.After introducing the job queue, when the sync part of the js code are executed by vm, it should check the queue, and do the pending jobs in FIFO order.
This patch only implemented the simplest queue, which actually cannot handle the true async callback.
e.g.