RowEditor: Add partialSubmit support

[ssibitz]

I am currently using Primefaces 6.2.2 Elite with Java 8
running on JBoss-EAP 7.1.0 application server.

The view i made has a lot of elements to fill out (our application parameters for a calculation),
so i split it into multiple pages using the Primefaces AccordionPanel.

I set the multiple option to false to show the user only one by one tabsheet.

When i use a tab with controls and a editable datatable in it
primefaces send the complete form on adding or general executing a ajax-request to process
datas on the server.
Because this are too much datas for the complete form i got a server error:
The number of parameters exceeded the maximum of 1000

So why sending the complete form datas via parameters when i only want to
execute the button's action ?
I also tried to use fragments or partial submit,
but there i can't see any differences to the submited datas....

The problem is to give you a sample out of our application i have to rebuild
our application - that's too much for reproducing...

So are there any workaround's for something like this ?
Is it possible to only send the current tabsheet-datas of the accordionpanel
when i set multiple to false instead of the complete form ?

Here are some screenshot's i can give you to show what i am meaning:

1.) The view in general with a lot of tab's inside a accordion panel:
Accordion (over)view

2.) The datas in one current tab:
Opened tab sheet in accordion

3.) The datas sent when i press the add/clear button:
Network-Analyse from posted datas

4.) Currently under 1000 parameters...
Network-Analyse other view with parameter count sent

5.) The source-code for the Add-Entry i got from the primefaces-forum:

    <p:commandLink process="@this"
        styleClass="LinkButtonDetail TextAlignCenter"
	actionListener="#{calcParam.onRowAdd('paramCustCatMarkupsTable')}"
	oncomplete="PF('custcatmarkups').addRow()">
	Add new<i class="fas fa-plus IconLeft"/>
    </p:commandLink>

    public void onRowAdd(String TableName)
    {
        if ("paramCustCatMarkupsTable".equalsIgnoreCase(TableName))
        {
            ParamcustcatmarkupEntity newCustcatmarkupEntityEntry = new ParamcustcatmarkupEntity();
            paramCustCatMarkups.add(newCustcatmarkupEntityEntry);
        }
     }

I write processing only @this, so why the whole form is submitted when i press this button ?
It seems that there is a bug or not ?

6.) When i press now the add-button occurs in a server exception - Here the stacktrace:
java.lang.RuntimeException: io.undertow.util.ParameterLimitException: UT000047: The number of parameters exceeded the maximum of 1000 at io.undertow.server.handlers.form.FormData.add(FormData.java:78) at io.undertow.server.handlers.form.FormData.add(FormData.java:68) at io.undertow.server.handlers.form.FormEncodedDataDefinition$FormEncodedDataParser.doParse(FormEncodedDataDefinition.java:173) at io.undertow.server.handlers.form.FormEncodedDataDefinition$FormEncodedDataParser.parseBlocking(FormEncodedDataDefinition.java:252) at io.undertow.servlet.spec.HttpServletRequestImpl.parseFormData(HttpServletRequestImpl.java:777) at io.undertow.servlet.spec.HttpServletRequestImpl.getParameterNames(HttpServletRequestImpl.java:674) at com.sun.faces.context.RequestParameterMap.get(RequestParameterMap.java:80) at com.sun.faces.context.RequestParameterMap.get(RequestParameterMap.java:57) at java.util.Collections$UnmodifiableMap.get(Collections.java:1454) at org.omnifaces.util.FacesLocal.getRequestParameter(FacesLocal.java:777) at org.omnifaces.cdi.viewscope.ViewScopeManager.isUnloadRequest(ViewScopeManager.java:236) at org.omnifaces.context.OmniExternalContext.getFlash(OmniExternalContext.java:61) at javax.faces.context.ExternalContextWrapper.getFlash(ExternalContextWrapper.java:1034) at com.sun.faces.lifecycle.Phase.handleBeforePhase(Phase.java:213) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:98) at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:133) at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:201) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:670) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.omnifaces.filter.GzipResponseFilter.doFilter(GzipResponseFilter.java:181) at org.omnifaces.filter.HttpFilter.doFilter(HttpFilter.java:108) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at com.skf.filters.DoCacheFilter.doFilter(DoCacheFilter.java:98) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at com.skf.filters.AuthorizationFilter.FilterRequest(AuthorizationFilter.java:194) at com.skf.filters.AuthorizationFilter.doFilter(AuthorizationFilter.java:221) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: io.undertow.util.ParameterLimitException: UT000047: The number of parameters exceeded the maximum of 1000 ... 72 more

Please do not write that i have to increase the maximum on the parameter.
This does not solve the issue but it only comes later on a view with dynamic datas
and does not solve the problem...

[cnsgithub]

I don't know what the reason for your original problem is, however if you want to avoid a full submit you might try one of the following:

1. Use immediate = true on the commandlink
2. Use a p:remotecommand outside of the datatable with immediate true and an action of your bean and call it via JS

( js call: remoteCommandFunctionName({name1:'value1', name2:'value2'});)
( get param in java: FacesContext.getCurrentInstance().getExternalContext().getRequestParameterMap().get("name1") )

[ssibitz]

Is there a ways to "exclude" in general sending/posting all datas from a Datatable ?

This would solve the issue i think because i have only this options for a datatable:

1.) Datatable is showing the datas readonly :
No sending of datatable-datas in form-datas required !

2.) Datatable is enabled for Row-Editing:
Changed datas are send by ajax-request to server.
No sending of datatable-datas in form-datas required !

3.) Datatable is enabled for Cell-Editing:
Changed datas are send by ajax-request to server.
No sending of datatable-datas in form-datas required !

So is it possible to never send datatable-datas
because i don't find a need for doing this...

[tandraschko]

Your "issues" are mostly like forum questions ;)
You can use partialSubmit, but really, thats nothing thats need to be discussed in the issue tracker.

[ssibitz]

The problem also occurs when i set to partialSubmit because in a simple Datatable with edit is set to on
and want to be able to dynamically add new datas the limit of 1000 Parameters is reached very quick when i have about 15 Columns in Edit Mode there are 30 Fields/Row (input+output).

So after 33 Row's i got a server exception because of 34 Rows x 15 Columns x 2 Fields/Column (Input+Output) = 1020 Parameters.

But in size we are talking about 600Bytes to transfer which makes a problem to send ? Really ?
So where is the bug here when i can upload megabytes of datas by using a file-transfer via AJAX ?

So why sending the whole datatable as form-parameter when i changed the datas via AJAX
and processed it directly ?

[tandraschko]

Now we have a real issue ;)

[tandraschko]

If you set the partialSubmit on your add-commandLink, that should work fine. Just the edit itself could still use partialSubmit=false of course.

[ssibitz]

It also crashes when i click on the add-commandlink
also when i set partialSubmit to true
and process only to @this :

<p:commandLink 	  
    process="@this"
    update="@none"
    partialSubmit="true"
    styleClass="LinkButtonDetail TextAlignCenter"
    actionListener="#{calcParam.onRowAdd('paramCustCatMarkupsTable')}"
    oncomplete="PF('custcatmarkups').addRow()">
    Add new<i class="fas fa-plus IconLeft"/>
</p:commandLink>

The whole datatable is then sended to the server,
and crashed after adding (Clicking the Add-Button) the 34th row :

The view with the datas before clicking the add-button

The parameter count sended to the server

All datatable parameters are sent to the server

Occurs in this error:
io.undertow.util.ParameterLimitException: UT000047: The number of parameters exceeded the maximum of 1000

[ssibitz]

I changed the current title of this issue because it currently occurs in CellEdit and RowEdit mode ...

[ssibitz]

I have one question:

Is there a needed from the primefaces side that all
datatable datas (coded as form data)
are send because in edit-mode (row or cell)
always AJAX is used for transfering the changed datas ?

[tandraschko]

About the add-commandLink, please provide a example based on the showcase....

There is no need, we just send the whole form per default. Therefore there is a "partialSubmit" mode.

[ssibitz]

And why sending the whole data-table datas per default ?
Is there any need for sending the data-table datas ?
In Edit-Mode always only the changed datas are send,
and in Readonly-Mode the datatable-datas not relevant for sending - or not ?

[ssibitz]

I will try to fork the showcase and modify a datatable-example in which the error can be reproduced...

[tandraschko]

The problem is actually this:
the DataTable must be proceed to correctly trigger the events on the serverside. So the roweditor must proceed the DataTable. PartialSubmit means that everything that is "process"ed, will be posted. In this case the whole DataTable.

And exactly this new functionally, to only send the current row for the rowEditor needs to be implemented.

[ssibitz]

So also when i edit only 1 cell the complete data-table will be transfered ?

[tandraschko]

I don't know, i always talked about the rowEditor. Just check it by yourself.

[ssibitz]

I will fork the primefaces-showcase and make a separate entry in the datatable-section
to show how to add a new row and to reproduce the issue.

Then i will come back to this ticket and report/send you the link...

[ssibitz]

I now got the exception also on the primefaces-showcase
in the "Datatable -> Edit" view.
(https://www.primefaces.org/showcase/ui/data/datatable/edit.xhtml)

To reproduce you have to click the "Add a new row" button 3 times.
The first and second click will produce the new generated entry,
but the 3rd click will produce the exception because then more than
1000 parameters are send to the server...

A unexpected error occured! Message: Form too many keys

So please try this example - I changed the
edit.xhtml
and
EditView.java
to use the Add-Button and add 123 rows to the datatable to reproduce the "overflow":

Here is the fork from the current Primefaces-Showcase i am using:
https://github.com/ssibitz/showcase.git

[tandraschko]

changed the title again. If CellEditor does also not work, we should add a new ticket.

[tandraschko]

For your other issue, i created a new ticket: #3564

The button itself does not post the datatable but the addRow method does.

[tandraschko]

It would be great if you could sponsor it or provide a PR. I don't have time to do this in my freetime.

[ssibitz]

PR ? What's this ? How can i help to solve this bug ?

[tandraschko]

https://help.github.com/articles/about-pull-requests/

[ssibitz]

OK, i now created a pull-request - can you check it ?

[ssibitz]

I think that the general problem is that
as default datas are send as "application/x-www-form-urlencoded" in jquery
and so also in the primefaces-send function in ajax.

Sending the form-datas to urlencoded will always limit
you in sending to a maximum of 1000 form-elements/params/datas

• even if the size of the POST is only less than 1KB.

So in general sending the datas via JSON, Text or a real XML will
solve the problem as i find out.
Then you can send datas as much as you like -
limited only by the upload-size which is in general
much greater than 1KB :-)

The bug is so located in the core.js which is sending
all datas as url-encoded form parameters.

I also found out on the jQuery's site the following
which sends the data as plain text without using the url-encoding:

The part's of the core.js are:

            send: function(e) {
/* ........*/
				// Send request as:
				// plain text
				// do not process datas (Not converting to application/x-www-form-urlencoded
                var p = {
                    url: x,
                    data: c,
                    type: "POST",
                    dataType: "xml",
					contentType: "text/plain; charset=UTF-8",
					processData: false,
                    cache: false,
                    portletForms: w,
                    source: e.source,
                    global: false,
                    beforeSend: function(z, i) {
                        z.setRequestHeader("Faces-Request", "partial/ajax");
                        z.pfSettings = i;
                        z.pfArgs = {};
                        PrimeFaces.nonAjaxPosted = false;
                        if (m) {
                            $(document).trigger("pfAjaxSend", [z, this])
                        }
                    }
                };

The main think is to use another type as
application/x-www-form-urlencoded
because this is limited to 1000 parameters as default on ALL servers !!

So this could be the right way:
contentType: "text/plain; charset=UTF-8",
processData: false,

I don't know what format is needed on the primefaces server side
, but in general you should switch from url-encoding to JSON/XML/Text-Encoded sending...

[tandraschko]

IMO thats not a generell problem, only a generell workaround ;)
partialSubmit should work there and used if possible, to reduce the data which is required nowadays for mobile devices.

[ssibitz]

1.) I think sending the datas via JSON/XML/Text-Encoded is an improvement
to really send big datas and be future-ready :-)

2.) Paritial handling of the datatable:
Sending the whole datatable in general always occurs in problems when
handling with big datas and here i start "changing" this.
The first think i want to try is to handle the partial sending of a datatable
so the whole datatable should never(!) be sended.

Why ?
I have only 5 use cases for the datatable and never a need to send the whole datatable back:

• 2.1) Table is readonly -> The datas will never send to the server because this also could be
  a security issue.
• 2.2) A general POST with all form datas is raised -> The same as if the Table is readonly because it has nothing to do with the datatable
• 2.3) Table is editable in Row-Edit-Mode -> Only the current row-datas should be sended
• 2.4) Table is editable in Cell-Edit-Mode -> Only the current cell-datas should be sended
• 2.5) Table is editable and AddRow was called -> Only get the new row from a server function

[ssibitz]

This issue has nothing to do with all components or the JSF-Core.
It only has to do with the datatable and sending unneeded datas.

A server side solution is not possible because it crashes before you got the datas on a filter function...

The exception occurs also in the Primefaces-Showcase
when you add about 125 rows to the 2 editable tables and add try to add a new one ...

[tandraschko]

for 1) and 2) partialSubmit was implemented ;)

3,4,5 is really component specific.

[ssibitz]

I currently found a simple, but effective workoround to
fix the mass-sending of datas for the primefaces-data table:

When the send function is called i filter out all fields with the parent class is from type
ui-cell-editor-input if the new config-param cfg.postDatatableEntries is not explizit set to true :

Here is the currently changed part in the core.ajax.js include a debugging:

// Remove all datatable entries if skipDatatableEntries set to true
var postDatatableEntries = cfg.postDatatableEntries === true;
if (postDatatableEntries == true) {
	PrimeFaces.debug('Do not skipp unwanted datatable entries.');
} else {
	PrimeFaces.debug('Skipping unwanted datatable entries.');
	// loop post params and remove it, if it's some kind of a datatable entry:
	postParams = $.grep(postParams, function(postParam, postParamIndex) {
		PrimeFaces.debug('Checking parameter: '+postParam.name+', with index:'+postParamIndex);
		var elements = document.getElementsByName(postParam.name);
		if (elements) {
			if (elements.length == 1) {
				var parentsClassName = elements[0].parentElement.className;
				PrimeFaces.debug('   Parameters parent is from type "'+parentsClassName+'"');
				if (parentsClassName.includes("ui-cell-editor-input")) {
					PrimeFaces.debug('   Ignoring parameter because its a datatable param !');
					return false;
				}
			}
		}
		return true;
	});
}

//serialize
var postData = $.param(postParams);

This currently fixes the point's 2.1), 2.2) and 2.5) -
so no sending the complete datatable datas any more in general
and calling the dynamic AddRow-Function now results
in sending about 10 parameters in Primefaces-Showcase
instead of 1000.
And also the new row is added sucessfully !

So only the points 2.3) and 2.4) are open
and i am checking it how to solve ...

[ssibitz]

I currently think for the point 2.3) and 2.4) only sending the
datas of the current row and filtering all other datas from the table should
also work similar to the cfg.postDatatableEntries parameter ...

[tandraschko]

Commited a prototype for it, you can activate it via:
<p:ajax event="rowEdit" .... partialSubmit="true" />

[ssibitz]

What's about the point's 2.1), 2.2) and 2.5) ?

Can you also filter the datatable's ui-cell-editor-input's by default
to skip unwanted "datatable-send-datas" ?
So that no datas of the datatable is included when
no editing is currently active ...

[tandraschko]

just don't render inputs in 2.1 and 2.2.... (or don't use a p:rowEditor.... )
2.5 is another issue and there is already a ticket for this one.

[ssibitz]

How can i don't render the input's when its inside
the datatable ?
I don't know when it should be rendered because
that's a "feature" of the datatable and should be handled here !

The whole datatable's input's are always send to the server
and that's the current problem - also when try to add a new row.

So : Please do not send all table datas by default on form-submit
because this is not a useable practice:
Only send changed datas via AJAX and not more as needed !

[ssibitz]

I see that your current fix only fixes the point 2.3) Row-Editing
but what's with the 2.4) Cell-Editing ?

Why you are closing this ticket even if the bug's are already exists ?

[tandraschko]

Dude, i'm not a premium support here to anwer all your questions :D

Did you ever checked the cell-editing if the problem happens there?
Did you ever checked if disabled inputs are posted to the server?

[tandraschko]

As i already said, this ticket is about the RowEditor. And this problem is fixed with the commit.

[ssibitz]

OK.

I will create new ticket's for the other point's,
but they are all the same base reason:
Sending to much datas instead of filtering them out before
sending them via AJAX-POST...

[tandraschko]

close in favor of: #3571

[kukel]

2.1: Real readonly inputs are NOT submitted by JSF, so 2.1 is a non-issue. 2.2 is or should like Thomas stated be done via the existing partialSubmit options. The core ajax should not have knowledge of the dataTable internals. 2.5 is already 'fixed', see #3564. It might be an option to in the #3564 fix use the skipDataTableChildren parameter/attribute