add size limited pymaps and pylists #530
Conversation
boulter
requested review from
jasmith-hs,
mattcoley,
hs-lsong and
kacperadach
There was a problem hiding this comment.
Throwing the OutputTooBigException in the LengthLimitingStringBuilder has been a bit of a mess trying to handle that runtime exception in multiple places. Would it make better sense to truncate the collection to keep it under the limit and add a warning? This would protect against the situation where data being pulled in gets continually larger until it trips the limit which could break templates rather than just truncate them.
|
This throws an |
There was a problem hiding this comment.
I like this idea. If it's wrapped in an ELException, it would be handled here by adding a warning-level template error
|
|
||
| @Override | ||
| public boolean addAll(int index, Collection<?> elements) { | ||
| if (size() + elements.size() >= maxSize) { |
There was a problem hiding this comment.
Integer overflow might be a concern. It should be safer to do size() >= maxSize - elements.size()
There was a problem hiding this comment.
I think we will run out of memory long before this if a collection has 2B items in it!
src/main/java/com/hubspot/jinjava/objects/collections/SizeLimitingPyList.java
Outdated
Show resolved
Hide resolved
|
@mattcoley I decided to create a new |
|
What if it's a warning when it's at like 90% capacity to help prevent the situation that Matt described |
Done. Check it out. |
| import com.hubspot.jinjava.objects.PyWrapper; | ||
| import java.util.ArrayList; | ||
| import java.util.Collection; | ||
| import java.util.List; | ||
|
|
||
| public class SizeLimitingPyList extends PyList implements PyWrapper { | ||
| private int maxSize; | ||
| private boolean hasWarned; |
User-provided templates should not be allowed to use unlimited resources. This is a failsafe that adds a limit the maximum size of lists and maps. The default is no (practical) limit.
This is similar to #137