-
Notifications
You must be signed in to change notification settings - Fork 1.3k
[core] Implement data-driven styling for {text,icon}-{color,opacity,halo-color,halo-blur,halo-width} #7939
Conversation
@anandthakker, thanks for your PR! By analyzing this pull request, we identified @jfirebaugh and @ansis to be potential reviewers. |
5a6df6d
to
9a2d6e0
Compare
Currently running into the following problem:
I think this is the reason that I'm getting a update: I believe the reason this isn't blowing up in GL JS is that, there, we:
|
I don't think that's the reason -- that pattern occurs elsewhere, like with |
Yeah - so it looks like we're ending up with attribute ids >= 16 because each data-driven mbgl::gl::Attributes<
mbgl::attributes::a_pos,
mbgl::attributes::a_offset<2>,
mbgl::attributes::a_texture_pos,
mbgl::attributes::a_data<4>,
mbgl::attributes::Min<mbgl::attributes::a_opacity>,
mbgl::attributes::Min<mbgl::attributes::a_fill_color>,
mbgl::attributes::Min<mbgl::attributes::a_halo_color>,
mbgl::attributes::Min<mbgl::attributes::a_halo_width>,
mbgl::attributes::Min<mbgl::attributes::a_halo_blur>,
mbgl::attributes::Min<mbgl::attributes::a_opacity>,
mbgl::attributes::Min<mbgl::attributes::a_fill_color>,
mbgl::attributes::Min<mbgl::attributes::a_halo_color>,
mbgl::attributes::Min<mbgl::attributes::a_halo_width>,
mbgl::attributes::Min<mbgl::attributes::a_halo_blur>,
mbgl::attributes::Max<mbgl::attributes::a_opacity>,
mbgl::attributes::Max<mbgl::attributes::a_fill_color>,
mbgl::attributes::Max<mbgl::attributes::a_halo_color>,
mbgl::attributes::Max<mbgl::attributes::a_halo_width>,
mbgl::attributes::Max<mbgl::attributes::a_halo_blur>,
mbgl::attributes::Max<mbgl::attributes::a_opacity>,
mbgl::attributes::Max<mbgl::attributes::a_fill_color>,
mbgl::attributes::Max<mbgl::attributes::a_halo_color>,
mbgl::attributes::Max<mbgl::attributes::a_halo_width>,
mbgl::attributes::Max<mbgl::attributes::a_halo_blur> > |
src/mbgl/programs/symbol_program.hpp
Outdated
@@ -94,7 +94,7 @@ class SymbolIconProgram : public Program< | |||
const TransformState&); | |||
}; | |||
|
|||
class SymbolSDFProgram : public Program< | |||
class SymbolSDFIconProgram : public Program< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I initially tried to just parameterize SymbolSDFProgram
on the paint property types, but it was proving sort of unweildy. Opted to duplicate the declarations and minimize code duplication in implementation via the shared makeSDFValues
function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about:
template <class PaintProperties>
class SymbolSDFProgram : public Program<
...
PaintProperties>
{
...
};
using SymbolSDFIconProgram = SymbolSDFProgram<style::IconPaintProperties>;
using SymbolSDFTextProgram = SymbolSDFProgram<style::TextPaintProperties>;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that's something like what I was trying before and having trouble with -- but it's very possible that it was just other mid-refactor issues that were making it hard to see clearly. Lemme try again.
e2d307e
to
1400ed8
Compare
@jfirebaugh Okay, I think this is ready for a 👀 .
|
@@ -28,6 +28,7 @@ class SymbolInstance { | |||
CollisionFeature textCollisionFeature; | |||
CollisionFeature iconCollisionFeature; | |||
WritingModeType writingModes; | |||
std::size_t featureIndex; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In GL JS, we did this with a direct reference to the associated feature. Would it be better to do a similar thing here, making this a SymbolFeature *
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ note: the reason I didn't was because, since SymbolInstance
:SymbolFeature
is many:1
, it seemed like there would have to be some weird acrobatics using std::shared_ptr
. Rather than introduce that machinery, I figured since both of these objects are really internal to SymbolLayout, this index-based coupling seemed okay.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is okay for now. I started learning more about the various classes used during symbol placement and I'm thinking maybe we can move to a hierarchy where SymbolFeature
owns SymbolInstance
s, and can provide a direct SymbolFeature&
reference at the appropriate time.
Almost there, I think:
Update: @tmpsantos explained the context of the Memory.Footprint test. Some additional memory usage is necessary, because we now need to hold on to feature properties after @tmpsantos indicates that this isn't necessarily a blocker. cc @jfirebaugh (also FYI the remaining tests appear to be passing now) |
13d1d30
to
dcacc17
Compare
So we're going ~660kb over the Memory.Footprint test limit. How much under the limit are we before this change, i.e. how much is the total increase? |
Force-logged it off of master (by setting the threshold to 0 in the unit test, because apparently googletest doesn't let you just arbitrary informational stuff 🙄 ):
( https://travis-ci.org/mapbox/mapbox-gl-native/jobs/204010132#L1013 ) On this branch, we get:
So that's a 1.7MB increase. Seems pretty significant. |
It occurs to me that this test is measuring RSS, and so I'm not sure whether these literal numbers are meaningfully interpretable as "the increase in memory usage". |
What this test is doing is:
Few notes:
|
One last thing that is worth mentioning. This test is focused on Map objects rather than tiles. If your code increases tile memory usage by 250 Kb per tile it is unlikely that this test will catch it, but the effects will be seen in an app when rendering 10 tiles at once in a tilted map (~2.5 MB memory increase). Additional manual testing with larger maps is advised, I personally like this tool: |
Capturing some notes from chat:
|
Did some tests to measure the memory impact of this change. On my Linux box, for rendering this map (1920x1200) using GLFW:
I started the map with the same configuration 10x with this PR applied and without (the hash just before the first commit of this branch. In average Repeating the method with a 1024x768 map, the observed increase was 8 MB. |
Looks like in the Memory.Footprint test, we're dealing with 980 symbol features total... that's 1.7KB per symbol feature, which would be plausible if we're keeping the geometries around but seems too big for just the text, placement, properties. @jfirebaugh two questions:
|
stats from profiling in Instruments: Obtained by profiling the Memory.Footprint test, and the filtering the Instruments time range to go from the beginning to point of peak heap usage. The +43MB delta in heap usage includes: +8MB - Other than SymbolFeature, hard to say what these others are... probably allocations by various container classes? |
Here are a couple things I would try to reduce the memory growth:
|
src/mbgl/layout/symbol_layout.cpp
Outdated
@@ -229,7 +232,8 @@ void SymbolLayout::prepare(uintptr_t tileUID, | |||
const bool textAlongLine = layout.get<TextRotationAlignment>() == AlignmentType::Map && | |||
layout.get<SymbolPlacement>() == SymbolPlacementType::Line; | |||
|
|||
for (const auto& feature : features) { | |||
for (auto it = features.begin(); it != features.end(); ++it) { | |||
auto feature = *it; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is creating a copy; it should be auto& feature = *it;
. Then feature.geometry.clear();
will have a meaningful effect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🙈
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alas, this change only seems to take us down by 51K in the Memory.Footprint test: 6.88349e+07 => 6.87836e+07.
src/mbgl/layout/symbol_layout.cpp
Outdated
@@ -496,6 +502,12 @@ std::unique_ptr<SymbolBucket> SymbolLayout::place(CollisionTile& collisionTile) | |||
layout.get<IconKeepUpright>(), iconPlacement, collisionTile.config.angle, symbolInstance.writingModes); | |||
} | |||
} | |||
|
|||
const auto feature = features.at(symbolInstance.featureIndex); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, should be const auto& feature = ...
.
I think that one consequence of this is that it would be problematic to then clear the geometries after we didn't need them anymore (unless we added something like a |
src/mbgl/layout/symbol_feature.hpp
Outdated
properties(feature->getProperties()) | ||
SymbolFeature(std::unique_ptr<GeometryTileFeature> feature_) : | ||
feature(std::move(feature_)), | ||
geometry(feature->getGeometries()) // we need a mutable copy of the geometry for mergeLines() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
☝️ @jfirebaugh note - mergeLines relies on mutating the SymbolFeatures' geometries, but getGeometries() returns a copy, so this bit of copying still needed to happen.
@jfirebaugh implemented your two suggestions from #7939 (comment). Moving |
src/mbgl/layout/symbol_layout.cpp
Outdated
} | ||
|
||
// Determine and load glyph ranges | ||
const size_t featureCount = sourceLayer.featureCount(); | ||
for (size_t i = 0; i < featureCount; ++i) { | ||
auto feature = sourceLayer.getFeature(i); | ||
if (!leader.filter(feature->getType(), feature->getID(), [&] (const auto& key) { return feature->getValue(key); })) | ||
SymbolFeature ft(sourceLayer.getFeature(i)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be better to construct SymbolFeature
only for features that pass the filter, to avoid unnecessarily building and copying the geometry.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, should have been more explicit:
auto feature = sourceLayer.getFeature(i);
if (!filter)
continue;
SymbolFeature ft(std::move(feature));
We don't want to call sourceLayer.getFeature(i)
twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh whoops, yeah that's what I'd intended -- trying to go too fast heh
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jfirebaugh amended.
src/mbgl/programs/symbol_program.cpp
Outdated
@@ -1,14 +1,21 @@ | |||
#include <iostream> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leftover from debugging?
src/mbgl/programs/symbol_program.cpp
Outdated
|
||
namespace mbgl { | ||
|
||
using namespace style; | ||
|
||
static_assert(sizeof(SymbolLayoutVertex) == 16, "expected SymbolLayoutVertex size"); | ||
|
||
MBGL_DEFINE_ENUM(SymbolSDFPart, { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MBGL_DEFINE_ENUM
is needed only where enum ↔ string conversion is required, which isn't the case with SymbolSDFPart
.
@@ -8,6 +8,7 @@ | |||
#include <mbgl/util/geometry.hpp> | |||
#include <mbgl/util/size.hpp> | |||
#include <mbgl/style/layers/symbol_layer_properties.hpp> | |||
#include <mbgl/style/layers/symbol_layer_impl.hpp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't be necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is needed here because it's where style::{Icon,Text}PaintProperties
are declared.
src/mbgl/programs/symbol_program.hpp
Outdated
using SymbolLayoutVertex = SymbolLayoutAttributes::Vertex; | ||
using SymbolAttributes = SymbolIconProgram::Attributes; | ||
using SymbolIconAttributes = SymbolIconProgram::Attributes; | ||
using SymbolGlyphAttributes = SymbolSDFTextProgram::Attributes; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename this to SymbolTextAttributes
.
9a44bd0
to
32a6d22
Compare
A little more on the memory usage increase. Running a single iteration from the Memory.Footprint test and estimating each symbol feature's footprint as update: Property values are shared among features within a tile, so this is actually overestimate. update 2: Corrected to not double-count shared property values in each tile layer: 434,632 |
32a6d22
to
7d78614
Compare
@@ -15,9 +15,17 @@ class VectorTileLayer; | |||
|
|||
using packed_iter_type = protozero::iterator_range<protozero::pbf_reader::const_uint32_iterator>; | |||
|
|||
struct VectorTileLayerData { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also have a shared_ptr<const std::string> data
member, to ensure that the data backing the tags_iter
and geometry_iter
is retained.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jfirebaugh 👍 done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@anandthakker @jfirebaugh - Landed here in a revised effort to integrate https://github.com/mapbox/vector-tile back into mbgl. I have some questions.
-
Without
VectorTileLayerData
and isshared_ptr<const std::string> data
member what exactly would go wrong? I figure there must be somewhere in the code that expects to create aVectorTileLayer
and use it/cache it in such a way beyond when the underlying data would be kept alive. So keeping a shared_ptr to the data keeps it alive as long as allVectorTileLayer
instances are alive? Can you point to the code that needs this? (I could not find it) -
Why does the
VectorTileFeature
need astd::shared_ptr<VectorTileLayerData> layerData
rather than justconst VectorTileLayer&
. This indicates to me that somewhere in the code expects to keep aVectorTileFeature
instance alive longer than the parentVectorTileLayer
. Is this happening and can you point to the place in the code that requires is? Or is this perhaps unneeded - in that theVectorTileLayer
will always be alive as long as theVectorTileFeature
created from it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does the VectorTileFeature need a std::shared_ptr layerData rather than just const VectorTileLayer&. This indicates to me that somewhere in the code expects to keep a VectorTileFeature instance alive longer than the parent VectorTileLayer
@springmeyer Yeah -- this is a possibility that did not exist before, but was introduced in this PR. (And apologies: I should have left a comment justifying the shared_ptr
s introduced here.)
The reason it can now happen is over in SymbolLayout
, and specifically SymbolFeature
(here). SymbolLayout
is constructed with a vector tile layer, and then later on--when there's no guarantee that the vector tile data is still around--expected to produce a "bucket" containing final rendering data. Before this PR, we copied the necessary data into SymbolFeature
structs which SymbolLayout
could hang on to; but because we need feature properties for data-driven styling--and because these turn out to be quite expensive to copy--SymbolFeature
now holds a reference to the VectorTileFeature
.
Thus, VectorTileFeature
s may need to outlive their parent VectorTileLayer
s, so they need shared ownership of VectorTileLayerData
.
As for the shared_ptr<const std::string> data
that's held by VectorTileLayerData
-- I think we need this basically as a consequence of the above, since VectorTileFeature
s may parse PBF data lazily. But even without these symbol-layout-related changes, it's making explicit a dependency that already existed, since there's no explicit guarantee that the parent VectorTileData
would outlive a particular VectorTileLayer
. The lifecycles happened to be okay before this, but only because of implicit constraints on how we used these objects.
@jfirebaugh should probably check my math on this, as I haven't spent time in the parts of our vector tile parsing/caching stuff outside of SymbolLayout
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that's correct. One minor note is that shared_ptr<const std::string>
predated the addition of VectorTileLayerData
and comes from the definition of Response
. I believe this was originally done to avoid string copies. Might be better to use move semantics now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@anandthakker and @jfirebaugh - awesome thank you for the confirmation. I'm grateful for the confirmation and will now proceed with my PR revising this code and keep all the explicit ownership relationships fully intact. The context I should have mentioned before is that I hope my PR will help performance so I was looking closely at the nearby code for any optimization potential, which is what lead me to the question of whether the shared pointers were absolutely needed. Great to know they are.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ticketed @jfirebaugh's idea: #9167
src/mbgl/tile/vector_tile.cpp
Outdated
VectorTileFeature::VectorTileFeature(protozero::pbf_reader feature_pbf, const VectorTileLayer& layer_) | ||
: layer(layer_) { | ||
VectorTileFeature::VectorTileFeature(protozero::pbf_reader feature_pbf, std::shared_ptr<VectorTileLayerData> layerData_) | ||
: layerData(layerData_) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
layerData(std::move(layerData_))
Latest changes bring us down to |
def4a1e
to
c99444a
Compare
Prepares for enabling DDS on symbol paint properties by allowing the SymbolFeatures, which we keep around after constructing SymbolLayout, to be used in evaluating data-driven paint properties later in the layout process.
The `Program` types are set up to bind GL attributes to each of the data-driven paint properties specified in the `PaintProperties` type provided. Since `SymbolPaintProperties` specifies both `Text*` and `Icon*` properties, the symbolIcon, symbolIconSDF, and symbolGlyph programs each attempt to bind roughly double the number of attributes that they actually need. This change addresses this by: - Adding the more specific `IconPaintProperties` and `TextPaintProperties` types, which are subsets of the full `SymbolPaintProperties`. - The symbol layer continues to use its `SymbolPaintProperties paint` member to track layer property state, but it provides helpers that construct objects of each the specific `{Icon,Text}PaintProperties::Evaluated` type, for use by the painter. - The three symbol programs instantiate `Program<>` using the appropriate `{Icon,Text}PaintProperties` type.
c99444a
to
7a7c2ae
Compare
No description provided.