-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Computed (on-demand) shape source #6940
Computed (on-demand) shape source #6940
Conversation
@JesseCrocker, thanks for your PR! By analyzing the history of the files in this pull request, we identified @boundsj, @1ec5 and @frederoni to be potential reviewers. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just some cursory feedback for starters.
@param x | ||
@param callback A block to call with the data that has been fetched for the tile. | ||
*/ | ||
- (void)getTileForZoom:(NSInteger)zoom x:(NSInteger)x y:(NSInteger)y callback:( void (^)(NSArray<id <MGLFeature>>*) )callback; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename "zoom" to "zoomLevel" for consistency with other SDK APIs. Should x and y be unsigned? Rename "callback" to "completionHandler" for consistency.
/** | ||
Request that the source reloads a tile. Tile will only be reloaded if it is currently on screen. | ||
*/ | ||
- (void)updateTile:(NSInteger)z x:(NSInteger)x y:(NSInteger)y; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename to -setNeedsUpdateAtZoomLevel:x:y:
.
extern const MGLGeoJSONSourceOption MGLGeoJSONSourceOptionSimplificationTolerance; | ||
|
||
|
||
@interface MGLGeoJSONSourceBase : MGLSource |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an unfortunate name for a class. Maybe someone else has an idea for what to call it. Also, it seems like it should declare some initializers and properties related to the options declared above.
impl(static_cast<Impl*>(baseImpl.get())) { } | ||
|
||
void CustomVectorSource::setTileData(uint8_t z, uint32_t x, uint32_t y, const mapbox::geojson::geojson& geoJSON) { | ||
impl->setTileData(z, x, y, geoJSON); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indent by four spaces.
|
||
- (instancetype)initWithIdentifier:(NSString *)identifier options:(NS_DICTIONARY_OF(NSString *, id) *)options | ||
{ | ||
if (self = [super initWithIdentifier:identifier]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use four spaces per tab stop.
@param zoom | ||
@param y | ||
@param x | ||
@param callback A block to call with the data that has been fetched for the tile. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fill in this documentation and rename callback
. Point out that completionHandler
must be called. Incidentally, is it absolutely necessary to make this API asynchronous? A synchronous data source method would be far less error-prone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't need to be async, since it's called on it's own NSOperationQueue. But it definitely would not work to have be it be synchronous on the main queue. Should i change this to be synchronous?
@end | ||
|
||
/** | ||
A source for vector data that is fetched 1 tile at a time. Usefull for sources that are |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/Usefull/Useful/
/** | ||
Reload all tiles. | ||
*/ | ||
- (void)reload; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename to -reloadData
for consistency with Cocoa classes like UITableView.
/** | ||
Request that the source reloads a tile. Tile will only be reloaded if it is currently on screen. | ||
*/ | ||
- (void)setNeedsUpdateAtZoomLevel:(NSUInteger)z x:(NSUInteger)x y:(NSUInteger)y; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It isn’t usually obvious to the developer what to pass in as x
and y
. Would it be possible to rework this method to take an MGLCoordinateBounds
, à la -[MKOverlayRenderer setNeedsDisplayInMapRect:zoomScale:]
?
proxyType = 1; | ||
remoteGlobalIDString = DA25D5B81CCD9EDE00607828; | ||
remoteInfo = settings; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert this change. The code signing issue was fixed in #6948.
@@ -2061,11 +2078,6 @@ | |||
/* End PBXSourcesBuildPhase section */ | |||
|
|||
/* Begin PBXTargetDependency section */ | |||
DA25D5C81CCDA0C100607828 /* PBXTargetDependency */ = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert this change too.
platform/ios/src/Mapbox.h
Outdated
@@ -53,3 +53,4 @@ FOUNDATION_EXPORT const unsigned char MapboxVersionString[]; | |||
#import "NSValue+MGLAdditions.h" | |||
#import "MGLStyleValue.h" | |||
#import "MGLTileSet.h" | |||
#import "MGLCustomVectorSource.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also import MGLGeoJSONSourceBase.h.
/** | ||
Request that the source reloads a tile. Tile will only be reloaded if it is currently on screen. | ||
*/ | ||
- (void)setNeedsUpdateAtZoomLevel:(NSUInteger)z x:(NSUInteger)x y:(NSUInteger)y; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On second thought, maybe this should be called -reloadTileInCoordinateBounds:zoomLevel:]
, for consistency with -reloadData
.
/** | ||
An object that implements the `MGLCustomVectorSourceDataSource` protocol that will be queried for tile data. | ||
*/ | ||
@property (nonatomic, readonly, copy) NSObject<MGLCustomVectorSourceDataSource> *dataSource; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Properties that refer to data sources should be weak to avoid a retain cycle, should not copy, and should be id
rather than NSObject. Is there a reason it’s read-only? It’s unusual for a data source to be used on a custom queue.
a614fe3
to
8cba5ea
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m excited to see this PR come together. Hope you don’t mind my continued waffling on API naming. Trying to get this right so we don’t confuse people who are familiar with Cocoa APIs.
Don’t forget to update the iOS and macOS changelogs and add any new classes to the jazzy configuration files so they show up at the right place in the table of contents.
So far I’ve only looked at the Objective-C side of these changes. @kkaefer, could you have a look at the C++ side? To kick things off, how about some tests? 😄
@param y | ||
@param x | ||
*/ | ||
- (NSArray<id <MGLFeature>>*)getTileForZoomLevel:(NSUInteger)zoomLevel x:(NSUInteger)x y:(NSUInteger)y; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that this is a simple getter, it should be called -featuresInTileAtX:y:zoomLevel:
, without the “get”.
I think it would be just as generally useful to have a -featuresInCoordinateBounds:zoomLevel:
, per #6940 (comment), to accommodate those migrating from MKOverlayRenderer. If you think it’s worth keeping a tile coordinate–based API around, perhaps both methods could be optional methods on this protocol. MGLCustomVectorSource would end up calling one or the other, depending on which one is implemented.
@param dataSource An object that implements the `MGLCustomVectorSourceDataSource` protocol that will be queried for tile data. | ||
@param options An `NSDictionary` of options for this source. | ||
*/ | ||
- (instancetype)initWithIdentifier:(NSString *)identifier dataSource:(NSObject<MGLCustomVectorSourceDataSource>*)dataSource options:(NS_DICTIONARY_OF(MGLGeoJSONSourceOption, id) *)options NS_DESIGNATED_INITIALIZER; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In Cocoa, data sources aren’t normally required; when absent, the class that normally uses the data source assumes no data is available. By following that model here, it’ll be possible to set up a custom vector source before setting up its data source, which may be handy if the data source takes some time to set up. This would entail removing the dataSource:
parameter from this method and making the dataSource
property nullable.
Fetch data for a tile | ||
@param zoom | ||
@param y | ||
@param x |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When filling in the documentation, make sure to mention that this method is always called on the operation queue in the requestQueue
property.
@1ec5 Thanks for working with me on this, I know it's added work for you that wasn't on the roadmap. One issue i'm running into is when I run |
@1ec5 to Implement featuresInCoordinateBounds:zoomLevel: It needs to find the bounds for a tile, it looks like mbgl::LatLngBounds already has a constructor to create a LatLngBounds from a CanonicalTileID, but mbgl/tile/tile_id.hpp can't be included in any of the obj-c++ classes, because it includes boost/functional/hash.hpp, and boost isn't in the search path for the iOS targets. Is it OK to add boost to the header search path for the iOS targets? |
Yes, after adding the files to core, run |
Yes, I think it would be OK, although we’ll want to watch the SDK for any significant size increase as a result. To add Boost, add it as a Mason package like we do in core. Be sure to update the macOS configuration too. |
The parts of tile_id.hpp that depend on Boost are the three template specializations of |
As I’ve mentioned before, we need better names for MGLCustomVectorSource and MGLGeoJSONSourceBase, especially the latter. Meanwhile, #7160 proposes renaming MGLGeoJSONSource to MGLFeatureSource, since most use cases for MGLGeoJSONSource don’t even involve actual GeoJSON source code. So, two proposals:
Any other ideas? /cc @incanus |
603f2f5
to
f159953
Compare
I've added a couple of tests for this, both on the obj-c side and the c++ side, but i dont think they are really testing anything useful. Any guidance as to how to test this, especially the core part of it, would be greatly appreciated. |
After doing some more testing with this, I turned up memory corruption when a source was removed, and a tile fetch was in progress. My last commit should fix that. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than the one nit, my feedback on the SDK side has all been addressed.
@implementation MGLComputedShapeSourceTests | ||
|
||
- (void)testInitializer { | ||
MGLComputedShapeSource *source = [[MGLComputedShapeSource alloc] initWithIdentifier:@"id" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: indent by four spaces for consistency.
Do not create instances of this class directly, and do not create your own | ||
subclasses of this class. Instead, create instances of `MGLShapeSource` or | ||
`MGLComputedShapeSource`. | ||
*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the way, this is totally optional, but we have a mechanism for inserting a Swift code example into a documentation comment via unit tests. Feel free to use MGLShapeSource’s documentation as a model.
Im still working on fixing a use-after-free issue with this. If any calls to the data source were in progress when the style was changed, they will end up calling Address sanitizer picks up the use after free, but without it turned on this manifests as all sort of random crashes from memory corruption. I've got a good test case for this at trailbehind@1f5f51e , It calls I see 2 different ways to look at this problem:
Fixing based on 2 seems like the better fix, but it seems like a big change that would be beyond the scope of this PR.
So im guessing fixing based on 1 is the way to go, but im having trouble getting it to work right. There's also a third option, let this crash, and add a note to the documentation that the source must be removed from the style before the style is removed from the map. @1ec5 Do you have any suggestion for how to approach this? |
I've got the crash mentioned above fixed by removing the reference to the MGLComputedShapeSource from the MGLComputedShapeSourceFetchOperation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your pull request! This is definitely something that we want to add, but it's very hard to review this pull request since it's scattered over so many commits. We generally require all commits to only contain self-contained changes, so before merging this, we'll either have to squash this into one commit (or preferably) isolate self-contained changes in this PR and commit them individually.
I realize that you're more familiar with the iOS/macOS side of things, but we also need Android and ideally Qt bindings before adding this change, as well as a lot more changes that also test rendering of generated tiles.
} | ||
|
||
void CustomVectorSource::Impl::setTileData(uint8_t z, uint32_t x, uint32_t y, const mapbox::geojson::geojson& geoJSON) { | ||
double scale = util::EXTENT / util::tileSize; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
constexpr
void CustomVectorSource::Impl::setTileData(uint8_t z, uint32_t x, uint32_t y, const mapbox::geojson::geojson& geoJSON) { | ||
double scale = util::EXTENT / util::tileSize; | ||
|
||
if(geoJSON.is<FeatureCollection>() && geoJSON.get<FeatureCollection>().size() == 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.size() == 0
=> !.empty()
if(geoJSON.is<FeatureCollection>() && geoJSON.get<FeatureCollection>().size() == 0) { | ||
for (auto const &item : tiles) { | ||
GeoJSONTile* tile = static_cast<GeoJSONTile*>(item.second.get()); | ||
if(tile->id.canonical.z == z && tile->id.canonical.x == x && tile->id.canonical.y == y) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of passing z/x/y
separately, can we use CanonicalTileID
and then compare the objects directly?
|
||
#pragma clang diagnostic push | ||
#pragma clang diagnostic ignored "-Wunused-parameter" | ||
const auto callback = ^void(uint8_t z, uint32_t x, uint32_t y) {}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we instead use C++11 lambdas rather than Apple-Clang specific syntax? You can also remove the z/x/y names to avoid the pragmas. An even better approach would be to pass a null pointer as the callback and check whether callback
is invokable before calling it.
}; | ||
|
||
|
||
} // namespace std |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why was this moved to a separate file? While not wrong, it doesn't look like it's necessary to include in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was split into a separate file so that tile_id.hpp could be included in objective-c++ files in the iOS/macOS side of things without having to include boost.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MGLPolylineFeature *feature = [MGLPolylineFeature polylineWithCoordinates:coords count:2]; | ||
feature.attributes = @{@"value": @(x)}; | ||
[features addObject:feature]; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this entire function something that should be in the core code rather than in a binding-specific code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean as an implementation of #119? I don’t think it satisfies the requirements for that feature, namely that the graticule would have to be visible before the style even starts to load. But it does make for an interesting demo. (This file is part of macosapp; it isn’t part of any SDK.)
@@ -26,7 +27,10 @@ using SuperclusterPointer = std::unique_ptr<mapbox::supercluster::Supercluster>; | |||
|
|||
struct GeoJSONOptions { | |||
// GeoJSON-VT options | |||
uint8_t minzoom = 0; | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
space
CustomVectorSource(std::string id, GeoJSONOptions options, std::function<void(uint8_t, uint32_t, uint32_t)> fetchTile); | ||
|
||
void setTileData(uint8_t, uint32_t, uint32_t, const mapbox::geojson::geojson&); | ||
void updateTile(uint8_t, uint32_t, uint32_t); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the difference between setTileData
and updateTile
? If they're distinct, can we add some documentation?
@kkaefer Thanks for reviewing this. I think i've addressed all of your comments. Sorry about the large number of commits, i had to do some iteration to make this work, and then there were a bunch of commits renaming things based on review feedback. Im well versed in Android development and capable of doing the Android bindings for this, but probably wont have time to get to it for a couple months. |
#8473 is a squashed, reorganized version of this PR. |
Wondering if there's any progress (or interests from other users) on this? |
→ #8473. |
Implements new API proposed in #6861 for a custom vector source, that queries a class supplied by the client application for the features to display on a tile.
This defines a new class in core,
mbgl::style::CustomVectorSource
, and a corresponding new source class for iOS/MacOSMGLCustomVectorSourceDataSource
. There is also an new interface defined i obj-c,MGLCustomVectorSourceDataSource
that defines a single method- (void)getTileForZoom:(NSInteger)zoom x:(NSInteger)x y:(NSInteger)y callback:( void (^)(NSArray<id <MGLFeature>>*) )callback
I see 4 uses for this:
My use case for this is displaying GPS traces, which is #2 and #4 from the above list. I develop an application where users frequently have hundreds of GPS traces, but occasionally have tens of thousands, and some of them have hundreds of thousands of points per trace. The data can also be modified, either by the user, or by a background sync process, in either case the data will need to be redisplayed.