New quest: Detail what landuse=orchard produces

[rugk]

What is possible is to ask users for the produce of farmland with perennial plants, a.k.a plantations: apples, pears, bananas, rubber, coconuts, cocoa, coffee, grapes etc. […]
This also requires a country-specific list of common produce types, like it is done for the sports quest because otherwise there are too many choices.

Sure, also trees are easy to answer. So we would have to let the user between trees and fruits first:

What is grown on this orchard?

• trees
• fruits
• (more menu:) "This is no orchard" and "Something else" (maybe "vineyard" 😃)

So according to the selection we would have to allow the user to select a type of tree and… seems there are only trees(?).
Or does OSM categorize bushes into the "trees" category too?

The UI could be similar to the one of the street paving.

Edit: We need images for these crops:

• Edit: tomatoe (later added)
• crop=pepper
• trees=banana_plants
• trees=orange_trees
• trees=lemon_trees|trees=lime_trees
• trees=guava_trees|trees=mango_trees
• trees=coconut_palms
• trees=apple_trees
• landuse=vineyard
• trees=pineapple_plants
• trees=pear_trees
• trees=plum_trees
• trees=coffee_plants
• trees=peach_trees
• trees=grapefruit_trees
• crop=strawberry
• trees=avocado_trees
• trees=apricot_trees
• trees=cherry_trees
• trees=papaya_trees
• trees=cacao_trees
• trees=walnut_trees
• trees=oil_palms
• trees=fig_trees
• trees=tea_plants
• trees=almond_trees
• trees=raspberry
• trees=palms
• trees=piper_plants
• trees=olive_trees
• trees=date_palms
• trees=rubber_trees
• trees=cashew_trees
• trees=hop_plants
• trees=hazel_plants
• trees=chestnut_trees
• trees=blueberry
• crop=sisal
• trees=kiwi_plants
• crop=nut
• trees=pistachio_trees
• trees=nutmeg_trees
• crop=jute
• trees=persimmon_trees
• crop=vanilla
• trees=cranberry
• trees=agave_plants
• trees=kola_trees
• trees=mate_plants
• trees=jojoba_plants

[westnordost]

Forget trees, bushes or fruits. The user (and we) don't care if apples grow on trees or bushes. If he sees apples, he will want to mark "apples" and be done with it.
The choices just must be ordered in an order that puts the most expected values (i.e. apples, pears etc for Germany) in front. That is why I mentioned the sports quest because that quest also applies a country-specific order (i.e. cricket goes first in GB). This order needs to be researched and I would say, a good old "world atlas" like we did have them in schools is a good starting point.

[rugk]

The user (and we) don't care if apples grow on trees or bushes.

Oh, this made my day. But you're right…

I am currently looking for statistics. For copyright reasons I do not want/cannot to share one here, so could you drop me a mail, so I can reply?

As for a worldwide overview you could take https://de.statista.com/statistik/daten/studie/28991/umfrage/erzeugung-von-obst-weltweit-nach-arten/ or https://en.wikipedia.org/wiki/Agriculture#Crop_statistics.

[westnordost]

Sent

[rugk]

Ah, I think I finally found a good (and also reliable, for what's it worth) source. It seems you can also download the data as CSV.

--> http://www.fao.org/faostat/en/#rankings/commodities_by_country

One has to tweak it a bit here, it e.g. also includes other things, such as meat, but apart from that, these are some stats.

[rugk]

I've finished my parser, so here it is: https://github.com/rugk/crop-parser
The data is here: https://github.com/rugk/crop-parser/blob/master/result/mostPlantedCrops_2014.yml

Now I also know that in Faroe Islands mostly only potatoes 🥔 are produced… 😃

In the data, we often have many variants of the same "basic" product. E.g. beans in all colours… 😉

[rugk]

However, one can also see the amount of different fruits there is. (and these are only the top 5 per country) Getting a picture for each one could be difficult/cumbersome, so you likely need to start with a few placeholder pictures.

[westnordost]

Wow, cool. I anticipated you'd go through it by hand. Some notes:

• The final YML should use the OSM tags as defined in the wiki
• Only crops grown on an orchard ("mehrjährig") should be in there (didn't check if this is the case already)
• Seems to be incomplete. I.e. for Germany I am missing i.e. pears, grapes (for wine) as well. So, I would say that the final YML is not something that comes out of your parser but that uses the output of the parser only as a starting point. So, from that perspective, the parser's function was single-use and has now served its term - we got the data from FAO, now we build on top of that

[rugk]

Yeah, converting the crops to OSM tags is another thing, which should be added. However, now you at least have an overview.

Only crops grown on an orchard ("mehrjährig") should be in there (didn't check if this is the case already)

Yeah, there is a blacklist.

Seems to be incomplete.

Ups, yeah. There was a mistake in the parser (it sorted one time too much). This is fixed in rugk/crops-parser@73793ba.

I.e. for Germany I am missing i.e. pears, grapes (for wine) as well.

Grapes are now included. Pears are only in Belgium in the top-5. Seems we overestimate the production… 😄 (at least in tonnes) Taken from the raw data in Germany only 44972 tonnes pears, 155300 pears (dry) and 28917 pears (green) were produced (Note they are not summed up in the parser currently.) . That's less than cabbage ("and other brassica") with 802888t.
When we sum up the pears, we still only get 229189, which is still less than a third of the production of cabbage.

[westnordost]

Now it is probably too late to say that, but today it came to me that it may also be a good approach to simply note down the climate zone(s) for each country in an YML and then show the different crops based on these zones. Because I have the impression that what is being planted is more dependent on the climate zone instead of the country. Could this be?

Because, there are just too many crops that can be grown within a country. Even if they are not in the top X, they should still be displayed right? Mentioning all of the possible ones for each and every country is perhaps too much work (to maintain).
Perhaps the per-country YML could serve the quest to change the order of the items being displayed and the climate zone determines which crops are shown at all.

[rugk]

Could this be?

Hehe, yeah, maybe. But we would need new data for it. However, island and so on (MH or the example I already mentioned above: Faroe Islands) may be special examples…

Perhaps the per-country YML could serve the quest to change the order of the items being displayed the climate zone determines which crops are shown in the first place.

Hmm, maybe.

[rugk]

So I've looked a bit for statistics and it was quite hard as mostly the natural vegetation is considered, but here at the bottom we have some data, at least for Africa, although the climate zones are of course the similar in the world. (They should be, actually…)

However the data there is still quite vague and the list is often very small. So it is not the data you want to use to limit what should be shown to users.
Maybe we need a searchable text input here.

There are also so-called agro-ecological zones, but these have to be made for each region and maybe are even smaller (more detailed) than climate zones, so we won't get far here.

In any case I actually like the hard facts/data we have per country, so maybe sticking to them is the best idea. We can, of course, generate another list of the top-20 crops per country, so we have really every fruit produced there in the list. In a test run I did so and here is the result. We could of course also just include all available data by not limiting the list… (For Germany there are around 70 crops listed)

[westnordost]

Uiuiui, so much data... I think with more than 4 rows of icons in an image list, it really gets unübersichtlich. If there are so many different types of orchard crops, keep in mind that all of the possible ones need to be translated into every language and possibly (if it'll be a list with images) find pictures for it. So I guess, to limit the work, the quest should only start with the most popular crops and can be extended over time to include more. The app could ignore those crop types from the YML it does not have resources for.

…

On 2 July 2017 15:34:15 CEST, rugk ***@***.***> wrote: So I've looked a bit for statistics and it was quite hard as mostly the natural vegetation is considered, but [here at the bottom](http://www.fao.org/docrep/S2022E/s2022e06.htm#2.2%20major%20crop%20zones) we have some data, at least for Africa, although the climate zones are of course the similar in the world. (They should be, actually…) However the data there is still quite vague and the list is often very small. So it is not the data you want to use to limit what should be shown to users. Maybe we need a searchable text input here. There are also so-called [agro-ecological zones](http://www.fao.org/docrep/W2962E/w2962e-04.htm), but these have to be made for each region and maybe are even smaller (more detailed) than climate zones, so we won't get far here. In any case I actually like the hard facts/data we have per country, so maybe sticking to them is the best idea. We can, of course, generate another list of the top-20 crops per country, so we have really every fruit produced there in the list. In a test run I did so and [here is the result](https://gist.github.com/rugk/d43700234e6a3023c6a8384ea6644ec9). We could of course also just include all available data by not limiting the list… (For Germany there are around 70 crops listed) -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #368 (comment)

-- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[rugk]

Okay, another idea: OSM allows "cereal" as a crop tag? Maybe we could include this? I mean the user may not be able to choose, which cereal exactly and the cereal may change, but at least "cereal" would be something, would not it?

BTW: It's also interesting that strawberries are not in the data. Not at all… I however, know some fields here in Germany. Mostly you can pick them up by yourself (and pay, afterwards). Edit: They are in the data.

[rugk]

Looking through OSM wiki, the tags for crop or produce are really rare. E.g: Are coconuts "crop=nut"??
So our different crops on orchards get less here… 😆 Bad for OSM, however.
They don't even have tomatoes 🍅 there and no soybeans (one of the most planted products of the world)…

I've adjusted the script to convert the output to OSM tags using a small database created out of the (englisch) OSM wiki.
So the result when only allowing OSM keys is really much more "cleaned up": https://github.com/rugk/crops-parser/blob/master/result/mostPlantedCrops_2014_OSMonly.yml
Due to OSM it, however, is not really a representation of what is mostly planted in a country. YOu can have a look on the other file to see where OSM keys are missing.

PS: As for the climate zone thing I noticed that at least the trees on OSM are actually categorised. (Tropical, Subtropical, Temperate)
PPS: Also strange, no cranberry in the source data. Okay, 2014 they maybe were not that popular, but anyway…

[rugk]

OSM or the OSM wiki is somehow bad. Just found some tree tags on another page not even mentioned on the main tree page. Maybe someone can help to fill in the CSV database there.

[westnordost]

I dare to say that by now, you did more research than the one who made the wiki page. You can add new trees if any one tree is not mentioned or documented wrongly

[westnordost]

@csv database: I'd use trees wherever it makes sense. I have the impression that the trees key is more for plantations/orchards - perennial plants and that the crop key is more for farm landuse.

[rugk]

Yeah, sure I could just add/invent my own tags there, but I want to stay compatible to existing mapping standards. If we assume everyone follows the wiki page I can surely do so…

I'll do some work here…

[westnordost]

The mapping standard is trees=*. Don't expect the wiki to be complete, it is a wiki.

[rugk]

In any case, this question is still open:

Okay, another idea: OSM allows "cereal" as a crop tag? Maybe we could include this? I mean the user may not be able to choose, which cereal exactly and the cereal may change, but at least "cereal" would be something, would not it?

[rugk]

So finally here is a list of missing OSM tags (created with the data from 2013/24, using the blacklist).

[westnordost]

Aren't all of the crops in the list non-permanent (non-perennial) crops?

cereal

Cereals are non perennial, it doesn't make sense to capture this data in OSM when it may change several times a year. Don't suppose that it will be "some kind of cereal" all year through on the same field everywhere. Even in Germany it could be canola ("Raps"), cabbages, beet etc in between, depending on the demand.

Also, only in industrial agriculture with intensive fertilization and pesticides it can be sustained to plant year after year the same crops. Non industrial agriculture will use crop rotation and that would be throughout most of the world.

[rugk]

Ah, yeah, right… Seems sometimes even potatoes are used.

Non industrial agriculture will use crop rotation and that would be throughout most of the world.

(Hopefully) not only there.* I just skimmed the German wiki article on "Fruchtfolgen" and really many things may be used there (potatoes and chick peas…), so we may have to extend our exclusion list even more… That's getting difficult here.

However, some not-so-famous fruits (Ginger, Vanilla, Gooseberries) are likely not used in crop rotation, …

*

Heute ist der Fruchtwechsel ein wichtiger Bestandteil der Landwirtschaft und des modernen Agrarmanagements, sowohl im konventionellen als auch im ökologischen Landbau. […] Es gibt heute weltweit auch eine Konzentration von Monokulturen, zum Beispiel beim Anbau von genverändertem Soja oder Mais in Südamerika und den USA. […] Auch der vereinzelte Anbau von mehrjährigen Selbstfolgen, also der 3- bis 5-malige Anbau gleicher Feldfrüchte, beispielsweise von Mais, gehören heute zur gängigen Praxis

[rugk]

Is not there any farmer with some experience in this field? Not that many on GitHub, probably… 😃

I now added some crop OSM tags missing and the list gets huge. And no, it still does not cover any majority of the top-5 as it differs heavily what is planted per country.
Also we get this crop rotation problem… So maybe we need to be more radical than using a blacklist there.

My ideas:

1. Use a whitelist?
2. Only include fruits growing on
   2.1. trees (farmers likely won't cut down trees to plant different ones that fast) and shrubs
   2.2. and lianas, like for kiwis (source)
   2.3. and maybe some crops, which can/are only (be) planted in monocultures (rice, source)…

BTW: We also more or less already have a whitelist as the osmcrops.csv, when set to only include OSM tags, acts like a whitelist, which does throw away all crops, where no OSM tag exists.

[rugk]

Actually the definition of an orchard is:

An orchard is an intentional planting of trees or shrubs that is maintained for food production.

However, the article goes on:

A fruit garden is generally synonymous with an orchard, although it is set on a smaller non-commercial scale and may emphasize berry shrubs in preference to fruit trees.

Sure, berries can also be mapped easily…

OSM says:

The tag can also apply to a fruit garden, generally synonymous with an orchard, although it is set on […] (and repeating Wikipedia here)

So we need to include fruit gardens.

The tag cannot be used for a garden producing vegetables under some fruit trees when the vegetable production is the main use and the trees are too scarce.

Good, so no vegetable, nothing growing on ground… (?)

[rugk]

So final question: Should be only include trees 🌳 ? Or are kiwi 🥝 plantages (does such a thing exist?) marked with "orchard" too?
And should we maybe drop everything, which may be mapped under "farmland"? So should we drop rice 🍚 ?

[westnordost]

Are you asking what would still fall under the OSM orchard tag or are you asking what what should be the scope of this quest?

In any case, we should assume the following: Anything tagged as orchard but without any crop or trees has been tagged from satellite imagery because "it looks like a plantation", ,i.e. systematically planted bushes or trees.
This is the input data. This does include kiwis, does include tree nurseries but does not include rice fields.

The user should be able to give a reasonable answer. How his answer is translated into OSM tags, is another thing. I did not concern myself so much with it (did not read OSM wiki), but I would draw the line between orchard (plantation) and farmland exactly between "cultivated on a yearly or less basis" (farmland) and "cultivated permanently / perennial" (orchard). This would put cotton into the farmland category.

[rugk]

POE still shows "100%" translated.

[westnordost]

Someone translated it immediately after I added the strings :-D

[rugk]

I did not do so… 😢

[rugk]

Maybe a friendly bot, considering that they are easy to translate… 😉

Could I somehow filter all those belonging to "orchard" to review them? (Maybe using tags, you currently only have a "html" one…)

Also it's sad I cannot see any stats, where (these strings) still need to be translated…

[westnordost]

Just go to the last page. If you hover over the values, it should also tell you who did the translation and when.

[rugk]

Could I somehow filter all those belonging to "orchard" to review them?

Ah, just using the "sort: updated DESC" now.

And no I cannot see who did it. Maybe only the admin can see this.

[rugk]

Okay, review finished. And it was good to review them, some mistakes had to be corrected. And I am a crops pro now… 😜

[rugk]

BTW, why are the strings in capital letters? Should not they be in lower case in most languages?

[rugk]

Current status: Images found, data complete, OSM list changed again, lists regenerated. I also checked, as good as I could, that we have at least images for all result data.

TODO: Batch-resize.

Also: All translators, please pay attention to the difference between peppers and pipers. See rugk/crops-parser#50 for details.

Reminder for @westnordost: Due to splitting of OSM keys, the YAML files may contain more than 15 elements for some countries. Just so you are aware. You can handle this as you want.

[rugk]

Data changed to use area instead of tonnes. Images resized.

That the per-area-data is used is not necessary for the initial version, the data can be always updated later. (In case you don't feel like looking at that PR.) But the pictures is (the last thing that is still) missing.

Ah, okay. I basically waited for that PR there. It was blocking that thing a bit. So I just picked the source data as the other part of the PR does not affect StreetComplete/this issue, so the data should be okay now.
I also generated each pair of the source data, so now you can choose the source. If you really want to have a complete list, you may even manually combine them. As you can see in the last line, in the "area version" the data of "Saint Pierre and Miquelon (PM)` is missing. In the tonnes version, you have some data there.

[westnordost]

[westnordost]

Merged.

[rugk]

Awesome! 🎉

[rugk]

I corrected it in the German translation, but again to emphasize for all translators:
REMOVED: CURRENTLY BEING CHANGED

[westnordost]

Huh? What you are saying is wrong. There are the following keys:

produce_bell_peppers
produce_pepper
produce_chili

[westnordost]

Paprika, Pfeffer, Chilli
in German

[rugk]

Oh, you actually switched to bell_pepper without any discussion or wiki adjustion? (except of expressing the idea) Okay, your choice/responsibility…

Also produce_bell_peppers should be singular, so at least produce_bell_pepper.

In any case I'd also like to adjust the image names, as one chooses the wrong one otherwise.

[westnordost]

This is just the key for strings, the exact name of the key does not matter. They reflect how it should be translated.

"What is being cultivated here"?

• Bell peppers
• Chili
• Pepper (not peppers)

[westnordost]

Also, I renamed your image names for use in StreetComplete anyway.
(i.e. "ananas")

[rugk]

Ah, so it is not the key being put into OSM?

[rugk]

BTW: Also cannot find produce_bell_peppers in PoEditor.