Upgrade to Meetup API v3 Since v2 Deprecated w/o Notice on Aug 20th

[allella]

@bcsnipes and @MarkMcDaniels noted the Meetup API is no longer supporting version 2 key authentication.

This majorly broke the ability to pull new/future events (UPDATE: removing key= from the app.py calls to Meetup's v2 API actually worked as a workaround - see below for more).

The script last generated a new file on August 20th and we now see Meetup had an Aug 15th cut-off date. I found a GitHub post from a month earlier, but why they wouldn't email active API key holders with a heads-up is beyond me. Big 🖕 to Meetup for that one.

It would seem the v3 API has been out for some time and this script used a v2 endpoint, perhaps for expedience of querying multiple groups at once.

We started exploring the use of Oauth 2, but it became apparent the pattern is more for creating an app where you need to allow users of your app to authorize certain permissions and then store an token to allow you to act on that user's behalf. This is not our use case, since we're looking to simply query mostly public data from the API from a single scheduled command line script.

In any case, it seems we can still do an API GET query for some endpoints, like the /:urlname/events, just one group at a time.

For instance, here is a list of Code For Greenville events, where (:urlname = Code-for-Greenville).
(if you get a CSRF auth error opening this URL then try a new browser session).

We'll likely also need to reformat the response fields a bit if the field names or values changed between v2 /events and v3 /:urlname/events responses.

Finally, it would seem v3 API uses :urlname and not the integer group_id we have in our organizations API. However, we can rather easily change these API key values in that API.

[allella]

Regarding the changes in the response "fields" between v2 and v3 of the events API, here is an example of v2 vs v3 event response for a single event.

The new v3 API returns

{
	"0": {
	"created": 1566015781000,
	"duration": 18000000,
	"id": "264078871",
	"name": "National Day of Civic Hacking",
	"date_in_series_pattern": false,
	"status": "upcoming",
	"time": 1569074400000,
	"local_date": "2019-09-21",
	"local_time": "10:00",
	"updated": 1567100506000,
	"utc_offset": -14400000,
	"waitlist_count": 0,
	"yes_rsvp_count": 7,
	"venue": {
		"id": 25052626,
		"name": "OpenWorks",
		"lat": 34.851924896240234,
		"lon": -82.39994812011719,
		"repinned": true,
		"address_1": "101 N. Main St 3rd Floor",
		"city": "Greenville",
		"country": "us",
		"localized_country_name": "USA",
		"zip": "",
		"state": "SC"
	},
	"group": {
		"created": 1398967208000,
		"name": "Code for Greenville",
		"id": 14215742,
		"join_mode": "open",
		"lat": 34.849998474121094,
		"lon": -82.4000015258789,
		"urlname": "Code-for-Greenville",
		"who": "Civic Innovators",
		"localized_location": "Greenville, SC",
		"state": "SC",
		"country": "us",
		"region": "en_US",
		"timezone": "US/Eastern"
	},
	"link": "https://www.meetup.com/Code-for-Greenville/events/264078871/",
	"description": "Truncated by Jim",
	"visibility": "public",
	"pro_is_email_shared": true,
	"member_pay_fee": false
	}
}

v2 /events

{
"results": [
	{
	"utc_offset": -14400000,
	"venue": {
		"country": "us",
		"localized_country_name": "USA",
		"city": "Greenville",
		"address_1": "101 N. Main St 3rd Floor",
		"name": "OpenWorks",
		"lon": -82.39995,
		"id": 25052626,
		"state": "SC",
		"lat": 34.851925,
		"repinned": true
	},
	"headcount": 0,
	"visibility": "public",
	"waitlist_count": 0,
	"created": 1566015781000,
	"maybe_rsvp_count": 0,
	"description": "Truncated by Jim",
	"event_url": "https://www.meetup.com/Code-for-Greenville/events/264078871/",
	"yes_rsvp_count": 7,
	"duration": 18000000,
	"announced": true,
	"name": "National Day of Civic Hacking",
	"id": "264078871",
	"time": 1569074400000,
	"updated": 1567100506000,
	"group": {
		"join_mode": "open",
		"created": 1398967208000,
		"name": "Code for Greenville",
		"group_lon": -82.4000015258789,
		"id": 14215742,
		"urlname": "Code-for-Greenville",
		"group_lat": 34.849998474121094,
		"who": "Civic Innovators"
	},
	"status": "upcoming"
	}
],
	"meta": {
		"next": "https://api.meetup.com/2/events?offset=1&sign=True&format=json&limited_events=False&group_urlname=Code-for-Greenville&photo-host=public&page=1&fields=&order=time&status=upcoming&desc=false",
		"method": "Events",
		"total_count": 25,
		"link": "https://api.meetup.com/2/events",
		"count": 1,
		"description": "Access Meetup events using a group, member, or event id. Events in private groups are available only to authenticated members of those groups. To search events by topic or location, see [Open Events](/meetup_api/docs/2/open_events).",
		"lon": "",
		"title": "Meetup Events v2",
		"url": "https://api.meetup.com/2/events?offset=0&sign=True&format=json&limited_events=False&group_urlname=Code-for-Greenville&photo-host=public&page=1&fields=&order=time&status=upcoming&desc=false",
		"signed_url": "https://api.meetup.com/2/events?offset=0&format=json&limited_events=False&group_urlname=Code-for-Greenville&photo-host=public&page=1&fields=&order=time&status=upcoming&desc=false&sig_id=12862924&sig=18683f2486a4ad36d382a4ef9ccc3d8bb356dce6",
		"id": "",
		"updated": 1567100506000,
		"lat": ""
	}
}

[allella]

It just occurred to me that the API key is not required for v2 GET /events. The same is true of the v3 GET of /:urlname/events

SOOOOO, removing key= from the Python script actually renders the v2 results, which provides a temporary fix.

Given Meetup v2 API is deprecated, and they cut off key authentication, we should still leave this issue open and convert over to v3 API calls.

[allella]

Commit 9ecf5cd removes use of Meetup key values, which proved to be unnecessary anyway.

[bcsnipes]

I actually just committed an update to my local branch, and it seems to have fixed the issue for v3.

Now I just need to update format_meetup_events() and then I can issue a pull request for you to have a look.

[allella]

Thanks. @bcsnipes I just did a commit so you can git stash your changes, git pull and check what I committed and see if there's anything else that needs to be done.

[allella]

@bcsnipes if you have changes to move to v3 events endpoint then that would be the next step, but the commit I just did worked around the v2 auth issue for the reasons mentioned above.

[allella]

Kudos to @bcsnipes for getting the start on a v3 Meetup integration.

Brett, I think an easy and cleaner way to get the group names is for us to put those values in the organizations API in the "field_events_api_key" field. This way you can just ask for the orgs and the API "key" will be in that data so you don't have to interrogate the URL fields for it.

If you have time to update the script to use the value of field_events_api_key then I can update the orgs API with the correct values.

[bcsnipes]

thanks, @allella, that's a great idea -- I'll work on updating the script.

[allella]

@bcsnipes Also, note that if you want to keep your GitHub fork up to date with this copy then one way to do that is similar to how I did a remote tracking branch on your fork.

If you see the old notes from our SC Codes map project then there's notes on how to setup an "remote upstream".

Basically, it would looks something like

git remote add upstream https://github.com/codeforgreenville/upstate_tech_cal_service.git
git fetch upstream

Run git branch -a and you'll see a new branch "remotes/upstream/master", along with any other remote branches from this project.

The fetch downloads the latest upstream code into a branch within your environment. You won't see anything change, but your local Git then has a copy of the upstream branches at your disposal.
Now, if within your git environment, if you want to update your "master" branch to include the latest from the remote's master branch, then you merge the upstream code into your branch, like

git checkout master
git merge upstream/master

Note that "upstream" could be called anything, but since you forked from this project the word "upstream" is a common vocabulary for it.

You'll notice in the pull request comments that I created a remote called "bcsnipes", but I could have called it "brett", or really anything unique besides origin, since "origin" is the default name for your primary remote repo.

You can look at .git/config to see where Git stores the relationships between remote names and URLs where they point.

[allella]

@bcsnipes

The field_event_service values are now the "group names" instead of the integers used in the v2 Meetup API.

Here's a list of the data, best viewed in Firefox
https://data.openupstate.org/rest/organizations?org_status=active&_format=json&event_service=meetup

[allella]

Closing this since @bcsnipes #21 and #22, along with removing the authentication, has us running on the v3 Meetup.com API.

If we need to request something from v3 that requires authentication then I have an API consumer registered under my account that we could use and/or we can register one under a CFG controlled account..