Use single API call to fetch Marathon resources.

[timoreimann]

Change Marathon provider to make just one API call instead of two per configuration update by means of specifying embedded resources, which enable retrieving multiple response types from the API at once. Apart from the obvious savings in API calls, we primarily gain a consistent view on both applications and tasks that allows us to drop a lot of correlation logic. Additionally, it will serve as the basis for the introduction of readiness checks which require application/task consistency for correct leverage on the proxy end.

Additional changes:

marathon.go:

• Filter on tasks now embedded inside the applications.
• Reduce/simplify signature on multiple template functions as we do not need to check for proper application/task correlation anymore.
• Remove getFrontendBackend in favor of just getBackend.
• Move filtering on enabled/exposed applications from taskFilter to applicationFilter. (The task filter just reached out to the applications anyway, so it never made sense to locate it with the
  tasks where the filter was called once for every task even though the result would never change.)
• Remove duplicate constraints filter in tasks, where it neither made sense to keep as it operates on the application level only.
• Add context to rendering error.

marathon_test.go:

• Simplify and reduce numerous tests.
• Convert tests with high number of cases into parallelized sub-tests.
• Improve readability/structure for several tests.
• Add missing test for enabled/exposed applications.
• Simplify the mocked Marathon server.

marathon.tmpl:

• Update application/task iteration.
• Replace getFrontendBackend by getBackend.

Fixes #1846

[timoreimann]

Note that I have explicitly tried to keep the changes to the tests as close to the bare minimum as possible, reflecting just the changes in the production code. Specifically, the Marathon tests badly need a builder pattern to reduce the amount of boilerplate.

I'll tackle that in a follow-up PR in order to keep this PR simple-ish.

[emilevauge]

Great work @timoreimann, design LGTM 👏

[juliens]

Great Job !
well spotted "embed=app.tasks" !! 😄
LGTM

[ldez]

Maybe you can add some integration tests (or in another PR)? 😉

[ldez]

Could you reduce tests definition with some helpers builder?

// ...
    task: aMarathonTask(withDefaultIPAddresses),
// ...

func aMarathonTask(builders ...func(marathon.Task)) marathon.Task {
	task := marathon.Task{
		Host:  "localhost",
		Ports: []int{80},
	}

	for _, builder := range builders {
		builder(task)
	}

	return task
}

func withDefaultIPAddresses(task marathon.Task) {
	task.IPAddresses =  []*marathon.IPAddress{
		{
			IPAddress: "127.0.0.1",
			Protocol:  "tcp",
		},
	}
}

I think you can do the same for marathon.Application, types.Backend, types.Frontend.

[timoreimann]

As noted, I intentionally kept modifications to the tests as minimal as possible in order to not risk changing both production code and the our test coverage safety net at the same time.

I'd like to do the test improvements in a follow-up PR. In fact, I'm already working on that and have started building a builder very similar to what you outlined.

Okay for you?

[ldez]

I don't really like to have 2 PRs for the same subject when 2-3 commits make the job: you can create a temporary test file, with the new tests, ensure all old tests always pass and replace old tests file by the temporary file. (with commit [and push to run CI] at each step)

But tests passed and code build.

Forget my request.

[emilevauge]

@ldez Marathon integration tests will be done #1406 ;)

[timoreimann]

Having at least some minimal integration test would actually be nice.

I'll wipe off the dust of #1406 and see if I can put something basic together. This PR doesn't change functionality, so ideally #1406 should provide immediate benefit.

[ldez]

@emilevauge I know, it's just a joke because @timoreimann and me have already speaking about Marathon integration tests. 😃

[ldez]

"Approval in disapproval." ©️

[ldez]

@timoreimann Could you rebase and resolve conflicts?

[timoreimann]

Rebased.

[juliens]

@timoreimann Could you resolve conflicts? Thx

[timoreimann]

@juliens Rebased.

[nmengin]

LGTM 👏