Add dogstreams.d for dynamic dogstreams configuration

[gtaylor]

Building on #1550 (fixing #753), I have added a dogstreams.d feature (similar to conf.d and checks.d). Users can stick *.yaml files in dogstreams.d, allowing the more easily dynamic configuration of streams.

The reason this ended up being important is that as we started writing Ansible playbooks for our infrastructure, we realized that it was very difficult to mix and match dogstreams across roles. This will be the case for anyone doing config management with things like Chef, Puppet, Salt, etc.

This PR includes unit tests for the feature, and I have updated the changelog. You may want to merge this instead of #1550, but should be able to merge it after #750 if you wanted to get that smaller change in first.

dogstreams.d also supports wildcard paths, which came from #1550.

[gtaylor]

@LeoCavaille Any feedback? I don't want to go through the trouble of rebasing this if you don't like the approach, or are already working on it separately.

[gtaylor]

@LeoCavaille Should I try to get this rebased sometime this weekend, or were you wanting to take a shot at it?

[LeoCavaille]

Currently working on it yea @gtaylor, should have something pushed at the end of the day probably.

[gtaylor]

@LeoCavaille Rock on. @joeasaurus sent a good PR on my PR (which I merged), if you hadn't already seen it.

Thanks for your work on this!

[Nuxij]

Thanks guys :)

[gtaylor]

@LeoCavaille hate to pester, but how are we looking? We could really use this so we can start using Datadog more thoroughly.

[benmccann]

hey @gtaylor this was added to the 5.5.0 milestone, but would need a rebase

[irabinovitch]

@gtaylor We are prepping for our 5.5 release. Would you have sometime this week to help complete the rebase?

[gtaylor]

We are now so far removed from when the initial PR was sent that we've moved on quite a bit at work. I doubt I'll have time in the near future to rebase this.

[talwai]

@gtaylor I think we'll re-assign this to the 5.6.0 milestone to keep time expectations more reasonable. We'll go over it again and give you more detailed feedback on the approach before any rebasing.

cc @irabinovitch

[remh]

@gtaylor Thanks a lot for this. Sorry about the delays, we are going to close this PR as we are planning to ship a better replacement for dogstream. Feel free to reach out if you have any questions and thanks a lot for your contribution.

Remi