Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposal and docs for changes to snapshot layout #43

Merged
merged 1 commit into from
Aug 16, 2017
Merged

Proposal and docs for changes to snapshot layout #43

merged 1 commit into from
Aug 16, 2017

Conversation

kensimon
Copy link

@kensimon kensimon commented Aug 9, 2017

This functions as both a proposal for how the snapshot tarballs should
be structured, as well as documentation for that structure.

Places with a "NOTE:" section are places where changes are proposed.

@kensimon kensimon force-pushed the snapshot-docs branch 2 times, most recently from d767ff3 to 22c16af Compare August 9, 2017 16:58
@jesscodez
Copy link
Contributor

@kensimon I have some screenshots that I think makes this info easier to digest, e.g. see below for /plugins structure:

screen shot 2017-08-08 at 4 41 48 pm

Once the structure is nailed down, let me know if you think these are helpful to incorporate. I can push them onto this branch if that's easiest.

^also, will we be unzipping nested tarballs?

@kensimon
Copy link
Author

kensimon commented Aug 9, 2017

Sure, screenshots would be awesome. Up to you if you want to put them before or after we merge... I think so long as it's clear the document is a work-in-progress we can merge early and often on this one.

The tar-within-tar problem will be fixed in a separate PR, I think we can update the documentation then, although I don't think the docs will have to change. Reason being: I think the fix for plugins that send tarfiles will be to implicitly extract them before putting them in the main tarball... that way, for someone consuming a sonobuoy snapshot, they'll never see a nested tarball in the first place (thus it's probably fine that the docs don't mention them.)

@jesscodez
Copy link
Contributor

Makes sense! (re the tar-within-tar)
Screenshots added, feel free to edit.

@kensimon
Copy link
Author

kensimon commented Aug 9, 2017

@timothysc if you're happy with this, feel free to approve (or just merge), I think we can iterate more if there's feedback. I have another PR that implements these changes too, but I'll wait for us to have a consensus on this first.

@kensimon kensimon requested a review from timothysc August 9, 2017 18:31
@timothysc
Copy link
Contributor

timothysc commented Aug 9, 2017 •
edited
Loading

@kensimon Let's give this some time for other folks and I'll send out an email too.

@timothysc timothysc added this to the v0.9.0 milestone Aug 9, 2017
timothysc
timothysc reviewed Aug 16, 2017
View reviewed changes
docs/snapshot.md
The `/resources` directory contains JSON-serialized Kubernetes objects, taken from querying the Kubernetes REST API. The directory has the following structure:

- `/resources/ns/<namespace>/<type>.json` - For all resources that belong to a namespace, where `<namespace>` is the namespace of that resource (eg. `kube-system`), and `<type>` is the type of resource, pluralized (eg. `Pods`).
- `/resources/cluster/<type>.json` - For all resources that don't belong to a namespace, where `<type>` is the type of resource, pluralized (eg. `Nodes`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your image shows non-ns vs. cluster

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

timothysc
timothysc reviewed Aug 16, 2017
View reviewed changes
docs/snapshot.md
The `/meta` directory contains metadata about this Sonobuoy run, including configuration and query runtime.

- `/meta/query-time.json` - Contains metadata about how long each query took, example: `{"queryobj":"Pods","time":12.345ms"}`
> NOTE: this file is currently distributed throughout the tarball in `/resources/.../results.json`. Proposal is to move it here, and create other `.json` files under `/meta` if we start capturing other things than query time.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have to see if this is problematic or not.

timothysc
timothysc reviewed Aug 16, 2017
View reviewed changes
docs/snapshot.md

### /serverversion.json

> NOTE: this is currently `/serverversion/serverversion.json`, proposal here is to just make it `/serverversion.json`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not cluster ?

timothysc
timothysc approved these changes Aug 16, 2017
View reviewed changes
Copy link
Contributor

@timothysc timothysc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm , we may need to revise as we implement b/c it may not be worth the effort to shuffle some things around, we'll have to see.

@timothysc
Copy link
Contributor

@kensimon or @abiogenesis-now feel free to rebase then merge.

Proposal and docs for changes to snapshot layout
d9c0681 
This functions as both a proposal for how the snapshot tarballs should
be structured, as well as documentation for that structure.

Places with a "NOTE:" section are places where changes are proposed.

Signed-off-by: Ken Simon <ninkendo@gmail.com>
@jesscodez jesscodez merged commit cb12d4e into vmware-tanzu:master Aug 16, 2017
@jesscodez
Copy link
Contributor

FYI added the snapshot file to the docs table-of-contents

@jesscodez jesscodez mentioned this pull request Aug 16, 2017
Closed
kensimon pushed a commit to kensimon/sonobuoy that referenced this pull request Aug 18, 2017
Initial stab at changing snapshot layout
5797188 
This shifts some of the layout based on the doc changes in vmware-tanzu#43.

Also adds some docs explaining the auto-extraction of plugin tar files

Signed-off-by: Ken Simon <ninkendo@gmail.com>
@kensimon kensimon mentioned this pull request Aug 18, 2017
Merged
kensimon pushed a commit to kensimon/sonobuoy that referenced this pull request Aug 21, 2017
Initial stab at changing snapshot layout
cd92061 
This shifts some of the layout based on the doc changes in vmware-tanzu#43. Also
adds some docs explaining the auto-extraction of plugin tar files

Changes:

- Pod logs moved to `/podlogs` inside the tarball
- Query times moved to `/meta/query-time.json` instead of dispersed
  throughout `/resources/**/results.json`
- `/resources/non-ns` is now `/resources/cluster`
- `/config.json` is now `/meta/config.json`

Signed-off-by: Ken Simon <ninkendo@gmail.com>
kensimon pushed a commit to kensimon/sonobuoy that referenced this pull request Aug 24, 2017
Initial stab at changing snapshot layout
cc81b23 
This shifts some of the layout based on the doc changes in vmware-tanzu#43. Also
adds some docs explaining the auto-extraction of plugin tar files

Changes:

- Pod logs moved to `/podlogs` inside the tarball
- Query times moved to `/meta/query-time.json` instead of dispersed
  throughout `/resources/**/results.json`
- `/resources/non-ns` is now `/resources/cluster`
- `/config.json` is now `/meta/config.json`

Signed-off-by: Ken Simon <ninkendo@gmail.com>
jhamilton1 pushed a commit to jhamilton1/sonobuoy that referenced this pull request Jun 27, 2018
Merge pull request vmware-tanzu#43 from kensimon/snapshot-docs
74ea73b 
Proposal and docs for changes to snapshot layout
Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com
jhamilton1 pushed a commit to jhamilton1/sonobuoy that referenced this pull request Jun 27, 2018
Initial stab at changing snapshot layout
cf5857f 
This shifts some of the layout based on the doc changes in vmware-tanzu#43. Also
adds some docs explaining the auto-extraction of plugin tar files

Changes:

- Pod logs moved to `/podlogs` inside the tarball
- Query times moved to `/meta/query-time.json` instead of dispersed
  throughout `/resources/**/results.json`
- `/resources/non-ns` is now `/resources/cluster`
- `/config.json` is now `/meta/config.json`

Signed-off-by: Ken Simon <ninkendo@gmail.com>

Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com
jhamilton1 pushed a commit to jhamilton1/sonobuoy that referenced this pull request Jun 27, 2018
Merge pull request vmware-tanzu#43 from kensimon/snapshot-docs
7aef87e 
Proposal and docs for changes to snapshot layout
Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com

Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com
jhamilton1 pushed a commit to jhamilton1/sonobuoy that referenced this pull request Jun 27, 2018
Initial stab at changing snapshot layout
3c8feeb 
This shifts some of the layout based on the doc changes in vmware-tanzu#43. Also
adds some docs explaining the auto-extraction of plugin tar files

Changes:

- Pod logs moved to `/podlogs` inside the tarball
- Query times moved to `/meta/query-time.json` instead of dispersed
  throughout `/resources/**/results.json`
- `/resources/non-ns` is now `/resources/cluster`
- `/config.json` is now `/meta/config.json`

Signed-off-by: Ken Simon <ninkendo@gmail.com>

Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com

Signed-off-by: Jesse Hamilton jesse.hamilton@heptio.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jesscodez jesscodez jesscodez left review comments

@timothysc timothysc timothysc approved these changes

Assignees
No one assigned
Labels
area/documentation Related to project documentation legacy/question p1-important
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kensimon @jesscodez @timothysc