Doc: clarify the confusion between integration and functional tests

[rachedbenmustapha]

As a followup to scality/IronMan-Data#96 here is a proposed update to the testing guidelines, which were leaving room for interpretation for classifying tests as functional instead of integration.

[adrienverge]

I think there's a misformatting in this paragraph.

[rachedbenmustapha]

That's ugly, fixing this!

[LaurenSpiegel]

Can we simplify along the lines set forth in this last quora answer: https://www.quora.com/What-is-the-difference-between-unit-testing-functional-testing-and-integration-testing

It is confusing to think about unit tests and integration tests as totally different animals. If you look at them as along one spectrum it makes more sense (at least to me). Unit tests examine less code and integration tests examine more code/functions working together. Then functional testing is clearly a different animal testing whether a feature works from the perspective of a user.

[ghost]

Well, originally, (I notice it wasn't updated in the guidelines anyways),
we intended for tests to be:

1. unit: test a single piece of code (function/object)
2. functional: Test one isolated functionality in one isolated
   component/program
3. integration: Test the functionality of the program while interacting
   with other components to validate the feature of the project as a whole
   (project as in IronMan component: Data, Metadata, etc..)
4. load test: Test the component fully, under a stressing load of requests,
   to create an environment that might happen once in a while
5. performance test: Test the limits of the performance of the component,
   identifying bottlenecks, design issues, etc.

This is something that felt clear to at least a few of us (only naming
Vianney, Adrien and me, and I'm missing some).

I see that here, we have a reversed definition of functional and
integration. We really need to discuss that all together, to take a sound
and common decision that all can follow.

David Pineau

On Fri, Dec 4, 2015 at 7:51 PM, Lauren Spiegel notifications@github.com
wrote:

Can we simplify along the lines set forth in this last quora answer:
https://www.quora.com/What-is-the-difference-between-unit-testing-functional-testing-and-integration-testing

It is confusing to think about unit tests and integration tests as totally
different animals. If you look at them as along one spectrum it makes more
sense (at least to me). Unit tests examine less code and integration tests
examine more code/functions working together. Then functional testing is
clearly a different animal testing whether a feature works from the
perspective of a user.

—
Reply to this email directly or view it on GitHub
#19 (comment)
.

[ghost]

On the semantic side, I'm with you on the functional being just assessing functionnalites, and integration as an overarching tests with components plugged together.

But that's just semantics. We'll need every kind of tests, from unit to performance ones.

[ghost]

By the way, this is related to #17.

[adrienverge]

I'm OK with any definition, as long as we make a decision once and for all and use it consistently.

PS: It looks like Quora agrees with @rachedbenmustapha's definition...

[ghost]

Does PR #26 obsoletize this one ? If yes, please close.