Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New config framework for the different schedulers #913

Merged
merged 40 commits into from
Mar 13, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
029db7c
Add the config framework for the different schedulers
na-- Jan 29, 2019
15a4128
Fix minor issues linting and test issues
na-- Jan 30, 2019
4a1c941
Break apart the scheduler config type initializations
na-- Jan 31, 2019
b50ff18
Clean up some commented-out code
na-- Jan 31, 2019
4821c50
Update and test the new scheduler configurations
na-- Feb 25, 2019
99b0994
Merge branch 'master' into scheduler-config-wip
na-- Feb 25, 2019
ad3f6f4
Fix linter issues
na-- Feb 25, 2019
62c910c
Remove the debug output
na-- Feb 26, 2019
8113438
Clean up and add more tests
na-- Feb 26, 2019
ce1c967
Restore the Split() method
na-- Feb 26, 2019
2451ead
Add copyright notices
na-- Feb 28, 2019
507c5c5
Improve the execution config generation from shortcuts
na-- Mar 1, 2019
54eb412
Warn if the user specifies only the new execution option
na-- Mar 1, 2019
9cab261
Improve the scheduler config tests
na-- Mar 1, 2019
977b10e
Refactor some CLI configs and add some basic configuration tests
na-- Mar 1, 2019
d1943ec
Improve the readability of the config test cases
na-- Mar 1, 2019
effbf30
Work arround some strage appveyor environment variable issues
na-- Mar 5, 2019
2120a12
Fix minor issues with the structure of the new config tests
na-- Mar 5, 2019
2c1adf3
Move TestConfigConsolidation() and its helpers to a separate file
na-- Mar 5, 2019
62ff4aa
Fix a typo in a comment
na-- Mar 5, 2019
704dcc3
Fix the duplicate config file path CLI flag
na-- Mar 6, 2019
a7d0b6f
Preserve the trailing spaces in the k6 ASCII banner
na-- Mar 6, 2019
b86d5cd
Automatically create the config file's parent folder
na-- Mar 6, 2019
c8c902a
Add a missing copyright notice
na-- Mar 6, 2019
13e506d
Fix the newline before the banner
na-- Mar 7, 2019
5d8b354
Move the root CLI persistent flags to their own flagset
na-- Mar 7, 2019
588ef66
Fix an env.var/CLI flag conflict and issues with CLI usage messages
na-- Mar 7, 2019
e76972a
Improve the CLI flags test framework in the config consolidation test
na-- Mar 7, 2019
295abd2
Add support for testing the JSON config in the consolidation as well
na-- Mar 7, 2019
9eb4379
Improve the comments on the funcitons that deal with the file config
na-- Mar 7, 2019
2de0923
Add tests and fix a minor bug
na-- Mar 8, 2019
bf84bc0
Merge pull request #935 from loadimpact/config-painful-testing
na-- Mar 8, 2019
93e4eae
Merge branch 'master' into scheduler-config-wip
na-- Mar 8, 2019
227f22c
Merge branch 'master' into scheduler-config-wip
na-- Mar 11, 2019
079282a
Fix or suppress linter errors
na-- Mar 11, 2019
238e224
Use a custom error type for execution conflict errors in the config
na-- Mar 11, 2019
6d19e61
Improve config consolidation, default values and tests
na-- Mar 12, 2019
9404af6
Extend config validation to the cloud and archive subcommands
na-- Mar 12, 2019
2498121
Silence a linter warning... for now!
na-- Mar 12, 2019
d88be34
Override the execution setting when execution shortcuts are used
na-- Mar 13, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 60 additions & 30 deletions lib/scheduler/configmap.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,45 +3,75 @@ package scheduler
import (
"encoding/json"
"fmt"
"sync"
)

// ConfigMap can contain mixed scheduler config types
type ConfigMap map[string]Config

// ConfigConstructor is a simple function that returns a concrete Config instance
// with the specified name and all default values correctly initialized
type ConfigConstructor func(name string, rawJSON []byte) (Config, error)

var (
configTypesMutex sync.RWMutex
mstoykov marked this conversation as resolved.
Show resolved Hide resolved
configConstructors = make(map[string]ConfigConstructor)
mstoykov marked this conversation as resolved.
Show resolved Hide resolved
)

// RegisterConfigType adds the supplied ConfigConstructor as the constructor for its
// type in the configConstructors map, in a thread-safe manner
func RegisterConfigType(configType string, constructor ConfigConstructor) {
mstoykov marked this conversation as resolved.
Show resolved Hide resolved
configTypesMutex.Lock()
defer configTypesMutex.Unlock()

if constructor == nil {
panic("scheduler configs: constructor is nil")
}
if _, configTypeExists := configConstructors[configType]; configTypeExists {
panic("scheduler configs: RegisterConfigType called twice for " + configType)
}

configConstructors[configType] = constructor
}

// GetParsedConfig returns a struct instance corresponding to the supplied
// config type. It will be fully initialized - with both the default values of
// the type, as well as with whatever the user had specified in the JSON
func GetParsedConfig(name, configType string, rawJSON []byte) (result Config, err error) {
switch configType {
case constantLoopingVUsType:
config := NewConstantLoopingVUsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case variableLoopingVUsType:
config := NewVariableLoopingVUsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case sharedIterationsType:
config := NewSharedIterationsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case perVUIterationsType:
config := NewPerVUIterationsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case constantArrivalRateType:
config := NewConstantArrivalRateConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case variableArrivalRateType:
config := NewVariableArrivalRateConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
default:
configTypesMutex.Lock()
defer configTypesMutex.Unlock()

constructor, exists := configConstructors[configType]
if !exists {
return nil, fmt.Errorf("unknown execution scheduler type '%s'", configType)
}
return
}
return constructor(name, rawJSON)
/*
switch configType {
case constantLoopingVUsType:
config := NewConstantLoopingVUsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case variableLoopingVUsType:
config := NewVariableLoopingVUsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case sharedIterationsType:
config := NewSharedIterationsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case perVUIterationsType:
config := NewPerVUIterationsConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
case constantArrivalRateType:

// ConfigMap can contain mixed scheduler config types
type ConfigMap map[string]Config
case variableArrivalRateType:
config := NewVariableArrivalRateConfig(name)
err = json.Unmarshal(rawJSON, &config)
result = config
*/
}

// UnmarshalJSON implements the json.Unmarshaler interface in a two-step manner,
// creating the correct type of configs based on the `type` property.
Expand Down
9 changes: 9 additions & 0 deletions lib/scheduler/constant_arrival_rate.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package scheduler

import (
"encoding/json"
"fmt"
"time"

Expand All @@ -10,6 +11,14 @@ import (

const constantArrivalRateType = "constant-arrival-rate"

func init() {
RegisterConfigType(constantArrivalRateType, func(name string, rawJSON []byte) (Config, error) {
config := NewConstantArrivalRateConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// ConstantArrivalRateConfig stores config for the constant arrival-rate scheduler
type ConstantArrivalRateConfig struct {
BaseConfig
Expand Down
9 changes: 9 additions & 0 deletions lib/scheduler/constant_looping_vus.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package scheduler

import (
"encoding/json"
"fmt"
"time"

Expand All @@ -10,6 +11,14 @@ import (

const constantLoopingVUsType = "constant-looping-vus"

func init() {
RegisterConfigType(constantLoopingVUsType, func(name string, rawJSON []byte) (Config, error) {
config := NewConstantLoopingVUsConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// The minimum duration we'll allow users to schedule. This doesn't affect the stages
// configuration, where 0-duration virtual stages are allowed for instantaneous VU jumps
const minDuration = 1 * time.Second
Expand Down
9 changes: 9 additions & 0 deletions lib/scheduler/per_vu_iterations.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package scheduler

import (
"encoding/json"
"fmt"
"time"

Expand All @@ -10,6 +11,14 @@ import (

const perVUIterationsType = "per-vu-iterations"

func init() {
RegisterConfigType(perVUIterationsType, func(name string, rawJSON []byte) (Config, error) {
config := NewPerVUIterationsConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// PerVUIteationsConfig stores the number of VUs iterations, as well as maxDuration settings
type PerVUIteationsConfig struct {
BaseConfig
Expand Down
9 changes: 9 additions & 0 deletions lib/scheduler/shared_iterations.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package scheduler

import (
"encoding/json"
"fmt"
"time"

Expand All @@ -10,6 +11,14 @@ import (

const sharedIterationsType = "shared-iterations"

func init() {
RegisterConfigType(sharedIterationsType, func(name string, rawJSON []byte) (Config, error) {
config := NewSharedIterationsConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// SharedIteationsConfig stores the number of VUs iterations, as well as maxDuration settings
type SharedIteationsConfig struct {
BaseConfig
Expand Down
8 changes: 8 additions & 0 deletions lib/scheduler/variable_arrival_rate.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ import (

const variableArrivalRateType = "variable-arrival-rate"

func init() {
RegisterConfigType(variableArrivalRateType, func(name string, rawJSON []byte) (Config, error) {
config := NewVariableArrivalRateConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// VariableArrivalRateConfig stores config for the variable arrival-rate scheduler
type VariableArrivalRateConfig struct {
BaseConfig
Expand Down
9 changes: 9 additions & 0 deletions lib/scheduler/variable_looping_vus.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package scheduler

import (
"encoding/json"
"fmt"

"github.com/loadimpact/k6/lib/types"
Expand All @@ -9,6 +10,14 @@ import (

const variableLoopingVUsType = "variable-looping-vus"

func init() {
RegisterConfigType(variableLoopingVUsType, func(name string, rawJSON []byte) (Config, error) {
config := NewVariableLoopingVUsConfig(name)
err := json.Unmarshal(rawJSON, &config)
return config, err
})
}

// Stage contains
type Stage struct {
Duration types.NullDuration `json:"duration"`
Expand Down