Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(UI): application data is now loaded in UI through the new GetAppDetails endpoint #2020

Merged
merged 90 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
b7d991a
channels
miguel-crespo-fdc Oct 10, 2024
66a3b82
PRogress on new endpoint
miguel-crespo-fdc Oct 11, 2024
f0f465d
Adapt rollout service for new app details
miguel-crespo-fdc Oct 14, 2024
aedf591
lint
miguel-crespo-fdc Oct 14, 2024
b57c420
linter
miguel-crespo-fdc Oct 14, 2024
2936020
Merge branch 'main' into mc/new-endpoint-rollout
miguel-crespo-fdc Oct 14, 2024
8bed2c5
remove repo metadata
miguel-crespo-fdc Oct 14, 2024
337e7e8
Merge branch 'mc/new-endpoint-rollout' of github.com:freiheit-com/kub…
miguel-crespo-fdc Oct 14, 2024
c8bbcac
team
miguel-crespo-fdc Oct 14, 2024
9ea51a6
final touches
miguel-crespo-fdc Oct 14, 2024
a42bc74
self PR review
miguel-crespo-fdc Oct 14, 2024
5f2773e
self review PR
miguel-crespo-fdc Oct 14, 2024
d68f2c9
reduce noise
miguel-crespo-fdc Oct 14, 2024
f8ccc9e
changed apps
miguel-crespo-fdc Oct 14, 2024
e40c48e
reducing noise
miguel-crespo-fdc Oct 14, 2024
cfad517
noise
miguel-crespo-fdc Oct 14, 2024
4a65847
Merge branch 'main' into mc/new-endpoint-rollout
miguel-crespo-fdc Oct 14, 2024
b6bb4bf
feat: new endpoint to get app details
sven-urbanski-freiheit-com Oct 2, 2024
60451a9
New endpoint
miguel-crespo-fdc Oct 3, 2024
c2acea9
small change to test
miguel-crespo-fdc Oct 3, 2024
f2d5c9b
Unit tests
miguel-crespo-fdc Oct 4, 2024
6b65d49
cleanup
miguel-crespo-fdc Oct 4, 2024
fe9402c
commit_deployments
miguel-crespo-fdc Oct 4, 2024
511b37a
rollout tests
miguel-crespo-fdc Oct 4, 2024
2f4d5db
redo sql, needs to get back locks by env, as group locks would be mer…
miguel-crespo-fdc Oct 7, 2024
bdd0727
start of ui changes
miguel-crespo-fdc Oct 3, 2024
6512478
docker compose
miguel-crespo-fdc Oct 3, 2024
453320f
no update
miguel-crespo-fdc Oct 8, 2024
42bfaa0
better no update
miguel-crespo-fdc Oct 8, 2024
a13fbba
Frontend implementation, missing tests
miguel-crespo-fdc Oct 10, 2024
88dc38d
restore files
miguel-crespo-fdc Oct 10, 2024
c7c5fb1
service lane tests
miguel-crespo-fdc Oct 10, 2024
cadee1a
trying to fix tests
miguel-crespo-fdc Oct 10, 2024
65aaed9
compiles
miguel-crespo-fdc Oct 14, 2024
4a0fdbe
Merge branch 'main' into mc/new-endpoint-rollout
miguel-crespo-fdc Oct 15, 2024
7e38be7
fixing UI tests
miguel-crespo-fdc Oct 15, 2024
f2a5b51
fixed tests
miguel-crespo-fdc Oct 15, 2024
d537166
pnpm
miguel-crespo-fdc Oct 15, 2024
0198524
warnings are compiling
miguel-crespo-fdc Oct 15, 2024
fb0f89c
Warnings are sort-of reactive
miguel-crespo-fdc Oct 16, 2024
754a387
Warnings are now reactive
miguel-crespo-fdc Oct 16, 2024
d25b806
style
miguel-crespo-fdc Oct 16, 2024
49be624
Merge branch 'main' into mc/app-details-ui
miguel-crespo-fdc Oct 16, 2024
808f58b
PR Concerns
miguel-crespo-fdc Oct 16, 2024
f68eca1
Merge branch 'mc/new-endpoint-rollout' of github.com:freiheit-com/kub…
miguel-crespo-fdc Oct 16, 2024
167a0fa
Flakeyness
miguel-crespo-fdc Oct 16, 2024
fa5d972
spinner margin
miguel-crespo-fdc Oct 16, 2024
d388041
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 16, 2024
7309fcf
Merge branch 'main' into mc/new-endpoint-rollout
miguel-crespo-fdc Oct 16, 2024
d7a3698
revert
miguel-crespo-fdc Oct 16, 2024
14b80f7
Merge branch 'mc/new-endpoint-rollout' of github.com:freiheit-com/kub…
miguel-crespo-fdc Oct 16, 2024
e9946da
more overview tests
miguel-crespo-fdc Oct 16, 2024
237571f
more flakeyness
miguel-crespo-fdc Oct 16, 2024
d187236
Merge branch 'main' into mc/new-endpoint-rollout
miguel-crespo-fdc Oct 16, 2024
6356ae3
Merge branch 'main' into mc/app-details-ui
miguel-crespo-fdc Oct 16, 2024
66123ac
Fixed broken tests
miguel-crespo-fdc Oct 16, 2024
e74c19b
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 16, 2024
2b4f0c0
linter and flakeyness
miguel-crespo-fdc Oct 16, 2024
eb38285
removing top level app
miguel-crespo-fdc Oct 16, 2024
ddfd0a9
PR concerns from Hannes
miguel-crespo-fdc Oct 16, 2024
989b26e
merge with origin
miguel-crespo-fdc Oct 16, 2024
4b3d49b
rename lightweight apps fields
miguel-crespo-fdc Oct 17, 2024
24e54fb
Merge branch 'main' into mc/app-details-ui
miguel-crespo-fdc Oct 17, 2024
3177198
restore applications
miguel-crespo-fdc Oct 17, 2024
0666ebb
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 17, 2024
3f47200
remove unecessary changes
miguel-crespo-fdc Oct 17, 2024
179cd87
revert faulty merge
miguel-crespo-fdc Oct 17, 2024
bff3332
more reverts
miguel-crespo-fdc Oct 17, 2024
dd0c11e
final reverts
miguel-crespo-fdc Oct 17, 2024
715050d
lint
miguel-crespo-fdc Oct 17, 2024
f76e545
overview
miguel-crespo-fdc Oct 17, 2024
3e8396a
merge with main
miguel-crespo-fdc Oct 17, 2024
61ba666
remove noise
miguel-crespo-fdc Oct 17, 2024
d0b37b7
grpc call success
miguel-crespo-fdc Oct 17, 2024
79d46bb
noise
miguel-crespo-fdc Oct 17, 2024
4581f1c
Update services/frontend-service/src/ui/components/ServiceLane/Servic…
miguel-crespo-fdc Oct 21, 2024
5adedfc
PR concerns
miguel-crespo-fdc Oct 21, 2024
55bf369
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 21, 2024
f73ee6d
test
miguel-crespo-fdc Oct 21, 2024
59038c6
imports
miguel-crespo-fdc Oct 21, 2024
7695127
prettier
miguel-crespo-fdc Oct 21, 2024
8d06ed4
Merge branch 'main' into mc/app-details-ui
miguel-crespo-fdc Oct 21, 2024
0d5bf7c
sort on server side
miguel-crespo-fdc Oct 21, 2024
0f8566f
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 21, 2024
2e5d3c5
Sort in backend
miguel-crespo-fdc Oct 21, 2024
8c60456
Merge branch 'main' into mc/app-details-ui
miguel-crespo-fdc Oct 21, 2024
eef9cfc
Update pkg/api/v1/api.proto
miguel-crespo-fdc Oct 21, 2024
08763e2
flakey
miguel-crespo-fdc Oct 21, 2024
3bb186b
Merge branch 'mc/app-details-ui' of github.com:freiheit-com/kuberpult…
miguel-crespo-fdc Oct 21, 2024
d980ab0
restore test
miguel-crespo-fdc Oct 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions pkg/api/v1/api.proto
Original file line number Diff line number Diff line change
Expand Up @@ -434,12 +434,19 @@ message GetOverviewRequest {
string git_revision = 1;
}

//Lightweight version of application. Only contains name, team and warnings
miguel-crespo-fdc marked this conversation as resolved.
Show resolved Hide resolved
message OverviewApplication {
string name = 1;
string team = 2;
}

message GetOverviewResponse {
map<string, Application> applications = 2;
repeated EnvironmentGroup environment_groups = 3;
string git_revision = 4;
string branch = 5;
string manifest_repo_url = 6;
repeated OverviewApplication lightweight_apps = 7;
}

message EnvironmentGroup {
Expand Down
1 change: 1 addition & 0 deletions pkg/db/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -5634,6 +5634,7 @@ func (h *DBHandler) ReadLatestOverviewCache(ctx context.Context, transaction *sq
Branch: "",
ManifestRepoUrl: "",
Applications: map[string]*api.Application{},
LightweightApps: []*api.OverviewApplication{},
EnvironmentGroups: []*api.EnvironmentGroup{},
GitRevision: "",
}
Expand Down
12 changes: 11 additions & 1 deletion services/cd-service/pkg/repository/repository.go
Original file line number Diff line number Diff line change
Expand Up @@ -2466,6 +2466,16 @@ func (s *State) DBInsertApplicationWithOverview(ctx context.Context, transaction
}
}
}
if shouldDelete {
lApps := make([]*api.OverviewApplication, len(cache.LightweightApps)-1)

for _, curr := range cache.LightweightApps {
if curr.Name != appName {
lApps = append(lApps, curr)
}
}
cache.LightweightApps = lApps
}

err = h.WriteOverviewCache(ctx, transaction, cache)
if err != nil {
Expand Down Expand Up @@ -2528,7 +2538,6 @@ func (s *State) UpdateTopLevelAppInOverview(ctx context.Context, transaction *sq
}
rels = retrievedReleasesOfApp
}

if releasesInDb, err := s.GetApplicationReleasesDB(ctx, transaction, appName, rels); err != nil {
return err
} else {
Expand Down Expand Up @@ -2556,6 +2565,7 @@ func (s *State) UpdateTopLevelAppInOverview(ctx context.Context, transaction *sq
result.Applications = map[string]*api.Application{}
}
result.Applications[appName] = &app
result.LightweightApps = append(result.LightweightApps, &api.OverviewApplication{Name: appName, Team: app.Team})
return nil
}

Expand Down
19 changes: 19 additions & 0 deletions services/cd-service/pkg/repository/repository_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2188,6 +2188,12 @@ func TestUpdateOverviewCache(t *testing.T) {
Warnings: nil,
},
},
LightweightApps: []*api.OverviewApplication{
{
Name: "app1",
Team: "",
},
},
EnvironmentGroups: []*api.EnvironmentGroup{},
},
},
Expand Down Expand Up @@ -2228,6 +2234,12 @@ func TestUpdateOverviewCache(t *testing.T) {
Warnings: nil,
},
},
LightweightApps: []*api.OverviewApplication{
{
Name: "app1",
Team: "",
},
},
EnvironmentGroups: []*api.EnvironmentGroup{
{
EnvironmentGroupName: "dev",
Expand Down Expand Up @@ -2311,6 +2323,12 @@ func TestUpdateOverviewCache(t *testing.T) {
Priority: 0,
},
},
LightweightApps: []*api.OverviewApplication{
{
Name: "app1",
Team: "",
},
},
GitRevision: "123",
Branch: "main",
ManifestRepoUrl: "https://example.com",
Expand All @@ -2337,6 +2355,7 @@ func TestUpdateOverviewCache(t *testing.T) {
GitRevision: "123",
Branch: "main",
ManifestRepoUrl: "https://example.com",
LightweightApps: []*api.OverviewApplication{},
},
},
}
Expand Down
6 changes: 6 additions & 0 deletions services/cd-service/pkg/repository/transformer_db_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1734,6 +1734,12 @@ func TestCreateEnvironmentUpdatesOverview(t *testing.T) {
},
},
},
LightweightApps: []*api.OverviewApplication{
{
Name: "app",
Team: "",
},
},
EnvironmentGroups: []*api.EnvironmentGroup{
&api.EnvironmentGroup{
EnvironmentGroupName: "development",
Expand Down
123 changes: 90 additions & 33 deletions services/cd-service/pkg/service/overview.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,24 +21,21 @@ import (
"database/sql"
"errors"
"fmt"
"github.com/freiheit-com/kuberpult/pkg/mapper"
"google.golang.org/protobuf/types/known/timestamppb"
"os"
"sync"
"sync/atomic"

api "github.com/freiheit-com/kuberpult/pkg/api/v1"
"github.com/freiheit-com/kuberpult/pkg/db"
"github.com/freiheit-com/kuberpult/pkg/grpc"
"github.com/freiheit-com/kuberpult/pkg/logger"
"go.uber.org/zap"

"github.com/freiheit-com/kuberpult/pkg/mapper"
"github.com/freiheit-com/kuberpult/services/cd-service/pkg/notify"
"github.com/freiheit-com/kuberpult/services/cd-service/pkg/repository"
git "github.com/libgit2/git2go/v34"
"go.uber.org/zap"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"

api "github.com/freiheit-com/kuberpult/pkg/api/v1"
"github.com/freiheit-com/kuberpult/pkg/db"
"github.com/freiheit-com/kuberpult/services/cd-service/pkg/notify"
"github.com/freiheit-com/kuberpult/services/cd-service/pkg/repository"
"google.golang.org/protobuf/types/known/timestamppb"
"os"
"sync"
"sync/atomic"
)

type OverviewServiceServer struct {
Expand Down Expand Up @@ -142,9 +139,6 @@ func (o *OverviewServiceServer) GetAppDetails(
}
envGroups := mapper.MapEnvironmentsToGroups(envConfigs)

result.UndeploySummary = deriveUndeploySummary(appName, envGroups)
result.Warnings = db.CalculateWarnings(ctx, appName, envGroups)

// App Locks
appLocks, err := o.DBHandler.DBSelectAllActiveAppLocksForApp(ctx, transaction, appName)
if err != nil {
Expand Down Expand Up @@ -217,14 +211,19 @@ func (o *OverviewServiceServer) GetAppDetails(
}
response.Deployments[envName] = deployment
}
result.UndeploySummary = deriveUndeploySummary(appName, response.Deployments)
miguel-crespo-fdc marked this conversation as resolved.
Show resolved Hide resolved
warnings, err := CalculateWarnings(ctx, transaction, o.Repository.State(), appName, envGroups)
if err != nil {
return nil, err
}
result.Warnings = warnings
return result, nil
})
if err != nil {
return nil, err
}
response.Application = resultApp
return response, nil

}

func (o *OverviewServiceServer) GetOverview(
Expand Down Expand Up @@ -302,6 +301,7 @@ func (o *OverviewServiceServer) getOverview(
Applications: map[string]*api.Application{},
EnvironmentGroups: []*api.EnvironmentGroup{},
GitRevision: rev,
LightweightApps: make([]*api.OverviewApplication, 0),
}
result.ManifestRepoUrl = o.RepositoryConfig.URL
result.Branch = o.RepositoryConfig.Branch
Expand Down Expand Up @@ -453,24 +453,14 @@ func (o *OverviewServiceServer) update(s *repository.State) {
o.notify.Notify()
}

func deriveUndeploySummary(appName string, groups []*api.EnvironmentGroup) api.UndeploySummary {
func deriveUndeploySummary(appName string, deployments map[string]*api.Deployment) api.UndeploySummary {
var allNormal = true
var allUndeploy = true
for _, group := range groups {
for _, environment := range group.Environments {
var app, exists = environment.Applications[appName]
if !exists {
continue
}
if app.Version == 0 {
// if the app exists but nothing is deployed, we ignore this
continue
}
if app.UndeployVersion {
allNormal = false
} else {
allUndeploy = false
}
for _, currentDeployment := range deployments {
if currentDeployment.UndeployVersion {
allNormal = false
} else {
allUndeploy = false
}
}
if allUndeploy {
Expand All @@ -481,3 +471,70 @@ func deriveUndeploySummary(appName string, groups []*api.EnvironmentGroup) api.U
}
return api.UndeploySummary_MIXED
}

func CalculateWarnings(ctx context.Context, transaction *sql.Tx, state *repository.State, appName string, groups []*api.EnvironmentGroup) ([]*api.Warning, error) {
result := make([]*api.Warning, 0)
for e := 0; e < len(groups); e++ {
group := groups[e]
for i := 0; i < len(groups[e].Environments); i++ {
env := group.Environments[i]
if env.Config.Upstream == nil || env.Config.Upstream.Environment == nil {
// if the env has no upstream, there's nothing to warn about
continue
}
upstreamEnvName := env.Config.GetUpstream().Environment
if upstreamEnvName == nil {
// this is already checked on startup and therefore shouldn't happen here
continue
}

versionInEnv, err := state.GetEnvironmentApplicationVersion(ctx, transaction, env.Name, appName)
if err != nil {
return nil, err
}

if versionInEnv == nil {
// appName is not deployed here, ignore it
continue
}

versionInUpstreamEnv, err := state.GetEnvironmentApplicationVersion(ctx, transaction, *upstreamEnvName, appName)
if err != nil {
return nil, err
}
if versionInUpstreamEnv == nil {
// appName is not deployed upstream... that's unusual!
var warning = api.Warning{
WarningType: &api.Warning_UpstreamNotDeployed{
UpstreamNotDeployed: &api.UpstreamNotDeployed{
UpstreamEnvironment: *upstreamEnvName,
ThisVersion: *versionInEnv,
ThisEnvironment: env.Name,
},
},
}
result = append(result, &warning)
continue
}

appLocks, err := state.GetEnvironmentApplicationLocks(ctx, transaction, env.Name, appName)
if err != nil {
return nil, err
}
if *versionInEnv > *versionInUpstreamEnv && len(appLocks) == 0 {
var warning = api.Warning{
WarningType: &api.Warning_UnusualDeploymentOrder{
UnusualDeploymentOrder: &api.UnusualDeploymentOrder{
UpstreamVersion: *versionInUpstreamEnv,
UpstreamEnvironment: *upstreamEnvName,
ThisVersion: *versionInEnv,
ThisEnvironment: env.Name,
},
},
}
result = append(result, &warning)
}
}
}
return result, nil
}
6 changes: 6 additions & 0 deletions services/cd-service/pkg/service/overview_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,12 @@ func TestOverviewService(t *testing.T) {
Team: "team-123",
},
},
LightweightApps: []*api.OverviewApplication{
{
Name: "test",
Team: "team-123",
},
},
GitRevision: "0",
},
Setup: []repository.Transformer{
Expand Down
12 changes: 6 additions & 6 deletions services/frontend-service/pkg/cmd/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -650,18 +650,18 @@ func (p *GrpcProxy) GetFailedEsls(
return p.EslServiceClient.GetFailedEsls(ctx, in)
}

func (p *GrpcProxy) GetAppDetails(
ctx context.Context,
in *api.GetAppDetailsRequest) (*api.GetAppDetailsResponse, error) {
return p.OverviewClient.GetAppDetails(ctx, in)
}

func (p *GrpcProxy) GetOverview(
ctx context.Context,
in *api.GetOverviewRequest) (*api.GetOverviewResponse, error) {
return p.OverviewClient.GetOverview(ctx, in)
}

func (p *GrpcProxy) GetAppDetails(
ctx context.Context,
in *api.GetAppDetailsRequest) (*api.GetAppDetailsResponse, error) {
return p.OverviewClient.GetAppDetails(ctx, in)
}

func (p *GrpcProxy) GetGitTags(
ctx context.Context,
in *api.GetGitTagsRequest) (*api.GetGitTagsResponse, error) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ func TestHandleCommitDeployments(t *testing.T) {
inputTail: "123456/",
failGrpcCall: false,
expectedStatusCode: http.StatusOK,
expectedResponse: "{\"deploymentStatus\":{\"app1\":{\"deploymentStatus\":{\"dev\":\"DEPLOYED\",\"prod\":\"UNKNOWN\",\"stage\":\"PENDING\"}}}}\n",
expectedResponse: "{\"deploymentStatus\":{\"app1\":{\"deploymentStatus\":{\"dev\":\"DEPLOYED\", \"prod\":\"UNKNOWN\", \"stage\":\"PENDING\"}}}}\n",
miguel-crespo-fdc marked this conversation as resolved.
Show resolved Hide resolved
},
}
for _, tc := range tcs {
Expand Down
2 changes: 2 additions & 0 deletions services/frontend-service/src/ui/App/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
FlushRolloutStatus,
PanicOverview,
showSnackbarWarn,
updateAppDetails,
UpdateFrontendConfig,
UpdateOverview,
UpdateRolloutStatus,
Expand Down Expand Up @@ -99,6 +100,7 @@ export const App: React.FC = () => {
UpdateOverview.set(result);
UpdateOverview.set({ loaded: true });
PanicOverview.set({ error: '' });
updateAppDetails.set({});
},
(error) => {
PanicOverview.set({ error: JSON.stringify({ msg: 'error in streamoverview', error }) });
Expand Down
Loading
Loading