Worker builds binary in case the binary doesn't match the worker arch #200

Skarlso · 2019-07-25T20:43:55Z

@michelvocks Alright! This works, but, as you can see I had to export two of the none exported functions, NewBuild and GitPull. I can't use the Create one directly because that tries to store things in the db to which the worker doesn't have access to.

I'll try to refactor this, I just wanted to share with you the current progress.

…e pipeline repo.

codecov-io · 2019-07-25T20:49:13Z

Codecov Report

Merging #200 into master will increase coverage by 0.01%.
The diff coverage is 51.63%.

@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage   62.11%   62.13%   +0.01%     
==========================================
  Files          48       49       +1     
  Lines        3944     4046     +102     
==========================================
+ Hits         2450     2514      +64     
- Misses       1086     1117      +31     
- Partials      408      415       +7

Impacted Files	Coverage Δ
workers/agent/agent.go	`49.63% <40.44%> (+0.95%)`	⬆️
store/sha_pair.go	`63.15% <63.15%> (ø)`
workers/server/worker.go	`50% <68%> (+1.94%)`	⬆️
store/store.go	`74.28% <70%> (+6.78%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c0f920...5ca5149. Read the comment docs.

workers/agent/agent.go

workers/proto/worker.pb.go

workers/server/worker.go

Skarlso · 2019-07-26T20:56:34Z

@michelvocks So this works, however. On the previous iteration I wasn't saving the pipeline as a new create run. Now I am with create. On the previous iteration this lead to a nice thing where it didn't rebuild the pipeline again once the worker rebuild it for itself. Now it's always rebuilding and I'm not exactly sure why. Probably because I'm always creating a new CreatePipeline entry?

I also refactored a code a bit. ;)

go.sum

Skarlso · 2019-08-22T15:22:06Z

This f*cker runs for 44 seconds.

…is probably the exec error test.

Skarlso · 2019-08-25T21:06:04Z

@michelvocks Alright, this is as ready as it gets. I'll add some more coverage later on. Tell me if you like the SHAPair approach. :)

store/memdb/memdb.go

workers/agent/agent.go

Skarlso · 2019-09-03T08:10:26Z

Buuu

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x78 pc=0x10b10eb]

goroutine 246 [running]:
github.com/gaia-pipeline/gaia/handlers.(*mockStorageService).PipelineGetScheduled(0xc0002ec000, 0x32, 0xd65c103098ff4d62, 0x49b949fa3e89058a, 0x64e2d54e4c776eb3, 0x729f546e92990536, 0x542df71cb26c69b0)
	<autogenerated>:1 +0x6b
github.com/gaia-pipeline/gaia/workers/scheduler.(*Scheduler).schedule(0xc0000dca20)
	/go/src/github.com/gaia-pipeline/gaia/workers/scheduler/scheduler.go:234 +0xd1
github.com/gaia-pipeline/gaia/workers/scheduler.(*Scheduler).Init.func1(0xc0003ec870, 0xc0000dca20)
	/go/src/github.com/gaia-pipeline/gaia/workers/scheduler/scheduler.go:126 +0x4d
created by github.com/gaia-pipeline/gaia/workers/scheduler.(*Scheduler).Init
	/go/src/github.com/gaia-pipeline/gaia/workers/scheduler/scheduler.go:121 +0xdd
FAIL	github.com/gaia-pipeline/gaia/handlers	7.157s
Exited with code 1

:(

I'll try to take a look at this and fix it. It's bat that it can do that.

Skarlso · 2019-09-03T12:01:40Z

store/store.go

-		return err
+
+	// Make sure buckets exist
+	setP.update(userBucket)


@michelvocks Sorry, I had to refactor this in this batch. It was too repetitive. :) Besides, you touch it you own it. :D

😄 good job!

Skarlso · 2019-09-03T15:21:17Z

Testing

Vagrant: vagrant init ubuntu/bionic64.
vim Vagrantfile -> uncomment public_network.
vagrant up
git clone gaia
checkout the pr
get a gaia config which creates a worker that points to the host machines IP address
run it -- it should connect
---- works ----

create a pipeline and run that pipeline
worker should rebuild the pipeline and execute it successfully

run it again
Huh, nope.. SHA mismatch. -- Gotta check this. :/
Oh I know why. Because uniqueID is actually always Unique. :D I need pipelineID here. :)
worker should just run what it already has without rebuilding
YESS!!

2019-09-03T15:42:55.064Z [INFO ] Gaia: received work from Gaia primary instance...
2019-09-03T15:42:55.134Z [DEBUG] Gaia: record found for pipeline... comparing.
2019-09-03T15:42:55.878Z [DEBUG] plugin: starting plugin: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang args=[]
2019-09-03T15:42:55.878Z [DEBUG] plugin: waiting for RPC address: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:42:56.150Z [DEBUG] plugin: plugin process exited: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:42:58.032Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:42:58.033Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:42:58.549Z [DEBUG] plugin: starting plugin: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang args=[]
2019-09-03T15:42:58.549Z [DEBUG] plugin: waiting for RPC address: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:42:58.561Z [DEBUG] plugin.test-worker-rebuild-2_golang: 2019/09/03 15:42:58 CreateUser has been started!
2019-09-03T15:43:01.031Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:43:01.033Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...

Found the record and moved on the run the rebuilt binary.

Success

change the pipeline
worker should rebuild again since the original pipeline changed
Worked!
Stop the worker then re-send the same pipeline. It should still work.

^C2019-09-03T15:44:43.786Z [INFO ] Gaia: exit signal received. Exiting...
vagrant@ubuntu-bionic:~/goproject/gaia$ ./main
2019-09-03T15:44:46.029Z [DEBUG] Gaia: executeable path found: path=/home/vagrant/goproject/gaia
2019-09-03T15:44:49.033Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:44:49.036Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:44:52.034Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:44:52.035Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:44:55.034Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:44:55.035Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:44:58.034Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:44:58.036Z [INFO ] Gaia: received work from Gaia primary instance...
2019-09-03T15:44:58.112Z [DEBUG] Gaia: record found for pipeline... comparing.
2019-09-03T15:44:58.562Z [DEBUG] plugin: starting plugin: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang args=[]
2019-09-03T15:44:58.563Z [DEBUG] plugin: waiting for RPC address: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:44:58.833Z [DEBUG] plugin: plugin process exited: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:45:01.034Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:45:01.041Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:45:01.430Z [DEBUG] plugin: starting plugin: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang args=[]
2019-09-03T15:45:01.431Z [DEBUG] plugin: waiting for RPC address: path=/home/vagrant/goproject/gaia/pipelines/test-worker-rebuild-2_golang
2019-09-03T15:45:01.447Z [DEBUG] plugin.test-worker-rebuild-2_golang: 2019/09/03 15:45:01 CreateUser has been started!
2019-09-03T15:45:04.036Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:45:04.045Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...
2019-09-03T15:45:06.448Z [DEBUG] plugin.test-worker-rebuild-2_golang: 2019/09/03 15:45:06 CreateUser has been finished!
2019-09-03T15:45:06.450Z [DEBUG] plugin.test-worker-rebuild-2_golang: 2019/09/03 15:45:06 MigrateDB has been started!
2019-09-03T15:45:07.033Z [TRACE] Gaia: try to pull work from Gaia primary instance...
2019-09-03T15:45:07.034Z [TRACE] Gaia: got no work from Gaia primary instance. Will try it again after a while...

ALL TESTS PASSED

…stead of per pipeline.

michelvocks

LGTM ❤️

store/sha_pair.go

…mistaken for pipeline.UniqueID

codecov-commenter · 2024-10-14T23:54:26Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 51.63399% with 74 lines in your changes missing coverage. Please review.

Project coverage is 62.13%. Comparing base (7c0f920) to head (5ca5149).
Report is 55 commits behind head on master.

Files with missing lines	Patch %	Lines
workers/agent/agent.go	40.44%	43 Missing and 10 partials ⚠️
workers/server/worker.go	68.00%	5 Missing and 3 partials ⚠️
store/sha_pair.go	63.15%	4 Missing and 3 partials ⚠️
store/store.go	70.00%	3 Missing and 3 partials ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage   62.11%   62.13%   +0.01%     
==========================================
  Files          48       49       +1     
  Lines        3944     4046     +102     
==========================================
+ Hits         2450     2514      +64     
- Misses       1086     1117      +31     
- Partials      408      415       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Gergely Brautigam and others added 18 commits July 23, 2019 16:45

Initial try to build binary by worker.

265fb7d

A little bit of refactoring.

9557f27

Cleaning up the errorous binary.

a56c272

A different approach.

763462d

Creating an api which the worker can call to get information about th…

fc756b9

…e pipeline repo.

Skeleton of the api

0e241f5

Added more logic.

43fab8f

Using the name instead

f355109

Moar info

4bde3b2

Added correct error display.

8c346ad

RPC endpoint.

3ab6e20

Changed impl to gRPC

b19d052

Fixed wording and nil panic

0b67ca3

Using the ID instead.

5b22447

Cannot save pipeline run

52843de

GitPull the repo...

0045758

Handling error

73d8c18

It works!

23198f9

Skarlso added the Needs Work The PR still requires some work from the submitter. label Jul 25, 2019

Skarlso added 2 commits July 25, 2019 22:46

Agent test build failure.

d5acea1

linter

9d19530

michelvocks reviewed Jul 26, 2019

View reviewed changes

workers/agent/agent.go Outdated Show resolved Hide resolved

workers/agent/agent.go Outdated Show resolved Hide resolved

workers/agent/agent.go Outdated Show resolved Hide resolved

workers/proto/worker.pb.go Outdated Show resolved Hide resolved

workers/server/worker.go Show resolved Hide resolved

Skarlso added 2 commits July 26, 2019 22:05

Trying out createpipeline again.

7c5648b

Extracting rebuilding of binary.

c5ff190

Skarlso force-pushed the issues_194_worker_builds_binary branch from 05ceb45 to f491cca Compare July 26, 2019 20:54

Skarlso added the needs review label Jul 26, 2019

Skarlso commented Jul 26, 2019

View reviewed changes

go.sum Outdated Show resolved Hide resolved

Linting fix

0e76372

Skarlso and others added 2 commits August 19, 2019 22:21

Added a basic test case for the re-builder.

1130104

Added test for the pipeline rebuilder.

c9e1bdf

Gergely Brautigam and others added 5 commits August 22, 2019 23:48

Some more coverage.

820ff51

TODO: Fix whatever is adding an extra run to the update workflow. It …

3c31dfb

…is probably the exec error test.

Fixed the test which was causing problems.

178a795

Passed test.

f773964

Added moar coverage for the worker.

48ca7f6

Skarlso force-pushed the issues_194_worker_builds_binary branch from 1029308 to 48ca7f6 Compare August 25, 2019 20:01

Added coverage to sha pair methods.

47ea5c9

michelvocks reviewed Sep 2, 2019

View reviewed changes

store/memdb/memdb.go Outdated Show resolved Hide resolved

michelvocks reviewed Sep 2, 2019

View reviewed changes

workers/agent/agent.go Outdated Show resolved Hide resolved

michelvocks reviewed Sep 2, 2019

View reviewed changes

workers/agent/agent.go Outdated Show resolved Hide resolved

Merge branch 'master' into issues_194_worker_builds_binary

f2c29d2

Shifted the sha storing process into bolt db.

e4744b9

Skarlso commented Sep 3, 2019

View reviewed changes

Added tests and fixed bucket names.

e098b75

Skarlso force-pushed the issues_194_worker_builds_binary branch from ba5371f to e098b75 Compare September 3, 2019 12:40

Changed to PipelineID instead of UniqueID which was unique per run in…

ca090e3

…stead of per pipeline.

Skarlso added the Ready To Merge PR is ready to be merged into master label Sep 4, 2019

michelvocks approved these changes Sep 4, 2019

View reviewed changes

store/sha_pair.go Outdated Show resolved Hide resolved

store/sha_pair.go Outdated Show resolved Hide resolved

store/sha_pair.go Outdated Show resolved Hide resolved

Gergely Brautigam added 2 commits September 4, 2019 12:25

Fixed comments.

41b0432

Renamed uniqueID field to pipelineid so it does not get accidentally …

5ca5149

…mistaken for pipeline.UniqueID

Skarlso merged commit 32bf82c into gaia-pipeline:master Sep 4, 2019

Skarlso deleted the issues_194_worker_builds_binary branch September 4, 2019 10:55

Skarlso mentioned this pull request Sep 4, 2019

The worker requires a different binary architecture than what the builder sent it #194

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Worker builds binary in case the binary doesn't match the worker arch #200

Worker builds binary in case the binary doesn't match the worker arch #200

Skarlso commented Jul 25, 2019

codecov-io commented Jul 25, 2019 •

edited

Loading

Skarlso commented Jul 26, 2019

Skarlso commented Aug 22, 2019

Skarlso commented Aug 25, 2019

Skarlso commented Sep 3, 2019

Skarlso Sep 3, 2019 •

edited

Loading

michelvocks Sep 3, 2019

Skarlso commented Sep 3, 2019 •

edited

Loading

michelvocks left a comment

codecov-commenter commented Oct 14, 2024

Worker builds binary in case the binary doesn't match the worker arch #200

Worker builds binary in case the binary doesn't match the worker arch #200

Conversation

Skarlso commented Jul 25, 2019

codecov-io commented Jul 25, 2019 • edited Loading

Codecov Report

Skarlso commented Jul 26, 2019

Skarlso commented Aug 22, 2019

Skarlso commented Aug 25, 2019

Skarlso commented Sep 3, 2019

Skarlso Sep 3, 2019 • edited Loading

Choose a reason for hiding this comment

michelvocks Sep 3, 2019

Choose a reason for hiding this comment

Skarlso commented Sep 3, 2019 • edited Loading

Testing

ALL TESTS PASSED

michelvocks left a comment

Choose a reason for hiding this comment

codecov-commenter commented Oct 14, 2024

Codecov Report

codecov-io commented Jul 25, 2019 •

edited

Loading

Skarlso Sep 3, 2019 •

edited

Loading

Skarlso commented Sep 3, 2019 •

edited

Loading