Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation about how to set up your own continuous integration server #904

Closed
gmarziou opened this issue Dec 24, 2014 · 21 comments
Closed

Comments

@gmarziou
Copy link
Contributor

Setting up your own continuous integration server using Jenkins for a JHipster 2 application is harder than for a classic Spring MVC application because of all tools required to build and test the client code.

In fact, for those who can it's probably a good idea to think about hosted continuous integration like TravisCI as the JHipster team does.

I am currently going through this process and I would like to contribute some how-to documentation if you think it is useful and maybe some other users may have some tips to share.

It's harder because you have to manage 3 software stacks in your build process:

  • Java/Maven for the server code and the build orchestration
  • Javascript/NodeJS/Grunt/Bower for client code
  • Ruby/Compass to pre-process CSS style sheets

Each stack comes with its own dependency management (artifacts, npm, gem) with potential conflicts to solve.

CSS pre-processing is optional though, but it could have been simpler by choosing LESS as it requires only NodeJS which is already a requirement of JHipster. Anyway, maybe in the future this dependecy on Ruby will go away by using node-sass which is based on libsass a C/C++ implementation but that would mean more work users (especially on Windows) as no pre-compiled binary is available.

I have started by setting up Jenkins on a Linux CentOS server without display and have faced issue #416 due to headless server. I have installed Xvfb and Firefox but Karma is still failing the same even though I modified karma.conf.js. I can give details if someone would like to help.

Finally, I have setup Jenkins on a Windows 2008 server and got a sucessful build with prod profile.
I can document instructions.

I had to change 2 things in karma.conf.js:

  • singleRun: true otherwise grunt test server stays alive after end of build and blocks next build.
  • added karma-junit-reporter plugin so that Jenkins can present results of Karma tests but I need to dig further this

This makes me think that there should be a different karma.conf.js for prod/CI and for dev, or maybe the test server on port 9001 should be configured as singleRun.

A minor thing, the Jenkins log console contains non printable characters probably used for color output:

    [4mRunning "ngtemplates:dist" (ngtemplates) task�[24m
@jdubois
Copy link
Member

jdubois commented Dec 25, 2014

I agree with you:

  • This is important
  • This is complex to set up (I've done it myself at my company)

Can you contribute to the documentation on https://github.com/jhipster/jhipster.github.io ? It is done with Jekyll.
You could add a new page "Setting up CI" under "documentation", and send a PR?

@gmarziou
Copy link
Contributor Author

OK I will, I have already one PR pending about client code documentation.

@telefunken
Copy link

I have a Dockerfile that sets up a Jenkins container with dependencies for building projects generated by jhipster (including Karma-tests). Let me know if it is of interest.

@gmarziou
Copy link
Contributor Author

@telefunken Yes I would really like to see your Docker file.

@telefunken
Copy link

Here's the Dockerfile and instructions: https://github.com/telefunken/jenkins-jhipster-docker

I've used this image on a couple of servers with jobs using gradle wrapper. I've also successfully tested it by pointing a job to https://github.com/jhipster/jhipster-sample-app-gradle.

I hope it's of use. Let e know if you want an example of my Jenkins settings or job configuration examples.

There are a couple of steps until the setup is fully automatic. See README.md.

Troubleshooting if running docker on osx:
You need to create the local folder (/var/jenkins_home) inside boot2docker host, since that is the actual parent host (or use a volume container).

npm caused "FATAL ERROR: Evacuation Allocation failed - process out of memory" first time. The reason is that boot2docker is by default allocated with only 2gb mem. Run: >VBoxManage modifyvm boot2docker-vm --memory 3000.

@jmirc jmirc added this to the 2.0.0 milestone Dec 29, 2014
@jdubois
Copy link
Member

jdubois commented Dec 30, 2014

@jmirc this is documentation, we don't need to put in the v2.0.0 milestone, this can be done afterwards

@jdubois jdubois removed this from the 2.0.0 milestone Dec 30, 2014
@gmarziou
Copy link
Contributor Author

Thanks @telefunken, I will look at it

@gmarziou
Copy link
Contributor Author

gmarziou commented Jan 5, 2015

@telefunken I used your Dockerfile, I had an issue with dbus dependency when running the "apt-get upgrade" which seems similar with this report.

As a workaround I commented out this line and finally successfully built my image but unfortunately the Jenkins job fails with same error as I had on my physical server "Fatal error: Cannot read property 'x-ua-compatible' of null" known as issue #416 and due to headless server.

It failed the same on 2 different Docker environments: Linux CentOS and Windows/boot2docker, maybe it works for you because you are on OSX, I may give it a try.

@PeterEltgroth
Copy link

@gmarziou what version of node where you using?

I had the "Fatal error: Cannot read property 'x-ua-compatible' of null" issue on MacOS when using node 0.11.14, but reverted to 0.11.13 (per reflux/refluxjs#90) and it went away. I had also attempted to upgrade my version of karma (since karma-runner/karma#1182 had been merged in); however, that did not work for me. I probably missed something as I'm fairly new to using node.

Here's the error I recieved with 0.11.14 when running './gradlew test':

ERROR [karma]: [TypeError: Cannot read property 'x-ua-compatible' of null]
TypeError: Cannot read property 'x-ua-compatible' of null
at getXUACompatibleMetaElement (.../node_modules/karma/lib/middleware/karma.js:40:19)
at .../node_modules/karma/lib/middleware/karma.js:145:45
at .../node_modules/karma/lib/middleware/common.js:57:35
at fs.js:295:14
at.../node_modules/karma/node_modules/graceful-fs/graceful-fs.js:104:5
at Object.oncomplete (fs.js:93:15)

@gmarziou
Copy link
Contributor Author

gmarziou commented Jan 6, 2015

@PeterEltgroth many thanks, you found the root cause.

I was using NodeJS 0.11.14, I downgraded to 0.11.13 and now my build passes on my physical server or @telefunken Docker container

@gmarziou
Copy link
Contributor Author

gmarziou commented Jan 6, 2015

This explains also why it was working on my Windows server as the NodeJS installer uses the current stable version: 0.10.35.
I don't know why I initially selected 0.11 in Jenkins, I'll recommend to use 0.10 in the coming documentation.

@MartinLichtblau
Copy link
Contributor

Just an idea on the topic (somehow)...
We could setup the generator-jhipster on Travis so that it auto-builds and deploys different kinds of sample-apps, like with compass/without, java8/7, authO/cookie, cache/no-cache/distributed, DbTypes. Auto deploy it all on e.g. heroku and we can see if it's all fully working.
I think this would it make easier to find errors from start to end (the generated app running in production).

@gmarziou
Copy link
Contributor Author

Sorry, my second pull request about setting up CI got appended to my first one about client code doc.

@gmarziou
Copy link
Contributor Author

For those who would like to review what I have documented before it gets merged, look at setting_up_ci_*.md files in https://github.com/gmarziou/jhipster.github.io/tree/v2.0

@jdubois
Copy link
Member

jdubois commented Jan 18, 2015

Great work @gmarziou !

Some comments:

  • The end of the setting_up_ci_windows.md file is in French, you need to translate it to English
  • For the Karma.js tests in the JUnit format -> shouldn't this be the default? I'd rather have this by default in the generator, and remove the documentation, what's your opinion?

@gmarziou
Copy link
Contributor Author

  • Sorry for the French part, it should be OK now.
  • I agree with you about Karma tests, changing gruntfile.js and package.json should be safe but I'm not sure about the 'singleRun: true' in karma.conf.js, normally it should not be required as it is already set in Gruntfile.js. I can't remember clearly, I'll try again tomorrow and let you know. I'd also like to run clean builds in both dev and jenkins .

@jdubois
Copy link
Member

jdubois commented Jan 20, 2015

@gmarziou OK, for the Karma tests it's cool if you can have a look at it, but anyway this is a great improvement over the existing documentation, can you submit a PR?

@gmarziou
Copy link
Contributor Author

@jdubois just to clarify, you mean a PR for Karma test configuration as the PR for the doc is merged within the one about client code doc, right? I'll do it.

@jdubois
Copy link
Member

jdubois commented Jan 20, 2015

Yes, if you can do both PR that's great, otherwise the documentation PR is already very good alone!

@jdubois
Copy link
Member

jdubois commented Jan 25, 2015

Hi, can you send the PR about the documentation?

@gmarziou
Copy link
Contributor Author

There's only one PR: jhipster/jhipster.github.io#21 which includes several commits about CI and client code structure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants