Strider 2.0 Brainstorming #667
Comments
|
One that I wont put up there, just because it's too controversial, is to rewrite the client-side with Ember.js. Especially with how angular is going and the spaghetti code that we have (even after browserify), it would really help bring in new contributors from the Ember community, since they'll be able to jump in very easily. |
|
Here's a good question: Would it be easier to start from scratch, with what we learned, and to write it from the ground up (feels like that would be better) or to refactor and hope that we don't introduce many new bugs? Since we have some fundamental changes that have been requested, and plenty of code that is just not up to par. |
|
Would certainly love to help where I can with UI/webapp tasks and any testing assistance. So I've got to ask about this Angular vs. Ember thing:
|
|
I'd support Ember over Angular. Especially in a large app like this, Ember's conventions get you a lot of benefits. |
|
I think that build configuration should be stored along with projects instead of the DB. So I'm suggesting to make Strider build git projects having strider.yml file without making any further configurations |
|
Please expound on this feature request. Are you saying they can have
|
|
Correct, I'm seeing a value in doing that if you're planning to add access for organizations. |
Javascript in HTML, which basically is the same thing that we as the web development community have been trying to minimize since the days of the dominance of PHP. Not only is this hard to debug, read, and keep dry, but it's also hard to extend and expand.
Ember is great for large applications that need to scale and grow. Making it an SPA is perfect for Strider, since it'll have a slightly longer load in the beginning, and then be lightning fast. Ember has nice conventions and really the best router, which is an absolute necessity for web apps. Ember Data, is amazing, and will totally fit the way strider is organized, providing nice abstractions. This will allow for making the backend a REST API that can serve multiple front ends. Also ember-cli and the build/addon system, which is super!
Both, since Angular doesn't portray any conventions that everyone should follow, it lends itself to being used poorly. |
|
@vsukhomlinov added that to must haves. |
|
@knownasilya Great answer! That gives some interesting food for thought on why Ember is possibly a better MVW framework for open source projects -> Conventions! It's also probably the best anti-Angular argument I've heard lately among all the FUD that seems to be trending lately :) I'll admit I've seen some lack of convention in Strider's plugin controller implementations. When working on the plugin template, I had to review a lot of plugins to try and extract what seemed like the ideal standard to build upon. The dependence on the plugin controller's $scope already having object's like A few debatable pros and cons of switching: big pro - Conventions. Great for getting all open source contributors working in better alignment. Could probably flesh this out more |
|
On another subject - I think community support for Strider needs improvement too, i.e. a better place than IRC to tell people where to direct questions. Possibly creating a forum or using Stack Overflow with a #strider tag to follow. I think in IRC sometimes no one is really around and questions get lost. There was a great discussion recently about improving the docs and linking directly to them from within the webapp ui - I think this is a great direction too |
|
What about using gitter? https://gitter.im/ along with an SO bot for IRC And yes, the wiki docs are a good idea! |
|
Stack overflow would be great. It's not something I've ever gotten into as On Tuesday, December 9, 2014, Ilya Radchenko notifications@github.com
Niall O'Higgins |
Right, @shaunc made some great suggestions, see here #208 (comment) Regarding client-side JS frameworks... I like the React stuff. I don't know anything about Ember. I like Meteor too. I also like Angular. cries |
|
:D I would order things React > Ember > Angular fwiw On Wed Dec 10 2014 at 12:21:16 PM Keyvan Fatehi notifications@github.com
|
|
React is only the V in MV*, so there's plenty missing, and it's just too low-level with an extra language (jsx) to learn. They do have nice ideas, and many of those have already landed in Ember behind feature flags. React and Angular are nice for custom brewed frameworks, like within a company, but not so nice for an opensource app where you're trying to get everyone on the same page. Meteor is a whole different story, but also has very loose conventions, and plenty of ways to get lost. Plus you have to buy into the whole stack, package management, server, client, db. |
|
I think was kidding with meteor w/ regards to Strider (although I use Meteor currently for rapid prototyping... kinda like how I still sometimes use ruby on rails) -- but I'm not kidding about the way it (and React) manage to give you two-way data binding without encouraging obtrusive javascript and cryptic new html tags like the way angular does it. You must admit, though, that angular is powerful and our plugin development has been made super easy as a result of how angular slots right into it. @bitwit makes some good points regarding that as well. In the case of plugins the solution is more docs for sure, not changing toolchains, not more coding. That said, I see React being just V as an advantage, not a disadvantage, and fitting of the super-modular style of strider. Maybe Ember is great but this has never been shown to me and I don't know how great the cost of transition is, I don't know how big it is and how modular it is. If it's a kitchen-sink framework (like Rails is when compared to Sinatra or Rack), then I'd wager the cost is high. I'd wager it's not very fitting of our modular style and that we'd be excluding more people from the project. I'd wager that the cost of sticking a jsx transform in our browserify and rewriting some (new) views in a React style is minimal in comparison to a switch to Ember or putting effort towards removal of existing Angular stuff. I know for a fact React will allow us to step-by-step move from Angular to React, over time. Can the same be said about Ember? Hell, even backbone seems safer / more realistic. I am not saying let's switch to React! I'm saying this: I'd rather focus on fixing, maintaining, documenting, cleaning up the current stuff than changing toolsets or technologies unless it really makes sense to do so. The one main pain point I heard was routing. The only client-side routing I know about in Strider was written by me really quickly because I got sick of clicking the tab I was working on ... we can totally write that properly, modularly, or find one that is well written and independent of these giant frameworks (angular, ember, etc). e.g. I think this might be good? https://github.com/RubaXa/Pilot As far as single-page apps, I think they're overrated and suck when added as an afterthought. Strider works well as a multi-page app and should be sticking to this (and very minimal javascript). That said, I am totally into the idea of creating a separate Strider Ember/SPA repository that consumes Strider as an API endpoint. In my opinion, this takes Strider in a good direction. |
|
Well said @keyvanfatehi, At the core, I think Strider's attractiveness is its ease of adoption. Being MEAN stack has lent really well to that. Having been someone who tried CI before and never understood where the benefit was given the time required to get it up and running, Strider was this amazing breath of fresh air that made it so simple. There was little argument NOT to do CI anymore. |
|
I'm busy porting my application from ember to ember-cli. I like Ember but it definitely requires that you "drink the cool-aid", so can see why it might not be the best choice here. I would think that an important consideration would be trying to decouple as much as possible, whatever the choice. I think its a great idea to split the client (or clients?) into another repo, and make interactions between components REST based as far as possible.... Maybe there should be a spec for how a client -- eg -- accepts a plugin. |
|
Getting back to the templating discussion, the usage of angular templating on top of swig is a big issue since they are using the same control structures. We should definitely pick either single page app approach or the clean express way. Or replace everything with something else haha. |
|
+1 for "single-page-app" in separated project (or projects) as that way you could connect various different clients or types of clients, all talking to a clear REST interface. |
|
And one more thing, I'd avoid using Bootstrap and Angular in plugins. That will extremely complicate upgrade of these libraries from both plugin maintainer and project owner perspective in case of any major API change (like Angular 2.0). It might be a part of an initiative to redesign plugins API |
|
@vsukhomlinov that is a good point. We would have to write an API that plugin authors would have to use, but at the same time keep it as flexible as it is now, and to prevent ugly plugins. This is complex enough that it would need it's own brainstorming thread 😄 ugly plugins = bad spaghetti code, bringing in libraries that are too fat, etc.. |
|
Lots of cool ideas here, I definitely like the direction of a Single Page/REST API style design.
|
I think we can allow the plugins to depend on styling, because we don't want the plugins to clash, or the authors to bring in bootstrap just for their plugin (overkill). The only issue is backwards compatibility, but that can be solved with a minimum/maximum semver version, e.g. "strider": {
"compatible": ">= 1.5"
} |
We are designing a plugin interface for our own application. We decided to go with a two-level structure: and “app” which contains several “plugins”. Perhaps that would be applicable here. A plugin means you are connecting to a particular interface. Sometimes you want to connect to more than one interface at once; other times you explicitly don’t want to provide one interface while plugging into another. With this structure, the “app” repo would contain a “strider.json” in the root that specifies where the app plugs in and any other configuration to use those plugs. |
|
How about just cleaning up and publishing a REST api for a start. Then various clients would work. |
|
👍 for React and their friends:
And for backend Meteor or Sails (which I like it more) and they have realtime features. |
|
I'm on your side @shaunc, that's priority number one. |
|
I'm also very pro API-first. The JS framework discussion will continue endlessly and the better Strider can support whatever you want to interface with sounds best to me. 2 new MV* JS frameworks were published by the time you finish reading this. On the subject of REST API features I would really like to see the job logs accessible via an endpoint. Maybe with a call that has a |
|
the parsing of live text streams for ANSI control characters is On Thursday, July 9, 2015, Kyle Newsome notifications@github.com wrote:
Niall O'Higgins |
|
Would love to see if there are better parsers, or if iojs has moved forward in this regard. Maybe even splitting out the phases some how so that the number of DOM elements isn't a big issue. |
|
Talks about rewriting certain aspects of Strider, just to arrive at the same point with a different technology, seem misplaced. Not being able to write a plugin for Strider, because you don't know Angular, is not a reason to rewrite Strider; it's a reason to learn Angular. If you want to rewrite part of an application, you should have a better reason than "It's written in X and I don't like X, we should use Y, because I like Y". Every framework sucks. If you don't think that the framework/tollkit/library/… you're using sucks, then you haven't been using it long enough. |
I agree completely, its hard to overstate this. If there's anything that practically every person in this thread shares, its knowledge on using REST.
Agreed. I only posted this because a "rewrite" was already on the table, seemingly, from the comments above mine. I would not suggest that Strider should begin refactoring to ease my personal pain, but if someone is going to rewrite it, then please consider not using Angular on the next round.
Again, I agree, mostly, and I tried to make that point (from a different angle). To reconcile my statements with yours, all we need to do is agree that HTML, CSS, and JS are, in themselves, neither X, nor Y, nor any other letter. Angular might be X, React is Y, and Polymer is Z.. and I think Strider should use "none of the above" and make a plugin system that uses the fundamental technologies that everyone already knows how to use. Also, my critique in this regard is limited to only the plugins. Strider can use a mixture of every client library out there for all I care, I just want to be able to write plugins without needing to add things to the already conflated "qualification requirements" we use to hire people. Strider plugins are, at their core, very simple devices.. and it seems inefficient to "invent" complexity by layering on alien technologies. Just do Strider is precisely on the right track when it emphasizes plugins as a way to extend the product without requiring direct effort by its core team. This is one of the golden ideas of our age, in some ways. However, that is, at minimum, weakened by opinionated libraries like Angular. I think its much worse than the minimum and if the Angular requirement can be removed, I expect Strider to take off to the levels of the other major CI/CD solutions on the market. I can at least be certain that we could (and would) share our ~10 or so unique Strider plugins with the world if we could ditch Angular. We only avoid sharing because we do not follow the community practices and guidelines. |
Share them anyway, and the community will come through, otherwise everyone misses out. |
|
Sure, we have them bundled as a single plugin.. the reason why will be a little more clear when you see it. As I said, we "circumvent" Strider in a particular way. I'll mirror the plugin to GitHub as soon as I can. |
|
Even though I might repeat what other people already mentioned: I think strider should increase its modularity. What I mean by this specifically is:
Additionally:
|
|
I agree about everything, but not sure about ProtoBuf. I feel like that would hinder community involvement since JSON is well understood but not so much for ProtoBuf. Although I totally see the benefits of using that format. |
|
The other concern is man-power. We don't have too many active contributors, but maybe a rewrite will solve that (developers love new things). |
|
I would love to do a combination of json api and ProtoBuf. I think it will make it simple to have a nice convention and have the speed of ProtoBuf. In the end any kind of client could consume, since implementations are being written for many clients for JSON API. If anyone is interesting in starting writing the API from scratch, let me know, and we can start a specific brain-storming issue (or a repo right off e.g. strider/api and discuss there). |
|
I've really enjoyed reading this thread! Thanks all! Ilya, I am very much for the idea of writing a new API from scratch. As an aside, I currently use Buildkite for CI and I love that by design it does very little. There is an agent (open source and cross platform) that you must install on your workers, and then point them at the orchestrator (buildkite's closed-source cloud software). For example I have two types of workers... A digital ocean amd64 instance tests, builds, deploys my web code. An ARM beaglebone tests, builds, and uploads binary artifacts of my 'embedded' code. As far as tools... I have been on a meteor stint for some time now. I have delivered several projects in meteor, and used it for a startup I'm currently working engaged in. However as my application grew more complex, the thing became slower and slower and the pub/sub stuff was getting in the way more often than helping. I scrapped it and rewrote... I used hapi and sequelize with a postgres adapter. It has been so nice and such a breath of fresh air. I really enjoy hapi's simplicity and I missed SQL's predictability, maintainability, and enforcement of documented integrity of the underlying schema. I have been using RabbitMQ (via CloudAMQP) and it's been just great for delivery work to downstream IoT devices. It's a good technology for delivering work. Redis and others are good too, but it was just interesting to me that I had been using Redis for pub sub so long (since I learned Rails) but that AMQP was what I was really after most of the time... I would keep that in my arsenal when designing a new CI... As for frontend, my post-meteor development has been pretty much using AngularJS with proper usage of angular-ui ui-router and then calling back to Hapi with What I do agree with very much is that the core should not include UI -- if a new design can pull this off well I think it could be very special. I can imagine having a plugin that provides a base UI for those that want it. That plugin could easily be forked and have features added to it or use a totally different frontend framework or whatever. That said, React seems quite interesting and I do believe (like we sorta have now in strider) in the idea of providing an easy way to generate plugin scaffolding. In it we can pre-bake some React boilerplate or Angular boilerplate, etc. Also, would these plugins run on the orchestrator or on the worker? Strider currently does not have a super clear separation between orchestrator and worker and I think that was both a strength and a weakness. It was a strength because you got up and running quickly, but it was a weakness because it was difficult to figure out how to set up strider to actually deliver jobs down, to say, a beaglebone to produce ARM builds. |
Sounds like a good plan. Write them as they come. I think that Yeoman is still a good option in this regard for writing something like that. It will allow for shared knowledge and a good platform for that exact use-case.
I've yet to use Hapi. What does it bring to the table that Express doesn't? Same in regards to sequelize vs knexjs (which is what I've used in the past). I agree on postgres, and if we need json storage for plugin data, that can be done with the json column. I know you like new things Keyvan 😉 Have you looked at rethinkdb (nosql)? It supports push notifications for an arbitrary query.
Plugins might have to be two faceted, a core portion and a drone portion. I can see some plugins implementing one or both, it would depend what the plugin needs to do I think. The drones can communicate back to core and report on what plugins they have, and those could be configured through core (or locally on the drone). Drone plugins could be simple node modules with a basic API, so that users can reuse the plethora of build/testing tools already out there. I think we can start on a rough design on how this new system should work, and lay down the non-negotiables, and we can talk technology stack after. |
|
I haven't looked at rethinkdb enough to comment on it. The main thing that I like so far about hapi versus express has been the way you define routes. Routes are just objects that get passed into the I also like the way 'auth strategies' work in Hapi -- it just makes sense to me. With express I vaguely remember doing some things with passportjs or something. On the other hand I might just be improving as a coder in general -- but I am pretty sure hapi has been a better toolset overall and it's not just me. I don't know anything about knexjs, but I can say that sequelize has been well-documented and easy to use. I like how it has a tool for generating migration files and running them, I've been using it in production and doing migrations with it without any issues. But anyway I agree with you about determining the non-negotiables, having an overall design we would like to achieve, and then looking at tools later. I'm pretty swamped with my current engagements but I will keep an eye on the new stuff :) |
|
Multiple build environments and Simultaneous builds are a must-have for devops. If it's as easy to test that a whole infra on a superior version of as it is to test with another version of a dependency then we have a sane test system imho. Simultaneous builds is a must for devops and blue green deployment too because devop projects (ie. codebases using saltstack, ansible, chef ...) would typically take time as the test would generally be "instanciate a set of containers, provision them (which is mostly self-tested) and then run functional tests". |
|
100% agree that we need true multiple build environments (this cannot be simply based on branches, it just doesn't work to tie the VCS into this). I'd like more info about what blue green deployment is @jpic if you don't mind |
|
Sure, is it good enough if I cite Fowler's article about BlueGreen ?
|
|
Guys, our 2c... Over at Panubo we are using Strider solely for deployments to CoreOS (Docker / Fleet) clusters. We chose Strider because it seems like a good lightweight, flexible tool that is focused on CD, rather than CI duties... To get Strider to handle CD tasks we've needed to write a bunch of different tools and even a Docker container to host it all. With all this we get an atomic (rolling) deployment capability. And our customers get visibility via a nice web UI. But right now it's tricky to configure as everything must be configured manually. We are using the GitHub / BitBucket plugins (depending on customer), Custom Script runner, and Meta data plugins along with our tools above. Some of our customers have over 100 projects. So we envisage that we might need to write our own plugin to make this easier. Given our usage, our wish list for Strider 2.0 would be:
Cheers, Andrew |
|
After manually running deployment scripts for far too long, I started looking into CI applications and decided on Strider, largely because it runs on Node and seems a lot less complex to set up than Jenkins or whatever. Since it's an easily-accessible portal to our private cloud, I've toyed with the idea of using it for more than just CI - like running database backups or running grunt tasks in arbitrary directories. That being said, I like @bitwit's suggestion of supporting more plugin types. If things were set up correctly, that functionality could be accomplished via plugin. I also think it makes a lot of sense to strip things down to just a REST API, and maybe have a separate repo for the "stock" UI, which, itself, could be a plugin orchestrator. Given @macropin's comments about continual deployment, and my personal goal of using this for things other than CI, I think it would be good to get away from the idea of "builds" and think about generic, nestable "tasks". (Something like this is also mentioned in #233) So from what exists now, the "deploy" task would contain a "test" task and a final task to actually push files and such. The "test" task would contain a task to copy code, install code, and run the tests. Plugins could then publish their own tasks that could be "dragged" into other tasks to add them to the workflow. Other plugins could consume these tasks as well, paving the way for a generic task scheduler, etc. Is this idea too crazy? |
No, I think this is really the way we want to go. Feel free to leave ideas in the |
|
definitely. the data model is structured around jobs not builds per se. at jobs can do more than just build and push code and can be triggered by On Wednesday, August 19, 2015, Ilya Radchenko notifications@github.com
Niall O'Higgins |
|
Hello @knownasilya ,Is the 2.0 development plan still going on? |
Just putting this out there, want to list out some of the "must have"s and "nice to have"s for 2.0.
Must Haves
Nice To Haves
Feel free to disagree, and contribute! Discussion welcome!
The text was updated successfully, but these errors were encountered: