background optimization opportunities

NOTE: This page is still being written

Optimization Opportunities

Observations

• Commonality in the needs of enterprise application projects is significantly higher than variability.
• Solutions to lots of common requirements are captured by well-established patterns and architectural approaches.
• The majority of implementation tasks are not unique, but rather repetitive, derivative, and mechanical.
• Numerous infrastructural frameworks and libraries exist
• Existing frameworks and libraries focus each on a specific layer or aspect, or cover only a subset of the overall architecture
• Projects have to combine their own technology stack out of multiple libraries and frameworks.
• The variety of technologies used in projects requires staffing the teams with people that have proper up-to-date expertise in many different areas.
• Certain architectures and services, such as in-memory data grid or runtime metric collection, are available as 3rd party products, or as part of PaaS offered by cloud vendors.
• Many projects attempt to capture recurring tasks and patterns into infrastructural frameworks of their own, each re-inventing the same wheels, as good as it is possible within the constraints of resources and time.

Problems

• There is no single off-the-shelf framework that covers architecture and implementation of typical enterprise applications through all layers and tiers from A to Z, while also minimizing or eliminating efforts invested in non-unique tasks. Let's name such framework a productivity platform.

• Existing infrastructural frameworks and libraries focus each on a specific layer or aspect, or cover only a subset of the architecture. Projects are left to combine their own technology stack out of multiple libraries and frameworks, each pair of which may or may not play well together.

• The variety of technologies used in applications requires the team to be staffed with professionals that have proper expertise in many different areas. Such requirement causes number of problems:

  • acquiring full-stack developers experienced with exactly the right set of technologies is hard
  • extending team with professionals in different technologies is expensive for both budget and timeframe - bear in mind that every additional team member doubles the coordination overhead
  • letting existing team members learn new technologies, while attractive to the team (or maybe not), imposes expensive learning curve and technical risks arising from improper use of a newly learnt technology
  • abandoning the use of a new technology for the above reasons, despite the fact that it is a perfect fit for the problem at hand, is bad, either.

• Application vendors and start-ups cannot afford home-grown productivity platform. They have to focus on delivering business value, or they risk running out of budget and time.

• Projects that proceed without a productivity platform, get into trouble later, as the system grows in size and complexity, the stakeholders' expectations grow even more, and lots of real users demand availability and reliability of operations. Such situation is usually handled in one of two ways (each one demanding great deal of additional investments):

  • re-engineering of the initial solution
  • expanding teams until there are enough people onboard to cope with ongoing issues.

• It is a deadlock situation, in which:

  • on one hand, for teams to sustain low costs and a high pace, applications must be based on a productivity platform.
  • on the other hand, application vendors and start-ups cannot afford (and should not attempt) development of their own productivity platform.
    • Improvised infrastructure layers that projects re-invent on their own, within very limiting constraints, don't do best in boosting team productivity and creating reliable operations, which can compromise project success.
  • thus, an off-the-shelf (preferably free and open sourced) productivity platform must exist, which teams can use in their projects right from the start. But such platform does not exist, at the moment of writing.

• With the absence of productivity platform, significant amounts of resources are invested in re-invention of wheels, as well as repetitive and mechanical tasks, resulting in high budgets, low stability, and long timeframes.

• Reaction to changes in requirements is slow, because projects drag the ballast of derivative mechanical tasks, which need to be re-done as part of change.

• Initial prototyping is far from instantaneous; it often takes considerable amount of time, before the business can try a prototype and give feedback. The later misconceptions are identified, the more rework is there to do.

• Beyond initial prototyping, it is often impossible to play with alternative domain models in order to discover one that fits best: the cost of exchanging one model for another is prohibitive, for reasons already mentioned.

• 3rd party products that support certain architectures and services, such as in-memory data grid and runtime metric collection, are often too expensive for lots of projects to purchase; such projects have to resort to less appropriate but cheaper solutions.

Solution

If an infrastructural ecosystem exists, which:

• Supplies ready A-to-Z architectural answers to common questions and concerns
• Supplies runtime platform that executes the architecture
• Provides high-productivity framework for implementation of reusable modules and unique features
• Provides highly adaptive implementations of common requirements and ready models for common domains
• Allows right people do the right job, so that code dealing with concrete technology stacks is contributed by experts in the corresponding technologies
• Minimizes developers' set of skills and areas of expertise required by the project
• Minimizes or eliminates developers' efforts invested in non-unique tasks
• Supports creation of free-style stunning user interface themes, native to their presentation platforms
• Deploys ready-to-use ALM/DevOps toolchain based on best known practices
• Is easily adaptable to requirements of a specific project
• Is built and maintained by an active and large open source community, which means that:
  • Many talents are involved
  • The code is massively reviewed and tested
  • Quality is not exchanged for marketing targets and deadlines
  • Development is driven by feedback and contribution

then:

• Significant amounts of work will be saved; resources and timeframes required by projects will drop dramatically
• The technical risks of the projects will decrease
• The deadlock situation of missing off-the-shelf framework will be resolved. Projects will be able to work both fast and right starting at day one.
• Reaction to changes will be quicker and easier.
• Prototyping will be almost instantaneous; experimentation with alternative models will require dramatically less work.
• The collective expertise of enterprise application development will begin to gather in one place in the form of shared code and reusable modules.