Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamic Routing #9601

Closed
15 tasks done
ErisDS opened this issue Apr 24, 2018 · 5 comments
Closed
15 tasks done

Dynamic Routing #9601

ErisDS opened this issue Apr 24, 2018 · 5 comments
Assignees
@kirrg001
Labels
feature [triage] New features we're planning or working on server / core Issues relating to the server or core of Ghost

Comments

@ErisDS
Copy link
Member

ErisDS commented Apr 24, 2018 •
edited by kirrg001
Loading

Refs: #9528 (YAML Settings), #9192 (Routing re-write), #9550 (Url Service).

The what and why of dynamic routing

Ghost's Routing has always been almost entirely static and hard-coded. The only config we've offered is the option to add dates to permalinks, but that setting is probably my least favourite part of Ghost 😂

People close to the project will have heard a lot about a mysterious feature called "channels", which was an attempt to expose more configuration around lists of posts. This has been working behind the scenes, but we haven't been clear on how to expose the functionality: Apps? Config? UI?

Early this year we sat down and rethought routing from the ground up with a view to answering the following questions: How can we make it possible to solve all the common routing use cases? How do we expose functionality to our users? How does this fit in with Apps? And where the fuck do Channels fit into all of this?

Top 3 common use cases we want to solve:

  1. custom home pages
  2. rendering a taxonomy at a non-standard URL, e.g. /tag/recipes/ -> /category/recipes/
  3. customising post collections e.g. removing posts with a certain tag, or using a different URL

There are 3 main problems we have with trying to reach these use cases:

  • How to make routes enumerable and configurable
  • Ability to generate sitemaps and other features that need to know about URLs rather than routes
  • Resource <-> URL 2 way binding - get resource from URL, get URL for resource

We think we have the answers, however the theory is never quite the same as the practice. This issue represents our current thinking at the time of writing and is very subject to change!

What's coming?

In #9528 we added the ability to customise Ghost through YAML files that live in content/settings. The first file that will live here will be routes.yaml, and this will be the gateway to all things dynamic routing. This solve the configuration problem.

The solution to everything else is the introduction of a URL service and a huge rewrite of all the code that generates content for Ghost blogs 😅

Beta Goals

In the first beta version, by editing content/settings/routes.yml it will be possible to:

  • Define a custom route that renders a specific template
  • Define a custom home page
  • Change the location of the standard collection of posts, e.g. so they all live at /blog/*.
  • Change the URL for taxonomies e.g /tag/:slug/ -> /category/:slug/
  • Customise the main post collection (v1 has severe known limitations)
  • Define new collections (v1 has severe known limitations)

Once the first beta goes out we will be looking for people to test their routing use-cases. At that point, we'll publish some documentation on how the default routes.yml file is structured and how to define custom routes and collections within the limitations.

Tracked tasks

Overall Goals

This issue will be closed when we achieve the following:

Note: There are some extensions to the dynamic routing behaviour that are known but not covered by this issue, which include custom controllers and custom taxonomies

💡 Please direct all questions & discussion to the forum. This issue will be used for collecting bugs & use cases once the beta goes out.
@ErisDS ErisDS added the feature [triage] New features we're planning or working on label Apr 24, 2018
@kirrg001 kirrg001 mentioned this issue Apr 24, 2018
Merged
17 tasks
This was referenced Apr 24, 2018
Closed
Closed
This was referenced Apr 24, 2018
Merged
Closed
Merged
Closed
Closed
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Apr 25, 2018
@kirrg001
Dynamic Routing
f982123 
refs TryGhost#9601

[ci skip]
kirrg001 added a commit that referenced this issue Apr 25, 2018
@kirrg001
Fixed wrong default filter for default collection
2a1be37 
refs #9601

- while i was testing different collections and different filters, i somehow thought that the default
  collection does not contain featured posts 😀 🙊
- this is wrong (!!!!)
- the url service is not yet connected
- so: this is not a bug
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Apr 25, 2018
@kirrg001
Use Bookshelf in our test env
efde5e4 
refs TryGhost#9601

- replace raw knex queries by Bookshelf
- optimise lot's of tests, so we don't experience a massive slow down in the test run
- this has troubled in the past e.g. with normalisation, any custom model logic
- there are for sure thousends things which can be optimised now, but because of time, we do them step by step
- this is especially important for the url service, because we have to ensure that inserting/updating/removing resources will trigger model events
kirrg001 added a commit that referenced this issue Apr 25, 2018
@kirrg001
Use Bookshelf in our test env (#9592)
e23fd51 
refs #9601

- replace raw knex queries by Bookshelf queries
- optimise lot's of test setups, so we don't experience a massive slow down in the test run
- this has troubled in the past e.g. with normalisation, any custom model logic - the test env always had to simulate things
- there are for sure thousands things which can be optimised now, but because of time, we do them step by step
- this is especially important for the url service (#9601), because we have to ensure that inserting/updating/removing resources will trigger model events

`grunt test-all` with SQLite finishes in 2,5-3min. (on master: 1-2min) 
`grunt test-all` with MySQL finishes in 4min. (on master: 3min)

**NOTE: We want to move as much as possible routing and integration tests to unit tests. This will speed up the test run again.** See #9342. But we need to find time for that. Any help is welcome!
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Apr 25, 2018
@kirrg001
Rewrite sitemaps
2731c4d 
refs TryGhost#9601

- listen on url events
- do not fetch resources
- the urlservice is the only component who knows which urls are valid
- ES6 adaptions
- keep caching logic -> only regenerate xml if there is a change
- update tests
- checked test coverage (100%)
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Apr 25, 2018
@kirrg001
Rewrite sitemaps
c2566a3 
refs TryGhost#9601

- listen on url events
- do not fetch resources
- the urlservice is the only component who knows which urls are valid
- ES6 adaptions
- keep caching logic -> only regenerate xml if there is a change
- update tests
- checked test coverage (100%)
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Apr 26, 2018
@kirrg001
Re-work usage of Url utility
4b1df3b 
refs TryGhost#9601

- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
  - only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink
- adapt url utility
- adapt tests
kirrg001 added a commit that referenced this issue Aug 15, 2018
@kirrg001
🎨 Dynamic Routing Beta: Enabled redirect by default when using data…
0b7412f 
… key

refs #9601

- when using the short form `data: tag.welcome` the redirect is enabled by default
  - /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
  e.g. data:
	tag:
	  resource: tags
          type: read
          slug: welcome
          redirect: false
kirrg001 added a commit that referenced this issue Aug 15, 2018
@kirrg001
Fixed express router reference after routes.yaml refresh
f209894 
refs #9601, refs #9744

- the express router reference wasn't updated fully
- the stack was the old router stack when you have uploaded a new routes.yaml file
  - this has caused e.g. that new redirects for channels/collections didn't work after the upload
kirrg001 added a commit that referenced this issue Aug 15, 2018
@kirrg001
🐛 Dynamic Routing Beta: Fixed wrong template order for static routes
0a2d0da 
refs #9601
@kirrg001 kirrg001 mentioned this issue Aug 16, 2018
Merged
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Aug 16, 2018
@kirrg001
🐛 Dynamic Routing Beta: Shortform author.foo is not allowed
87db84d 
refs TryGhost#9601

- otherwise you get access to {{author}} in the theme, which is deprecated
- we define both "user" and "author" alias for now, because taxonomies use the author alias to create it's data query object
- i don't want to refactor this now
- we only disallow `data: author.foo` in the yaml validator (usage)
- internal usage is allowed
- add a validation error when resolving short to long form
ErisDS pushed a commit that referenced this issue Aug 16, 2018
@kirrg001 @ErisDS
🐛 Dynamic Routing Beta: Disallow using author resource and author…
816e911 
… data key name (#9790)

* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed

refs #9601

- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
kirrg001 pushed a commit that referenced this issue Aug 16, 2018
@ErisDS @kirrg001
✨Added advanced filtering to Dynamic Routing (#9757)
290f74f 
refs #9601, refs #9742

- Upgraded NQL to 0.1.0
- The new version of NQL supports aliases e.g. `tag: tags.slug`, which makes it possible to define `filter=tag:support`
- Furthermore, this allows us to support advanced filtering like tag:[a,b]
- In dynamic routing, we use mingo via NQL which has a slightly different feature set to GQL in the API:
   - AND NOT, OR and other advanced logic combos DO work on joined tables
   - Counts are not yet supported
- The Dynamic Routing beta docs should describe that API filtering and Dynamic Routing filtering is different
kirrg001 added a commit that referenced this issue Aug 16, 2018
@kirrg001
🎨 Dynamic Routing Beta: Enabled redirect by default when using data…
17833a6 
… key

refs #9601

- when using the short form `data: tag.welcome` the redirect is enabled by default
  - /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
  e.g. data:
	tag:
	  resource: tags
          type: read
          slug: welcome
          redirect: false
kirrg001 added a commit that referenced this issue Aug 16, 2018
@kirrg001
Fixed express router reference after routes.yaml refresh
0a4f34c 
refs #9601, refs #9744

- the express router reference wasn't updated fully
- the stack was the old router stack when you have uploaded a new routes.yaml file
  - this has caused e.g. that new redirects for channels/collections didn't work after the upload
kirrg001 added a commit that referenced this issue Aug 16, 2018
@kirrg001
🐛 Dynamic Routing Beta: Fixed wrong template order for static routes
f49073a 
refs #9601
kirrg001 added a commit that referenced this issue Aug 16, 2018
@kirrg001
🐛 Dynamic Routing Beta: Disallow using author resource and author…
cd690ef 
… data key name (#9790)

* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed

refs #9601

- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
kirrg001 added a commit to kirrg001/Ghost that referenced this issue Oct 18, 2018
@kirrg001
Changed how we determine if a resource is owned by an url generator
f28bd71 
refs TryGhost#9601

- api v2 returns absolute urls
- api v0.1 returns relative urls
- it's easier if we compare by id
kirrg001 added a commit that referenced this issue Oct 18, 2018
@kirrg001
Changed how we determine if a resource is owned by an url generator
3f9d0f5 
refs #9601

- api v2 returns absolute urls
- api v0.1 returns relative urls
- it's easier if we compare by id
@kirrg001 kirrg001 mentioned this issue Oct 22, 2018
Closed
@kirrg001 kirrg001 added the server / core Issues relating to the server or core of Ghost label Jan 2, 2019
This was referenced Jan 2, 2019
Closed
Closed
@kirrg001
Copy link
Contributor

kirrg001 commented Jan 2, 2019

Dynamic Routing Beta was shipped in Ghost 1.24.0. I am closing this issue for now. I extracted two sub features, which we aren't able to achieve yet.

Any further improvement or bug should be tracked separately 👻

@kirrg001 kirrg001 closed this as completed Jan 2, 2019
daniellockyer pushed a commit that referenced this issue Jul 26, 2022
@kirrg001
Dynamic Routing: Added migration for routes.yaml file (#9692)
7d9e2a2 
refs #9601

- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
  - the backwards compatible behaviour was different
  - only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kirrg001 kirrg001

Labels
feature [triage] New features we're planning or working on server / core Issues relating to the server or core of Ghost
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ErisDS @kirrg001 @vikaspotluri123 @pascalandy @fernandops26