Collection items not appearing in CMS colection listing

[darranmorris]

Describe the bug
Netlify CMS is not listing items in a collection. If I add a new file using the CMS, it appears in the listing. If I refresh the page, then the new item also disappears. The items appear fine on the built website. This site uses Bitbucket. I have a very similar config for another site which uses GitHub and the collection items display fine in the CMS.

To Reproduce

1. Select a collection.
2. Collection contains no items.
3. Add a new file and save it.
4. The item appears in the collection.
5. Refresh the browser window.
6. Collection contains no items.

Expected behavior
Collection should contain the list of all items in the collection folder.

Screenshots

Applicable Versions:

• Netlify CMS version: 2.9.7
• Git provider: BitBucket
• OS: Mac OS 10.14
• Browser version: Chrome 75

CMS configuration

backend:
  name: bitbucket
  repo: darranmorris/xxxxxxxx

media_folder: "source/images/posts/"
public_folder: "/images/posts/"

slug:
  encoding: "ascii"
  clean_accents: true
  sanitize_replacement: "-"
  
collections: # A list of collections the CMS should be able to edit
  - label: "Website"
    name: "website"
    files:
      - label: "Home Page"
        name: "home"
        file: "data/cms/home.yml"
        fields:
          - {label: "Hero Title", name: hero_title, widget: string, default: "AUTOMATE<br /> YOUR OIL & GAS<br /> CONTROL ROOM<br /> OPERATIONS"}
          - {label: "Hero Subtitle", name: hero_subtitle, widget: string, default: "Operate flow rate transitions and facility re-starts up to 40% faster."}
          - {label: "Hero CTA", name: hero_cta, widget: string, default: "Get in touch"}
          - {label: "Intro Title", name: introtitle, widget: string, default: "What is CruxOCM?"}
          - {label: "Intro Paragraph", name: intro_para, widget: markdown}
          - {label: "Intro Method Title", name: intro_method_title, widget: string}
          - {label: "Intro Method Paragraph", name: intro_method_para, widget: string}
          - {label: "Intro Block Pullout", name: intro_pullout, widget: string}
      - label: "About Page"
        name: "about"
        file: "data/cms/about.yml"
        fields:
          - {label: "Main Title", name: main_title, widget: string}
          - {label: "Main Intro", name: main_intro, widget: string}
          - {label: "Main Block Pullout", name: main_pullout, widget: string}
          - {label: "Partner Logos Title", name: logos_title, widget: string}
      - label: "Contact Page"
        name: "contact"
        file: "data/cms/contact.yml"
        fields:
          - {label: "Main Title", name: main_title, widget: string}
          - {label: "Main Title", name: main_title, widget: string}
  - name: news
    label: News
    folder: source/news/
    extension: .html.md
    format: frontmatter
    create: true
    slug: "{{year}}-{{month}}-{{day}}-{{title}}"
    fields:
      - {label: "Title", name: title, widget: string}
      - {label: "Date", name: date, widget: datetime}
      - {label: "Layout", name: layout,, widget: hidden, default: "blog-layout"}
      - {label: "Nav Item", name: nav_item,, widget: hidden, default: "news"}
      - {label: "Blog Image", name: image, widget: image, default: "https://cruxocm.netlify.com/images/posts/default.jpg"}
      - {label: "Body", name: body,, widget: markdown}

[erquhart]

Any errors in the console?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[neilkinnish]

I am seeing this issue, there are no errors listed @erquhart. The item is also not found in the cms search, but definitely exists.

Is the CMS using local storage to cache the list? I'm assuming if this is true it's not updating in some cases. I have a lot of items in my collections.

The order of items in collections also appears to be random? Is there anyway to set the order.

[barthc]

@neilkinnish can you share the repo?

[neilkinnish]

@barthc apologies, just realised my issue is different...

I'm running into the situation where I have two collections and both have over 1k items. Due to the GitHub greater api limit of 1k, new items are not listed or searchable (as the cms loads all items to then search it seems).

Attempting to use gitlab or bitbucket (which both have paged results) but running into api rate limits when running a search due to the way the cms loads all items to then search.

Do you know it these issues will be resolved?

[barthc]

@neilkinnish can't give you a time estimate when the limit issue will be fixed. I just assigned myself to the issue. You can follow up there for any further updates.

[erezrokah]

I'm putting back the stale label I removed due to @neilkinnish initial comment. If this issue is still relevant please let me know.

[aemonm]

hi - I'm having the same issue. I do get some errors in the network panel:
/.netlify/git/github/contents/.gitattributes?ts=1573531696037&ref=develop

returns 404:
{ "message": "Not Found", "documentation_url": "https://developer.github.com/v3/repos/contents/#get-contents" }

I'm losing all my collections as soon as I refresh. Assume this is a permissions issue related to git-gateway, though I've gone through the docs and tried the obvious fixes.

[barthc]

@aemonm can you share your repo ?

[aemonm]

@barthc

Sure: https://github.com/aemonm/fire-works

Just a further note, I tried to disable and then re-enable Identity in the Netlify console, but now its stuck on this:

[matteocargnelutti]

Having this exact same issue since 2.10.6: Some collections showing, but empty.
Forcing 2.10.5 circumvents the bug.

Checked the Network inspector: all the .netlify/git/github/git/trees/x requests are returning 200 and the payload they are expected to.

Thank you.

Orignal post

[erezrokah]

Hi @matteocargnelutti, as the original issue happens when using BitBucket and not git-gateway, do you mind opening a new issue with the relevant info (config.yml, network traffic, a reproduction repo would be amazing but I understand if you can't share it).

@aemonm the request to .gitattributes is expected to fail as it is a part of the mechanism the CMS uses to auto detect large media configuration with git-gateway (tries to check for existing large media configuration).

I'll try to reproduce the original issue and update here

[erezrokah]

Reproduces here.

The underlying issue is that we filter collection files based on the extension: .html.md field, but when getting each file extension we were retrieving just the md part.

@matteocargnelutti I stand corrected. I think what you're experiencing is related since in #3014 I aligned the all the backends to use the same utility method to filter files which made the GitHub backend use the same faulty method the BitBucket backend was using (sorry for the long explanation).

[matteocargnelutti]

@erezrokah Fantastic, thank you very much.
Do you still need me to open an issue, since you apparently figured it out: #3097 ?

Thanks,

[erezrokah]

No need for a new issue at the moment. It would be great if you can confirm that you're having this issue with collections that specify an extension, at least until we get new release published.

[matteocargnelutti]

No need for a new issue at the moment. It would be great if you can confirm that you're having this issue with collections that specify an extension, at least until we get new release published.

I can confirm that. It appears that only our .html.md-based collections are impacted 👍

PS: Netlify CMS is great and y'all doing a great job 🎉 .

[erezrokah]

Thanks @matteocargnelutti! This is now released in netlify-cms@2.10.8 or netlify-cms-app@2.11.5

[maciejmatu]

Thanks @maciejmatu! This is now released in netlify-cms@2.10.8 or netlify-cms-app@2.11.5

@erezrokah Not sure if you mentioned the right person, I didn't do anything here 😄

[ChenMorSays]

I'm having the exact same problem. Running on netlify-cms-app@2.11.5^.
Posts are created, displayed fine on the blog, but can't see them on the admin dashboard.

Thoughts?

---

Here are the contents of my config.yaml file:

backend:
  name: github
  repo: (org)/(repo)
  branch: "master"

media_folder: "static/assets"
public_folder: "assets"

collections:
  - name: "blog"
    label: "Blog"
    folder: "content/blog"
    create: true
    fields:
      - { name: "path", label: "Path" }
      - { name: "date", label: "Date", widget: "date" }
      - { name: "title", label: "Title" }
      - { name: "description", label: "Description" }
      - { name: "body", label: "Body", widget: "markdown" }