-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BEP Proposal: Atlas specification #1281
Comments
We currently aim to organize a meeting to further discuss the atlas-related |
yo, count me in |
Hi @bids-standard/maintainers, as we aim to share our draft with the community to receive and integrate feedback soon, we would like to ask if it would be possible to move our BEP to the next stage. More specifically, this refers to step 5 of the BEP development guide, ie making the BEP official and obtaining a number. We followed and conducted steps 1-3 and would now like to access if the requirements for step 4 were fulfilled and if not, what would be missing to achieve this. We are aware of the current status concerning the two atlas-related If you have any questions, please don't hesitate to ask. Best, Peer |
Hi everyone, I just wanted to follow up on my earlier message: would it be possible to evaluate if the requirements for step 4 were fulfilled and if not, indicate what we would need to do to achieve this? It would be great to hear from you. Best, Peer |
@effigies you were there at the steering group meeting, and while we are still a little unsure of some specific aspects, I cannot see any reasons not to move on stage 4 - IMO +1 |
reposted from other thread: Hi @PeerHerholz, thanks for the patience -- I am not sure I'm the correct person to judge this, but to obtain a number you may open a PR to the bids-website repo by editing these files. We can then discuss in that PR and potentially approve and merge it, and then you'll have the official number :-) |
Hi @sappelhoff, thanks for getting back to us and the information. We opened a respective PR here. Looking forward to the discussion! Cheers, Peer |
Hello @bids-standard/maintainers, @bids-standard/steering & everyone, I hope you're doing fine. We (@jdkent, @dorahermes, @francopestilli, @dlevitas, @poldrack, @CPernet, @melanieganz, @mnoergaard, @mathesong, @bendhouseart, @MaxvandenBoom and others) continued to work on BEP038 - Atlases. We would like to finalize Step 9 of the BEP development process: If y'all could have a look at the draft again and share your thoughts re the current status and if something/what definitely still needs to be addressed, that would be great! I'll start a list below and will keep editing it based on your comments. Needs to be addressed
Would be cool to address but not necessaryThanks so much again for all your help and effort, we highly appreciated it. |
Thanks @PeerHerholz! And please check out the atlas PR and voice your opinions, so we ensure to address all concerns before moving forward. |
Thanks @melanieganz! I/we keep monitoring the |
Hi everyone, given that #1579 was merged, I think we can proceed with finalizing the draft. Could I please ask everyone, the moderators/leads (@eduff, @jdkent) and contributors (@dorahermes, @francopestilli, @dlevitas, @poldrack, @CPernet, @melanieganz, @mnoergaard, @mathesong, @bendhouseart, @MaxvandenBoom, @dlevitas) to have another look at the BEP and indicate problems that should be addressed before we're moving forward (except the ones listed above)? Thank you very much for your continued supported, we highly appreciate it. Cheers, Peer |
|
Thx for the reply and comments @CPernet!
Sure thing, we can remove it for the time being and add one if needed.
I'm actually not sure if #1602 applies here, ie for |
@melanieganz I added the needed discussion re BEP017 files to the list. |
@melanieganz and @CPernet, I was trying to come up with an approach to exclude |
is that the files to check? PR are fine but a pain to edit, can you invite people to your fork instead |
Hi @CPernet, sorry for my unclear message. I was referring to the |
?? @melanieganz and I were wondering about community feedback -- posting the google doc to the community? |
Hi @CPernet, the google doc was shared earlier this year and promoted at different events (e.g. conferences and workshops). Are you suggesting to make another call for feedback? Cheers, Peer |
well no one else but 'us' look at it, and that's a little problematic -- nothing says we can keep working on the markdown, but another public email would be good IMO |
re good your point about relmat -- sine we want this BEP first, we have to define it here. |
can you point me to the markdown doc? do you need help - it is an important extension for Open Neuro PET so @bendhouseart can definitively spend time on that |
Hi Cyril,
yeah, I would be cool to get more ideas and feedback. However, I'm not sure how likely this is given the
So, introducing the files here but outlining that they're part of
We're still working on the P.S.: Sorry for closing and re-opening the issue, I hit buttons by mistake. |
Yes - let's not get stuck in the waiting loop ... I say let's fork the repo and start the markdown. In parallel, send another community email JIC. |
Hi folks, as we would like to bring this BEP to the next stage of the development process, ie moving it to
The second point will also include another announcement over the mailing list and social media to ask others for feedback. Cheers, Peer |
Dear @PeerHerholz, sorry, for leaving you hanging, I simply missed this thread. So regarding the files related to BEP017 (see below your earlier comment), I would vote for not mentioning them in the atlas BEP at all. They are no introduced yet in the spec and otherwise we would have to wait for BEP017 before merging the straightforward and basically relatively finished BEP038. So I would remove them for now and then mention in BEP017 that files can be saved according to the atlas spec and give examples there. And then one can always later on make a minor change and add more atlas examples once BEP017 is finished and merged. I just think it's a bad idea to keep holdign the straightforward atlas BEP that goes way beyond BEP017 back. So remove those, freeze draft and get a date with the maintainers and afterwards steering group scheduled. |
Just to clarify, @oesteban here you are addressing cons of @PeerHerholz point 1, correct? Point 1 is regarding The "derivative" dataset type Wrt the status of the derivatives, the common derivatives define some guidelines for the derivatives, but there are approximately 5-6 other BEPs going on that also define this (BEP011, 012, 016, 023 per modality plus BEP017 and 038). So this is a huge ongoing effort between all modalities and there are not fixed decisions yet, correct? So we do agree that So what exactly is the issue with defining an atlas when as a derivative with the examples provided in the PR? They do not break any of the existing modality specific derivatives - we took great care to invite and solicit feedback from other modalities like the EEG, MEG and MR crowd and based our experience on PET. Can you please in one to two sentences formulate the a con list? The whole argumentation is quite convoluted for me to follow. |
And @oesteban wrt your other reply which I believe addresses point 3 of @PeerHerholz's list just as I my comment did. Your con would be that the atlas should be named differently and that it's not versionable? I would like to address this in two parts:
|
Hi All, I'm a big fan of this BEP and would like to thank @PeerHerholz and everyone else for their hard work putting this together. I have some questions about suffixes. Of the three suffixes introduced ( I'm not exactly sure how this process works, but I'm happy to open a PR to the |
dseg, probseg and mask are defined in derivatives: https://bids-specification.readthedocs.io/en/stable/derivatives/imaging.html mimap will need to be addressed by BEP23. |
hi @oesteban @CPernet @melanieganz @yarikoptic @effigies etc, Thanks for the heartful discussion. I have been observing. I want to move forward with the BIDS-Connectivity project this BEP needs to be resolved. A summary of what I understand the issues might be (more things have been said, I understand but I am trying to close): As a general background, Templates in this context are Atlases in a specific space (say the sub-01 space). Atlases instead are (1) If we change, the name from Atlas to Template (see A). @oesteban believes this BEP becomes unnecessary because derivatives can already be saved. Recently, @TheChymera and @yarikoptic started what seems to be a very similar BEP here, should these two be merged or the discussion coordinated? In general, Atlases are coming to be recognized here as something that BIDS will need to manage and data archives and platforms will need to be able to host atlases for purposes of analysis of individuals or groups. @oesteban can you try to make a proposal that does not require the deletion of the current BEP, but a either a change in its operations or a repurpose? Otherwise, I fear the support seems to go in favor of keeping a version of this BEP. But I do not want to be the judge of that, we had a similar roadblock for a different issue for DWi and I think we have mechanisms in place to resolve the issue, but I was hoping we could find a solution here before we go that route. [Edited for clarity] |
Well, if this is almost-finished, and we're just derailing the discussion, perhaps this could be a subsequent BEP. It's not just that one dataset, there are numerous “libraries” of some feature or another, which I think are best contextualized as atlases. They're no different from segmentation areas in the sense that they can be used as spatial references, and the things which are important for them are a superset of what's covered in BEP038. Resolution, corresponding template, that's all good. The two I've been using are the ABI gene expression and the ABI connectivity atlases, and what they need would be a In any case let me know, here on the other issue. I'd be happy to help with either extending this, or starting a new BEP. |
I think this request (adding feature- is similar to @pwighton's request in our discussion here. it seems that we need ways to define content types of images in an Atlas. Does it seem reasonable? |
Yes. The way I understand it, and perhaps I don't, is that currently the BEP assumes parcellation atlasses. The category range is defined broadly “such as landmarks, labeled brain regions, parcellated surfaces, and other stereotaxy-referenced tables, or textual matter” but I see that as all boiling down to parcellation.
I would agree, but is this something which should be done now, perhaps at the cost of delaying the BEP, or is it something that can be easily tacked on later? |
case 1 with atlas shared at the root works fine - it is compliant with the current spec (except the entity atlas) , @oesteban was fine with this, I think (TBC). @TheChymera this does NOT boil down to parcellation - re qMRI and PET (ie quantitative voxel wise values) I must admit - I'm also puzzled by how to move forward. |
Hello everyone, sorry again for the late reply but I'm most likely afk for the rest of this week as well but we will be back next week. Just to follow up on the discussion on how to move forward. IMHO it becomes increasingly more challenging to follow all the discussion points and arguments in this thread. Please don't get me wrong: I think all of them are warranted and very helpful. However, I also think a more precise and clear overview would be beneficial for folks. This essentially goes back to my proposal of a collection of use cases/examples based on the different positions that were brought up (ie WDYT? Cheers and thanks again, Peer |
Meta/moderation of the BEP: @PeerHerholz @francopestilli @melanieganz @effigies @CPernet - I'll shortly send an email proposing the logistics of how to go forward with this BEP. As you'll see in the email, it's definitely not perfect, but I think it could be piloted as a more specific way for all BEPs to work out. I agree with Peer that the process of BEPs is very undefined beyond "we discuss on Google Docs until a PR to the specs is created". Most (if not all) BEPs struggle to have such a clean and straightforward trajectory. Specific answers in this BEP:
I didn't get a sense that my effort will be considered, so I didn't go ahead and try. But I can give it a go if it will be actually considered - I don't think it requires much (beyond the main talking point we've already initiated above).
Not at all, and please, I would ask everyone not to simplify my interventions this way because it very badly represents my view (@francopestilli, I know this is a consequence of my poorly explaining rather than an intended oversimplification). The real (and very important) value of this BEP is on the metadata descriptions. That is already there, and I am in favor of that. The structure may change, but the metadata descriptions will remain practically unmodified. It worries me a little that the discussion is perceived as a complete slamming of the BEP when it is not. One potential way to move this forward more rapidly is to split the atlas metadata and the file structure into two different BEPs. I can anticipate that atlas metadata will be a low-hanging fruit to go in smoothly; I see the file structure and the new DatasetType as very problematic for the points discussed above and below. (highlight for @PeerHerholz @francopestilli)
This admittedly worries me. If the validator is "flexibilized" with derivatives, as a result of realizing that derivatives is such a wild space that the validation cannot be implemented, then I would stop here and recommend everyone to move into a self-sufficient data description such as NIDM and hope that the NIDM-BIDS BEP sees the light one day. This also deflects the discussion over the point that the new DatasetType opens the door to having datasets with two possible (and equally valid) encodings within BIDS. This is problematic (with dangerous consequences to reproducibility) beyond whether the validator can distinguish them and also needs addressing. For more on this point, please check my PR #1530 (although I see this duality way more problematic than what my PR tried to address).
It seems my argument was poorly delivered. I'm not suggesting it doesn't follow the BIDS specs. What I meant is that, if the DatasetType were to be "atlas", then BIDS will need some sort of "registry" of names so that your choice "PS13" is not overshadowed by another researcher, and when papers come out, mentioning "PS13" unambiguously refers to your template (atlas), not others that happen to have been called the same way. As we describe in the templateflow paper, ambiguity is a big issue for reproducibility and expanding flexibility here will have negative consequences overall.
There are a few issues I see with this:
@yarikoptic thanks a lot for the paper and the picture you posted. Following the figure with the ontology, we can clearly see how BIDS encodes the most important concept -the subject- on the left side** -yellow, "spatial module"- (with session IMHO omitted as assumed) with entities and file structure tools, and the concepts on the right side (green, "semantic module") more predominantly with metadata. The argument I'm trying to bring about is that template is a concept that falls within the spatial module of the ontology, while atlas is a semantic construct (right side of the ontology). ** EDIT: changed wording order for clarity |
This thread has honestly gone way past the point where I have time to
follow it in detail, but I do want to respond to @oesteban's last point:
The argument I'm trying to bring about is that template is a concept that
falls within the spatial module of the ontology, while atlas is a semantic
construct (right side of the ontology).
Isn't an atlas really a mapping between the semantic domain and the spatial
domain? e.g. mapping of a particular anatomical region or neurotransmitter
density (semantics concepts from neuroscience) into a location in the
sterotactic space?
|
I would not dare contradict your argument(s) @poldrack :D. My point was more about Yarik's paper - within the thought framework of the figure he pasted there, it becomes clear that templates belong more on the left side (spatial domain) and atlases more on the right side (semantic domain). Obviously, there are many grades and colors to definitions and concepts, and so I agree there's a connection between the idea of an atlas and the spatial domain, which goes through the idea of template as I discuss next. Circling back to your question, Isn't an atlas really a mapping between the semantic domain and the spatial domain?, of course, yes, but as very nicely put by @effigies above, that mapping needs to go necessarily through the concept of templates (T1w, T2w, or any other MRI, radioligand receptor binding, etc.), which map a particular property that may be used to engender individual or group stereotaxy. Obviously, the actual brains Talairach used to engender stereotaxy and establish a reference where the brain could be atlased are way less semantic or abstract than digital brains obtained with 3D imaging. This is to make an epistemic argument supporting the equivalence between template (averaging individuals or repetitions of a single individual) and subject as a coordinate-system-engendering spatial reference. There may be a day when the scanner will produce TheOneAndBestAtlas' parcellations and region-wise averaged fMRI time series of the brain directly - but I doubt it will allow us to discard the original fMRI time series or the anatomical images when retrieving the data from the scanner. Because then that atlas' reference is lost, and any possibility of understanding how things were generated with it also is. EDIT: Perhaps one point was missing above, in the sense that @yarikoptic's suggestion of looking into the paper he proposed is to me very important - it would allow moving from "how individuals look at this problem" (i.e., in terms of I like this or that) into "how this problem can be categorized and resolved with continuity to BIDS' principles and foundations". |
@oesteban what we need in the google doc is the proposed changes for cases 1, 2, 3 - along with whatever name changes are needed (maybe use a different colors) -- then we can ask the technical committee to decide since otherwise we are stuck -- that rule was decided by the steering group committee last week (to be merged FYI https://github.com/bids-standard/bids-extensions/pull/29/files) |
This is a misinterpretation of bids-standard/bids-extensions#29 for the reasons I have provided there bids-standard/bids-extensions#29 (comment) IMHO that call to the technical committee cannot be done if the BEP is not in the PR phase unless I'm misinterpreting the governance. No, I do not intend to work on a Google Doc document that is out of date and should not be used (as we have discussed above). I can work as a PR against @francopestilli's lab Github repo, but this is precisely why I proposed over email to fork the specs to work in this PR, to make it open to everyone and under the control of BIDS. |
the PR reflects exactly the google doc -- and the steering group decided on that procedure |
but feel free to make a parallel PR for the technical committee to review |
What PR? |
Apologies, I've been too quick -- in several instances above we linked to the PR, i.e. the Atlas BEP is/was ready --> you can see it there https://github.com/PESTILLILAB/bids-specification/blob/bep038/src/atlas.md. What I was proposing is to use your framework to write the 3 use cases/examples that the BEP deals with. I think that's the best way for everyone to appreciate the changes you want to make. Without those 3 cases fully fleshed out, it can be difficult to follow, Then we can see if we can reach a consensus, if and only if we cannot, would we need to ask an 'external' group to look into it (as per BEP guideline PR). I hope this makes sense to you. Thx. |
I don't know what my framework is in this context. Are you asking me to write a PR against PESTILLILAB/bids-specification?
I have done that above, but okay if a PR against the fork would be a better way of driving the conversation. Expand to see my concerns about the "BEP guideline PR"
The "BEP guideline PR" has so many problems that it is absolutely uninterpretable. However, as per the BIDS' bylaws, it is the BIDS SC who decides its interpretation anyway. In effect, the "BEP guideline PR" is just creating a loophole to move the responsibility for making an ad-hoc decision on a BEP to a group (we don't know the name yet) of referees instead of having the SC behind that decision. Before proceeding that way, I have several questions (please expand if interested):
Linking bids-standard/bids-extensions#29 as all these questions should be in that thread. |
I think we should go ahead and open a PR for BEP038. I suggest that we push the current @oesteban I think it would be good to make a concrete PR against that branch, once it's created. I've tried to keep up with this thread, but still don't understand what alternative you would like to see. I did say that I would also try to make a PR against the PESTILLILAB branch a few weeks ago, but have not had the time. I will try to get to that ASAP. Please do not wait on me to get a PR in if you end up having time to put together a proposal first. |
Dear all, I wanted to highlight that @effigies has turned the PESTILLI lab PR into a proper BIDS repo PR now and that we are continuing/restarting the discussions there. We really want to get your input and see your alternative proposals for how we can share atlases. So please check it out! |
Thanks @effigies. You suggested in the PR to have the conversation here - but I think it actually would be better to lock this conversation and redirect everyone to the PR as the main discussion point about the BEP. |
I suggest that future conversations be focused in individual PRs against the bep038 branch, rather than creating extensive discussion in that PR, which could grow into something sprawling like this issue. I think it would be good to have very specific proposals that can be discussed and accepted/rejected. I'm not a huge fan of locking conversations, but I can if it seems like it will be difficult to keep new posts to this thread to a minimum. |
Your idea
EDIT
github draft for current discussion: https://github.com/PESTILLILAB/bids-specification/blob/bep038/src/atlas.md
the google doc draft (deprecated in favor of the github draft) : https://docs.google.com/document/d/1RxW4cARr3-EiBEcXjLpSIVidvnUSHE7yJCUY91i5TfM/edit?usp=sharing
The text was updated successfully, but these errors were encountered: