Skip to content

Commit

Permalink
Polish developmental guidelines
Browse files Browse the repository at this point in the history
  • Loading branch information
Gammerdinger authored Dec 19, 2024
1 parent 47264ef commit 354f8fc
Showing 1 changed file with 31 additions and 28 deletions.
59 changes: 31 additions & 28 deletions training_development.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ This is a guide to help create continuity across our training materials. It is a

Backward Design principles are a set of principles for course design which focus on how students will be assessed and then work to develop materials that align with those assessment goals. When we apply backward design principles in our course development it should follow these steps:

1. Identify what goal we want participants to accomplish
2. Create an assessment that will certify that the goal was obtained
3. Develop materials aimed at ensuring that students will be successful at the assessment
1. Identify what goals we want participants to accomplish
2. Create an assessments that will certify that the goals was obtained
3. Develop materials aimed at ensuring that students will be successful at these assessments

When you are planning a workshop or a lesson it can be really helpful to develop, even if it is very rudimentary (you can always come back to this) what exercises you want the participants to do first. This will tremendously help guide your development process, because you will be developing *toward* an assessment.
When you are planning a workshop or a lesson it can be really helpful to develop, even if it is very rudimentary (you can always come back to this later), what exercises you want the participants to do first. This will tremendously help guide your development process, because you will be developing *toward* an assessment.

Vanderbilt has a nice [webpage](https://cft.vanderbilt.edu/guides-sub-pages/understanding-by-design/) on Backward Design.

## Work as a team
## Working as a team

When working on developing course materials, if can be helpful to work as a team. This has a few advantages over solo course development:

Expand All @@ -32,17 +32,17 @@ What we have found when people work alone on material development is that it can

When we work as a team on materials development, we need to:

1. Explicitly lay out an overarching timeline for the project at the beginning with ample cushion time built in
1. Explicitly lay out an overarching timeline for the project at the beginning with ample cushion time built-in
2. Create sub-timelines for each week or two weeks of development
3. Create time budgets for certain tasks so that everyone has clear expectations on the expected timeframe is and there can be a discussion if more time is needed and why
4. Assess and give feedback on the prgress made since the last meeting
4. Assess and give feedback on the progress made since the last meeting
5. **Try to avoid spending more than ~5 hours on development without getting at least some feedback on it.** It can be very easy to spend too much time and go too far down a rabbit hole and bill too much time for development.

## Phases of design

When we are creating a course and we have an idea for each lesson, it is a good idea to work within this framework:

1. Identify your Learning Objectives or atleast a draft or what you want them to be
1. Identify your Learning Objectives or atleast a draft of what you want them to be
2. Draft a rough idea of your assessments for those Learning Objectives
3. Create a framework of the lesson. This framework should be very minimalistic in content but still contain:
- Learning objectives
Expand All @@ -51,28 +51,29 @@ When we are creating a course and we have an idea for each lesson, it is a good
- Subheadings
- Psuedocode of what the participants for the workshop will be doing (don't worry about the exact code or parameters at this point unless you are just copy-and-pasting them from somewhere, but something like `bwa align to mm10 reference` is fine
- Drafts of exercies/assessments, these don't need to be final and it could be something like `have participants run FASTQC on sample 2`
- Text where image will go describing it like `image showing pseudobulk process`. If you know this image exists and we already have developed it somewhere, feel free to embed it but don't create any images at this point
- Text where image will go describing it like `image showing the pseudobulk process`. If you know this image exists and we already have developed it somewhere, feel free to embed it but don't create any images at this point
- Navigation buttons, but don't link them to any pages yet as names will likely change
- Footer
5. Get approval from your collaborator on the framework before proceeding
6. Flesh out draft materials by learning objective
7. After completing a learning objective, show it to your colloborator for feedback
8. Incorporate your collaborator's feedback and design initial drafts for any figures/GIFs/apps (guidelines below on these) that will be embedded in the materials
9. Repeat steps 5-8 for each learning objective in a lesson
10. Have a third-party review your materials and provide feedback. The thrid-party should recognize that this is still in a draft state. Fixing typos, etc. are not the goal here but rather, does the flow make sense to them, is the lesson clear, etc.
10. Have a third-party review your materials and provide feedback. The third-party should recognize that this is still in a draft state. Fixing typos, etc. is not the goal here but rather, does the flow make sense to them, is the lesson clear, does the assessment address the learning objective, etc.
11. Incorporate third-party's feedback into the lesson
12. First pass at polish on the lesson. Try tidying the lesson up, fixing typos, checking links, making sure things are rendering correctly in the HTML
13. Third-party polish. This is the final polish for a lesson, the final polisher must:
12. Create more finished products for the figures/GIFs/apps as needed
13. First pass at polish on the lesson. Try tidying the lesson up, fixing typos, checking links, making sure things are rendering correctly in the HTML
14. Third-party polish. This is the final polish for a lesson, the final polisher must:
- Run all of the code to ensure it works
- Work from the HTML page to ensure renderings are correct
- Address any typos they see
- Check each link to ensure they work

# Identifying a dataset

This *can* be one of the most time-consuming parts of development. When designing our workshops, we want to be using publically availible data. People outside of Harvard oftentimes use our resources and they need to be able to access the data set via SRA/GEO. If we have toy datasets we should host them on Dropbox or GitHub (**Meeta we should make a verdict on this** Personally, I like having everything on GitHub, but sometimes there are file size issues with GitHub. Sometimes I get e-mails about access issue with people getting odds and even from Dropbox and that is annoying. I sort of feel like the policy should be GitHub if we can, but if File Size constricts us, then Dropbox?).
This *can* be one of the most time-consuming parts of development. When designing our workshops, we want to be using publically available data. People outside of Harvard oftentimes use our resources and they need to be able to access the dataset via SRA/GEO. If we have toy datasets we should host them on Dropbox or GitHub (**Meeta we should make a verdict on this** Personally, I like having everything on GitHub, but sometimes there are file size issues with GitHub. Sometimes I get e-mails about access issue with people getting odds and even from Dropbox and that is annoying. I sort of feel like the policy should be GitHub if we can, but if File Size constricts us, then Dropbox?).

We also want to avoid to much unnecessary complexity to the data. What is the minimum number of samples we can trim the data down to and still be within best practices? Perhaps the experiment had multiple conditions, can we narrow the dataset to just case and control conditions. We don't want the complexity of the dataset to take away from the learning of the concepts.
We also want to avoid to much unnecessary complexity to the data. What is the minimum number of samples we can trim the data down to and still be within best practices? Perhaps the experiment had multiple conditions, can we narrow the dataset to just case and control conditions? We don't want the complexity of the dataset to take away from the learning.

# Format

Expand All @@ -88,7 +89,7 @@ The general format of the page should be:


## YAML
The top of the page should have a YAML
The top of the page should have a YAML that contains some metadat for the page and looks like:

```
---
Expand All @@ -108,23 +109,25 @@ Contributors: List of contributing authors
Approximate time: XX minutes
```

Also, include a cutdown workflow image to show particiapnts where they are in the workflow process.

## Learning Objectives

The next section we need to add is arguably one of the most important sections in our layout, the Learning Objectives. It is critical that learning objectives are:

- *Measurable* - This is really important that you can assess whether a learning objective has been achieved.
- *Measurable* - It is critical that you can assess whether a learning objective has been achieved
- *Specific* - Focus on what we are specifically aiming for participants to get out of the lesson
- *Achievable* - Partipcants can accomplish the learning objective given the resources in the lesson
- *Achievable* - Participants can accomplish the learning objective given the resources in the lesson

You will likely find that this might be one of the more difficult parts of the lesson to develop, but it can be a great guide for how you develop the lesson. Whe you are developing learning objectives, being considering where on Bloom's Taxonomy (see below) you are aiming for workshop participants to be reaching. Not all learning objectives for the lesson need to be at the higher levels of Bloom's taxonomy. A variety can be beneficial and also is probably an accurate reflection of the level our materials should be taught at. For example, we likely won't ask participants to "Construct a read alignment algorithm", but being able to "Paraphrase or identify key steps in the read alignment process" is likely a more reasonable expectation.
You will likely find that this might be one of the more difficult parts of the lesson to develop, but it can be a great guide for how you develop the lesson. When you are developing learning objectives, consider where on Bloom's Taxonomy (see below) you are aiming for workshop participants to be reaching. Not all learning objectives for the lesson need to be at the higher levels of Bloom's taxonomy. A variety can be beneficial and also is probably an accurate reflection of the level our materials should be taught at. For example, we likely won't ask participants to "Construct a read alignment algorithm", but being able to "Paraphrase or identify key steps in the read alignment process" is likely a more reasonable expectation.

<p align="center">
<img src="img/blooms_taxonomy.png" width="600">
</p>

*Image from https://tigerlearn.fhsu.edu/the-revised-blooms-taxonomy-as-a-framework-for-writing-learning-objectives/*

Create creating a Learning Objective, try to pick a verb that is going to pair well with the level in Bloom's Taxonomy you are hoping particiapnts will reach AND with your assessment. For example, if your Learning Objective is "Identify the key steps in bwa's read alignment algorithm" then the assessment should be something like "Which of these steps are part of bwa's read alignment algorithm" and not something like "Compare bwa and HISAT2's read alignment algorithms".
When creating a Learning Objective, try to pick a verb that is going to pair well with the level in Bloom's Taxonomy you are hoping participants will reach _AND_ with your assessment. For example, if your Learning Objective is "Identify the key steps in bwa's read alignment algorithm" then the assessment should be something like "Which of these steps are part of bwa's read alignment algorithm" and not something like "Compare bwa and HISAT2's read alignment algorithms".

Picking the right verb for your learning objective can be really helpful for creating your assessment of that learning objective. Below is a great table of more verbs associated with each level of Bloom's Taxonomy.

Expand All @@ -142,7 +145,7 @@ Below we have outlines for the different formats of content we will find in our

### Text

Text is important for conveying ideas, but long paragraphs and text-heavy lessons can be diificult for retaining participants' attention. Major deas should be stated in text, but they can also be supplmented by:
Text is important for conveying ideas, but long paragraphs and text-heavy lessons can be diificult for retaining participants' attention. Major ideas should be stated in text, but they can also be supplmented by:

- Images
- GIFs
Expand All @@ -166,26 +169,26 @@ Sometimes people have created excellent figures in papers and there is no need t

Figures that we create should follow some guidelines:

- *High-resolution* - Others in our group or elsewhere may recycle this figure and we would like the figure to be clear at different size scaling than we present. By providing high-resolution images, we can try to future proof this.
- *High-resolution* - Others in our group or elsewhere may recycle this figure and we would like the figure to be clear at different size scalings than we present. By providing high-resolution images, we can try to future-proof this.

- *Clarity* - **Each figure should try to convey a single takeaway point.** Only include details relevant for the point we are trying to convey with an image. We can make multiple images for different points. Try to avoid cramming an image with too many details it can be distracting and lose sight of the overall goal.

Be sure to label the axes on any plots.

Use colorblind-friendly color palettes when creating figures.

If working from screenshots, make sure the interface is clean, hide the dock on Mac, clean up your Desktop, etc.
If working from screenshots, make sure the interface is clean (i.e. hide the dock on Mac, clean up your Desktop, etc.)

- *Attribution* - On the figures we develop, put "Harvard Chan Bioinformatics Core" in the bottom right corner. This builds attribution into a figure, so if someone uses our figure and forgets to attribute us, we are still attributed.

- *Management* - It is encouraged to use Adobe Illustrator to create images. Please keep any images used in creating images, like screenshots, in the same folder as the Illustrator file (.ai) along with the final PNG/SVG image on Dropbox.

### GIFs

When using graphical user interfaces (GUIs) like IGV or FileZilla, it can be beneficial to create supplement the text with GIFs rather than screenshots to show how to interact with the interface. When deciding to create a GIF, have these considerations:
When using graphical user interfaces (GUIs) like IGV or FileZilla, it can be beneficial to supplement the text with GIFs rather than screenshots to show how to interact with the interface. When deciding to create a GIF, have these considerations:

- **Is this GUI rather stable (i.e. IGV/RStudio/FileZilla)?** - Creating GIFs are more time-consuming than screenshots so if a tool is not in a stable state, it would likely be preferable to use screenshots over spending the time to create, and then inevitably re-create, the GIF
- **Can I break this task into short segments?** - To work within the framework of High-Definition and not drop too many frames (less than about 12.5fps), you'll need to keep the final GIF to about 10 seconds or less in order for it to be small enough to upload to GitHUb which has a 25MB file size limit.
- **Can I break this task into short segments?** - To work within the framework of High-Definition and not drop too many frames (less than about 12.5fps), you'll need to keep the final GIF to about 10 seconds or less in order for it to be small enough to upload to GitHub which has a 25MB file size limit.

Many of the same considerations that existed for images, also exist for GIFs.

Expand All @@ -201,9 +204,9 @@ Many of the same considerations that existed for images, also exist for GIFs.

Sometimes we people have a habit of writing out equations in words like:

"The gravitational force is gravitational constant times the mass of first object times the mass of the second object divided by the distance between the centre of two bodies squared"
"The gravitational force is the gravitational constant times the mass of first object times the mass of the second object divided by the distance between the center of two bodies squared"

Perhaps some may find this useful, but it can be a lot more clear to provide equations ***AND*** define the elements of the equation. [Codecogs](https://editor.codecogs.com/) provides a nice way to embed equations as images in our GitHub documents
Perhaps some may find this useful, but it can be a lot more clear to provide equations ***AND*** define the elements of the equation. For example in the previous example is only the distance between the center of the two bodies squared or is the entire equation squared. [Codecogs](https://editor.codecogs.com/) provides a nice way to embed equations as images in our GitHub documents

If possible, providing toy examples and tables to help make the outcomes of the equations more intuitive. For example, instead of the gravitational example above, we could do:

Expand Down Expand Up @@ -239,7 +242,7 @@ Sometimes not all of the content that is developed is 100% relevant to the works

#### Dropdowns

Oftentimes you will come across places where there are two reasonable approaches to a task or want to provide more information on that is not critical for participant's immediate success, but is still information that people might like to have. If brief, these are a great place to use a dropdown. The format for these looks like:
Oftentimes you will come across places where there are two reasonable approaches to a task or want to provide more information on that is not critical for participant's immediate success, but is still information that people might like to have. If brief, these are a great place to use a dropdown. The syntax for these dropdowns look like:

```
<details>
Expand Down Expand Up @@ -271,4 +274,4 @@ All of the pages need to have the following footer:

*These materials have been developed by members of the teaching team at the [Harvard Chan Bioinformatics Core (HBC)](http://bioinformatics.sph.harvard.edu/). These are open access materials distributed under the terms of the [Creative Commons Attribution license](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.*

The main page for a repository should also have our Zenodo citation and include this need to include our RRID, *SCR_025373*.
The main page for a repository should also have our Zenodo citation and include our RRID, *SCR_025373*.

0 comments on commit 354f8fc

Please sign in to comment.