diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 000000000..676f220eb
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: Create a bug report to help us improve
+labels: "bug"
+---
+
+**Describe the bug**
+
+<!-- A clear and concise description of what the bug is. -->
+
+**To Reproduce**
+
+Steps to reproduce the behavior:
+
+1. ...
+2. ...
+3. ...
+
+**Expected behavior**
+
+<!-- A clear and concise description of what you expected to happen. -->
+
+**System [please complete the following information]:**
+
+-   OS: e.g. [Ubuntu 18.04]
+-   Python version: [e.g. Python 3.8]
+-   Pytorch version: [e.g. Pytorch 1.11.0]
+-   Transformers version: [e.g. transformers 4.34.0]
+
+**Additional context**
+
+<!-- Add any other context about the problem here. -->
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 000000000..5065058d6
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest a new feature
+labels: "enhancement"
+---
+
+**Is your feature request related to a problem? Please describe.**
+
+<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when ... -->
+
+**Describe the solution you would like**
+
+<!-- A clear and concise description of what you want to happen. -->
+
+**Additional context**
+
+<!-- Add any other context or screenshots about the feature request here. -->
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
new file mode 100644
index 000000000..4a1217df6
--- /dev/null
+++ b/.github/pull_request_template.md
@@ -0,0 +1,10 @@
+**PR Checklist**
+
+- [ ] A description of the changes is added to the description of this PR.
+- [ ] If there is a related issue, make sure it is linked to this PR.
+- [ ] If you've fixed a bug or added code that should be tested, add tests!
+- [ ] If you've added or modified a feature, documentation in `docs` is updated
+
+**Description of changes**
+
+<!-- Please state what you've changed and how it might affect the users. -->