Upstreaming fbsync changes #1367
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fbshipit-source-id: bb7d2eb52240c7223b57c3c9624e61d116e77e39
Summary: [20:45:34: cpuhrsch@devvm3140 pytorch]$ ./fb_build/import_text.sh Reviewed By: pbelevich Differential Revision: D21320577 fbshipit-source-id: ac2148b9f0d58e5538443c879845bfb4f6ca7202
Summary: ./fb_build/import_text.sh Reviewed By: zhangguanheng66 Differential Revision: D21343124 fbshipit-source-id: c08ecad2cc6f439fa40130aeaf91383be9403fe8
Summary: See title Reviewed By: pbelevich Differential Revision: D21344211 fbshipit-source-id: a8bcf7f3ab9bb2c2853e27f612e82caa341d3651
Summary: Import torchtext up to pytorch#786 Reviewed By: cpuhrsch Differential Revision: D21483116 fbshipit-source-id: bc8ab38db9dc9ce4a8734ca8ea991c20e4ef0882
Summary: Import up to pytorch#798 Addresses T67599333 Reviewed By: zhangguanheng66 Differential Revision: D21764935 fbshipit-source-id: f44d1db637799f2e95f420a8099fbf19545c7cbd
Summary: Import from github master Reviewed By: zhangguanheng66 Differential Revision: D21886238 fbshipit-source-id: a8f098e299466dd1701fe7ceb6a97c2a2fc54b9d
Summary: Import from github master Reviewed By: zhangguanheng66 Differential Revision: D21907519 fbshipit-source-id: f22370d97796da5f2cb9f76f506c80f18fefea7f
Summary: Original commit changeset: f22370d97796 Reviewed By: zhangguanheng66 Differential Revision: D21964222 fbshipit-source-id: c316836596fc3e232e63abc59e172f237b551cc5
Summary: Import from github torchtext/master Reviewed By: zhangguanheng66, cpuhrsch Differential Revision: D22168183 fbshipit-source-id: 7d96ade64f18942d9bd19437011be2f65f0b2a5e
Reviewed By: Nayef211 Differential Revision: D22315715 fbshipit-source-id: 6b8b8544b0aa458cf5e7e9ca380d0dc85c98189f
Summary: Import from github torchtext/master Reviewed By: cpuhrsch Differential Revision: D22420576 fbshipit-source-id: 4d2c19d7f1db8f698894ca406c1c44b2ad8e0506
fbshipit-source-id: 5113fe0c527595e4227ff827253b7414abbdf7ac
fbshipit-source-id: 798decc90db4f13770e97cdce3c0df7d5421b2a3
Summary: Import from github torchtext/master Reviewed By: zhangguanheng66 Differential Revision: D22641140 fbshipit-source-id: 8190692d059a937e25c5f93506581086f389c291
Reviewed By: ashwinp-fb Differential Revision: D22955630 fbshipit-source-id: f00ef17a905e4c7cd9196c8924db39f9cdfe8cfa
Summary: Import from github torchtext/master Reviewed By: zhangguanheng66 Differential Revision: D22989210 fbshipit-source-id: 083464e188b758a8746123f4dd2197cc7edc4bc4
Summary: Import from github torchtext/master Reviewed By: cpuhrsch Differential Revision: D23190596 fbshipit-source-id: 1568a25a5bd6431bcef3c6539f64a3ab1f5bccd7
Reviewed By: hudeven Differential Revision: D23451795 fbshipit-source-id: 73e6130c16716919c77862cef4ca4c8048428670
Reviewed By: Nayef211 Differential Revision: D23539397 fbshipit-source-id: 88dce59418a3071cbc9e944cf0a4cf2117d7d9f7
Reviewed By: cpuhrsch Differential Revision: D23616189 fbshipit-source-id: 365debc987326145eead7456ed48517fe55cac96
Summary: Pull Request resolved: pytorch/pytorch#42390 **Summary** This commit extends support for properties to include ScriptModules. **Test Plan** This commit adds a unit test that has a ScriptModule with a user-defined property. `python test/test_jit_py3.py TestScriptPy3.test_module_properties` Test Plan: Imported from OSS Reviewed By: eellison, mannatsingh Differential Revision: D22880298 Pulled By: SplitInfinity fbshipit-source-id: 74f6cb80f716084339e2151ca25092b6341a1560
Reviewed By: cpuhrsch Differential Revision: D23721167 fbshipit-source-id: 13b32091c422a3ed0ae299595d69a7afa7136638
Reviewed By: cpuhrsch Differential Revision: D23962265 fbshipit-source-id: 0d042878fe9119aa725e982ab7d5e96e7c885a59
Summary: Pull Request resolved: pytorch/pytorch#45261 **Summary** This commit enables `unused` syntax for ignoring properties. Inoring properties is more intuitive with this feature enabled. `ignore` is not supported because class type properties cannot be executed in Python (because they exist only as TorchScript types) like an `ignored` function and module properties that cannot be scripted are not added to the `ScriptModule` wrapper so that they may execute in Python. **Test Plan** This commit updates the existing unit tests for class type and module properties to test properties ignored using `unused`. Test Plan: Imported from OSS Reviewed By: navahgar, Krovatkin, mannatsingh Differential Revision: D23971881 Pulled By: SplitInfinity fbshipit-source-id: 8d3cc1bbede7753d6b6f416619e4660c56311d33
Summary: Sync torchtext up to GH commit eb5e39d Reviewed By: parmeet Differential Revision: D27400885 fbshipit-source-id: 1f8f92ca42ba36d070db6740b3bb4c148f69586b
Summary: Imported latest from github Master PR#1267 Reviewed By: cpuhrsch Differential Revision: D27503970 fbshipit-source-id: 853ff895ba42b1feb7442abe1c87478e43d62e5b
Summary: Import torchtext from github Reviewed By: parmeet Differential Revision: D27803909 fbshipit-source-id: 9cb0f15858b1417cb5868d5651513eb2df998fbe
Reviewed By: parmeet Differential Revision: D27922562 fbshipit-source-id: 3c18cd9e2583e03471461ad8a22ac6b0ceb596a2
Summary: Importing torchtext from github for regular sync. Reviewed By: cpuhrsch Differential Revision: D27983819 fbshipit-source-id: 5806421d788afaa872f5320b5f4cbcd913e103ea
Reviewed By: parmeet Differential Revision: D28101664 fbshipit-source-id: a8643b3ecf85de2cb815dcfa5789a4a5d246d80f
Reviewed By: cpuhrsch Differential Revision: D28111696 fbshipit-source-id: fef195941492493a399adb37339cfa64795e22a0
Summary: This diff syncs torchtext GH with fbcode Reviewed By: cpuhrsch Differential Revision: D28321356 fbshipit-source-id: 7736f0d100941627b58424911a1329b1ce66c123
Reviewed By: parmeet Differential Revision: D28478153 fbshipit-source-id: bfcaffe8fe48e96d8df454f7df0d25ec39d5d4a6
…ytorch#1289) Summary: allow-large-files to commit wikitext103_vocab.pt Reviewed By: cpuhrsch Differential Revision: D28478152 fbshipit-source-id: c2a871439f054024b95c05f7664a84028aacaca3
Summary: Importing from Github Reviewed By: cpuhrsch Differential Revision: D28572929 fbshipit-source-id: 2e7b00aadeda6ab0596ef23295f41c5b0fa246e7
Summary: Adding API usage logging for Vocab module Reviewed By: colin2328 Differential Revision: D28585537 fbshipit-source-id: 38975b523fb597412fbcb18ef831bfb4834cb420
Reviewed By: parmeet Differential Revision: D28683381 fbshipit-source-id: 7bfbf445dd512f0ce21c34096cf3f08332d90138
Reviewed By: NicolasHug Differential Revision: D28994054 fbshipit-source-id: 4c679f56ef37b18f6d2acaaaed8518facbeaa41c
Summary: Import torchtext pytorch#1328 ca514f6 Reviewed By: NicolasHug Differential Revision: D29120370 fbshipit-source-id: 229586f3470bd61bfb2f6a390d79e45d4eae3b4d
Summary: Fixes pytorch/pytorch#58988. Pull Request resolved: pytorch/pytorch#59067 Reviewed By: jbschlosser Differential Revision: D28986642 Pulled By: heitorschueroff fbshipit-source-id: 3ef2d26b4010fc3519d0a1a020ea446ffeb46ba0
Summary: Import torchtext pytorch#1300 0435df1 Reviewed By: parmeet Differential Revision: D29371832 fbshipit-source-id: 624280ddfa787a4e7628e60fa673cb9df0a66641
Summary: Import from github Reviewed By: hudeven Differential Revision: D29441995 fbshipit-source-id: 27731ce2714c16180d11bfb26af5d5a2dba408b1
Summary: Import from github Reviewed By: NicolasHug Differential Revision: D29537684 fbshipit-source-id: 25b1fc1e6d9f930e83f5f2939788b90b083aeaa2
Summary: We would like to add and access torchtext datasets on manifold. This Diff unifies the dataset download from external links and through manifold for internal access. This is enabled via io_path package. The main idea is to plugin the download hooks in the download_from_url function. The download hooks will delegate the download to appropriate Path Handler. In OSS we have enabled download via https and google drive. Internally, we replace the download hook to download data from manifold. We have created a _download_hooks.py file under /fb/ folder which will replace the corresponding file in OSS. The file under /fb/ folder converts the http/https URL paths into corresponding manifold paths and download the data from there. Reviewed By: hudeven Differential Revision: D28892389 fbshipit-source-id: 3b66544dd2345075e2e7c524f344db04aa2a24e3
Summary: Import from github Reviewed By: hudeven Differential Revision: D29856211 fbshipit-source-id: 6332f9bdf3cf4eef572c5423db15101ea904d825
Summary: Import torchtext pytorch#1365 c57b1fb Reviewed By: parmeet Differential Revision: D29940816 fbshipit-source-id: 6b2495b550a7e6b6110b0df12de51a87b0d31c1c
Summary: This is the first step in moving Roberta Model from pytext_lib into PyTorch Text Library. Here we moved the Roberta building blocks into pytorch/text/fb/nn/modules. The code-base is organized according to WIP document https://docs.google.com/document/d/1c0Fs-v97pndLrT3bdfGRGeUeEC38UcDpibvgOXkbS-g/edit#heading=h.3ybcf0ic42yp Reviewed By: hudeven Differential Revision: D29671800 fbshipit-source-id: d01daa99e0a5463716660722381db9a0eeb083f8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reviewed By: hudeven
fbshipit-source-id: d01daa99e0a5463716660722381db9a0eeb083f8