Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Parser] Parser 2.0 part 2 #6162

Merged
merged 48 commits into from
Aug 11, 2020
Merged

[Parser] Parser 2.0 part 2 #6162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
fbe3a5b
Add code from livestream with JK
jroesch Jul 11, 2020
4c61844
Fix errors parsing ResNet
jroesch Jul 11, 2020
432e0d9
Parse metadata section efficiently and do most of plumbing to resolve…
jroesch Jul 23, 2020
71dc90c
WIP
jroesch Jul 24, 2020
b940e8b
Change meta reference to an operator
jroesch Jul 27, 2020
089b342
Meta references now work
jroesch Jul 28, 2020
a4cfb3f
MetaReference expansion now works
jroesch Jul 28, 2020
0cb6c85
Start working on source map and move diagnostic context
jroesch Jul 28, 2020
4502adc
Convert tokenizer and parser to use new machinery
jroesch Jul 28, 2020
39df873
Kill to_json
jroesch Jul 28, 2020
b7dd570
Fix comment in type_infer.cc
jroesch Jul 29, 2020
9968fdf
Remove old parser
jroesch Jul 29, 2020
1113953
Rename parser tests and remove old ones
jroesch Jul 29, 2020
669aafc
Record span end information
jroesch Jul 29, 2020
9be4e7b
Convert to using spans everywhere
jroesch Jul 29, 2020
ab782e6
Add span fields back to all Relay constructors
jroesch Jul 29, 2020
b7e7fc5
Start passing spans
jroesch Jul 29, 2020
486f779
Pass spans around visitors
jroesch Jul 30, 2020
4daa1fe
Format
jroesch Jul 30, 2020
07f8789
Fix
jroesch Jul 30, 2020
1fcde63
Fix
jroesch Jul 30, 2020
13a4e96
disable reference lint from string helpers
jroesch Jul 30, 2020
4314584
Fix tokenizer
jroesch Jul 30, 2020
46dbc41
Fix issue with empty metadata section
jroesch Jul 30, 2020
9932b9d
Document new span fields and small tweaks
jroesch Jul 30, 2020
bb8171c
Formatting
jroesch Jul 30, 2020
597bf25
Add span doc fields
jroesch Jul 30, 2020
dc158c1
Add format tweak
jroesch Jul 30, 2020
0e54e6f
Improve errors and fix the semantic version tags in Prelude
jroesch Jul 30, 2020
4557fed
Update gradient.rly
jroesch Jul 30, 2020
7ef4e76
Clean up broken spans
jroesch Jul 31, 2020
10f2531
Clean up parser tests and turn on previously skipped tests
jroesch Jul 31, 2020
16c9276
Update errors to handle skipped cases
jroesch Jul 31, 2020
d521757
Tweak
jroesch Jul 31, 2020
aaea54a
Tweak
jroesch Jul 31, 2020
99bee2a
Format
jroesch Jul 31, 2020
4510899
Fix some minor issues with ADT tests
jroesch Jul 31, 2020
c991440
Format
jroesch Jul 31, 2020
50b517c
Fix path
jroesch Aug 3, 2020
6d1a38d
WIP
jroesch Aug 6, 2020
2e7bdbd
WIP
jroesch Aug 7, 2020
e3e5ef8
Fix ir_text_printer
jroesch Aug 8, 2020
2a8d67a
format
jroesch Aug 8, 2020
470bd86
Formatted
jroesch Aug 8, 2020
43dfe42
More formatting
jroesch Aug 9, 2020
7fa4c3f
Repair test cases
jroesch Aug 11, 2020
dd61083
Fix CI
jroesch Aug 11, 2020
77397ba
Retrigger CI
jroesch Aug 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,3 @@ conda/pkg
# nix files
.envrc
*.nix

# antlr files
*.tokens
*.interp
3 changes: 0 additions & 3 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ include(cmake/util/FindCUDA.cmake)
include(cmake/util/FindVulkan.cmake)
include(cmake/util/FindLLVM.cmake)
include(cmake/util/FindROCM.cmake)
include(cmake/util/FindANTLR.cmake)

if(EXISTS ${CMAKE_CURRENT_BINARY_DIR}/config.cmake)
include(${CMAKE_CURRENT_BINARY_DIR}/config.cmake)
Expand Down Expand Up @@ -66,7 +65,6 @@ tvm_option(USE_SORT "Build with sort support" OFF)
tvm_option(USE_NNPACK "Build with nnpack support" OFF)
tvm_option(USE_RANDOM "Build with random support" OFF)
tvm_option(USE_MICRO_STANDALONE_RUNTIME "Build with micro.standalone_runtime support" OFF)
tvm_option(USE_ANTLR "Build with ANTLR for Relay parsing" OFF)
tvm_option(USE_CPP_RPC "Build CPP RPC" OFF)
tvm_option(USE_TFLITE "Build with tflite support" OFF)
tvm_option(USE_TENSORFLOW_PATH "TensorFlow root path when use TFLite" none)
Expand Down Expand Up @@ -311,7 +309,6 @@ include(cmake/modules/Metal.cmake)
include(cmake/modules/ROCM.cmake)
include(cmake/modules/LLVM.cmake)
include(cmake/modules/Micro.cmake)
include(cmake/modules/ANTLR.cmake)
include(cmake/modules/contrib/BLAS.cmake)
include(cmake/modules/contrib/CODEGENC.cmake)
include(cmake/modules/contrib/DNNL.cmake)
Expand Down
40 changes: 0 additions & 40 deletions cmake/modules/ANTLR.cmake
View file
Open in desktop

This file was deleted.

65 changes: 0 additions & 65 deletions cmake/util/FindANTLR.cmake
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions docker/Dockerfile.ci_cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,6 @@ ENV PATH $PATH:$CARGO_HOME/bin:/usr/lib/go-1.10/bin
COPY install/ubuntu_install_java.sh /install/ubuntu_install_java.sh
RUN bash /install/ubuntu_install_java.sh

COPY install/ubuntu_install_antlr.sh /install/ubuntu_install_antlr.sh
RUN bash /install/ubuntu_install_antlr.sh

# Chisel deps for TSIM
COPY install/ubuntu_install_chisel.sh /install/ubuntu_install_chisel.sh
RUN bash /install/ubuntu_install_chisel.sh
Expand Down
3 changes: 0 additions & 3 deletions docker/Dockerfile.ci_gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,6 @@ RUN bash /install/ubuntu_install_vulkan.sh
COPY install/ubuntu_install_redis.sh /install/ubuntu_install_redis.sh
RUN bash /install/ubuntu_install_redis.sh

COPY install/ubuntu_install_antlr.sh /install/ubuntu_install_antlr.sh
RUN bash /install/ubuntu_install_antlr.sh

# NNPACK deps
COPY install/ubuntu_install_nnpack.sh /install/ubuntu_install_nnpack.sh
RUN bash /install/ubuntu_install_nnpack.sh
Expand Down
3 changes: 0 additions & 3 deletions docker/Dockerfile.ci_wasm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@ RUN bash /install/ubuntu1804_install_llvm.sh
COPY install/ubuntu_install_java.sh /install/ubuntu_install_java.sh
RUN bash /install/ubuntu_install_java.sh

COPY install/ubuntu_install_antlr.sh /install/ubuntu_install_antlr.sh
RUN bash /install/ubuntu_install_antlr.sh

COPY install/ubuntu_install_nodejs.sh /install/ubuntu_install_nodejs.sh
RUN bash /install/ubuntu_install_nodejs.sh

Expand Down
25 changes: 0 additions & 25 deletions docker/install/ubuntu_install_antlr.sh
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion docker/install/ubuntu_install_python_package.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ set -u
set -o pipefail

# install libraries for python package on ubuntu
pip3 install pylint==1.9.4 six numpy pytest cython decorator scipy tornado typed_ast pytest mypy orderedset antlr4-python3-runtime attrs requests Pillow packaging
pip3 install pylint==1.9.4 six numpy pytest cython decorator scipy tornado typed_ast pytest mypy orderedset attrs requests Pillow packaging
4 changes: 2 additions & 2 deletions docs/README.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,12 @@ You can run the following script to reproduce the CI sphinx pre-check stage.
This script skips the tutorial executions and is useful for quickly check the content.

```bash
./tests/scrpts/task_sphinx_precheck.sh
./tests/scripts/task_sphinx_precheck.sh
```

The following script runs the full build which includes tutorial executions.
You will need a gpu CI environment.

```bash
./tests/scrpts/task_python_docs.sh
./tests/scripts/task_python_docs.sh
```
7 changes: 0 additions & 7 deletions docs/install/from_source.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,13 +213,6 @@ like ``virtualenv``.

pip3 install --user tornado psutil xgboost

* If you want to build tvm to compile a model, you must use Python 3 and run the following

.. code:: bash

sudo apt install antlr4
pip3 install --user mypy orderedset antlr4-python3-runtime


Install Contrib Libraries
-------------------------
Expand Down
3 changes: 2 additions & 1 deletion include/tvm/ir/attrs.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,8 @@ struct AttrInitEntry {
~AttrInitEntry() DMLC_THROW_EXCEPTION {
if (value_missing_) {
std::ostringstream os;
os << type_key_ << ": Cannot find required field \'" << key_ << "\' during initialization";
os << type_key_ << ": Cannot find required field \'" << key_ << "\' during initialization."
<< "If the key is defined check that its type matches the declared type.";
throw AttrError(os.str());
}
}
Expand Down
19 changes: 15 additions & 4 deletions include/tvm/ir/span.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,21 +80,29 @@ class Span;
class SpanNode : public Object {
public:
/*! \brief The source name. */
SourceName source;
SourceName source_name;
/*! \brief The line number. */
int line;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe what you are trying to achieve here is the range for line[begin, end], column[begin, end].
nit: May be we can rename the variables to reflect the same.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think I agree with @ANSHUMAN87. Maybe its better to use "begin_line", "begin_column", "end_line", "end_column"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be begin_line, end_line, begin_column, end_column, as both values are related. Maybe even make a small range struct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kind of painful to refactor due some of the Python code. I still need to update the Python bindings how about I add to tracking issue and we can do in follow up work?

/*! \brief The column offset. */
int column;
/*! \brief The end line number. */
int end_line;
/*! \brief The end column number. */
int end_column;

// override attr visitor
void VisitAttrs(AttrVisitor* v) {
v->Visit("source", &source);
v->Visit("source_name", &source_name);
v->Visit("line", &line);
v->Visit("column", &column);
v->Visit("end_line", &end_line);
v->Visit("end_column", &end_column);
}

bool SEqualReduce(const SpanNode* other, SEqualReducer equal) const {
return equal(source, other->source) && equal(line, other->line) && equal(column, other->column);
return equal(source_name, other->source_name) && equal(line, other->line) &&
equal(column, other->column) && equal(end_line, other->end_line) &&
equal(end_column, other->end_column);
}

static constexpr const char* _type_key = "Span";
Expand All @@ -103,7 +111,10 @@ class SpanNode : public Object {

class Span : public ObjectRef {
public:
TVM_DLL Span(SourceName source, int lineno, int col_offset);
TVM_DLL Span(SourceName source_name, int line, int end_line, int column, int end_column);

/*! \brief Merge two spans into one which captures the combined regions. */
TVM_DLL Span Merge(const Span& other);

TVM_DEFINE_OBJECT_REF_METHODS(Span, ObjectRef, SpanNode);
};
Expand Down
2 changes: 1 addition & 1 deletion include/tvm/parser/parser.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
namespace tvm {
namespace parser {

IRModule Parse(std::string file_name, std::string file_content);
IRModule ParseModule(std::string file_name, std::string file_content);

} // namespace parser
} // namespace tvm
Expand Down
Loading