Skip to content

Building ANTLR 4

Jump to bottom
linuxonz edited this page Oct 13, 2023 · 40 revisions

Building ANTLR v4.13.1

Below are the versions of ANTLR (ANother Tool for Language Recognition) that are available in their respective distributions at the time of creation of these build instructions:

  • Ubuntu 20.04 has 4.7.2-2
  • Ubuntu (22.04, 23.04) have 4.7.2-5

The instructions provided below specify the steps to build ANTLR v4.13.1 on Linux on IBM Z for the following distributions:

  • RHEL (7.8, 7.9, 8.6, 8.8, 9.0, 9.2)
  • SLES (12 SP5, 15 SP4, 15 SP5)
  • Ubuntu (20.04, 22.04, 23.04)

If only the Java runtime is required, then the pre-built jars for ANTLR v4.13.1 can be downloaded from here.

If you want to build AntLR4 using manual steps, go to STEP 1

Use the following commands to build AntLR4 using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/AntLR/4.13.1/build_antlr4.sh

# Build antlr4, skipping test cases
bash build_antlr4.sh [Provide -j (Semuru11|Semuru17|Temurin11|Temurin17|OpenJDK11|OpenJDK17) option for building with specific java]

The instructions below describe:

  1. Installing some tools needed for the installation or building.
  2. How to download and install the required Java SDK.
  3. If only the ANTLR Java runtime is required - how to download and install ANTLR pre-built binaries
  4. If other ANTLR build targets are required - how to download and install the required packages, build ANTLR and the supported build targets from source.

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Install dependencies.

  • RHEL (7.8, 7.9)

    sudo yum install -y wget tar which curl diff

  • RHEL (8.6, 8.8, 9.0, 9.2)

    sudo yum install -y wget tar which curl diffutils

  • SLES (12 SP5, 15 SP4, 15 SP5)

    sudo zypper install -y wget gzip tar curl xz diffutils

  • Ubuntu (20.04, 22.04, 23.04)

    sudo apt-get update
sudo apt-get install -y wget tar curl diffutils

Step 2: Install Java SDK

2.1) Download and install Java

  • With IBM Semeru Runtime (previously known as AdoptOpenJDK openj9)

    • Download and install IBM Semeru Runtime (Java 11, 17) from here.

  • With Eclipse Adoptium Temurin Runtime (previously known as AdoptOpenJDK hotspot)

    • Download and install Eclipse Adoptium Temurin Runtime (Java 11, 17) from here.

  • With OpenJDK:

    • RHEL (7.8, 7.9, 8.6, 8.8, 9.0, 9.2)

      # For JAVA 11
sudo yum install -y java-11-openjdk-devel

# For JAVA 17 on RHEL 8.x, 9.x
sudo yum install -y java-17-openjdk-devel

    • SLES (12 SP5, 15 SP4, 15 SP5)

      # For JAVA 11
sudo zypper install -y java-11-openjdk java-11-openjdk-devel

# JAVA 17 for SLES15-sp4 only
sudo zypper install -y java-17-openjdk-17.0.4.0-150400.3.3.1 java-17-openjdk-devel-17.0.4.0-150400.3.3.1

    • Ubuntu (20.04, 22.04, 23.04)

      # For JAVA 11
sudo apt-get update
sudo apt-get install -y openjdk-11-jre openjdk-11-jdk

# For JAVA 17
sudo apt-get update
sudo apt-get install -y openjdk-17-jre openjdk-17-jdk

Note: At the time of creation of these build instructions, ANTLR was verified with Eclipse Adoptium Temurin Runtim (build 11.0.19+7, 17.0.7+7) and IBM Semeru Runtime Open Edition (build 11.0.18+10, 17.0.6+10).

2.2) Set Environment variables

export SOURCE_ROOT=/<source_root>/
export JAVA_HOME=<path to installed java>
export PATH=$JAVA_HOME/bin:$PATH

Step 3: If only the ANTLR Java runtime is required

3.1) Download and install ANTLR pre-built binaries

cd $SOURCE_ROOT
curl -s -S -L -O https://www.antlr.org/download/antlr-4.13.1-complete.jar
JAR=$SOURCE_ROOT/antlr-4.13.1-complete.jar
export CLASSPATH=".:$JAR:$CLASSPATH"

3.2) Test the ANTLR tool and Java runtime

Test the tool and runtime using the example provided here.

alias antlr4='java -Xmx500M -cp $CLASSPATH org.antlr.v4.Tool'
alias grun='java -Xmx500M -cp $CLASSPATH org.antlr.v4.gui.TestRig'

cd $SOURCE_ROOT
mkdir antlrtest
cd antlrtest
cat > Hello.g4 <<'EOF'
  grammar Hello;
  r  : 'hello' ID ;
  ID : [a-z]+ ;
  WS : [ \t\r\n]+ -> skip ;
EOF

antlr4 Hello.g4
javac Hello*.java
diff -w <(grun Hello r -tree <<<"hello world") <(echo "(r hello world)")

Step 4: If other ANTLR runtimes are required

4.1) Install the required packages

  • RHEL (7.8, 7.9)
sudo yum install -y unzip xz xz-devel libuuid-devel curl wget git make diffutils devtoolset-11-gcc \
  devtoolset-11-gcc-c++ llvm-toolset-11.0 python2 bzip2-devel gdbm-devel libdb-devel libffi-devel ncurses-devel \
  readline-devel sqlite-devel tar tk-devel zlib-devel openssl-devel libcurl-devel

#switch to GCC 11
source /opt/rh/llvm-toolset-11.0/enable
source /opt/rh/devtoolset-11/enable
  • RHEL (8.6, 8.8)
sudo yum install -y unzip xz libuuid-devel curl wget git make diffutils gcc gcc-c++ python2 python38 cmake libarchive \
  clang
  • RHEL (9.0, 9.2)
sudo yum install -y unzip xz libuuid-devel curl wget git make diffutils gcc gcc-c++ python3 cmake libarchive clang
  • Ubuntu 20.04
sudo apt-get update
sudo apt-get install -y unzip xz-utils uuid-dev curl wget git make python python3.8 gcc g++ cmake clang pkg-config \
  libssl-dev libcurl4-openssl-dev zlib1g-dev
  • Ubuntu 22.04
sudo apt-get update
sudo apt-get install -y unzip xz-utils uuid-dev curl wget git make python2 bzip2 tk-dev libghc-bzlib-dev gcc g++ cmake \
  clang pkg-config
  • Ubuntu 23.04
sudo apt-get update
sudo apt-get install -y unzip xz-utils uuid-dev curl wget git make bzip2 tk-dev libghc-bzlib-dev gcc g++ cmake clang \
  pkg-config
  • SLES 12 SP5
sudo zypper install -y unzip tar xz xz-devel libuuid-devel curl wget git make diffutils gcc8 gcc8-c++ gcc11 gcc11-c++ \
  python python-typing cmake gawk gdbm-devel libbz2-devel libdb-4_8-devel libffi48-devel ncurses-devel readline-devel \
  sqlite3-devel tk-devel zlib-devel openssl-devel libcurl-devel

#switch to GCC 11
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-11 50
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 50
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 50
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-11 50
  • SLES (15 SP4, 15 SP5)
sudo zypper install -y unzip xz xz-devel libuuid-devel curl wget git make diffutils gcc11 gcc11-c++ python cmake \
  clang7 gawk gdbm-devel libbz2-devel libdb-4_8-devel libffi-devel libnsl-devel libopenssl-devel libuuid-devel make \
  ncurses-devel readline-devel sqlite3-devel tar tk-devel zlib-devel gzip

#switch to GCC 11
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-11 40
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 40
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 40
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-11 40

4.2) Download(or Install) required binaries(packages)

  • Install Go 1.19.5, then use the following command to set the GO111MODULE environment variable to auto:
cd $SOURCE_ROOT
export GO_VERSION="1.19.5"
export GOROOT=$SOURCE_ROOT/go
wget -q https://storage.googleapis.com/golang/go"$GO_VERSION".linux-s390x.tar.gz
sudo tar -xzf go"$GO_VERSION".linux-s390x.tar.gz
export PATH=$GOROOT/bin:$PATH
export CC=gcc
go env -w GO111MODULE=auto
go version
  • Build and install OpenSSL from source (Only on RHEL 7.x and SLES 12 SP5)
cd $SOURCE_ROOT
wget https://www.openssl.org/source/openssl-1.1.1h.tar.gz
tar -xzf openssl-1.1.1h.tar.gz
cd openssl-1.1.1h
./config --prefix=/usr/local --openssldir=/usr/local
make
sudo make install

export LDFLAGS="-L/usr/local/lib/ -L/usr/local/lib64/"
export LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/lib64/:/usr/lib/:/usr/lib64/${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export CPPFLAGS="-I/usr/local/include/ -I/usr/local/include/openssl"
  • Install Python 3 (Only on RHEL 7.x, SLES 12 SP5)
cd $SOURCE_ROOT
ver=3.8.8
wget https://www.python.org/ftp/python/$ver/Python-$ver.tgz
tar -xzf Python-$ver.tgz
cd Python-$ver
./configure
make && sudo make install
python3 -V
  • Install cmake (Only on RHEL 7.x, SLES 12 SP5)
cd $SOURCE_ROOT
wget https://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2.tar.gz
tar -xzf cmake-3.24.2.tar.gz
cd cmake-3.24.2
./bootstrap --system-curl
make
sudo make install
export PATH=/usr/local/bin:$PATH

  • Install Clang 8.0.1 (Only on SLES 12 SP5)

    • Build and Install LLVM Dependency
    cd $SOURCE_ROOT
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-8.0.1/llvm-8.0.1.src.tar.xz
tar -xf llvm-8.0.1.src.tar.xz
cd llvm-8.0.1.src/
mkdir build && cd build
cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=release -DCMAKE_C_COMPILER=/usr/bin/gcc-8 -DCMAKE_CXX_COMPILER=/usr/bin/g++-8 ..
make && sudo make install
    • Build and Install CLANG
    cd $SOURCE_ROOT
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-8.0.1/cfe-8.0.1.src.tar.xz
tar -xf cfe-8.0.1.src.tar.xz
cd cfe-8.0.1.src/
mkdir build && cd build
cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=release -DCMAKE_C_COMPILER=/usr/bin/gcc-8 -DCMAKE_CXX_COMPILER=/usr/bin/g++-8 ..
make && sudo make install

  • Download NodeJS, Maven and ANTLR source for all distros.

cd $SOURCE_ROOT

export NODE_TAR=node-v16.17.1-linux-s390x.tar.xz
curl -s -S -L -O https://nodejs.org/dist/v16.17.1/${NODE_TAR}
tar xJf ${NODE_TAR}

export MAVEN_TAR=apache-maven-3.8.6-bin.tar.gz
curl -s -S -L -O https://archive.apache.org/dist/maven/maven-3/3.8.6/binaries/${MAVEN_TAR}
tar xzf ${MAVEN_TAR}

export ZIP=antlr4-4.13.1.zip
curl -s -S -L -o ${ZIP} https://github.com/antlr/antlr4/archive/4.13.1.zip
unzip -q ${ZIP}

4.3) Build ANTLR and the supported runtimes

  • Set the PATH environment variable
export PATH=$SOURCE_ROOT/node-v16.17.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.8.6/bin:${PATH}
  • Build the ANTLR tool, then build the supported runtimes.
export MAVEN_OPTS="-Xmx1G"

cd $SOURCE_ROOT/antlr4-4.13.1
mvn install -DskipTests=true
cd runtime/Cpp
mkdir build && mkdir run
cd build
cmake -DWITH_LIBCXX=Off -DCMAKE_BUILD_TYPE=release ..
make -j 8
DESTDIR=$SOURCE_ROOT/antlr4-4.13.1/runtime/Cpp/run make install

Note: If multiple gcc versions are installed, DCMAKE_CXX_COMPILER and DCMAKE_C_COMPILER flags can be used in cmake command to give correct version path.
Note: ANTLR 4.13.1 runtimes supported for Linux on IBM Z are Java, Python, Go, NodeJS and C++.

Step 5: Test (Optional)

cd $SOURCE_ROOT/antlr4-4.13.1/runtime-testsuite
mvn -Dtest=java.** test
mvn -Dtest=python2.** test     # except for RHEL 9.x, SLES 15.x, and Ubuntu 23.04 as Python 2 has been removed from these distros
mvn -Dtest=python3.** test
sudo env "PATH=$PATH" "GOROOT=$GOROOT" mvn -Dtest=go.** test
mvn -Dtest=javascript.** test
mvn -Dtest=cpp.** test

References:

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally