Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support manylinux build #165

Merged
merged 6 commits into from
Jan 16, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@
# VisualDL (Visualize the Deep Learning)

## Introduction
VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace.
VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace.
New features will be continuously added.

At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that


Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
can be integrated into other platforms.


Expand All @@ -27,8 +27,8 @@ VisualDL now provides 4 components:
- histogram

### Graph
Graph is compatible with ONNX(Open Neural Network Exchange)[https://github.com/onnx/onnx],
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
Graph is compatible with ONNX(Open Neural Network Exchange)[https://github.com/onnx/onnx],
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
PaddlePaddle, PyTorch and MXNet.

<p align="center">
Expand Down Expand Up @@ -75,7 +75,7 @@ logger = LogWriter(dir, sync_cycle=10)
with logger.mode("train"):
# create a scalar component called 'scalars/scalar0'
scalar0 = logger.scalar("scalars/scalar0")


# add some records during DL model running, lets start from another block.
with logger.mode("train"):
Expand All @@ -98,12 +98,12 @@ namepsace cp = visualdl::components;
int main() {
const std::string dir = "./tmp";
vs::LogWriter logger(dir, 10);

logger.SetMode("train");
auto tablet = logger.NewTablet("scalars/scalar0");

cp::Scalar<float> scalar0(tablet);

for (int step = 0; step < 1000; step++) {
float v = (float)std::rand() / RAND_MAX;
scalar0.AddRecord(step, v);
Expand Down Expand Up @@ -131,7 +131,7 @@ Board also supports the parameters below for remote access:
### How to install
```
python setup.py bdist_wheel
pip install --upgrade dist/visualdl-0.0.1-py2-none-any.whl
pip install --upgrade dist/visualdl-*.whl
```

### Run a demo from scratch
Expand All @@ -144,6 +144,6 @@ that will start a server locally.

### Contribute

VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/).
VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/).
We welcome everyone to use, comment and contribute to Visual DL :)
2 changes: 1 addition & 1 deletion build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ build_frontend_fake() {

build_backend() {
cd $BUILD_DIR
cmake ..
cmake .. ${PYTHON_FLAGS}
make -j2
}

Expand Down
2 changes: 1 addition & 1 deletion demo/mxnet/TUTORIAL_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ VisualDL的安装很简单。请按照VisualDL的[官方网站](https://github.c

```
python setup.py bdist_wheel
pip install --upgrade dist/visualdl-0.0.1-py2-none-any.whl
pip install --upgrade dist/visualdl-*.whl
```

## 开始编写训练MNIST的程序
Expand Down
17 changes: 10 additions & 7 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from distutils import sysconfig, dep_util, log
import setuptools.command.build_py
import setuptools
from setuptools import setup, find_packages
from setuptools import setup, find_packages, Distribution, Extension
import subprocess

TOP_DIR = os.path.realpath(os.path.dirname(__file__))
Expand Down Expand Up @@ -79,22 +79,25 @@ def run(self):
data_root = os.path.join(TOP_DIR, 'visualdl/server/dist')
for root, dirs, files in os.walk(data_root):
for filename in files:
path = 'dist/'+os.path.join(root, filename)[len(data_root)+1:]
path = 'dist/' + os.path.join(root, filename)[len(data_root) + 1:]
datas.append(path)
print datas

setup(
name="visualdl",
version=VERSION_NUMBER,
author="PaddlePaddle and Echarts team.",
description="Visualize Deep Learning.",
author="PaddlePaddle and Echarts team",
description="Visualize Deep Learning",
license=LICENSE,
keywords="visualization deeplearning",
long_description=read('README.md'),
install_requires=install_requires,
package_data={'visualdl.server': datas,
'visualdl':['core.so'],
'visualdl.python':['core.so', 'dog.jpg']},
package_data={
'visualdl.server': datas,
'visualdl': ['core.so'],
'visualdl.python': ['core.so', 'dog.jpg']
},
packages=packages,
ext_modules=[Extension('_foo', ['stub.cc'])],
scripts=['visualdl/server/visualDL', 'demo/vdl_scratch.py'],
cmdclass=cmdclass)
Empty file added stub.cc
Empty file.
5 changes: 3 additions & 2 deletions tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,13 @@ readonly TOP_DIR=$(pwd)
readonly core_path=$TOP_DIR/build/visualdl/logic
readonly python_path=$TOP_DIR/visualdl/python
readonly max_file_size=1000000 # 1MB
readonly version_number=`cat VERSION_NUMBER`

sudo="sudo"

if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then sudo=""; fi

if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
Expand Down Expand Up @@ -41,7 +42,7 @@ package() {

cd $TOP_DIR
python setup.py bdist_wheel
$sudo pip install dist/visualdl-0.0.1-py2-none-any.whl
$sudo pip install dist/visualdl-${version_number}-*.whl
}

backend_test() {
Expand Down
11 changes: 9 additions & 2 deletions visualdl/logic/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,19 @@ add_dependencies(im storage_proto)
add_dependencies(sdk entry storage storage_proto eigen3)

## pybind
set(OPTIONAL_LINK_FLAGS)

if(NOT APPLE)
set(OPTIONAL_LINK_FLAGS "rt")
endif()

add_library(core SHARED ${PROJECT_SOURCE_DIR}/visualdl/logic/pybind.cc)

if (NOT ON_RELEASE)
add_dependencies(core pybind python im entry tablet storage sdk protobuf glog eigen3)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf glog)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf glog ${OPTIONAL_LINK_FLAGS})
else()
add_dependencies(core pybind python im entry tablet storage sdk protobuf eigen3)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf ${OPTIONAL_LINK_FLAGS})
endif()
set_target_properties(core PROPERTIES PREFIX "" SUFFIX ".so")