Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise about #295

Merged
merged 5 commits into from
Jun 7, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions docs/_static/pangeo-style.css
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,17 @@ a:hover, a:focus {
}

.navbar-inverse .navbar-nav>li>a:hover, .navbar-inverse .navbar-nav>li>a:focus {
background-color: #e50051;
background-color: #00a3b0;
}

.navbar-inverse .navbar-nav>.open>a, .navbar-inverse .navbar-nav>.open>a:hover, .navbar-inverse .navbar-nav>.open>a:focus {
background-color: #e50051;
background-color: #00a3b0;
}

.dropdown-menu>li>a:hover, .dropdown-menu>li>a:focus {
text-decoration: none;
/* color: #ffffff; */
background-color: #00a3b0;
}

.navbar-brand {
Expand Down
87 changes: 33 additions & 54 deletions docs/about.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,31 @@
About Pangeo
============

Pangeo is first and foremmost a :ref:`community of people <collaborators>`
working collaboratively to develop software and infrastructure to enable
Big Data geoscience research.

Some of the products produced by this commmunity include interconnected
:ref:`software pacakge <packages>` and :ref:`deployments <deployments>` of
this software in cloud and high-performance-computing environments. Such
a deployment is sometimes referred to as a *Pangeo Environment*.

Motivation
----------

There are several building crises facing the Atmosphere / Ocean / Land /
Climate (AOC) science community:
There are several building crises facing the geoscience community:

- Big Data: datasets are growing too rapidly and legacy software tools
- *Big Data*: datasets are growing too rapidly and legacy software tools
for scientific analysis can't handle them. This is a major obstacle
to scientific progress.
- Technology Gap: a growing gap between the technological
- *Technology Gap*: a growing gap between the technological
sophistication of industry solutions (high) and scientific software
(low).
- Reproducibility: a fragmentation of software tools and environments
renders most AOC research effectively unreproducible and prone to
- *Reproducibility*: a fragmentation of software tools and environments
renders most geoscience research effectively unreproducible and prone to
failure.

We believe these challenges can all be addressed through a unified
effort.
Pangeo aims to address these challenges through a unified, collaborative effort.

Mission Statement
-----------------
Expand All @@ -30,56 +37,28 @@ open-source analysis tools for ocean, atmosphere and climate science can
be developed, distributed, and sustained. These tools must be scalable
in order to meet the current and future challenges of big data, and
these solutions should leverage the existing expertise outside of the
AOC community.

Vision
------

We envision a collection of related but independent open-source packages
that meet specific scientific needs within the AOC fields. These
packages will follow modern best practices for software development,
including:

- hosting on GitHub,
- testing,
- coverage,
- continuous integration,
- comprehensive documentation, and
- a welcoming and inclusive development culture.

As much as possible, we will build on top of existing solutions and
leverage expertise from the broader technology world, including
industry.

In practice, the "python data" software stack (see above) currently
provides the most stable and powerful foundation layer for our desired
tools. In particular the xarray and dask projects provide a mechanism to
easily build scalability into scientific analysis. Our vision of future
AOC software involves the adoption of these common software layers, and
a clear communication between developers to define project scope and
dependency that eliminates redundancy and fragmentation.
geoscience community.

Get Involved
------------
Goals
-----

The scientific culture in the AOC community must be tied to, and evolve
from, the community's software culture. Hence, we depend upon
contributions from the entire community, both scientific and industrial.
To accomplish this mission, we have identified three specific goals.

We encourage everyone to get involved by:
#. Foster collaboration around the open source scientific python ecosystem for
ocean / atmosphere / land / climate science.
#. Support the development with domain-specific geoscience packages.
#. Improve scalability of these tools to to handle petabyte-scale datasets on
HPC and cloud platforms.

- contributing to the goals and vision of the organization,
- contributing to the design documents of the proposed software,
- contributing to the software, via issues and pull requests, and/or
- using the software for your scientific analysis and letting us know
about your experiences (e.g., contributing to examples)

For now, community discussion is happening on our `pangeo google
group <https://groups.google.com/forum/#!forum/pangeo>`__. This is an
open group, and we invite anyone interested to join.
Get Involved
------------

The pangeo project is completely open to involvement from anyone with interest.
There are many ways to get involved:

Pangeo receives support from the NSF_ via the EarthCube_ program.
- *Science users*: read the :ref:`quickstart`, browse the :ref:`use-cases`,
read about the :ref:`packages`, or visit a :ref:`deployment <deployments>`.
- *Developers / system administrators*: learn about the
:ref:`Technical Architecture` or read the :ref:`setup-guides`.

.. _NSF: https://pangeo-data.github.io/
.. _EarthCube: https://earthcube.org/
For more information, consult the :ref:`FAQ`.
6 changes: 5 additions & 1 deletion docs/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,11 @@ datasets.
These datasets are directly accessible from the Google Cloud Pangeo deployment:
`pangeo.pydata.org <http://pangeo.pydata.org>`_.

**HOW DO WE LIST THE DATASETS? CAN WE DIRECTLY PARSE THE INTAKE CATALOG?**
+--------+--------------+
| Name | Description |
+========+==============+
| Under | Construction |
+--------+--------------+


.. _cloud-data-guide:
Expand Down
23 changes: 23 additions & 0 deletions docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,26 @@ Frequently Asked Questions
has the modularity and collaborative structure required to achieve our goals.

#. *Why don't you use Julia or R?*

We would like Pangeo to evolve into a multi-language platform, including
support for Julia, R, and other programming languages used for data science.
Because Pangeo uses Jupyter to provide interactive commputing, this is
already possible today; Jupyter can be configured to run Kernels in many
different languages (see
`Jupyter docs on Kernels <http://jupyter.readthedocs.io/en/latest/projects/kernels.html>`_).
These kernels can be configured to run on any of the
Pangeo :ref:`Deployments`. However, the higher-level features of the Pangeo
platform, in particular, parallel execution with Dask, are limited to
Python at the moment. If you would like to work on enabling multi-language
support, please engage with us via the `Pangeo GitHub issue tracker`_.

#. *Can I use Pangeo for X?*

Yes! Our goal is to point you towards the building blocks to create your
own Pangeo, whatever that means for you. If you want to create up a custom
Pangeo environment, start by consulting the :ref:`setup-guides`. If you
want help or want to discuss customization options, reach out via the
`Pangeo GitHub issue tracker`_.


.. _Pangeo GitHub issue tracker: https://github.com/pangeo-data/pangeo/issues
40 changes: 28 additions & 12 deletions docs/packages.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
.. _packages:

Packages
========

Expand Down Expand Up @@ -42,6 +40,27 @@ as well as powerful tools for computation and visualization.
.. _Common Data Model: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM
.. _netCDF: http://www.unidata.ucar.edu/software/netcdf


Iris
~~~~

.. image:: _static/Iris_logo_banner.png

- Website: https://scitools.org.uk/iris/docs/latest/
- GitHub: https://github.com/SciTools/iris

Iris seeks to provide a powerful, easy to use, and community-driven Python
library for analysing and visualising meteorological and oceanographic data sets.

With Iris you can:

- Use a single API to work on your data, irrespective of its original format.
- Read and write (CF-)netCDF, GRIB, and PP files.
- Easily produce graphs and maps via integration with matplotlib and cartopy.

Iris is an alternative to Xarray. Iris is developed primarily by the
`UK Met Office Informatics Lab <http://www.informaticslab.co.uk/>`_.

Dask
~~~~

Expand Down Expand Up @@ -77,11 +96,11 @@ live.
Pangeo Affiliated Packages
--------------------------

xgcm
~~~~

xesmf
~~~~~
There are many other python packages that can work with the core packages
to provide additional functionality.
We plan to eventually catalog these packages here on the Pangeo website.
For now, please refer to the
`Xarray list of related projects <http://xarray.pydata.org/en/latest/faq.html#what-other-projects-leverage-xarray>`_.


Guidelines for New Packages
Expand Down Expand Up @@ -128,12 +147,9 @@ additional recommendations.
inevitable as developers try implementing different solutions to the
same general problems.)
4. *Consume and Produce Xarray Objects:* Xarray data structures
facilitate mutual interoperability between packages. (For more about
Xarray , see below.)
facilitate mutual interoperability between packages.
5. *Operate Lazily:* whenever possible, packages should avoid explicitly
triggering computation on
`Dask <http://dask.pydata.org/en/latest/array.html>`__ objects. (For
more about Dask, see below)
triggering computation on Dask objects.

Why Xarray and Dask?
~~~~~~~~~~~~~~~~~~~~
Expand Down
2 changes: 2 additions & 0 deletions docs/setup_guides/cloud.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ We hope to soon expand it to include AWS and Azure.
Step One: Install the Necessary Software
----------------------------------------

.. _google-cloud-sdk:

Google Cloud SDK
~~~~~~~~~~~~~~~~

Expand Down