Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Conda Installer #6604

Merged
merged 66 commits into from
Aug 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
b0aea46
reopen PR
deeleeramone Aug 1, 2024
493d852
updating some windows things
andrewkenreich Aug 1, 2024
fc8cea0
Merge branch 'develop' into feature/conda-installer
deeleeramone Aug 1, 2024
4767d8d
text file edit
deeleeramone Aug 1, 2024
8abe5d1
Merge branch 'feature/conda-installer' of https://github.com/OpenBB-f…
deeleeramone Aug 1, 2024
73153e3
couple touch ups
deeleeramone Aug 2, 2024
0fd3c5f
Merge branch 'develop' into feature/conda-installer
deeleeramone Aug 2, 2024
30d4be2
Merge branch 'develop' into feature/conda-installer
deeleeramone Aug 5, 2024
7e9a583
update post_install.sh to install via poetry
deeleeramone Aug 5, 2024
047fbdc
readme
deeleeramone Aug 5, 2024
e7544b5
poetry.toml
deeleeramone Aug 5, 2024
ae50d61
try windows scripts
deeleeramone Aug 5, 2024
6d59e8e
fixes
deeleeramone Aug 5, 2024
cf39276
vbs poetry env use
deeleeramone Aug 5, 2024
680c787
codespell
deeleeramone Aug 5, 2024
5b3aa2e
Merge branch 'develop' into feature/conda-installer
deeleeramone Aug 6, 2024
20a12df
expose uvicorn kwargs
deeleeramone Aug 7, 2024
cd2516c
try windows version of modified launcher
deeleeramone Aug 7, 2024
6239102
try something else
deeleeramone Aug 7, 2024
8c95889
try something else
deeleeramone Aug 7, 2024
879895b
vbs api shortcut
deeleeramone Aug 7, 2024
91b34a2
wrong shortcut
deeleeramone Aug 7, 2024
ffdac4b
launch windows shortcuts with cmd
deeleeramone Aug 7, 2024
2f8d2b9
windows conclusion script
deeleeramone Aug 7, 2024
f0e66e9
add WorkingDirectory
deeleeramone Aug 7, 2024
7122e36
windows python path is not in /bin
deeleeramone Aug 7, 2024
9ba3534
missed a shortcut WorkingDirectory
deeleeramone Aug 7, 2024
5c44795
notebook windows arg syntax
deeleeramone Aug 7, 2024
593916c
cwd for notebooks
deeleeramone Aug 7, 2024
93a3818
do a cd .. move
deeleeramone Aug 7, 2024
a1f5dd9
cli wrapper typo
deeleeramone Aug 7, 2024
8fd80f2
openbb-cli
deeleeramone Aug 7, 2024
e36b2da
merge conflicts
deeleeramone Aug 8, 2024
837c3ac
yaml
deeleeramone Aug 8, 2024
1e04b59
add conda
deeleeramone Aug 8, 2024
097fd09
Merge branch 'feature/conda-distribution' of https://github.com/OpenB…
deeleeramone Aug 8, 2024
0bd9232
merge branch develop
deeleeramone Aug 8, 2024
dd13aa5
fix reset_index erasing the name of the column (#6619)
deeleeramone Aug 9, 2024
bcc4433
handle edge case where user_settings is not formatted as expected
deeleeramone Aug 9, 2024
e86735b
add welcome message and entry point for python interface build script
deeleeramone Aug 9, 2024
818ca8a
Merge branch 'develop' of https://github.com/OpenBB-finance/OpenBB in…
deeleeramone Aug 9, 2024
2ceafb4
flag to not install conda on PATH
deeleeramone Aug 10, 2024
66146ae
add condarc
deeleeramone Aug 10, 2024
eefd841
condarc setting
deeleeramone Aug 10, 2024
f967886
update requirements.txt
deeleeramone Aug 10, 2024
c0c5f11
use poetry groups for flexible post-install config
deeleeramone Aug 11, 2024
a5c2bf1
update folder structure
deeleeramone Aug 12, 2024
52fa1a1
post install
deeleeramone Aug 12, 2024
a4347d9
try something for windows
deeleeramone Aug 12, 2024
d558d37
no nsi_template
deeleeramone Aug 12, 2024
6312eac
some updates
deeleeramone Aug 13, 2024
29b17c6
remove leading space.
deeleeramone Aug 13, 2024
ec93741
update aiohttp for dependabot (#6621)
deeleeramone Aug 14, 2024
fb04325
example extensions
deeleeramone Aug 15, 2024
3b74939
install examples script
deeleeramone Aug 15, 2024
31b2f19
update readme
deeleeramone Aug 16, 2024
b2ab84f
[BugFix] Fix Pandas FutureWarning In `derivatives.futures.curve` Char…
deeleeramone Aug 16, 2024
76777d7
codespell
deeleeramone Aug 16, 2024
07d8232
[BugFix] `obb.news.world` Add Validators To Providers Where Symbol Is…
deeleeramone Aug 17, 2024
8f7555b
data.__fields__ is deprecated - use data.model_fields instead (#6629)
deeleeramone Aug 19, 2024
04ee2a3
some updates
deeleeramone Aug 20, 2024
506f75e
Merge branch 'develop' of https://github.com/OpenBB-finance/OpenBB in…
deeleeramone Aug 20, 2024
a578d83
difflib check
deeleeramone Aug 20, 2024
d074360
merge prompt
deeleeramone Aug 20, 2024
e6fa654
windows updates
deeleeramone Aug 20, 2024
699fbad
cmd cwd
deeleeramone Aug 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .codespell.skip
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
./tests
.git
*.bat
*.css
*.csv
*.html
Expand All @@ -8,6 +9,7 @@
*.js
*.json
*.lock
*.nsi
*.scss
*.txt
*.yaml
Expand Down
33 changes: 23 additions & 10 deletions build/conda/README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,36 @@
# OpenBB Platform Installer

The code in this folder allows creating a packaged installer for the OpenBB Platform using conda constructor.
The code in this folder creates a packaged installer for the OpenBB Platform, using Conda Constructor.

The installers can be created for Windows (.exe), MacOS (.pkg).

The installer is distributed with a conda environment that contains python and package management tools.
The installer is created by conda constructor.
The installer is distributed with a Conda environment that contains Python and package management tools.

The dependencies required to run the application are installed from pypi by the constructor's post_install script.
The dependencies required to run the application are installed from PyPI by the constructor's post_install script,
and are defined in the `requirements.txt` file.

The version of Python is defined in the `construct.yaml` file, it is set for 3.12.

**Conda should be installed already, and initialized in the current shell profile.**

# Build Instructions

1. Create the build environment (see the [Build environment](#build-environment) section)
2. Build the installer (see the [Build command](#build-command) section)
Navigate into the `build/conda` folder and then begin.

1. Create the Conda environment.
- `conda env create -n constructor --file environments/constructor.yml`
2. Activate the environment.
- `conda activate constructor`
3. Build the installer.
- `constructor installer/.`


# For when we have a full NSIS

## Build environment
To check nsis file - run

The build environment should be created off the `environments/constructor-env.yml` file.
miniconda3\\envs\\constructor\\NSIS\\makensis.exe /V2 assets\\installer.nsi

## Build command
gives better output

In the `installer` folder run `constructor .` to build the installer.
miniconda3\\envs\\constructor\\NSIS\\makensis.exe /V4 assets\\installer.nsi
7 changes: 7 additions & 0 deletions build/conda/environments/constructor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: constructor
channels:
- conda-forge
- defaults
dependencies:
- conda-build
- constructor
180 changes: 180 additions & 0 deletions build/conda/installer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# OpenBB Platform

Thanks for installing the OpenBB Platform. To get the most out of your experience, we recommend reviewing the contents of this document.

If you have never used Python before, don't worry, your first time is just a double-click away. The sections below outline everything you need to know for getting started.

## What Did I Install?

This is a kitted out development container for Pythonic, open source, financial research. It has everything you need to get started immediately.

Included, is a complete installation of the OpenBB Platform and CLI, along with Conda and Jupyter Notebook, in an isolated Python 3.12 Conda environment.

If you already have an installation of Conda or Anaconda, the version in this folder will not rely on, interfere, or share base environment paths with, existing versions.

Tools for managing, updating, and building environments allow you to take full control of your installation, as well as quickly spin up new containers within this project folder.

To get started using right away, open one of the shortcuts described below.

If you want to build your own end points, connections, and features, try installing the [example extensions](#examples)

## Shortcuts

At the root of the installation folder (where this document is) are shortcuts for launchers and to key file locations.

- Bash (Windows: CMD): Opens a command line shell with the base Conda environment activated. Activate the OpenBB environment with:
- Mac/Linux: `source activate obb`
- Windows: `conda activate obb`
- Environments: Shortcut to the root "envs" folder, where the `obb` and any created environments are stored.
- "widgets.json" is located in the "assets" subdirectory.
- openbb-api: Opens a Terminal window and launches the OpenBB API, with optional prompts to login with your OpenBB Hub PAT.
- openbb-cli: Opens a Terminal window and launches the OpenBB CLI (formerly OpenBB Terminal).
- openbb-ipython: Opens a Terminal window and starts an IPython session with the OpenBB Platform package imported.
- openbb-notebook: Opens a Terminal window, starts the Jupyter development server, and opens a browser window with the Notebook application.
- OpenBBUserData: Shortcut to the OpenBBUserData folder. Files exported from CLI are saved here.
- Settings: Shortcut to the `~/.openbb_platform` location where `.env` files and `user_settings.json` are stored.
- If you cannot see the ".env" file, set Finder/Explorer to display hidden/system files.
- Update: Opens a Terminal window and updates the OpenBB Platform environment (obb) and packages.

## Command Line Entry Points

There are several command line entry points available when either:
- The OpenBB Platform environment (obb) is active.
- The "openbb_platform" package has been installed in the active Python environment, including when `--no-deps` or `--only-root` flags were used to install.

### OpenBB Entry Points

- openbb: Launches the OpenBB CLI.
- openbb-api: Launches the API and accepts select arguments.
- All args/kwargs available to `uvicorn run` are exposed.
- `--login`: True/False. Defeats the login prompt when False.
- `--no-build`: Skips building "widgets.json"
- To launch the API with no input prompts enter: `openbb-api --login False --no-build
- openbb-build: Runs the build script that generates the static assets for the OpenBB Pythobn interface.
- Run this after installing/uninstalling/updating OpenBB extensions
- Automatically run when `openbb-update` is run.
- openbb-update: Updates the environment packages defined in `pyproject.toml` and `poetry.lock` and rebuilds the OpenBB Python interface.
- Location: "/extensions/openbb_platform"
- Passes all args/kwargs to `poetry install`.

## Installed Folder Structure

At the root of the installation (where this document is), there will be two folders:

- conda (Windows: This is named after the last folder path if the default was not used)
- extensions

### Conda

This is the folder where Conda is installed and all environment data (site-packages, etc.) is stored. The "Environments" shortcuts takes you to the root of the "envs" folder.

### Extensions

This folder contains two subfolders:

#### examples

Contains empty example extensions for the three types of OpenBB Platform extensions:

- Router
- Provider
- OBBject

To install all three examples in the OpenBB Platform environment, open the Bash/CMD shortcut and enter:

```console
source activate obb # CMD: conda activate obb
cd extensions/examples
python install_examples.py
```

#### openbb_platform

This is a meta package for installing and managing OpenBB Platform installation within any Python environment.
Poetry is used to update and resolve any dependencies that are defined in the `pyproject.toml` file.

The `poetry.lock` file is updated every time the `Update` shortcut is run.

There are several configurations available, via combinations of `--with`, `--without`, and `--only` parameters, when running `poetry install`.

Choices are:

- main
- openbb-all
- cli
- notebook

Each parameter will accept multiple items, but `--without` takes priority for solving for the environment.

## Uninstallation

Mac/Linux users can uninstall by deleting the folder, while Windows users can run the Uninstall shortcut.

The leftover artifacts will include:

- Global configuration files stored by Poetry, PIP, Conda, and Jupyter.
- Third-party Python package caches and `.env` files.
- Folders:
- ~/.openbb_platform
- ~/OpenBBUserData

Windows users will also need to remove the parent folder where it was originally installed, after running Uninstall.

Your OpenBB Hub account can be deleted, along with all associated data, from the Account page in [my.openbb.co](https://my.openbb.co)

## Additional Information

### Conda Install vs. PIP Install vs. Poetry Install

The three ways to install packages may appear to accomplish the same objectives - installing Python packages - but each has a distinctly different purpose.

An easy way to think about the difference is, both `pip` and `poetry` require an existing Python installation; `conda` installs Python so that you can run `pip install poetry`.

In general, you use:

- `conda`
- When a new environment has been created. Packages here can be included in the creation command, `conda create`.
- When the specific package is only distributed through Conda or the `conda-forge` channel.
- When changing the version of Python used within the environment.
- `pip`
- When installing/uninstalling packages to the activated Python environment.
- Resort to `conda install` only if packages fail to install or wheels cannot be built.
- `poetry`
- Installing a local Python package.
- Syncing all package dependencies according to provided specs in `pyproject.toml` and `poetry.lock` files.
- Building and distributing Python packages.

Conda is for container management, Poetry resolves the project's dependencies and provides build/distribution tools, while `pip` is for installing from [PyPI](https://pypi.org)

### Documentation Resources

### OpenBB
- [Quckstart](https://docs.openbb.co/platform/getting_started/quickstart)
- [OpenBB Pro](https://docs.openbb.co/pro)
- [OpenBB Pro Data Connectors](https://docs.openbb.co/pro/data-connectors)
- [OpenBB Platform](https://docs.openbb.co/platform)
- [OpenBB CLI](https://docs.openbb.co/cli)
- [OpenBB User Settings](https://docs.openbb.co/platform/user_guides/settings_and_environment_variables)
- [API Keys and Authorization](https://docs.openbb.co/platform/getting_started/api_keys)
- [REST API Requests](https://docs.openbb.co/platform/getting_started/api_requests)
- [OpenBB Platform Architecture Overview](https://docs.openbb.co/platform/developer_guide/architecture_overview)
- [Create A New Router Extension](https://docs.openbb.co/platform/getting_started/create_new_router_extension)
- [OpenBB Charting]()
- [PyWry](https://github.com/OpenBB-finance/pywry)

### External

- [python](https://docs.python.org/3/)
- [conda create](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#managing-environments)
- [pyproject.toml](https://python-poetry.org/docs/pyproject/)
- [requirements.txt](https://pip.pypa.io/en/stable/reference/requirements-file-format/)
- [pip](https://pip.pypa.io/en/stable/)
- [poetry config](https://python-poetry.org/docs/configuration)
- [pydantic](https://docs.pydantic.dev/latest/)
- [uvicorn](https://www.uvicorn.org/)
- [fastapi](https://fastapi.tiangolo.com/)
- [jupyter](https://docs.jupyter.org/en/latest/)
- [pandas](https://pandas.pydata.org/docs/)
- [numpy](https://numpy.org/doc/)
- [Plotly Graph Objects](https://plotly.com/python/graph-objects/)

Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
76 changes: 65 additions & 11 deletions build/conda/installer/assets/create_shortcut.vbs
Original file line number Diff line number Diff line change
@@ -1,16 +1,70 @@
Set objShell = CreateObject("WScript.Shell")
sDesktopPath = objShell.SpecialFolders("Desktop")
sdesktopPath = objShell.SpecialFolders("Desktop")

' Create shortcut for openbb-cli
Set objShortcut = objShell.CreateShortcut(sDesktopPath & "\openbb-cli.lnk")
objShortcut.TargetPath = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\Scripts\openbb-cli.exe"
objShortcut.IconLocation = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\assets\openbb.ico"
objShortcut.Description = "OpenBB CLI launcher"
Set fso = CreateObject("Scripting.FileSystemObject")
prefixPath = objShell.ExpandEnvironmentStrings("%PREFIX%")
userProfilePath = objShell.ExpandEnvironmentStrings("%USERPROFILE%")
currentPath = fso.GetAbsolutePathName(prefixPath)
parentFolder = fso.GetParentFolderName(currentPath)
shortcutFolder = parentFolder

Set objFSO = CreateObject("Scripting.FileSystemObject")
If Not objFSO.FolderExists(shortcutFolder) Then
objFSO.CreateFolder(shortcutFolder)
End If

If prefixPath = "%PREFIX%" Or userProfilePath = "%USERPROFILE%" Then
WScript.Echo "Environment variables %PREFIX% or %USERPROFILE% are not set correctly."
WScript.Quit 1
End If

Sub CreateShortcut(name, targetPath, iconPath)
Set objShell = CreateObject("WScript.Shell")
Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\" & name & ".lnk")
objShortcut.TargetPath = objShell.ExpandEnvironmentStrings(targetPath)
objShortcut.IconLocation = objShell.ExpandEnvironmentStrings(iconPath)
objShortcut.Description = name
objShortcut.Save
End Sub

CreateShortcut "openbb-cli", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "openbb-api", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "openbb-notebook", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "openbb-ipython", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "Update", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "CMD", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "Environments", prefixPath & "\envs", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "Settings", userProfilePath & "\.openbb_platform", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "OpenBBUserData", userProfilePath & "\OpenBBUserData", prefixPath & "\assets\openbb_icon.ico"
CreateShortcut "Uninstall", prefixPath & "\Uninstall-OpenBB.exe", prefixPath & "\assets\openbb_icon.ico"

Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-notebook.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && cd " & userProfilePath & " && jupyter-notebook && exit"
objShortcut.WorkingDirectory = prefixPath
objShortcut.Save

Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-ipython.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && ipython -c ""from openbb import obb;obb"" -i"
objShortcut.WorkingDirectory = prefixPath
objShortcut.Save

' Create shortcut for openbb-api
Set objShortcut = objShell.CreateShortcut(sDesktopPath & "\openbb-api.lnk")
objShortcut.TargetPath = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\Scripts\openbb-api.exe"
objShortcut.IconLocation = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\assets\openbb.ico"
objShortcut.Description = "OpenBB API launcher"
Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-api.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-api --login && exit"
objShortcut.WorkingDirectory = prefixPath
objShortcut.Save


Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\Update.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-update && exit"
objShortcut.WorkingDirectory = prefixPath
objShortcut.Save

Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\CMD.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate base && echo Conda base environment is active. Use this shell to create new environments. && echo To activate the OpenBB environment, run 'conda activate obb'."
objShortcut.WorkingDirectory = shortcutFolder
objShortcut.Save

Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-cli.lnk")
objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb && exit"
objShortcut.WorkingDirectory = prefixPath
objShortcut.Save
Loading
Loading