Virtual graph #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

niclasko merged 22 commits into main from virtual_graph

Jan 24, 2026

.github/workflows/python-publish.yml

-Original file line number
+Diff line change
@@ -0,0 +1,97 @@
+    name: Publish Python Package to PyPI
+    on:
+      push:
+        branches:
+          - main
+        paths:
+          - "flowquery-py/**"
+      pull_request:
+        branches:
+          - main
+        paths:
+          - "flowquery-py/**"
+      release:
+        types: [published]
+      workflow_dispatch:
+        inputs:
+          target:
+            description: "Publish target"
+            required: true
+            default: "testpypi"
+            type: choice
+            options:
+              - testpypi
+              - pypi
+    jobs:
+      build:
+        name: Build distribution
+        runs-on: ubuntu-latest
+        defaults:
+          run:
+            working-directory: flowquery-py
+        steps:
+          - uses: actions/checkout@v4
+          - name: Set up Python
+            uses: actions/setup-python@v5
+            with:
+              python-version: "3.12"
+          - name: Install build dependencies
+            run: pip install build
+          - name: Build package
+            run: python -m build
+          - name: Upload distribution artifacts
+            uses: actions/upload-artifact@v4
+            with:
+              name: python-package-distributions
+              path: flowquery-py/dist/
+      publish-testpypi:
+        name: Publish to TestPyPI
+        if: github.event_name == 'workflow_dispatch' && github.event.inputs.target == 'testpypi'
+        needs: build
+        runs-on: ubuntu-latest
+        environment:
+          name: testpypi
+          url: https://test.pypi.org/p/flowquery
+        permissions:
+          id-token: write
+        steps:
+          - name: Download distribution artifacts
+            uses: actions/download-artifact@v4
+            with:
+              name: python-package-distributions
+              path: dist/
+          - name: Publish to TestPyPI
+            uses: pypa/gh-action-pypi-publish@release/v1
+            with:
+              repository-url: https://test.pypi.org/legacy/
+      publish-pypi:
+        name: Publish to PyPI
+        if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.target == 'pypi')
+        needs: build
+        runs-on: ubuntu-latest
+        environment:
+          name: pypi
+          url: https://pypi.org/p/flowquery
+        permissions:
+          id-token: write
+        steps:
+          - name: Download distribution artifacts
+            uses: actions/download-artifact@v4
+            with:
+              name: python-package-distributions
+              path: dist/
+          - name: Publish to PyPI
+            uses: pypa/gh-action-pypi-publish@release/v1

flowquery-py/.gitignore

-Original file line number
+Diff line change
@@ -0,0 +1,84 @@
+    # Byte-compiled / optimized / DLL files
+    __pycache__/
+    *.py[cod]
+    *$py.class
+    # C extensions
+    *.so
+    # Distribution / packaging
+    .Python
+    build/
+    develop-eggs/
+    dist/
+    downloads/
+    eggs/
+    .eggs/
+    lib/
+    lib64/
+    parts/
+    sdist/
+    var/
+    wheels/
+    *.egg-info/
+    .installed.cfg
+    *.egg
+    # PyInstaller
+    *.manifest
+    *.spec
+    # Installer logs
+    pip-log.txt
+    pip-delete-this-directory.txt
+    # Unit test / coverage reports
+    htmlcov/
+    .tox/
+    .nox/
+    .coverage
+    .coverage.*
+    .cache
+    nosetests.xml
+    coverage.xml
+    *.cover
+    *.py,cover
+    .hypothesis/
+    .pytest_cache/
+    # Environments
+    .env
+    .venv
+    env/
+    venv/
+    ENV/
+    env.bak/
+    venv.bak/
+    # IDE
+    .idea/
+    .vscode/
+    *.swp
+    *.swo
+    *~
+    .project
+    .pydevproject
+    .settings/
+    # Jupyter Notebook
+    .ipynb_checkpoints
+    # mypy
+    .mypy_cache/
+    .dmypy.json
+    dmypy.json
+    # Pyre type checker
+    .pyre/
+    # pytype static type analyzer
+    .pytype/
+    # OS
+    .DS_Store
+    Thumbs.db

flowquery-py/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,166 @@
+    # FlowQuery Python Implementation
+    This is the Python implementation of FlowQuery, a declarative query language for data processing pipelines.
+    ## Installation
+    ### From Source
+    ```bash
+    git clone https://github.com/microsoft/FlowQuery.git
+    cd FlowQuery/flowquery-py
+    pip install -e .
+    ```
+    ### With Development Dependencies
+    ```bash
+    pip install -e ".[dev]"
+    ```
+    ## Quick Start
+    ### Command Line Interface
+    After installation, you can start the interactive REPL:
+    ```bash
+    flowquery
+    ```
+    ### Using Conda (Alternative)
+    **Windows (PowerShell):**
+    ```powershell
+    cd flowquery-py
+    .\setup_env.ps1
+    conda activate flowquery
+    ```
+    **Linux/macOS:**
+    ```bash
+    cd flowquery-py
+    chmod +x setup_env.sh
+    ./setup_env.sh
+    conda activate flowquery
+    ```
+    The setup scripts automatically:
+. Read the Python version from `pyproject.toml`
+. Create a conda environment named `flowquery`
+. Install the package with all dev dependencies
+    ## Requirements
+    - Python 3.10+ (defined in `pyproject.toml`)
+    - pytest (for running tests)
+    - pytest-asyncio (for async test support)
+    - aiohttp (for HTTP requests)
+    All dependencies are managed in `pyproject.toml`.
+    ## Programmatic Usage
+    ```python
+    import asyncio
+    from flowquery import Runner
+    runner = Runner("WITH 1 as x RETURN x + 1 as result")
+    asyncio.run(runner.run())
+    print(runner.results)  # [{'result': 2}]
+    ```
+    ## Running Tests
+    ```bash
+    pytest tests/
+    ```
+    ## Project Structure
+    ```
+    flowquery-py/
+    ├── pyproject.toml       # Dependencies & project config (single source of truth)
+    ├── setup_env.ps1        # Windows conda setup script
+    ├── setup_env.sh         # Linux/macOS conda setup script
+    ├── README.md
+    ├── src/
+    │   ├── __init__.py          # Main package entry point
+    │   ├── extensibility.py     # Public API for custom functions
+    │   ├── compute/
+    │   │   └── runner.py        # Query execution engine
+    │   ├── graph/
+    │   │   ├── node.py          # Graph node representation
+    │   │   ├── relationship.py  # Graph relationship representation
+    │   │   ├── pattern.py       # Pattern matching
+    │   │   └── database.py      # In-memory graph database
+    │   ├── io/
+    │   │   └── command_line.py  # Interactive REPL
+    │   ├── parsing/
+    │   │   ├── parser.py        # Main parser
+    │   │   ├── ast_node.py      # AST node base class
+    │   │   ├── expressions/     # Expression types (numbers, strings, operators)
+    │   │   ├── functions/       # Built-in and custom functions
+    │   │   ├── operations/      # Query operations (WITH, RETURN, UNWIND, etc.)
+    │   │   ├── components/      # LOAD clause components
+    │   │   ├── data_structures/ # Arrays, objects, lookups
+    │   │   └── logic/           # CASE/WHEN/THEN/ELSE
+    │   ├── tokenization/
+    │   │   ├── tokenizer.py     # Lexer
+    │   │   ├── token.py         # Token class
+    │   │   └── ...              # Token types and mappers
+    │   └── utils/
+    │       ├── string_utils.py  # String manipulation utilities
+    │       └── object_utils.py  # Object utilities
+    └── tests/
+        ├── test_extensibility.py
+        ├── compute/
+        │   └── test_runner.py
+        ├── graph/
+        │   ├── test_create.py
+        │   ├── test_data.py
+        │   └── test_match.py
+        ├── parsing/
+        │   ├── test_parser.py
+        │   ├── test_context.py
+        │   └── test_expression.py
+        └── tokenization/
+            ├── test_tokenizer.py
+            ├── test_token_mapper.py
+            └── test_trie.py
+    ```
+    ## Creating Custom Functions
+    ```python
+    from flowquery.extensibility import Function, FunctionDef
+    @FunctionDef({
+        "description": "Converts a string to uppercase",
+        "category": "string",
+        "parameters": [
+            {"name": "text", "description": "String to convert", "type": "string"}
+        ],
+        "output": {"description": "Uppercase string", "type": "string"}
+    })
+    class UpperCase(Function):
+        def __init__(self):
+            super().__init__("uppercase")
+            self._expected_parameter_count = 1
+        def value(self) -> str:
+            return str(self.get_children()[0].value()).upper()
+    ```
+    ## License
+    MIT License - see [LICENSE](LICENSE) for details.
+    ## Links
+    - [Homepage](https://github.com/microsoft/FlowQuery/flowquery-py)
+    - [Repository](https://github.com/microsoft/FlowQuery/flowquery-py)
+    - [Issues](https://github.com/microsoft/FlowQuery/issues)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Virtual graph #8

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!