Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase staging #343

Merged
merged 27 commits into from
Aug 21, 2023
Merged

Rebase staging #343

merged 27 commits into from
Aug 21, 2023

Commits on Feb 12, 2021

  1. Release/1.1.3 (#290)

    * [MLOP-634] Butterfree dev workflow, set triggers for branches staging and master (#280)
    
    * Change github actions pipelines.
    
    * Change pipeline logic.
    
    * [BUG] Fix Staging GithubActions Pipeline (#283)
    
    * New step on pipelie.
    
    * Some adjusts.
    
    * Apply only wheel. (#285)
    
    * [BUG] Change version on setup.py to PyPI (#286)
    
    * Add new make command to change version.
    
    * Change command order.
    
    * Change desc and variable name.
    
    * Change command name.
    
    * Keep milliseconds when using 'from_ms' argument in timestamp feature (#284)
    
    * changed timestamp resolution
    
    * fix import
    
    * simple refactor
    
    Co-authored-by: Henrique Camargo <henriquecamargo@spf1lt-pj000560.ldap.quintoandar.com.br>
    
    * Change trigger for pipeline staging (#287)
    
    * Change trigger to publish dev pipeline.
    
    * Some fix.
    
    * Create a dev package. (#288)
    
    * [MLOP-633] Butterfree dev workflow, update documentation (#281)
    
    * Update workflow doc.
    
    * Update README
    
    * Add pre-release.
    
    * Fix typo.
    
    * [MLOP-632] Butterfree dev workflow, automate release description (#279)
    
    * release 1.1.4
    
    * update changelog
    
    Co-authored-by: Mayara Moromisato <44944954+moromimay@users.noreply.github.com>
    Co-authored-by: Henrique Camargo <henriquecamargo@spf1lt-pj000560.ldap.quintoandar.com.br>
    Co-authored-by: AlvaroMarquesAndrade <45604858+AlvaroMarquesAndrade@users.noreply.github.com>
    4 people authored Feb 12, 2021
    Configuration menu
    Copy the full SHA
    e443db9 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2021

  1. Release/1.2.0 (#328)

    * [MLOP-636] Create migration classes (#282)
    
    * [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)
    
    * Add interval branch modifications.
    
    * Add interval_runs notebook.
    
    * Add tests.
    
    * Apply style (black, flack8 and mypy).
    
    * Fix tests.
    
    * Change version to create package dev.
    
    * Allow slide selection (#293)
    
    * Fix Slide Duration Typo (#295)
    
    * [MLOP-637] Implement diff method (#292)
    
    * [MLOP-640] Create CLI with migrate command (#298)
    
    * [MLOP-645] Implement query method, cassandra (#291)
    
    * [MLOP-671] Implement get_schema on Spark client (#301)
    
    * [MLOP-648] Implement query method, metastore (#294)
    
    * Fix Validation Step (#302)
    
    * [MLOP-647] [MLOP-646] Apply migrations (#300)
    
    * add apply migration method
    
    * add test apply migration
    
    * add migrate actor with tests
    
    * mypy compliant
    
    * fix test interaction with mocked object
    
    * Rebase and some adjusts.
    
    Co-authored-by: Mayara Moromisato <may.alveslima@gmail.com>
    
    * [BUG] Apply create_partitions to historical validate (#303)
    
    * Apply create_partitions to historical validate.
    
    * Remove comments and adjusts.
    
    * [BUG] Fix key path for validate read (#304)
    
    * Fix key path
    
    * bump version
    
    Co-authored-by: AlvaroMarquesAndrade <1a789766b1c4c8b679e80f11fa6d63d42fa4bcdf>
    
    * [FIX] Add Partition types for Metastore (#305)
    
    * [MLOP-639] Track logs in S3 (#306)
    
    * Apply tracking logs and logging config.
    
    * Adjusts in CLI and logging.conf.
    
    * Some adjusts.
    
    * Change version to generate new dev package
    
    * Fix version.
    
    * Apply style.
    
    * Add new assert in the migrate unit test.
    
    * [BUG] Change logging config (#307)
    
    * Change logging config.
    
    * Some adjusts.
    
    * Remove a code smell.
    
    * Change solution for tracking logs (#308)
    
    * Change tracking logs method.
    
    * Change version to generate dev package.
    
    * Change path name in S3
    
    * Read and write consistency level options (#309)
    
    * modify cassandra client to be region aware
    
    * add option for the user to set read and write consistency levels on cassandra config
    
    * add tests
    
    * use env vars instead
    
    * Update butterfree/configs/db/cassandra_config.py
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Update butterfree/configs/db/cassandra_config.py
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Fix kafka reader. (#310)
    
    * Fix path validate. (#311)
    
    * Add local dc property (#312)
    
    * add local dc property
    
    * update version
    
    * Remove metastore migrate (#313)
    
    * Remove metastore migrate.
    
    * Change version to create a dev package.
    
    * Fix link in our docs. (#315)
    
    * [BUG] Fix Cassandra Connect Session (#316)
    
    * Fix Cassandra Connect Session.
    
    * Apply style.
    
    * Fix migration query. (#318)
    
    * Fix migration query add type key. (#319)
    
    * Fix db-config condition (#321)
    
    * Fix db-config condition.
    
    * Apply style.
    
    * MLOP-642 Document migration in Butterfree (#320)
    
    * update docs
    
    * add more information and reference new cli.md file
    
    * [MLOP-702] Debug mode for Automate Migration (#322)
    
    * Create flag debug-mode.
    
    * Fix tests.
    
    * Fix migrate test.
    
    * [MLOP-727] Improve logging messages (#325)
    
    * Fix logging message for local file
    
    * Remove json import
    
    * [MLOP-728] Improve logging messages (#324)
    
    * Improve logs.
    
    * Revert debug-mode condition.
    
    * Fix method to generate agg feature name. (#326)
    
    * [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)
    
    * Change writer type for interval mode.
    
    * Some adjusts.
    
    * Release 1.2.0
    
    Co-authored-by: AlvaroMarquesAndrade <45604858+AlvaroMarquesAndrade@users.noreply.github.com>
    Co-authored-by: Igor Gustavo Hoelscher <19557581+roelschr@users.noreply.github.com>
    Co-authored-by: Felipe Victorino Caputo <13631451+fvcaputo@users.noreply.github.com>
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    Co-authored-by: Gabriel Brandão <37742275+GaBrandao@users.noreply.github.com>
    6 people authored Jun 15, 2021
    Configuration menu
    Copy the full SHA
    1058c31 View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2023

  1. Release 1.2.1 (#338)

    * Add the missing link for H3 geohash (#330)
    
    * Add the missing link for H3 geohash
    
    * Update the H3 geohash link.
    
    * Update the same link 
    
    Update the same link in in spark_function_and_window.ipynb example
    
    * Update README.md (#331)
    
    * Update Github Actions Workflow runner (#332)
    
    * Update Workflow runner version
    
    * bump flake8-bandit
    
    * chore: bypass false positive for S105
    
    Co-authored-by: Lucas Cardozo <lucasecardozo@gmail.com>
    
    * Delete sphinx version. (#334)
    
    * Update files to staging (#336)
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Update butterfree/configs/db/cassandra_config.py
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Fix kafka reader. (#310)
    
    * Fix path validate. (#311)
    
    * Add local dc property (#312)
    
    * release 1.2.1
    
    Co-authored-by: Jay Vala <24193355+jdvala@users.noreply.github.com>
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    Co-authored-by: Lucas Fonseca <lucas.fonseca@quintoandar.com.br>
    Co-authored-by: Lucas Cardozo <lucasecardozo@gmail.com>
    Co-authored-by: Felipe Victorino Caputo <13631451+fvcaputo@users.noreply.github.com>
    6 people authored Jan 3, 2023
    1 Configuration menu
    Copy the full SHA
    adebeb9 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. [BUG] Fix Staging GithubActions Pipeline (#283)

    * New step on pipelie.
    
    * Some adjusts.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    6cc5b0d View commit details
    Browse the repository at this point in the history
  2. Apply only wheel. (#285)

    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    219ce7c View commit details
    Browse the repository at this point in the history
  3. [BUG] Change version on setup.py to PyPI (#286)

    * Add new make command to change version.
    
    * Change command order.
    
    * Change desc and variable name.
    
    * Change command name.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    dbb1a57 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9a2fff9 View commit details
    Browse the repository at this point in the history
  5. [MLOP-635] Rebase Incremental Job/Interval Run branch for test on sel…

    …ected feature sets (#278)
    
    * Add interval branch modifications.
    
    * Add interval_runs notebook.
    
    * Add tests.
    
    * Apply style (black, flack8 and mypy).
    
    * Fix tests.
    
    * Change version to create package dev.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    c4eb0f1 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    ead426d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3976944 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    5004971 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    f7557b3 View commit details
    Browse the repository at this point in the history
  10. Fix Validation Step (#302)

    AlvaroMarquesAndrade authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    230dc99 View commit details
    Browse the repository at this point in the history
  11. [MLOP-647] [MLOP-646] Apply migrations (#300)

    * add apply migration method
    
    * add test apply migration
    
    * add migrate actor with tests
    
    * mypy compliant
    
    * fix test interaction with mocked object
    
    * Rebase and some adjusts.
    
    Co-authored-by: Mayara Moromisato <may.alveslima@gmail.com>
    2 people authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    ec1c267 View commit details
    Browse the repository at this point in the history
  12. [MLOP-639] Track logs in S3 (#306)

    * Apply tracking logs and logging config.
    
    * Adjusts in CLI and logging.conf.
    
    * Some adjusts.
    
    * Change version to generate new dev package
    
    * Fix version.
    
    * Apply style.
    
    * Add new assert in the migrate unit test.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    7336e89 View commit details
    Browse the repository at this point in the history
  13. [BUG] Change logging config (#307)

    * Change logging config.
    
    * Some adjusts.
    
    * Remove a code smell.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    3f51677 View commit details
    Browse the repository at this point in the history
  14. Change solution for tracking logs (#308)

    * Change tracking logs method.
    
    * Change version to generate dev package.
    
    * Change path name in S3
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    c2c38ad View commit details
    Browse the repository at this point in the history
  15. Remove metastore migrate (#313)

    * Remove metastore migrate.
    
    * Change version to create a dev package.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    e59fe8d View commit details
    Browse the repository at this point in the history
  16. [MLOP-727] Improve logging messages (#325)

    * Fix logging message for local file
    
    * Remove json import
    GaBrandao authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    d73103d View commit details
    Browse the repository at this point in the history
  17. [MLOP-728] Improve logging messages (#324)

    * Improve logs.
    
    * Revert debug-mode condition.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    766c4c0 View commit details
    Browse the repository at this point in the history
  18. [MLOP-691] Include step to add partition to SparkMetastore during wri…

    …ting of Butterfree (#327)
    
    * Change writer type for interval mode.
    
    * Some adjusts.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    f635553 View commit details
    Browse the repository at this point in the history
  19. Update files to staging (#336)

    * Release/1.1.3 (#290)
    
    * [MLOP-634] Butterfree dev workflow, set triggers for branches staging and master (#280)
    
    * Change github actions pipelines.
    
    * Change pipeline logic.
    
    * [BUG] Fix Staging GithubActions Pipeline (#283)
    
    * New step on pipelie.
    
    * Some adjusts.
    
    * Apply only wheel. (#285)
    
    * [BUG] Change version on setup.py to PyPI (#286)
    
    * Add new make command to change version.
    
    * Change command order.
    
    * Change desc and variable name.
    
    * Change command name.
    
    * Keep milliseconds when using 'from_ms' argument in timestamp feature (#284)
    
    * changed timestamp resolution
    
    * fix import
    
    * simple refactor
    
    Co-authored-by: Henrique Camargo <henriquecamargo@spf1lt-pj000560.ldap.quintoandar.com.br>
    
    * Change trigger for pipeline staging (#287)
    
    * Change trigger to publish dev pipeline.
    
    * Some fix.
    
    * Create a dev package. (#288)
    
    * [MLOP-633] Butterfree dev workflow, update documentation (#281)
    
    * Update workflow doc.
    
    * Update README
    
    * Add pre-release.
    
    * Fix typo.
    
    * [MLOP-632] Butterfree dev workflow, automate release description (#279)
    
    * release 1.1.4
    
    * update changelog
    
    Co-authored-by: Mayara Moromisato <44944954+moromimay@users.noreply.github.com>
    Co-authored-by: Henrique Camargo <henriquecamargo@spf1lt-pj000560.ldap.quintoandar.com.br>
    Co-authored-by: AlvaroMarquesAndrade <45604858+AlvaroMarquesAndrade@users.noreply.github.com>
    
    * Release/1.2.0 (#328)
    
    * [MLOP-636] Create migration classes (#282)
    
    * [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)
    
    * Add interval branch modifications.
    
    * Add interval_runs notebook.
    
    * Add tests.
    
    * Apply style (black, flack8 and mypy).
    
    * Fix tests.
    
    * Change version to create package dev.
    
    * Allow slide selection (#293)
    
    * Fix Slide Duration Typo (#295)
    
    * [MLOP-637] Implement diff method (#292)
    
    * [MLOP-640] Create CLI with migrate command (#298)
    
    * [MLOP-645] Implement query method, cassandra (#291)
    
    * [MLOP-671] Implement get_schema on Spark client (#301)
    
    * [MLOP-648] Implement query method, metastore (#294)
    
    * Fix Validation Step (#302)
    
    * [MLOP-647] [MLOP-646] Apply migrations (#300)
    
    * add apply migration method
    
    * add test apply migration
    
    * add migrate actor with tests
    
    * mypy compliant
    
    * fix test interaction with mocked object
    
    * Rebase and some adjusts.
    
    Co-authored-by: Mayara Moromisato <may.alveslima@gmail.com>
    
    * [BUG] Apply create_partitions to historical validate (#303)
    
    * Apply create_partitions to historical validate.
    
    * Remove comments and adjusts.
    
    * [BUG] Fix key path for validate read (#304)
    
    * Fix key path
    
    * bump version
    
    Co-authored-by: AlvaroMarquesAndrade <1a789766b1c4c8b679e80f11fa6d63d42fa4bcdf>
    
    * [FIX] Add Partition types for Metastore (#305)
    
    * [MLOP-639] Track logs in S3 (#306)
    
    * Apply tracking logs and logging config.
    
    * Adjusts in CLI and logging.conf.
    
    * Some adjusts.
    
    * Change version to generate new dev package
    
    * Fix version.
    
    * Apply style.
    
    * Add new assert in the migrate unit test.
    
    * [BUG] Change logging config (#307)
    
    * Change logging config.
    
    * Some adjusts.
    
    * Remove a code smell.
    
    * Change solution for tracking logs (#308)
    
    * Change tracking logs method.
    
    * Change version to generate dev package.
    
    * Change path name in S3
    
    * Read and write consistency level options (#309)
    
    * modify cassandra client to be region aware
    
    * add option for the user to set read and write consistency levels on cassandra config
    
    * add tests
    
    * use env vars instead
    
    * Update butterfree/configs/db/cassandra_config.py
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Update butterfree/configs/db/cassandra_config.py
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    
    * Fix kafka reader. (#310)
    
    * Fix path validate. (#311)
    
    * Add local dc property (#312)
    
    * add local dc property
    
    * update version
    
    * Remove metastore migrate (#313)
    
    * Remove metastore migrate.
    
    * Change version to create a dev package.
    
    * Fix link in our docs. (#315)
    
    * [BUG] Fix Cassandra Connect Session (#316)
    
    * Fix Cassandra Connect Session.
    
    * Apply style.
    
    * Fix migration query. (#318)
    
    * Fix migration query add type key. (#319)
    
    * Fix db-config condition (#321)
    
    * Fix db-config condition.
    
    * Apply style.
    
    * MLOP-642 Document migration in Butterfree (#320)
    
    * update docs
    
    * add more information and reference new cli.md file
    
    * [MLOP-702] Debug mode for Automate Migration (#322)
    
    * Create flag debug-mode.
    
    * Fix tests.
    
    * Fix migrate test.
    
    * [MLOP-727] Improve logging messages (#325)
    
    * Fix logging message for local file
    
    * Remove json import
    
    * [MLOP-728] Improve logging messages (#324)
    
    * Improve logs.
    
    * Revert debug-mode condition.
    
    * Fix method to generate agg feature name. (#326)
    
    * [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)
    
    * Change writer type for interval mode.
    
    * Some adjusts.
    
    * Release 1.2.0
    
    Co-authored-by: AlvaroMarquesAndrade <45604858+AlvaroMarquesAndrade@users.noreply.github.com>
    Co-authored-by: Igor Gustavo Hoelscher <19557581+roelschr@users.noreply.github.com>
    Co-authored-by: Felipe Victorino Caputo <13631451+fvcaputo@users.noreply.github.com>
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    Co-authored-by: Gabriel Brandão <37742275+GaBrandao@users.noreply.github.com>
    
    Co-authored-by: hmeretti <hmeretti@gmail.com>
    Co-authored-by: Henrique Camargo <henriquecamargo@spf1lt-pj000560.ldap.quintoandar.com.br>
    Co-authored-by: AlvaroMarquesAndrade <45604858+AlvaroMarquesAndrade@users.noreply.github.com>
    Co-authored-by: Igor Gustavo Hoelscher <19557581+roelschr@users.noreply.github.com>
    Co-authored-by: Felipe Victorino Caputo <13631451+fvcaputo@users.noreply.github.com>
    Co-authored-by: Rodrigo Martins de Oliveira <allrod5@users.noreply.github.com>
    Co-authored-by: Gabriel Brandão <37742275+GaBrandao@users.noreply.github.com>
    8 people authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    7df5e42 View commit details
    Browse the repository at this point in the history
  20. Revert "Update files to staging (#336)" (#337)

    This reverts commit 35bcd30.
    moromimay authored and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    2d4aefb View commit details
    Browse the repository at this point in the history
  21. Less strict requirements (#333)

    * bump a few requirements; increase lower bound for h3 version range; adding pyarrow dev dependency
    
    * fix type repr for spark types; fix: broken tests (pyspark 3.4)
    
    ---------
    
    Co-authored-by: Ralph Rassweiler <ralphrass@gmail.com>
    lecardozo and ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    98c2420 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    f43a09c View commit details
    Browse the repository at this point in the history
  23. fix: parameter, libs (#341)

    ralphrass committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    6142bb0 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    9110091 View commit details
    Browse the repository at this point in the history