README.md

Python learning

Some notebooks I have created as I have been learning how to do various things in Python. Some need additional datasets which are referenced as links in the notebook. These are work in progress as I learn more and add more to this repo.

Requirements

Need to add some sort of requirements.txt file. The required libraries can be found from the imports in the notebooks.

Geospatial things

This includes notebooks for generating regular geometries (hexgrids and square grids) and transforming quantities from one geometry to another:

• HexGrid.ipynb
• RegularGridShapefiles.ipynb
• UniformPopulationEstimates.ipynb

Cleaning US postal addresses

• us_postal_address_data_cleaning.ipynb

Some pandas notes

Some notes that I've made to remind me how to do things that I regularly use like groupby operations and working with time variables:

• pandas_notes.ipynb
• time_in_pandas.ipynb

PySpark notes

Some notes I have been making while reluctantly learning PySpark, including instructions on setup and avoiding some errors with the Java SDK

• pyspark_examples.ipynb

Webscraping

From my blog post on Introduction to Webscraping

• Intro_to_webscraping_010121.ipynb

Machine learning / AI / generative AI

Re-creating results from papers to understand some techniques for data labelling, large language models and genAI.

• ravfogel_2023.ipynb
• active_inference.ipynb

Some stats

Worked examples of things like positive predictive values, linear regression, stats tests:

• diagnostic_tests.ipynb
• essential_medical_stats_ch12.ipynb
• linear_regression_basics.ipynb
• ppv.ipynb
• stats_tests.ipynb
• ml_stats_animations.ipynb