02_python/2882@bildungsstatistik-kanton-zuerich.ipynb

{"cells": [{"cell_type": "markdown", "id": "db7f08c1", "metadata": {}, "source": ["## Open Government Data, provided by **Canton Zurich**\n", "*Autogenerated Python starter code for data set with identifier* **2882@bildungsstatistik-kanton-zuerich**"]}, {"cell_type": "markdown", "id": "164c4327", "metadata": {}, "source": ["## Dataset\n", "# **Entwicklung der Bildungsabschl\u00fcsse an Mittelschulen im Kanton Z\u00fcrich ab 2000**"]}, {"cell_type": "markdown", "id": "46d13c8b-4ac3-4e2f-940f-0c14ec5b7c0d", "metadata": {}, "source": ["## Description\n", "\n", "Entwicklung der Bildungsabschl\u00fcsse an Mittelschulen im Kanton Z\u00fcrich nach Mittelschultyp seit dem Kalenderjahr 2000. Die Daten sind differenziert nach Jahr und Abschlusstyp. Ausgewiesen werden die Anzahl Abschl\u00fcsse an Mittelschulen im Kanton Z\u00fcrich.\n\nDie Daten werden im Rahmen der Statistik der Bildungsabschl\u00fcsse f\u00fcr das Bundesamtes f\u00fcr Statistik erhoben. Es handelt sich um eine Vollerhebung. Die Daten werden J\u00e4hrlich im Januar/Februar erhoben."]}, {"cell_type": "markdown", "id": "cad813bb-c986-4bb4-b52b-f78bb608086f", "metadata": {}, "source": ["## Data set links\n", "\n", "[Direct data shop link for dataset](https://www.zh.ch/de/politik-staat/statistik-daten/datenkatalog.html#/datasets/2882@bildungsstatistik-kanton-zuerich)"]}, {"cell_type": "markdown", "id": "9d4813e9", "metadata": {}, "source": ["## Metadata\n", "- **Issued** `2025-02-28T08:53:29`\n- **Modified** `2025-03-12T17:39:16`\n- **Startdate** `2000-01-01`\n- **Enddate** `2023-12-31`\n- **Theme** `['http://publications.europa.eu/resource/authority/data-theme/EDUC']`\n- **Keyword** `['ausbildung', 'berufsmaturitaetsschule', 'bildung', 'gymnasien', 'lernende', 'matura', 'mittelschulen', 'schueler*innen', 'ogd']`\n- **Publisher** `['Bildungsstatistik des Kantons Z\u00fcrich']`\n- **Landingpage** `https://www.zh.ch/de/bildung/bildungssystem/zahlen-fakten.html`\n"]}, {"cell_type": "markdown", "id": "8a857d65", "metadata": {"jp-MarkdownHeadingCollapsed": true, "tags": []}, "source": ["## Imports and helper functions"]}, {"cell_type": "code", "execution_count": null, "id": "93b39602-1c1e-46d2-ae70-1716b1481e9b", "metadata": {"tags": []}, "outputs": [], "source": ["%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.style.use('ggplot')\n", "\n", "params = {\n", "    'text.color': (0.25, 0.25, 0.25),\n", "    'figure.figsize': [18, 6],\n", "   }\n", "\n", "plt.rcParams.update(params)\n", "\n", "import pandas as pd "]}, {"cell_type": "code", "execution_count": null, "id": "aa6611d7-e1c0-40a7-b0ff-601a8ef8439b", "metadata": {"tags": []}, "outputs": [], "source": ["# helper function for reading datasets with proper separator\n", "def get_dataset(url):\n", "    if url[-3:] != \"csv\":\n", "        print(\"The data set URL has no proper 'csv' extension. Reading the dataset might not have worked as expected.\\nPlease check the dataset link and adjust pandas' read_csv() parameters accordingly.\")\n", "    data = pd.read_csv(url, sep=\",\", on_bad_lines='warn', encoding_errors='ignore', low_memory=False)\n", "    # if dataframe only has one column or less the data is not comma separated, use \";\" instead\n", "    if data.shape[1] <= 1:\n", "        data = pd.read_csv(url, sep=';', on_bad_lines='warn', encoding_errors='ignore', low_memory=False)\n", "        if data.shape[1] <= 1:\n", "            print(\"The data wasn't imported properly. Very likely the correct separator couldn't be found.\\nPlease check the dataset manually and adjust the code.\")\n", "    return data"]}, {"cell_type": "markdown", "id": "02ce518f", "metadata": {}, "source": ["## Load data\n", "\n", "- The dataset has **`1` distribution(s)** in CSV format.\n", "- All available CSV distributions are listed below and can be read into a pandas dataframe."]}, {"cell_type": "code", "execution_count": null, "id": "0", "metadata": {"tags": []}, "outputs": [], "source": "# Distribution 0\n# Ktzhdistid               : 5943\n# Title                    : Entwicklung der Bildungsabschl\u00fcsse an Mittelschulen im Kanton Z\u00fcrich\n# Description              : Die Daten sind differenziert nach Jahr und Abschlusstyp. Ausgewiesen werden die Anzahl Abschl\u00fcsse an Mittelschulen im Kanton Z\u00fcrich. Jahr: Das Kalenderjahr in dem ein Abschluss stattfand. Abschlusstyp: Erlangter Abschlusstyp resp. Typ der Mittelschule die abgeschlossen wurde. Anzahl: Die Anzahl Personen die eine Ausbildung erfolgreich abgeschlossen haben. Die Diplommittelschule wurde ab  Schuljahr 2007/08 durch die Fachmittelschule ersetzt. Aufgrund der K\u00fcrzung der Ausbildungsdauer von 4.5 auf 4 Jahre, haben im Kalenderjahr 2002 zwei Jahrg\u00e4nge die gymnasiale Ausbildung abgeschlossen. Seit dem Kalenderjahr 2022 werden zus\u00e4tzliche ausl\u00e4ndische Abschl\u00fcsse wie Franz\u00f6sisches Baccalaur\u00e9at, GCSE oder A-Level erfasst. Der Anstieg bei den internationalen Mittelschulen ist vorwiegend durch diese methodische Anpassung zu begr\u00fcnden.\n# Issued                   : 2025-02-28T08:53:29\n# Modified                 : 2025-03-19T08:39:52\n\ndf = get_dataset('https://www.web.statistik.zh.ch/ogd/data/bista/ZH_Entwicklung_der_Bildungsabschluesse_an_Mittelschulen.csv')\n\n"}, {"cell_type": "markdown", "id": "4ce1f78f", "metadata": {}, "source": ["## Analyze data"]}, {"cell_type": "code", "execution_count": null, "id": "3e3dab86", "metadata": {}, "outputs": [], "source": ["# drop columns that have no values\n", "df.dropna(how='all', axis=1, inplace=True)"]}, {"cell_type": "code", "execution_count": null, "id": "841bd8d2", "metadata": {}, "outputs": [], "source": ["print(f'The dataset has {df.shape[0]:,.0f} rows (observations) and {df.shape[1]:,.0f} columns (variables).')\n", "print(f'There seem to be {df.duplicated().sum()} exact duplicates in the data.')"]}, {"cell_type": "code", "execution_count": null, "id": "75e73c96", "metadata": {}, "outputs": [], "source": ["df.info(memory_usage='deep', verbose=True)"]}, {"cell_type": "code", "execution_count": null, "id": "02f3df4d", "metadata": {}, "outputs": [], "source": ["df.head()"]}, {"cell_type": "code", "execution_count": null, "id": "a0d7d898", "metadata": {}, "outputs": [], "source": ["# display a small random sample transposed in order to see all variables\n", "df.sample(3).T"]}, {"cell_type": "code", "execution_count": null, "id": "786806dd", "metadata": {}, "outputs": [], "source": ["# describe non-numerical features\n", "try:\n", "    with pd.option_context('display.float_format', '{:,.2f}'.format):\n", "        display(df.describe(exclude='number'))\n", "except:\n", "    print(\"No categorical data in dataset.\")"]}, {"cell_type": "code", "execution_count": null, "id": "e744a6b6", "metadata": {}, "outputs": [], "source": ["# describe numerical features\n", "try:\n", "    with pd.option_context('display.float_format', '{:,.2f}'.format):\n", "        display(df.describe(include='number'))\n", "except:\n", "    print(\"No numercial data in dataset.\")"]}, {"cell_type": "code", "execution_count": null, "id": "7a65d95d", "metadata": {}, "outputs": [], "source": ["# check missing values with missingno\n", "# https://github.com/ResidentMario/missingno\n", "import missingno as msno\n", "msno.matrix(df, labels=True, sort='descending');"]}, {"cell_type": "code", "execution_count": null, "id": "fcc604b7", "metadata": {}, "outputs": [], "source": ["# plot a histogram for each numerical feature\n", "try:\n", "    df.hist(bins=25, rwidth=.9)\n", "    plt.tight_layout()\n", "    plt.show()\n", "except:\n", "    print(\"No numercial data to plot.\") "]}, {"cell_type": "code", "execution_count": null, "id": "e13cc6eb", "metadata": {}, "outputs": [], "source": ["# continue your code here..."]}, {"cell_type": "code", "execution_count": null, "id": "18886378", "metadata": {}, "outputs": [], "source": []}, {"cell_type": "code", "execution_count": null, "id": "59fce071", "metadata": {}, "outputs": [], "source": []}, {"cell_type": "code", "execution_count": null, "id": "f42f25aa", "metadata": {}, "outputs": [], "source": []}, {"cell_type": "markdown", "id": "c6b87d83", "metadata": {}, "source": ["**Contact**: Bildungsstatistik des Kantons Z\u00fcrich | bista@bi.zh.ch"]}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.15"}}, "nbformat": 4, "nbformat_minor": 5}