Skip to content

Commit

Permalink
Merge branch 'main' into quant_qa
Browse files Browse the repository at this point in the history
  • Loading branch information
ZiyueXu77 authored Feb 27, 2025
2 parents c3396a5 + 36a40a2 commit e169e87
Show file tree
Hide file tree
Showing 37 changed files with 4,010 additions and 202 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@
"id": "bd1c7385-dd22-4d5c-8d2e-7a73f0b3ac2d",
"metadata": {},
"source": [
"In this chapter, we visited NVFlare's offerings in enabling efficient and robust federated training of language models, especially in the era of LLMs.\n",
"In this chapter, we visited NVFlare's offerings in enabling efficient and robust federated training of language models, especially in the era of LLMs. \n",
"\n",
"Specifically, the following items have been covered:\n",
"1. **[Federated NLP with BERT Model](../08.1_fed_bert/federated_nlp_with_bert.ipynb)**: task-specific model training with BERT in a \n",
"2. **[Federated LLM Tuning with SFT](../08.2_llm_sft/LLM_SFT.ipynb)**: supervised Fine-Tuning and its role in adapting LLMs in federated learning\n",
"3. **[Federated LLM Tuning with PEFT](../08.3_llm_peft/LLM_PEFT.ipynb)**: PEFT in adapting LLMs for specific tasks, which can be achieve in a federated setting\n",
"1. **[Federated NLP with BERT Model](../08.1_fed_bert/federated_nlp_with_bert.ipynb)**: task-specific model training with BERT in a federated setting. \n",
"2. **[Federated LLM Tuning with SFT](../08.2_llm_sft/LLM_SFT.ipynb)**: supervised Fine-Tuning and its role in adapting LLMs in federated learning. \n",
"3. **[Federated LLM Tuning with PEFT](../08.3_llm_peft/LLM_PEFT.ipynb)**: PEFT in adapting LLMs for specific tasks, which can be achieve in a federated setting. \n",
"4. **[Model Quantization for Transmission](../08.4_llm_quantization/LLM_quantization.ipynb)**: reduce the message size with quantization methods so as to address the significant communication burden when performing federated LLM learning with SFT. \n",
"5. **[Message Streaming for Model Transmission](../08.5_llm_streaming/LLM_streaming.ipynb)**: with quantization reducing communication cost, system memory requirement is still high for prepareing the message on either side. Therefore, we enabled streaming capabilities for more efficient and robust model communication."
]
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "84c0c9f9-ed49-4070-81f1-d255dd8a3dce",
"metadata": {},
"source": [
"# Federated Learning in Financial Services\n",
"\n",
"This chapter includes an end-to-end example demonstrating the use of federated learning in a financial application - credit card fraud detection.\n",
"\n",
"The experiments are based on the [kaggle credit card fraud dataset](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud), \n",
"\n",
"As compared with other tutorails, in this chapter to illustrate the end-to-end process that is realistic for financial applications, we manually duplicated the records to extend the data time span from 2 days to over 2 years, and added random transactional information. Our primary goal is to showcase the process with a more realistic dataset.\n",
"\n",
"The overall steps of the end-to-end process include the following:"
]
},
{
"cell_type": "markdown",
"id": "29c44b19-be11-4d05-b0bd-76a623966d51",
"metadata": {},
"source": [
"## Step 1: Data Preparation \n",
"First, we prepare the data by adding random transactional information to the base creditcard dataset.\n",
"## Step 2: Feature Analysis\n",
"Second, we analyze the data, understand the features, and derive (and encode) secondary features that can be more useful for building the model. Rule-based and GNN-based feature enrichments can be adapted.\n",
"## Step 3: Federated XGBoost \n",
"With the enriched data, we can fit them with federated XGBoost. "
]
},
{
"cell_type": "markdown",
"id": "8f0d090a-fbeb-437c-b6c3-68aea50d3244",
"metadata": {},
"source": [
"Now let's move on to see the details of this [end-to-end illustration](../12.1_end_to_end_federated_fraud_detection/end_to_end_federated_fruad_detection_process.ipynb), after which we will do a [recap](../12.2_recap/recap.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5f3916f6-493d-49d4-96d8-daccf5ded8dd",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit e169e87

Please sign in to comment.