Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 120 compatibility with rdt issue72 #121

Merged
merged 47 commits into from
Nov 6, 2019
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
1ffdfd5
Removed old PositiveNumberTransformes usage
JDTheRipperPC Oct 4, 2019
36d109b
change rdt hypertransformer usage
JDTheRipperPC Oct 4, 2019
8293bc9
Adapt to the new RDT API
csala Oct 7, 2019
6ad8e1d
Add example notebooks
csala Oct 7, 2019
35535d1
WIP
JDTheRipperPC Oct 7, 2019
c9555f0
Rename DataNavigator to Metadata and isolate metadata inside it
csala Oct 8, 2019
756ce7a
Reduce data loading and remove tables from modeler
csala Oct 8, 2019
d64e1b8
new tests for test_sdv and changes on test_modeler
JDTheRipperPC Oct 11, 2019
b768490
added more tests
JDTheRipperPC Oct 14, 2019
e166c4e
added unittests and fixed linting
JDTheRipperPC Oct 16, 2019
d134271
added save and load SDV intance tests
JDTheRipperPC Oct 17, 2019
802512e
fix docstring and tests
JDTheRipperPC Oct 17, 2019
0037b5d
added metadata test
JDTheRipperPC Oct 17, 2019
7e4151a
added save and load tests on test_modeler
JDTheRipperPC Oct 17, 2019
e7f219a
WIP, added tests, one test is marked as skip
JDTheRipperPC Oct 17, 2019
f7df984
Fix tests.
JDTheRipperPC Oct 21, 2019
8148718
fix test_sampler
JDTheRipperPC Oct 21, 2019
27ef996
Remove distribution args and fix primary key generation
csala Oct 22, 2019
de965a3
Add airbnb example
csala Oct 22, 2019
96493e2
Added mountain and olympic game datasets
JDTheRipperPC Oct 25, 2019
cca88e0
CPA usage refactoring. Skip id columns on model
csala Oct 25, 2019
3c8f706
Remove data from tests folder
csala Oct 25, 2019
91276bf
Reorganize examples
csala Oct 25, 2019
f9f7c71
Remove unused argument
csala Oct 25, 2019
030aefe
Add Quickstart
csala Oct 28, 2019
3e94b96
fixed sdv and metadata test failure, metadata tests wip
JDTheRipperPC Oct 28, 2019
5639a66
Merge branch 'issue_120_compatibility_with_rdt_issue72_fixes' of http…
JDTheRipperPC Oct 28, 2019
014cb96
fixed modeler tests
JDTheRipperPC Oct 29, 2019
75961b0
fixed sampler tests
JDTheRipperPC Oct 29, 2019
3aa3bc1
fix linting and py35 tests
JDTheRipperPC Oct 29, 2019
a8e1bd3
fix isort
JDTheRipperPC Oct 29, 2019
446b4bb
added docstrings
JDTheRipperPC Oct 29, 2019
a15ac9f
docstrings
JDTheRipperPC Oct 30, 2019
a89153c
updated README.md
JDTheRipperPC Oct 30, 2019
107ec44
added tests and sdv demo
JDTheRipperPC Oct 30, 2019
79ccd26
Merge branch 'issue_120_compatibility_with_rdt_issue72_fixes' into is…
JDTheRipperPC Oct 30, 2019
7d75faa
fit tests py35
JDTheRipperPC Oct 30, 2019
d99ebe1
Update docstrings.
pvk-developer Oct 31, 2019
af49713
Isort.
pvk-developer Oct 31, 2019
13cc674
Fix docstrings, tests and examples. Minor code improvements
csala Nov 4, 2019
6864e46
Fix py35 test
csala Nov 4, 2019
d6aba39
Update readme.
pvk-developer Nov 4, 2019
c552367
Update README.md
pvk-developer Nov 4, 2019
f001a34
Improve README
csala Nov 6, 2019
110d9e7
Fix json formatting
csala Nov 6, 2019
75ba5a7
Add clarifications about the data format
csala Nov 6, 2019
d1376b4
Cleanup examples
csala Nov 6, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
512 changes: 164 additions & 348 deletions README.md

Large diffs are not rendered by default.

678 changes: 678 additions & 0 deletions examples/RDT HyperTransformer + GaussianMultivariate.ipynb

Large diffs are not rendered by default.

Empty file removed examples/__init__.py
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Airbnb Dataset Subsampling\n",
"\n",
"This notebooks shows how to generate the `users_sample.csv` and `sessions_sample.csv` files.\n",
"Before running this notebook, make sure to having downloaded the `train_users_2.csv` and `sessions.csv`\n",
"files from https://www.kaggle.com/c/airbnb-recruiting-new-user-bookings/data"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"users = pd.read_csv('train_users_2.csv')\n",
"sessions = pd.read_csv('sessions.csv')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(213451, 16)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"users.shape"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(10567737, 6)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sessions.shape"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"users_sample = users.sample(1000).reset_index(drop=True)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"sessions_sample = sessions[\n",
" sessions['user_id'].isin(users_sample['id'])\n",
"].reset_index(drop=True).copy()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(24988, 6)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sessions_sample.shape"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"user_ids = users_sample['id'].reset_index().set_index('id')['index']"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"id\n",
"c2cga16ny5 0\n",
"08yitv7nz0 1\n",
"3oobewtnls 2\n",
"owd2csvj6u 3\n",
"i66dlbdyrt 4\n",
"Name: index, dtype: int64"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"user_ids.head()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"users_sample['id'] = users_sample['id'].map(user_ids)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"sessions_sample['user_id'] = sessions_sample['user_id'].map(user_ids)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"users_sample.to_csv('users_sample.csv', index=False)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"sessions_sample.to_csv('sessions_sample.csv', index=False)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading