Skip to content

Commit

Permalink
Faster Faker (#18021)
Browse files Browse the repository at this point in the history
* Faster Faker

* unit tests

* tests passing

* update pr link

* guard against small ranges

* Fixup product timezones

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
  • Loading branch information
evantahler and octavia-squidington-iii authored Oct 15, 2022
1 parent 1b134d8 commit c81ea48
Show file tree
Hide file tree
Showing 15 changed files with 229 additions and 188 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@
- name: Faker
sourceDefinitionId: dfd88b22-b603-4c3d-aad7-3701784586b1
dockerRepository: airbyte/source-faker
dockerImageTag: 0.1.8
dockerImageTag: 0.2.0
documentationUrl: https://docs.airbyte.com/integrations/sources/faker
sourceType: api
releaseStage: alpha
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2948,7 +2948,7 @@
oauthFlowInitParameters: []
oauthFlowOutputParameters:
- - "access_token"
- dockerImage: "airbyte/source-faker:0.1.8"
- dockerImage: "airbyte/source-faker:0.2.0"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/faker"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_faker ./source_faker
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.8
LABEL io.airbyte.version=0.2.0
LABEL io.airbyte.name=airbyte/source-faker
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ Make sure to familiarize yourself with [pytest test discovery](https://docs.pyte
First install test dependencies into your virtual environment:

```
pip install .[tests]
pip install ".[tests]"
```

### Unit Tests
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,26 @@
"created_at": {
"type": "string",
"format": "date-time",
"airbyte_type": "timestamp_without_timezone"
"airbyte_type": "timestamp_with_timezone"
},
"updated_at": {
"type": "string",
"format": "date-time",
"airbyte_type": "timestamp_without_timezone"
"airbyte_type": "timestamp_with_timezone"
},
"job": { "type": "string" },
"company": { "type": "string" },
"ssn": { "type": "string" },
"residence": { "type": "string" },
"current_location": { "type": "array" },
"blood_group": { "type": "string" },
"website": { "type": "array" },
"username": { "type": "string" },
"name": { "type": "string" },
"sex": { "type": "string" },
"address": { "type": "string" },
"mail": { "type": "string" }
"title": { "type": "string" },
"age": { "type": "integer" },
"email": { "type": "string" },
"telephone": { "type": "string" },
"gender": { "type": "string" },
"language": { "type": "string" },
"academic_degree": { "type": "string" },
"nationality": { "type": "string" },
"occupation": { "type": "string" },
"height": { "type": "string" },
"blood_type": { "type": "string" },
"weight": { "type": "integer" }
}
},
"supported_sync_modes": ["incremental", "full_refresh"],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,29 @@
"type": "object",
"properties": {
"id": { "type": "number" },
"created_at": { "type": "date" },
"updated_at": { "type": "date" },
"job": { "type": "string" },
"company": { "type": "string" },
"ssn": { "type": "string" },
"residence": { "type": "string" },
"current_location": { "type": "array" },
"blood_group": { "type": "string" },
"website": { "type": "array" },
"username": { "type": "string" },
"created_at": {
"type": "string",
"format": "date-time",
"airbyte_type": "timestamp_with_timezone"
},
"updated_at": {
"type": "string",
"format": "date-time",
"airbyte_type": "timestamp_with_timezone"
},
"name": { "type": "string" },
"sex": { "type": "string" },
"address": { "type": "string" },
"mail": { "type": "string" }
"title": { "type": "string" },
"age": { "type": "integer" },
"email": { "type": "string" },
"telephone": { "type": "string" },
"gender": { "type": "string" },
"language": { "type": "string" },
"academic_degree": { "type": "string" },
"nationality": { "type": "string" },
"occupation": { "type": "string" },
"height": { "type": "string" },
"blood_type": { "type": "string" },
"weight": { "type": "integer" }
}
},
"supported_sync_modes": ["incremental", "full_refresh"],
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{"stream": "Users", "data": {"job": "Musician", "company": "Williams-Sheppard", "ssn": "498-52-4970", "residence": "Unit 5938 Box 2421\nDPO AP 33335", "current_location": [52.958961, 143.143712], "blood_group": "B+", "website": ["http://www.rivera.com/", "http://grimes-green.net/", "http://www.larsen.com/"], "username": "leeashley", "name": "Gary Cross", "sex": "M", "address": "711 Golden Overpass\nWest Andreaville, MA 71317", "mail": "tamaramorrison@hotmail.com", "id": 1, "created_at": "1976-03-27T12:40:22", "updated_at": "2000-04-13T06:17:38"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Therapist, occupational", "company": "Stewart-Allen", "ssn": "189-25-3060", "residence": "1122 Megan Squares Suite 848\nPort Jason, OR 55475", "current_location": [65.610695, -32.24732], "blood_group": "O-", "website": ["https://www.salazar-tucker.com/", "http://www.dennis.com/", "https://www.simmons-brown.com/", "http://www.walters.com/"], "username": "myersmitchell", "name": "Chelsea Greer", "sex": "F", "address": "Unit 0903 Box 2173\nDPO AP 08507", "mail": "stephenschristine@yahoo.com", "id": 2, "created_at": "1974-11-24T20:47:03", "updated_at": "1985-01-10T17:47:40"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Illustrator", "company": "Graham-Brown", "ssn": "479-06-9773", "residence": "45792 Tammy Centers Apt. 258\nDavidmouth, HI 02231", "current_location": [36.600983, 40.066283], "blood_group": "AB+", "website": ["https://castro.info/", "http://williams.info/", "https://howard.org/"], "username": "smithjames", "name": "Jasmine Perry", "sex": "F", "address": "76960 Savage Port\nBartonton, NV 08874", "mail": "jacqueline78@yahoo.com", "id": 3, "created_at": "2011-02-06T22:32:40", "updated_at": "2021-05-30T16:13:19"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Academic librarian", "company": "Simpson LLC", "ssn": "858-60-0817", "residence": "81206 Stewart Forest Apt. 089\nEast Davidborough, ME 37198", "current_location": [-41.0853825, -126.215901], "blood_group": "O-", "website": ["http://www.sloan-marsh.com/", "https://www.meyer.com/", "https://www.williams.com/", "https://browning.org/"], "username": "monica23", "name": "Steven Bowman", "sex": "M", "address": "22455 Higgins Junction Apt. 042\nNew Keith, OH 17493", "mail": "danny30@yahoo.com", "id": 4, "created_at": "2005-08-05T11:36:52", "updated_at": "2019-08-02T23:04:14"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Building services engineer", "company": "Powell-Murphy", "ssn": "425-14-1629", "residence": "5517 Holly Meadow Apt. 452\nLake Anne, SC 11894", "current_location": [-72.327639, -134.543372], "blood_group": "B-", "website": ["http://garcia.net/", "https://kramer-klein.com/"], "username": "ryanhoward", "name": "Daniel Duarte", "sex": "M", "address": "731 Sanders Fords\nPort Jasonberg, ID 60585", "mail": "jward@gmail.com", "id": 5, "created_at": "1980-02-06T13:15:11", "updated_at": "2003-03-15T20:31:32"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Surveyor, rural practice", "company": "Little-Henderson", "ssn": "534-64-5284", "residence": "389 Alec Squares Suite 508\nPort Jonathan, FL 50177", "current_location": [66.4839605, -21.954682], "blood_group": "O-", "website": ["http://www.perez.com/"], "username": "mistymurray", "name": "Joan Atkins", "sex": "F", "address": "809 Erika Valley Apt. 634\nPetersenfort, WY 51431", "mail": "melissayates@hotmail.com", "id": 6, "created_at": "1982-05-20T20:41:25", "updated_at": "2020-05-29T07:50:41"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Immigration officer", "company": "Hudson, Cook and Snyder", "ssn": "269-39-6686", "residence": "22345 Sheri Orchard Suite 279\nLake Hollystad, AZ 80687", "current_location": [35.165804, 27.598858], "blood_group": "O-", "website": ["http://smith.org/"], "username": "james71", "name": "Ashley Dunn", "sex": "F", "address": "519 Kramer Crossroad Suite 418\nNorth Kimberly, MN 99672", "mail": "stanleyclarke@gmail.com", "id": 7, "created_at": "1972-09-20T17:24:07", "updated_at": "2003-01-31T16:46:49"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Freight forwarder", "company": "Duffy Ltd", "ssn": "583-74-3539", "residence": "412 Snow Manors Apt. 161\nSouth Kimtown, NV 57004", "current_location": [21.942823, -163.821807], "blood_group": "O-", "website": ["https://www.jones-howard.com/", "https://smith.com/", "http://www.smith.com/"], "username": "michelelopez", "name": "Melissa Cantu", "sex": "F", "address": "420 Michael Mountains Suite 485\nNew Victoria, ND 76634", "mail": "eric22@gmail.com", "id": 8, "created_at": "1986-07-11T18:55:27", "updated_at": "2004-08-04T16:06:58"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Fast food restaurant manager", "company": "Carter Group", "ssn": "680-06-2167", "residence": "12648 Yang Divide Suite 451\nSouth Cynthia, NC 08084", "current_location": [11.047629, 39.379532], "blood_group": "A+", "website": ["https://watson.com/", "http://www.rodriguez-jacobs.com/", "https://saunders.com/", "http://giles-thomas.info/"], "username": "amontgomery", "name": "Christina Smith", "sex": "F", "address": "489 Roger Terrace\nDavisfort, IN 71770", "mail": "bramirez@gmail.com", "id": 9, "created_at": "1972-10-04T19:18:09", "updated_at": "2015-06-29T09:01:24"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"job": "Academic librarian", "company": "Ross-Zamora", "ssn": "728-51-7285", "residence": "4391 Chad Greens Suite 851\nPort Frank, LA 37561", "current_location": [49.5419055, -107.833532], "blood_group": "B-", "website": ["https://osborne.com/"], "username": "qtaylor", "name": "Melissa James", "sex": "F", "address": "637 Neal Island Suite 074\nLake Tyler, RI 28775", "mail": "kellypeter@gmail.com", "id": 10, "created_at": "1974-07-10T19:02:25", "updated_at": "2005-11-07T09:09:06"}, "emitted_at": 1649892395000}
{"stream": "Users", "data": {"id": 1, "name": "Reda", "title": "M.Sc.Tech.", "age": 47, "email": "locations1983@protonmail.com", "telephone": "+1-(110)-795-7610", "gender": "Male", "language": "Tamil", "academic_degree": "Master", "nationality": "Italian", "occupation": "Word Processing Operator", "height": "1.55", "blood_type": "B\u2212", "weight": 58, "created_at": "2009-08-12T18:57:58+00:00", "updated_at": "2012-07-02T08:32:31+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 2, "name": "Tristan", "title": "M.Sc.Tech.", "age": 32, "email": "variations1847@duck.com", "telephone": "683-770-9281", "gender": "Other", "language": "Bosnian", "academic_degree": "Bachelor", "nationality": "Estonian", "occupation": "Tiler", "height": "2.00", "blood_type": "AB\u2212", "weight": 44, "created_at": "2008-09-23T19:57:09+00:00", "updated_at": "2016-03-10T04:48:06+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 3, "name": "Yuki", "title": "Miss", "age": 50, "email": "vacuum2027@yahoo.com", "telephone": "1-321-809-2061", "gender": "Female", "language": "Armenian", "academic_degree": "Bachelor", "nationality": "Swiss", "occupation": "Valuer", "height": "1.84", "blood_type": "O\u2212", "weight": 71, "created_at": "2003-06-14T10:39:40+00:00", "updated_at": "2003-12-03T21:21:30+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 4, "name": "Fred", "title": "MMath", "age": 47, "email": "causes1859@outlook.com", "telephone": "(827) 127-3811", "gender": "Female", "language": "Assamese", "academic_degree": "PhD", "nationality": "Russian", "occupation": "Turkey Farmer", "height": "1.80", "blood_type": "A+", "weight": 39, "created_at": "2001-09-30T00:05:46+00:00", "updated_at": "2006-09-16T14:55:33+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 5, "name": "Emmitt", "title": "DPhil", "age": 39, "email": "athens1899@gmail.com", "telephone": "(470) 656-8003", "gender": "Other", "language": "English", "academic_degree": "Bachelor", "nationality": "Jordanian", "occupation": "Stone Sawyer", "height": "1.52", "blood_type": "A+", "weight": 82, "created_at": "2012-12-27T21:40:00+00:00", "updated_at": "2015-06-08T23:20:45+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 6, "name": "Hollis", "title": "MSc", "age": 52, "email": "fisheries1881@yandex.com", "telephone": "(519) 606-9896", "gender": "Other", "language": "Swati", "academic_degree": "Master", "nationality": "Chilean", "occupation": "Writer", "height": "1.85", "blood_type": "AB\u2212", "weight": 85, "created_at": "2002-04-30T18:14:15+00:00", "updated_at": "2004-09-15T02:05:20+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 7, "name": "Kip", "title": "M.D.", "age": 31, "email": "numbers1983@example.com", "telephone": "346-013-2638", "gender": "Other", "language": "Armenian", "academic_degree": "Master", "nationality": "Swiss", "occupation": "Salesman", "height": "1.89", "blood_type": "O+", "weight": 48, "created_at": "2003-09-11T17:13:51+00:00", "updated_at": "2016-08-04T09:35:18+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 8, "name": "Carie", "title": "Madam", "age": 49, "email": "watershed1819@example.com", "telephone": "(348) 881-9607", "gender": "Male", "language": "Kyrgyz", "academic_degree": "PhD", "nationality": "Guatemalan", "occupation": "Park Ranger", "height": "1.50", "blood_type": "O+", "weight": 83, "created_at": "2012-06-19T07:18:11+00:00", "updated_at": "2017-10-10T14:05:38+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 9, "name": "Steven", "title": "Mr.", "age": 54, "email": "llp1893@yahoo.com", "telephone": "830.247.8156", "gender": "Fluid", "language": "Catalan", "academic_degree": "Bachelor", "nationality": "Egyptian", "occupation": "Ambulance Driver", "height": "1.52", "blood_type": "AB+", "weight": 81, "created_at": "2002-11-25T04:56:09+00:00", "updated_at": "2005-01-20T21:16:30+00:00"}, "emitted_at": 1665791425000}
{"stream": "Users", "data": {"id": 10, "name": "Lore", "title": "Madam", "age": 61, "email": "resident2075@example.com", "telephone": "321.233.0702", "gender": "Female", "language": "Polish", "academic_degree": "Master", "nationality": "French", "occupation": "Registrar", "height": "1.99", "blood_type": "B+", "weight": 56, "created_at": "2001-02-23T17:43:25+00:00", "updated_at": "2022-09-09T16:51:15+00:00"}, "emitted_at": 1665791425000}
4 changes: 2 additions & 2 deletions airbyte-integrations/connectors/source-faker/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from setuptools import find_packages, setup

MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1", "Faker==13.3.1"]
MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1", "mimesis==6.1.1"]

TEST_REQUIREMENTS = [
"pytest~=6.1",
Expand All @@ -14,7 +14,7 @@

setup(
name="source_faker",
description="Source implementation for Faker.",
description="Source implementation for fake but realistic looking data.",
author="Airbyte",
author_email="evan@airbyte.io",
packages=find_packages(),
Expand Down
Loading

0 comments on commit c81ea48

Please sign in to comment.