Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1685 Add support for multiple results in salesforce batch jobs #1686

Merged
merged 9 commits into from May 19, 2016
Merged

#1685 Add support for multiple results in salesforce batch jobs #1686

merged 9 commits into from May 19, 2016

Conversation

dacort
Copy link
Contributor

@dacort dacort commented May 11, 2016

Addresses #1685 by adding support to fetch multiple results from a single Salesforce batch job.

Open Items

  • Merge downloaded files

Description

  • New get_batch_result_ids method that is more accurately documented
  • Backwards compatible get_batch_results method, including warning message
  • Small set of tests for the changes

Motivation and Context

Fixes #1685

Have you tested this? If so, how?

Included unit tests as well as verified locally.

@mention-bot
Copy link

By analyzing the blame information on this pull request, we identified @dlstadther to be a potential reviewer

@dlstadther
Copy link
Collaborator

Based on TravisCI results, looks like you just need to fix the few flake8 errors and then run the tests again. All the others look to be errors when creating the environment (stuff you didn't cause).

I haven't run this myself, but the logic looks good. Thanks!

@dacort
Copy link
Contributor Author

dacort commented May 12, 2016

D'oh. Of course I forget to run the flake8 test after I add the unit test. ;)

Addressing shortly.

@dacort dacort changed the title #1685 Add support for multiple results in salesforce batch jobs [WIP] #1685 Add support for multiple results in salesforce batch jobs May 12, 2016
@dlstadther
Copy link
Collaborator

dlstadther commented May 17, 2016

Looks like you need to get @Tarrasch 's updates to Tox (that'll fix these failing tests).

git fetch upstream
git checkout master
git merge upstream/master
git checkout <this branch>
git rebase master

git push origin <this branch> --force

Also, did you fix the duplicated header issue?

Adds a new method 'get_batch_result_ids' and maintains backwards compatibility
with the old method, adding a warning.
Note that this only supports CSV since this is what the library does by default.
@dacort
Copy link
Contributor Author

dacort commented May 17, 2016

@dlstadther Thanks for the note re: Tox updates. Just pulled that in and pushed up an update that merges batch results (from CSV).

@dacort
Copy link
Contributor Author

dacort commented May 17, 2016

Looks like I have to some work to do re: python 3 compat.

if sys.version_info.major > 2:
patch_name = 'builtins.open'

@mock.patch(patch_name, side_effect=mocked_open)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use six.py for the Python 2.x vs 3.x compatibility?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will take a look!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I can use it to import a generic builtins, but I can at least use it to check for the Python version.

The challenge here is the patch_name string gets interpreted further down the stack and I'm not actually importing builtins, so flake8 throws an error.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, tests passed. So if @Tarrasch approves, then i think this pr is good to go.

@codecov-io
Copy link

codecov-io commented May 17, 2016

Current coverage is 76.10%

Merging #1686 into master will decrease coverage by 2.52%

@@             master      #1686   diff @@
==========================================
  Files            94         94          
  Lines         10256      10285    +29   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits           8063       7827   -236   
- Misses         2193       2458   +265   
  Partials          0          0          

Powered by Codecov. Last updated by 187517b...8171a19

@@ -135,6 +135,10 @@ def rename(self, *args, **kwargs):
def path(self):
return self._fn

@property
def fn(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using .fn can't you just use .path and there's less code in total?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I can, yep! Had just been following the convention that was already in the module.

Switched over to using .path - that works fine. Also removed extra code that had been added in mock.py.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad for setting a poor example within salesforce.py!

@dacort dacort changed the title [WIP] #1685 Add support for multiple results in salesforce batch jobs #1685 Add support for multiple results in salesforce batch jobs May 18, 2016
@Tarrasch Tarrasch merged commit 2a0ff1f into spotify:master May 19, 2016
@Tarrasch
Copy link
Contributor

Good job. Thanks guys :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants