Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage streams use 'complete' event for end of stream but built-in streams use 'finish' #362

Closed
ryanseys opened this issue Jan 23, 2015 · 5 comments
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: question Request for information or clarification. Not an issue.

Comments

@ryanseys
Copy link
Contributor

As was raised in #340, I looked into why there was a discrepancy between what the developer thought and what was the real case. There was a suggestion made to update our docs, but our docs aren't the issue here. A snippet from the tests shows the issue:

Using 'finish' event:

file.createReadStream()
.pipe(fs.createWriteStream(tmpFilePath))
.on('error', done)
.on('finish', function() {
  file.delete(function(err) {
    assert.ifError(err);

    fs.readFile(tmpFilePath, function(err, data) {
      assert.equal(data, fileContent);
      done();
    });
  });
});

Using 'complete' event:

var file = bucket.file(filenames[0]);
fs.createReadStream(files.logo.path)
  .pipe(file.createWriteStream())
  .on('error', done)
  .on('complete', function() {
    file.copy(filenames[1], function(err, copiedFile) {
      assert.ifError(err);
      copiedFile.copy(filenames[2], done);
    });
  });

Seems the only difference is the type of file that is getting piped to. In the first case, it's a regular stream from fs and in the second it's our implementation of the storage file write stream.

So my question is, should we use a consistent finish event everywhere or is this by-design or otherwise okay?

@ryanseys ryanseys added type: question Request for information or clarification. Not an issue. opinions wanted api: storage Issues related to the Cloud Storage API. labels Jan 23, 2015
@stephenplusplus
Copy link
Contributor

Our complete event comes from request: https://github.com/request/request/blob/e33a883dd412dc7a1fdd1e1f282e840faa540609/request.js#L1224

Being consistent with fs would be nice, but off hand, I'm not sure we can do that.

@ryanseys
Copy link
Contributor Author

Are you sure? I thought we intercept pretty much every event and emit our own events.

I'm looking at code here and here for example.

@stephenplusplus
Copy link
Contributor

Ooh, I forgot about our new proxy stream. Yeah, we should be able to emit whatever we choose in these cases.

@jgeewax jgeewax modified the milestone: Storage Stable Feb 2, 2015
@stephenplusplus
Copy link
Contributor

This is quite complex.

A native readable stream is done when it emits 'end', which is the same time the stream is .end()-ed. The end listener doesn't receive arguments, and if you pass arguments to stream.end(), they get written to the stream immediately prior to being ended. I wonder if this is why request uses complete, so they can do some post-processing before emitting the response headers/body/status arguments.

I'm okay with keeping complete as the consistent event for readable and writable end signals.

@ryanseys
Copy link
Contributor Author

ryanseys commented Feb 5, 2015

Oh yeah, good observation. That looks like exactly what they are doing. I'm happy with this too. It just makes it look weird when you have pipe with different streams, but in either case you have to make sure you're using the right event. complete as a consistent "yo, im done now" event is good for me.

@ryanseys ryanseys closed this as completed Feb 5, 2015
sofisl pushed a commit that referenced this issue Sep 27, 2022
Adds the following region tags:

video_detect_person_beta
video_detect_person_gcs_beta
video_detect_faces_beta
video_detect_faces_gcs_beta
sofisl pushed a commit that referenced this issue Sep 27, 2022
🤖 I have created a release \*beep\* \*boop\* 
---
## [2.7.0](https://www.github.com/googleapis/nodejs-video-intelligence/compare/v2.6.5...v2.7.0) (2020-02-13)


### Features

* face and person detection samples ([#362](https://www.github.com/googleapis/nodejs-video-intelligence/issues/362)) ([cff2f36](https://www.github.com/googleapis/nodejs-video-intelligence/commit/cff2f36a4e6294908a4e26587ed840c1ec1b03f8))


### Bug Fixes

* adds spaces to region tags, other fixes ([#369](https://www.github.com/googleapis/nodejs-video-intelligence/issues/369)) ([2b6943e](https://www.github.com/googleapis/nodejs-video-intelligence/commit/2b6943ee0685761a0076c7b8023eed4f12f93d64))
* fixes face and people detection region tags ([#367](https://www.github.com/googleapis/nodejs-video-intelligence/issues/367)) ([ab039b5](https://www.github.com/googleapis/nodejs-video-intelligence/commit/ab039b56b3bea27edf93c4db7c97241599d38419))
* refactors person and face detection samples into separate files ([#370](https://www.github.com/googleapis/nodejs-video-intelligence/issues/370)) ([eb9b400](https://www.github.com/googleapis/nodejs-video-intelligence/commit/eb9b400c24bdf306d8263ec402922b3235754034))
* updates README file with correct links ([#371](https://www.github.com/googleapis/nodejs-video-intelligence/issues/371)) ([fb2701a](https://www.github.com/googleapis/nodejs-video-intelligence/commit/fb2701a81c7476ef06ab279a8d4572f006abe831))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
sofisl added a commit that referenced this issue Oct 13, 2022
* chore(deps): upgrade gapic-generator-java to 2.4.1

PiperOrigin-RevId: 422607515

Source-Link: googleapis/googleapis@ba2ffd6

Source-Link: googleapis/googleapis-gen@73ba4ad
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNzNiYTRhZGQyMzlhNjE5ZGE1NjdmZmJkNGU1NzMwZmRkNmRlMDRkMyJ9

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: sofisl <55454395+sofisl@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 9, 2022
Adds the following region tags:

video_detect_person_beta
video_detect_person_gcs_beta
video_detect_faces_beta
video_detect_faces_gcs_beta
sofisl pushed a commit that referenced this issue Nov 9, 2022
🤖 I have created a release \*beep\* \*boop\* 
---
## [2.7.0](https://www.github.com/googleapis/nodejs-video-intelligence/compare/v2.6.5...v2.7.0) (2020-02-13)


### Features

* face and person detection samples ([#362](https://www.github.com/googleapis/nodejs-video-intelligence/issues/362)) ([cff2f36](https://www.github.com/googleapis/nodejs-video-intelligence/commit/cff2f36a4e6294908a4e26587ed840c1ec1b03f8))


### Bug Fixes

* adds spaces to region tags, other fixes ([#369](https://www.github.com/googleapis/nodejs-video-intelligence/issues/369)) ([2b6943e](https://www.github.com/googleapis/nodejs-video-intelligence/commit/2b6943ee0685761a0076c7b8023eed4f12f93d64))
* fixes face and people detection region tags ([#367](https://www.github.com/googleapis/nodejs-video-intelligence/issues/367)) ([ab039b5](https://www.github.com/googleapis/nodejs-video-intelligence/commit/ab039b56b3bea27edf93c4db7c97241599d38419))
* refactors person and face detection samples into separate files ([#370](https://www.github.com/googleapis/nodejs-video-intelligence/issues/370)) ([eb9b400](https://www.github.com/googleapis/nodejs-video-intelligence/commit/eb9b400c24bdf306d8263ec402922b3235754034))
* updates README file with correct links ([#371](https://www.github.com/googleapis/nodejs-video-intelligence/issues/371)) ([fb2701a](https://www.github.com/googleapis/nodejs-video-intelligence/commit/fb2701a81c7476ef06ab279a8d4572f006abe831))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
sofisl pushed a commit that referenced this issue Nov 10, 2022
sofisl pushed a commit that referenced this issue Nov 10, 2022
sofisl pushed a commit that referenced this issue Nov 11, 2022
sofisl pushed a commit that referenced this issue Nov 18, 2022
[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [sinon](https://sinonjs.org/) ([source](https://togithub.com/sinonjs/sinon)) | [`^9.0.1` -> `^10.0.0`](https://renovatebot.com/diffs/npm/sinon/9.2.4/10.0.0) | [![age](https://badges.renovateapi.com/packages/npm/sinon/10.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/sinon/10.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/sinon/10.0.0/compatibility-slim/9.2.4)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/sinon/10.0.0/confidence-slim/9.2.4)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>sinonjs/sinon</summary>

### [`v10.0.0`](https://togithub.com/sinonjs/sinon/blob/master/CHANGELOG.md#&#8203;1000--2021-03-22)

[Compare Source](https://togithub.com/sinonjs/sinon/compare/v9.2.4...v10.0.0)

==================

-   Upgrade nise to 4.1.0
-   Use [@&#8203;sinonjs/eslint-config](https://togithub.com/sinonjs/eslint-config)[@&#8203;4](https://togithub.com/4) => Adopts ES2017 => Drops support for IE 11, Legacy Edge and legacy Safari

</details>

---

### Renovate configuration

:date: **Schedule**: "after 9am and before 3pm" (UTC).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-security-center).
sofisl pushed a commit that referenced this issue Nov 30, 2022
* feat!: Update library to use Node 12

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Jan 10, 2023
sofisl pushed a commit that referenced this issue Jan 26, 2023
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 468790263

Source-Link: googleapis/googleapis@873ab45

Source-Link: googleapis/googleapis-gen@cb6f37a
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiY2I2ZjM3YWVmZjJhMzQ3MmU0MGE3YmJhY2U4YzY3ZDc1ZTI0YmVlNSJ9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

3 participants