[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsub: API refactor #98

Closed
stephenplusplus opened this issue Aug 7, 2014 · 57 comments · Fixed by #245
Closed

pubsub: API refactor #98

stephenplusplus opened this issue Aug 7, 2014 · 57 comments · Fixed by #245
Assignees
Labels
api: pubsub Issues related to the Pub/Sub API. 🚨 This issue needs some love. triage me I really want to be triaged.
Milestone

Comments

@stephenplusplus
Copy link
Contributor

9/30/14:

PR: #245

9/29/14:

http://stephenplusplus.github.io/gcloud-node/#/docs/master/pubsub

9/22/14:

Outdated:

Initializing

var gcloud = require('gcloud')({ projectId: 'test-project' });
var pubsub = gcloud.pubsub();
// to override global defaults:
// var pubsub = gcloud.pubsub({ keyFilename: '/', etc: '...' });

Topics

// Create a topic
pubsub.createTopic('a-new-topic-name', function(err, topic) {});

// Get topics
pubsub.getTopics(function(err, topics) {});
pubsub.getTopics({ maxResults: 3 }, function(err, topics) {});

// Reference a topic
var topic = pubsub.topic('an-existing-topic-name');

// Publish message
topic.publish('Hi', function(err) {});

// Publish raw message
topic.publishRaw({
  data: 'Hi',
  label: [
    { key: 'priority', numValue: 0 },
    { key: 'foo', stringValue: 'bar' }
  ]
}, function(err) {});

Subscriptions

// Create a subscription
topic.subscribe('a-new-subscription-name', {
  autoAck: true
}, function(err, subscription));

// Get subscriptions
topic.getSubscriptions(function(err, subscriptions) {});
topic.getSubscriptions({ maxResults: 5 }, function(err, subscriptions) {});

// Reference a subscription
var subscription = topic.subscription('an-existing-subscription-name');

// Handle a message event
subscription.on('message', function(message) {});

// Close a subscription
subscription.close();

Example Usage

Create a topic and publish a message
pubsub.createTopic('newTopic', function(err, topic) {
  if (err) {/*...*/}
  topic.publish('Hey!', function(err) {
    if (err) {/*...*/}
  });
});
Publish a message to a topic
pubsub.topic('myTopic')
  .publish('Hey!', function(err) {
    if (err) {/*...*/}
  });
Create a subscription on a topic to listen for messages
pubsub.topic('myTopic')
  .subscribe('newSubscription', function(err, subscription) {
    if (err) {/*...*/}
    subscription.on('message', function() {
      console.log(msg);
    });
  });
Get an existing subscription on a topic to listen for messages, and auto ack them.
pubsub.topic('myTopic')
  .subscription('mySubscription', { autoAck: true })
  .on('message', function() {
    console.log(msg);
  });

Old recommended api, for historic purposes:

// New PubSub creation
var PubSub = require('lib/pubsub');
var pubsub = new PubSub({ projectId: 'test-project' });

// Create Topic
var TOPIC_NAME = 'puppies';
var PuppiesTopic = new pubsub.Topic(TOPIC_NAME);

// Publish Message (simple)
PuppiesTopic.publish('Hi', function(err) {});

// Publish Message (specific [raw format])
PuppiesTopic.publishRaw({
  data: 'Hi',
  label: [
    { key: 'priority', numValue: 0 },
    { key: 'foo', stringValue: 'bar' }
  ]
}, function(err) {});

// Subscribe to a Topic (specific [subscription name specified])
var puppyLogger = PuppiesTopic.subscribe({
  name: 'puppyLogger',
  autoAck: true,
  ackDeadlineSeconds: 60
});

puppyLogger.on('message', function(message) {
  console.log('New message from a puppy!', message);
});

// Close subscription after 5 minutes
setTimeout(puppyLogger.close, 300000);

// Get Subscriptions for a topic (simple)
PuppiesTopic.getSubscriptions(function(err, subscriptions) {});

// Get Subscriptions for a topic (specific)
PuppiesTopic.getSubscriptions({ maxResults: 5 }, function(err, subscriptions) {});

// Get Individual Subscription
PuppiesTopic.getSubscription('puppyLogger', function(err, subscription) {});

// Get Topics (simple)
pubsub.getTopics(function(err, topics) {});

// Get Topics (specific)
pubsub.getTopics({ maxResults: 5 }, function(err, topics) {});

// Get Inidividual Topic
pubsub.getTopic(TOPIC_NAME, function(err, topic) {});
@rakyll
Copy link
Contributor
rakyll commented Aug 7, 2014

Looks generally good.

Two issues:

@stephenplusplus
Copy link
Contributor Author

Is there a way we provide creation of topics...

I was thinking that would happen with the initial new pubsub.Topic('Topic Name') - if a topic with that name already exists, it would still return a "new Topic" object, but would refer to the already existing topic.

...and subscriptions?

I forgot to annotate what I did with that. Both of these bits would create a subscription:

// Subscribe to a Topic (simple [auto subscription name generated])
var puppyLogger = PuppiesTopic.subscribe({
  autoAck: true,
  ackDeadlineSeconds: 60
});

// Subscribe to a Topic (specific [subscription name specified])
var puppyLogger = PuppiesTopic.subscribe({
  name: 'puppyLogger',
  autoAck: true,
  ackDeadlineSeconds: 60
});

The first one would generate a subscription name for you, but if you wanted to provide one, simply provide a "name" property in the configuration object. I'm not sure if that's a great idea, definitely need more thoughts on that. My thought was, I may not care about naming it, as I never intend to refer to it again from another connection. If I do need to, I can provide a name. Is that a common enough scenario, or should we require user specification of a name?

publishMessage may be misleading.

publish and publishRaw sgtm. They definitely can't be mistaken for anything else that way.

@rakyll
Copy link
Contributor
rakyll commented Aug 8, 2014

If I do need to, I can provide a name. Is that a common enough scenario, or should we require user specification of a name?

I filed an issue against API that name should be generated if not provided, and team said my assumption about subscriptions being transient is wrong. In the typical scenario, a user will create a set of topics and subscriptions via a script before deploying their app. They will embed the topic and subscription names to their app config and will never run CRUD operations on topics and subscriptions from the application context.

@rakyll
Copy link
Contributor
rakyll commented Aug 8, 2014

b/16462864

@stephenplusplus
Copy link
Contributor Author

No problemo. Updated the first post with these changes.

@rakyll
Copy link
Contributor
rakyll commented Aug 8, 2014

Looks great!

@rakyll rakyll modified the milestones: M1: core, datastore & storage, M2: bigquery & pubsub Aug 8, 2014
@rakyll
Copy link
Contributor
rakyll commented Aug 11, 2014

@stephenplusplus, I think we can start working on this issue. Would you like to focus on it?

@stephenplusplus
Copy link
Contributor Author

Yep! I actually have a good start, just fine tuning and writing tests
currently.

On Monday, August 11, 2014, Burcu Dogan notifications@github.com wrote:

@stephenplusplus https://github.com/stephenplusplus, I think we can
start working on this issue. Would you like to focus on it?


Reply to this email directly or view it on GitHub
#98 (comment)
.

@rakyll
Copy link
Contributor
rakyll commented Aug 11, 2014

Great!

stephenplusplus pushed a commit to stephenplusplus/gcloud-node that referenced this issue Aug 12, 2014
stephenplusplus pushed a commit to stephenplusplus/gcloud-node that referenced this issue Aug 12, 2014
stephenplusplus pushed a commit to stephenplusplus/gcloud-node that referenced this issue Aug 12, 2014
@stephenplusplus
Copy link
Contributor Author

#107 (comment)

@rakyll
Copy link
Contributor
rakyll commented Aug 26, 2014

Opening this issue again, we still need to improve the API.

@tmatsuo
Copy link
Contributor
tmatsuo commented Aug 26, 2014

Have 2 questions.

// Subscribe to a Topic (specific [subscription name specified])
var puppyLogger = PuppiesTopic.subscribe({
name: 'puppyLogger',
autoAck: true,
ackDeadlineSeconds: 60
});

In my understanding, the above code will create a new subscription if not exist. If the subscription with that name exists, what will happen? What if the existing subscription has a push endpoint (push subscription)?

Why the connection needs projectId? Potentially Pub/Sub can be used for building a kind of system dashboard across multiple projects. In that case, you need to create as much connections as project numbers with the current lib.

@tmatsuo
Copy link
Contributor
tmatsuo commented Aug 26, 2014

Generally I like the natural taste of the suggested API. Do you have any plan for providing a similar shortcut for push subscriptions?

@rakyll
Copy link
Contributor
rakyll commented Aug 26, 2014

I think you meant push notifications. There is currently no plan for an out of the box solution, because it involves adding a handler to user's existing web server code. We can provide a middleware for that, but currently not in our scope.

The rest of the questions above is what we should address once we get back working on the API improvements.

@rakyll rakyll reopened this Aug 26, 2014
@stephenplusplus
Copy link
Contributor Author

I've updated the first post with a new proposal and simple example usages. I liked the other approach as well (at the bottom of the first post), but there was too much magic involved that could be confusing.

Please let me know your thoughts!

@stephenplusplus
Copy link
Contributor Author

// @ryanseys @silvolu

@tmatsuo
Copy link
Contributor
tmatsuo commented Sep 17, 2014

Does pubsub.getTopic() invoke the underlining API call?

@stephenplusplus
Copy link
Contributor Author

It does. To eliminate two api calls when doing one operation like subscribing to a topic, we would have to keep a subscribe function outside the scope of a topic, similar to what we have now on master:

pubsub.subscribe({
  topic: 'name',
  name: 'sub-name'
}, function(err) {});

My proposals have mainly been for creating a clear hierarchy: Topic > Subscription. However, situations like above make that tricky to pull off efficiently.

@tmatsuo
Copy link
Contributor
tmatsuo commented Sep 17, 2014

So do you think that hierarchy matters over performance?

@stephenplusplus
Copy link
Contributor Author

So do you think that hierarchy usability matters over performance?

Sometimes. In this case, I'm not sure, but am leaning towards sticking to a similar API to what we have on master, just with a fresh coat of paint. What are your thoughts? (and anyone else here?)

@tmatsuo
Copy link
Contributor
tmatsuo commented Sep 17, 2014

How about to just allow creating Topic/Subscription object without any API calls?

sofisl pushed a commit that referenced this issue Nov 11, 2022
sofisl pushed a commit that referenced this issue Nov 11, 2022
* feat: support regapic LRO

Use gapic-generator-typescript v2.15.1.

PiperOrigin-RevId: 456946341

Source-Link: googleapis/googleapis@88fd18d

Source-Link: googleapis/googleapis-gen@accfa37
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWNjZmEzNzFmNjY3NDM5MzEzMzM1YzY0MDQyYjA2M2MxYzUzMTAyZSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 11, 2022
🤖 I have created a release *beep* *boop*
---


## [2.1.0](googleapis/nodejs-service-usage@v2.0.0...v2.1.0) (2022-06-30)


### Features

* support regapic LRO ([#98](googleapis/nodejs-service-usage#98)) ([b433238](googleapis/nodejs-service-usage@b433238))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
sofisl pushed a commit that referenced this issue Nov 11, 2022
* chore(main): release 2.1.0

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 11, 2022
* feat: Add Secure Boot support to TPU v2alpha1 API

PiperOrigin-RevId: 474644226

Source-Link: googleapis/googleapis@f90b329

Source-Link: googleapis/googleapis-gen@4ad8763
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNGFkODc2M2JkZTY3NmY5MmEzZWI3MDc1M2FlMWNmZWQwZTgxMzg3ZSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 11, 2022
🤖 I have created a release *beep* *boop*
---


## [2.2.0](https://togithub.com/googleapis/nodejs-cloud-tpu/compare/v2.1.0...v2.2.0) (2022-11-10)


### Features

* Add Secure Boot support to TPU v2alpha1 API ([#98](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/98)) ([e4fc278](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/e4fc27883278b8161bb7ad598dd83021e2467d99))


### Bug Fixes

* Allow passing gax instance to client constructor ([#96](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/96)) ([d636ecf](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/d636ecf4798258a71f289bd6a6add2cf45e6a2cb))
* Better support for fallback mode ([#91](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/91)) ([a291abd](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/a291abd4a0418eb375f9c4a27f19735afee4acca))
* Change import long to require ([#92](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/92)) ([5de09bb](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/5de09bb8786a790ff5a6d643f8493b6f6ea3c4ec))
* **deps:** Use google-gax v3.5.2 ([#104](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/104)) ([86b8617](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/86b86173fe7f8dd33e5cb6abb683f32d148670c6))
* Do not import the whole google-gax from proto JS ([#1553](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/1553)) ([#95](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/95)) ([e4289c1](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/e4289c164ea2123947328ceddfb09cf083e50a19))
* Preserve default values in x-goog-request-params header ([#97](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/97)) ([42310b7](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/42310b7a99b51320e9924f8ecd1d75513b28b598))
* Regenerated protos JS and TS definitions ([#107](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/107)) ([7e3cba5](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/7e3cba5fd489373a7c460369d37f04c32e3cd9c3))
* Remove pip install statements ([#1546](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/1546)) ([#94](https://togithub.com/googleapis/nodejs-cloud-tpu/issues/94)) ([7a26fe6](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/7a26fe63b51c661bdd22bfafecc3d4291247dddf))
* use google-gax v3.3.0 ([e4289c1](https://togithub.com/googleapis/nodejs-cloud-tpu/commit/e4289c164ea2123947328ceddfb09cf083e50a19))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
sofisl pushed a commit that referenced this issue Nov 11, 2022
Source-Link: googleapis/synthtool@d229a12
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:74ab2b3c71ef27e6d8b69b1d0a0c9d31447777b79ac3cd4be82c265b45f37e5e
sofisl pushed a commit that referenced this issue Nov 11, 2022
* feat: support regapic LRO

Use gapic-generator-typescript v2.15.1.

PiperOrigin-RevId: 456946341

Source-Link: googleapis/googleapis@88fd18d

Source-Link: googleapis/googleapis-gen@accfa37
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWNjZmEzNzFmNjY3NDM5MzEzMzM1YzY0MDQyYjA2M2MxYzUzMTAyZSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 11, 2022
🤖 I have created a release *beep* *boop*
---


## [2.0.1](googleapis/nodejs-essential-contacts@v2.0.0...v2.0.1) (2022-06-30)


### Bug Fixes

* **docs:** describe fallback rest option ([#98](googleapis/nodejs-essential-contacts#98)) ([1ed45c6](googleapis/nodejs-essential-contacts@1ed45c6))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
sofisl pushed a commit that referenced this issue Nov 16, 2022
... chore: update gapic-generator-ruby to the latest commit chore: release gapic-generator-typescript 1.5.0

Committer: @miraleung
PiperOrigin-RevId: 380641501

Source-Link: googleapis/googleapis@076f7e9

Source-Link: googleapis/googleapis-gen@27e4c88
sofisl pushed a commit that referenced this issue Jan 24, 2023
* chore(deps): update dependency uuid to v3.3.0

* Update package.json
sofisl pushed a commit that referenced this issue Jan 25, 2023
* chore(deps): update dependency uuid to v3.3.0

* Update package.json
sofisl pushed a commit that referenced this issue Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the Pub/Sub API. 🚨 This issue needs some love. triage me I really want to be triaged.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants