[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: Add support to specifying If-Match/If-None-Match when reading an object #3228

Open
segevfiner opened this issue Nov 18, 2020 · 7 comments
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@segevfiner
Copy link

Is your feature request related to a problem? Please describe.
I'm trying to implement support for conditional requests and caching. The documentation states that GCS does support If-Match/If-None-Match (But not If-Modified-Since/If-Unmodified-Since?), but setting them is not supported in the client.

Describe the solution you'd like
Support setting If-Match/If-None-Match for reading an object.

Describe alternatives you've considered
Sending the request without this client?

Additional context
Documentation: https://cloud.google.com/storage/docs/generations-preconditions#_JSONAPI https://cloud.google.com/storage/docs/json_api/v1/parameters

google/go-cloud#2881

@segevfiner segevfiner added the triage me I really want to be triaged. label Nov 18, 2020
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Nov 18, 2020
@segevfiner segevfiner changed the title storage: Add support to specifying If-Match/If-None-Match for reading an object storage: Add support to specifying If-Match/If-None-Match when reading an object Nov 19, 2020
@tritone tritone added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Nov 19, 2020
@tritone
Copy link
Contributor
tritone commented Nov 19, 2020

Thanks for pointing this out. It looks like we have support for IfGenerationMatch and IfMetagenerationMatch in this library, but not If-Match/If-None-Match.

One likely reason is that it looks like If-Match and If-None-Match are only supported via the XML API, while this library uses the JSON API for most operations. (e.g. see https://cloud.google.com/storage/docs/json_api/v1/objects/insert#parameters which only includes the generation and metageneration headers). Reads are the main operation that actually do use XML currently but I'm hesitant to add support for a feature that's not present in JSON at all.

@frankyn any more thoughts on this, or suggestions for how to work around this gap?

@segevfiner
Copy link
Author
segevfiner commented Nov 19, 2020

Look at the documentation I pointed out in the issue's description. It states that If-Match/If-None-Match are supported by the JSON API (But not If-Modified-Since/If-Unmodified-Since?). The request headers are just documented on other pages and not on every request like the documentation of the XML API, leading to this confusion.

@tritone
Copy link
Contributor
tritone commented Nov 19, 2020

Ah I see that it is present here: https://cloud.google.com/storage/docs/json_api/v1/parameters#ifmatch . Sorry I was confused about that. And indeed it looks like If-Modified-Since and In-Unmodified-Since are not present.

I'll follow up internally to see if there's any reason we shouldn't look into adding support. In the mean time, it is possible in this library to add arbitrary headers to a request by wrapping the HTTP transport and using the WithHTTPClient option, though it's a bit clunky to execute this. Let me know if you'd like a code snippet for how to do this workaround.

@frankyn
Copy link
Member
frankyn commented Nov 23, 2020

Hi @segevfiner,

Do you have more background on your use-case and if GCS Generation (ifGenerationMatch) and Metageneration (IfMetaGenerationMatch) preconditions would be enough in this case?

@tritone is following up with the GCS backend team to get information on if Etag (If-Match/If-None-Match) should also be discouraged in libraries. We try to follow public documentation in most cases but it's not super clear if there are any reasons anyone would want to use them.

Side note: I saw a similar issue in java-storage: googleapis/java-storage#628

@segevfiner
Copy link
Author
segevfiner commented Nov 24, 2020

My use case is supporting HTTP conditional requests when serving objects from a bucket by an HTTP web server implemented in Go. This boils down to forwarding and returning the relevant headers to the object/blob store and the requesting client.

I'm using the gocloud.dev/blob package and originally asked for this there, where I discovered that this is supported by the other cloud object/blob stores, and their libraries, but not by the GCS Go library, and that it is documented that in GCS (implicitly by it being omitted) that it doesn't support If-Modified-Since & If-Unmodified-Since.

Generation and metageneration obviously won't help here, the browsers don't know about those things and require the standard HTTP headers like ETag, If-None-Match, etc.

@segevfiner
Copy link
Author

Another thing is that passing/retrieving such headers is also interesting for HEAD HTTP requests, not sure how those are exposed in the client.

@tritone
Copy link
Contributor
tritone commented Nov 30, 2020

We don't support HEAD requests specifically in the client. If you want to access metadata for example, we provide the Object.Attrs and Bucket.Attrs surfaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants