-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Batch - Support setting the working directory #10670
Comments
What Maven artifact do you use?
Would you share URL of the document? |
We're using The documentation I use is here: https://cloud.google.com/batch/docs/create-run-job-storage#console |
Checking |
Yes, I realize that, what I ask is to be able, via the Java SDK, to set the working directory. I only refer to the cd command on the doc to explain what I want to avoid, using We mount a volume in the batch job, this volume is created from a bucket where we create a random folder for each Job. We use this random folder as the mount path so we would like to set the working directory of the Job to the mount path as we upload files in the bucket before launching the job so it would be more convenient. We have a generic task runner that can run arbitrary scripts (shell, Python, Julia, ...) in a lot of different platform including Google Cloud so we need to offer a coherent way of working on all environments and the user of our tasks runner doesn't know anything about Google Batch or any other supported task runner. |
I'd like to know more about the abstraction. Is that Docker containers (e.g., https://cloud.google.com/batch/docs/samples/batch-create-container-job)? |
Yes we use container Job. The abstraction that our user use are very high level, they describe a task in YAML and a runner (Docker, k8s, Google Batch, AWS Batch, ...) and we create the needed resources on the target runner. |
If you use Docker container, doesn't WORKDIR (https://docs.docker.com/reference/dockerfile/#workdir) solve your problem? |
I don't understand, I'm using the Batch Java SDK not the Docker SDK. |
(Re-reading your comments) Am I right this is randomness is the reason why Dockerfile's WORKDIR does not work for your case?
|
We run arbitrary Docker images that the user can supply so we cannot rely on the Dockerfile's WORKDIR, and even if we could, we create a random directory (we should be able to overcome this but the first reason will still stood). |
For the record, CloudRun allow setting a working directory from the SDK when creating a container |
Can I get the URL you read for Cloud Run's case? |
Line 3627 in 02d2b5e
|
Thank you for the reference. I explored the API definition but didn't find similar properties in the Batch API (https://cloud.google.com/batch/docs/reference/rest/v1/projects.locations.jobs#Job). Unfortunately this problem cannot be solved by Java SDK (this repository). Would you sent the feature request of setting the working directory through https://cloud.google.com/batch/docs/get-started#get-support?
|
Is your feature request related to a problem? Please describe.
When you create a Batch Job, there is no way to set the working directory; the documentation says to
cd WORKING_DIRECTORY
.It would be convenient, especially when working with volumes, to be able to se the working directory to the volume mount path.
Describe the solution you'd like
We should be able to set the task runnable working directory or at least the container when using containers as container engine support setting the working directory.
Describe alternatives you've considered
Using
cd
as a first command as the documentation explain.The text was updated successfully, but these errors were encountered: