[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parametrized Quantum Circuits for Reinforcement Learning stuck if using qsim backend #766

Open
jccalvojackson opened this issue Apr 26, 2023 · 4 comments
Open

Parametrized Quantum Circuits for Reinforcement Learning stuck if using qsim backend #766

jccalvojackson opened this issue Apr 26, 2023 · 4 comments

Comments

@jccalvojackson
Copy link
jccalvojackson commented Apr 26, 2023 •
edited
Loading

I've ran this notebook with no problem. However, if I try to use qsim as a backend by passing the backend backend = qsimcirq.QSimSimulator() explicitly to ReUploadingPQC, then I get the following message

2023-04-26 09:20:03.862016: W tensorflow/core/grappler/optimizers/loop_optimizer.cc:907] Skipping loop optimization for Merge node with control input: cond/branch_executed/_12

and it gets stuck there indefinitely.

Any idea of what might be going on?

To reproduce:

replace

def generate_model_policy(
    qubits: List[cirq.GridQubit],
    n_layers: int,
    n_actions: int,
    beta: float,
    observables: List[cirq.PauliString],
) -> tf.keras.Model:
    """Generates a Keras model for a data re-uploading PQC policy."""

    input_tensor = tf.keras.Input(shape=(len(qubits),), dtype=tf.dtypes.float32, name="input")
    re_uploading_pqc = ReUploadingPQC(qubits, n_layers, observables)([input_tensor])
    process = tf.keras.Sequential(
        [Alternating(n_actions), tf.keras.layers.Lambda(lambda x: x * beta), tf.keras.layers.Softmax()],
        name="observables-policy",
    )
    policy = process(re_uploading_pqc)
    model = tf.keras.Model(inputs=[input_tensor], outputs=policy)

    return model

with

def generate_model_policy(
    qubits: List[cirq.GridQubit],
    n_layers: int,
    n_actions: int,
    beta: float,
    observables: List[cirq.PauliString],
    backend="noiseless",
) -> tf.keras.Model:
    """Generates a Keras model for a data re-uploading PQC policy."""

    input_tensor = tf.keras.Input(shape=(len(qubits),), dtype=tf.dtypes.float32, name="input")
    re_uploading_pqc = ReUploadingPQC(qubits, n_layers, observables, backend=backend)([input_tensor])
    process = tf.keras.Sequential(
        [Alternating(n_actions), tf.keras.layers.Lambda(lambda x: x * beta), tf.keras.layers.Softmax()],
        name="observables-policy",
    )
    policy = process(re_uploading_pqc)
    model = tf.keras.Model(inputs=[input_tensor], outputs=policy)

    return model

and then replace

model = generate_model_policy(qubits, n_layers, n_actions, 1.0, observables)

backend = qsimcirq.QSimSimulator()

model = generate_model_policy(qubits, n_layers, n_actions, 1.0, observables, backend=backend)

versions:

gym = "==0.22.0"
tensorflow = "2.7.0"
tensorflow-quantum = "0.7.2"
qsimcirq = "0.13.3" # <0.16.0 because of incompatibility with tensorflow-quantum
@lockwo
Copy link
Contributor
lockwo commented Apr 26, 2023

My guess would be there is something going on with the interface here: https://github.com/tensorflow/quantum/blob/v0.7.2/tensorflow_quantum/core/ops/cirq_ops.py#L125, might be a good place to investigate more. Although I am interested in seeing why you want to use qsim as the backend when TFQ already uses qsim by default.

@jccalvojackson
Copy link
Author

My guess would be there is something going on with the interface here: https://github.com/tensorflow/quantum/blob/v0.7.2/tensorflow_quantum/core/ops/cirq_ops.py#L125, might be a good place to investigate more.

thank you, ill look into it.

Although I am interested in seeing why you want to use qsim as the backend when TFQ already uses qsim by default.

It was a sanity check. I actually want to benchmark QRL using different backends, including cuQuantum. So for example I want run QRL using different QSimSimulator options like:

ops = qsimcirq.QSimOptions(gpu_mode=0, use_gpu=True, max_fused_gate_size=nfused)

ops = qsimcirq.QSimOptions(gpu_mode=0, disable_gpu=False, use_sampler=False, max_fused_gate_size=nfused)

ops = qsimcirq.QSimOptions(gpu_mode=1, use_gpu=True, max_fused_gate_size=nfused)

ops = qsimcirq.QSimOptions(gpu_mode=1, disable_gpu=False, use_sampler=False, max_fused_gate_size=nfused)

ops = qsimcirq.QSimOptions(use_gpu=False, cpu_threads=ncpu_threads, max_fused_gate_size=nfused)

ops = qsimcirq.QSimOptions(disable_gpu=True, use_sampler=False, cpu_threads=ncpu_threads, max_fused_gate_size=nfused, gpu_mode=0)

Im now trying on a gpu using these images but now I face other problems that possibly require to compile tensor flow from source.

Another question. Any particular reason TFQ has pinned dependencies (as opposed to a range), could it be used with cirq>0.13?

thank you

@lockwo
Copy link
Contributor
lockwo commented Apr 27, 2023

GPU support for ops is super nascent (see: #759), I'm not 100% what state it is in so if you encounter errors, be sure to share them. Regarding pinned dependencies, there is a PR to update that (#697) but idk if it will happen.

@jccalvojackson
Copy link
Author

thank you very much!

I've added the error I get on gpu

regarding the original issue about getting stuck on cpu mode. I've run these tests tensorflow_quantum/python/layers/high_level/controlled_pqc_test.py using the qsimcirq.QSimSimulator() and they all go through with no problem. This means it is able to construct the corresponding operator. But somehow, for the inputs of the qrl model it gets stuck whereas for the inputs of the tests it does not

@jccalvojackson jccalvojackson mentioned this issue Apr 28, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lockwo @jccalvojackson