Need preprocessor config error #488

mtkgeek · 2024-06-29T05:22:37Z

Hi, I'm trying to run the phi3-vision model using local files. I followed the instructions but ran into this error

panicked at mistralrs-core/src/pipeline/vision.rs:192:22: Need preprocessor config

Even though the instructions do not say a preprocessor config file is needed, I added that as well but still ran into the same error. I'm on a Mac btw

MY CODE

from mistralrs import Runner, Which, ChatCompletionRequest, VisionArchitecture

runner = Runner(
    which=Which.VisionPlain(
        model_id="/Users/path/to/my/phi3v_model",
        tokenizer_json=None,
        repeat_last_n=64,
        arch=VisionArchitecture.Phi3V,
    ),
)

res = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="phi3v",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "https://www.nhmagazine.com/content/uploads/2019/05/mtwashingtonFranconia-2-19-18-108-Edit-Edit.jpg"
                        },
                    },
                    {
                        "type": "text",
                        "text": "<|image_1|>\nWhat is shown in this image? Write a detailed response analyzing the scene.",
                    },
                ],
            }
        ],
        max_tokens=256,
        presence_penalty=1.0,
        top_p=0.1,
        temperature=0.1,
    )
)
print(res.choices[0].message.content)
print(res.usage)

The text was updated successfully, but these errors were encountered:

EricLBuehler · 2024-06-29T08:16:59Z

Hi @mtkgeek, I just merged #489 which should fix this. As you are using the Python API, can you please follow the instructions to install from source, and try it again?

mtkgeek · 2024-06-29T08:32:51Z

Oh nice. I'm using the pip version. I tried installing from source but keep running into errors when I run pip install maturin[patchelf].

Building wheels for collected packages: patchelf
  Building wheel for patchelf (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for patchelf (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      Traceback (most recent call last):
        File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 643, in setup
          cmkr = cmaker.CMaker(cmake_executable)
        File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 145, in __init__
          self.cmake_version = get_cmake_version(self.cmake_executable)
        File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 102, in get_cmake_version
          raise SKBuildError(msg) from err
      
      Problem with the CMake installation, aborting build. CMake executable is cmake
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for patchelf
Failed to build patchelf
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (patchelf

I guess I'll just wait for these changes to make it to pypi.

EricLBuehler · 2024-06-30T00:05:07Z

@mtkgeek I just released v0.1.23 which includes this fix. Can you please try it again after a pip reinstall?

mtkgeek · 2024-06-30T06:08:08Z

Thanks for the update @EricLBuehler. Now pip install doesn't work though

ERROR

Compiling png v0.17.13
     Compiling candle-core v0.6.0 (https://github.com/EricLBuehler/candle.git?rev=b438cba3f8742519d6f2154209e096c123fc3eb9#b438cba3)
Compiling hyper v1.3.1
  error[E0308]: mismatched types
     --> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/quantized/metal.rs:132:39
      |
  132 |         let src = crate::Storage::Cpu(src);
      |                   ------------------- ^^^ expected `CpuStorage`, found `&CpuStorage`
      |                   |
      |                   arguments to this enum variant are incorrect
      |
  note: tuple variant defined here
     --> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/storage.rs:10:5
      |
  10  |     Cpu(CpuStorage),
      |     ^^^
  help: consider using clone here
      |
  132 |         let src = crate::Storage::Cpu(src.clone());
      |                                          ++++++++

  error[E0599]: no method named `as_slice` found for enum `Storage` in the current scope
      --> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/quantized/metal.rs:136:36
       |
  136  |             storage.from_float(src.as_slice::<f32>()?)?;
       |                                    ^^^^^^^^ method not found in `Storage`
       |
      ::: /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/storage.rs:9:1
       |
  9    | pub enum Storage {
       | ---------------- method `as_slice` not found for this enum
       |
  note: the method `as_slice` exists on the type `CpuStorage`
      --> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/cpu_backend/mod.rs:1541:5
       |
  1541 |     pub fn as_slice<D: WithDType>(&self) -> Result<&[D]> {
       |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

     Compiling signal-hook-mio v0.2.3
     Compiling derive_builder_macro v0.20.0
     Compiling darling_macro v0.11.0
  Some errors have detailed explanations: E0308, E0599.
  For more information about an error, try `rustc --explain E0308`.
     Compiling itertools v0.11.0
  error: could not compile `candle-core` (lib) due to 2 previous errors
  warning: build failed, waiting for other jobs to finish...
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101"

EricLBuehler · 2024-06-30T06:51:05Z

@mtkgeek thank you for reporting this, can you please try it again after you update via pip - it should be fixed?

mtkgeek · 2024-06-30T07:03:41Z

@EricLBuehler the new version installs but the original problem still persists. PanicException: Need preprocessor config when I run the code.

EricLBuehler · 2024-06-30T09:32:05Z

@EricLBuehler the new version installs but the original problem still persists. PanicException: Need preprocessor config when I run the code.

Can you please paste the output of ls /Users/path/to/my/phi3v_model?

mtkgeek · 2024-06-30T09:34:10Z

Sure @EricLBuehler . It's

config.json                             model.safetensors.index.json
model-00001-of-00002.safetensors        tokenizer.json
model-00002-of-00002.safetensors        tokenizer_config.json

EricLBuehler · 2024-06-30T09:39:13Z

Ah ok! I think I must have forgotten to update the docs. As documented here now, if you add preprocessor_config.json this should work.

mtkgeek · 2024-06-30T09:46:55Z

Thanks, it works now. I also wanted to point out that I can see Loading model /Users/path/to/my/phi3v_model on cpu. is this normal, I was expecting "on metal" or "on mps" ?

EricLBuehler · 2024-06-30T16:23:11Z

Ok, that is good that it works. Regarding the device, can you close this issue and open another for that?

EricLBuehler added the resolved label Jun 29, 2024

EricLBuehler mentioned this issue Jun 30, 2024

Bump version to 0.1.24 #497

Merged

mtkgeek closed this as completed Jul 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need preprocessor config error #488

Need preprocessor config error #488

Need preprocessor config error #488

Need preprocessor config error #488

Comments