[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenAI for retrieval seems not working #103

Open
j-a-m-a-l-i opened this issue Mar 14, 2024 · 4 comments
Open

OpenAI for retrieval seems not working #103

j-a-m-a-l-i opened this issue Mar 14, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@j-a-m-a-l-i
Copy link
j-a-m-a-l-i commented Mar 14, 2024 •
edited
Loading

Using OpenAI for retrieval I encountered two problems:

1- it cannot read the openai key from hal-9100.toml. I had to add it to environment section of docker-compose file. Now it have access to the key.

2- It does not have access to the pdf that I've added to the assistant. Looking at the log it shows that chunks are empty (Im not sure if this the reason for not answering question from the pdf)

|     <instructions>
hal-9100  |
hal-9100  |     </instructions>
hal-9100  |     <function_calls>
hal-9100  |
hal-9100  |     </function_calls>
hal-9100  |     <action_calls>
hal-9100  |
hal-9100  |     </action_calls>
hal-9100  |     <previous_messages>
hal-9100  |     <message>
hal-9100  |     {"content":[{"text":{"annotations":[],"value":"What are the values of QPs?"},"type":"text"}],"role":"user"}
hal-9100  |     </message>
hal-9100  |
hal-9100  |     </previous_messages>
hal-9100  |     <file>
hal-9100  |     []
hal-9100  |     </file>
hal-9100  |     <chunk>
hal-9100  |     []
hal-9100  |     </chunk>
hal-9100  |
hal-9100  |     </instructions>
hal-9100  |     <function_calls>
hal-9100  |
hal-9100  |     </function_calls>
hal-9100  |     <action_calls>
hal-9100  |
hal-9100  |     </action_calls>
hal-9100  |     <previous_messages>
hal-9100  |     <message>
hal-9100  |     {"content":[{"text":{"annotations":[],"value":"What are the values of QPs"},"type":"text"}],"role":"user"}
hal-9100  |     </message>
hal-9100  |
hal-9100  |     </previous_messages>
@j-a-m-a-l-i j-a-m-a-l-i added the bug Something isn't working label Mar 14, 2024
@louis030195
Copy link
Collaborator

Can you share what you did? What code did you run, cli, etc. @j-a-m-a-l-i

@j-a-m-a-l-i
Copy link
Author

I cloned the repo and did change the hal-9100.toml as follow:


# if you use anthropic llm
anthropic_api_key = "..."

# if you use openai - does not really make sense but possible!
openai_api_key = "..."
model_url = "https://api.openai.com/v1/chat/completions"

# if you use your own llm deployed, for example with anyscale:
#model_url = "http://ollama:11434/v1/chat/completions"
# or use ollama, check docker compose too!
# model_url = "http://ollama:11434/v1/chat/completions"
# docker compose --profile api --profile ollama -f docker/docker-compose.yml up
# or use your own llm
# model_url = "http://localhost:8000/chat/completions"

# if your own llm needs an api key (authorization bearer token), for example with anyscale:
#model_api_key = "get it here https://app.endpoints.anyscale.com/credentials"
#model_url = "https://api.endpoints.anyscale.com/v1/chat/completions"
#model_api_key = "..."

database_url = "postgres://postgres:secret@localhost:5432/mydatabase"
redis_url = "redis://127.0.0.1/"
s3_endpoint = "http://localhost:9000"
s3_access_key = "minioadmin"
s3_secret_key = "minioadmin"
s3_bucket_name = "mybucket"

I added my openai api key to hal-9100.toml and run the following:

docker compose --profile api -f docker/docker-compose.yml up

But I got the following error:


hal-9100  | [2024-03-15T15:17:42Z INFO  hal_9100_extra::llm] Automatically computed max_tokens_to_sample: 2671
hal-9100  | thread 'main' panicked at /home/runner/work/hal-9100/hal-9100/hal-9100-extra/src/openai.rs:257:41:
hal-9100  | OPENAI_API_KEY must be set: NotPresent
hal-9100  | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I added OPENAI_API_KEY to environment section in docker-compose file and it fixed that error.

Then I tried to use retrieval tool with this code:


def initialize_assistant(): 
    client = OpenAI(base_url='http://10.0.1.88:3000') 
    model = "gpt-4-0125-preview" 
    
    file_data = client.files.create(                
    file=open("../data/2019paper.pdf", "rb"),
    purpose="assistants",
    )    
    
    content = f""" You are a helpful assistant.
"""
    assistant = client.beta.assistants.create(    
    instructions=content,  
    model=model,
    tools=[{"type": "retrieval"}],
    file_ids=[file_data.id],
    )
    
    thread = client.beta.threads.create()
    return assistant.id, thread.id, file_data.id

def chat_with_assistant(user_input, assistant_id, thread_id, file_id):
    client = OpenAI(base_url='http://10.0.1.88:3000')     
        
    message = client.beta.threads.messages.create(
            thread_id=thread_id,
            role="user",
            content=user_input,            
            file_ids=[file_id],              
        )
    
    run = client.beta.threads.runs.create(
            thread_id=thread_id,
            assistant_id=assistant_id,
            #tools=[{"type": "retrieval"}],            
            )
    
    while run.status not in ['completed', 'failed', 'requires_action']:
        run = client.beta.threads.runs.retrieve(thread_id=thread_id, run_id=run.id)
        print("\n    run.status:", run.status)
        time.sleep(0.3)    

    messages = client.beta.threads.messages.list(thread_id=thread_id)    
    reply1 = messages.data[0].content[0].text.value
    reply2 = messages.data[-1].content[0].text.value
    print("\n    reply1 (message): ", reply1)  
    print("\n    reply2 (message): ", reply2)  
    
    return reply2

assistant_id, thread_id, file_id = initialize_assistant()
user_input = "What are the values of QPs in this article?"
response = chat_with_assistant(user_input, assistant_id, thread_id, file_id)
print("\nresponse: ", response)

Connecting directly to OpenAI assistant api give me the correct answer from the pdf. Connecting to hal-9100 give me this:
“I'm sorry, but I can't access or provide values from an article without specific details or content from the article itself.”
Which indicates it does not have access to the pdf.
Same thing happens if I use anyscale.
I even added the file to both assistant and the message (as u can see in the code), again no success.

@j-a-m-a-l-i
Copy link
Author

And this is the log:

hal-9100  | [2024-03-15T15:45:25Z INFO  lopdf::document] StandardEncoding
hal-9100  | [2024-03-15T15:45:25Z INFO  lopdf::document] StandardEncoding
hal-9100  | [2024-03-15T15:45:25Z INFO  lopdf::document] StandardEncoding
hal-9100  | [2024-03-15T15:45:25Z INFO  lopdf::document] StandardEncoding
hal-9100  | [2024-03-15T15:45:25Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:25Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:25Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_core::run_steps] Creating step for run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_core::executor] Calling LLM API with instructions: <instructions>
hal-9100  |     <instructions>
hal-9100  |
hal-9100  |     </instructions>
hal-9100  |     <function_calls>
hal-9100  |
hal-9100  |     </function_calls>
hal-9100  |     <action_calls>
hal-9100  |
hal-9100  |     </action_calls>
hal-9100  |     <previous_messages>
hal-9100  |     <message>
hal-9100  |     {"content":[{"text":{"annotations":[],"value":"What are the values of QPs in this article?"},"type":"text"}],"role":"user"}
hal-9100  |     </message>
hal-9100  |
hal-9100  |     </previous_messages>
hal-9100  |     <file>
hal-9100  |     []
hal-9100  |     </file>
hal-9100  |     <chunk>
hal-9100  |     []
hal-9100  |     </chunk>
hal-9100  |
hal-9100  |     </instructions>
hal-9100  |     <function_calls>
hal-9100  |
hal-9100  |     </function_calls>
hal-9100  |     <action_calls>
hal-9100  |
hal-9100  |     </action_calls>
hal-9100  |     <previous_messages>
hal-9100  |     <message>
hal-9100  |     {"content":[{"text":{"annotations":[],"value":"What are the values of QPs in this article?"},"type":"text"}],"role":"user"}
hal-9100  |     </message>
hal-9100  |
hal-9100  |     </previous_messages>
hal-9100  |
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_extra::llm] Calling OpenAI API with messages: [Message { role: "user", content: "<instructions>\n<instructions>\n\n</instructions>\n<function_calls>\n\n</function_calls>\n<action_calls>\n\n</action_calls>\n<previous_messages>\n<message>\n{\"content\":[{\"text\":{\"annotations\":[],\"value\":\"What are the values of QPs in this article?\"},\"type\":\"text\"}],\"role\":\"user\"}\n</message>\n\n</previous_messages>\n<file>\n[]\n</file>\n<chunk>\n[]\n</chunk>\n\n</instructions>\n<function_calls>\n\n</function_calls>\n<action_calls>\n\n</action_calls>\n<previous_messages>\n<message>\n{\"content\":[{\"text\":{\"annotations\":[],\"value\":\"What are the values of QPs in this article?\"},\"type\":\"text\"}],\"role\":\"user\"}\n</message>\n\n</previous_messages>\n" }, Message { role: "system", content: "You are an assistant that help a user based on tools and context you have.\n\nRules:\n- Do not hallucinate\n- Obey strictly to the user request e.g. in <message> tags - EXTREMELY IMPORTANT\n- Answer directly the user e.g. 'What is the solution to the equation \"x + 2 = 4\"?' You should answer \"x = 2\" even though receiving bunch of context before.\n- Do not add tags in your answer such as <function_calls> etc. nor continue the user sentence. Just answer the user.\n\nThese are additional instructions from the user that you must obey absolutely:\n\n You are a helpful assistant.\n\n\n" }]
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_extra::llm] Automatically computed max_tokens_to_sample: 3733
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:26Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:27Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:27Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:27Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:28Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:28Z INFO  hal_9100_core::runs] Getting run from database for thread_id: 0e4cfe20-a485-498e-ad27-7b7e4b7b3c91 and run_id: 1a006196-4e27-4c4d-831b-ad06194811b4
hal-9100  | [2024-03-15T15:45:28Z INFO  hal_9100_core::executor] LLM API output: I'm sorry, but you haven't provided an article for me to analyze and find the values of QPs. Could you please provide more details or specify the article you're referring to?

@j-a-m-a-l-i
Copy link
Author

@louis030195 Am I doing something wrong? or this can be a bug?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@louis030195 @j-a-m-a-l-i