# Connecting to an LLM

When integrating a Large Language Model (LLM) into your application, whether using **Streamlit** or **FastAPI**, it's important to securely manage the connection credentials (such as an API key) and ensure smooth communication with the model. Below are the steps to connect to an LLM in a general manner.

#### 1. S**et up your LLM API Key** Additional Notes:

To securely store and access your LLM API key, use the [**Secrets Management**](/overview/building-apps-in-health-universe/deploying-your-app-to-health-universe/secret-management.md).

For example:

```python
import os

LLM_API_KEY = os.environ.get("LLM_API_KEY")
```

#### 2. **Connect to the LLM from your code**:

Once you have the API key, you can establish a connection to the LLM. The steps for connecting to the model will depend on your specific LLM provider, but here's a general example of how you might set it up:

* **Install the required library:** If your LLM provider requires a specific Python library, ensure it’s installed.
* For example: `pip install some-llm-library`
* Set up the connection in your Streamlit or FastAPI app:

```python
import some_llm_library
import os

# Set up the API key
some_llm_library.api_key = os.environ.get("LLM_API_KEY")

# Example function to call the LLM
def query_llm(prompt):
    response = some_llm_library.Completion.create(
        model="your-model-name",  # Adjust model as needed
        prompt=prompt,
        max_tokens=150
    )
    return response["choices"][0]["text"].strip()
```

* Integrate with Streamlit or FastAPI:

In Streamlit, you could use the `query_llm()` function to process user input in real-time:

```python
import streamlit as st

st.title("LLM Query Example")

user_input = st.text_input("Ask something to the model:")
if user_input:
    response = query_llm(user_input)
    st.write(response)
```

In FastAPI, you could expose an endpoint for querying the LLM:

```python
from fastapi import FastAPI

app = FastAPI()

@app.get("/query_llm")
async def query_llm_endpoint(prompt: str):
    response = query_llm(prompt)
    return {"response": response}
```

#### Additional Notes:

* **Security Considerations**: Always retrieve your API keys securely using the Secrets Management process. Never hardcode your API keys directly into your codebase.
* **Rate Limits and Costs**: Be mindful of the rate limits and associated costs of using the LLM service. Ensure that your application handles errors and retries gracefully.
* **Error Handling**: Implement error handling for cases like invalid API keys, rate limits being exceeded, or network issues.

```python
try:
    response = query_llm(user_input)
except some_llm_library.LLMError as e:
    st.error(f"Error connecting to the model: {e}")
```

By following these steps, you can securely integrate an LLM into your Streamlit or FastAPI application, ensuring both security and functionality.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.healthuniverse.com/overview/building-apps-in-health-universe/deploying-your-app-to-health-universe/connecting-to-an-llm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.