Connecting to an external data source
Last updated
Was this helpful?
Last updated
Was this helpful?
When integrating an external data source (such as a database, file system, etc.) into your application, whether using Streamlit or FastAPI, it’s important to manage your credentials securely and establish a reliable connection. Below are the steps to connect to an external data source in a secure and efficient manner.
To securely store and access credentials (such as API keys, database passwords, or other sensitive information), use the feature on the Health Universe website.
Important: Secrets can only contain letters, numbers, or the -
and _
characters. They must start with a letter or number. For example, DB_PASSWORD
, API_KEY
, or DATA_SOURCE_CREDENTIALS
.
Once you've created the secret, you can retrieve it securely using os.environ.get()
in your Python code. Make sure to use uppercase for the secret name when accessing it:
Once you have your credentials, you can establish a connection to the external data source. The process for connecting depends on the type of data source. Below are examples for connecting to a database and an API.
Connecting to a Database: To connect to a database (e.g., PostgreSQL, MySQL, etc.), you can use a library like psycopg2
or sqlalchemy
.
For PostgreSQL: pip install psycopg2
Connecting to an External API: To connect to an external API, you can use the requests
library.
For requests: pip install requests
Once the connection to the external data source is established, you can easily use it within your Streamlit or FastAPI app.
Streamlit Example:
FastAPI Example:
Security Considerations: Always retrieve your credentials securely using the Secrets Management process. Never hardcode sensitive data directly into your codebase.
Connection Pooling: For databases, consider using connection pooling to optimize performance and resource usage. Libraries like SQLAlchemy
or psycopg2
support pooling.
Error Handling: Always include error handling when working with external data sources, as network issues, incorrect credentials, or unavailable resources may cause failures.
By following these steps, you can securely and efficiently connect to an external data source from your Streamlit or FastAPI application.