Data Sources
Native Data Integrations
Credal integrates with many different data sources. The table below provides the status of each of the current integrations and what functionality is currently available. These integrations require an admin to enable them - as an admin, they are configurable at https://app.credal.ai/admin/data-sources.
If you don’t yet see the data source you’re looking for, we may be able to help. Feel free contact your Credal team about your use case.
Bring Your Own Data
If Credal doesn’t have out of the box support for your data source yet, there are several ways to bring your own data to Credal.
Upload to an existing system
If your content is low volume and changes infrequently, exporting to PDF or similar and uploading to a supported system like Google Drive or SharePoint is a good option.
REST API
Credal’s REST API lets you upload documents and metadata to Credal. You can use this to upload data from any source that can be accessed programmatically. For more information, see the Document Catalog API.
If you wish to opt in to permissions, you can specify permissioned principals in the API calls.
MongoDB Collection
If you can land your data in a MongoDB collection, Credal can sync this natively and map fields in MongoDB to the various fields needed to make a source usable in Credal. Your administrator can configure this in the data sources page. Contact your Credal team for help setting up.
Metadata & Smart Filtering
Collection Schemas
Collection schemas describe the expected structure of a set of documents, or a set of entities that you’d want associated with each individual one for filtering purposes. For example, a set of sales call transcripts could have a Client Name field. A schema is a way to specify the expected structure to Credal so that we can filter data and only send the relevant pieces of information to the LLM. They can be specified by pressing the Add Schema button on any document collection and specifying the expected fields and their type.
AI Entity Extraction (beta)
AI Entity Extraction alows you to specify a predetermined set of values for a field in a schema, and have an LLM auto extract these values as we crawl data.
For example, if you had a list of customer documents in Google Drive, or sales call transcripts in Sharepoint, you could specify Customer Name as an entity and a list of possible Customers to try to extract on syncing. This gives you automatic data curation and tagging, and allows users to filter on these entities just by asking questions to your Copilot. Contact your Credal team for help setting up AI Entity Extraction.
Smart Filtering (beta)
We support smart filtering for all of our data sources. This allows you to filter on fields that you may have imported. (from semi structured data sources like Salesforce and Jira) or metadata you have tagged documents with manually using the data catalog metadata API endpoint, or auto extracted entities above. This allows us to only send relevant information to the LLM. For example, you could have a series of call transcripts that you want to upload that you want to tag with whether or not they are Active. Once uploading the data, you can use the API endpoint above to tag the docs with an “active” field, and the search across only the docs where the active field is set to true. To use smart filtering, toggle on the Smart Filtering setting on your copilot and make sure you have a schema specified for your document collection. This is preconfigured automatically for the Salesforce and Jira data sources - but please confirm the types of these schemas are what you’d expect.
Note that smart filtering is still in beta. If you have any questions/see any issues, please reach out to the Credal team and we’d be happy to help.