Data Sources

Credal integrates with many different data sources. The table below provides the status of each of the current integrations and what functionality is currently available. These integrations require an admin to enable them - as an admin, they are configurable at https://app.credal.ai/admin/data-sources.

Collection Schemas

Collection schemas describe the expected structure of a set of documents, or a set of entities that you’d want associated with each individual one for filtering purposes. For example, a set of sales call transcripts could have a Client Name field. A schema is a way to specify the expected structure to Credal so that we can filter data and only send the relevant pieces of information to the LLM. They can be specified by pressing the Add Schema button on any document collection and specifying the expected fields and their type.

Beta: We have a feature called entity extraction specifically for fields that are of type string. This alows you to specify a predetermined set of values for a field in a schema, and have an LLM auto extract these values. For example, if you had a list of customer documents in Google Drive and a set of documents, you could specify Customer Name as an entity and a list of possible Customers to try to extract.

Smart Filtering

We support smart filtering for all of our data sources. This allows you to filter on fields that you may have imported (from semi structured data sources like Salesforce and Jira) or metadata you have tagged documents with manually using https://docs.credal.ai/api-reference/document-catalog/metadata, or auto extracted entities above. This allows us to only send relevant information to the LLM. For example, you could have a series of call transcripts that you want to upload that you want to tag with whether or not they are Active. Once uploading the data, you can use the API endpoint above to tag the docs with an “active” field, and the search across only the docs where the active field is set to true. To use smart filtering, toggle on the Smart Filtering setting on your copilot and make sure you have a schema specified for your document collection. This is preconfigured automatically for the Salesforce and Jira data sources - but please confirm the types of these schemas are what you’d expect.

Note that smart filtering is still in beta. If you have any questions/see any issues, please reach out to the Credal team and we’d be happy to help.