Admin-only tools for monitoring system security and evaluating AI response quality. Track user activity logs, detect security threats, and measure response completeness and accuracy against baseline benchmarks.
Both Observability and Analytics pages are only accessible to users with the Admin role.
The Observability page provides administrators with a detailed security log of all user activity in the system, enabling oversight of access and detection of suspicious behaviour.
A timestamped record of every successful login to the system, including the user, method (password or SSO), and IP address.
Tracks when users sign out of the system, including session duration and logout method.
Flags and logs any attempts to manipulate the AI through malicious prompt injection or RAG poisoning inputs designed to alter system behaviour.
Records unauthorised or invalid attempts to modify the RAG knowledge base, including rejected uploads and policy violations.
The Analytics section provides an AI-driven evaluation of generated responses, comparing them against a baseline to measure quality across key dimensions. This section is only accessible to Admin users.
Measures whether the generated response fully addresses all aspects of the RFP question, with no key points omitted compared to the baseline.
Evaluates factual correctness of the AI-generated response against source documentation and the established baseline answers.
Tracks how response quality evolves as the content library and brand voice settings are updated, highlighting gains in completeness and accuracy.