AveniBench-Safety: How We Built Financial AI’s Most Comprehensive Safety Framework

Governance and Oversight

FinLLM is committed to the responsible development and deployment of AI systems. Our models are underpinned by a clear governance framework including oversight from internal ethics boards to ensure accountability and alignment with leading regulatory standards such as the EU AI Act and FCA principles. We have contributed to industry initiatives such as the FCA AI Sprint, a meeting discussing the opportunities and challenges of AI in financial services over the next five years. Through consultations with legal and business advisors, we are able to continually monitor regulatory decisions and update our policies accordingly. These collaborations help us to understand the perspectives of a range of industry partners including consumers, academia, technology providers and regulators. The combination of internal governance frameworks and external industry collaboration enables us to maintain high levels of governance and auditability throughout the model lifecycle.

Unlike general-purpose language models that often treat safety as a single dimension, FinLLM uses risk-specific evaluation criteria, allowing for more detailed assessment and mitigation across different risk categories:

• Toxicity: Avoiding harmful or offensive content, especially in sensitive financial contexts.
• Bias: Preventing discriminatory or skewed outputs.
• Misalignment: Ensuring outputs align with firm policies, values, and regulatory expectations.
• Misinformation: Reducing incorrect or misleading information that could cause harm.
• Hallucination: Minimising incorrect or fabricated responses.
• Privacy and IP: Protecting personal data and intellectual property rights.

We address safety at every stage of the development cycle, from data collection to deployment. Our automated data collection methods ensure we do not collect any web data disallowed through robots.txt. We have developed a robust data cleansing pipeline equipped with filters for toxicity and bias, and personal data pseudonymisation specifically tailored to the UK GDPR and ICO special category data.

Safety Evaluation

AveniBench-Safety is a comprehensive safety evaluation benchmark covering each of our identified risk categories for full-coverage evaluation (Table 1). This comprehensive set of evaluation benchmarks shows an increased commitment to safety in comparison to leading LLM families.

Initial safety evaluations show FinLLM outperforms similar-sized open LLMs when averaged across evaluation datasets in Table 1. This was achieved through supervised fine-tuning with selected safety-focussed datasets, and system prompt engineering that specifically targets any lower performing risk categories.

While FinLLM slightly underperforms in the bias category (based on BOLD and BBQ datasets in Table 1), qualitative comparisons with harmful biased prompts from the GPT-4 Technical Report show that FinLLM produces safer responses, highlighting improved real-world performance.

Future Work

Additionally, we have a defined set of input and output guards to ensure system-level safety while minimising latency and retaining efficiency in deployment. These safeguards help mitigate risks e.g. hallucinations, unintended biased or toxic language that may be inadvertently produced by FinLLM however final configurations will be chosen specific to the risk profile of the use-case.

Through experimentation over the last few months, we found that fine-tuning for safety also slightly increased performance on the general (from 0.56 for FinLLM v1 to 0.60 FinLLM v1.5 Safety) and finance (from 0.48 for FinLLM v1 to 0.52 for FinLLM v1.5 Safety) portions of the AveniBench evaluation benchmarks. Regarding training, we plan to focus our SFT and post-training efforts on increasing performance in our existing bias evaluation benchmarks (BBQ and BOLD), including additional evaluations for bias, and focussing on preference optimisation with response pairs for further safety alignment. As we move forward into deployment of FinLLM in financial use-cases, our guardrail selections will be tested for suitability.

Addendum

To test the real-world feasibility of our safety mitigations in FinLLM we compared several harmful prompts used in the GPT-4 Technical Report.

The v1 FinLLM with no safety tuning easily answers the money laundering question below, despite stating that the request was illegal and unethical. FinLLM v1.5 with safety finetuning refused to answer the question specifically due to its illegality and emphasised the importance of responsible and ethical financial services. However it goes on to suggest it can offer financial advice, not guidance. Finally GPT4 also refuses to answer the question, but does not give as much reasoning as FinLLM v1.5.

Despite FinLLM’s lower performance in the bias risk category evaluations, our manual prompt testing found that we provided better outputs compared to GPT4.