Banque des Territoires (CDC Group) x Polyconseil x Hugging Face: Enhancing a Major French Environmental Program with a Sovereign Data Solution
AnthonyTruchet-Polyconseil, jcailton, StacyRamaherison, florentgbelidji, Violette
## Table of contents

- Case Study in English - Banque des Territoires (CDC Group) x Polyconseil x Hugging Face: Enhancing a Major French Environmental Program with a Sovereign Data Solution
    - [Executive summary](#executive-summary)   
    - [The power of RAG to meet environmental objectives](#power-of-rag)
    - [Industrializing while ensuring performance and sovereignty](#industrializing-ensuring-performance-sovereignty)
    - [A modular solution to respond to a dynamic sector](#modular-solution-to-respond-to-a-dynamic-sector)
    - [Key Success Factors Success Factors](#key-success-factors)
- Case Study in French - Banque des Territoires (Groupe CDC) x Polyconseil x Hugging Face : améliorer un programme environnemental français majeur grâce à une solution data souveraine
    - [Résumé](#resume)
    - [La puissance du RAG au service d'objectifs environnementaux](#puissance-rag)
    - [Industrialiser en garantissant performance et souveraineté](#industrialiser-garantissant-performance-souverainete)
    - [Une solution modulaire pour répondre au dynamisme du secteur](#solution-modulaire-repondre-dynamisme-secteur)
    - [Facteurs clés de succès](#facteurs-cles-succes)


<a name="executive-summary"></a>
## Executive summary

The collaboration initiated last January between Banque des Territoires (part of the Caisse des Dépôts et Consignations group), Polyconseil, and Hugging Face illustrates the possibility of merging the potential of generative AI with the pressing demands of data sovereignty.

As the project's first phase has just finished, the tool developed is ultimately intended to support the national strategy for schools' environmental renovation. Specifically, the solution aims to optimize the support framework of Banque des Territoires’ EduRénov program, which is dedicated to the ecological renovation of 10,000 public school facilities (nurseries, grade/middle/high schools, and universities).

This article shares some key insights from a successful co-development between:
- A data science team from Banque des Territoires’ Loan Department, along with EduRénov’ Director ;
- A multidisciplinary team from Polyconseil, including developers, DevOps, and Product Managers ;
- A Hugging Face expert in Machine Learning and AI solutions deployment.

<a name="power-of-rag"></a>
## The power of RAG to meet environmental objectives

Launched by Banque des Territoires (BdT), EduRénov is a flagship program within France's ecological and energy transformation strategy. It aims to simplify, support, and finance the energetic renovation of public school buildings. Its ambition is reflected in challenging objectives: assisting 10,000 renovation projects, from nurseries to universities - representing 20% of the national pool of infrastructures - to achieve 40% energy savings within 5 years. Banque des Territoires mobilizes unprecedented means to meet this goal: 2 billion euros in loans to finance the work and 50 million euros dedicated to preparatory engineering. After just one year of operation, the program signed nearly 2,000 projects but aims to expand further. As program director Nicolas Turcat emphasizes: 

> _EduRénov has found its projects and cruising speed; now we will enhance the relationship quality with local authorities while seeking many new projects. We share a common conviction with Polyconseil and Hugging Face: the challenge of ecological transition will be won by scaling up our actions._

The success of the EduRénov program involves numerous exchanges - notably emails - between experts from Banque des Territoires, Caisse des Dépôts Group (CDC) leading the program, and the communities owning the involved buildings. These interactions are crucial but particularly time-consuming and repetitive. However, responses to these emails rely on a large documentation shared between all BdT experts. Therefore, a Retrieval Augmented Generation (RAG) solution to facilitate these exchanges is particularly appropriate.

Since the launch of ChatGPT and the growing craze around generative AI, many companies have been interested in RAG systems that leverage their data using LLMs via commercial APIs. Public actors have shown more measured enthusiasm due to data sensitivity and strategic sovereignty issues.

In this context, LLMs and open-source technological ecosystems present significant advantages, especially as their generalist performances catch up with proprietary solutions currently leading the field. Thus, the CDC launched a pilot data transformation project around the EduRénov program, chosen for its operational criticality and potential impact, with an unyielding condition:  to guarantee the sovereignty of compute services and models used.

<a name="industrializing-ensuring-performance-sovereignty"></a>
## Industrializing while ensuring performance and sovereignty

Before starting the project, CDC teams experimented with different models and frameworks, notably using open-source solutions proposed by Hugging Face (Text Generation Inference, Transformers, Sentence Transformers, Tokenizers, etc.). These tests validated the potential of a RAG approach. The CDC, therefore, wished to develop a secure application to improve the responsiveness of BdT's support to communities.

Given Caisse des Dépôts (CDC) status in the French public ecosystem and the need to ensure the solution’s sovereignty and security for manipulated data, the CDC chose a French consortium formed by Polyconseil and Hugging Face. Beyond their respective technical expertise, the complementarity of this collaboration was deemed particularly suited to the project's challenges.

- Polyconseil is a technology firm that provides digital innovation expertise through an Agile approach at every stage of technically-intensive projects. From large corporations to startups, Polyconseil partners with clients across all sectors, including ArianeGroup, Canal+, France Ministry of Culture, SNCF, and FDJ. Certified Service France Garanti, Polyconseil has demonstrated expertise in on-premise and cloud deployment ([AWS Advanced Tier Services partner and labeled Amazon EKS Delivery](https://www.linkedin.com/feed/update/urn:li:activity:7201588363357827072/), GCP Cloud Architect, Kubernetes CKA certified consultants, etc.). The firm thus possesses all the necessary resources to deploy large-scale digital projects, with teams composed of Data Scientists, Data Engineers, full-stack/DevOps developers, UI/UX Designers, Product Managers, etc. Its generative AI and LLM expertise is based on a dedicated practice: Alivia, through the [Alivia App](https://www.alivia.app/), plus custom support and implementation offers.

- Founded in 2016, Hugging Face has become, over the years, the most widely used platform for AI collaboration on a global scale. Initially specializing in Transformers and publisher of the famous open-source library of the same name, Hugging Face is now globally recognized for its platform, the 'Hub',  which brings together the machine learning community. Hugging Face offers widely adopted libraries, more than 750,000 models, and over 175,000 datasets ready to use. Hugging Face has become, in a few years, an essential global player in artificial intelligence. With the mission to democratize machine learning, Hugging Face now counts more than 200,000 daily active users and 15,000 companies that build, train, and deploy models and datasets.

<a name="modular-solution-to-respond-to-a-dynamic-sector"></a>
## A modular solution to respond to a dynamic sector

The imagined solution consists of an application made available to BdT employees, allowing them to submit an email sent by a prospect and automatically generate a suitable and sourced project response based on EduRénov documentation. The agent can then edit the response before sending it to their interlocutor. This final step enables alignment with the agents' expectations using a method such as Reinforcement Learning from Human Feedback (RLHF).

The following diagram illustrates this:

<p align="center">
    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/llama2-non-engineers/diagram_en.png" alt="RLHF" width=90%>
</p>

### Diagram explanation

1. A client sends a request by email through existing channels.
2. This request is transferred to the new user interface.
3. Call to the Orchestrator, which builds a query based on an email for the Retriever.
4. The Retriever module finds the relevant contextual elements indexed by their embeddings from the vector database.
5. The Orchestrator constructs a prompt incorporating the retrieved context and calls the Reader module by carefully tracing the documentary sources.
6. The Reader module uses an LLM to generate a response suggestion, which is returned to the agent via the user interface.
7. The agent evaluates the quality of the response in the interface, then corrects and validates it. This step allows for the collection of human intelligence feedback.
8. The response is transferred to the messaging system for sending.
9. The response is delivered to the client, mentioning references to certain sources.
10. The client can refer to the public repository of used documentary resources.

To implement this overall process, four main subsystems are distinguished:
- In green: the user interface for ingesting the documentary base and constituting qualitative datasets for fine-tuning and RLHF.
- In black: the messaging system and its interfacing.
- In purple: the Retrieval Augmented Generation system itself.
- In red: the entire pipeline and the fine-tuning and RLHF database.

<a name="key-success-factors"></a>
## Key Success Factors Success Factors

The state-of-the-art in the GenAI field evolves at a tremendous pace; making it critical to modify models during a project without significantly affecting the developed solution. Polyconseil designed a modular architecture where simple configuration changes can adjust the LLM, embedding model, and retrieval method. This lets data scientists easily test different configurations to optimize the solution's performance. Finally,  this means that the optimal open and sovereign LLM solution to date can be available in production relatively simply.

We opted for a [modular monolith](https://www.milanjovanovic.tech/blog/what-is-a-modular-monolith) in [hexagonal architecture](https://herbertograca.com/2017/11/16/explicit-architecture-01-ddd-hexagonal-onion-clean-cqrs-how-i-put-it-all-together/) to optimize the design workload. However, as the efficient evaluation of an LLM requires execution on a GPU, we outsourced LLM calls outside the monolith. We used Hugging Face's [Text Generation Inference (TGI)](https://huggingface.co/docs/text-generation-inference/index), which offers a highly performant and configurable dockerized service to host any LLM available on the Hub.

To ensure data independence and sovereignty, the solution primarily relies on open-source models deployed on a French cloud provider: [NumSpot](https://numspot.com/). This actor was chosen for its SecNumCloud qualification, backed by Outscale's IaaS, founded by Dassault Systèmes to meet its own security challenges.

Regarding open-source solutions, many French tools stand out. In particular, the unicorn [Mistral AI](https://mistral.ai/fr/) is one of them, whose [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) model is currently used within the system’s Reader. However, other more discreet yet specific projects present strong potential to meet our challenges, such as [CroissantLLM](https://huggingface.co/blog/manu/croissant-llm-blog), which we are evaluating. This model results from a collaboration between the [MICS laboratory](https://www.mics.centralesupelec.fr/) of CentraleSupélec and [Illuin Technology](https://www.illuin.tech/). They aim to provide an ethical, responsible, and performant model tailored to French data.

Organizationally, we formed a single Agile team operating according to a flexible ScrumBan methodology, complemented by a weekly ritual of monitoring and training on AI breakthroughs. The latter is led by the Hugging Face expert from its [Expert Support program](https://huggingface.co/support). This structure facilitates a smooth transfer of skills and responsibilities to the BdT Data teams while ensuring regular and resilient deliveries amidst project context changes. Thus, we delivered an early naive MVP of the solution and both qualitative and quantitative evaluation notebooks. To this end, we utilize open-source libraries specializing in the evaluation of generative AI systems, such as RAGAS. This serves as the foundation upon which we iterate new features and performance improvements to the system.

Final Words from Hakim Lahlou, OLS Groups Innovation and Strategy Director at Banque des Territoires loan department:
> _We are delighted to work at Banque des Territoires alongside these experts, renowned both in France and internationally, on a cutting edge fully sovereign data solution. Based on this pilot program, this approach opens a new pathway: this is likely how public policies will be deployed in the territories in the future, along with the necessary financing for the country's ecological and energy transformation. Currently, this approach is the only one that enables massive, efficient, and precise deployment._

_Are you involved in a project that has sovereignty challenges? Do you want to develop a solution that leverages the capabilities of LLMs? Or do you simply have questions about our services or the project? Reach out to us directly at alivia@polyconseil.fr._

_If you are interested in the Hugging Face Expert Support program for your company, please contact us [here](https://huggingface.co/contact/sales?from=support) - our sales team will get in touch to discuss your needs!_