The following scientific partners involved:

NCC-Bulgaria was founded by the Institute of Information and Communication Technologies at the Bulgarian Academy of Sciences, Sofia University “St. Kliment Ohridski” and the University of National and World Economy.

NCC-Bulgaria is focused on:

• Creating a roadmap for successful work in the field of HPC, big data analysis, and AI;
• Analyzing the existing competencies and facilitating the use of HPC/HPDA/AI in Bulgaria;
• Raising awareness and promoting HPC/HPDA/AI use in companies and the public sector.

Scientific/governmental/private partners involved involved:

TBI Bank is driven by the goal to build the leading challenger bank in Southeastern Europe. As a diverse team of 19 nationalities and expertise from all around the globe, they can make their customers’ lives easier. They embrace constant learning and put clients at the heart of everything they do. This places TBI as one of the most efficient and profitable banks in SEE. Their core markets are Bulgaria, Romania, and Greece as we also have activities in Germany and Lithuania, serving a customer base of over 2.2 million individuals across these regions.

Technical/scientific Challenge:

Effectively integrating multiple high-performance Generative AI systems (such as Large Language Models) into business workflows presents multiple key technical and scientific challenges. Most state-of-the-art LLMs are built on autoregressive frameworks, meaning they generate text sequentially, predicting each token based solely on preceding tokens. Models have a finite context window, which restricts the amount of background information and detail that can be incorporated into prompts and responses. Generating lengthy or highly detailed responses often leads to issues with maintaining consistent logic, style, and accuracy across the entire output.

Currently, there is no standardized, streamlined workflow to manage the lifecycle of generative AI interactions effectively. Manually consolidating outputs from multiple LLMs requires human judgment or additional software tools, adding complexity and inefficiency to workflows.

Ensuring the final output meets high standards for quality, accuracy, readability, and originality poses additional scientific and technical problems. Combining multiple outputs intelligently requires advanced natural language processing techniques and specialized tools. Adjusting the final merged text for consistent tone, style, and grammatical correctness is critical for professionalism but introduces additional layers of complexity. Reliably detecting and mitigating potential plagiarism or originality issues is essential yet challenging, particularly when balancing automated processes and manual oversight.

Solution:

To ensure the highest quality and consistency in the final results provided to the end user, the outputs from the selected Large Language Models (OpenAI’s GPT-4o, Anthropic’s Claude 3.5 Sonnet, and Google’s Gemini 1.5) undergo a structured merging and refinement process. The merging procedure involves pairwise combinations of responses leveraging external tools such as EasyPeasy.ai and the OpenAI o1 API [1] (the current state-of-the-art reasoning model).

• Pairwise Merging Outputs from two models are merged using EasyPeasy.ai’s text-merging capability [2]. This tool intelligently consolidates distinct strengths, clarity, and coherence into one unified text, ensuring comprehensive coverage of key points and reducing redundancy. The merging is performed in pairs of two – the first two outputs – Text_Output_1 and Text_Output_2 are merged into Result_1 and then Result_1 is merged with Text_Output_3 to get to the final merged result. An alternative approach is using a reasoning model (like OpenAI’s o1 or DeepSeek R1) to perform the merging since these models can use their “thinking” to reason about the similarities and differences among the three different outputs.
• Quality Assurance of the Merged Output Once the merged output is finalized into a single coherent piece, the text undergoes additional rigorous quality assurance steps.