This development was supported by GENIAC, a project implemented by the Ministry of Economy, Trade and Industry and the New Energy and Industrial Technology Development Organization (NEDO) to strengthen the development capability of generative AI in Japan.

405B LLM/foundation model for tourism industry
In this project, we conducted continuous pre-training on a scale of 100 billion tokens for Chinese, Japanese, and Korean based on the Llama3.1 405B model, and then fine-tuned the dialogue format to develop a generic AI model similar to GPT-4. The dataset was built and trained by Deepreneur for Japanese and Ubitus for Chinese.
We have also fine-tuned the system using tourism data with a view to applying it to the tourism industry.
benchmark
The model developed by Deepreneur et al. performs well in Japanese, Traditional Chinese, Korean, and English, with significant improvements in accuracy and fluency in natural dialogue tasks.
In benchmark evaluations, the model showed excellent performance on several metrics and outperformed GPT-4 on certain tasks. All project goals were met, with the Japanese model achieving 9.18 on JA MT-Bench and 4.44 on ELYZA Tasks 100. The Traditional Chinese model also showed high performance on TMMLU+ with over 76.061 TP3T.

About Deepreneur
Deepreneur is an AI startup from the Matsuo-Iwasawa Laboratory of the University of Tokyo.
We provide customized DX solutions for corporate clients based on our philosophy of "using AI technology to create a new and unprecedented relationship between humans and technology." We provide customized DX solutions for corporate clients based on this philosophy.
We promote corporate DX through the development of AI and systems that utilize our proprietary large-scale language model (LLM) "Blue-Lizard" and other cutting-edge Deep Learning technologies.
Deepreneuer provides services for the construction of LLM unique to each company, utilizing the know-how and knowledge acquired through research and development. We provide comprehensive support from the construction of each company's unique LLM to its implementation in accordance with each company's individual issues and requirements, including not only cloud-based LLM but also on-premise LLM, which has been difficult to implement in the past.
If you are interested in our services for building your own LLM, please contact us through the link below.https://www.deepreneur.com/contact
Deepreneur Corporation|Company Profile
| name of company | Deepreneur Inc. |
| Establishment | April 28, 2023 |
| Location | HONGO EGG, 6-25-14 Hongo, Bunkyo-ku, Tokyo |
| representative | Representative Director Yuta Sawada |
| uniform resouce locator | https://www.deepreneur.com/ |









