Tuesday, November 17, 2026

AI development startup rinna has open-sourced a large-scale GPT-2 language model specifically for Japanese!

rinna Corporation, a startup formed by spinning off the AI character business (XiaoIce) of Microsoft's AI & Research division, has built a large-scale GPT-2 language model specifically for Japanese and released it as open source. HuggingFace for the language model and training code.

Background of Open Source

rinna is an AI development company founded in June 2020 as a spin-out of the team that worked on AI chatbots in Microsoft's AI & Research divisionIt is.

We have presented technologies such as the "Empathetic Chat Model," which uses deep learning technology to enable AI to automatically generate context-sensitive conversational text for natural conversations with humans, and the "Speech-to-Speech System," which enables AI to express rich emotions through spoken and sung voices.

These state-of-the-art technologies have been applied to the development of our AI chatbot "Rinna" and "AI Character," an AI chatbot with customized conversational content and voice expressions that give it character, for use in corporate marketing and other applications. 

In the process of experimenting with natural language processing (NLP) for product development, we have now built a large-scale language model of GPT-2 specifically for Japanese.To contribute to the Japanese NLP research community, the language models developed and the training code for researchers to reproduce the experimental results on their own machines are available as open source on GitHub and HuggingFace, an NLP model library.I will do so. 

What are the features of the Japanese GPT-2 model?

Language models are based on machine learning of linguistic data, and are used to model conversational and writtenModel "human-like language" as a probability.The model is the one that is used to estimate the probability of the language. A good language model is one that can accurately estimate probabilities. For example,"Probability (I am a cat) > Probability (I am a cat)."The ability of the language model to estimate the 

 GPT-2 is a language model (autoregressive language model) that computes sentence probabilities from word-level probability combinations. For example,"Probability (I am a cat) = Probability (I am) x Probability (is|I am) x Probability (cat|I am, is) x Probability (in|I am, is, cat) x Probability (is|I am, is, cat, in)"The estimation is performed in a manner similar to that of the GPT-2. Using this capability, GPT-2 can automatically generate a sentence given the prefix "I am a cat" by selecting "is" as the next word from the probability estimation. 

The Japanese GPT-2 model released by rinna, Inc,Automatic generation of advanced Japanese text with common Japanese text featuresThe user/researcher can Users/researchers can also create their own models from this model by fine-tuning specific textual data.  

For example, when the sentence "Together, to the heights of artificial intelligence that no one has reached yet" is given as a Prefix,Generate response sentences in specific contexts (Demo 1: Impressions of a lecture, Demo 2: Introduction to a book)(*1). 

Demonstration 1] Sentence generation in the context of lecture impressions

Demo 2] Sentence generation in the context of a book introduction

(*1) The above demo sets the maximum number of characters for the text to be generated, not the actual full text to be generated. 

Features of rinna's Japanese GPT-2 model 

rinna's Japanese GPT-2 model has the following features 

  • As training data, the CC-100 (http://data.statmt.org/cc-100/) of open source data. 
  • Using a Tesla V100 GPU, 70 gigabytes of Japanese text was trained over an extended period of approximately one month. As a result, the model achieved a performance of approximately 18 perplexity (*2).Models are well trained and versatileIt can be said that 
  • For easy access by users/researchers,Publish developed models on HuggingFaceWe do this. In addition, to enable researchers to reproduce our results on their own machines, we have developed a toPublished the training code on GitHub.The company is doing so. 
  • The model we have released is a medium size model defined as GPT2-medium. In the future, so that users/researchers can make the best choice based on the trade-off between performance and cost,Models of different sizes will also be availableIt is. We also plan to release new models trained on different data. 
    (Â 18 perplexity refers to the performance of the GPT-2 model in predicting the next word from a previously given word, leaving only 18 options that contain the correct one. 

Future Development 

The large-scale language models being developed by rinna's research team are already widely used in our products. We will continue to research and develop more accurate and larger-scale language models that include different text styles and different amounts of data,Increase the capabilities of AI chatbots.We will continue to work on this project. We will also open source these models for the Japanese language research community. 

Company Profile 

Company name: rinna Corporation 
LocationWeWork: 39F WeWork, Shibuya Scramble Square, 2-24-12 Shibuya, Shibuya-ku, Tokyo 
Date of establishment:June 17, 2020 
Representative Director:Jean "Cliff" Chen 
Web site:https://corp.rinna.co.jp/ 
Business Description:Research, planning, development, operation and sales of AI services 

PORT Editorial Officehttps://port.creww.me/
PORT by Creww is an open innovation media for startups, entrepreneurs, entrepreneurs in recovery, and companies seeking to create new businesses on the theme of social issues, operated by Creww Co.
- Advertisment -
- Advertisment -

Featured

東芝のオープンイノベーション、協業検討に進む2社が決定 ダイナミックマッププラットフォームとArithmerを選定

ReGACY Innovation Group株式会社(東京都千代田区)は、企画・運営パートナーとして支援する株式会社東芝の「Toshiba OPEN INNOVATION PROGRAM 2026」で、協業検討に進む参加企業2社が決定したと発表した。7月に募集した共創プランの中から、ダイナミックマッププラットフォーム株式会社とArithmer株式会社を選定。両社は東芝グループの「Hybrid-3D LiDAR」と「気象データサービス」を活用し、物流・インフラ分野などでの新たなソリューション共創を目指す。12月中旬の成果発表会に向け、実証と事業性検証を進める。 本プログラムでは、2026年7月6日から7月26日まで参加企業を募集しており、応募のあった共創プランの中から、ダイナミックマッププラットフォーム株式会社、Arithmer株式会社の2社が選定されました。 両社は、東芝グループが保有する「Hybrid-3D LiDAR」および「気象データサービス」の技術・データを用いて、物流・インフラ分野やデータ活用領域における新たなビジネスソリューションの共創を目指します。 今後は、12月中旬に予定されている成果発表会に向けて実証や事業性検証を進めるとともに、事業化を見据えた協業の検討を加速していきます。

新潟日報社、オープンイノベーションプログラム2026ピッチ審査会で「三ツ目」「想ひ人」2社を選定

新潟日報社は8月4日、「新潟日報社 オープンイノベーションプログラム2026 ピッチ審査会」を開催した。Creww株式会社が提供する「Creww Growth」を活用し、5月15日から6月15日にかけて新潟県や社会が抱える課題解決に向けた協業アイデアを募集。応募61件の中から書類審査を通過した8社がピッチに参加し、審査の結果「三ツ目」(新潟県三条市)と「想ひ人」(東京都)の2社を選定した。今後は実証事業等を通じて事業化を進める予定。

【ヤマサキ × BeeInventor】高温下での過酷な作業環境から、データは命を守れるか——創業151年の工業炉建設会社とスマートヘルメット新興企業が挑む「熱中症ゼロ」の現場

明治8年創業の株式会社ヤマサキは、建設業界向けアクセラレータープログラム「ヤマサキ×オープンイノベーションプログラム2024」を通じて、スマートヘルメットを手がけるビーインベンターとのPoCを実施した。内部温度800度を超える製鉄炉の傍らで働く現場作業員の熱中症リスクをAIで早期検知するという挑戦は、今まさに第2フェーズへと移行しようとしている。プロジェクトリーダーの向井裕一氏と、BeeInventor(ビーインベンター)のNao SeoChon氏に話を伺った。

【ヤマサキ × CHAOSRU】「誰も知らない技術」を、誰もが体験できる価値へ——創業151年の工業炉建設会社とVRスタートアップが挑む、採用×認知の壁の壊し方

明治8年創業の株式会社ヤマサキは、製鉄に欠かせない築炉の技術を持ちながら、業界・企業の認知度の壁が採用の課題となっていた。この課題の突破口を探るべく参加したのが、『ヤマサキ×オープンイノベーションプログラム2024』だ。プログラムを通じて、VR・XR技術を核とする株式会社CHAOSRUとの協業をスタート。2025年7月から始まったPoC(実証実験)では、高校生向け採用説明会へのVR体験の導入により、ブース来訪者数に明らかな変化が生まれている。アクセラレータープログラムを通じたスタートアップとの協業で何が動き、何が見えてきたのか。株式会社ヤマサキのプロジェクトリーダー 田中敏彦氏と、CHAOSRU代表取締役の内藤薫氏に話を伺った。