Publications

2024

1. 

   Can Large Language Models Invent Algorithms to Improve Themselves?

   Yoichi Ishibashi, Taro Yano, and Masafumi Oyamada

   arXiv, 2024

   PDF

2. 

   Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization

   Yoichi Ishibashi, and Nishimura Yoshimasa

   arXiv, 2024

   PDF Code

3. 

   Subspace Representations for Soft Set Operations and Sentence Similarities

   Yoichi Ishibashi, Sho Yokoi, Katsuhito Sudoh, and 1 more author

   NAACL, 2024

   Abs PDF Code

   In the field of natural language processing (NLP), continuous vector representations are crucial for capturing the semantic meanings of individual words. Yet, when it comes to the representations of sets of words, the conventional vector-based approaches often struggle with expressiveness and lack the essential set operations such as union, intersection, and complement. Inspired by quantum logic, we realize the representation of word sets and corresponding set operations within pre-trained word embedding spaces. By grounding our approach in the linear subspaces, we enable efficient computation of various set operations and facilitate the soft computation of membership functions within continuous spaces. Moreover, we allow for the computation of the F-score directly within word vectors, thereby establishing a direct link to the assessment of sentence similarity. In experiments with widely-used pre-trained embeddings and benchmarks, we show that our subspace-based set operations consistently outperform vector-based ones in both sentence similarity and set retrieval tasks.

2023

1. 

   Knowledge Sanitization of Large Language Models

   Yoichi Ishibashi, and Hidetoshi Shimodaira

   arXiv, 2023

   PDF Code

2. 

   Evaluating the Robustness of Discrete Prompts

   Yoichi Ishibashi, Danushka Bollegala, Katsuhito Sudoh, and 1 more author

   In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, May 2023

   Abs PDF Code

   Discrete prompts have been used for fine-tuning Pre-trained Language Models for diverse NLP tasks. In particular, automatic methods that generate discrete prompts from a small set of training instances have reported superior performance. However, a closer look at the learnt prompts reveals that they contain noisy and counter-intuitive lexical constructs that would not be encountered in manually-written prompts. This raises an important yet understudied question regarding the robustness of automatically learnt discrete prompts when used in downstream tasks. To address this question, we conduct a systematic study of the robustness of discrete prompts by applying carefully designed perturbations into an application using AutoPrompt and then measure their performance in two Natural Language Inference (NLI) datasets. Our experimental results show that although the discrete prompt-based method remains relatively robust against perturbations to NLI inputs, they are highly sensitive to other types of perturbations such as shuffling and deletion of prompt tokens. Moreover, they generalize poorly across different NLI datasets. We hope our findings will inspire future work on robust discrete prompt learning.

2021

1. Multilingual Machine Translation Evaluation Metrics Fine-tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task

   Kosuke Takahashi, Yoichi Ishibashi, Katsuhito Sudoh, and 1 more author

   In Proceedings of the Sixth Conference on Machine Translation, Nov 2021

   Abs PDF

   This paper describes our submission to the WMT2021 shared metrics task. Our metric is operative to segment-level and system-level translations. Our belief toward a better metric is to detect a significant error that cannot be missed in the real practice cases of evaluation. For that reason, we used pseudo-negative examples in which attributes of some words are transferred to the reversed attribute words, and we build evaluation models to handle such serious mistakes of translations. We fine-tune a multilingual largely pre-trained model on the provided corpus of past years’ metric task and fine-tune again further on the synthetic negative examples that are derived from the same fine-tune corpus. From the evaluation results of the WMT21’s development corpus, fine-tuning on the pseudo-negatives using WMT15-17 and WMT18-20 metric corpus achieved a better Pearson’s correlation score than the one fine-tuned without negative examples. Our submitted models,hyp+src_hyp+ref and hyp+src_hyp+ref.negative, are the plain model using WMT18-20 and the one additionally fine-tuned on negative samples, respectively.

2020

1. Reflection-based Word Attribute Transfer

   Yoichi Ishibashi, Katsuhito Sudoh, Koichiro Yoshino, and 1 more author

   In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, Jul 2020

   Abs PDF Code

   Word embeddings, which often represent such analogic relations as king - man + woman queen, can be used to change a word’s attribute, including its gender. For transferring king into queen in this analogy-based manner, we subtract a difference vector man - woman based on the knowledge that king is male. However, developing such knowledge is very costly for words and attributes. In this work, we propose a novel method for word attribute transfer based on reflection mappings without such an analogy operation. Experimental results show that our proposed method can transfer the word attributes of the given words without changing the words that do not have the target attributes.

2018

1. Generating Responses based on Information Visually-Induced by Text Utterance

   Yoichi Ishibashi, and Hisashi Miyamori

   Jul 2018

   PDF

2017

1. KSU Team’s Dialogue System at the NTCIR-13 Short Text Conversation Task 2

   Yoichi Ishibashi, Sho Sugimoto, and Hisashi Miyamori

   In The 13th NTCIR Conference, Evaluation of Information Access Technologies, National Center of Sciences, Tokyo, Japan, December 5-8, 2017, Jul 2017

   PDF