Please tell us about your research.

Since graduate school, I have consistently conducted research at the intersection of computer vision and computer graphics.

Computer vision is the field of recognizing and analyzing images captured by a camera using a computer. For example, it leads to real-world applications such as enabling robots to perceive real-world information through cameras and operate autonomously. On the other hand, computer graphics is the field of visually presenting computer-generated information to humans.

In other words, computer vision involves converting real-world objects into information, while computer graphics involves visualizing that information; the two are two sides of the same coin.

Virtual reality is precisely at the intersection of these two fields. During my graduate school years, I was part of a research lab in this area and also conducted research related to image-generation AI. While virtual reality has recently gained attention through films like *Avatar* (directed by James Cameron, released in 2009), the work that personally influenced me was the novel *The Perfect Insider* (written by Hiroshi Mori, published in 1996).

After securing a position as a researcher at the University of Osaka, I gradually shifted my focus to research closer to computer vision. Specifically, this involves virtually reconstructing the three-dimensional (3D) shapes of objects on a computer using images captured by a camera. In recent years, I have been particularly focused on image analysis and 3D reconstruction of plants.

Until now, computer vision has primarily focused on cars and humans—areas leading to automated driving and the use of robots to reduce labor in factories and logistics. Because vast amounts of image data can be acquired through dashcams and surveillance cameras, recent years have seen significant progress in the development of deep learning—the technology that trains AI on large volumes of images.

However, while computer vision for plants is a field that could help solve problems facing agriculture, such as reducing labor in robotic cultivation and streamlining plant breeding, research in this area has not progressed much until now.

There are two main reasons for this. The first is the scarcity of images suitable for analysis. Although some farmers have recently begun installing cameras in greenhouse cultivation, there is no accumulated dataset.

The second reason is the extreme diversity of plant morphology. Humans and cars have basically the same structure, so even if parts are not visible in an image, it is easy to infer, for example, that “an arm extends from here.” However, plants not only come in a wide variety of species, but even within the same species, there are individual differences in branch structure, leaf arrangement, and number of leaves. Even with images taken from multiple angles, it is difficult—even with the latest computer vision technology—to analyze and reconstruct what lies hidden behind the leaves.

I believe that plant image analysis and 3D reconstruction are an intriguing field where both technical challenges and social needs coexist, and I continue my research in this area.

What led you to focus your research on plants?

In terms of how I got started, I would say it was my year-long study abroad in France after earning my degree. I was affiliated with a laboratory at the National Institute for Research in Computer Science and Control (INRIA) that was working on the integration of 3D reconstruction and image generation. Incidentally, that lab has become very well-known in recent years for developing a technology called “3D Gaussian Splatting.” It’s not just the United States and China; France, the United Kingdom, Switzerland, and Germany are also leading nations in this field.

During my year-long stay in France, as a wine lover, I traveled around wine-producing regions, starting with Bordeaux. I gazed at the vineyards and enjoyed wine at wineries… At the time, it was purely a leisure trip, but after returning to Japan, I was blessed with an unexpected connection.

Before joining the Graduate School of Information Science and Technology (IST), I first secured a position at the Institute of Scientific and Industrial Research at the University of Osaka, where my supervisor, Professor Yasushi Yagi, happened to be a wine enthusiast. Through Professor Yagi’s connections, I participated in a gathering of academic researchers with a shared interest in wine and ended up visiting a vineyard in Yamanashi.

What struck me there was a scene where, while the vineyard staff was pruning the vines based on their own experience, a distinguished professor of agricultural science was offering advice, saying, “That’s not quite right.”

If pruning is done even slightly incorrectly, the grapes will not grow well. Realizing that agriculture is by no means a straightforward task, even for experienced professionals, I began to think about how computers could support them in overcoming these challenges. I reasoned that if a computer could analyze images of plants and recommend, “Prune here,” we could avoid relying on individual expertise while also aiming to improve efficiency and reduce the need for manual labor. That’s where my current research began. From 2017 to 2021, I served concurrently as a researcher for JST’s (Japan Science and Technology Agency) PRESTO program, which allowed me to meet others working in related fields and expand my research network.

In terms of efforts toward social implementation, we frequently receive inquiries from manufacturers involved in crop production and processing, as well as agricultural machinery manufacturers. It seems that corporate representatives are particularly interested in physical AI—that is, entrusting agricultural work to robots.

As such, the introduction of information technology into agriculture is a hot area for social implementation. However, personally, I feel a sense of urgency that this field must become even more prominent, or agriculture will not be able to survive. I am on good terms with researchers in related fields at the Graduate School of Agricultural and Life Sciences at the University of Tokyo and with agricultural startups, and we are conducting joint research. I hope that this field will accelerate further in both the information technology and agricultural sectors.