It is with great honor that I have been named to the TIME100 Most Influential People in AI. Below is the full text of the interview.
1. How did AI help you create "Cyberpunk: Peach John"? Can you explain the process?
In a collection of essays I published in 2019, I wrote the following story:
Since 1997, when the supercomputer Deep Blue defeated the then chess champion Garry Kimovich Kasparov, humans have been unable to beat machines at chess. Even in Shogi and Go, humans cannot beat machines today. However, it is neither humans nor machines that are currently the best performers in these board games. The best results are achieved by joint human-machine teams. The same thing will be seen everywhere in the world in the future with the advancement of AI. The manga industry will be no exception.
As I wrote this story in 2019, I was eagerly awaiting the arrival of AI like Midjourney. The cyberpunk images generated by Midjourney inspired me to write the story. Then, with the help of Midjourney, I put that story into comic book form.
2. Did you feel that Midjourney created the images you really liked? Why or why not? (Can you send us an example of how one image changed into another?)
My day job is as a manga scriptwriter. I need a human manga artist to publish the stories I write. From this perspective, the current AI is not far behind compared to human artists.
There are two reasons.
First, there are extremely few things they can draw. For example, AI is still not good at drawing "hands". The hand is an emotional part that should be called a second face. In the field of manga drawing, being bad at drawing hands is a major disadvantage. Also, it is impossible to draw food, tools, animals, etc. well without special fine-tuning.
Second, AI cannot "interpret" a story as well as humans can. It can't draw anything beyond what it is commanded to do at a prompt. In comparison, a human manga artist can interpret the story independently and draw great illustrations that far exceed the scriptwriter's expectations.
From the perspective of a manga scriptwriter, image-generating AI is like MIDI from the perspective of a musician: with MIDI, you can make a machine play music. However, most musicians would probably prefer to have their music played by a human orchestra or singer.
3. How long and specific were the text requests you sent to Midjourney AI to create an image? (Can you send us an example?)
For example, the attached file is from a scene where Momotaro is found unconscious in a capsule. The prompt used to generate this image is as follows:
Unlike a year ago, when Cyberpunk Momotaro was created, it is no longer mainstream to generate illustrations using only prompts. AI users are generally using image2image to finish the text2image-generated images. It is also common to use extensions such as Openpose (ContolNet) together.
こんな感じ。無加筆なので細部はおかしいのだけど、ControlNet自体は使えている。ように見える。 pic.twitter.com/PatFwwikHq
— Rootport🧬 (@rootport) 2023年5月24日
①txt2imgで生成したやつ
— Rootport🧬 (@rootport) 2023年7月31日
②クリスタで色調補正＆背景削除
③img2imgで拡大したやつ
④Tiled Diffusionで印刷可能レベルまで拡大＆マンガ原稿用に線画追加したやつ#AIart #StableDiffusion #AIマンガ #AI呪文研究部 pic.twitter.com/QKKsRnaTT5
The public image of AI as "being able to generate the illustration you want just by typing words" is now incorrect.
4. How often would you type in the text request, and how often do you have to change the text request? What did you learn about generating images about AI?
In total, nearly 9,000 images were generated to produce Cyberpunk Momotaro. Since the full-length manga is 108 pages, I generated about 80 images per page, or roughly 16 to 26 images per frame, since there are 3 to 5 frames per page.
5. When you were using Midjourney, did you think of how it would affect the mangaka industry?
In the Japanese manga industry, a single work is almost always produced by a single manga artist. As a result, overwork has become a problem. Pulling all-nighters before deadlines is commonplace, and young manga artists often fall ill or even die. In addition, skilled assistants are always in short supply. Young manga artists without financial resources find it difficult to hire good assistants.
The development of image generation AI has the potential to dramatically improve these labor problems, and if AI can make work more efficient, it should make it possible to devote more time to creative work such as information gathering, story development, and research on manga expression.
6. You say that "Cyberpunk: Peach John" should be considered as art. Why should people consider it as art?
It is up to the reader to decide if they consider it art or not. I cannot force them to do so.
But here is a hint:
Andy Warhol's arrangement of Campbell's soup cans is recognized as art; Marcel Duchamp has many works that use randomness, all of which are recognized as art. I did not infringe on Campbell's trademark, but I did represent a story by arranging AI-generated material. If this representation is not art, then the works of Warhol and Duchamp can no longer be called art.
7. Did you receive any criticism or support from Shinchosha about your manga being fully AI-generated?
I did not receive any specific input from Shincho-sha on how to use AI. I received very general support for publishing comics.
8. You said using AI is like using a MIDI player. At what point do you draw the line between sampling from other artworks to create an AI-generated image (like in MIDI), and potentially infringing the copyright of other artists with the use of AI and its database?
Do not confuse "learning" with "plagiarism." For example, children learn their native language by learning what adults say. However, a child does not plagiarize an adult's lines every time he or she speaks. In most cases, new sentences are "generated" in the child's brain.
AI is no different: AI does not store image fragments of a dataset. For example, Stable Diffusion's model data, trained on about 2.3 billion images, is only about 4 GB. That's only one or two bytes of data per image, or in other words, one or two letters of the alphabet.
AI learns by quantifying image features. It learns images as if they were "concepts" in human terms. It is impossible to call this "plagiarism" or "collage".
We should also separate the "use" stage from the "learning" stage.
I consider image generation AI as "just a tool" like a paintbrush or Photoshop. What is allowed to be done with a paintbrush should also be allowed to be done with AI. What is not allowed to be done with a paintbrush is not allowed to be done with AI. If someone uses AI to create a work that infringes on the copyrights of others, the responsibility should be borne by the user of the AI, not the AI.
9. Do you plan to continue down this track of creating AI-generated manga? Do you plan on extending the AI-generation to writing and scripting too?
I intend to continue producing manga by AI. I already make extensive use of AI-generated images for storyboards and other applications that convey images to human manga artists.
Also, I could not have answered this interview without the translation AI. I started my career as a blogger, and my publications have been translated into several languages. My writings may have been "unauthorized learned" by the translation AI or LLM. I am one of the parties in this regard.
If there is a translation AI that has learned my writings, I would be happy to help expand the possibilities for humanity.
10. What did creating "Cyberpunk: Peach John" teach you about the manga industry in Japan?
In the Japanese illustrator industry, image-generating AI was received with near panic by some. On the other hand, in the Japanese manga artist industry, I have the impression that it was received somewhat calmly. I get the impression that only a few people are hysterically critical, and that most manga artists see it as "one more interesting tool."
In the past, when vinyl records and radio came along, the job of playing the violin in the bar was forced to change its business model. Similarly, the advent of generative AI should force many of us creators to change our business models.
However, the level of urgency differs from person to person. Manga artists may be said to be a relatively less urgent profession. This is because today's AI has not yet reached the level of being able to generate manga at the touch of a button.
On the other hand, those who used to draw illustrations of fan-art of anime and game characters and sell them through commission services are in danger because of the emergence of image-generating AI. To follow the previous analogy, the business model is similar to that of a violin-playing job in a bar.
If you go to Blue Note, you will see that the job of playing violin in a bar is not lost even today. But it would also be too optimistic to assume that everything is as it has always been.
Nevertheless, the advent of disruptive innovation is also an opportunity. In the past, vinyl records and radio were a threat to musicians before their time. But these new technologies did not diminish music; throughout the 20th century, music grew into a huge industry. As AI is used in all areas of this world, there will be as much wealth in all areas of this world in the future as radio and records have brought to music industry.
11. With a less robust manga industry now compared to before, what do you think AI can do for the industry?
I do not know the situation in other countries, but the Japanese manga industry is not stagnant.
Indeed, after peaking in 1995, paper-based manga sales have been declining in size. However, with the growth of e-book sales, the amount of sales has been growing steadily over the past four years since 2019.
By 2020, it has surpassed its former peak of 1995.
(It has been analyzed that this is probably due to the increase in the number of people reading comics as a result of the COVID-19 pandemic, which restricted out-of-home entertainment.)
In short, the Japanese manga industry is still a get-rich-quick industry. It is likely that talented young people will continue to be attracted to the industry in the future.
I believe that AI can improve every aspect of the manga industry.
I have already written about the topic of improving the working environment.
Even people who are not good at drawing will be able to produce manga. Manga artists who can already draw will be able to easily create extravagant pictures, including full color. Also, a new business may be born, such as borrowing a " style " from a popular illustrator, producing manga with an AI that can generate his/her own style, and sharing the royalties with him/her.
Also, as LLMs like ChatGPT develop, it will be possible to have AI generate interesting stories. Those who are good at drawing but not at creating stories will be greatly helped. Shuei-sha has already released an AI that consults with manga artists and gives them advice, just like an editor.
If these are combined with translation AI, it should make it easier to produce manga with an eye on overseas markets from the start.
AI will make everything better.
People ask, "Will AI take away human jobs?" However, It is wrong to think in the framework of "human vs. AI." In the world of board games, the best results today are achieved by joint human-machine teams. Keep this in mind.
=== === === === ===
Follow up questions:
(1)You said that AI can help solve the problem of overworked and underpaid manga artists. But if publishers know AI exist, won’t manga artists remain underpaid, and won’t the industry introduce more budget cuts to gain more profit? Can you be specific about the business model you see that would benefit those mangaka using AI to be paid more?
Good point! To answer this question, I need to explain the production flow of manga in Japan. In Japan, a manga is published through the following six stages:
1. planning
2. script writing
3. storyboarding
4. drawing
-4.1. rough sketching
-4.2. pen drawing
-4.3. finishing
5. phototypesetting
6. publication
Image generation AI only makes stage "4" of these a little easier.
Also, LLMs like ChatGPT are good brainstorming partners and advisors at stage “1” and “2”, but they can't write scripts as interesting as we humans can.
(Note: LLMs can only write stories that have a safe development due to the way they statistically predict the next word. In order to surprise the reader with an innovative manga story, we need to mix in statistically unpredictable scenes as part of the plot.)
A manga storyboard is called a "name" in Japan. As stage “3”, storyboarding determines how the panels of the manga will be arranged and how the story will be presented. For this task, the current image generation AI is of no help at all. Perhaps an AI dedicated to this task is needed, but I have not heard any news that it is being developed.
AI is not the first time that technological innovation has made manga production more efficient. When word processors came along, they streamlined screenplay writing. When screen-tones and Photoshop appeared, the "4. drawing" stage became more efficient. However, there is no evidence that these past innovations have caused the manga industry to decline, or that there are no more young people who want to become manga artists. Rather, manga artists have greedily adopted new technologies and challenged themselves to create new expressions. At least from the perspective of the manga industry, generative AI is nothing more than the latest iteration of a history that has been repeated in the past.
Manga is a comprehensive art form. Even if an AI that can draw like a human or write like a human emerges, it cannot replace all human work.
Of course, in the future, AI will be able to handle the entire process of manga production. Whether that will be in 5 years or 50 years, I do not know. But there will come a time when a single push of a button will automatically generate a 32-page manga. I predict that in that era, manga production will not be based on the current model of finishing a single work and publishing it, but will be a form of entertainment similar to live-streaming on YouTube or Twitch. When that era arrives, the "artists vs. publishers" argument itself will no longer be valid. The publishing industry and publishers will not go unscathed, and a whole new business will have to be established. Discussing such a distant future is an interesting idea for a science fiction novel, but I don't think it is very meaningful.
(2)To benefit from AI as a tool, access is important, but do all mangaka have equitable access to AI technology?
As you say, access is the key to getting benefits from AI.
In this regard, I am optimistic: Stable Diffusion is being developed as open source and is available to everyone for free. The capital investment to run Stable Diffusion is also not very expensive: for the equivalent of one or two months of the average monthly income of a Japanese in his/her 20s, one can buy a computer with sufficient specifications for manga production.
Incidentally, if a Japanese person in his/her 20s were to purchase a new car, he/she would need an amount equivalent to 6 to 12 months of monthly income, even if the car is inexpensive. The price of a computer that can run image-generating AI should be one of the cheapest capital investments needed for the job. Even more exciting, computer performance is improving with each passing year. On the other hand, the computer specs required to run AI are dropping rapidly due to optimization of technology.
What should be rather alarming is the regulation of AI.
Currently, anyone with a computer of modest performance can freely use AI for free. If this were to become a licensing system, accessibility would be fatally compromised.
Regulations on AI performance are also contrary to the principles of a free and democratic society.
For example, in a dictatorship, an AI that can draw a dictator's face should be banned because it can easily produce a caricature of the dictator. Those who call for restrictions on AI would do well to put their hands on their hearts and reflect on what they are advocating. Don't you think the same thing as this dictator?
We should separate the means from the ends. I believe that the ends don't justify the means, and also the means don't justify the ends too.
Obviously, we should be thoughtful about whether or not we should publish the work we produce. If it is pornographic or violent, there should be age restrictions on viewing. These are restrictions on the "ends". On the other hand, we should be cautious about restricting the "means" of expression itself. AI is not like guns or automobiles. It does not kill anyone as long as it is used privately on a local computer.
As the saying goes, "Nescit vox missa reverti," we are responsible for the words we utter and the expressions we publish. But we must not allow anyone to pull out our tongues and try to take away our voices. Whoever seeks to deprive us of our "means" of expression is an enemy of freedom, even if their motive is based on good intentions. The road to hell is paved with good intentions, in most cases.
(3)The “hand-drawn” style is still part of manga’s appeal. Do you think AI affects or diminishes this appeal?
Entertainment is not the same as transportation or communication. With automobiles, there is no need for horse-drawn carriages. With the Internet, there is no need for carrier pigeons. But radio and records do not eliminate the need for human singers and instrumentalists. Computer games do not eliminate the need for chess and Go. As Johan Huizinga pointed out, "play" is outside the realm of economic necessity and demand.
The same is true of manga. As mentioned above, Japanese manga has experienced many technological innovations in the past. However, that has not detracted from the appeal of manga. Rather, manga artists have developed manga culture by successfully blending new and old technologies.
In the early days, all visual effects in manga were drawn by human hands. Character outlines, shadows, explosions, and mental images were all drawn by hand. When a product called ScreenTone was introduced, it was widely used for visual effects. When Photoshop was introduced, some of the first manga artists began using it to create their manga. When it became possible to extract illustration-like line drawings from photographs, some manga artists began applying them to backgrounds. Today, it is not uncommon to see manga artists using software such as Unreal Engine and Unity to apply 3DCG to their manga.
On the other hand, even with such technological advances, there are still manga artists who create almost all of their work by hand.
For most manga artists, the "story they want to express" should be their first priority.
Technology is merely a means to express that story. They choose the technology that best suits the story they want to express, or the technology that is easy for them to use. Image generation AI and LLM are just new options. The performance of the current generation AIs will not detract from the appeal of manga, because manga artists are not horses and pigeons.
