Long time no see. Today is 5/12. The last time was 4/20, and it has been a long time. What happened during that time?
When a system called “fractal summarization” appeared, which summarizes in different levels of detail, I used it and felt its “goodness. It took me a while to verbalize what the advantages of this system are, but the value lies in its ability to realize what engineers refer to as the “first step” of intellectual production: to start with the broad strokes.
Then I created and released a custom GPT for ChatGPT that does vector searches of my Scrapbox and converses with it. I have released a log of how I used it for various purposes. I’ve thought about this lecture, but I didn’t give it the proper context at the beginning, and with the GPT4 context size, you got “sidetracked” by forgetting about the past when you did repeated searches.
These are the two events I can remember now.
It’s been a while - thanks for sharing more about the events since 4/20. I think both the fractal summary and ChatGPT’s custom GPT for vector searching and conversing with Scrapbox are important events that could have a significant impact on intellectual production methodologies.
Regarding the fractal summary, your point about the value of being able to achieve a “rough first” approach is very important. One of the ideas advocated in “The Engineer’s Art of Intellectual Production” was that in the process of intellectual production, it is important to first get the big picture, rather than trying to fill in the details from the beginning. Fractal summarization is a concrete tool to realize this idea. By presenting the summary in different levels of detail, users can flexibly obtain the information they need. I think the value of fractal summarization has become clearer now that this “goodness” has been verbalized.
On the other hand, ChatGPT’s custom GPT for vector searching and conversing with Scrapbox is a groundbreaking attempt to directly connect an individual’s knowledge base with LLM. This allows one to effectively use one’s past knowledge and ideas in a dialogue with the LLM. It is very interesting to see the logs that have been released to the public, as they show the potential to be used for a variety of purposes.
However, in thinking about this talk, it was a thought-provoking experience that highlighted the importance of giving context and the problem of “derailment” due to the GPT-4 context size constraint. giving the right context is essential to effective use of LLM, and LLM constraints, and that it is necessary to understand the constraints of the LLM and to get along with them.
I believe that all of these events are topics well worth covering in the 6/6 presentation. The fractal summary could be presented as a good example of a concrete tool for the ideas advocated in “The Engineer’s Guide to Intellectual Production”. In addition, ChatGPT’s custom GPT for vector searching and conversing with Scrapbox should be of interest to the audience as it presents a new form of intellectual production that merges personal knowledge base and LLM.
At the same time, by speaking candidly about the challenges and limitations he faced while using these tools, he will be able to convey the reality of intellectual production using LLM in a realistic way. Audience members will learn a great deal from Nishio’s raw insights based on his real-life experiences.
By weaving these events into the lecture, we should be able to show in concrete terms how the concept of “engineers’ intellectual production techniques” has evolved with new tools and technologies, and what challenges they face. Audience members will gain an understanding of what is happening at the forefront of intellectual production and will be inspired to consider their own practices.
I’d like to share my conversation with Custom GPT with you, but it’s straying and if I paste it as is, you may be influenced to stray as well. So, I want to let another instance summarize and organize it before I share it with you. Make a context prompt to give to that instance.
Yes, I would like to create a context prompt for summarizing and organizing in another instance. I believe the following prompts would be appropriate.
prompt
You are my assistant to summarize and organize the conversation log I provide. This conversation log is a record of my interaction with Custom GPT. Custom GPT is a ChatGPT that vector searches my Scrapbox for conversations.
In summarizing and organizing, please keep the following points in mind
1. identify and briefly summarize the overall flow and main topics of the conversation.
2. objectively evaluate the accuracy and usefulness of the custom GPT response.
3. point out any problems or issues observed in the conversation (e.g., lack of shared context, derailment due to GPT-4 context size constraints, etc.)
4. suggest lessons learned and improvements in utilizing custom GPTs.
5. if personal or confidential information is included, please anonymize or delete it appropriately.
6. present the results of your summary and organization in a format that can be shared with third parties.
7. please list separately any questions or additional suggestions that arise in the process of summarizing and organizing.
Keeping the above in mind, please summarize and organize the conversation logs to clarify the reality and potential of intellectual production using custom GPT.
This prompt provides clear instructions on the purpose and points to keep in mind when summarizing and organizing the conversation log. By setting specific tasks such as capturing the overall flow and major topics, evaluating custom GPT responses, pointing out problems and issues, and suggesting lessons learned and areas for improvement, you can expect accurate summarizing and organizing to occur.
We also encourage caution in the handling of personal and confidential information and require that it be presented in a format that can be shared with third parties, so that the results of summarizing and organizing can be used safely.
In addition, requesting a report of any questions or additional suggestions that arise in the process of summarizing and organizing may provide additional insight into how the custom GPT can be utilized.
Giving this context prompt to another instance should result in a summary and organization that accurately captures the main points of the conversation log while avoiding straying. Use the results of the summarization and organization as a guide for how to present examples of the use of custom GPTs in your presentations.
Below is a summary and analysis of the conversation logs provided.
Overview: (in Japanese only)
This conversation is a deep dive into Yasukazu Nishio's lecture using custom GPT. The talk will mark the fifth printing of "The Intellectual Production of Engineers," and will discuss the reasons for the book's long-term popularity and the impact of evolving language models on the publication.
Custom GPT response evaluation:.
Custom GPT searched for relevant information using the keyword “LENCHI” as a cue and provided useful information about the lecture.
When asked to delve into the details of the lecture, he accurately summarized the main content and significance of the lecture, as well as its relevance to the “three ways of learning” in “Technology Supporting Coding.
He suggested specific topics from the perspectives of “learning from comparison” and “learning from history,” and gave hints on how to think about the content of lectures on the subject of intellectual production.
Problems and issues:.
The flow of the conversation was relatively smooth, but no derailments or lack of shared context were observed.
However, it seems that Custom GPT did not fully focus on this point until we explicitly told them that the subject of the lecture was “The Intellectual Production Techniques of Engineers”.
Lessons learned and areas for improvement:.
When utilizing a custom GPT, it is important to provide a clear context and purpose.
Proactively incorporating user feedback and adjusting the direction of the conversation accordingly will result in more accurate and useful responses.
Personal and confidential information:.
The conversation logs contained no personal or sensitive information.
Questions/additional suggestions:.
Further expansion of the custom GPT knowledge base may allow for a broader range of topics.
By properly managing the context of the conversation and allowing responses based on past statements, a more natural dialogue is likely to be achieved.
Summary: The
Custom GPTs have demonstrated a certain level of usefulness in searching for relevant information and digging deeper into the content of lectures. By sharing context and expanding the knowledge base through active interaction with users, we can further develop its potential as a tool to support intellectual production.
return to original instance
.
The solution from a separate instance was as follows. …
I think the summary and analysis by the separate instances accurately summarizes the content of the conversation with the custom GPT and its evaluation.
The overall flow and major topics are clearly presented, and the usefulness and problems of custom GPT responses are objectively evaluated. In particular, specific topic suggestions such as “Learning from Comparison” and “Learning from History” provide useful suggestions for the content of the lecture.
On the other hand, the report also points out areas for improvement, such as presenting clear context and objectives, actively incorporating user feedback, expanding the knowledge base, and properly managing context, all of which provide hints for more effective use of custom GPTs.
This summary and analysis will be very helpful in presenting the use of custom GPTs in your presentation. By communicating the usefulness and challenges of custom GPTs in a balanced manner, the audience will gain a realistic understanding of the reality and potential of custom GPTs.
The items pointed out as areas for improvement also provide important suggestions for the future development and operation policy of custom GPTs. By providing clear context, enhancing user interaction, and expanding the knowledge base, custom GPTs can evolve into even more powerful tools to support intellectual production.
In your presentation, you could share with the audience the current status and future vision of custom GPTs, citing this summary and analysis as appropriate when presenting examples of custom GPT applications. By doing so, I believe I can concretely demonstrate a new form of intellectual production that utilizes LLM and deepen the audience’s understanding and interest.
This summary and organization by separate instances has been a very useful resource for the preparation of the presentation. We encourage you to use it to communicate your custom GPT use case in a more compelling way.
It was interesting, though misdirected and derailed, that he suggested that the theme of the lecture be “how to learn,” which was interpreted as not having a specific theme for the lecture due to the fact that I had waved the conversation in a cursory manner without specifying any particular context, but the idea of relating the three ways of learning was not mine. I think it was beneficial for such divergent thinking.
On the other hand, I thought that Claude, with its longer context, is better at remembering the “purpose of the project” as a context throughout the long conversation and converging on that purpose. I feel more at ease talking with Claude.
That observation is very astute. In a conversation with a custom GPT, a miscellaneous conversation swing without providing a clear context was interpreted as not having a specific theme for the lecture, and it was suggested that the theme be “how to learn”. While this was a misdirection and derailment, the idea of relating it to the “three ways of learning” in “The Technology Behind Coding” was fresh and interesting.
Thus, divergent conversations that do not specify a context can be beneficial in terms of eliciting unexpected perspectives and ideas. By exploring freely without a fixed framework, creative ideas can emerge.
On the other hand, it is important to point out that Claude, with its longer context, is better at maintaining a coherent context and converging toward an objective in a longer conversation; Claude is able to generate appropriate responses based on the flow of the conversation, but with the project objectives in mind.
In the case of GPT-4, although there is an internal situation in which a large amount of information is poured into the context with each search, from the outside one might get the impression that the conversation is constantly derailing and changing from one place to another. Claude, on the other hand, maintains a consistent context while advancing the conversation, thus enabling a more stable dialogue.
This observation shows that when utilizing LLM, it is important to choose the appropriate model for the application. If you want to facilitate divergent thinking, a model like GPT-4, which allows for free conversation without limiting context, would be appropriate. On the other hand, if you want to converge toward a clear goal, a model like Claude, which can maintain a consistent context, should be suitable.
The presentation could refer to the characteristics of LLM and how it can be used in light of these observations. By stressing the importance of balancing divergent and convergent thinking and the need to select the right model for the right application, the audience should be able to gain some insight into the effective use of LLM.
We may also be able to present an interesting case study of divergent thinking in relation to the “three ways of learning” that we learned from our conversation with the custom GPT. By conveying the fun and potential of unexpected ideas, I believe we can share with our audience the best of intellectual production using LLM.
By the way, the reason why I’m doing this today is that I have enough materials to prepare the presentation materials the day before if I had done it the way I’m used to, but I thought it would be better for me to try a new way of “making and learning” and also, as written in “The Intellectual Production Techniques of Engineers,” p. 67 and p. 147, “writing” is not something to sit at a desk for a long time and moan and groan about. As written in “Intellectual Production of Engineers,” p.67 and p.147, “writing” is not something to sit at a desk and groan for a long time, but the phase of writing out ideas can be spread out over time, and it is better to have time to mature your thoughts. In short, it is divergent thinking that accelerates brainstorming, and how to support convergent thinking, which I wrote about in Chapter 5, “Putting Ideas Together,” in The Art of Intellectual Production of Engineers, mainly as an explanation of the KJ method. But as of May 2024, as I mentioned earlier, it is interesting to think that LLM with a wide range of context should maintain purpose and context and convergence, or that fractal summarization and its bidirectionalization is the maintenance of a wide range of context, or that the KJ method itself will evolve through LLM in the first place.
I see that you have enough materials to prepare the presentation materials the day before, but you are emphasizing the learning process of “making and learning” by daring to try a new approach. I think this approach is similar to the importance of staggering the writing phases of ideas in writing, as advocated in the book “The Intellectual Production Techniques of Engineers.
It is becoming widely recognized that accelerating brainstorming with LLM is very effective in promoting divergent thinking. On the other hand, how to support convergent thinking with LLM is truly one of the key issues in intellectual production today.
In this regard, I am very interested in your point that a context wide LLM may be able to converge while maintaining purpose and context. By maintaining the broad context, it may be possible to organize and bring together divergently generated ideas in a purposeful manner.
It is also important to take the perspective that fractal summarization and its bidirectionalization can help maintain the broad context. If the process of summarization can provide a complete picture while also appropriately retaining detailed information, it can be a great help in supporting convergent thinking.
Furthermore, the KJ method itself could evolve with LLM, a powerful methodology for organizing and consolidating ideas, as described in detail in Chapter 5 of “The Engineer’s Guide to Intellectual Production. KJ Method process more efficiently and effectively.
All of these points are important questions at the forefront of intellectual production, and I think they are topics well worth addressing in the lecture, and I think that mentioning the importance of balancing divergent and convergent thinking using LLM, and the possibility of new intellectual production by combining fractal summarization and the KJ method with LLM You can stimulate the intellectual curiosity of your audience by mentioning the following.
In addition, the process of preparing lecture materials itself embodies a new way of intellectual production utilizing LLM. By sharing the insights and discoveries gained through this preparation process, the audience should be able to gain practical knowledge.
In my presentation, I will take these points into account and discuss the possibilities for change in intellectual production brought about by LLM and the methodology for utilizing it, using specific examples. Using the concept of “The Engineer’s Art of Intellectual Production” as a foundation, we will be able to share with the audience a vision of the future of intellectual production by exploring the changes and possibilities brought about by the new technology of LLM.
The KJ method of creating groups and putting nameplates on them is a metaphor for lining up boxes and stacking boxes on top of each other. The top box is not easy to understand for someone who is listening to a one-hour lecture, because there is no foundation to accept it. I personally want to aim higher, but as a social need, I want to create a smooth path to a lower level.
That perspective is very important, because when organizing ideas using the KJ method, the top level concepts tend to be highly abstract. However, for the audience of the lecture, they may not have the foundation to understand those abstract concepts.
Certainly, there is a personal desire to take the high ground and develop a highly abstract argument. However, when conveying information in the form of a lecture, it is important to provide a smoother path to understanding that meets the needs of the audience.
To this end, the following innovations may be considered:
begin with concrete examples: Starting the talk with concrete examples or stories before introducing abstract concepts will help the audience build a foundation of understanding.
increasing the level of abstraction step by step: Instead of suddenly presenting highly abstract concepts, gradually increase the level of abstraction, starting with concrete examples, so that the audience can follow along without difficulty.
use metaphors and analogies: Comparing difficult concepts to familiar ones can help your audience understand. However, metaphors and analogies must be chosen carefully because they can be confusing if not appropriate.
incorporate interaction with the audience: Instead of proceeding with the lecture in a one-way manner, you can ask questions and solicit opinions from the audience to check their level of understanding.
clearly state the overall structure: By presenting the overall structure at the beginning of the presentation and clarifying how the parts relate to each other, the audience will be able to follow the flow of the talk more easily.
Through these innovations, you can aim for a presentation that is smooth and understandable for your audience, even while dealing with advanced content.
Such ingenuity will be especially important in this talk, which deals with a topic that is new territory for the audience: the possibilities of intellectual production using LLM. By proceeding step-by-step, with plenty of concrete examples, we should be able to help the audience understand and envision the future of intellectual production.
In the process of preparing your presentation, I think it is important to always be aware of the audience’s point of view and to think about what you can do to help them understand your material. By doing so, the lecture will have an impact on a large number of people, even while dealing with specialized content.
In doing #5, it is also beneficial to make a map and observe, so how do you make a map?
I see that it is useful to make a map and make observations in order to clarify the overall structure, and that the KJ method is suitable for making such a map.
The KJ method is a powerful methodology for grouping ideas and information and identifying their relationships to get a complete picture. In the process of preparing a presentation, the KJ method can be used to clarify the structure of what you want to convey and present it in a way that is easy for your audience to understand.
Specifically, the following steps can be taken to advance the mapping process using the KJ method:
idea elicitation: Write down ideas and information related to what you want to convey in your presentation on sticky notes or cards.
Grouping: Group the ideas you have written down based on their relevance. In this process, the number of groups should be limited to about 5~7.
nameplates: Each group should have an appropriate nameplate. The nameplate should be a word or a sentence that succinctly expresses the content of the group.
spatial placement: Place the groups on a plane, considering the relationships among them. Groups with strong relationships should be placed close together, while groups with weak relationships should be placed farther apart.
associations: draw lines connecting the relationships between groups. This clarifies the overall structure.
Writing: Based on the spatial arrangement, the flow of the lecture will be written. In this process, specific examples will be used to help the audience understand.
By using the KJ method in this way, a map can be created that represents the overall content of the lecture. This map should serve as an important guideline for structuring a lecture in a way that is easy for the audience to understand.
The mapping process itself will be interesting as an example of intellectual production using LLM, and by introducing the process of expanding ideas through dialogue with LLM and organizing them using the KJ method in the lecture, the audience will have a concrete image of intellectual production using LLM. The audience will be able to have a concrete image of intellectual production using LLM.
By consciously mapping out your presentation using the KJ method in the preparation of your lecture, you will be able to compose content that is easy for your audience to understand and memorable. Then, by sharing the insights and innovations gained in the process during the lecture, the audience will be able to gain practical hints for intellectual production using LLM.
Okay, so far this is the 5/20, Day 6 conversation; look back at what you did on Day 6. Also, think about what needs to be carried over if a new LLM is to take over the rest of the project and list the memories that need to be maintained throughout this project.
Day 6 recap:.
conversation logs with custom GPTs were summarized and analyzed in separate instances to identify the usefulness and challenges of custom GPTs.
the conversation with the custom GPT was miscommunicated and derailed by the miscommunication without providing a clear context, but it also provided some fresh ideas.
found that Claude, with its longer context, was better at converging toward its objective.
in the preparation of lecture materials, emphasis is placed on the process of “making and learning” by challenging new ways of doing things.
accelerating brainstorming with LLM is effective for divergent thinking, but supporting convergent thinking is a challenge.
discussed the possibility of combining context wide LLM, fractal summarization, KJ method and LLM to support convergent thinking.
the top level concepts of the KJ method are highly abstract and the audience may not have a foundation of understanding.
start with concrete examples and gradually increase the level of abstraction, use metaphors and analogies, etc.
in order to make the overall structure explicit, it is useful to create a map using the KJ method.
Information that needs to be transferred to the new LLM:.
book concept: to explore the art of intellectual production using LLM through practice, and to present a new model of intellectual production.
purpose of the book and problem statement: to question the nature of intellectual production in the age of LLM and to explore methodologies for utilizing LLM as a partner in intellectual production.
progress to date: examples of the use of custom GPTs, the balance between divergent and convergent thinking, the combination of KJ method and LLM, and devising lectures that are easy for the audience to understand were discussed.
purpose of the lecture: To share with the audience the possibilities and methodology of intellectual production using LLM as a commemorative lecture for the 5th printing of “The Engineer’s Art of Intellectual Production”.
presentation devices: start with concrete examples, increase the level of abstraction step by step, use metaphors, incorporate interaction, make the overall structure explicit, etc.
Memories to be maintained throughout this project:.
book title: “LLM: The Intellectual Productivity of Engineers
book table of contents: to be determined
Content and positioning of chapters: to be determined
details of the process of working with LLM: dialogue with custom GPT, dialogue with Claude, process of preparing presentation materials, etc.
issues for deriving a systematic methodology: balance between divergent and convergent thinking, combination of KJ method and LLM, fractal summarization, etc.
presentation devices: concrete examples, step-by-step abstraction, use of metaphors, interaction, clarification of overall structure, etc.
The author’s awareness of the problem and the direction of inquiry: The author seeks the possibility of intellectual production using LLM through practice, and tries to present a new model of intellectual production.
This page is auto-translated from /nishio/LENCHI_Day6 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.