Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Architecture | R. Müller

Summary

Ral Miller, an expert in software and architecture, discusses the utilization of AI in software design, specifically focusing on the capabilities of GPT models. He highlights the importance of context in AI interactions and shares practical tips for effectively using chatbots like GPT for complex tasks such as preparing for the iSAQB Advanced exam. Miller emphasizes the iterative process of refining prompts and the potential of AI as a collaborative tool in architectural decision-making, while also addressing considerations around data protection and the ethical use of AI.

Takeaways

• 🚀 Ral Miller is a renowned software and architecture expert, author, and accredited trainer known for initiating the open-source project doc toolchain.
• 📈 Ral discusses the effective use of AI, specifically chatbots like CPT, in software design and creating solution architectures.
• 💡 The importance of data protection and copyright is highlighted when using AI, emphasizing the need to handle personal and company information with care.
• 📚 Ral shares his experience and tips on using AI for preparing for the iSAQB Advanced exam, noting the differences between GPT-3 and GPT-4 models.
• 🌟 GPT-4 is recognized for its superior performance in a medical exam study, showcasing its ability to provide higher quality responses compared to GPT-3.
• 🧠 The core functionality of AI models like GPT-4 is likened to an all-knowing monkey rather than a statistical parrot, indicating a more complex and knowledgeable system.
• 🔍 Ral introduces the concept of 'embeddings' and their role in enhancing AI models, allowing them to access and utilize additional information beyond the core neural network.
• 🔗 The significance of context in AI interactions is discussed, with strategies provided for maintaining context and ensuring effective communication with AI.
• 📝 Tips for priming AI models are shared, including the use of custom instructions to establish long-term memory and refine the AI's responses.
• 🛠️ Ral demonstrates the practical application of AI in solving complex tasks, such as the iSAQB Advanced exam, by iteratively working with the AI to generate and refine solutions.
• 📈 The potential of AI as an architectural co-pilot is emphasized, with Ral noting that while AI can provide a good starting point, human guidance and review are still necessary.

Q & A

What is Ral Miller known for in the field of software and architecture?

Ral Miller is a well-known software and architecture expert, author, and accredited trainer. He started the open-source project doc toolchain, which focuses on the effective documentation of software architecture. He is also responsible for creating various formats of the arc 42 template.

What is the main topic of Ral Miller's session?

The main topic of Ral Miller's session is 'Using AI in software design', specifically discussing how chat can assist in creating a solution architecture.

What are the potential dangers of using CCK GPT mentioned in the transcript?

The potential dangers of using CCK GPT include issues with data protection, both for personal data and company information, and copyright concerns when working with non-public files.

How does Ral Miller describe the core functionality of large language models like GPT?

Ral Miller describes the core functionality of large language models like GPT as being based on a neural network that performs auto-completion, similar to how a mobile phone predicts text based on probabilities.

What is the difference in performance between GPT-3 and GPT-4 as discussed in the transcript?

The transcript mentions a study where GPT-4 scored an 82% accuracy rate on a medical exam with roughly 1,000 questions, compared to GPT-3.5 which scored 65% on average and GPT-3 with 75%. This indicates that GPT-4 has a higher quality and more capabilities than GPT-3.

How does Ral Miller suggest maintaining context when interacting with chatbots like GPT?

Ral Miller suggests starting every session with a prompt that includes a special character like a greater sign (>) to help identify whether the system is still within the context. As long as the context is active, the first paragraph of each output will include the dash. Once the context is lost, the system will no longer display the quote character.

What is the role of embeddings in enhancing the capabilities of a neural network like GPT?

Embeddings are used to extend the neural network with additional data. They are created from text fragments and stored in a vector database. When interacting with the system, it may query this database for text fragments that fit the prompt, pulling them into the context and enhancing the system's understanding and responses.

How does Ral Miller prepare the chatbot for a specific task?

Ral Miller prepares the chatbot by providing it with a detailed context, or 'priming', which includes information about his background, the goal of the session, and any specific requirements or constraints. He also uses custom instructions to guide the chatbot's responses and to create a 'long-term memory' for the chatbot.

What is the significance of the context size for GPT models?

The context size determines how much of the chat history the model takes into account when generating responses. GPT-3 has a context size of 2,000 tokens, while GPT-4 has increased this to 128,000 tokens. A larger context size allows the model to consider more information, potentially leading to more accurate and relevant responses.

How does Ral Miller use chat GPT to assist with an advanced certification exam?

Ral Miller uses chat GPT to generate solutions for the tasks in the exam by providing detailed prompts and context. He also uses it to understand the task better by asking the chatbot questions and refining the prompts based on the responses. The chatbot helps in creating documents, diagrams, and strategies as part of the exam.

What are Ral Miller's views on the future capabilities of chatbots like GPT?

Ral Miller believes that chatbots like GPT can serve as an architectural body co-pilot, helping to generate ideas and make decisions. He expects that advancements in the model will lead to an even larger context size, which will help to overcome current limitations. He also sees potential in using GPT for passing oral exams by integrating it with virtual avatars and video technology.

Outlines

00:00

🤖 Introduction to AI in Software Design

Ral Miller, a renowned software and architecture expert, discusses the use of AI in software design. He highlights the importance of considering data protection and copyright when using AI tools like GPT. Ral introduces the concept of chatbots in creating solution architectures and shares his experience with GPT, emphasizing that while GPT is often seen as a stochastic parrot, he views it as an all-knowing monkey due to its extensive data training and ability to provide detailed answers.

05:02

🧠 Understanding the AI Model's Architecture

The speaker delves into the architecture of AI models, comparing GPT-3 and GPT-4, and noting the differences in their training data and parameters. He explains that GPT-4 has a more complex structure and can process multimodal inputs, including images and voice. The discussion includes the concept of 'embeddings' and how they extend the AI's knowledge base beyond its training data, suggesting that the AI model is more than just a neural network.

10:02

📝 The Role of Context in AI Interaction

The importance of context when interacting with AI is emphasized, with the speaker explaining how the AI model uses context to generate accurate responses. He provides tips on maintaining context during a chat session and how the model's context window can affect the quality of its answers. The speaker also discusses the concept of priming the AI with information to ensure it understands the discussion's goals and background.

15:06

🎯 Preparing for the iSAQB Advanced Certification

The speaker shares his approach to preparing for the iSAQB Advanced Certification using AI. He outlines the process of priming the AI with personal and professional context, discussing his preferred programming languages and architectural styles. The speaker demonstrates how to use the AI to understand and work with exam materials, including rules and glossaries, to build a solid foundation for tackling the certification exam.

20:06

📚 AI's Ability to Interpret and Generate Content

The speaker explores AI's capability to interpret and generate content, such as diagrams and tables, from provided documents. He discusses the limitations of AI in reading images within PDFs and shares his experience in guiding the AI to produce the desired outputs. The speaker also highlights the AI's ability to ask relevant questions about a task, showing its active engagement in understanding and solving problems.

25:10

🛠️ Iterative Approach to AI-assisted Task Solving

The speaker describes an iterative approach to solving tasks with AI assistance, where the AI generates parts of the response to refine the solution. He discusses the process of working with limited output space and the need to edit prompts to keep the context concise. The speaker provides examples of how he refined his prompts to achieve better results, such as creating a utility tree and assessing quality scenarios.

30:11

🏗️ Building a Solution Architecture with AI

The speaker details the process of building a solution architecture with AI, discussing the creation of a context diagram, business models, and a technology stack. He shares his experiences in refining prompts to improve the AI's outputs and emphasizes the importance of checking and iterating on the AI's results. The speaker also discusses the potential of AI to serve as an architectural co-pilot, providing valuable insights and options for problem-solving.

35:11

🚀 The Future of AI in Architectural Decision-Making

In the concluding remarks, the speaker reflects on the potential of AI in architectural decision-making. He discusses the advancements in AI models and their ability to transform input into solution outputs. The speaker acknowledges that while AI has limitations and requires guidance, it serves as a valuable tool for generating ideas, making decisions, and identifying errors and inconsistencies in architectural designs.

40:13

💬 Q&A Session with Ral Miller

The Q&A session with Ral Miller covers topics such as the time and effort required to solve problems with AI, the creation of embeddings, and the AI's ability to surprise with new ideas. Ral discusses the ethical considerations of using AI in exams and his experiences with using AI for open-source software inquiries. He also addresses concerns about the use of IP in training sets and the importance of using AI responsibly.

Keywords

💡AI in software design

AI, or Artificial Intelligence, plays a pivotal role in the field of software design by assisting in the creation of solution architectures. In the context of the video, AI is used to enhance the design process through chatbots like GPT, which can generate ideas, make decisions, and provide architectural recommendations. The speaker discusses how AI tools like GPT can be utilized to improve efficiency and innovation in software design, as demonstrated by their application in tackling a complex exam task.

💡Chatbot

A chatbot is an AI-powered conversational agent designed to interact with humans through text or voice interfaces. In the video, the chatbot GPT is used as a tool to assist in the software design process. It is capable of understanding context, generating responses, and even asking questions to clarify or expand on the given task. The chatbot's ability to process natural language and provide meaningful outputs makes it a valuable asset in the design and decision-making process.

💡Solution architecture

Solution architecture refers to the high-level design of a system that outlines the structure, components, and their relationships. It is a blueprint for the software's construction and serves as a guide for developers. In the video, the speaker discusses how AI, specifically through the use of chatbots, can aid in creating effective solution architectures by generating ideas, analyzing requirements, and proposing designs that meet specific objectives.

💡GPT-3 and GPT-4

GPT-3 and GPT-4 are versions of Generative Pre-trained Transformer models developed by OpenAI. These models are advanced language models capable of understanding and generating human-like text based on the input they receive. GPT-4, being the newer version, has been trained on more data and has more parameters, resulting in improved performance and understanding compared to GPT-3. In the video, the speaker contrasts the capabilities of these models and discusses how GPT-4's enhanced abilities make it more effective for tasks like software design and exam problem-solving.

💡Context

In the context of AI and chatbots, context refers to the background information or previous interactions that the system uses to generate relevant and coherent responses. It is crucial for understanding the continuity of a conversation and providing accurate information. The video emphasizes the importance of context in chatbot interactions, as it allows the AI to maintain a consistent and informed dialogue, which is essential for complex tasks like solving an exam problem.

💡Embeddings

Embeddings are a critical concept in natural language processing and AI, where words or phrases are represented as numerical vectors in a high-dimensional space. These vectors capture the semantic meaning of the text, allowing the AI to understand relationships between words and generate contextually relevant responses. In the video, the speaker mentions embeddings as a way to extend the AI's knowledge beyond its training data by incorporating additional text fragments into the chatbot's responses.

💡Prompt engineering

Prompt engineering is the process of crafting effective prompts or questions for AI models to elicit desired responses. It involves carefully designing the input to guide the AI towards specific outcomes, making it a crucial skill when using chatbots for complex tasks. In the video, the speaker emphasizes the importance of prompt engineering in working with GPT, as it allows for better control over the AI's output and helps in achieving the objectives of the task at hand.

💡Data protection

Data protection refers to the measures and policies implemented to ensure the privacy and security of personal and sensitive data. It is a critical concern when using AI tools, as these systems can handle large amounts of information, some of which may be confidential. In the video, the speaker issues a disclaimer about the importance of data protection, cautioning against the use of non-public files and personal data with AI tools like GPT to prevent potential breaches.

💡Advanced software architecture certification

An advanced software architecture certification is a professional qualification that validates an individual's expertise and knowledge in the field of software architecture. The video focuses on the speaker's attempt to use AI to assist in achieving such a certification, specifically through the iSAQB (International Software Architecture Qualification Board) exam. The process involves solving complex architectural tasks and demonstrating a deep understanding of software design principles.

💡Multimodal capabilities

Multimodal capabilities refer to the ability of an AI system to process and understand multiple types of inputs, such as text, images, and voice. This enhances the system's versatility and interaction capabilities. In the video, the speaker mentions that GPT-4 has multimodal capabilities, allowing it to not only work with text but also process images and generate speech, which indicates a more comprehensive and interactive AI experience.

Highlights

Ral Miller is a well-known expert in software and architecture, and the creator of the open-source project doc toolchain.

Ral Miller emphasizes the importance of considering data protection and copyright when using AI in software design.

GPT-4 has been trained on terabytes of text data, resulting in more parameters for providing better answers compared to GPT-3.

GPT-4 scores higher on tests, with an 82% accuracy compared to GPT-3's 65% and human average of 75%.

Ral Miller refers to GPT-4 as an 'all-knowing monkey' rather than a 'statistical parrot' due to its extensive knowledge base and improved capabilities.

GPT-4 has a context size of 128,000 tokens, a significant increase from GPT-3's 32,000 tokens, allowing for a larger window of context.

Ral Miller shares his experience using GPT-4 to prepare for the iSAQB Advanced exam, demonstrating its potential as a study tool.

The speaker discusses the use of embeddings to extend the neural network with additional data, which is stored in a vector database.

Ral Miller provides tips on maintaining context during chat sessions with GPT-4 by starting each session with a specific prompt.

The importance of priming the system with relevant context, such as profession and preferred technologies, is emphasized for effective communication with GPT-4.

Ral Miller demonstrates the ability of GPT-4 to understand and work with complex diagrams and images from PDFs.

The speaker highlights the iterative process of refining prompts and checking the model's understanding to achieve high-quality outputs.

Ral Miller shares his success in using GPT-4 to generate a solution strategy for a software architecture task, showcasing its problem-solving capabilities.

The speaker discusses the potential of GPT-4 as an architectural body co-pilot, aiding in decision-making and providing different perspectives on architecture.

Ral Miller addresses the issue of bias in AI systems and shares his experience with how the system's generated image reflected a bias.

The speaker emphasizes the importance of checking and refining the results produced by GPT-4, as it serves as a starting point rather than a final solution.

Ral Miller concludes that GPT-4, with its advancements, is a valuable tool for transforming input into solution outputs, though it requires guidance and analysis.

Transcripts

Ral Miller is a very well-known software

architecture expert author and iqb

accredited trainer Ral started the

open-source project doc toolchain which

deals with the effective

documentation of software

architecture uh he's also responsible

for creating the various formats of the

arc 42

template yeah and enjoy now his session

on using AI in software design

thank you MCO for the uh introduction so

I can skip my first slide hello and

welcome to my talk using Ai and software

design how chat can help with creating a

solution architecture let's skip the

slide because you know already know

everything about me and um time is

precious today because I have lots of

slides but first up a

disclaimer be aware of the dangers when

you use cck GPT keep data protection in

mind both in regards of personal data

and company

information copyright might be an issue

when you work with um with CPT if we

drop files which are not public

available and we will I will show you

how to solve the um isq aqb Advanced

exam with chpt but don't do this with a

real exam you not allowed to do

so

now another

disclaimer I am not an AI expert I am

only a prompt engineer I'm a user of

chat GPT and um I think I I gathered

quite some experience by now so I will

show you today my tips and tricks

most of the time when um people talk

about CPT and other large language

medals models they say it's a stochastic

parrot because there's a neuron Network

at the core of the system and it just

yeah does some kind of autoc completion

like when you type in text on your

mobile phone based on

probabilities that's the core of the

large language

model but I believe it is more the all

knowing monkey and why I think so I will

now first try to tell you and introduce

some Basics about

chpt so if you compare the freely aaable

CET

gpt3 with the not free aaable CG 4 you

will notice that there's a huge

difference um cpt4 has been trained on

lots of data terabytes of Text data by

the way most of the information I

present here I Googled in the inter on

the internet but um there's not much

reliable data out there but it gives you

some yeah rough estimates

so much more data in the training and

many more

parameters which are needed to give good

answers so gpt3 is available for free so

many people who talk about CET GPT and

who have checked it out are talking

about the free

model because you have to pay 20 bucks

per month to use the GPT 4 model at

least at least if you use a um chat GPT

4 by open

Ai and if

you search deeper for uh comparisons

here's for instance a study about a

medical exam with roughly 1,000

questions and here you see the

difference between those two models GPT

3.5 scores uh 65% of the test humans on

average

75% GPT 4

82% so it has really a different

quality the GPT 4 model than the gpt3

model and it is more than just the

neural

network but it's hard to find details

about it um it's based on the

Transformer model architecture and you

find this picture on the internet and

yes we are Architects and we should be

able to read this uh this diagram but um

for me it's not so yeah doesn't give me

so much meaning so much background as I

expected so for this talk I came up with

my own diagram and it's um yeah

partly guesses it's quite abstract but

you will notice what I want to tell you

at the core there's a neuron

Network and you input text to the model

and before this text hits the neural

network we already have a natural

language processor so it's not only the

neural network at the core but there's

something in front of it

and then when we output data there's an

output

processor and then we get our text

output and um there's also the system

also feeds context to the NLP

processor we will talk in a minute about

the

context and it seems to be not only one

neural network in at core but a mixture

of

experts so in the past when um you asked

a question how to mathematical question

what's 1 +

one it went through the normal neuronal

Network and it had some problems with

bigger numbers or more complex

calculations now at the core seems to be

this mixture of experts so something

decides uh to which expert model the

question should be

rooted and so it gained much more

knowledge and can um lead to better

answers but it also now is a

multimodal um yeah tool because it not

only works on text but you can now also

drop images which are then processed and

fed to the NLP

processor and even um you can ask it to

Output images through Deli

3 and voice you can speak to

it and it can generate speech so you can

um not only type in your questions but

you can just um work with it with your

voice there's a Code

interpretor uh which is capable of um

yeah doing using Python scripts to work

on your

prompt and there are many more plugins

aailable which makes the model quite

powerful so you see it's not only just a

neon Network at a core it's many more of

it and there was one article where they

stated that jet gptt 4 has around

110 layers whatever this means I think

it's not the layer of the neural network

it's the

layers before and after

it but as I said I'm not the AI expert

I'm the prompt

engineer so let's talk about context

when you chat with the

system you type in your text and first

should be some kind of priming you

should give the system some context who

you are what you know and what you're

are going to do in this example hey

let's work on a web- based application

with spring Boot and

MySQL and then you chat with a system

and

everything which is here in the chat in

the history will be part of the context

of the system will be fed to the

neuronal network this is quite important

because if you continue your

chat there's a window of

context and

now some messages has left this context

will not be fed anymore to the

model and this means that your first

statements the priming leaves the

context and the system doesn't know

anymore what we are talking about and it

will start to give wrong

answers so how important is this

fact chck gpt3 has a context of 2,000

tokens GPT 4 or has a context yeah in

the past of uh 32,000

tokens since roughly four weeks um they

increased managed to increase the

context to

128,000 tokens a token is not a word

it's a part of a word so those um 32,000

tokens are roughly 20,000 worth

so it takes some time to get out of this

context but it depends on the model you

work with and for each model there are

different models with different context

sizes so the context size of

128 uh K is only aailable through the

API at the

moment so here's my first tip to see

whether you are still in cont text or

not just start every session with this

prompt which tells the system to start

every response with a greater

sign because it

outputs um it's text with markdown

formatted the greater sign will be

outputed as a

quote which will look like this so the

first paragraph of each output

will be this

Dash as long as we are in

context the first time we leave the

context it will forget about this prompt

and will also not display this uh quote

anymore but I wanted to tell you

that the system is not just a

probability machine not Sy stastical

parot and for this embeddings are quite

important um you want to extend the

network the neural network with your own

data but that's not possible because

it's already fully trained and you can't

easily train it with additional data the

solution for this is to use

text

and create vectors from these um text

fragments um there are some algorithms

which um create those vectors um which

just check out the words and see how

yeah how near they are to each

other quite complex I don't want to get

get into detail here but you have to

know about those embeddings that they

are different than what the model knows

um within the neuron Network and it's

also not yeah how it how it belongs to

the context because those text fragments

those vectors are stored in a vector

database and when you talk to the system

it always for each prompt might query

the vector database check out out

whether there is a text fragment which

fits your prompt and pull it into your

context so if it does this your context

will be smaller than what you chat with

a system so that's why it's important to

know about it and even if you have a big

context it's always a question how much

context you you really

have again GPT 4 32,000

tokens and we will see when we are in

the chat model the the chat

GPT we will have less we will have

something about 4,000 or 8,000

tokens so going back to the all knowing

monkey or the stastical parrot I I

believe with this approach those

multiple layers multimodal approach and

those

embeddings um the system behaves like

Yeah The all- Knowing monkey which has

access to all the knowledge of the world

at least the knowledge with which is

aailable to

it but let's get started with the real

details with the

yeah with trying to get a solution for

the advanced certification of the is

aqb so as I already said we have to

prepare the model we have to do some

priming every session every chat session

starts from scratch the model doesn't

know what you chatted with it before in

another session so you have to tell it

the context of your discussion it

doesn't know you you have to tell it

what it should know about you you have

to tell it what's the goal of the

session and maybe also what the solution

should look

like so for instance my context is that

I'm an experienced software architect in

the field of web development my favorite

programming language is Java or groovy

JavaScript and Cor responding Frameworks

are not my thing I prepare to get by

with minimal JavaScript okay that's me

interesting I um want to create an image

for this and here you see some kind of

bias and it's so funny that not even I

am depicted as as an old man it's also

that there really old um technology

calculator on the table so I gave it a

try and um see what kind of bias it uses

when I tell it U my favorite programming

language is Python and JavaScript that's

cool and I love modern JavaScript

Frameworks hey now I'm young and cool

and

tip so bias is a problem even where you

don't expect

it but let's move on with the

priming

um I shouldn't not only tell the system

yeah this this small paragraph I also

extended it to tell it more about um how

I work which Frameworks I I like to use

by the way there's lots of text on those

slides you don't have to read it um you

will get the slides

later so this is really enough context

uh for the model to understand how I

think how I work um what's my profession

is

um it isn't long ago that you had to

copy and paste such a priming every time

into your chat session which was not fun

at all but um now catp uh has custom

instructions and you have two text

Fields where you can put in um what

would you like chpt to know about you to

provide better responses so that is the

text from the last SL side and I call

this long-term memory because I can put

in there something which CHP should

memorize and the second part of the

custom instruction is how would you like

chpt to respond and there I also have

lots of prompts uh for instance start

every response with a greater sign um we

already had this but also many things

for instance um

be excellent at reasoning don't mention

your knowledge cut off um I already know

this and um that really

helps and um I also instructed that I

want to create ASD do output for my

documents and even how I want to have my

ask do output

formatted okay even more priming

I want to have the best basis for my

chat session with cck

GPT so I tell it today I want to talk

with you about a software architecture

certification do you know the is

aqb every time I start a session and um

talk with chat chpt I ask the model

whether it knows about the

Technologies um or other abstract things

I want to talk about it um so it first

answers yes it knows it then it

describes the is aqb and this tells me

that it yeah knows what it's talking

about and it also creates some

context for instance here I ask about

the advanced level

certification and it repeat the key

aspects of the certification adding this

to the context and helping both chpt and

me um for this

session by the way I think the answer

was much longer but um that isn't too

important for this

session now we are still at the priming

mode um so there are some um public

aaable PDFs for the exam and I um just

dropped in here the official

rules the model should know about the

rules I also dropped the glossery

because I'm not a native speaker of the

English language and um when we talk

chat GPT and me about um the the exam we

should use the same wording

and so I think it's important that it

knows about this

document and as you can see um those

documents are now put into the system as

embeddings and that should be enough for

the priming at a moment now we can start

with the real task we have a third PDF

document the example task big spender

for a um Advanced

certification I asked chpt to read the

document carefully but not start to

solve the task yet why every time you

drop something into chat GPT it tries to

come up with a good answer and here it

would try to yeah already um solve the

certification task but this would

clutter up the the context

so even with this it just tells me

something about uh the document that it

understood it and

um ask me if I have any

questions feel free to ask

great the PDF contains two images and I

already experienced that somehow images

in PDFs are hard to read for C

GPT but if I just drop them into the

chat and tell him hey here are my images

from the

PDF it really manages to read and

understand those

diagrams it can read the text in the

boxes it uh will notice the connections

and uh output what it sees there which

is quite

helpful 6 months ago when I submitted

the idea for the talk um jgpt wasn't

capable of doing all of this and I

wasn't sure whether I can create a

solution uh with chpt for it um it

couldn't read those images but what was

quite interesting there was a

description underneath um this class

diagram in in form of a table and I did

it the other way around I put in the

description and asked CHP to create a

diagram and the diagram didn't look like

this one and I was looking for the

reason and found out that the

description didn't fit the diagram in

all parts which is also quite

interesting one last

instruction I learned that CH gbt is

capable of asking me questions

so before you solve a task check if you

have any questions regarding the task

and ask me those

questions I pressed return and oops Yeah

it already had some questions it came up

with a list of 10 questions about the

task and I think quite good questions

like regulatory

requirements it does doesn't know that

I'm from Germany and U maybe uh one

regulatory requirement is a um data

Protection Law and other German laws so

this was quite

interesting and I answered all those

questions and you can answer them one by

one and every time you submit one it

will tell you okay now I understand but

please answer also the the other

questions so you can turn side

but the First subtask Quality

attributes I always let CH gbt repeat

the objective for the

subtask and um it repeats it adds to the

context and I see that it really knows

about the subtask

great so this worked this is the subtask

the objective now we can go on derve the

quality scenarios trade me an asky do

document okay and wow I got a table

quite nice I scanned it and looks

reasonable the importance we could work

on this it shouldn't be everything high

but quite a good

sample now we also want to have a

utility tree I asked CH to create one in

plant oml and as you can see here this

says three of

three so when I work with those prompts

I refine them and it doesn't make sense

because of the limited context to refine

The Prompt in the next prompt it makes

sense to edit your prompt

to keep the context short and here you

can see um I had three tries and uh

first I I just had the idea to Output it

as a plant uml mind map but then I

noticed it didn't came up with the um

IDs of the quality scenarios so I

refined this

and wow it tells me it created a file

which can be downloaded I wasn't aware

well that it can do so but it worked it

was not a hallucination so I downloaded

this file displayed it in plant uml and

here you go we have a utility

tree we have the quality attributes

everything's

fine now you can

see um yeah that's that's a third step

for this

subtask um again a table and um it's a

special table because I again refined my

prompt because first it had some

problems um with the motivation and I

then asked uh the system to to put the

motivation below those lines with a call

span of three and it knows how to work

with asid do tables and uh came up with

a good result

the result is a little bit questionable

I would have uh post other weights on it

so system reliable and availability only

10% yeah okay I think it should be more

but these are all things you can discuss

with a model and um so change its

mind so as you have seen I I let the

model create parts of the uh

response um just to to work on the

details because it has a limited output

space and uh it will not output the the

whole um solution for this one task if

you do not um work iteration by

iteration on it and so I then ask it to

uh give a solution for the whole subtask

and here we are it repeats the other

things it already created and looks good

to

me solution strategy that was an easy

one um

so but I had a problem with repeating

the objective um you can see I refined

The Prompt as stated in the document was

important because if I do not say as

stated in the document it just uh tries

to get it from the cont context if I say

as stated in the document it starts to

read the document again and outputs what

it finds in a document and this seems to

be quite okay it's it's a word nearly

word by word um repetion of the the

objective from the document so with this

I just um told it hey give me a solution

strategy

and um ah yeah and again I I told it if

you have any questions about this task

ask them

now and again I was surprised it came up

with some

questions and um it doesn't work as

expected that when I at the start tell

it whenever you have question ask a

questions you always have to repeat this

um the thing but it came up with those

questions I answered those questions

told it to um give me a solution

strategy and it came up as expected uh

with a key constraints assumptions and

basic

principles talking about the chat

context I can ask it how much of the

context is left and it tells me 2,478

tokens of 4,000 aable tokens so we are

still in context

great I repeat it and it tells me 2,300

tokens okay that's an

estimate of 8,000 tokens so it doesn't

know whether it has 4,000 or 8,000

tokens but we are still in context I

believe because what happened here we

don't have the quote anymore here in

front of it but that is just uh because

it sometimes forgets to Output it here

it is

again again uh objective for the third

statement for third subtask this time it

forgot something overview of purchased

and open source

Parts okay so in my

prompts

I yeah in the next prompt I I tell it to

that it forgot it but first I wanted to

to have the context diagram I thought

this is an easy task I told it to create

it as plant TL in the C4 notation but

what I got was too simple far too simple

it didn't remember what it needed for

this context diagram so this is

worthless

give the model time to think is an a

prompting approach where you yeah let

the model first generate the information

it needs so first name all actors who

will interact with the system name all

external systems and create a table with

uh the external

interfaces and now we get an answer and

with this answer we just repeat our time

ask our prompt from

um just two prompts ago and the result

is quite better and now I'm I'm happy

with um the actors and the external

systems let's go on with business

models we need a business

structure and um chat GPT can generate

it again we have a plant ml diagram uh

as

download and again I'm not happy with

this

diagram and again I do did some prompt

engineering and toed the system to

display all external components from the

subtask three in the diagram actors and

external systems and that changed the

display the the generated

diagram

the fifth subtask wants to have a

technology

stack again let's repeat the

objective and hey that is not the whole

OB objective because if you take a look

at um at the document subtask five is a

long

subtask and the problem seems to be that

it goes over a page break

this is a problem for

chip and in this case we I fix it by

putting all the text in the

context when working with uh those

embeddings it just fetches for each

answer those embeddings it needs now I

put it in the context and grow my

context and with this full

context um

I can

get a technology stack I'm happy with

this and uh it was also requested to um

explain the technology stack with the

workflow through the system and yeah it

tells me the system parts involved for

each step and the contribution to the

Quality goals looks fine for

me let's check the context again

tells me we used up 4,600 tokens out of

8,000 tokens

okay let's go

forward the final subtask

evaluation and in this case I extended

my prompt to repeat the full

objective so you have to be quite

detailed and concise with your prompt

and then I got um the the full objective

fine and this time I asked whether it

understands the objective and whether it

has questions regarding it and I was

surprised it says yes I understand the

objective and I don't have any specific

questions okay so go ahead identify the

top five riskiest and most important

quality scenarios here they are

fine and then I asked to create the

whole

document and as you can see data in inte

Integrity it has a rational essential

for legal compliance great architectural

decision implementing strict Access

Control good tradeoffs this might may

slightly increase system complexity but

it's

necessary I think those results are

quite good and now we finished the

exam and the question is how much

context is now left and now it tells me

we used up 5,000 tokens out of 4,000

tokens and the earliest parts of the

conversation are no longer in the

immediate

context we just used up all the the

context so I guess when you do it in a

real

scenario you will start the context from

fresh for each

subtask and

um I I guess it's also just a matter of

time that we get a chat GPT with a

really huge context and then these

problems will be

gone

conclusion CH GPT is a generative

pre-tin Transformer that's what GPT is

about and it is capable of transforming

the advanced exam input to a solution

output I think that's quite great as I

said six months ago I think that

wouldn't be such a good solution but now

with the advancements in the model I

think the solution is quite okay um it

has some problems but it is a good

starting point it's it's a architectural

body co-pilot for me and I now can work

from this Solution on and make it

better it needs Guidance the results

have to be checked

yes it's quite helpful can walk alone

that's what I mean it's a buddy um to

help me to get good ideas to um make

good decisions and to um for for

instance um get different views on uh on

the

architecture interly interesting if you

ask CHT to analyze the

results and ask it to find errors and

inconsistencies it will find the

shortcomings by its own so this is also

quite helpful to just ask the

system whether the things it produced

are good or

not now the advanced exam um consists of

two parts the homework which we just did

with

chup and an oral exam

where you have to show up and um answer

some questions and will jgpt be able to

pass the oral exam the technology for

this is already there you can create

virtual avatars you can speak to the

system and you can even uh fake video so

the technology is there but I think that

the it will be yeah you will notice that

the the solution was not created by a h

a human

architect and so you still have to

analyze the

solution understand it and be prepared

for the oral

exam to say it with the words of William

Gibson the future is already here it's

just not very evenly distributed so when

you think of gpt3 the free model and GPT

4 where you have to pay 20 bucks that's

not evenly distributed not everybody has

access to the better

models but it's already a very

interesting um

technology so thank you for your

attention the slides are aailable behind

this QR code there's also a discussion

board there and um the full transcript

and the solution as

PDF thank you for your attention I think

we still have three minutes left for

questions thank you Ral this was an

amazing talk and about a very important

topic um I have two questions for you um

the first would

be how long did it take you to solve the

problem how many attempts were

needed H it was quite fast it um I would

say two days to work with the system and

to um rework some of the

prompts but I also have to say that I I

only did a quick view over the results

they

look from a first view quite good but um

you will find uh problem s with

it okay thank you um the another

question um are there other means of

creating embeddings other than priming

interactivity uh by dropping documents

in the CET

GPT so when you just work with CET GPT

um I think this is a um main way to do

embeddings you can also by now um State

some URL which it then goes and fetches

them and um if you create your own

systems through the API uh you can use

the API to create embeddings to um for

instance use a larger um set of PDFs

create your own Vector database and thus

create those embeddings you then have

have the full control of it and um

because you always um you yeah you

create the chat you put in the the

context to the API and you decide how

much of the context you will fill with

the

embeddings um thank you uh there here's

another question um did Jet Chet GPT

really help you or did you try to make

Chet GPT understand insights you already

had

beforehand did it Sur surpris you with

good ideas you did not have

before yes it it's always surprises me

with good ideas and uh always helps me

um for instance it was closer to the

objectives of the task and I was and um

that was something where I thought yes

it's it's right these these were the

expected results I would have written

much more not just to to um tell how I

came to those uh

results but that's one thing I was

surprised of another thing um that it

asked me questions about the task and

the if you take a look at those

questions they make

sense and this also helps me to get a

better understanding of the task and the

next thing is when you work on

architectural decision records with cat

GPT um you can ask it for several

options how to solve the problem and it

might find options you didn't think of

so yes it helps it helps to extend your

your

mind okay and uh now a last question

because uh the time is running um for

the Break um what is your view on the IP

used in the training sets and in whether

your input or questions were then added

as training for the

model so most of the time when I work

with CET GPT um I uh asked very general

questions about um open source software

so that that's something I do not

consider as a problem when you work um

with a exam I could only work with a um

public aaable um example you're not

allowed to do this with a real exam and

also um I wouldn't put in um some

internal documents or something like

this or personal data um but the

interesting point is that um all those

companies who provide those large

language models um if you buy the

Enterprise Edition they promise you that

they don't use the data you put into the

chat for trainings or something like

this so I think there will be

yeah a switch in how you think about

this problem and how it will be

solved okay thank you uh there is one

thing uh um someone is asking for the QR

code uh maybe you can bring it up again

the QR code but um

I think it was in in your um

presentation you you can download the

presentations to the attendees you can

download all presentations uh uh later

uh we will provide it for you yeah so uh

I think or maybe I think I will also put

the link in in the chat okay this is

great okay thank you