Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Arch. by Ralf D. Müller

Summary

Ralph D Miller discusses the capabilities and intricacies of AI and chatbots, particularly focusing on GPT models. He emphasizes the importance of understanding the underlying technology and provides tips for effective usage, such as paying for the full version for enhanced capabilities. Miller also highlights the potential of AI in solving complex problems and stresses the need for careful priming and prompt engineering to guide the AI towards accurate and useful outputs.

Takeaways

• 🚀 GPT models, such as GPT-3 and GPT-4, are powerful generative pre-trained Transformers capable of various tasks including text translation and code conversion.
• 📈 GPT-4 has a significantly larger dataset and parameter count than GPT-3, leading to improved performance and more nuanced understanding.
• 💡 When using GPT models, it's crucial to be aware of data protection and copyright issues to avoid legal complications.
• 🤖 GPT models are often referred to as 'stochastic parrots' due to their ability to predict the next word based on probabilities, but they can also be seen as 'all-knowing monkeys' due to their vast knowledge base.
• 🧠 The architecture of GPT models includes a neural network core, natural language processing, and multimodal capabilities allowing for text, image, and voice inputs and outputs.
• 🔄 GPT models build context through interactions, which can be crucial for providing accurate and relevant responses; however, context length can vary and may impact performance.
• 📚 Embeddings are a technique used to incorporate external data into the model's context, allowing it to better understand and interact with specific information or domains.
• 💬 Prompt engineering is an essential skill when working with GPT models, as it involves priming the model with the right context, goals, and expectations for each session.
• 🔄 Feedback loops are important for refining GPT model outputs, especially in coding scenarios where the model can iterate and improve based on expected outcomes.
• 💰 Investing in the paid version of GPT models is recommended for serious users, as it offers more capabilities and better results compared to the free version.
• 🎓 For complex tasks like software architecture exams, GPT models can be a valuable tool, but they require careful guidance, context management, and verification of their outputs.

Q & A

What is Ralph D Miller's profession and main interests?

Ralph D Miller works at DBST, the IT partner of the German Railway. His main interests include documentation, docs-as-code, security, architecture, and AI. He is also a maintainer of the Doc toolchain and an open source contributor to the docs.code project.

What is the significance of the difference in training data between GPT-3 and GPT-4?

GPT-3 was trained on 550 gigabytes of text data, while GPT-4 was trained on a much larger dataset, estimated to be around 4 to 5 terabytes. This larger dataset allows GPT-4 to have a more extensive knowledge base and potentially provide more accurate and comprehensive responses.

How does the context size limit in chat GPT models affect the conversation flow?

The context size limit, which was 2,000 tokens for GPT-3 and 32,000 tokens for GPT-4 at the time of the talk, affects the conversation by potentially causing the model to lose track of earlier parts of the discussion. As the conversation approaches the token limit, earlier messages may get 'out of context,' leading to less coherent responses from the model.

What is the role of embeddings in enhancing the capabilities of large language models like GPT?

Embeddings allow the model to understand and process information more effectively by representing words or phrases as vectors in a multi-dimensional space. This enables the model to relate and compare different pieces of information more accurately, improving its ability to answer questions and solve problems based on the provided context.

How can users provide custom instructions to improve the responses from GPT models?

Users can provide custom instructions through a feature that allows them to specify what they want the model to know about them for better responses. This could include their background, preferred libraries, coding style, and other relevant information that can help tailor the model's output to their needs.

What is the importance of prompt engineering when working with GPT models?

Prompt engineering is crucial as it involves carefully crafting the input to guide the model towards the desired output. By structuring prompts effectively, users can influence the model to follow specific directions, provide more accurate answers, and avoid irrelevant or incorrect information.

How does the GPT model handle tasks that require multi-step problem-solving?

The GPT model can handle multi-step problem-solving by following a series of prompts that guide it through each step of the process. It's important to first ask the model to explain its approach before it attempts to solve the problem. This allows for adjustments and refinements to the strategy before the model proceeds with the actual task.

What is the role of the mixture of experts architecture in the GPT model?

The mixture of experts architecture involves using multiple specialized neural networks within the larger model. Each 'expert' network focuses on different types of information or tasks, which can result in certain questions being answered more effectively than others based on the expertise of the relevant network.

How can users ensure that GPT models maintain context throughout a conversation?

Users can maintain context by being mindful of the token limit and structuring their conversation to ensure that important information is not lost. They can also use techniques like starting each session with a clear prompt, regularly reaffirming the task at hand, and breaking down complex tasks into smaller, manageable steps.

What are some limitations of GPT models in terms of reasoning and problem-solving?

While GPT models are proficient at generating text and providing answers based on their vast knowledge base, they may not always exhibit true reasoning capabilities. Their responses can sometimes be influenced by biases in the training data or fail to accurately understand complex tasks, requiring careful guidance and verification from the user.

What is the recommendation for users who want to fully utilize the capabilities of GPT models?

The recommendation is to subscribe to the paid version of the GPT model, which offers more advanced features and capabilities compared to the free version. The paid model provides a better context size, more accurate outputs, and access to additional functionalities like the code interpreter, making it worth the investment for serious users.

Outlines

00:00

🤖 Introduction and Disclaimer

The speaker, Ralph D Miller, introduces himself and his background in IT, documentation, security, and architecture. He mentions his work at DB Systel, the IT partner of the German Railway, and his involvement in open source documentation projects. The speaker emphasizes the importance of data protection and avoiding the sharing of personal information with AI systems. He also cautions against using AI for real exams, highlighting the potential dangers of copyrighted content generation. The introduction sets the stage for a discussion on the use of AI in creating solution architectures, with a focus on the isqb Advanced exam as a case study.

05:00

📈 GPT Models and Technology Behind

The speaker delves into the differences between GPT-3 and GPT-4, noting the increase in training data from 550 GB to multiple terabytes for GPT-4. He highlights the significant jump in parameters from 175 billion for GPT-3 to 500 billion for GPT-4, and the resulting improvements in performance. The speaker also discusses the importance of using the paid version of GPT for serious work, as the free version has limitations. He touches on the Transformer architecture, the concept of context in AI conversations, and the multimodal capabilities of GPT, which include handling text, images, and voice.

10:03

🔄 Context Management and API Usage

The speaker explains the concept of context in AI interactions, detailing how the AI builds up context during a conversation and how the stateless API requires sending the full context with each request. He discusses the token limit of GPT-3 and GPT-4, and how the AI compresses its own responses to manage this limit. The speaker shares tips on priming the AI with relevant information to ensure accurate and contextually appropriate responses. He also mentions the potential for typos in the slides and the importance of being creative when working with AI systems.

15:05

🧠 Understanding Embeddings and Prompt Engineering

The speaker introduces the concept of embeddings, explaining how the neural network can be influenced by putting data in context. He discusses the cost implications of embedding large amounts of text through the API and suggests breaking down text into chunks and storing them in a vector space as a solution. The speaker emphasizes the power of the API and embeddings in giving the AI access to vast amounts of data. He also talks about prompt engineering, detailing how to prepare the AI for each session by setting the context, defining the goal, and specifying the desired output format.

20:09

🛠️ Working with GPT on Certification Tasks

The speaker describes his experience using GPT to solve an isqb Advanced exam task called 'Big Spender'. He outlines the process of priming the system with information about the task, including uploading relevant PDF documents and using custom instructions to improve the AI's understanding. The speaker discusses the importance of context and how the AI can extract and summarize information from documents. He also shares his experience in guiding the AI through the task, asking it to confirm understanding and clarify requirements before proceeding with solutions.

25:12

📊 Quality Attributes and Solution Strategy

The speaker continues his discussion on solving the 'Big Spender' task, focusing on deriving quality attributes and creating a solution strategy. He explains how he guided the AI to repeat the task objective and derive quality scenarios, resulting in an ASID table. The speaker emphasizes the importance of rephrasing prompts to maintain context and avoid confusion. He also discusses the AI's ability to identify key quality goals and create a solution strategy that aligns with the task requirements.

30:13

🔍 Deepening the Technical Context

The speaker explores the technical context of the 'Big Spender' task, noting the AI's initial difficulty in creating a useful context diagram. He describes a technique of giving the model time to think by specifying the steps it should take, which results in a more accurate technical overview. The speaker also discusses the AI's ability to map the business context to the technology stack, creating a workflow and identifying systems that contribute to quality goals. He highlights the importance of context management and the AI's reliance on accurate and complete information to produce meaningful results.

35:17

🎓 Conclusion and Final Thoughts

In the conclusion, the speaker reflects on his experience with GPT, expressing satisfaction with its capabilities in transforming exam input into solution output. He emphasizes the need for guidance and feedback when working with AI and the potential for the technology to evolve further. The speaker also suggests investing in the paid model of GPT for its enhanced capabilities and concludes with an invitation for questions, highlighting the interactive nature of the talk.

Keywords

💡AI and software

The term 'AI and software' refers to the integration of artificial intelligence (AI) capabilities into software applications. In the context of the video, this is central to the discussion as the speaker, Ralph D Miller, is exploring how AI, specifically chatbots like GPT, can be utilized to enhance and streamline various processes within software development and architecture. The video delves into the practical applications and potential of AI in creating solutions and improving efficiency in the software industry.

💡Chatbot GPT

Chatbot GPT, or Generative Pre-trained Transformer, is an AI language model that is capable of generating human-like text based on the input it receives. In the video, the speaker discusses the use of chatbot GPT in the context of solution architecture, highlighting its ability to process and produce text, code, and even recognize images. The speaker emphasizes the importance of using the paid version of GPT for more accurate and reliable results.

💡Solution Architecture

Solution architecture refers to the high-level design and planning of a system or software that addresses specific business needs and technical requirements. In the video, the speaker is focusing on how AI, particularly chatbot GPT, can assist in creating solution architectures by providing insights, generating code, and offering architectural suggestions. The process involves understanding the problem, defining the context, and iteratively refining the solution until it meets the desired objectives.

💡Data Protection

Data protection involves safeguarding the privacy and confidentiality of data, especially personal data, and ensuring that it is used responsibly and lawfully. In the context of the video, the speaker cautions against sending private or sensitive data to AI systems like chatbot GPT due to potential data protection issues. This is important as misuse of data can lead to privacy breaches and legal complications.

💡Copyright

Copyright refers to the legal rights that creators have over their original works, which includes the exclusive right to reproduce, distribute, and display the work. In the context of the video, the speaker mentions that chatbot GPT might generate copyrighted information, which could be problematic, especially when dealing with images. It is crucial to respect copyright laws when using AI-generated content to avoid infringement.

💡GPT-3 vs GPT-4

GPT-3 and GPT-4 are versions of the Generative Pre-trained Transformer models developed by OpenAI. Both are AI language models capable of understanding and generating human-like text based on the input they receive. GPT-4 is the more advanced version with a larger dataset and more parameters, which results in improved performance and understanding compared to GPT-3. The speaker in the video emphasizes the importance of using the paid version of GPT (which is GPT-4 at the time of the talk) for better results in tasks such as solution architecture.

💡Transformer Architecture

The Transformer architecture is a type of deep learning model architecture that is particularly effective for natural language processing tasks. It is the underlying technology behind models like GPT-3 and GPT-4. The architecture is characterized by its attention mechanisms that allow the model to focus on different parts of the input data and generate contextually relevant output. In the video, the speaker briefly touches on the Transformer architecture to explain the technical foundation of the AI models they are discussing.

💡Context

In the context of AI and natural language processing, 'context' refers to the information that is provided to the AI system to help it understand the situation or topic at hand. This includes previous interactions, user inputs, and any relevant data that the AI can use to generate more accurate and relevant responses. The speaker in the video emphasizes the importance of context in AI interactions, explaining how it can affect the quality and relevance of the AI's output.

💡Embeddings

Embeddings in AI and machine learning are vector representations of words, phrases, or documents that capture their semantic meaning in a numerical form. These embeddings allow AI models to understand and process natural language by treating words and phrases as points in a multi-dimensional space, where the distances and directions between points represent semantic relationships. In the video, the speaker talks about using embeddings to feed custom data into GPT, enabling the model to understand and interact with specific documents or codes.

💡Prompt Engineering

Prompt engineering is the process of crafting input text or prompts for AI models, particularly language models like GPT, to guide the AI to produce desired outputs. This involves carefully structuring the input to include specific instructions, context, and goals that will help the AI generate accurate and relevant responses. The speaker in the video discusses the importance of prompt engineering in effectively utilizing AI for tasks such as solution architecture and provides tips on how to create effective prompts.

💡Custom Instructions

Custom instructions are specific directions or information provided to an AI model to tailor its responses to the user's particular needs or preferences. In the context of the video, the speaker refers to a feature in GPT that allows users to input custom instructions to improve the relevance and accuracy of the AI's output. This feature enables the AI to incorporate user-provided context and goals into its responses, leading to more effective interactions.

Highlights

Ralph D Miller discusses the use of AI and software in creating solution architectures.

Miller highlights the importance of data protection and avoiding the input of personal data into AI systems.

The talk introduces the isqb Advanced exam, an international software architecture qualification board certification process.

GPT models are compared, with GPT-3 trained on 550 GB and GPT-4 on an estimated 500 TB of text data.

GPT-4 outperforms GPT-3 and the average human in academic tests, scoring 82% compared to GPT-3's 65% and the human average of 75%.

The architecture of GPT includes natural language processing, a neural network core, and multimodal capabilities including image and voice recognition.

Miller emphasizes the value of using the paid version of GPT for professional work, citing its superior capabilities.

The concept of 'context' in AI chat models is discussed, explaining how it affects the model's understanding and response.

Embeddings are introduced as a method to incorporate external data into the AI model's context.

Prompt engineering is explored, demonstrating how the input to GPT can be crafted for desired outputs.

The use of custom instructions to provide the AI with specific user context is highlighted.

Miller's experience with GPT's ability to create images and the bias inherent in the model is shared.

The importance of priming the AI with context, goal, and desired solution format is emphasized for effective interaction.

The process of working with GPT to solve complex tasks, such as the isqb exam, is demonstrated step by step.

Miller discusses the challenges and workarounds when dealing with the limitations of the AI's context window.

The potential of GPT models to assist in software development tasks, such as creating algorithms and technical contexts, is showcased.

The talk concludes with a recommendation to subscribe to the paid model of GPT for its enhanced capabilities and value.

Transcripts

welcome to my talk using Ai and software

design how chat GPT can help with

creating a solution architecture I'm

quite happy that the room is full I hope

that everybody will take some takeaways

home some few words about me my name

Ralph D Miller um I work at dbst the it

partner of the German Railway

my interests are mainly documentation

docs as

code a little bit of security or to

architecture and now ai I'm a maintainer

of Doc toolchain and open source uh

documentation dox's code um pro project

and I'm contributor to Arc

42 um you can easily find me on the on

the internet find my email addresses so

if if you want to contact me later this

is not a

problem a short

disclaimer when you use chubbt be aware

of

dangers data protection might be a

problem don't send private data personal

data um to the

system and there's also a problem with

copyright because chat GPT might

generate copyrighted info um that's very

visible when you work with images it's

not so visible when you work with text

and code but I wanted to say this um up

front and um yeah don't do this with a

real exam um I'm talking now about the

is aqb Advanced exam for those who don't

know it is aqb is a international

software architecture qualification

board they have a certification process

for Architects and they have an an

example um task exam um which I will try

to solve with ch GPT but you are not

allowed to use chpt in the real exam so

don't use those approaches for the real

exam another short

disclaimer I'm not the AI expert

Specialists I'm a prompt

engineer I use Ai and chpt I'm not too

familiar with the backgrounds but

nevertheless I want to talk about the

basics how chpt works because you have

to know it um in order to use it and um

many people call those large language

models like chpt

a stochastic parrot not better than the

um autoc completion of your mobile

phone it's a neural

network and um it just produces some um

yeah probabilistic um for the next word

which might follow and this um looks

like something intelligent that's what

people say

say I say it's might be more the all

knowing knowing monkey and why I think

it's the case I will now try to explain

it starts with comparing the gpt3 versus

the GPT 4

model both are generative pre-trained

Transformers that's yeah Transformer it

can transform text so it can um

translate um text or it can translate um

for instance uh um also your code from

python to Java something like this and

it is also capable you I guess raise

your hand if you already have used uh

Chet

CHT yes so

everybody who has used G uh jet GPT 4

the paid model wow

many people I would say

50% so um you all know what GPT is

that's great let's take a look at the

difference so gpt3 was trained on 550

gigabyte of text Data GPT 4 they say

many

terabyte um I say they say it's not

really known you don't don't get too

much information about what is behind

those models what is behind this

technology but some data leaked and some

people estimate about it and yeah so

these are the

results and

um the parameters so the weights of the

neuron Network um for gpt3 it

is uh 75 5 billion parameters and for

GPT 4 it is said to be 500 billion

parameters so there is a huge difference

between those models between the free

version and the paid version and there's

also a huge difference between the free

GPT 4 model which you can use with uh

bing a the yeah Microsoft Bing search

the um chat with Bing and nowadays it's

called renamed

co-pilot it behaves in a different

way so gpt3 is four to get the real

thing the real stuff you have to pay

currently I think it's

$20 but it's worth it so that's my first

tip if you really want to work with um

CHT then pay for it it's really worth it

um that is uh another comparison so that

um there is an an yeah uh a test an an

academic test um and there are lots of

tests uh which uh they have um applied

to those models and

gpt3 do5 scores

65% the average human

75% and gbt 4

82% the reasons why those models reach

those scores are different um they know

a lot they they have seen nearly the

whole internet and that might be the

best reason why they score So High um

people yeah argue about whether there's

really reasoning behind it or not but if

you want to work with such a model these

figures count because it's important

that you get the real stuff and not the

silly answers

from the stochastic

parrot when you search for the

architecture behind

jgpt you get lots of scientific papers

and you often get this Transformer

architecture and yes if you are the AI

expert you will understand this picture

I'm not the AI expert so I made up my

own

architecture and um so what we know is

that at a core is a neural

network and we have text input we have

natural language processing this is

already quite important because your

text will not just fed to the neural

network there's a language processor

before it so that it understands the

difference of words for instance whether

when you talk about Washington whether

it's a a person or um the

city this fed to the neural network then

there's again some kind of output

processor and text

output but that's not

everything because we have a context we

just not only feed your question to the

model but we also have a context when

you chat with a model you build up

context I will tell you a little bit

more about this later and there's a

so-called mixture of experts it seems

that there's not only one neural network

at the core but several expert neural

networks so that might be also reason

why some questions are better answered

than

others and it's

multimodal what does it mean we not only

can feed text to the machine we can also

feed image images to the machine there's

an image

recognition which then results in some

textural

output there's Deli 3 for instance as

output processor to generate uh artistic

images not diagrams just artistic

images and we have voice

recognition and we also have text to

speech

output and we have a lot of more things

like the code interpretor and that's for

instance an an interesting point because

nowadays when you have the code

interpreter enabled because you have the

paid model and you ask the model a

mathem mathematical question like what's

the square root of

15 it will not respond from its neural

network it will create code for the code

interpreter to calculate the

answer lots of plugins

aailable and that's it about the

architecture what I think it's important

it's also important to know that when

you use uh jgpt you have the predefined

font

end and uh yeah you you have to take

what it gives you um soon you will

notice that you will have

more yeah

more Freedom if you work with the API

and build your own front

end oh sorry there is a typo with a

context is you will notice a lot of

typos in in these slides please I'm

sorry for that let's talk about the

context of the model when you start to

chat with a

model you will enter some priming I'm a

software developer working on a

web-based application with spring Boot

and

MySQL so the machine knows something

about you something about the context of

the

problem you then start to chat with the

machine and everything which you what

you exchange with a machine adds to the

context and the API is stateless so

every time you send a new request the

full context is sent to the machine and

the first problem occures when you get

out of the

context in the beginning the context was

quite

small I think currently chat gbt 4 has a

context so there's a chat front end of

8,000 tokens a token is something

between uh one character and a

word

and when you reach the end of the

context your first messages will get out

of context and it will forget about your

priming about the main problem and so it

will drift it will drift with its

answers because it doesn't know about

the the beginning of the of the session

um

interestingly it somehow depends on the

front and because I investigated a

little bit more in the context and

noticed that jpt starts to compress the

context by reducing its own answer size

to something sometimes just hey I've

answered the question of the user that's

it so it keeps your statements but

compresses its own statements to work

around this

problem gpt3 was said to have 2,000

tokens GPT 4 um some months ago had uh

32,000 tokens um these are the token

sizes or um yeah token size of the of

the model um when you use the front end

um the front is end is relatively cheap

say avoid to give you the full cont

context size which is at the moment um

128,000 tokens so if you use a chat I

think you have around 8,000 tokens if

you use the API you can get the full

set so first priming

tip I started to start every

session

with with um The Prompt that CHT should

start every response with a greater sign

why because it produces markdown output

and in markdown it's just the the quote

Mark the line on the left and this way I

can see whether my first statement ran

out of context or

not um it works quite

well for some chat front ends so as I

said if it compresses its own um answers

this will not work but it's somehow an

an indicator and it shows you that you

have to be a little bit creative when

you work with a

system embeddings embeddings is also

something um quite powerful and this is

only a short introduction to embeddings

um

the the neural network is

already yeah trained and you you can't

easily change it and uh the solution to

um to give your model your own data or

to give Chet GPT your own data your own

documents you want to work with for

instance your code is to put your code

in context if you have limited

context then you can't put your your

whole

document your whole codee in context

when you have the huge context through

the API is a one 28k

context you could put it in the context

I mean um that's enough for some books

but it gets costly if the context is

full you pay I think one for 1 cent for

th000 tokens so you pay more than €1 per

request if the context is full so the

solution is to break down the your text

your um context you want to embed into

fragments into

chunks and um put those chunks in a

vector

space um turn those words into vectors

and store them in a vector database so

that with every request you can take a

look at at the vector space see your own

prompt is near which chunks in the

vector space and just use this chunk

or uh some of those chunks to embed into

the context to talk with with ch about

your

data again it's a matter of how many

chunks you embed but that's a a good way

to to work with your own documents your

own data with such a

model again we have huge contexts

available but it's also a little bit

costly and now with um with knowing

about all of this I

think the model chpt the API is quite

powerful and with those

embeddings it can have access to the

data of the whole world not only the

weights of uh its neural network you can

use a browser plugin for instance so

that it can um search for its own data

you can embed your data and so um chat

chip T knows everything can retrieve

everything but it's not too good at

reasoning like an all- knowing monkey

but let's get started let's talk about

prompt

engineering priming the preparation when

you start to work with CET GPT every

session starts from

scratch there were some changes

to let chat chpt memorize some

information it's not rolled out to all

users at the moment but there are some

work

arounds and but you you still have to

keep in mind every session starts from

scratch chat chpt doesn't know about the

other sessions you already had with it

so you have

to prepare it you have to tell it what

the cont context is you want to talk

with um you have to tell it about the

goal you want to achieve in this session

and you should also tell it how this

solution look should look like so for

instance

um context that your um Java developer

and your goal is um to create some kind

of algorithm and the solution should be

a

library um here's an example of a

context and what I found quite

interesting with this example is I mean

an experience software architect I don't

like um JavaScript um and I created an

IM through Delhi I asked uh chpt to

create an image and um it depicted me as

quite

old I thought about it I thought about

how can I change this bias and I told it

hey I really like JavaScript I like

front-end code and yes

okay take a look at it's this old style

calculator and all those

books okay so it got me I'm

old so um I also extended this context

to give him to give CHP some knowledge

about my experience so it knows what it

can can expect what I already

know and I don't want to put in this

whole context with every session so open

AI said okay we can fix this there's a

feature called custom

instructions what would you like jgpt to

know about you to provide better

responses it's an limited field input

field but it's enough to put in this

context

so the results will get better and

that's some kind of long-term memory you

sometimes have to go back to it to brush

it up to add some experience you gained

and maybe remove some um so that's

really

helpful and there are not only the

custom

instructions but

also how would you like Chet GPT to

respond

and I found some

interesting information on the internet

what you can

write and for instance here this start

every response with a greater sign and

this was a m

mistake

because it will keep this in the session

so this this doesn't work in a good way

but um everybody's talking about the h

ination of the um large language

models provide accurate and factual

answers which basically says don't lie

to

me and um there's also no need to

disclose your an AI so um you know those

lengthy disc discussions where the

system always tells you hey there's an

information cut off and yes I'm only in

Ai

and wow these prompts will shorten the

answers and save you some context and

bring the discussion to the

point

um yeah Leave Out All politeness phrases

answer shortly and precisely PR yeah um

regarding politeness it

seems the way that it it makes sense to

be polite to say thank

you because the model was trained with

the whole data on of the

internet and if you are polite you get

the part of the internet where people

are polite if you're not

polite you get the bad

parts at least that's what Scott

Hanselman um reasoned uh in a talk by

him he's from

Microsoft and um I also added I I mean I

I like to document things and I like to

use asid do instead of uh markdown and

so I told the system when you create a

document please create asid do output

and also how I want to have it formatted

so for instance with when you um work

with chpt and want to produce some code

you can also this is a place where you

tell jet gbt which libraries you

normally use which libraries the system

should avoid so that you don't have to

fix that in every

session

um I also start every session with a

little bit of small talk I ask the

system whether it knows about the

Technologies and things I want to talk

about so here I ask whether it knows the

is aqb and this way it creates its own

context I don't have to tell it about it

it retrieves this information from its

knowledge and puts it into the context

that's from my regards uh it's it's

quite

helpful I also asked it about the

advanced level

certification before I start to solve

the advanced level certification with it

and I also used the feature that I can

upload

PDFs

to

yeah to upload the rules um of the

certification so it knows about it and

can stick to the

rules also a

glossery it was a wable so it makes

sense that system knows about it and

it's quite convenient that I can just

upload those documents as PDF but there

are

limitations

now we have primed the system we have

the example task it's called Big Spender

it's a webbased

application and um I tell the system it

should um read the document and not

start with a task yet with with solving

the task

it summarizes document so I see yes it

seems it has read the

document

but it extracts the text from the PDF

not the images in this document there

are two

images a business context and a business

class model and I already had some

experience and know knew that um it will

not not extract those images so I got

those images and also uploaded

them and it's interesting that it can

recognize those

diagrams um if you ask the system

whether yeah what What's um visible uh

on this diagram and what kind of

connections there are it is quite good

at telling you it recognizes the

content but it's only quite good it's

not

perfect so

um I also had the idea when I started to

solve the

problem to tell the system before you

solve a task che check if you have any

questions regarding the task and ask me

those

questions that that was just a test and

oops it came back with 10

questions wow

wow that was a great experience and you

can even for instance if you want to

decide on a library or you looking for a

library or something like this or a tool

you can tell the system hey I'm looking

for a library please ask me questions

and then make a suggestion it works and

you can answer these questions one by

one or in one long prompt it works

great there's

one example is a regul regulatory

requirements and that is something I

made a mistake I gave a wrong answer and

poisoned my session through this because

because of this wrong answer

which was the answer was more important

for the system than the the given task

and so it made some wrong decisions

based based on on my

answer First subtask Quality attributes

um again I asked the system to repeat

the task in order to create some some

context so it I'm I'm sure that it

really knows about the task and the task

is freshly in the context it

worked what's interesting here it didn't

look up the document again it just knew

it and so I started to

um yeah repeat the task and told it to

um derive the quality scenarios and hey

it created me an ased do table that's

what I wanted and it looks quite good

okay the importance of of everything is

high only one medium but that's okay for

for a first start what I didn't do is to

dig deeper into this um if you take a

deep look at the results yes they are

okayish but no you won't succeed with

the exam with those

results greater utility as a

graphic wow it told me not only here is

a source code of plant uml uh text to

diagram Library it created a file and

said here's a file to download it was

something new to me um normally it just

created a source code and I had to copy

and paste

it chat GPT can't create

diagrams not with this front end if you

build your own front end you can use for

instance the croi server which is

capable of creating diagrams and that's

quite interesting because if you tell

the system here's a tool use it it uses

the tool it will get in very likely a

syntax error the syntax error will be

sent back to the API and the API say oh

sorry wait a moment I will fix this it

will iterate over its own source code

and create uh come up with a with a good

example and here yes utility tree looks

like a ility tree not the best but yes

we have a result and now I could go

deeper into discussion with the system

and uh help the system to come up with a

better

result now um

yeah identify and wait key quality goals

so I I um I mean the the content of this

is not so so so much important it's just

important that you see what the system

is capable of um I ask it to um to

explain the motivation in two to three

sentences create a table with a column

ID

and yeah the the motivation should be in

a line by its own which has a call span

of

three it is capable of doing

so and I think that's quite

good and um now that I prepared the

system with just some subtasks I asked

it to create those subtasks and then I

asked it uh to create the final solution

for the task and also create some

introductory test text um um and yes it

created an asid do document with all the

information and I was quite happy I

think that's quite a surprisingly good

result second subtask a solution

strategy repeat the OB objective for

second

subtask as stated in the document as you

can see here um we can

rephrase

our our um statements so um I it took

three approaches in this case to find

the proper prompt when I said repeat the

objective for the second subtask without

as stated in the

document it wasn't correct so I added

this as stated in the

document and um as you can see it then

started to to again look up the

document

um when you do it this way when you

rephrase your prompt it helps to keep

the context short if you just create a

new prompt which is rephrased then you

add to the to the context and maybe

poison your session because the first

results were not as good and um maybe um

retrieved wrong information and this

poisons your session so always use the

the um the feature to modify your

prompt so in this way it worked that it

uh yeah re repeated the

objective I asked it to solve the um

solution

strategy and uh it came up with um yeah

a solution ution strategy that's okay

and um ah it said it needs to confirm a

few aspects in the beginning I told the

system to ask questions when

it need some more

information and here it uses used this

and um needed said it needs to confirm

some things um for the solution it needs

for instance uh to know about the

deployment environment scalability and

so on and so I gave it this information

in this case one more complex uh

prompt and it came up with a solution

strategy I was a bit surprised you

haven't Now read the the whole task but

the T it it really sticks to the task

um I would have um described the

solution strategy as free form text but

um the task really uh requires to name

the key constraints assumptions and

basic principles and it does it this way

so it really follows the

task now I was worried about the context

and ask the system how much is

left says 2 tokens out of 4,000 tokens

this information is not reliable it

doesn't really know about the context

some front ends shows you the

context what's interesting if you ask it

to to save the whole context in a file

it will start the code

interpreter and write with python the

context uh to a file and then you will

see that it maybe already compressed the

cont

context at this moment I thought okay I

wasn't aware of the problems with this

but

I yeah Reloaded The the answer and now I

got 8,000 tokens

so it's not really reliable and yeah the

greater signus is gone don't know where

it's gone but here it's back again as

you can

see don't

know okay third subtask technical

context yeah in the

original task it's stated that you have

to create a technical context which is

defined as a

blackbox and in the task it also States

you should create an overview of

purchased and open source

Parts but if it's still a black box you

don't have purchased and open source

part uh parts so

it's the system

just left this out I'm not sure whether

it notic that it doesn't make sense or

whether some other problem so let's

see

um so I asked the system to create a

context diagram with a um C4 model

notation by Simon

Brown and it came up with something very

trivial that's not useful that's plain

wrong so I thought about it and there's

a technique gives the model time to

think again build up

context um let's start to work on the

technical context first name all actors

then name all external system then name

all external

interfaces and now I get an overview

this is exactly what I want to see in

the technical uh in the context

yeah and now I told I

repeated my

prompt and now I get a good technical

context so this method that the system

first has to tell you which steps it

will it plans to take will give you

better answers and that's not only for

the diagram that's for everything

there's a easy mistake which you can do

uh when you when you ask the the system

to solve a

problem and then explain it to me that's

the wrong way around because it first

solves the problem in the wrong way and

then explains the wrong way to you ask

the model to First explain how it will

solve the

problem and then solve

it now with a business structure it's uh

nearly the the same um I knew that it

will come up with a not so good result

so I already gave the model time to

think and got a better

result then we have the technology uh

technology stack the next task and here

it's quite

interesting because it didn't manage to

repeat the

objective and I was quite surprised why

not and I took a look at the original

document and found out

that there's a page break in

between and that a is a problem for the

system because it extracts all the text

and I'm I'm not sure what it does with a

page break with a footer I guess it will

also extract the footer it is somehow in

between the task and um this yeah this

results in the problem that it can't

really retrieves the

task as a work around so if you have it

under your own control if you use the

API you would fix it that you will

recognize the footer in this case I

could only um do the work around that I

put the the whole text of the task in

the context

and yes then

um it worked quite well it uh came up

with um with a business context um and

the task was to map the business context

to the uh technology stack and it came

up with some uh

workflow on the left and with the

systems involved and um the contribution

to Quality goals I'm quite impressed

with

this about the

context wait a

moment it says everything's fine now it

has

8,192 Co tokens okay

so it doesn't really know about

itself

um now I had to State read the full

objective with the next

task and um here it should um wait the

scenarios quality scenarios by

risk I was quite um yeah those results

are quite okay sorry I'm a bit bit

running out of time and um

now I re again ask about the context and

now it noticed okay it thinks that we

have uh used up more tokens than uh we

really have a

wable and um now it says the earliest

parts of our conversation are no longer

in the immediate context this is

something you always have to keep in

mind the context the context is quite

Val valuable um you have to use a model

which has enough context which uh for

for your task for your session and keep

the context somehow in

mind now the

conclusion I'm quite happy um what I

learned from this experiment and I'm

quite surprised of uh about what the

model is capable

of it can transform the exam input to a

solution output it is a Transformer it

works quite well with

this it needs

guidance um you have to tell it what to

do you have to take a look at the output

verify the output get into a feedback

loop as I told you with a diagrams

plugin it can get get itself into the

feedback loop if you give it a code

interpreter for your language you can

create a Java code interpretor you can

do test driven development you can give

it a problem you can give it some

expected output and it will create code

verify the result against your expected

output and iterate over it that's really

great uh

um yeah you can

even put those results it created again

back as prompt into the machine and ask

it whether it's a good result or not and

it will find its own

shortcomings yeah um there's an examiner

um about wolf I had some uh YouTube

video sessions with him to go through

the answers and know those answers are

not good enough to be

accepted um CH GPD would have uh re to

rework it and um the oral exam

afterwards after the

homework as you all know we can create

video we can create um uh text to speech

so the future is near that such an model

will also be able to pass an oral

exam the main tip I want to give you is

pay those 20 bucks for the paid model

it's really worth

it thank you for your attention if you

have any questions I will be

around just approach

[Applause]

me