Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Arch. by Ralf D. Müller
Summary
TLDRRalph D Miller discusses the capabilities and intricacies of AI and chatbots, particularly focusing on GPT models. He emphasizes the importance of understanding the underlying technology and provides tips for effective usage, such as paying for the full version for enhanced capabilities. Miller also highlights the potential of AI in solving complex problems and stresses the need for careful priming and prompt engineering to guide the AI towards accurate and useful outputs.
Takeaways
- 🚀 GPT models, such as GPT-3 and GPT-4, are powerful generative pre-trained Transformers capable of various tasks including text translation and code conversion.
- 📈 GPT-4 has a significantly larger dataset and parameter count than GPT-3, leading to improved performance and more nuanced understanding.
- 💡 When using GPT models, it's crucial to be aware of data protection and copyright issues to avoid legal complications.
- 🤖 GPT models are often referred to as 'stochastic parrots' due to their ability to predict the next word based on probabilities, but they can also be seen as 'all-knowing monkeys' due to their vast knowledge base.
- 🧠 The architecture of GPT models includes a neural network core, natural language processing, and multimodal capabilities allowing for text, image, and voice inputs and outputs.
- 🔄 GPT models build context through interactions, which can be crucial for providing accurate and relevant responses; however, context length can vary and may impact performance.
- 📚 Embeddings are a technique used to incorporate external data into the model's context, allowing it to better understand and interact with specific information or domains.
- 💬 Prompt engineering is an essential skill when working with GPT models, as it involves priming the model with the right context, goals, and expectations for each session.
- 🔄 Feedback loops are important for refining GPT model outputs, especially in coding scenarios where the model can iterate and improve based on expected outcomes.
- 💰 Investing in the paid version of GPT models is recommended for serious users, as it offers more capabilities and better results compared to the free version.
- 🎓 For complex tasks like software architecture exams, GPT models can be a valuable tool, but they require careful guidance, context management, and verification of their outputs.
Q & A
What is Ralph D Miller's profession and main interests?
-Ralph D Miller works at DBST, the IT partner of the German Railway. His main interests include documentation, docs-as-code, security, architecture, and AI. He is also a maintainer of the Doc toolchain and an open source contributor to the docs.code project.
What is the significance of the difference in training data between GPT-3 and GPT-4?
-GPT-3 was trained on 550 gigabytes of text data, while GPT-4 was trained on a much larger dataset, estimated to be around 4 to 5 terabytes. This larger dataset allows GPT-4 to have a more extensive knowledge base and potentially provide more accurate and comprehensive responses.
How does the context size limit in chat GPT models affect the conversation flow?
-The context size limit, which was 2,000 tokens for GPT-3 and 32,000 tokens for GPT-4 at the time of the talk, affects the conversation by potentially causing the model to lose track of earlier parts of the discussion. As the conversation approaches the token limit, earlier messages may get 'out of context,' leading to less coherent responses from the model.
What is the role of embeddings in enhancing the capabilities of large language models like GPT?
-Embeddings allow the model to understand and process information more effectively by representing words or phrases as vectors in a multi-dimensional space. This enables the model to relate and compare different pieces of information more accurately, improving its ability to answer questions and solve problems based on the provided context.
How can users provide custom instructions to improve the responses from GPT models?
-Users can provide custom instructions through a feature that allows them to specify what they want the model to know about them for better responses. This could include their background, preferred libraries, coding style, and other relevant information that can help tailor the model's output to their needs.
What is the importance of prompt engineering when working with GPT models?
-Prompt engineering is crucial as it involves carefully crafting the input to guide the model towards the desired output. By structuring prompts effectively, users can influence the model to follow specific directions, provide more accurate answers, and avoid irrelevant or incorrect information.
How does the GPT model handle tasks that require multi-step problem-solving?
-The GPT model can handle multi-step problem-solving by following a series of prompts that guide it through each step of the process. It's important to first ask the model to explain its approach before it attempts to solve the problem. This allows for adjustments and refinements to the strategy before the model proceeds with the actual task.
What is the role of the mixture of experts architecture in the GPT model?
-The mixture of experts architecture involves using multiple specialized neural networks within the larger model. Each 'expert' network focuses on different types of information or tasks, which can result in certain questions being answered more effectively than others based on the expertise of the relevant network.
How can users ensure that GPT models maintain context throughout a conversation?
-Users can maintain context by being mindful of the token limit and structuring their conversation to ensure that important information is not lost. They can also use techniques like starting each session with a clear prompt, regularly reaffirming the task at hand, and breaking down complex tasks into smaller, manageable steps.
What are some limitations of GPT models in terms of reasoning and problem-solving?
-While GPT models are proficient at generating text and providing answers based on their vast knowledge base, they may not always exhibit true reasoning capabilities. Their responses can sometimes be influenced by biases in the training data or fail to accurately understand complex tasks, requiring careful guidance and verification from the user.
What is the recommendation for users who want to fully utilize the capabilities of GPT models?
-The recommendation is to subscribe to the paid version of the GPT model, which offers more advanced features and capabilities compared to the free version. The paid model provides a better context size, more accurate outputs, and access to additional functionalities like the code interpreter, making it worth the investment for serious users.
Outlines
🤖 Introduction and Disclaimer
The speaker, Ralph D Miller, introduces himself and his background in IT, documentation, security, and architecture. He mentions his work at DB Systel, the IT partner of the German Railway, and his involvement in open source documentation projects. The speaker emphasizes the importance of data protection and avoiding the sharing of personal information with AI systems. He also cautions against using AI for real exams, highlighting the potential dangers of copyrighted content generation. The introduction sets the stage for a discussion on the use of AI in creating solution architectures, with a focus on the isqb Advanced exam as a case study.
📈 GPT Models and Technology Behind
The speaker delves into the differences between GPT-3 and GPT-4, noting the increase in training data from 550 GB to multiple terabytes for GPT-4. He highlights the significant jump in parameters from 175 billion for GPT-3 to 500 billion for GPT-4, and the resulting improvements in performance. The speaker also discusses the importance of using the paid version of GPT for serious work, as the free version has limitations. He touches on the Transformer architecture, the concept of context in AI conversations, and the multimodal capabilities of GPT, which include handling text, images, and voice.
🔄 Context Management and API Usage
The speaker explains the concept of context in AI interactions, detailing how the AI builds up context during a conversation and how the stateless API requires sending the full context with each request. He discusses the token limit of GPT-3 and GPT-4, and how the AI compresses its own responses to manage this limit. The speaker shares tips on priming the AI with relevant information to ensure accurate and contextually appropriate responses. He also mentions the potential for typos in the slides and the importance of being creative when working with AI systems.
🧠 Understanding Embeddings and Prompt Engineering
The speaker introduces the concept of embeddings, explaining how the neural network can be influenced by putting data in context. He discusses the cost implications of embedding large amounts of text through the API and suggests breaking down text into chunks and storing them in a vector space as a solution. The speaker emphasizes the power of the API and embeddings in giving the AI access to vast amounts of data. He also talks about prompt engineering, detailing how to prepare the AI for each session by setting the context, defining the goal, and specifying the desired output format.
🛠️ Working with GPT on Certification Tasks
The speaker describes his experience using GPT to solve an isqb Advanced exam task called 'Big Spender'. He outlines the process of priming the system with information about the task, including uploading relevant PDF documents and using custom instructions to improve the AI's understanding. The speaker discusses the importance of context and how the AI can extract and summarize information from documents. He also shares his experience in guiding the AI through the task, asking it to confirm understanding and clarify requirements before proceeding with solutions.
📊 Quality Attributes and Solution Strategy
The speaker continues his discussion on solving the 'Big Spender' task, focusing on deriving quality attributes and creating a solution strategy. He explains how he guided the AI to repeat the task objective and derive quality scenarios, resulting in an ASID table. The speaker emphasizes the importance of rephrasing prompts to maintain context and avoid confusion. He also discusses the AI's ability to identify key quality goals and create a solution strategy that aligns with the task requirements.
🔍 Deepening the Technical Context
The speaker explores the technical context of the 'Big Spender' task, noting the AI's initial difficulty in creating a useful context diagram. He describes a technique of giving the model time to think by specifying the steps it should take, which results in a more accurate technical overview. The speaker also discusses the AI's ability to map the business context to the technology stack, creating a workflow and identifying systems that contribute to quality goals. He highlights the importance of context management and the AI's reliance on accurate and complete information to produce meaningful results.
🎓 Conclusion and Final Thoughts
In the conclusion, the speaker reflects on his experience with GPT, expressing satisfaction with its capabilities in transforming exam input into solution output. He emphasizes the need for guidance and feedback when working with AI and the potential for the technology to evolve further. The speaker also suggests investing in the paid model of GPT for its enhanced capabilities and concludes with an invitation for questions, highlighting the interactive nature of the talk.
Mindmap
Keywords
💡AI and software
💡Chatbot GPT
💡Solution Architecture
💡Data Protection
💡Copyright
💡GPT-3 vs GPT-4
💡Transformer Architecture
💡Context
💡Embeddings
💡Prompt Engineering
💡Custom Instructions
Highlights
Ralph D Miller discusses the use of AI and software in creating solution architectures.
Miller highlights the importance of data protection and avoiding the input of personal data into AI systems.
The talk introduces the isqb Advanced exam, an international software architecture qualification board certification process.
GPT models are compared, with GPT-3 trained on 550 GB and GPT-4 on an estimated 500 TB of text data.
GPT-4 outperforms GPT-3 and the average human in academic tests, scoring 82% compared to GPT-3's 65% and the human average of 75%.
The architecture of GPT includes natural language processing, a neural network core, and multimodal capabilities including image and voice recognition.
Miller emphasizes the value of using the paid version of GPT for professional work, citing its superior capabilities.
The concept of 'context' in AI chat models is discussed, explaining how it affects the model's understanding and response.
Embeddings are introduced as a method to incorporate external data into the AI model's context.
Prompt engineering is explored, demonstrating how the input to GPT can be crafted for desired outputs.
The use of custom instructions to provide the AI with specific user context is highlighted.
Miller's experience with GPT's ability to create images and the bias inherent in the model is shared.
The importance of priming the AI with context, goal, and desired solution format is emphasized for effective interaction.
The process of working with GPT to solve complex tasks, such as the isqb exam, is demonstrated step by step.
Miller discusses the challenges and workarounds when dealing with the limitations of the AI's context window.
The potential of GPT models to assist in software development tasks, such as creating algorithms and technical contexts, is showcased.
The talk concludes with a recommendation to subscribe to the paid model of GPT for its enhanced capabilities and value.
Transcripts
welcome to my talk using Ai and software
design how chat GPT can help with
creating a solution architecture I'm
quite happy that the room is full I hope
that everybody will take some takeaways
home some few words about me my name
Ralph D Miller um I work at dbst the it
partner of the German Railway
my interests are mainly documentation
docs as
code a little bit of security or to
architecture and now ai I'm a maintainer
of Doc toolchain and open source uh
documentation dox's code um pro project
and I'm contributor to Arc
42 um you can easily find me on the on
the internet find my email addresses so
if if you want to contact me later this
is not a
problem a short
disclaimer when you use chubbt be aware
of
dangers data protection might be a
problem don't send private data personal
data um to the
system and there's also a problem with
copyright because chat GPT might
generate copyrighted info um that's very
visible when you work with images it's
not so visible when you work with text
and code but I wanted to say this um up
front and um yeah don't do this with a
real exam um I'm talking now about the
is aqb Advanced exam for those who don't
know it is aqb is a international
software architecture qualification
board they have a certification process
for Architects and they have an an
example um task exam um which I will try
to solve with ch GPT but you are not
allowed to use chpt in the real exam so
don't use those approaches for the real
exam another short
disclaimer I'm not the AI expert
Specialists I'm a prompt
engineer I use Ai and chpt I'm not too
familiar with the backgrounds but
nevertheless I want to talk about the
basics how chpt works because you have
to know it um in order to use it and um
many people call those large language
models like chpt
a stochastic parrot not better than the
um autoc completion of your mobile
phone it's a neural
network and um it just produces some um
yeah probabilistic um for the next word
which might follow and this um looks
like something intelligent that's what
people say
say I say it's might be more the all
knowing knowing monkey and why I think
it's the case I will now try to explain
it starts with comparing the gpt3 versus
the GPT 4
model both are generative pre-trained
Transformers that's yeah Transformer it
can transform text so it can um
translate um text or it can translate um
for instance uh um also your code from
python to Java something like this and
it is also capable you I guess raise
your hand if you already have used uh
Chet
CHT yes so
everybody who has used G uh jet GPT 4
the paid model wow
many people I would say
50% so um you all know what GPT is
that's great let's take a look at the
difference so gpt3 was trained on 550
gigabyte of text Data GPT 4 they say
many
terabyte um I say they say it's not
really known you don't don't get too
much information about what is behind
those models what is behind this
technology but some data leaked and some
people estimate about it and yeah so
these are the
results and
um the parameters so the weights of the
neuron Network um for gpt3 it
is uh 75 5 billion parameters and for
GPT 4 it is said to be 500 billion
parameters so there is a huge difference
between those models between the free
version and the paid version and there's
also a huge difference between the free
GPT 4 model which you can use with uh
bing a the yeah Microsoft Bing search
the um chat with Bing and nowadays it's
called renamed
co-pilot it behaves in a different
way so gpt3 is four to get the real
thing the real stuff you have to pay
currently I think it's
$20 but it's worth it so that's my first
tip if you really want to work with um
CHT then pay for it it's really worth it
um that is uh another comparison so that
um there is an an yeah uh a test an an
academic test um and there are lots of
tests uh which uh they have um applied
to those models and
gpt3 do5 scores
65% the average human
75% and gbt 4
82% the reasons why those models reach
those scores are different um they know
a lot they they have seen nearly the
whole internet and that might be the
best reason why they score So High um
people yeah argue about whether there's
really reasoning behind it or not but if
you want to work with such a model these
figures count because it's important
that you get the real stuff and not the
silly answers
from the stochastic
parrot when you search for the
architecture behind
jgpt you get lots of scientific papers
and you often get this Transformer
architecture and yes if you are the AI
expert you will understand this picture
I'm not the AI expert so I made up my
own
architecture and um so what we know is
that at a core is a neural
network and we have text input we have
natural language processing this is
already quite important because your
text will not just fed to the neural
network there's a language processor
before it so that it understands the
difference of words for instance whether
when you talk about Washington whether
it's a a person or um the
city this fed to the neural network then
there's again some kind of output
processor and text
output but that's not
everything because we have a context we
just not only feed your question to the
model but we also have a context when
you chat with a model you build up
context I will tell you a little bit
more about this later and there's a
so-called mixture of experts it seems
that there's not only one neural network
at the core but several expert neural
networks so that might be also reason
why some questions are better answered
than
others and it's
multimodal what does it mean we not only
can feed text to the machine we can also
feed image images to the machine there's
an image
recognition which then results in some
textural
output there's Deli 3 for instance as
output processor to generate uh artistic
images not diagrams just artistic
images and we have voice
recognition and we also have text to
speech
output and we have a lot of more things
like the code interpretor and that's for
instance an an interesting point because
nowadays when you have the code
interpreter enabled because you have the
paid model and you ask the model a
mathem mathematical question like what's
the square root of
15 it will not respond from its neural
network it will create code for the code
interpreter to calculate the
answer lots of plugins
aailable and that's it about the
architecture what I think it's important
it's also important to know that when
you use uh jgpt you have the predefined
font
end and uh yeah you you have to take
what it gives you um soon you will
notice that you will have
more yeah
more Freedom if you work with the API
and build your own front
end oh sorry there is a typo with a
context is you will notice a lot of
typos in in these slides please I'm
sorry for that let's talk about the
context of the model when you start to
chat with a
model you will enter some priming I'm a
software developer working on a
web-based application with spring Boot
and
MySQL so the machine knows something
about you something about the context of
the
problem you then start to chat with the
machine and everything which you what
you exchange with a machine adds to the
context and the API is stateless so
every time you send a new request the
full context is sent to the machine and
the first problem occures when you get
out of the
context in the beginning the context was
quite
small I think currently chat gbt 4 has a
context so there's a chat front end of
8,000 tokens a token is something
between uh one character and a
word
and when you reach the end of the
context your first messages will get out
of context and it will forget about your
priming about the main problem and so it
will drift it will drift with its
answers because it doesn't know about
the the beginning of the of the session
um
interestingly it somehow depends on the
front and because I investigated a
little bit more in the context and
noticed that jpt starts to compress the
context by reducing its own answer size
to something sometimes just hey I've
answered the question of the user that's
it so it keeps your statements but
compresses its own statements to work
around this
problem gpt3 was said to have 2,000
tokens GPT 4 um some months ago had uh
32,000 tokens um these are the token
sizes or um yeah token size of the of
the model um when you use the front end
um the front is end is relatively cheap
say avoid to give you the full cont
context size which is at the moment um
128,000 tokens so if you use a chat I
think you have around 8,000 tokens if
you use the API you can get the full
set so first priming
tip I started to start every
session
with with um The Prompt that CHT should
start every response with a greater sign
why because it produces markdown output
and in markdown it's just the the quote
Mark the line on the left and this way I
can see whether my first statement ran
out of context or
not um it works quite
well for some chat front ends so as I
said if it compresses its own um answers
this will not work but it's somehow an
an indicator and it shows you that you
have to be a little bit creative when
you work with a
system embeddings embeddings is also
something um quite powerful and this is
only a short introduction to embeddings
um
the the neural network is
already yeah trained and you you can't
easily change it and uh the solution to
um to give your model your own data or
to give Chet GPT your own data your own
documents you want to work with for
instance your code is to put your code
in context if you have limited
context then you can't put your your
whole
document your whole codee in context
when you have the huge context through
the API is a one 28k
context you could put it in the context
I mean um that's enough for some books
but it gets costly if the context is
full you pay I think one for 1 cent for
th000 tokens so you pay more than €1 per
request if the context is full so the
solution is to break down the your text
your um context you want to embed into
fragments into
chunks and um put those chunks in a
vector
space um turn those words into vectors
and store them in a vector database so
that with every request you can take a
look at at the vector space see your own
prompt is near which chunks in the
vector space and just use this chunk
or uh some of those chunks to embed into
the context to talk with with ch about
your
data again it's a matter of how many
chunks you embed but that's a a good way
to to work with your own documents your
own data with such a
model again we have huge contexts
available but it's also a little bit
costly and now with um with knowing
about all of this I
think the model chpt the API is quite
powerful and with those
embeddings it can have access to the
data of the whole world not only the
weights of uh its neural network you can
use a browser plugin for instance so
that it can um search for its own data
you can embed your data and so um chat
chip T knows everything can retrieve
everything but it's not too good at
reasoning like an all- knowing monkey
but let's get started let's talk about
prompt
engineering priming the preparation when
you start to work with CET GPT every
session starts from
scratch there were some changes
to let chat chpt memorize some
information it's not rolled out to all
users at the moment but there are some
work
arounds and but you you still have to
keep in mind every session starts from
scratch chat chpt doesn't know about the
other sessions you already had with it
so you have
to prepare it you have to tell it what
the cont context is you want to talk
with um you have to tell it about the
goal you want to achieve in this session
and you should also tell it how this
solution look should look like so for
instance
um context that your um Java developer
and your goal is um to create some kind
of algorithm and the solution should be
a
library um here's an example of a
context and what I found quite
interesting with this example is I mean
an experience software architect I don't
like um JavaScript um and I created an
IM through Delhi I asked uh chpt to
create an image and um it depicted me as
quite
old I thought about it I thought about
how can I change this bias and I told it
hey I really like JavaScript I like
front-end code and yes
okay take a look at it's this old style
calculator and all those
books okay so it got me I'm
old so um I also extended this context
to give him to give CHP some knowledge
about my experience so it knows what it
can can expect what I already
know and I don't want to put in this
whole context with every session so open
AI said okay we can fix this there's a
feature called custom
instructions what would you like jgpt to
know about you to provide better
responses it's an limited field input
field but it's enough to put in this
context
so the results will get better and
that's some kind of long-term memory you
sometimes have to go back to it to brush
it up to add some experience you gained
and maybe remove some um so that's
really
helpful and there are not only the
custom
instructions but
also how would you like Chet GPT to
respond
and I found some
interesting information on the internet
what you can
write and for instance here this start
every response with a greater sign and
this was a m
mistake
because it will keep this in the session
so this this doesn't work in a good way
but um everybody's talking about the h
ination of the um large language
models provide accurate and factual
answers which basically says don't lie
to
me and um there's also no need to
disclose your an AI so um you know those
lengthy disc discussions where the
system always tells you hey there's an
information cut off and yes I'm only in
Ai
and wow these prompts will shorten the
answers and save you some context and
bring the discussion to the
point
um yeah Leave Out All politeness phrases
answer shortly and precisely PR yeah um
regarding politeness it
seems the way that it it makes sense to
be polite to say thank
you because the model was trained with
the whole data on of the
internet and if you are polite you get
the part of the internet where people
are polite if you're not
polite you get the bad
parts at least that's what Scott
Hanselman um reasoned uh in a talk by
him he's from
Microsoft and um I also added I I mean I
I like to document things and I like to
use asid do instead of uh markdown and
so I told the system when you create a
document please create asid do output
and also how I want to have it formatted
so for instance with when you um work
with chpt and want to produce some code
you can also this is a place where you
tell jet gbt which libraries you
normally use which libraries the system
should avoid so that you don't have to
fix that in every
session
um I also start every session with a
little bit of small talk I ask the
system whether it knows about the
Technologies and things I want to talk
about so here I ask whether it knows the
is aqb and this way it creates its own
context I don't have to tell it about it
it retrieves this information from its
knowledge and puts it into the context
that's from my regards uh it's it's
quite
helpful I also asked it about the
advanced level
certification before I start to solve
the advanced level certification with it
and I also used the feature that I can
upload
PDFs
to
yeah to upload the rules um of the
certification so it knows about it and
can stick to the
rules also a
glossery it was a wable so it makes
sense that system knows about it and
it's quite convenient that I can just
upload those documents as PDF but there
are
limitations
now we have primed the system we have
the example task it's called Big Spender
it's a webbased
application and um I tell the system it
should um read the document and not
start with a task yet with with solving
the task
it summarizes document so I see yes it
seems it has read the
document
but it extracts the text from the PDF
not the images in this document there
are two
images a business context and a business
class model and I already had some
experience and know knew that um it will
not not extract those images so I got
those images and also uploaded
them and it's interesting that it can
recognize those
diagrams um if you ask the system
whether yeah what What's um visible uh
on this diagram and what kind of
connections there are it is quite good
at telling you it recognizes the
content but it's only quite good it's
not
perfect so
um I also had the idea when I started to
solve the
problem to tell the system before you
solve a task che check if you have any
questions regarding the task and ask me
those
questions that that was just a test and
oops it came back with 10
questions wow
wow that was a great experience and you
can even for instance if you want to
decide on a library or you looking for a
library or something like this or a tool
you can tell the system hey I'm looking
for a library please ask me questions
and then make a suggestion it works and
you can answer these questions one by
one or in one long prompt it works
great there's
one example is a regul regulatory
requirements and that is something I
made a mistake I gave a wrong answer and
poisoned my session through this because
because of this wrong answer
which was the answer was more important
for the system than the the given task
and so it made some wrong decisions
based based on on my
answer First subtask Quality attributes
um again I asked the system to repeat
the task in order to create some some
context so it I'm I'm sure that it
really knows about the task and the task
is freshly in the context it
worked what's interesting here it didn't
look up the document again it just knew
it and so I started to
um yeah repeat the task and told it to
um derive the quality scenarios and hey
it created me an ased do table that's
what I wanted and it looks quite good
okay the importance of of everything is
high only one medium but that's okay for
for a first start what I didn't do is to
dig deeper into this um if you take a
deep look at the results yes they are
okayish but no you won't succeed with
the exam with those
results greater utility as a
graphic wow it told me not only here is
a source code of plant uml uh text to
diagram Library it created a file and
said here's a file to download it was
something new to me um normally it just
created a source code and I had to copy
and paste
it chat GPT can't create
diagrams not with this front end if you
build your own front end you can use for
instance the croi server which is
capable of creating diagrams and that's
quite interesting because if you tell
the system here's a tool use it it uses
the tool it will get in very likely a
syntax error the syntax error will be
sent back to the API and the API say oh
sorry wait a moment I will fix this it
will iterate over its own source code
and create uh come up with a with a good
example and here yes utility tree looks
like a ility tree not the best but yes
we have a result and now I could go
deeper into discussion with the system
and uh help the system to come up with a
better
result now um
yeah identify and wait key quality goals
so I I um I mean the the content of this
is not so so so much important it's just
important that you see what the system
is capable of um I ask it to um to
explain the motivation in two to three
sentences create a table with a column
ID
and yeah the the motivation should be in
a line by its own which has a call span
of
three it is capable of doing
so and I think that's quite
good and um now that I prepared the
system with just some subtasks I asked
it to create those subtasks and then I
asked it uh to create the final solution
for the task and also create some
introductory test text um um and yes it
created an asid do document with all the
information and I was quite happy I
think that's quite a surprisingly good
result second subtask a solution
strategy repeat the OB objective for
second
subtask as stated in the document as you
can see here um we can
rephrase
our our um statements so um I it took
three approaches in this case to find
the proper prompt when I said repeat the
objective for the second subtask without
as stated in the
document it wasn't correct so I added
this as stated in the
document and um as you can see it then
started to to again look up the
document
um when you do it this way when you
rephrase your prompt it helps to keep
the context short if you just create a
new prompt which is rephrased then you
add to the to the context and maybe
poison your session because the first
results were not as good and um maybe um
retrieved wrong information and this
poisons your session so always use the
the um the feature to modify your
prompt so in this way it worked that it
uh yeah re repeated the
objective I asked it to solve the um
solution
strategy and uh it came up with um yeah
a solution ution strategy that's okay
and um ah it said it needs to confirm a
few aspects in the beginning I told the
system to ask questions when
it need some more
information and here it uses used this
and um needed said it needs to confirm
some things um for the solution it needs
for instance uh to know about the
deployment environment scalability and
so on and so I gave it this information
in this case one more complex uh
prompt and it came up with a solution
strategy I was a bit surprised you
haven't Now read the the whole task but
the T it it really sticks to the task
um I would have um described the
solution strategy as free form text but
um the task really uh requires to name
the key constraints assumptions and
basic principles and it does it this way
so it really follows the
task now I was worried about the context
and ask the system how much is
left says 2 tokens out of 4,000 tokens
this information is not reliable it
doesn't really know about the context
some front ends shows you the
context what's interesting if you ask it
to to save the whole context in a file
it will start the code
interpreter and write with python the
context uh to a file and then you will
see that it maybe already compressed the
cont
context at this moment I thought okay I
wasn't aware of the problems with this
but
I yeah Reloaded The the answer and now I
got 8,000 tokens
so it's not really reliable and yeah the
greater signus is gone don't know where
it's gone but here it's back again as
you can
see don't
know okay third subtask technical
context yeah in the
original task it's stated that you have
to create a technical context which is
defined as a
blackbox and in the task it also States
you should create an overview of
purchased and open source
Parts but if it's still a black box you
don't have purchased and open source
part uh parts so
it's the system
just left this out I'm not sure whether
it notic that it doesn't make sense or
whether some other problem so let's
see
um so I asked the system to create a
context diagram with a um C4 model
notation by Simon
Brown and it came up with something very
trivial that's not useful that's plain
wrong so I thought about it and there's
a technique gives the model time to
think again build up
context um let's start to work on the
technical context first name all actors
then name all external system then name
all external
interfaces and now I get an overview
this is exactly what I want to see in
the technical uh in the context
yeah and now I told I
repeated my
prompt and now I get a good technical
context so this method that the system
first has to tell you which steps it
will it plans to take will give you
better answers and that's not only for
the diagram that's for everything
there's a easy mistake which you can do
uh when you when you ask the the system
to solve a
problem and then explain it to me that's
the wrong way around because it first
solves the problem in the wrong way and
then explains the wrong way to you ask
the model to First explain how it will
solve the
problem and then solve
it now with a business structure it's uh
nearly the the same um I knew that it
will come up with a not so good result
so I already gave the model time to
think and got a better
result then we have the technology uh
technology stack the next task and here
it's quite
interesting because it didn't manage to
repeat the
objective and I was quite surprised why
not and I took a look at the original
document and found out
that there's a page break in
between and that a is a problem for the
system because it extracts all the text
and I'm I'm not sure what it does with a
page break with a footer I guess it will
also extract the footer it is somehow in
between the task and um this yeah this
results in the problem that it can't
really retrieves the
task as a work around so if you have it
under your own control if you use the
API you would fix it that you will
recognize the footer in this case I
could only um do the work around that I
put the the whole text of the task in
the context
and yes then
um it worked quite well it uh came up
with um with a business context um and
the task was to map the business context
to the uh technology stack and it came
up with some uh
workflow on the left and with the
systems involved and um the contribution
to Quality goals I'm quite impressed
with
this about the
context wait a
moment it says everything's fine now it
has
8,192 Co tokens okay
so it doesn't really know about
itself
um now I had to State read the full
objective with the next
task and um here it should um wait the
scenarios quality scenarios by
risk I was quite um yeah those results
are quite okay sorry I'm a bit bit
running out of time and um
now I re again ask about the context and
now it noticed okay it thinks that we
have uh used up more tokens than uh we
really have a
wable and um now it says the earliest
parts of our conversation are no longer
in the immediate context this is
something you always have to keep in
mind the context the context is quite
Val valuable um you have to use a model
which has enough context which uh for
for your task for your session and keep
the context somehow in
mind now the
conclusion I'm quite happy um what I
learned from this experiment and I'm
quite surprised of uh about what the
model is capable
of it can transform the exam input to a
solution output it is a Transformer it
works quite well with
this it needs
guidance um you have to tell it what to
do you have to take a look at the output
verify the output get into a feedback
loop as I told you with a diagrams
plugin it can get get itself into the
feedback loop if you give it a code
interpreter for your language you can
create a Java code interpretor you can
do test driven development you can give
it a problem you can give it some
expected output and it will create code
verify the result against your expected
output and iterate over it that's really
great uh
um yeah you can
even put those results it created again
back as prompt into the machine and ask
it whether it's a good result or not and
it will find its own
shortcomings yeah um there's an examiner
um about wolf I had some uh YouTube
video sessions with him to go through
the answers and know those answers are
not good enough to be
accepted um CH GPD would have uh re to
rework it and um the oral exam
afterwards after the
homework as you all know we can create
video we can create um uh text to speech
so the future is near that such an model
will also be able to pass an oral
exam the main tip I want to give you is
pay those 20 bucks for the paid model
it's really worth
it thank you for your attention if you
have any questions I will be
around just approach
[Applause]
me
Browse More Related Video
![](https://i.ytimg.com/vi/gyJ_mDj-s4I/hq720.jpg?sqp=-oaymwEmCIAKENAF8quKqQMa8AEB-AH-CYAC0AWKAgwIABABGDkgOCh_MA8=&rs=AOn4CLCA2Zax3ImkzWitiMaj0YE5L6g9Yg)
Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Architecture | R. Müller
![](https://i.ytimg.com/vi/1UXyhj5iPBU/hq720.jpg)
Prompt Engineering - Corso professionale Parte 1/2
![](https://i.ytimg.com/vi/DlrOke9FdIU/hqdefault.jpg?sqp=-oaymwEXCJADEOABSFryq4qpAwkIARUAAIhCGAE=&rs=AOn4CLD01zp4GtBPQVFbTL9mWem1u8itMA)
The implications of AI on a Center of Excellence
![](https://i.ytimg.com/vi/Bq-ncjOGeVU/hqdefault.jpg?sqp=-oaymwEXCJADEOABSFryq4qpAwkIARUAAIhCGAE=&rs=AOn4CLB3Q86yIFS7UxWBcb88IY04sw0APw)
What is Prompt Engineering? (in about a minute)
![](https://i.ytimg.com/vi/H9Hz6mmn5vY/hq720.jpg)
Basi e Principi utili del Prompt Engineering, l'arte di saper parlare con le AI Generative
![](https://i.ytimg.com/vi/dz-iF5N9WZE/hq720.jpg)
Промпт інженер. Як правильно написати промпт ChatGPT
5.0 / 5 (0 votes)