Memory Hierarchy & Interfacing

Summary

This educational session delves into the concept of memory hierarchy, explaining how different memory units are ranked based on access time and size, with registers being the fastest and secondary storage the slowest. It highlights the purpose of the hierarchy in bridging the speed gap between processors and memory while keeping costs reasonable. The video also explores two methods of memory interfacing, discussing the hit and miss ratios and their impact on average memory access time, aiming to provide a clear understanding of computer organization and memory efficiency.

Takeaways

• 📚 The concept of memory hierarchy in computing is analogous to ranking or order based on different parameters, similar to how movies can be ranked by release dates or ratings.
• 🏰 Memory units are ranked based on access time and size, with registers being the fastest and secondary memory being the slowest but largest in capacity.
• 💻 The purpose of the memory hierarchy is to bridge the speed gap between the fast processor and the slower memory while keeping costs reasonable.
• 🔍 Memory interfacing is a part of computer organization that involves connecting memory units to the processor and I/O peripherals to feed instructions efficiently.
• ⏱️ Processor speed is measured in MIPS (Million Instructions Per Second), and the goal is to provide this many instructions from memory quickly and cost-effectively.
• 🎯 In the memory hierarchy, a 'hit' occurs when the processor finds the required information in the current level of memory, and a 'miss' means it must look in the next level.
• 📈 The hit ratio is calculated as the percentage of times the processor successfully accesses the required information from a memory level without missing.
• 🔢 The effective or average memory access time can be calculated using a formula that considers hit ratios and access times of different memory levels.
• 🔄 There are two ways of interfacing memory units: simultaneous connection to the processor, allowing parallel search across levels, and level-wise connection, where the processor searches level by level sequentially.
• ⏳ In simultaneous interfacing, the average access time is calculated considering the parallel search and the probabilities of hits and misses at each level.
• 📉 In level-wise interfacing, the average access time formula accounts for the sequential search, including the cumulative access times of all levels in case of a miss.
• 📚 The next session will involve solving numerical problems related to memory hierarchy to enhance understanding of the concept.

Q & A

What does the term 'hierarchy' refer to in the context of memory hierarchy?

In the context of memory hierarchy, 'hierarchy' refers to the ranking or organization of different memory units based on various parameters such as access time, size, and cost.

Why is the memory hierarchy important in computer systems?

The memory hierarchy is important because it helps bridge the speed mismatch between the fast processor and the slower memory, ensuring efficient data retrieval at a reasonable cost.

What are the different levels of memory units in the hierarchy based on access time and size?

The different levels of memory units, from fastest to slowest, are registers, SRAM caches, DRAM main memory, and secondary memory storages.

How does the cost and usage frequency relate to the memory hierarchy?

As we move up in the memory hierarchy, the cost and frequency of usage increase, with the processor being the most expensive and most frequently used component.

What is the purpose of the memory interfacing in computer organization?

Memory interfacing in computer organization deals with the way of connecting various levels of memory units, especially to the processor and I/O peripherals, to efficiently feed the processor with instructions.

What does MIPS stand for and what does it measure?

MIPS stands for Million Instructions Per Second and it measures the speed of the processor by counting how many instructions it can execute in a second.

What is a 'hit' in the context of memory hierarchy?

A 'hit' in the context of memory hierarchy occurs when the processor finds the required instruction or data in the memory level it is currently referring to.

What is a 'miss' and how does it relate to the memory hierarchy?

A 'miss' occurs when the processor does not find the required instruction or data in the current memory level and must look for it in the next level of the hierarchy.

What is the hit ratio and how is it calculated?

The hit ratio is the percentage of times the processor finds the required instructions or data in a particular memory level. It is calculated by dividing the number of hits by the total number of instructions.

What are the two ways of interfacing memory units with the processor as described in the script?

The two ways of interfacing memory units with the processor are: 1) Simultaneously connecting all different memory levels to the processor, allowing it to look for information in all levels side by side. 2) Connecting memory units level-wise, where the processor looks for information in one level at a time, moving to the next level only if the information is not found in the current one.

Can you explain the formula for effective or average memory access time in the context of simultaneous memory interfacing?

The formula for effective or average memory access time in simultaneous memory interfacing is the sum of the products of hit ratios and access times for each memory level. It accounts for the time taken to access each level based on the probability of finding the instruction there and includes all levels from the fastest to the slowest.

Outlines

00:00

📚 Introduction to Memory Hierarchy and Interfacing

In this educational session, the presenter introduces the concept of memory hierarchy, explaining it with the analogy of ranking in Nolan's Dark Knight trilogy. The hierarchy of memory units is discussed based on access time and size, with registers being the fastest and secondary memory being the slowest. The purpose of the memory hierarchy is to bridge the speed gap between the fast processor and slow memory at a reasonable cost. The session also covers the basics of memory interfacing, explaining how memory units are connected to the processor and I/O peripherals, and introduces the concept of hits and misses in memory access. The presenter aims to provide a foundational understanding before delving into numerical problems in the next session.

05:02

🔍 Understanding Memory Access Time and Interfacing Methods

This paragraph delves deeper into the calculation of effective or average memory access time, using a formula that considers the hit ratios and access times of different memory levels. The presenter explains the concept of hit ratio, which is the probability of finding an instruction in a particular memory level, and how it affects the average access time. Two methods of memory interfacing are discussed: simultaneous connection of all memory levels to the processor, and level-wise interfacing where the processor sequentially checks each level if the instruction is not found in the previous one. The presenter provides a detailed explanation of the formulas used for calculating access time in both interfacing methods and sets the stage for solving related numerical problems in the upcoming session.

Keywords

💡Memory Hierarchy

Memory hierarchy refers to the organization of a computer's memory system into a series of levels, each with varying access times and capacities. In the script, it is explained as a ranking of memory units based on access time and size, with registers being the fastest and secondary memory being the slowest. The concept is crucial for understanding how a computer manages data flow to match the speed of the processor with the efficiency of memory storage.

💡Processor

The processor, also known as the CPU, is the component of a computer that performs arithmetic and logical operations. It is central to the video's theme as the speed of the processor is contrasted with the slower speed of memory, highlighting the need for a memory hierarchy to bridge this speed mismatch.

💡Access Time

Access time is the duration it takes for a memory unit to return data once an access request is made. The script discusses how different levels of the memory hierarchy have different access times, with registers having the least and secondary memory having the most, which is a key factor in the ranking of the memory hierarchy.

💡SRAM Caches

SRAM (Static Random-Access Memory) caches are a type of high-speed memory used in the memory hierarchy, placed between the processor and main memory to reduce access time. The script mentions SRAM caches as part of the hierarchy, emphasizing their role in improving the efficiency of data retrieval for the processor.

💡DRAM Main Memory

DRAM (Dynamic Random-Access Memory) is the primary memory used in computers, where the majority of the operating system, applications, and data are stored. The script positions DRAM as a level in the memory hierarchy that is slower than caches but faster than secondary memory.

💡Secondary Memory

Secondary memory refers to non-volatile storage devices like hard drives or SSDs, which hold data even when power is off. In the script, secondary memory is described as the slowest level of the memory hierarchy but is essential for permanent data storage.

💡Hit and Miss

In the context of the memory hierarchy, a 'hit' occurs when the data the processor requests is found in the current level of memory, while a 'miss' means the data is not found and must be sought in a lower level. The script explains the importance of hit and miss in calculating the efficiency of memory access.

💡Hit Ratio

The hit ratio is the percentage of times that data is found in a particular level of the memory hierarchy when requested. The script uses the hit ratio to illustrate the efficiency of caches and main memory in providing data to the processor.

💡Interfacing

Interfacing in the script refers to the method of connecting different levels of memory to the processor and I/O peripherals. It discusses two main ways of interfacing: simultaneous connection of all levels to the processor and level-wise connection, which affects how data is accessed and the overall performance of the system.

💡MIPS

MIPS (Million Instructions Per Second) is a measure of a processor's speed, indicating how many instructions it can execute in a second. The script mentions MIPS to emphasize the goal of feeding the processor with instructions as quickly and cost-effectively as possible.

💡Effective or Average Memory Access Time

Effective or average memory access time is a metric used to calculate the average time taken to access data in a memory hierarchy. The script provides a formula to understand this concept, which is essential for evaluating the performance of different memory interfacing methods.

Highlights

Introduction to memory hierarchy and its purpose to bridge the speed mismatch between fast processors and slow memory.

Explanation of hierarchy as ranking based on different parameters, using Nolan's Dark Knight Trilogy as an analogy.

Ranking of memory units based on access time and size, with registers being the fastest and secondary memory being the slowest.

Cost and usage frequency hierarchy being reversed compared to access time, with higher levels being more expensive and used less frequently.

The processor's speed is measured in MIPS (million instructions per second), and the goal is to feed it instructions efficiently.

Definition of 'hit' and 'miss' in the context of memory access, where a hit is finding the required information in the current level.

The time taken for a miss includes searching both the current and the next level of memory.

Illustration of two ways of memory interfacing: simultaneous connection and level-wise connection to the processor.

Effective or average memory access time formula explained for simultaneous memory interfacing.

Explanation of hit ratio, which is the probability of finding an instruction in a particular memory level.

The importance of the last level of memory hierarchy having a 100% hit ratio due to its permanent storage nature.

Calculation of average memory access time considering hit ratios and access times of different memory levels.

The sequential search process in level-wise memory interfacing, where the processor looks for information level by level.

Different formula for calculating access time in level-wise interfacing, considering both access times of the current and next levels.

Upcoming session预告 to solve numerical problems related to memory hierarchy for a deeper understanding.

Closing remarks and thanks to the audience for watching the session on memory hierarchy and interfacing.

Transcripts

hello

everyone welcome back in this session

we are going to learn about first the

memory hierarchy

and then we will have a brief

introduction regarding how the different

memory units

are interfaced with the processor so

let's get to learning

now the term hierarchy if put simply

means ranking

also ranking can be done based on

different parameters

for an instance if we consider nolan's

dark knight trilogy

according to the release dates batman

begins comes first

then the dark knight and finally the

dark knight rises

but if rotten tomatoes tomato meter is

considered the dark knight is on the top

having the freshness of 94 percent

because

honestly heath ledger just rocked it

right then comes the dark knight rises

and finally batman begins now this

ranking remains the same

if metacritic scores are also considered

similarly

based on different aspects the memory

storages

can also be ranked if we consider access

time and size we can rank the memory

units

like this registers are made up of flip

flops

and are embedded within the processor

itself therefore

time to access them is the least the

sram caches

are next followed by the dram main

memory

last but not least the secondary memory

storages

however in terms of cost and usage

frequency

this hierarchy is reversed i mean as we

move up in this

the cost and the frequency of usage

increases in our previous discussion

we already understood that the processor

is very fast and the sole purpose of

having the memory hierarchy

is to bridge the speed mismatch between

the fast processor to slow memory

that too at a reasonable cost now coming

to memory interfacing it is a part of

computer organization

which deals with the way of connecting

various levels of memory units

especially to the processor also to the

i o peripherals

generally the speed of the processor is

counted

using the unit mips some call it

mips it stands for million instructions

per second

and our goal is to try to feed the

processor

with those many instructions from the

memory in a faster

yet cost effective approach let's try to

understand

various levels of the memory if i say

information at nth level is a subset of

the information available in the n plus

1 level it means if the processor refers

to the nth level memory for something

may that be any instruction or data if

that's found there only

we term it as hit otherwise it's called

a mess

and the processor then goes to the n

plus one level that is the next level to

look for the same

so in case of a miss the time taken to

look for the information in both

nth level and n plus one level should be

considered

however it's not always the case it

actually depends on the way the

interfacing has been done

let's dig a bit deeper now generally

interfacing is done in two ways

the following illustrations will help us

understand these ways

way one all the different memory levels

are simultaneously connected to the

processor and whenever the processor

wants some information

it can look for it in all the different

levels side by side

here the processor is connected to three

memory units of different levels

now for the time being let's call them

m1 m2 and

m3 assuming the access time for these

are

t1 t2 and t3 also t1 is less than t2

which in turn

is less than t3 now let's presume that

for a program

or a set of instructions the hit ratio

of the m1 and m2 are h1 and h2

respectively all right now the question

that you must be having is

what on earth is hit ratio right

now we already know that during

execution the set of instructions are

brought into the main memory from the

non-volatile secondary storage

also the frequently accessed portions

are stored inside the cache

let's just say there are 100

instructions in a program and it's

permanently stored inside the last level

of memory hierarchy that is the

secondary memory

suppose from those 100 80 of them are

brought into the main memory

in that case the hit ratio would be 80

by 100

that is 0.8 or 80 percent

this means during execution there is 80

chance for the processor of getting the

required instructions

inside the main memory now as the last

level is nothing but the permanent

storage

there is the hit ratio is naturally

hundred percent so we don't really

bother about it because

if the program isn't there there is no

way for the processor to execute it

it's like if we don't really have the

audio file how on earth we can play it

so coming back to our illustration with

all these informations

if we now try to find out how much time

is needed on an

average to find out an information in

this organization

which is by the way popularly known as

effective

or average memory access time the

formula is as follows

let's understand this formula so that we

don't really have to memorize it

so there is h1 percent chance of finding

the instruction

inside m1 and this is the time taken to

access m1

now 1 minus 81 percent chance is also

there that the processor

may not find the instruction required

inside

m1 which is called the miss rate or miss

ratio of m1

for those cases the processor will look

inside the next level that is m2

which has 82 percent chance of having

the instructions in it

also the time taken to access m2 is t2

hence 1 minus h1 multiplied by

h2 and t2 so finally there are

1 minus 82 percent chance of not finding

the instructions

inside m2 that is the miss ratio or the

miss rate of

m2 is 1 minus h2 therefore

1 minus h1 multiplied by 1 minus h2 that

is considering

all the chances where we can't find the

required instructions either in m1

or in m2 in that case we will have to

look for it

inside m3 that is the last level and the

time required to access

m3 is t3 hence 1 minus

h1 multiplied by 1 minus h2 multiplied

by t3

now remember the memory devices of the

different levels are connected to the

processor simultaneously and that's why

these all will take place parallelly now

the second way

is to interface the memory units level

wise

as this that means if the required

instruction

can't be found in m1 then the processor

will go and look for it inside the next

level that is m2

and if it is not even there the seeking

process will continue to the next level

so in that case the formula will become

this

because here the processor is not

simultaneously connected to all the

memory units

also if the processor is looking for an

instruction

in the n plus one level that means it

already has looked for it in the nth

level or the previous level

and it couldn't find it there that's why

both the access times are being

considered

well that was all about memory hierarchy

and the different ways the memory

interfacing takes place

in the next session we will try to solve

a few numerical problems

associated to this concept to have a

lucid understanding

so i hope to see you in the next one

thank you all for watching

you