How to Start an AWS Step Function Workflow From Lambda | STEP BY STEP GUIDE
Summary
TLDRThis AWS tutorial demonstrates how to initiate a Step Function from a Lambda function. The video guides viewers through creating a Step Function instance and a Lambda function in Python, setting the necessary permissions, and writing code to interact with Step Functions. It showcases the process of defining a unique transaction ID using the UUID library, crafting a JSON input, and invoking the Step Function's start execution API. The tutorial concludes with testing the Lambda function and verifying successful Step Function instances.
Takeaways
- 😀 The video demonstrates how to start a Step Function instance from a Lambda function in AWS.
- 🔗 The presenter has already created a Step Function instance and predefined some AWS Step Language (ASL) code for a simple task involving an SQS queue.
- 📝 The ASL code takes an input and communicates with the SQS 'SendMessage' API, posting a JSON object with a transaction ID and type derived from the Step Function input.
- 🆔 The Step Function input includes a unique transaction ID and a type, both of which are strings.
- 👉 The presenter copies the Amazon Resource Name (ARN) of the Step Function for use in the Lambda function creation process.
- 🤖 The Lambda function named 'step function invoker' is created in Python 3.6 or 3.7, with permissions set to access the Step Function.
- 🔑 It's important to use the least privilege model for permissions, but the presenter uses full access for simplicity in this tutorial.
- 💾 The Lambda function's code uses the 'boto3' library to interact with AWS services, specifically the Step Functions client.
- 🔄 A unique transaction ID is generated using the 'UUID' library to ensure the Step Function instance has a unique key.
- 📑 The input for the Step Function is defined as a JSON object, with the transaction ID and type hardcoded as 'purchase' in the example.
- 🚀 The 'start_execution' API of the Step Functions client is called with the state machine ARN, a unique name (transaction ID), and the JSON payload as parameters.
Q & A
What is the main topic of the video?
-The main topic of the video is demonstrating how to start a Step Function instance from a Lambda function in AWS.
What is ASL in the context of the video?
-ASL in this context refers to Amazon States Language, which is used to define the state machine for Step Functions in AWS.
What is the purpose of the SQS SendMessage API in the Step Function?
-The SQS SendMessage API is used in the Step Function to post a JSON object containing a transaction ID and type to an SQS queue.
What are the inputs required for the Step Function in the script?
-The Step Function requires an input containing a transaction ID and a type, both of which are strings.
What is the ARN used for in the Lambda function?
-The ARN (Amazon Resource Name) is used to reference the Step Function that the Lambda function will interact with.
Why is it important to use the least privilege model when creating an IAM role for the Lambda function?
-Using the least privilege model ensures that the Lambda function has only the permissions it needs to perform its tasks, enhancing security by minimizing potential access to sensitive resources.
What is the role of the UUID library in the Lambda function code?
-The UUID library is used to generate a unique transaction ID for each Step Function instance, ensuring that each instance has a unique identifier.
How does the Lambda function interact with the Step Functions client?
-The Lambda function uses the boto3 library to create an instance of the Step Functions client and then calls the start_execution API to start a Step Function instance.
What is the significance of the 'name' field in the start_execution API call?
-The 'name' field serves as the unique identifier for the Step Function instance, allowing AWS to distinguish between different instances.
How does the Lambda function handle the input for the Step Function?
-The Lambda function constructs a JSON object with the required transaction ID and type, converts it to a string using the JSON library, and passes it as the input to the start_execution API call.
How can you verify that the Lambda function has successfully started the Step Function instances?
-You can verify the success by checking the logs in the Lambda function and by looking at the Step Functions console to see the instances and their progress.
Outlines
🚀 Starting a Step Function from Lambda
This paragraph introduces the video's objective to demonstrate initiating a Step Function instance from within a Lambda function. The presenter has already set up a Step Function instance and predefined ASL code for a task that interacts with the SQS 'send message' API. The input for the Step Function consists of a transaction ID and a type, both of which are strings. The presenter emphasizes the importance of using unique identifiers for Step Function instances and proceeds to show how to create a Lambda function with the necessary permissions to start the Step Function.
🛠 Creating and Testing the Lambda Function
The second paragraph details the process of creating a Lambda function named 'step function invoker' in Python 3.6 or 3.7. The function requires specific permissions for accessing the Step Function service, and the presenter advises using the least privilege model. The Lambda function's code is written in an external editor, with the presenter explaining the need for libraries such as 'json', 'moto3', and 'uuid'. The code generates a unique transaction ID using the 'uuid' library and constructs a JSON input object for the Step Function. The Lambda function then uses the 'boto3' Step Function client to start the execution of the Step Function, passing the state machine ARN, a unique name, and the JSON payload. The paragraph concludes with instructions on testing the Lambda function by creating a test event and verifying successful execution in the Step Functions console.
Mindmap
Keywords
💡AWS
💡Lambda
💡Step Functions
💡Event Orchestration
💡ASL
💡SQS
💡JSON
💡IAM Role
💡UUID
💡Start Execution API
💡Moto3
Highlights
Introduction to integrating AWS Lambda with Step Functions for event orchestration.
Demonstration of starting a Step Function instance from a Lambda function.
Creation of a Step Function instance named 'step function from lambda' with predefined ASL code.
Explanation of the Step Function's task to communicate with the SQS Send Message API.
Details on the JSON object structure for the transaction ID and type from the Step Function input.
Copying the ARN for use in creating the Lambda function to start the Step Function instance.
Creating a new Lambda function named 'step function invoker' in Python 3.6.
Importance of using the correct permissions for the Lambda function to access Step Functions.
Using the least privilege model to assign only necessary permissions for the Lambda function.
Writing code in the Lambda function to interact with Step Functions using the boto3 client.
Generating a unique transaction ID using the UUID library for the Step Function instance.
Defining the input object for the Step Function with a hardcoded 'purchase' type.
Calling the Step Function's start_execution API with the necessary parameters.
Conversion of the Python dictionary object to a JSON string for the API call.
Testing the Lambda function with a test event to ensure successful Step Function execution.
Verification of multiple successful Step Function instances in the AWS console.
Exploration of individual Step Function instances to track progress and success.
Encouragement to check out additional videos on Step Functions and related AWS services.
Transcripts
hello everyone and welcome back to AWS
simplified in my previous videos I've
shown you how to integrate lambdas into
step functions as part of event
orchestration but in this video I want
to show you how to start a step function
instance from a lambda so let's jump
right into it so here we are in the step
function console so I've actually
already created a step function instance
and it's called step function from
lambda and I predefined some ASL also
some step function code that does a
fairly trivial task so let's just take a
look at this really quick so it takes an
input and it's going to be communicating
with the sqs send message API and here
is the URL to the Q that I've created in
advance of this video and the body that
it's going to be posting to this sq bus
it's going to be a JSON object that has
a transaction ID and the value is going
to be coming from the input of the step
function and the type same deal so
there's a type key and the value is
going to be coming from the input so
that's to say our step function input is
going to contain a transaction ID which
is a string and a type which is also a
string okay so it's a very
straightforward application here so this
all looks well and good I'm going to
copy the AR n because we're gonna need
it when we create our lambda function
and actually try to start an instance of
the step function so this is all fine
here now let's go over to lambda and
actually create our lambda function
that's going to do this okay going to
click on create function now we're going
to be doing this from scratch get a name
my function names step function invoker
and we're going to be doing this in
Python 3.6 3.7 rather and then this is
actually quite important the permission
section so let's scroll down here so I'm
going to be using on an existing role
that actually has the step function
access associated with it however if you
are doing this from scratch you need to
go out and create a role that has the
permission that you need I'm using full
access but I encourage you to use lease
privilege models so only use the
permissions that you need for this
exercise so you actually need the
functions start execution policy and you
need to associate that with a role so
I've already done that here except that
I'm using the full access policy so I
get all those permissions for free so
after that's done you can go ahead and
click on create a function in the bottom
right here you can sometimes take a bit
that was pretty quick that's great okay
so that was created successfully so
let's scroll down here so now we can
actually write our code to interact with
step functions so you can do this in the
editor if you choose I'm gonna do it in
sublime just because I'm a little bit
more comfortable there so let's switch
over to sublime the first step I want to
do here is just paste in the AR n that
corresponds with the step function that
we created in the previous step and
we're gonna need that in a moment or so
when we interact with the client library
that needs an AR n that's an input and
then let's just take whatever we have
here as a templates I get rid of the
stuff we don't need
alright perfect okay so let's get to it
so we're gonna need the JSON library so
that's so let's leave that as it is I'm
gonna need the moto3 library and we're
also going to need the UUID library so
we're gonna be using a function from
this library that generates a random
hash of values and this will make sense
in a moment it's actually the first step
here so I'll probably have to get into
it right away and then the other thing
that we need is an instance of the boat
of three step function client so we need
to say client is equal to boat o 3 dot
clients then you say step functions so
now we have a reference to that client
ok so the first thing that we need is
the transaction ID but before we do that
let's just recall what the input to this
step function is going to look like so
the input is going to be a JSON that
looks like this so we're gonna have
transaction ID and then something excuse
me there we go foo and then we have type
and that can be either purchase or
refund we're just gonna hard code this
is purchase here just in this example so
just a reminder that's what our input is
going to look like ok so the first thing
we need to do is actually generate a
value that corresponds with the
transaction ID and this can be anything
you want to be a random string of values
it could be something meaningful like a
customer ID plus a date
it could be absolutely anything the only
condition though is that this value
whatever you're using as the key for the
step function instance needs to be
unique you cannot have two step function
instances that correspond with the same
key value now since we're using
transaction ID is the kind of identifier
in this example it makes sense to have a
unique value here so let's actually do
that right now through the UUID library
so let's say our transaction ID and
we're going to store this in a variable
is equal to uu ID dot u u ID 1 and what
this thing will generate is just a
random string of characters separated by
dashes so it'll look like like something
like this and many many more of these
variables I think it's limited to 4
delimiter so probably 30 or so
characters so that's what it's gonna
look like so we need to actually convert
this to string since UUID 1 actually
returns an object so now we have a
string instance of that so this is going
to be the value that we populate into
the body of the step function it's also
going to be the name field or the
identifier of the step function instance
as well okay so let's proceed to the
next step and actually define our input
object so let's say our input is equal
to a json and it's gonna look similar to
what i have above here on line 10 and
let's say the transaction ID is equal to
what we just defined so that's going to
be that UUID and we're gonna say the
type surrounded in quotes of course and
we're gonna say purchase okay so now we
have a JSON sorry dictionary object in
Python that corresponds to this object
here that we're looking at okay so next
we need to actually call the step
function API it's called the start
execution API and we can store the
result in a response object if we want
but we can say response is equal to
client dot start
underscore execution and we have a
couple things that we need to pass in
here okay so the first one that's
absolutely mandatory state machine AR N
and you can see now why I copied this
from the previous step because we need
to paste this in here so let's do that
all right now based on
make sure you put a comma at the end
here let's just Center this guy all
right cool
and the second field that you need is
the name so this is the unique
identifier of this step function
instance so it's going to be transaction
ID which is the same value that we're
populating in the JSON payload and the
third value is the actual JSON payload
that you want to pass in and this can be
used throughout your step function for
orchestration or to identify certain key
values to determine what you want to do
in this step function so for our
purposes is this JSON object for yours
it can be anything you like so let's say
input is equal to a now there's a key
step here recall that this is a Python
dictionary object this API requires a
string so we need to string Phi and
convert this dictionary object into a
JSON and so the way we do that is just
use the JSON library JSON dump s and
pass in that object the input object was
which corresponds to the dictionary
object we created previously okay so
everything looks good here let's copy
this out you can go back to our staff
function he's not in click on save in
the top right in order to test it out we
need to create a test event you can put
whatever you want in the event name
leave this as default we're not reading
off the input so it doesn't matter if
you were reading off the input in your
lab well you may want to change this but
we don't care so we don't need it
clicking on create in the bottom right
okay and let's click on test now to see
if this worked okay you can see that
it's exceeded a whole bunch of jargony
we don't care about let's click this a
few times test plus whatever how many
times you want okay so it looks like
it's succeeding from the logs here let's
head over to step functions and actually
verify that there's multiple instances
so we can see eight instances have been
successful if we actually click into
here now now we can look at the separate
instances that correspond to each click
of the button on that lambda function so
these values all kind of our first
glance may look at the same but you can
see here ad 580 0 so these are all
unique values that have all their own
unique key so if we click on one of
these we can see what the step function
was doing and its progress so you can
see through the green box here that it
was successful at this step and you can
see here
you got a 200 okay when we were
communicating with the SQS sendmessage
api sofa lee found this video useful I
have quite a few videos on step
functions and set function orchestration
including dynamo and SQS and us and us
how they suggest to check those out I'll
put a playlist in the pop up that's
coming up in a second thanks so much
guys and I'll see you next time
Browse More Related Video
![](https://i.ytimg.com/vi/wbsbXfkv47A/hq720.jpg)
How to Run a Python Docker Image on AWS Lambda
![](https://i.ytimg.com/vi/rzdSuiU_TQc/hq720.jpg)
Implementing Infrastructure as Code with Terraform | AWS Cloud Resume Challenge - Part 6
![](https://i.ytimg.com/vi/s0XFX3WHg0w/hq720.jpg)
AWS Step Functions with Lambda Tutorial | Step by Step Guide
![](https://i.ytimg.com/vi/979XewnHJmc/hq720.jpg?v=61405986)
Evaluating Functions with Value of X as Fraction and Radical | General Mathematics
![](https://i.ytimg.com/vi/HFfXvfFe9F8/hq720.jpg)
End To End Youtube Video Transcribe Summarizer LLM App With Google Gemini Pro
![](https://i.ytimg.com/vi/qXDhD4y_GtQ/hq720.jpg)
Intégrales - partie 1 : l'intégrale de Riemann
5.0 / 5 (0 votes)