Azure Service Fabric - Tutorial 17 - Data Packages, Config and Environment Variables

Summary

This video tutorial delves into using data packages, config packages, and environment variables in Service Fabric. It demonstrates how to package and deploy various file types, such as JSON, XML, and CSV, with a Service Fabric deployment. The video also covers registering data packages in the service manifest, accessing data within the service, and using configuration to adapt settings based on deployment scenarios. Additionally, it explains the process of setting environment variables and accessing them within the application code for different types of services in Service Fabric.

Takeaways

• 📦 Data packages in Service Fabric allow for packaging files like JSON, XML, CSV as part of the deployment.
• 📂 To include a data file, create a 'data' folder in the service package route and add the file there.
• 🔄 Change the file's properties to 'Copy to Output Directory if newer' to ensure it's included in the deployment.
• 🌐 Register the data package in the service manifest with a name and version number.
• 🔍 Access the data file in the service by using the `this.ActorService.Context.CodePackageActivationContext.GetDataPackageObject` method.
• 📋 Configuration packages enable different settings based on deployment, using application parameters files.
• 🔧 Add configuration values to the application parameters files and register them in the application manifest.
• 🔄 Use configuration overrides in the service manifest import to set different values for different environments.
• 🎯 Access configuration values in the service code via the `this.ActorService.Context.CodePackageActivationContext.GetConfigurationPackageObject` method.
• 🏗️ Environment variables can be set in the application manifest's service manifest import section for service-specific settings.
• 🔑 Use environment variables in the service by accessing them with `Environment.GetEnvironmentVariable` in C#.
• 🔄 Actor services and stateful/stateless services differ slightly in how they access configuration and data files, but the process is largely the same.

Q & A

What are the three main components of Service Fabric services?

The three main components of Service Fabric services are code, configuration, and data.

How do data packages work in Service Fabric?

Data packages in Service Fabric allow you to package up a file, such as JSON, XML, or CSV, as part of your deployment. They are added to the service and registered in the service manifest with a version number.

How can you add a data file to a Service Fabric service?

To add a data file, you create a 'data' folder in the service package route, add the data file there, set the 'Copy to Output Directory' property to 'Copy if newer', and then register the data package in the service manifest with a name and version.

How do you access data files within a Service Fabric actor service?

In the actor service class, you can access the data file by using the `this.ActorService.Context.CodePackageActivationContext.GetDataPackageObject` method, specifying the name of the data package, and then combining the data package path with the file name to read the contents.

What is the purpose of configuration in Service Fabric services?

Configuration in Service Fabric services allows you to set different values for parameters based on the deployment environment, enabling flexible and adaptable service behavior across different cluster configurations.

How do you set up different configurations for different deployment environments?

You can set up different configurations by adding parameters in the application parameters files for each environment, such as 'Local.OneNode' and 'Local.FiveNode', and then referencing these parameters in the application and service manifests.

How do you override configuration settings for a specific service in Service Fabric?

You override configuration settings by adding a 'Config Overrides' section in the service manifest import for the specific service, defining the settings and parameters that should be used for that service.

What are environment variables in the context of Service Fabric applications?

Environment variables in Service Fabric applications provide a way to configure services with external settings that can be changed without modifying the application code, allowing for灵活的配置管理.

How do you add environment variables to a Service Fabric service?

You add environment variables by defining them in the 'EnvironmentOverrides' section of the service manifest import in the application manifest, and then referencing them in the service manifest under the 'EnvironmentVariables' section for the specific service.

How can you access environment variables within a Service Fabric actor service?

You can access environment variables in an actor service by using the `Environment.GetEnvironmentVariable` method, providing the name of the environment variable as a string.

What is the difference between accessing configuration and data files in actor services versus stateful and stateless services?

The main difference is that in actor services, you access configuration and data files through the `this.ActorService.Context` object, whereas in stateful and stateless services, you use the `this.Context` object from the service's context.

How can you verify that your Service Fabric service is correctly accessing configuration and environment variables?

You can verify access by deploying the service in different configurations, triggering calls to the service's APIs, and observing the output for the configuration and environment variable values, ensuring they match the expected settings for the given environment.

Outlines

00:00

📦 Understanding Data Packages in Service Fabric

This paragraph introduces the concept of data packages in Service Fabric, explaining how they allow for the inclusion of files like JSON, XML, or CSV as part of a service deployment. It details the process of adding a data file to a service, such as creating a 'data' folder within the service package route and placing the data file there. The video also covers how to set the 'Copy to Output Directory' property for the file, register the data package in the service manifest, and access the data within the service using the `this.ActorService.Context.CodePackageActivationContext.GetDataPackageObject` method. The example used is a CSV file named 'test.csv' with sample data, and it demonstrates how to read the contents of the file within the service.

05:01

🔧 Configuring Services with Configuration Packages

The second paragraph delves into the use of configuration packages in Service Fabric, which allow for different configurations based on deployment scenarios. It explains how to add parameters to application parameters files for different environments (e.g., local one-node vs. local five-node) and how to use these parameters within the service. The process involves adding the parameter to the application manifest, creating a config override in the service manifest import, and accessing the parameter value in the service code. The example given involves adding a 'DB config' parameter that can have different values depending on whether the service is deployed in a one-node or five-node cluster.

10:01

🔄 Accessing Configuration Settings at Runtime

This section focuses on accessing configuration settings at runtime within an actor service. It explains how to retrieve configuration settings using the `this.ActorService.Context.CodePackageActivationContext.GetConfigurationPackageObject` method and how to access specific settings and sections. The example demonstrates retrieving a 'DB config' parameter value that was set differently for one-node and five-node configurations. The video also clarifies how to access the same configuration settings in stateful and stateless services compared to actor services.

15:03

🌐 Utilizing Environment Variables in Service Fabric Applications

The final paragraph discusses the use of environment variables in Service Fabric applications. It outlines the process of adding environment variable overrides in the application manifest and service manifest import sections. The example adds an environment variable named 'DB port' with a sample port value. The video then shows how to access this environment variable in the service code using the `Environment.GetEnvironmentVariable` method and emphasizes the importance of using the correct environment variable name and case sensitivity. It concludes with a brief mention of the difference in accessing environment variables between actor, stateful, and stateless services.

Keywords

💡Service Fabric

Service Fabric is a distributed systems platform for building and deploying microservices, which is the main focus of the video. It provides a framework for deploying and managing scalable and reliable services. In the context of the video, Service Fabric is used to package and deploy data, configuration, and environment variables for an application.

💡Data Packages

Data packages in Service Fabric allow developers to include files, such as JSON, XML, or CSV, as part of the service deployment. These packages are essential for including static data that the service might need. In the video, the creator demonstrates how to add a CSV file as a data package to a Service Fabric service.

💡Configuration

Configuration in Service Fabric refers to the settings and parameters that can be changed for a service without requiring a code change. This allows for flexibility in how the service operates based on the deployment environment. The video explains how to use configuration files to set different values for a 'DB config' parameter based on whether the service is deployed in a one-node or a five-node cluster.

💡Environment Variables

Environment variables are values that can be set outside of the application code and can influence the behavior of an application. In Service Fabric, they can be defined in the application manifest and used within the service code to adapt to different deployment scenarios. The video demonstrates adding an environment variable 'DB port' and accessing it within the service code.

💡Actor Service

Actor service is a concept in Service Fabric based on the Actor model, which encapsulates state and behavior within objects called actors. Each actor has a unique identifier and communicates with other actors through messages. The video uses the actor service as an example to demonstrate how to access data, configuration, and environment variables.

💡Service Manifest

The service manifest is an XML file that describes the service in Service Fabric, including its code, configuration, and data packages. It is used to register the service and its components with the fabric. The video explains how to add data packages and configuration parameters to the service manifest.

💡Code Package Activation Context

The code package activation context is an object in Service Fabric that provides access to the runtime environment of a service's code package. It is used to interact with the fabric and retrieve information such as the path to data or configuration packages. The video demonstrates using the activation context to access a data file's contents.

💡Configuration Package Object

The configuration package object in Service Fabric is used to retrieve configuration settings from the service's configuration package. It allows services to access configuration data at runtime, which can be different based on the deployment environment. The video shows how to obtain this object and use it to access a 'DB config' parameter.

💡Environment Overrides

Environment overrides in Service Fabric allow developers to define different settings for different environments, such as development, testing, or production. These overrides are applied to the service's environment variables, allowing for灵活 configuration changes without altering the code. The video explains how to add an environment variable 'DB port' and reference it in the service.

💡Application Parameters

Application parameters in Service Fabric are used to set values for parameters defined in the application manifest. These parameters can be different for each deployment environment and are used to configure the application based on the environment it is running in. The video discusses adding 'DB config' parameters to different environment files like 'Local.OneNode' and 'Local.FiveNode'.

💡Package Root

The package root in Service Fabric refers to the directory where the application package is located. This is important as it is the base directory from which the fabric deploys and runs services. The video mentions the package root in the context of adding data and configuration packages to the service.

Highlights

Introduction to using data packages, config packages, and environment variables in Service Fabric.

Data packages allow for packaging files like JSON, XML, CSV as part of the Service Fabric deployment.

Service Fabric services consist of three main components: code, configuration, and data.

Adding a data file to a service involves creating a 'data' folder in the package route and adding the file there.

To use a data file in a service, it must be registered in the service manifest with a version number.

Accessing data in the service is done through the 'this.ActorService.Context' and 'CodePackageActivationContext' in the actor service class.

Configuration allows for different settings based on deployment, using application parameters files.

Configuration values are added in the application manifest and can be overridden in the service manifest import.

Environment variables can be set in the application manifest for different deployment scenarios.

Environment variables are accessed in the service using 'Environment.ExpandEnvironmentVariables'.

The video provides a step-by-step tutorial on adding and using data packages, config packages, and environment variables in Service Fabric.

Demonstration of deploying a service with a data package and accessing its contents in the 'UnactivatedAsync' method.

Explanation of how to configure different database connection settings for one-node and five-node clusters using config packages.

Illustration of changing cluster configuration and observing the dynamic update in configuration values.

Discussion on the slight difference in accessing configuration and data files between actor, stateful, and stateless services.

The video concludes with a summary and a call to action for viewers to like and subscribe for more content.

Transcripts

hi guys so welcome back to another video

on service fabric in this video we're

gonna start to look at data packages

config packages and environment

variables and how we can use these in

service fabric so the first thing we

look at is data packages and data

packages allow us to package up a file

as part of our service fabric deployment

so this could be a JSON XML CSV file

whatever we want and as we remember all

service fabric services have three main

components they have code which we've

mostly looked at so far configuration

which we'll look at in this video and

data which we'll look at now in order to

add a data file to our service we simply

go to our service and open the package

route we can add a new folder here calls

data and inside this data folder is

where we're gonna add our data files so

we might want to add a file called test

and it's a CSV file so test dot CSV and

inside that file we will just have some

random CSV data or random tape tabular

data which I'll fill out now so we

filled out the test CSV file here with

just a couple of columns so we can see

we've got the title column ID name age

and then four rows of data and this

could be a much much bigger file in a

real system so the first thing we need

to do is need to right-click on the file

here in the solution Explorer and go to

properties and then change copy to April

directiy to copy if newer and then save

so next we can go back to our product

actor service we'll be using the actor

for this tutorial but the concepts are

pretty much identical with stateful

services I'll just show the one

different detail in a few minutes so we

also need to register the data package

we've added in our service manifest for

the service that we are using the data

package in so if we open the service

manifest dot XML for our product actor

service we add the data package right

underneath the config package here so

you want to

a data package we want to call it data

how do we need to give their version

number as well so we'll just give it a

version 1.0.0 and that will allow us to

add this data package into our actor

service and then you access the data in

our actor service we can go back to the

product actor service class which is the

base class for our actor and in any of

the methods we can access the data so

we'll do it in unactivated async for now

we simply need to say this actor service

context and then we want to call code

package activation context and get data

package object and then we give the name

of the data package so data and our

semicolon and then we can save this in

the variable called data package so as

you can see this is a type of system a

fabric data package and then we want to

get the path from the data package and

combine it with the name of our CSV file

so our data path will be path dot

combined to combine both the data

package path and the name of the file so

to do this we just simply type data

package and Darth path and the name of

our file that we added is test csv if

the file was called something different

this is where we change the name of the

file and finally we want to read out of

the contents from this path and use it

in our service so we can add another

variable called contents and we have

that equal to file dot read alt text and

we simply pass it the path so in this

case it's the data path here and this

should give us a string object which

contains the contents of the file we're

looking to use so we'll quickly run our

service deploy it and runners will call

any method on the actor we should

on activate a sink and we'll see what we

get in the contents of our file okay so

our service is deployed now and we're

gonna trigger a call to on activate a

sink using postman so we can see we hit

our API endpoint and we jump into on

activate a sink so we see we have a

reference that data package object here

and to the data path which is C service

fabric dev cluster and what note the

data is stored on and then we're able to

access the contents of the file here so

we can then parse the contents of this

file and use it however we want in our

service so the next thing we'll look at

is configuration so in our service we

might want to have a different

configuration based on our deployment so

we might want to look in our application

parameters file we see that we have a

local one node file and we have a local

5 node file we might want to set some

values different depending on how we're

deployed so to do this we can simply add

a parameter in any of these files in the

application parameters folder so for

instance in the local 5 node XML filing

on add a parameter and it's gonna be

called DB config and we'll simply set

its value to be called 5 node I will do

something very similar in one node we'll

call DB come fake except we'll change

the value to one node so this means we

can use a different value for a DB

config when we're running in a 5 node as

opposed to a one node cluster it's

important that we keep the name the same

so we can access it no matter how we're

deployed so once we've added the

configuration value in our application

parameters files we need to go to the

application package root and the

application manifest and then we need to

add it here at the top in the parameter

section one more time and in this case

we're giving it a default value or you

can leave this blank if we don't want to

have a default value so the name as we

had is called

DB config in this case we won't give it

a default value we'll leave that blank

so the final thing we need to do in this

file is we need to add this new

parameter in a config section in our

service manifest import for whatever

service we want to use the parameter in

we can use the parameter in multiple

services if we want but we have to

ensure we add the correct configuration

for each service so I'll just delete

these comments and split it and a bit so

it's a bit easier to see we can see we

have two service manifest imports we

have one for the API service and one for

the actor service the actor service is

the one that we're gonna add this DB

config config parameter to so we do that

right order the service manifest ref and

we add a new section called config

overwrites inside that we have a config

override which we give a name and

usually the convention is to call this

one config and inside our config we can

set certain settings and we have a

section which we give a default name say

we might call the section database and

inside the section is finally where we

can add our parameter so we'll give it

the same name as above DB config

although this isn't required to have the

exact same name in this instance and we

give it the value so the value we want

to give it needs to be in square

brackets like this and needs to be the

same name as the parameter here so that

means it's reading in the value from the

application parameters files so here

we've created a config override called

config we have a section inside the

settings called database and inside that

section we have a parameter called DB

config and we can have multiple

parameters in a single section and we'll

just change this section name to be

uppercase to be inline with conventions

so the last thing we need to do to

configure our service to use this Sun

fig is we need to go to the service in

question which is our product actor

service we need to go into the package

roof and we need to open config and

settings and here we need to add another

section using the names used here so in

the settings file

we can see we have several sections

already to find we need to add another

section here underneath so we'll have

the section the section needs a name in

this case the name should be the exact

same as the section that we've given it

in the application manifest simply

database and then we can add a parameter

here so the parameter also has a name

and again the name should be the exact

same as the name of the parameter in the

application manifest and the value is

left blank which means it will be read

in again from the application manifest

which in turn is reading it in from the

application parameters files so then we

have to access this value in our code so

again we can jump into the actor service

for simplicity we'll do it in the on

activate async for now we'll just

comment out the data package code so

again we start this off the same way we

call this actor service dot context then

we want to get the code package

activation context and in this case we

want to call get configuration package

object we need to pass this a screen

called a package name remember in our

application manifest we gave the package

name the name config so we put that in

our configuration package object and

then we want to call dot settings and

sections and then we need to pass in our

section name which again in the

application manifest we called database

and finally what parameter we want so

parameters can is a dictionary so we

need to pass it in the key and the key

is the name of our parameter in this

case DB config and then to get the value

we can simply call dot value and we can

save that to a variable called DB config

which is a screen so when we run this

service and deploy it in a one node

configuration we should get the value

one note here if you deploy it in five

no config we should get the value five

node so let's boot up the service in one

node config and see what we get so our

service is deployed now

so let's trigger a call to activate a

sync or any extra method which will

trigger a call to on activate a sync and

see what comes out in this DB config

value so we'll just call one of the API

endpoints again which should proxy to

our actor and we can see we jump into

all activate async and let's continue on

here and see what we get for DB config

as expected we get one node so we'll

change our cluster configuration to be

in the five node configuration just to

see that we get the DB config changed to

five node so we'll just redeploy our

cluster now and we'll jump back in when

we get that set up

okay so we've redeployed on a five node

configuration and we'll hit the API

endpoint again and see what we get for

the value of DB config this time so

we'll hit the API endpoint which will

proxy to our actor service we'll try get

the DB config from the config section

and as expected their value for DB

config in a five node configuration is

five node so the final thing I'm going

to talk about in this video is

environment variables in service fabric

so just to clean everything up we'll

just close all of our documents for now

just so we have a clear workspace and

we'll see how we can add environment

variable in our service fabric

application so we want to again open our

application manifest and the place where

we add environment overrides is in our

service manifest import section in our

application manifest so we can see we

have two service manifest imports we

have one for the API service which by

default actually already has an

environment override and one for hour

after hour in vironment overrides so we

want to add a section here in our second

sir

Manifest import for environment

overrides so we can just copied that in

from our section above and we don't want

to have the environment name asp.net

core environment instead we'll call it

something different to say DB port and a

value of just say a random port like a

peasant and we can see here we have the

environment overrides and we have to

define the code package reference and we

have an equal to code with a lowercase C

this I think relates to the service

manifest so if we look in the service

manifest for the actor service remember

that this is the section for the actor

service we can see that we have a code

package called code with an uppercase so

I think in the application manifest in

our environment override section this

has to be a lowercase version of what we

have in our code package here so

speaking of the service manifest we also

have to add some configuration for our

environment variable here so we can add

environment variables in our service

manifest so we have a section right

under the entry point for environment

variables and then we just want to add

the environment variable that we just

added in our application manifest into

our service and the name has to be the

same so we'll cover that as DB port and

the value can be just blank because we

want to use the value from our

application manifest so the final step

is we want to actually use this

environment variable inside our

application so we can open up our

product actor service again and we'll go

down to the on activate async just

comment at this DB config section and we

want to access our environment variable

which we call DB ports so I set a

variable equal to DB board and we'll

access this just like any order

environment variable that we access in

c-sharp so we'll just say environment

yeah environment variable and we'll give

it the name DB port

just double-check that that is what we

expect with a lowercase B here so we'll

just put that in so we'll run our

service again and we'll just see can we

access this DB port here and remember

this is imported as a string so if it

was in numeric value we'd have to cast

it I'll just deploy and we'll come back

and we'll call a method on our actor to

see if we can get this DV port from our

environment variable okay so our service

has started now so we're just gonna

again hit the API endpoint to trigger

and proxy to our actor service so we've

hit the endpoint little proxy director

service will call the on activate async

and we can see we have accessed the DB

port 8000 so the last quick thing I want

to mention before wrapping up the video

is that this was obviously for actor

services so we called this dot actor

service the only difference between

accessing in config and data files in

actor services and in stateful and

stateless services is that we call this

dot actor service in actor services put

in order services we simply just need to

call this dot context and everything

else should be the exact same so thanks

for watching this video guys if you

enjoyed it please give it a thumbs up

and subscribe to the channel