|
| 1 | +title: How to Monitor Python Functions on AWS Lambda with Sentry |
| 2 | +slug: monitor-python-functions-aws-lambda-sentry |
| 3 | +meta: Learn how to monitor your Python 3 functions on AWS Lambda using Sentry. |
| 4 | +category: post |
| 5 | +date: 2021-04-22 |
| 6 | +modified: 2021-04-23 |
| 7 | +newsletter: False |
| 8 | +headerimage: /img/headers/python-lambda-sentry.jpg |
| 9 | +headeralt: The Python, AWS Lambda and Sentry logos are copyright their respective owners. |
| 10 | + |
| 11 | + |
| 12 | +[Amazon Web Services (AWS) Lambda](/aws-lambda.html) is a usage-based |
| 13 | +compute service that can run [Python 3](/why-use-python.html) code. Errors |
| 14 | +can happen in any environment you are running your application in, so |
| 15 | +it is necessary to have reliable [monitoring](/monitoring.html) in place |
| 16 | +to have visibility when a problem occurs. |
| 17 | + |
| 18 | +In this post we will install and configure |
| 19 | +[Sentry](https://sentry.io/welcome/)'s application monitoring |
| 20 | +service that works specifically for code running on AWS Lambda. |
| 21 | + |
| 22 | + |
| 23 | +## Application Dependencies |
| 24 | +A local [development environment](/development-environments.html) is not |
| 25 | +required to follow this tutorial because all of the coding and configuration |
| 26 | +can happen in a web browser through the |
| 27 | +[AWS Console](https://console.aws.amazon.com/console/). |
| 28 | + |
| 29 | +The example code can be copy and pasted from this blog post or you |
| 30 | +can access it on GitHub under the |
| 31 | +[Full Stack Python blog-post-examples](https://github.com/fullstackpython/blog-code-examples) |
| 32 | +repository within the |
| 33 | +[monitor-python-aws-lambda-sentry directory](https://github.com/fullstackpython/blog-code-examples/tree/master/monitor-python-aws-lambda-sentry). |
| 34 | + |
| 35 | + |
| 36 | +## Accessing the AWS Lambda Service |
| 37 | +[Sign into your existing AWS account](https://aws.amazon.com/console) |
| 38 | +or sign up for a [new account](https://aws.amazon.com/). Lambda |
| 39 | +gives you the first 1 million requests for free so that you can execute |
| 40 | +basic applications without no or low cost. |
| 41 | + |
| 42 | +<img src="/img/210406-python-sentry-aws-lambda/aws-lambda-landing.jpg" width="100%" class="shot rnd outl" alt="The AWS Lambda landing page."> |
| 43 | + |
| 44 | +When you log into your account, use the search box to enter |
| 45 | +"lambda" and select "Lambda" when it appears to get to the right |
| 46 | +page. |
| 47 | + |
| 48 | +<img src="/img/210406-python-sentry-aws-lambda/lambda-search-bar.png" width="100%" class="shot rnd outl" alt="Use the search bar to find AWS Lambda."> |
| 49 | + |
| 50 | +If you have already used Lambda before, you will see your existing Lambda |
| 51 | +functions in a searchable table. We're going to create a new function so |
| 52 | +click the "Create function" button. |
| 53 | + |
| 54 | +<img src="/img/210406-python-sentry-aws-lambda/create-function.png" width="100%" class="shot rnd outl" alt="Click the create function button."> |
| 55 | + |
| 56 | +The create function page will give you several options for starting a new |
| 57 | +Lambda function. |
| 58 | + |
| 59 | +<img src="/img/210406-python-sentry-aws-lambda/create-function-detail.png" width="100%" class="shot rnd outl" alt="The create function details page."> |
| 60 | + |
| 61 | +Click the "Browse Serverless App Repository" selection box, then choose |
| 62 | +the "hello-world-python3" starter app from within the |
| 63 | +"Public applications" section. |
| 64 | + |
| 65 | +<img src="/img/210406-python-sentry-aws-lambda/create-function-detail.png" width="100%" class="shot rnd outl" alt="The create function details page."> |
| 66 | + |
| 67 | +The hello-world-python3 starter app details page should look something |
| 68 | +like the following screen: |
| 69 | + |
| 70 | +<img src="/img/210406-python-sentry-aws-lambda/hello-world-python3.png" width="100%" class="shot rnd outl" alt="Hello world Python3 example app and Lambda function."> |
| 71 | + |
| 72 | +Fill in some example text such as "test" under `IdentityNameParameter` |
| 73 | +and click the "Deploy" button: |
| 74 | + |
| 75 | +<img src="/img/210406-python-sentry-aws-lambda/deploy-starter-app.png" width="100%" class="shot rnd outl" alt="Click the deploy button to use the starter app."> |
| 76 | + |
| 77 | +The function will now be deployed. As soon as it is ready we can |
| 78 | +customize it and test it out before adding Sentry to capture any errors |
| 79 | +that occur during execution. |
| 80 | + |
| 81 | + |
| 82 | +## Testing the starter Python app |
| 83 | +Go back to the Lambda functions main page and select your new deployed |
| 84 | +starter app from the list. |
| 85 | + |
| 86 | +<img src="/img/210406-python-sentry-aws-lambda/functions-list.jpg" width="100%" class="shot rnd outl" alt="List of AWS Lambda functions you have created."> |
| 87 | + |
| 88 | +Find the orange "Test" button with a down arrow next to it like you |
| 89 | +see in the image below, and then click the down arrow. Select |
| 90 | +"Configure Test Event". |
| 91 | + |
| 92 | +<img src="/img/210406-python-sentry-aws-lambda/configure-test.jpg" width="100%" class="shot rnd outl" alt="Configure the test event."> |
| 93 | + |
| 94 | +Fill in the Event name as "FirstTest" or something similar, then |
| 95 | +press the "Create" button at the bottom of the modal window. |
| 96 | + |
| 97 | +Click the "Test" button and it will run the Lambda function with |
| 98 | +the parameters from that new test event. You should see something |
| 99 | +like the following output: |
| 100 | + |
| 101 | +```python |
| 102 | +Response |
| 103 | +"value1" |
| 104 | + |
| 105 | +Function Logs |
| 106 | +START RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Version: $LATEST |
| 107 | +value1 = value1 |
| 108 | +value2 = value2 |
| 109 | +value3 = value3 |
| 110 | +END RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 |
| 111 | +REPORT RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Duration: 0.30 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB Init Duration: 1.34 ms |
| 112 | + |
| 113 | +Request ID |
| 114 | +62fa2f25-669c-47b7-b4e7-47353b0bd914 |
| 115 | +``` |
| 116 | + |
| 117 | +That means the test case was successful, but what happens even if there |
| 118 | +is a straightforward mistake in the code, such as trying to access an |
| 119 | +undeclared variable? |
| 120 | + |
| 121 | +Go into the code editor and you should see the starter code like this: |
| 122 | + |
| 123 | +<img src="/img/210406-python-sentry-aws-lambda/lambda-code-editor.jpg" width="100%" class="shot rnd outl" alt="Code editor within AWS Lambda."> |
| 124 | + |
| 125 | +Update the code with the new highlighted line, which tries to access |
| 126 | +a fourth variable, which does not exist in the test configuration |
| 127 | +we try to run it with. |
| 128 | + |
| 129 | +```python |
| 130 | +import json |
| 131 | + |
| 132 | +print('Loading function') |
| 133 | + |
| 134 | + |
| 135 | +def lambda_handler(event, context): |
| 136 | + #print("Received event: " + json.dumps(event, indent=2)) |
| 137 | + print("value1 = " + event['key1']) |
| 138 | + print("value2 = " + event['key2']) |
| 139 | + print("value3 = " + event['key3']) |
| 140 | +~~ print("value4 = " + event['key4']) |
| 141 | + return event['key1'] # Echo back the first key value |
| 142 | + #raise Exception('Something went wrong') |
| 143 | +``` |
| 144 | + |
| 145 | +After adding that one new line of code, hit the "Deploy" button, |
| 146 | +then the "Test" button. You should see some error output: |
| 147 | + |
| 148 | +``` |
| 149 | +Response |
| 150 | +{ |
| 151 | + "errorMessage": "'key4'", |
| 152 | + "errorType": "KeyError", |
| 153 | + "stackTrace": [ |
| 154 | + [ |
| 155 | + "/var/task/lambda_function.py", |
| 156 | + 11, |
| 157 | + "lambda_handler", |
| 158 | + "print(\"value4 = \" + event['key4'])" |
| 159 | + ] |
| 160 | + ] |
| 161 | +} |
| 162 | +
|
| 163 | +Function Logs |
| 164 | +START RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb Version: $LATEST |
| 165 | +value1 = value1 |
| 166 | +value2 = value2 |
| 167 | +value3 = value3 |
| 168 | +'key4': KeyError |
| 169 | +Traceback (most recent call last): |
| 170 | + File "/var/task/lambda_function.py", line 11, in lambda_handler |
| 171 | + print("value4 = " + event['key4']) |
| 172 | +KeyError: 'key4' |
| 173 | +
|
| 174 | +END RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb |
| 175 | +REPORT RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb Duration: 0.81 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB Init Duration: 1.61 ms |
| 176 | +
|
| 177 | +Request ID |
| 178 | +a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb |
| 179 | +``` |
| 180 | + |
| 181 | +It is obvious when we are working in the Console that an error just |
| 182 | +occurred. However, in most cases an error will happen sporadically |
| 183 | +which is why we need a monitoring system in place to catch and report |
| 184 | +on those exceptions. |
| 185 | + |
| 186 | + |
| 187 | +## AWS Lambda function monitoring with Sentry |
| 188 | +The easiest way to add Sentry to Lambda for this application |
| 189 | +is to configure an |
| 190 | +[AWS Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) |
| 191 | +with the necessary dependency for Sentry. Sentry has concise |
| 192 | +[documentation on addin gvia Lambda Layers](https://docs.sentry.io/platforms/python/guides/aws-lambda/layer/) |
| 193 | +so we will walk through that way to configure it and test it |
| 194 | +out. |
| 195 | + |
| 196 | +First, scroll down to the "Layers" section while in your Lambda |
| 197 | +function configuration. Click the "Add a layer" button": |
| 198 | + |
| 199 | +<img src="/img/210406-python-sentry-aws-lambda/add-lambda-layer.png" width="100%" class="shot rnd outl" alt="Add Lambda layer."> |
| 200 | + |
| 201 | +In the "Add layer" screen, select the "Specify an ARN" option. |
| 202 | + |
| 203 | +<img src="/img/210406-python-sentry-aws-lambda/add-layer-specify-arn.jpg" width="100%" class="shot rnd outl" alt="Select Specify ARN in the Add Layer screen."> |
| 204 | + |
| 205 | +Now to specify the Amazon Resource Name (ARN), we need to use |
| 206 | +the Sentry documentation to get the right configuration string. |
| 207 | + |
| 208 | +US-East-1 is the oldest and most commonly-used region so I'll |
| 209 | +use that here in this tutorial but you should check which one |
| 210 | +you are in if you are not certain. |
| 211 | + |
| 212 | +<img src="/img/210406-python-sentry-aws-lambda/arn-region.png" width="100%" class="shot rnd outl" alt="Select the AWS for the ARN string."> |
| 213 | + |
| 214 | +Copy that value into the Lambda Layer configuration, like this: |
| 215 | + |
| 216 | +<img src="/img/210406-python-sentry-aws-lambda/layer-with-arn.png" width="100%" class="shot rnd outl" alt="Select the AWS for the ARN string."> |
| 217 | + |
| 218 | +Then press the "Add" button. Now you have the Sentry dependency |
| 219 | +in your environment so code that relies upon that library can be |
| 220 | +used in the Lambda function. |
| 221 | + |
| 222 | +Next we need to go into the Sentry dashboard to create a project, |
| 223 | +get our unique identifer, and connect it to our Lambda function. |
| 224 | + |
| 225 | +Sentry can be [self-hosted](https://github.com/getsentry/onpremise) or |
| 226 | +used as a cloud service through [Sentry.io](https://sentry.io). We will |
| 227 | +use the cloud hosted version because it is quicker than |
| 228 | +setting up your own server as well as free for smaller projects. |
| 229 | + |
| 230 | +Go to [Sentry.io's homepage](https://sentry.io). |
| 231 | + |
| 232 | +<img src="/img/210406-python-sentry-aws-lambda/sentry-homepage.jpg" width="100%" class="shot rnd outl" alt="Sentry.io homepage where you can sign up for a free account."> |
| 233 | + |
| 234 | +Sign into your account or sign up for a new free account. You will be at |
| 235 | +the main account dashboard after logging in or completing the Sentry sign |
| 236 | +up process. |
| 237 | + |
| 238 | +There are no errors logged on our account dashboard yet, which is as |
| 239 | +expected because we have not yet connected our account to our Lambda |
| 240 | +function. |
| 241 | + |
| 242 | +Click "Projects" on the left navigation bar, then "Create Project" |
| 243 | +in the top right corner. |
| 244 | + |
| 245 | +Under "Choose a Platform", select "Serverless" and then "AWS Lambda (Python)" |
| 246 | +as shown below: |
| 247 | + |
| 248 | +<img src="/img/210406-python-sentry-aws-lambda/aws-lambda-python.jpg" width="100%" class="shot rnd outl" alt="Choose AWS Lambda (Python) under the platform options."> |
| 249 | + |
| 250 | +Decide under what criteria it should send error information out of |
| 251 | +Lambda. For this tutorial, we will have it send every exception. |
| 252 | +Then click the "Create Project." button. |
| 253 | + |
| 254 | +You can have Sentry handle the instrumentation automatically but |
| 255 | +we will handle it manually for our function. On the next screen, Sentry |
| 256 | +will provide you with your unique DSN string, which we will need for |
| 257 | +our function. |
| 258 | + |
| 259 | +<img src="/img/210406-python-sentry-aws-lambda/sentry-dsn-string.jpg" width="100%" class="shot rnd outl" alt="Copy the Sentry DSN string so we can export it as an environment variable."> |
| 260 | + |
| 261 | +Typically you will want to |
| 262 | +[use environment variables on AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html) |
| 263 | +to store and access values like your Sentry key. |
| 264 | + |
| 265 | +Copy the contents of the Sentry DSN string, and go into the Lambda console |
| 266 | +to create a new environment variable. To do that, click the "Configuration" |
| 267 | +tab within Lambda like you see here: |
| 268 | + |
| 269 | +<img src="/img/210406-python-sentry-aws-lambda/aws-lambda-configuration.jpg" width="100%" class="shot rnd outl" alt="Click the Lambda Configuration tab."> |
| 270 | + |
| 271 | +Then click "Edit" and add a new environment variable with the key of `SENTRY_DSN` |
| 272 | +and the value of the DSN string that you copied from the Sentry screen. |
| 273 | + |
| 274 | +<img src="/img/210406-python-sentry-aws-lambda/add-env-var.jpg" width="100%" class="shot rnd outl" alt="Add the environment variable in AWS Lambda."> |
| 275 | + |
| 276 | +Click the "Save" button and go back to your Lambda function code. |
| 277 | + |
| 278 | +Update your Lambda function with the following highlighted new lines of code |
| 279 | +to send errors to Sentry. |
| 280 | + |
| 281 | +```python |
| 282 | +import json |
| 283 | +~~import os |
| 284 | +~~import sentry_sdk |
| 285 | +~~from sentry_sdk.integrations.aws_lambda import AwsLambdaIntegration |
| 286 | + |
| 287 | +~~SENTRY_DSN = os.environ.get('SENTRY_DSN') |
| 288 | +~~sentry_sdk.init( |
| 289 | +~~ dsn=SENTRY_DSN, |
| 290 | +~~ integrations=[AwsLambdaIntegration()] |
| 291 | +~~) |
| 292 | + |
| 293 | +print('Loading function') |
| 294 | + |
| 295 | + |
| 296 | +def lambda_handler(event, context): |
| 297 | + #print("Received event: " + json.dumps(event, indent=2)) |
| 298 | + print("value1 = " + event['key1']) |
| 299 | + print("value2 = " + event['key2']) |
| 300 | + print("value3 = " + event['key3']) |
| 301 | + print("value4 = " + event['key4']) |
| 302 | + return event['key1'] # Echo back the first key value |
| 303 | + #raise Exception('Something went wrong') |
| 304 | +``` |
| 305 | + |
| 306 | +Click the "Deploy" button and then "Test". The code will throw |
| 307 | +an error and when we go back to our Sentry dashboard we will |
| 308 | +see it captured and viewable for further inspection. |
| 309 | + |
| 310 | +<img src="/img/210406-python-sentry-aws-lambda/sentry-error-dashboard.jpg" width="100%" class="shot rnd outl" alt="AWS Lambda exception in the Sentry dashboard."> |
| 311 | + |
| 312 | +It works! Next you will likely want to tune your exception reporting |
| 313 | +criteria to make sure you get alerted to the right number of exceptions |
| 314 | +if you do not want to see all of them. |
| 315 | + |
| 316 | + |
| 317 | +## What's Next? |
| 318 | +We just wrote and executed a Python 3 function on AWS Lambda then |
| 319 | +captured the exception message into the Sentry logs. You can |
| 320 | +now continue building out your Python code knowing that when something |
| 321 | +goes wrong you will have full visibility on what happened. |
| 322 | + |
| 323 | +Check out the [AWS Lambda section](/aws-lambda.html) for |
| 324 | +more tutorials by other developers. |
| 325 | + |
| 326 | +Further questions? Contact me on Twitter |
| 327 | +[@fullstackpython](https://twitter.com/fullstackpython) |
| 328 | +or [@mattmakai](https://twitter.com/mattmakai). I am also on GitHub with |
| 329 | +the username [mattmakai](https://github.com/mattmakai). |
| 330 | + |
| 331 | +Something wrong with this post? Fork |
| 332 | +[this page's source on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/210422-monitor-python-aws-lambda-sentry.markdown) |
| 333 | +and submit a pull request. |
0 commit comments