AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the compute resources for you. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications, and only pay for what you use.

Let's break down its components and the reasoning behind its invention.

AWS Lambda Explained:

1. Serverless: This doesn't mean there are no servers involved; rather, it means you don't have to provision, maintain, or administer the servers yourself. AWS takes care of all the infrastructure, scaling, patching, and administration.

2. Event-driven: Lambda is designed to use events like changes to data in an Amazon S3 bucket or an update to a DynamoDB table to trigger the execution of code. There are numerous AWS and non-AWS services that can act as event sources for Lambda.

3. Automatic Scaling: Lambda automatically scales your application by running code in response to each trigger. Your trigger can be an uploaded image, a new log file, a new row in a database, etc. Each trigger is handled as a separate instance of the function.

4. Stateless: Each function execution is independent. AWS Lambda assumes no retention of state between function executions. If you want to maintain state, you'd typically use another service like Amazon S3, RDS, or DynamoDB.

5. Short-lived: Lambda functions are designed to be short-lived. Currently, the maximum execution duration per request is set at 15 minutes.

6. Built-in fault tolerance: AWS Lambda maintains compute capacity and infrastructure reliability, including monitoring, logging via Amazon CloudWatch, and automatic retries.

Why Was AWS Lambda Invented?

1. Simplify Infrastructure Management: Before serverless and Lambda, developers had to set up, configure, and maintain servers to run their applications, which could be time-consuming and distract from the actual application development. With Lambda, developers can focus purely on their code.

2. Cost-effective: With AWS Lambda, you only pay for the compute time that you consume. There's no charge when your code isn't running. This can be a cost-saving model for applications with sporadic traffic patterns.

3. Scalability: Traditional scaling involves predicting traffic and provisioning servers accordingly, which could lead to over-provisioning (wasting money) or under-provisioning (poor user experience). Lambda automatically scales by running code in response to each trigger, ensuring that the application scales with the size of the workload.

4. Rapid Development and Deployment: Without the need to worry about the underlying infrastructure, developers can rapidly develop, test, and deploy applications or functionalities.

5. Enable Event-driven Architectures: AWS Lambda facilitates architectures where certain events automatically trigger code execution, enabling real-time data processing, and reactivity.

6. Integration with AWS Ecosystem: AWS Lambda is deeply integrated with other AWS services, making it easy to set up data pipelines, workflows, and other integrations.

In essence, AWS Lambda was invented to abstract away the complexities of infrastructure management and allow developers to focus solely on their code, making application development faster, more efficient, and cost-effective. It's part of the broader move towards serverless computing, where the emphasis is on building functionality without the overhead of infrastructure management.

How it works?

AWS Lambda operates on an event-driven paradigm. Here's a step-by-step explanation of how it works:

1. Event Source: An event source publishes events to the AWS Lambda function. This could be a variety of AWS services, such as:

   • An S3 bucket when a new file gets uploaded.

     

   • A DynamoDB stream when a new record is added.

   • An API Gateway endpoint when it receives a request.

     

   • A CloudWatch alarm or custom event.

   • ...and many more.

2. Function Invocation: Once the event source publishes an event, AWS Lambda executes (or "invokes") the function asynchronously or synchronously based on the type of event source.

   • Asynchronous invocation: After the event source pushes the event, Lambda queues it for processing and immediately returns a 202 status (Accepted). The function execution proceeds in the background. If the function execution fails, Lambda attempts to retry.

   • Synchronous invocation: The event source waits for the response and Lambda executes the function immediately. Examples include API Gateway, where a client is waiting for a response.

3. Environment Setup: When a function is invoked, AWS Lambda launches an "execution role" that contains the function code, its dependencies, and a runtime. If there is a "warm" container available (from a recent function execution), Lambda might reuse it, which reduces the initialization time (commonly referred to as "cold start" vs. "warm start").

4. Handler Function: Inside the Lambda function code, there is a handler function that AWS Lambda uses as the entry point to execute the function logic. The handler processes the incoming event and can interact with other AWS services or resources as needed.

5. Statelessness: AWS Lambda functions are stateless by design. Each function execution is independent and should not assume any retention of state between executions. If you need to maintain state, you must use an external service like Amazon S3, RDS, or DynamoDB.

6. Scaling: Lambda functions scale automatically by running code in parallel. Each event triggers an independent invocation of the Lambda function. If 1000 events are triggered, AWS Lambda can run 1000 function instances in parallel.

7. Resource Specification: When you create or update a Lambda function, you can specify the amount of memory allocated to it. AWS Lambda allocates CPU power linearly in proportion to the amount of memory configured.

8. Short-lived Execution: Each function execution can run for up to 15 minutes. After that, AWS Lambda terminates the execution.

9. Logging and Monitoring: Execution logs can be forwarded to Amazon CloudWatch Logs, allowing you to debug or monitor your Lambda function's invocation and execution. Additionally, metrics related to the function's execution are automatically reported to Amazon CloudWatch.

10. Security: AWS Lambda maintains the compute fleet that offers a balance of memory, CPU, network, and other resources. This is done on behalf of the user, in Amazon VPC, and maintains the infrastructure, including OS and server maintenance, patch deployment, and more.

AWS Lambda abstracts away the infrastructure management tasks and allows you to focus purely on your code. It receives events, processes them in stateless, auto-scaling, and short-lived functions, and seamlessly integrates with other AWS services to create powerful and scalable applications without the overhead of server management.

Handler Function

In AWS Lambda, the handler function acts as the entry point for the execution of your Lambda function. It is the method that's called when your function is invoked. Let's dive deeper into the Lambda handler function:

1. Definition:

The handler function receives two arguments:

def handler_name(event, context): 
    ...
    return some_value

• event: This contains information about the triggering event. Its structure varies based on the Lambda trigger; for instance, an S3 event would have details about the bucket and the object that changed, while an API Gateway trigger would provide details about the HTTP request.

• context: This provides methods and properties that provide information about the invocation, function, and execution environment.

2. Syntax:

In your Lambda function code, the handler function is typically defined as:

def handler_name(event, context):
    # Your code here
    return some_value

The name of the handler function can be anything you like; however, you need to specify this name in the AWS Lambda configuration so the service knows which function to call.

3. Configuration:

When you create or update a Lambda function, you specify the handler as a string in the format:

file_name.handler_function_name

For instance, if your file is named my_function.py and your handler function is named lambda_handler, then the handler string you'd provide to AWS Lambda would be my_function.lambda_handler.

4. Return Values:

The handler's return value type depends on the integration of the Lambda function:

• For Lambda functions triggered by an API Gateway proxy integration, you should return an appropriately formatted response for the API Gateway.

• For other event sources, AWS Lambda expects the handler to return nothing.

5. Error Handling:

If the handler encounters an error, you can use standard logging to log the error, and AWS Lambda writes it to CloudWatch Logs. If you'd like to notify about an error in your function's logic, you can set up a Dead Letter Queue (DLQ) or use AWS Lambda Destinations for more advanced scenarios.

6. Examples:

• For an S3 trigger:

def lambda_handler(event, context):
    # Print the event to get details about the S3 object that triggered this
    print(event)
    # ... Your processing logic here ...
    return "Successfully processed S3 event"

• For an API Gateway trigger:

def lambda_handler(event, context):
    # Extracting query parameters from the event
    query_string = event['queryStringParameters']

    # ... Your processing logic here ...

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!'),
        'headers': {
            'Content-Type': 'application/json'
        }
    }

The Lambda handler function is the core of your Lambda function's logic, processing incoming events, and determining the behavior of the function in response to triggers. Its implementation will depend heavily on the specifics of the event source and the purpose of the Lambda function.

Tutorial: Run Hello World AWS Lambda function via AWS CLI

Here's a tutorial on how to create and run a serverless "Hello World" function using AWS Lambda via the AWS Command-Line Interface (CLI). This tutorial assumes you already have the AWS CLI installed and configured with the necessary access rights.

Code files are available at https://github.com/Brain2life/aws-cookbook

1. Create a Lambda Function

Step 1.1: First, let's write a simple Lambda function:

# hello_world.py
def lambda_handler(event, context):
    return {
        'statusCode': 200,
        'body': 'Hello, World!'
    }

Save the above code to a file named hello_world.py.

Step 1.2: Package the function into a zip file:

zip hello_world.zip hello_world.py

2. Create an IAM Role for Lambda

AWS Lambda needs permissions to run. For simplicity, we'll give it basic Lambda execution rights.

Step 2.1: Create a trust policy saved as trust-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Step 2.2: Create a new role using the trust policy:

aws iam create-role --role-name HelloWorldLambdaRole --assume-role-policy-document file://trust-policy.json

Step 2.3: Attach the basic execution role policy to the role:

aws iam attach-role-policy --role-name HelloWorldLambdaRole --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

3. Create the Lambda Function

Use the AWS CLI to create a new Lambda function:

aws lambda create-function \
--function-name HelloWorldFunction \
--zip-file fileb://hello_world.zip \
--handler hello_world.lambda_handler \
--runtime python3.9 \
--role <ARN-OF-THE-ROLE>

Replace <ARN-OF-THE-ROLE> with the ARN of the HelloWorldLambdaRole role you created in step 2.

4. Test the Lambda Function

Step 4.1: Create an event file, event.json, to simulate an input event for the Lambda function:

{
  "key1": "value1",
  "key2": "value2",
  "key3": "value3"
}

Step 4.2: Invoke the Lambda function using the AWS CLI:

aws lambda invoke \
    --function-name HelloWorldFunction \
    --payload file://event.json \
    --cli-binary-format raw-in-base64-out \
    output.txt

Step 4.3: View the response in output.txt:

cat output.txt

You should see the Hello, World! response.

5. Cleanup

Once you're done, it's good practice to remove the resources you created to avoid incurring unnecessary charges:

aws lambda delete-function --function-name HelloWorldFunction
aws iam detach-role-policy --role-name HelloWorldLambdaRole --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
aws iam delete-role --role-name HelloWorldLambdaRole

And there you have it! You've just created, deployed, and tested a serverless "Hello World" function using AWS Lambda via the AWS CLI.

References:

1. AWS Lambda

2. Run a Serverless "Hello, World!" with AWS Lambda

3. AWS Lambda Docs

4. How is the return object in a Python-AWS lambda used?

5. AWS Lambda Pricing

6. Lambda execution role