How to schedule AWS Lambda functions using EventBridge
AWS EventBridge is a serverless event bus service that facilitates building application architectures driven by events. It enables you to set up routes between your source events and targeted AWS services.
Let's break down AWS EventBridge and its integration with AWS Lambda functions.
Components of AWS EventBridge:
Event Buses: A conduit to receive events. You can use the default event bus or create custom ones.
Events: An event is a signal that a system's state has changed. In AWS, these could come from AWS services, integrated SaaS applications, or custom sources.
Rules: Determine how events on an event bus are routed to targets. A rule matches incoming events and routes them to targets for processing.
Targets: An AWS service or resource that acts on matched events. AWS Lambda is one of the supported targets.
How it integrates with AWS Lambda:
Source: Start with an event source. This could be any AWS service like Amazon S3 (e.g., when a new object is added to a bucket) or custom applications sending events to the event bus.
Event Bus: The event from the source is sent to the EventBridge event bus.
Rules: Within EventBridge, you set up rules to determine how to handle and where to send these incoming events. A rule listens for specific events defined by the event pattern or schedule.
Targeting Lambda: If a rule's conditions are met (i.e., an incoming event matches the criteria specified in the rule), EventBridge can trigger one or more target actions. One such action can be invoking an AWS Lambda function. So, you can set AWS Lambda as a target for matched events.
Lambda Execution: Once AWS Lambda is invoked, it runs your function code securely within an execution role, which grants the function permissions to use other AWS services.
Steps to integrate EventBridge with Lambda:
Create a Lambda function: This will contain the logic you want to execute when a specific event occurs.
Define an EventBridge Rule:
Navigate to the EventBridge console.
Create a new rule.
Define the event pattern or schedule that will trigger this rule.
Set Lambda as a Target:
In the targets section of the rule, select "Lambda function".
Choose the Lambda function you created in step 1.
Test the Integration: Send events to the event bus that match your rule's criteria. This should trigger your Lambda function.
Monitor: Use services like Amazon CloudWatch to monitor the Lambda invocations and view logs for debugging and tracking.
Benefits:
Decoupling: EventBridge facilitates decoupling between services, making systems more modular and easier to maintain and scale.
Flexibility: EventBridge can handle events from a vast array of AWS services, integrated SaaS applications, and custom sources, offering a lot of flexibility in terms of event sources.
Scalability: Both AWS Lambda and EventBridge are serverless, so they scale automatically with the number of incoming events.
AWS EventBridge's integration with AWS Lambda provides a powerful mechanism for event-driven architectures, allowing for efficient, real-time processing and reaction to system changes.
Use Cases:
The integration of EventBridge and AWS Lambda is particularly useful for crafting event-driven architectures. Here are several common use cases:
Application Integrations:
Microservices Coordination: When microservices architectures are used, EventBridge can relay events from one service to another. Lambda functions can be triggered to process data, transform it, or trigger another microservice.
SaaS Application Integration: Integrate with third-party SaaS applications like Datadog, Auth0, or Zendesk. When an event occurs in these applications (like a new ticket in Zendesk), a Lambda function could be triggered to process or react to that event.
Operational Tasks:
Automated Backup: Trigger Lambda to initiate backups of databases or other services based on scheduled events.
Cleanup Tasks: Automatically delete unused resources, such as old EBS snapshots or unused EC2 instances based on specific triggers or schedules.
Real-time Analytics and Monitoring:
Log Processing: Ingest logs into EventBridge and use Lambda to process, analyze, and store them in databases or send to monitoring tools.
Real-time Alerts: Trigger Lambda functions to send alerts or notifications when specific operational events occur, like errors or thresholds being reached.
Workflow Automation:
- Order Processing: When a new order is placed in an e-commerce application, an event is sent to EventBridge. A Lambda function could then handle various stages of the order lifecycle, like inventory check, payment processing, or sending notifications.
IoT (Internet of Things):
- Device Data Processing: IoT devices can send data to AWS IoT Core, which then routes the data to EventBridge. Lambda can process this data, for instance, to update device shadows, store readings, or trigger alarms based on device metrics.
Custom Application Events:
- User Sign-up: When a user signs up on a platform, an event can be emitted. Lambda can then process this event, sending a welcome email or populating default user settings.
Security and Compliance:
Automated Remediation: If a non-compliant resource is detected (like an S3 bucket being made public), an event can be sent. A Lambda function could then automatically modify the bucket's permissions to make it private again.
Suspicious Activity Alerts: When suspicious or anomalous activities are detected, Lambda functions can be triggered to send alerts, take protective actions, or integrate with other security tools.
Distributed Tracing and Auditing:
- Audit Trail: Create an audit trail by capturing events related to resource changes, user activities, etc. Lambda can process these events and store them in a structured manner for future reference or analysis.
System Health Monitoring:
- Health Checks: Trigger Lambda functions at regular intervals to perform health checks on services and applications. If anomalies are found, corrective actions can be initiated, or notifications can be sent.
Content and Data Management:
- Content Moderation: When users upload content, an event can be emitted. Lambda can be used to trigger moderation tools or processes to ensure the content adheres to guidelines before it's publicly accessible.
The beauty of the EventBridge and Lambda integration lies in its versatility. The above use cases are just the tip of the iceberg. With custom events and the flexibility to define specific reactions to these events, the possibilities are vast.
Tutorial: Schedule AWS Lambda functions using EventBridge
You can set up a rule to run an AWS Lambda function on a schedule. This tutorial shows how to use the AWS Management Console or the AWS CLI to create the rule.
For schedules, EventBridge doesn't provide second-level precision in schedule expressions. The finest resolution using a cron expression is one minute. Due to the distributed nature of EventBridge and the target services, there can be a delay of several seconds between the time the scheduled rule is triggered and the time the target service runs the target resource.
Step 1: Create a Lambda function
Create a Lambda function to log the scheduled events.
To create a Lambda function
Open the AWS Lambda console at https://console.aws.amazon.com/lambda/.
Choose Create function.
Choose Author from scratch.
Enter a name and description for the Lambda function. For example, name the function
LogScheduledEvent
.Change Runtime to Node.js 14.x and choose Create function.
On the Code tab of the function page, double-click index.js.
Replace the existing code with the following code.
'use strict'; exports.handler = (event, context, callback) => { console.log('LogScheduledEvent'); console.log('Received event:', JSON.stringify(event, null, 2)); callback(null, 'Finished'); };
Choose Deploy.
Step 2: Create a Rule
Create a rule to run the Lambda function you created in step 1 on a schedule.
You can use either the console or the AWS CLI to create the rule. To use the AWS CLI, you first grant the rule permission to invoke your Lambda function. Then you can create the rule and add the Lambda function as a target.
To create a rule (console)
Open the Amazon EventBridge console at https://console.aws.amazon.com/events/.
In the navigation pane, choose Rules.
Choose Create rule.
Enter a name and description for the rule.
A rule can't have the same name as another rule in the same Region and on the same event bus.
For Event bus, choose the event bus that you want to associate with this rule. If you want this rule to match events that come from your account, select AWS default event bus. When an AWS service in your account emits an event, it always goes to your account’s default event bus.
For Rule type, choose Schedule.
For Schedule pattern, choose A schedule that runs at a regular rate, such as every 10 minutes. and enter
5
and choose Minutes from the drop-down list.Choose Next.
For Target types, choose AWS service.
For Select a target, choose Lambda function from the drop-down list.
For Function, select the Lambda function that you created in the Step 1: Create a Lambda function section. In this example, select
LogScheduledEvent
.Choose Next.
Choose Next.
Review the details of the rule and choose Create rule.
Step 3: Verify the rule
Wait at least five minutes after completing step 2, and then you can verify that your Lambda function was invoked.
View the output from your Lambda function
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.
In the navigation pane, choose Logs.
Select the name of the log group for your Lambda function (
/aws/lambda/function-name
).Select the name of the log stream to view the data provided by the function for the instance that you launched.