AWS Log Insights Query with String Contains

Querying AWS Log Insights for Specific Strings

Amazon CloudWatch Log Insights is a powerful tool for analyzing logs generated by AWS services. One common task is filtering log messages to find entries containing a specific string. Various methods can be used, each suited to different needs.

1. Basic Filtering with strcontains

The simplest way to find logs containing a specific word or phrase is the strcontains function. It checks if a specific substring exists in a field, such as @message:

fields @timestamp, @message
| filter strcontains(@message, "Error")
| sort @timestamp desc
| limit 50

This query filters logs for messages containing the word “Error,” showing the most recent 50 results.

2. Advanced Matching with LIKE

The LIKE operator enables pattern matching using regular expressions, making it highly flexible for queries:

fields @timestamp, @message  
| filter @message like /timeout|failure/  
| sort @timestamp desc  
| limit 30 

This example filters messages containing either “timeout” or “failure.” Regular expressions can handle variations in text format or multiple keywords.

3. Regex Matching with =~

Another method is the =~ operator for pattern-based searches:

fields @timestamp, @message
| filter @message =~ /.*Critical Error.*/
| sort @timestamp desc
| limit 25

This finds messages containing the phrase “Critical Error,” regardless of its position in the log.

4. Performance Optimization

To enhance query performance:

• Use indexed fields (like @message, @logStream) in filters.
• Reduce result sizes with limit and refine time ranges to narrow down searches.
• Use the fields command to include only relevant fields, reducing data processing overhead.

5. Combining Filters

You can combine filters for complex conditions:

fields @timestamp, @message
| filter strcontains(@message, "Error") and not strcontains(@message, "Debug")
| sort @timestamp desc
| limit 20

This query finds log entries with “Error” but excludes those containing “Debug.”

Practical Use Cases

• Error Detection: Quickly identify error messages or failed processes in your logs.
• Pattern Matching: Use regex for sophisticated searches, like finding IP addresses or timestamps.
• Exclusion Filters: Simplify data analysis by removing irrelevant entries.

Conclusion

AWS Log Insights offers versatile ways to filter logs by content, from simple substring matching to advanced regex queries. Whether you’re debugging issues or monitoring application performance, these methods ensure accurate and efficient log analysis.