Script Monitoring

Introduction

In the world of Linux scripting, monitoring the execution and performance of scripts is crucial for ensuring they run efficiently and as expected. This tutorial will guide you through various methods to monitor script execution and performance, from basic logging to more advanced tools like strace and perf.

Monitoring helps in identifying bottlenecks, debugging issues, and optimizing scripts for better performance. Whether you're a beginner or an intermediate developer, understanding how to monitor your scripts can significantly enhance your productivity.

Concept

Script monitoring involves tracking various aspects of script execution, such as:

• Execution Time: How long the script takes to complete.
• Resource Usage: CPU, memory, and disk I/O usage.
• Error Handling: Capturing and logging errors for debugging.
• Performance Profiling: Identifying specific parts of the script that are slow or resource-intensive.

In this section, we'll explore different tools and techniques to monitor these aspects effectively.

Examples

Basic Logging

The simplest way to monitor a script is by adding logging statements. This helps in understanding the flow of execution and identifying where things might be going wrong.

Example 1: Adding Simple Logging

#!/bin/bash

# Start time
start_time=$(date +%s)

echo "Script started at $(date)"

# Simulate some work
sleep 5

# End time
end_time=$(date +%s)
execution_time=$((end_time - start_time))

echo "Script ended at $(date)"
echo "Execution time: $execution_time seconds"

Script started at Mon May 15 10:00:00 UTC 2023  
Simulating work...  
Script ended at Mon May 15 10:00:05 UTC 2023  
Execution time: 5 seconds

Example 2: Logging to a File

#!/bin/bash

# Log file
LOG_FILE="script.log"

echo "Script started at $(date)" >> $LOG_FILE

# Simulate some work
sleep 5

echo "Script ended at $(date)" >> $LOG_FILE

Script started at Mon May 15 10:00:00 UTC 2023  
Simulating work...  
Script ended at Mon May 15 10:00:05 UTC 2023

Advanced Monitoring Tools

For more detailed monitoring, you can use advanced tools like strace and perf.

Example 3: Using strace for System Calls

strace is a powerful tool that traces system calls made by a process. It's useful for debugging and understanding what your script is doing at the system level.

#!/bin/bash

# Run the script with strace
strace -o trace.log ./your_script.sh

execve("./your_script.sh", ["./your_script.sh"], 0x7ffc9d1f5b28 /* 43 vars */) = 0  
brk(NULL)                               = 0x563a2c2e7000  
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)  
...

Example 4: Using perf for Performance Profiling

perf is a performance analysis tool that can provide detailed insights into CPU usage, cache misses, and other performance metrics.

#!/bin/bash

# Run the script with perf
perf record -g ./your_script.sh

After running the script, you can analyze the data using:

perf report

# To display the perf.data header info, please use: /usr/bin/perf report --header
#
# Samples: 100  of event 'cycles'
# Event count (approx.): 3456789012 cycles
#
# Overhead  Command  Shared Object      Symbol  
# ........  .......  ................  .......
#
#    50.00%  your_script.sh  [kernel.kallsyms]  [k] sys_read
#    30.00%  your_script.sh  [kernel.kallsyms]  [k] sys_write
#    20.00%  your_script.sh  /lib/x86_64-linux-gnu/libc-2.31.so  __GI__IO_getline_info

Monitoring Resource Usage

You can use tools like top, htop, and vmstat to monitor resource usage in real-time.

Example 5: Using top for Real-Time Monitoring

#!/bin/bash

# Run the script with top monitoring
top -p $(pgrep your_script.sh)

top - 10:00:00 up 2 days, 5:30,  1 user,  load average: 0.78, 0.65, 0.54
Tasks: 223 total,   1 running, 222 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.0 us,  2.0 sy,  0.0 ni, 93.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  7816.0 total,   543.2 free,  2345.6 used,  4927.2 buff/cache
MiB Swap:  2048.0 total,  2048.0 free,     0.0 used.  4812.2 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
123456 user      20   0   1234m   567m   123m R  98.7   7.2   0:05.43 your_script.sh

Example 6: Using vmstat for Detailed System Metrics

#!/bin/bash

# Run the script with vmstat monitoring
vmstat -s

12345 total memory
    5678 used memory
    1234 free memory
     456 buffers
     789 cached

What's Next?

Now that you have a good understanding of how to monitor script execution and performance, you can explore automation tools like Ansible or Puppet for managing and monitoring scripts across multiple systems. These tools can help in automating the deployment and monitoring of your scripts, making it easier to manage large-scale environments.

By combining basic logging with advanced tools like strace and perf, you can gain deep insights into how your scripts are performing and make informed decisions to optimize them further.

Happy scripting!