Shell scripting case share - business system log custom retention or deletion requirements


Description of requirements: Logs for certain online business systems are not regularly generated, Some produce daily, Some of them take days to produce, Because the system only generates logs when it is in use, Log files are stored in a directory named after the date of the day. When the log directory gets bigger and bigger it needs to be processed, This led to a requirement from a development colleague, Need a custom script to delete or keep these log directories, as follows:

[root@localhost pay-sign-oper]# ls
2018-09-13  2018-09-20  2018-10-11  2018-10-28  2018-11-02  2018-11-14  2018-12-04  2018-12-09          
2018-09-14  2018-09-27  2018-10-24  2018-10-29  2018-11-06  2018-11-15  2018-12-05  2018-12-10
2018-09-15  2018-09-29  2018-10-25  2018-10-30  2018-11-07  2018-11-30  2018-12-06  2018-12-11
2018-09-17  2018-09-30  2018-10-26  2018-10-31  2018-11-08  2018-12-02  2018-12-07  2018-12-12
2018-09-19  2018-10-08  2018-10-27  2018-11-01  2018-11-09  2018-12-03  2018-12-08  pay-sign-oper.log

[root@localhost pay-sign-oper]# ls 2018-09-13/
pay-sign-oper.2018-09-13.log

[root@localhost pay-sign-oper]# ls 2018-12-12/
pay-sign-oper.2018-12-12.log

 For example, a development colleague now wants to delete logs from before November 08, 2018, and now needs a smart script that can be used to do this in one click.

The script reads as follows:

[root@localhost pay-sign-oper]# cat date_log_delete.sh 
#!/bin/bash

echo -n " Please enter the date,  Log directories up to that date will be deleted next:"  
read  date1

date2=$(echo ${date1}|awk -F"-" '{print $1$2$3}')

cd `pwd`
for date3 in $(ls -l|grep "drwxr"|awk '{print $9}'|awk -F"-" '{print $1$2$3}')
do
   a=$(echo ${date3}|cut -c 1-4)
   b=$(echo ${date3}|cut -c 5-6)
   c=$(echo ${date3}|cut -c 7-8)
   date4=$(echo ${a}-${b}-${c})

     if [ ${date3} -lt ${date2} ];then
        rm -rf ${date4} && echo "deleted ${date4}"
     else
        echo "${date4} do not need to delete"
     fi
done

 Grant script execution privileges
[root@localhost pay-sign-oper]# chmod 755 date_log_delete.sh
[root@localhost pay-sign-oper]# ll date_log_delete.sh
-rwxr-xr-x 1 root root 557 Dec 13 14:50 date_log_delete.sh

For example, if you want to delete the logs before September 27, 2018, then after the script executes, follow the prompt and enter:2018-09-27 to do so:

[root@localhost pay-sign-oper]# sh date_log_delete.sh 
 Please enter the date,  Log directories up to that date will be deleted next:2018-09-27
deleted 2018-09-13
deleted 2018-09-14
deleted 2018-09-15
deleted 2018-09-17
deleted 2018-09-19
deleted 2018-09-20
2018-09-27 do not need to delete
2018-09-29 do not need to delete
2018-09-30 do not need to delete
2018-10-08 do not need to delete
2018-10-11 do not need to delete
2018-10-24 do not need to delete
2018-10-25 do not need to delete
2018-10-26 do not need to delete
2018-10-27 do not need to delete
2018-10-28 do not need to delete
2018-10-29 do not need to delete
2018-10-30 do not need to delete
2018-10-31 do not need to delete
2018-11-01 do not need to delete
2018-11-02 do not need to delete
2018-11-06 do not need to delete
2018-11-07 do not need to delete
2018-11-08 do not need to delete
2018-11-09 do not need to delete
2018-11-14 do not need to delete
2018-11-15 do not need to delete
2018-11-30 do not need to delete
2018-12-02 do not need to delete
2018-12-03 do not need to delete
2018-12-04 do not need to delete
2018-12-05 do not need to delete
2018-12-06 do not need to delete
2018-12-07 do not need to delete
2018-12-08 do not need to delete
2018-12-09 do not need to delete
2018-12-10 do not need to delete
2018-12-11 do not need to delete
2018-12-12 do not need to delete

 Script execution look,  look over,  find that2018 year9 month27 The logs and log directories were deleted before the number.
[root@localhost pay-sign-oper]# ls
2018-09-27  2018-10-11  2018-10-27  2018-10-31  2018-11-07  2018-11-15  2018-12-04  2018-12-08  2018-12-12
2018-09-29  2018-10-24  2018-10-28  2018-11-01  2018-11-08  2018-11-30  2018-12-05  2018-12-09  date_log_delete.sh
2018-09-30  2018-10-25  2018-10-29  2018-11-02  2018-11-09  2018-12-02  2018-12-06  2018-12-10  pay-sign-oper.log
2018-10-08  2018-10-26  2018-10-30  2018-11-06  2018-11-14  2018-12-03  2018-12-07  2018-12-11

The above script seems to work very well, very smart! This script can be used in any such logging scenario . The following is a breakdown of some of the small shell scripts involved in the script:

1)
echo -n " Please enter the date,  Log directories up to that date will be deleted next:" 
read  date1
 
 The above two lines of script use the read parameter in the shell script.
 The read parameter indicates that input from the standard input (keyboard), or other file descriptors, is received.
 After getting the input content, the read parameter passes that content to a standard variable.
 That is, the date that the script prompts for after execution, and passes that date to the ${date1} variable
 
2)
date2=$(echo ${date1}|awk -F"-" '{print $1$2$3}')
 
 This line of script converts the input date into numeric format and passes it to the variable ${date2}, which is used to make the size determination in the if statement with the later variable ${date3}.
 For example: the script executes and prompts for input on 2018-09-27, which is the value of the variable ${date1}.  Then the value of the variable ${date} is 20180927.
[root@localhost pay-sign-oper]# echo 2018-09-27|awk -F"-" '{print $1$2$3}'
20180927
 
3)
cd `pwd`
 
 This line of the script indicates the switch to the current directory path (you can actually leave this line out, since the script file is in the current directory)
 
4)
for date3 in $(ls -l|grep "drwxr"|awk '{print $9}'|awk -F"-" '{print $1$2$3}')
 
 This script line takes the date directory in the current directory, converts it to numeric format, and passes it to the variable ${date3}, which is used to make the size determination in the if statement with ${date2}.  as follows:
[root@localhost pay-sign-oper]# ls -l
total 136
drwxr-xr-x 2 root root 4096 Dec 13 14:56 2018-09-27
drwxr-xr-x 2 root root 4096 Dec 13 14:56 2018-09-29
drwxr-xr-x 2 root root 4096 Dec 13 14:55 2018-09-30
drwxr-xr-x 2 root root 4096 Dec 13 14:55 2018-10-08
...........
...........
drwxr-xr-x 2 root root 4096 Dec 13 12:40 2018-12-10
drwxr-xr-x 2 root root 4096 Dec 13 12:40 2018-12-11
drwxr-xr-x 2 root root 4096 Dec 13 12:40 2018-12-12
-rwxr-xr-x 1 root root  557 Dec 13 14:50 date_log_delete.sh
-rw-r--r-- 1 root root    0 Dec 13 12:40 pay-sign-oper.log


[root@localhost pay-sign-oper]# ls -l|grep "drwxr"|awk '{print $9}'
2018-09-27
2018-09-29
2018-09-30
2018-10-08
...........
...........
2018-12-10
2018-12-11
2018-12-12

[root@localhost pay-sign-oper]# ls -l|grep "drwxr"|awk '{print $9}'|awk -F"-" '{print $1$2$3}'
20180927
20180929
20180930
20181008
...........
...........
20181210
20181211
20181212
 
5)
a=$(echo ${date3}|cut -c 1-4)
b=$(echo ${date3}|cut -c 5-6)
c=$(echo ${date3}|cut -c 7-8)
 
 These three lines of script represent the extraction of certain bits from a numeric string.  The three variables a , b, and c denote taking bits 1-4, bits 5-6, and bits 7-8 of the variable ${date3}, respectively.
 For example, if the value of the variable ${date3} is 20180927, then:
[root@localhost pay-sign-oper]# echo 20180927|cut -c 1-4
2018
[root@localhost pay-sign-oper]# echo 20180927|cut -c 5-6
09
[root@localhost pay-sign-oper]# echo 20180927|cut -c 7-8
27
 
6) date4=$(echo ${a}-${b}-${c})
 
 This line of scripting indicates that the value of the variable ${date3} is split and then composed into a date format.  In order to do the date directory delete operation after the result of the if statement later.
 
7)
if [ ${date3} -lt ${date2} ];then
        rm -rf ${date4} && echo "deleted ${date4}"
     else
        echo "${date4} do not need to delete"
     fi
 
The above script content represents the judgment in the if statement, :
 When the value of the variable ${date3} is less than the value of the variable ${date2}, the date directory of the variable ${date4} is deleted and the deletion message is printed.
 When the value of the variable ${date3} is greater than or equal to the value of the variable ${date2}, a do not delete message is printed.

Recommended>>
1、City Central Finance Office to promote indepth cybersecurity financial day campaign
2、Mysterious text message makes iPhone die instantly Apple says next update fixes the issue
3、MySQL Occupancy OneClick Query Practice
4、8 million downloaded npm packages hacked your device may become a mining machine
5、Chinas softwaredefined storage and hyperconverged market expected to enter a period of rapid growth

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送

    已发送

    朋友将在看一看看到

    确定
    分享你的想法...
    取消

    分享想法到看一看

    确定
    最多200字,当前共

    发送中

    网络异常,请稍后重试

    微信扫一扫
    关注该公众号